Our main problem How to implement a query that retrieves data from multiple services in a microservice architecture when database per-service is involved and shared database is not an option? In a microservices architecture, every service work independent from each other and has his own database and communicate using messaging…

In distributed systems, it might be multiple different data stores are involved so let assume we have a tangle of different datastores cache, SQLDB, and indexes and we need to synchronize the data between each other. Actually, when you are dealing with distributed systems you should put your old mindset…

In fact, most of the projects have some elements of legacy and new systems and legacy is part of our life as software engineers and it might be hard to refactor or rebuild this legacy to be integrated with modern parts in your software. the problem with legacy is there…

In this article, I will explain the term in distributed computing called Partitioning so I will explain the following points What is Partitioning meaning? When do I need it? Explaining the kinds of Partitioning with advantages and disadvantages Secondary index and kinds of Secondary index Rebalancing and why I need…

In this article, I will explain Redis in system design as I experienced a lot of issues in a legacy system that depends on Redis as a major part of the design and after reading and understanding Redis I understood the issues What is Redis? Redis is an open-source, in-memory…

What is Kafkana ? Kafkana is an open-source multiple Kafka clusters dashboard management What are the features offered by kafkana ? Support multiple clusters stored in your browser local-storage List of brokers with a configuration map List of topics with details about replica — ISR etc. List of consumers and…

In this article, I will explain an idea to have a semi-mirror repoistory feature without a premium version from GitLab as the mirror with pull mode is a premium What is a mirror repository feature first? If you have for example a codebase on a repository on GitHub or azure-DevOps…

In distributed system world nothing 100 % reliable so we always consider faults so when designing a distributed system model is how we designate our premises about faults. In general, we can categorize models into three 1- Network behavior 2- Node behavior 3- Timing behavior Network behavior As no network is reliable…

In this article I will explain some data pipeline use cases and how can Kafka be useful there with a prof of concpt how to build data streaming pipeline. In many large/medium scale project you need to use some sort of data system that can be valid in different cases…