Before AI and blockchain, microservices was the ruler of the buzzword world. Even now, if you don't preach microservices, you won't get hired in a trendy engineering organization. If you remotely disagree with seeing microservices as the unequivocal holy grail of a modern stack, you risk yourself being seen as either illiterate or a dinosaur. Often, CTOs sell microservices too hard to the board simply to give an impression that investor money is being spent well. But, is it a good starting point for a solution?
Table of contents
- The illusion of complexity and seeing microservices as the savior
- The monetary penalties of microservices
- Reasons to choose microservices
- The right number of microservices and the team size
The illusion of complexity and seeing microservices as the savior
A "complex" system is a network of many components, and they communicate in a variety of ways. Complexity doesn't signify difficulty. It is challenging to reduce a complex system to a collection of formulas and variables. For example, predicting an economic downturn precisely (if possible at all) could be a very complex system.
A "complicated" system is driven by hundreds of formulas and rules written one over the other over a long period, directing a system's outcome. It isn't easy to understand as a whole, mainly because business rules override and overlap. A back-office system could be a complicated system comprising hundreds of rules that determine the result of a particular dataset, event, or action.
When a system starts to behave unpredictably for an engineer who does not have excellent visibility across the entire system, the illusion they get is that the system is "complex." Don't automatically assume your system is complex. Instead, it was possibly made complicated. Mainly the absence of an architect and/or accumulation of a pile of bad practices contribute to a "complicated" system design.
The first question you might ask yourself before assuming that the problem you are solving is complex, did you make your design complicated? Generally, there are two mental blockers engineers have about simplifying their architecture.
- Simply poor/bad design (accept it), and in some cases, engineering is clueless as to what makes a good design
- Many engineers jumped into coding for years without thinking, and now its too complicated to simplify
If you think microservices will salvage you down the road, you are in it for a great surprise, if you don't simplify your system first. Microservices will increase complexity in order of magnitude if you refuse to refactor your architecture.
In the beginning, microservices should be a special case, and it shines when the business and technical operations satisfy a set of parameters. Instead, you can start with a modular monolithic architecture. Modularity will allow you to extract and expose as services when needed. Don't exclusively commit to one pattern from where it becomes very difficult to opt-out. Let microservices be a natural evolution, not an enforced default in the organization.
The monetary penalties of microservices
A slightly advanced function doesn't warrant a separate service. Instead of invoking shared functions, making service calls has huge taxes, to name a few, per microservice:
- network connectivity
- compliance & audit
- authentication & authorization
- data transmission & security
- resource provision & management
- configuration management
- extra code for communications
- serialization & context loading
- service instance auto-scaling
- service discovery & load balancing
- redundancy, backup & recovery
- logging & error handling
- re-implement cloud patterns (timeouts, retry, etc.)
- fault tolerance
- message bus & queues
- separate build and deploy processes
- monitor performance and meet SLAs
- potential areas of waste, e.g., cyclic dependency
- cost implications (possibly higher TCO)
- the introduction of many unnecessary points of failure
Put a monetary value in everything. Is this worth paying the price for a microservice for trivial reasons vs. the benefits? The above qualities are great to have and maintain, but it is a waste of recurrent effort if the cost vs. benefits can't be justified when repetitively implemented across multiple services which could easily be served by one. Lets accept it — not all companies are Google, Uber and Amazon, and have similar userbases, organizational assets and funding. You will probably not get the full benefits from fully microservices-oriented architecture from day 1, unlike the companies mentioned earlier.
Reasons to choose microservices
Always start with a monolithic approach and branch out to microservices only when it proves to have benefits that can be materialized. Attempt microservices when most engineers understand and practice modular software construction, clean code, clean architectures, and CI/CD is in place. Here are a few examples when it is a lower risk or it can easily be reasoned about to go for microservices in the beginning.
- Business function: When a business function is significantly different from the rest, for example, a Computer Vision API doesn't need to remain with IT services API, e.g., accounting.
- More frequent releases: A set of functionalities may require several production deployments daily compared to the rest of the monolith deployed once a week/month. A microservice would enjoy faster build, integration, automated tests, and deployment.
- Different compute model: A business case requires unusual computability compared to the rest. For example, AI model training is a significantly different workload than an email marketing engine.
- Specialized performance: An unusual scaling, performance, and throughput requirement may exist compared to the rest of the system. For example, a DNS server has no business in staying with a back-office system.
- Proof of concepts or unstable data structures: PoCs are great candidates for microservices but aim to merge into the monolithic product that you have built when the PoC receives a go-ahead.
- When merging is problematic: For any reason, if it becomes impossible to hire competent engineers and make them owners of respective functional business areas to prevent stepping on others' shoes (given the architecture is already good), merging into a large codebase becomes difficult. If your teams fight with code merge for more than a reasonable amount of time, it is time to think about splitting.
- Teams on a collision course: Your teams can't stand each other, refuse to work together, and there's no peaceful solution you can offer. If there is enough separation, split into multiple services and assign teams to the services they own.
- Special cases: When you have them, but don't treat everything as a special case.
The right number of microservices and the team size
This is more art than science, but typically don't create more microservices than the number of teams you have unless you want the teams to engulf in the flames of insanity, burnout and you want them to spend 90% of the time in non-coding activities.
What's the ideal team size, then?
"We try to create teams that are no larger than can be fed by two pizzas. We call that the two-pizza team rule." — Jeff Bezos (Amazon)
How many people can two pizzas feed? Possibly 5-8 people. According to Jeff Bezos, that's the ideal size of any team.
The success in the right blend of microservices and monolithic approach and reaping its benefits still depend on how well the trade-offs are considered, the lessons learned and the planning exercise executed. Here, CTOs/architects play an enormous role by educating the teams and instilling an open-minded engineering culture.
"Any organization that designs a system (defined broadly) will produce a design whose structure is a copy of the organization's communication structure." — Melvin E. Conway (1967)
Engineering culture plays quite an influential role in determining the teams' shape and type of architectural decisions. The kind of designs the teams come up with can be said a reflection of how they are managed, empowered and communicated.
Wise engineers/architects/CTOs can see beyond the hype and buzzwords and exercise greater clarity on how software is engineered, evolved, scaled, and maintained. Please don't get married to a single architectural pattern and be blinded by it. A sustainable software development practice is all about doing the right thing. Stay true to to it.