Communicating microservices the proper way

Introduction

Communicating microservices is one of the most challenging parts of implementing a microservices architecture. If you do badly, you will fail and yet another "back to the monolith" article will be written.

Things to avoid when communicating microservices

First, let's talk a little bit about things that must avoid when you are trying to communicate microservices:

• Don't use shared databases: this includes traditional databases but also key-value systems like Redis. If you do so, you are immediately coupling the different microservices that are using these systems: every time you modify the database because of a requirement of some microservice, you are probably breaking another one; and you can also forget deployment autonomy if you do so. Each microservice should have its own database -if needed.
• Don't use FTP: yes, it's asynchronous, and yes, you can get to a versioned protocol or something if you work enough, but man, there are better tools these days.

We've seen what NOT to do. Let's how to properly communicate microservices.

Option 1: Communicating microservices through HTTP calls (the acceptable one)

Using HTTP calls such as RESTful APIs or even SOAP is one of the two best options we have to communicate microservices. This one is acceptable but has some drawbacks:

• You need to implement circuit breakers: what if the endpoint fails or is down? You need a plan B, probably shaped as a messaging queue -which is option 2, the preferred one.
• If the communication starts with a user action, then the user will be waiting while you call another -or even more than one- microservices. This can take some time due to network latency and processing time.
• To avoid the previous one, you will need to work with better approaches, such as gRPC, which is not always easy and is not supported -or decently supported- by all the programming languages.
• You'll need to build backward compatibility for the endpoints if you want -and you want- to keep deployment autonomy.

If you absolutely need to do synchronous communication (and consistency) this is your only option. If you can live with asynchronous communication, so eventual consistency, keep reading.

Option 2: Communicating microservices through messaging queues (the best one)

If you can live with eventual consistency, communicating microservices using messages in queues is the best possible way. How does it work?

You can use a choreography, so:

• One microservice sends a message to a queue.
• Other microservice listen to the queue, and process the message, resulting in some action and potentially sending another message.
• A third microservice listens to the second message and processes it.
• ...

As you can see, no direct communication is involved anymore. Advantages:

• If something goes wrong when processing a message, it goes back to the queue and can be reprocessed later, so the system is more resilient.
• If the receiving microservice is down, the daemon listening messages will be down, so the messages will remain in the queue; as soon as the microservice is up again, all the messages will be processed: again, it's more resilient.
• If processing a message takes a long time, no one is waiting. The user is not affected.
• We can process many messages at the same time by having many daemons listening. This is very useful for long processes.
• We can a different amount of resources to the daemons and the rest of the system, so we don't need to overpower the whole microservice only because we have some slow processes.

This method has also some drawbacks:

• You need to reason in terms of eventual consistency: this is not easy at the beginning, and testing is also more difficult.
• Many times you need to adapt your UI to eventual consistency: if the users ask for some task that needs to involve several microservices and you use a choreography, then the UI should have some mechanism to make the user note that this is an asynchronous operation and also a mechanism to let the user know that the process has finished.
• Again, you need to version your messages to have backward compatibility and conserve deployment autonomy.

Conclusion

Communicating microservices is one of these difficult things we need to manage in a microservices architecture. But, if you do things well, you'll probably enjoy microservices as much as I do.