Wednesday, 30 March, 2016 UTC
Summary
A reader of my RabbitMQ Patterns email course recently asked a question about using messaging systems within an web application. Specifically, they wanted to know why you would use a request/reply pattern over a messaging server, instead of just handling the request within the HTTP call and the web server, itself.
The short answer is, “it depends”.
There are times when the request can and should be handled by the web server, directly. Loading a blog post to view? Yeah, that’s probably a good call for the web server to handle. But there are also times when it doesn’t make sense for the web server to handle the workload, directly.
The following is a short conversation that took place via email, regarding messaging vs HTTP.
Shifting The Workload
Them:
What are the benefits of using a request / response queue over just doing the work as part of http request?For fire and forget messages I can see the performance benefits…Is it just being able to shift the work load to another service? Or abstract some complexity of the work?
Me:
That’s the core of it, really – shifting the workload to another service.
If you have an external system that contains the information you need, or can do the work within a few hundred milliseconds, it might make sense to do request/reply.
Scheduled Code Example
Me, continued:
For example, I have a system that uses a node.js module for running tasks at specific times. It’s a bit like a cron job, but it runs entirely in node.
This module lives in a back-end service that runs in it’s own process. I used a separate process because this service has nothing to do with handling HTTP requests and should not live in the HTTP context or web server, directly.
But, my web app needs to get a list of what’s running next, and display them on a particular page. So how do I do that? What are the options?
Read The Database Directly
I could read the database collection that the scheduler module uses – it’s in the same database that the web app uses. But, that would be a backdoor into the functionality the module provides. I would have to re-create the logic of the scheduler module within my web app, to translate the data into something meaningful for the user.
No, thanks.
A New Schedule Module Instance
I could create new instance of the scheduler module in my web app. But this would create two sources of the truth on what is running next, and they would be in conflict with each other.
Each instance would assume it owns the data in the database (because the instances don’t know about each other). This would cause problems when one updates the database and the other tries to update it again.
Again: No, thanks.
Request/Reply
The solution, in this case, was to use request/reply to get the list of items from the existing back-end service. That way I have a single source of the truth for the scheduled items, and don’t have to re-create logic from an existing module.
Decoupling The Database
Them:
Ah thanks, that helps. Nice, interesting idea. Means you web app isn’t so coupled to the database.
Me:
Exactly!
There’s a saying I heard a long time ago, “a database is not an integration layer.”
I don’t remember where I first heard that, but it continues to ring true after many years.
I could have tried to use the database as an integration layer, but that would have added complexity and potential problems to the system.
It made more sense for me to use messaging in this scenario, and allow the one source of truth on the upcoming schedules to be that one source of truth.
There are far more examples of what can (and should) be done with messaging, when handling HTTP requests. The list is nearly endless, and can go so far as to say database writes should be handled through messages!
In my RabbitMQ For Developers training package, I spoke with Anders Ljusberg about Event Sourcing and messaging systems for database writes.
It was an enlightening conversation, to hear about the need for pushing a database write to a back-end service, and how this affects the over-all architecture of a web application.
Be sure to check out the complete RabbitMQ For Developers course, for this and other interviews, screencasts, ebooks and more!