3

I am currently working with a legacy system that consists of several services which (among others) communicate through some kind of Enterprise Service Bus (ESB) to synchronize data.

I would like to gradually work this system towards the direction of micro services architecture. I am planning to reduce the dependency on ESB and use more of message broker like RabbitMQ or Kafka. Due to some resource/existing technology limitation, I don't think I will be able to completely avoid data replication between services even though I should be able to clearly define a single service as the data owner.

What I am wondering now, how can I safely do a database backup restore for a single service when necessary? Doing so will cause the service to be out of sync with other services that hold the replicated data. Any experience/suggestion regarding this?

Improve this question

1 Answer 1

Reset to default
5

Have your primary database publish events every time a database mutation occurs, and let the replicated services subscribe to this event and apply the same mutation on their replicated data.

You already use a message broker, so you can leverage your existing stack for broadcasting the events. By having replication done through events, a restore being applied to the primary database will be propagated to all other services.

Depending on the scale of the backup, there will be a short period where the data on the other services will be stale. This might or might not be acceptable for your use case. Think of the staleness as some sort of eventual consistency model.

Improve this answer
Sign up to request clarification or add additional context in comments.

4 Comments

"a restore being applied to the primary database will be propagated to all other services" -> could you elaborate this a bit more? How do I design my application to be able to detect changes applied by database restore?
@habsq Without knowing the specifics of the database that you are using, that's a bit hard. On most databases, you can define post-commit triggers that will be triggered when a mutation occurs. You can then emit your events from within the trigger. Some databases (e.g. Oracle) provide an out of the box replication mechanism that works the same way, using publish-subscribe pattern for each database mutation to propagate the replicated data.
Thanks for the explanation! I am marking this as the answer because at least it gave me partial solution. Problem is, some of the services (and their databases) are deployed in a PaaS which means that setting the database to publish events when mutation occurs not possible.
@habsq Assuming there's a gatekeeper service that do all the mutations, the events can be triggered from the service instead. For an answer to the specific use case that you are having, feel free to post a separate question detailing the said use case.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.