Return an error ? But what's error ? 500 ?
Yes return an error. There is nothing wrong in returning an Error from your micro-service. Usually "500 Internal Server Error" is the right error in the case you have a failure in your database. This is a standard behavior of an Rest API.
But isn't the microservice architecture supposed to avoid this type of
coupling ? Shall we make the system more loosely coupled ?
I think there is a confusion here. A micro-service communication with its own database is not considered as coupling. Micro-service-A which is using its won database micro-service-A-database is considered as one logical unit or vertical. The micro-service-A would not be much of a use without its database and vise versa. This is totally ok and you can look at in similar way as with standard WebApplication with its Frontend, Backend(similar as your service) and Database. Coupling should be avoided across different micro-services. For example micro-service-A should not be tightly couple with with micro-service-B or micro-service-C. Each micro-service should be atomic and as independent as possible but not from its database, cache or similar. You can consider the database as logical part of it.
Shall the micro-service save the data in a local file or queue and
retry to insert in db ?
No it is expected that the database could fail or at least you have to deal with the option as well. From the User prospective you would just return the Error Code 500. At least for most cases this would be the expected behavior. There are some special cases where you want at any cost to save the data and not lose it for that request(there are ways to deal with this as well).
But what about user ?
If it is a standard Web user then he would retry a couple of times and if the problem persists probably come later and try again(there is nothing wrong in returning the Error 500 here). If by user you mean another micro-service is doing the call then that caller-micro-service has to expect that failure can happen. Now it depends what you are doing here? Example consider micro-service-A is calling micro-service-B with an Http request:
- Get Call: here you can build a retry policy in micro-service-A and if micro-service-B responds with 500 after retries you can return an error to the user who called micro-service-B.
- Post/Put/Patch Call: here you can also try similar as with the Get Calls but only if 1 service is involved. If you have micro-service-A calling micro-service-B and then micro-service-C and if one call was successfully(which saved some data) and another one failed you have to consider Sagas(if the operation should be transactional).
Imagine the event bus crashes.... If so, the consumer will crash too
in the microservice.
If your local micro-service database crushes then all the other channels should crash as well. If you can not save your entity in your local micro-service db why would you go further with the operation like publishing a message to a queue? The source of truth of your data/entities is the database(at least in most cases). So if the database fails you should throw an exception and return an error to the caller/user.
If db insert occur, the producer could not emit the event to the event
bus.... So data is lost ? So shall the producer keep data in its local
storage for retrying ?
So in the case where you save your data/entity in the database and the queue is not available you could simply save the Message/Event to a table in the DB and then publish the Message/Event when the queue is up and running again. Actually this is a very common pattern in this situations. For example your implementation could be Transactional:
- Save entity to its table and
- Save event/message to Event table. This way if one fails the
operation will be rolled back.
You can have a background worker(depending on the tech you are using for your Back-end) to publish messages into the queue in async way.
