0

I'm looking for the right way so use ElasticSearch with MongoDB. I want to save several informations in MongoDB. Additionally i want to save a larger text with ElasticSearch to support complex fulltext-search.

My problem at the moment is: I'm not sure what the best solution is for this. Most solutions i found to synchronize MongoDB with ElasticSearch are using "river" which is deprecated!

What is the best way to combine these two technologies?

Is it even the best way to save it in MongoDB and ElasticSearch?

I found multiple articles that explained, that ElasticSearch alone is not safe enough and that you have to use another DBMS.

Also under robustness on the mongoDB website I found this:

Unfortunately, Elasticsearch (and the components it's made of) does not currently handle OutOfMemory-errors very well. [source]

So saving the data redundant is probably the best way.

Thanks in advance!

Improve this question

1 Answer 1

Reset to default
3

Hei,

We are also working with both Elasticsearch and MongoDb. We started with a river and after having a lot of issues with it we got rid of it before becoming deprecated. The way we do it is: when saving data to mongo we create a message in a queue which notifies the search storage to do the insert/delete operation with the given data.

So basically we keep them in sync manually and there will always be a delay between mongo and elaticsearch. The good part is that if elasticsearch would fail, we have implemented an endpoint which reimports the data from mongo to ES. Also, the structure inside ES it's different from the one in mongo. Before, it was a lot more complicated to do this with the river. Imagine that we even had our own custom implementation.

Hope my answer helps at least a bit.

Improve this answer
Sign up to request clarification or add additional context in comments.

4 Comments

hello, thank you for your answer. I've actually found this solution alot. I think it's surprising that there is no built-in solution already for this problem. Thank you!
@alexandra are you guys still using your custom built solution? what are your thoughts on mongo-connector?
Hei, we still keep things in sync manually, yes. I haven't worked with mongo-connector but it looks similar to the old river implementation. What I can say is that when we used to have the river in place, the problems appeared after starting to use it, so probably you need to try it for a while to reach a conclusion.
Mongo Connector is not officially supported by mongo and moreover, mongo-connector doesn't support LDAP auth which is very basic for mongo enterprise. It only supports limited AUTH.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.