ArangoDB Java Batch mode insert performance

Question

I'm using ArangoDb 3.0.5 with arangodb-java-driver 3.0.1. ArangoDB is running on a 3.5ghz i7 with 24gb ram and an ssd.

Loading some simple Vertex data from Apache Flink seems to be going very slowly, at roughly 1000 vertices/sec. Task Manager shows it is CPU bound on the ArangoDB process.

My connector is calling startBatchMode, iterating through 500 calls to graphCreateVertex (with wait for sync set to false), and then calling executeBatch.

System resources in the management interface shows roughly 15000 (per sec?) while the load is running, and used CPU time fixed to 1 for user time. I'm new to ArangoDB and am not sure how to profile what is going on. Any help much appreciated!

Rob

mpv89 · Accepted Answer · 2016-09-15 12:06:12Z

1

Your performance result is the expected behavior. The point with batchMode is, that all of you 500 calls are send in one and executed on the server in only one thread.

To gain better performance, you can use more than one thread in your client for creating your vertices. More requests in parallel will allow the server to use more than one thread.

Also you can use createDocument instead of graphCreateVertex. This avoids consistency checks on the graph, but is a lot faster.

If you don't need these checks, you can also use importDocuments instead of batchMode + createDocument which is even faster.

answered Sep 15, 2016 at 12:06

mpv89

1,89111 silver badges10 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Arkay Over a year ago

Thanks, the importDocuments method looks ideal, especially given the importOptions allows me to set behavior on error, duplicates etc. Ill get back with performance numbers asap. Does with also work for edges as well as vertices? Edges therefore being documents with _from/_to attributes?

Arkay Over a year ago

importDocuments runs at around 200,000 (simple) documents per thread, much better, thanks!

mpv89 Over a year ago

Yes, edges are documents with _from and _to, so importDocuments will also work.

Collectives™ on Stack Overflow

ArangoDB Java Batch mode insert performance

1 Answer 1

3 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

3 Comments

Your Answer

Sign up or log in

Post as a guest

Related