4

I have decided to work with Python for processing graphs. I use NetworkX for loading the edge list to construct the graph. When I try to load edges with nx.read_edgelist I receive MemoryError.

The graph has 4 million nodes and 34 million edges and my PC has 12GB of RAM. Is it possible to load that graph into 12GB? Is there any optimal solution?

Improve this question
0

1 Answer 1

Reset to default
3

NetworkX uses dictionaries to implement graph data structures. Each edge uses at least 100 bytes of memory. Hence, with 34 million edges, you would would likely need more than 3.4 GB.

One solution would be to use cloud computing and scale up memory as needed.

An alternative to NetworkX is graph-tool, which is a Python module that implements core data structures and algorithms in C++.

See also

Improve this answer
Sign up to request clarification or add additional context in comments.

1 Comment

I suppose you are mistaken, 100 bytes x 34 * 10^6 = 34 * 10^8 is approximately 3.4 GB, not 40 GB

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.