Python: numpy memmap in parallel

Question

Does it make sense to use numpy's memmap across multiple cores (MPI)?

I have a file on disk.

Can I create a separate memmap object on each core, and use it to read different slices from the file?

What about writing to it?

Could you describe your application in a bit more detail? Also, there is a pretty helpful answer here: stackoverflow.com/questions/16149803/… — Jan
– Jan, Commented Jun 20, 2020 at 1:29

user3666197 · Accepted Answer · 2020-06-20 09:23:17Z

4

Q : "Does it make sense to use numpy's memmap across multiple cores (MPI)?"

Yes _{( ... even without MPI, using just Python native { thread- | process-}-based forms of concurrent-processing )}

Q : "Can I create a separate memmap-object on each core, and use it to read different slices from the file?"

Yes.

Q : "What about writing to it?"

The same _{( sure, if having been opened in write-able mode ... )}

answered Jun 20, 2020 at 9:23

user3666197

1

Sign up to request clarification or add additional context in comments.

1 Answer 1