2

Currently, I can create (nested) lists of objects that are a mix of eagerly computed items and delayed items.

If I pass that list to dask.compute, it can create the graph and computes the result as a new list replacing the delayed items with their computed counterparts.

The list has a very well defined structure that I would like to exploit. As such, before using Dask, I had been using numpy array with dtype=object.

Can I pass these numpy arrays to dask.compute? Are there other collections, that support ND slicing à la numpy, that I can use instead?

My current workaround is to either use dictionaries, or nested lists, but the ability to slice numpy arrays is really nice and I would not like to loose that.

Thanks,

Mark

code example as notebook

Improve this question

1 Answer 1

Reset to default
2

Dask.compute currently only searches through core Python data structures like lists and dictionaries. It does not search through Numpy arrays.

You might consider using Numpy arrays until the very end, then calling .tolist() then calling np.array again.

result = dask.compute(*x.tolist()) result = np.array(result)

Improve this answer
Sign up to request clarification or add additional context in comments.

4 Comments

great thanks, that was my workaround too, I just wanted to make sure I wasn't missing something.
Do you have any plans to support numpy arrays in your nested packing and unpacking?
Me personally? No I don't plan to work on this. If you are curious about the project then you should raise an issue.
you included the dask team. I'll raise an issue on github. Thanks

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.