0

I want to do something like this:

client = Client()
def sub():
    client.submit(lambda: 'ok')

Calling sub() does not submit the task. Calling client.submit(lambda: 'ok') directly does. I can't find anything in the docs discribing this behavior. (I am on Dask version 1.1.4 and am currently not able to upgrade)

2 Answers 2

1

In your original code, the future generated by submit was not stored in a variable, and there was no reference to it after your function sub completed. Therefore, Dask will have noticed that no client is any more interested in that future, and it will have been cleaned from the system - probably it had already run, and the memory of the result will have been released.

This is expected behaviour, and deciding in your code which futures have references to them is a fundamental concept in managing memory on the system, see https://distributed.dask.org/en/latest/memory.html#clearing-data

I suspect that when yo originally called submit while not in a function, a reference to the result was kept by your repl - if you had added a ; to the end of the line to suppress output, you would have seen the same behaviour as calling sub.

Sign up to request clarification or add additional context in comments.

Comments

0

The task was submitted also when I executed sub() but I did not see this reflected in the Dask GUI after refreshing the GUI. You can check for certain that it works if you do

def sub():
    return client.submit(lambda: 'ok')

sub().result()
# 'ok'

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.