16

What would cause a Clojure program to not immediately exit upon finishing the last statement in the main function?

All I did was to change a (doall (map ...)) to a (doall (pmap ...)), and suddenly the program would hang upon completion of its tasks for a good number of seconds before exiting. I would put a (println "Finished everything!") on the last line of the -main function, and it would print that, yet still not exit for some time. What could cause this behavior, and how should I fix it?

EDIT: The pmap is the only part of the program that is parallelized (mostly because everything else runs more or less instantly). As latter parts of the program require all the results from pmap in order to function correctly, and as the program output is the same for both map and pmap, I somewhat doubt that pmap would still be running at the end of the program. Putting (System/exit 0) at the end instead of the println does not change program output, either.

Improve this question
2
  • 1
    I'd guess that pmap must not be completely done doing its processing when the code after it fires. Hard to tell without more code shown. Commented Jan 1, 2012 at 22:04
  • 1
    possible duplicate of Why does Clojure hang after having performed my calculations? Commented Jan 1, 2012 at 22:31

2 Answers 2

Reset to default
19

pmap will spin up multiple threads from a threadpool to service your concurrent tasks.

Calling shutdown-agents is necessary to allow the JVM to exit in an orderly manner because the threads in the agent threadpools are not daemon threads.

You need to explicitly tell them to gracefully shutdown (but only when your program is done). This looks to have been answered before here.

Quoting from that answer:

"You need to call shutdown-agents to kill the threads backing the threadpool used by pmap."

Docs are here: http://clojuredocs.org/clojure_core/1.3.0/clojure.core/shutdown-agents

Improve this answer
Sign up to request clarification or add additional context in comments.

1 Comment

Great solution! I've run into this problem in multiple projects where I shell out to run other commands, and adding (shutdown-agents) fixes it. Presumably clojure.java.shell is using some sort of other thread to do its thing.
3

According to the docs:

...[pmap is O]nly useful for computationally intensive functions where the time of f dominates the coordination overhead.

So this probably means there is more overhead using pmap for your scenario than using map. The pmap operations are probably taking a little extra time to finish up before the code after pmap fires.

Improve this answer

3 Comments

I don't mind being down-voted, but please at least give an explanation.
Thank you for the answer Jason, but I doubt that pmap has not finished its operations for reasons I added to the question. Would you have any other ideas to this behavior? Also, just fyi, I did not downvote you.
@wrongusername: No other suggestions. And no need by the looks of it, thanks to the answer provided by Scott Lowe.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.