2

I'd like to use reflection in combination with parallel processing in Scala, but I'm getting bitten by reflection's lack of thread safety. So, I'm considering just running each task in its own process (not thread). Is there any easy way to do this?

For example, is there a way to configure .par so it spawns processes, not threads? Or is there some function fork that takes a closure and runs it in a new process?

EDIT: Futures are apparently a good way to go. However, I still need to figure out how to run them in separate processes.

EDIT 2: I'm still having concurrency issues, even when using Akka's "fork-join-executor" dispatcher, which sure sounds like it should be forking processes. However, when I run ManagementFactory.getRuntimeMXBean().getName() inside the Futures, it seems everything still lives in the same process. Is this the right way to check for actual process-level parallelism? Am I using the correct Akka dispatcher?

EDIT 3: I realize reflection sucks. Unfortunately it is used in a library I need.

Improve this question
3
  • It sounds like most of the problems associated with lack of thread safety for reflection might be solvable by using a separate Class Loader on each thread. Does anyone else have thoughts on that? Commented Oct 24, 2013 at 23:56
  • 1
    If you're a newcomer to Scala, you almost certainly should not be looking to reflection to solve programming or design problems. A very strong principle of Scala programming and design is static typing. You should know "up front" (when you write the code) what types your program traffics in. So-called "dynamic typing" is a kind of mealy-mouthed, "you know what I mean, compiler, right?" programming that is ultimately counter-productive, at least in large systems. Avoid it! Commented Oct 25, 2013 at 4:15
  • could post post a sample code demonstrating the issue? Commented Oct 28, 2013 at 10:17

2 Answers 2

Reset to default
4

Have you looked into Scala Actors or Akka? There may be no more compelling reason to use Scala than for parallel and asynchronous programming. It's baked into the language. Check out these facilities. I'm pretty sure you'll find what you need.

Improve this answer
Sign up to request clarification or add additional context in comments.

3 Comments

Um, no, but now that you mention it, I totally should have. But I've never used it, directly. Any pointers for this simple use case?
Sorry, I don't know enough about what you're trying to do to help. I can say though that Akka is probably overkill. Actors should be sufficient. This should help if you can get past the monotonous delivery: youtube.com/watch?v=bNLeo36pFls
It looks like Akka Futures will do the trick, so long as they're configured to run in their own processes: doc.akka.io/docs/akka/2.2.3/scala/futures.html
0

There's little information as regards the problem you're trying to solve here...previous answers are pretty much on the ball - look at Actors etc...Akka and you may find that you don't need to necessarily do anything too complicated. Introspection/reflection in a multi-threaded environment usually means a messy and not well thought-out strategy in terms of decomposing the problem in hand.

Improve this answer

2 Comments

Unfortunately, reflection is unavoidable as it's baked into (the extremely useful) scala-pickling at the moment.
So is it a problem around serialisation and complexity thereof?

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.