Creating a distributed memory service in Scala

Scala deals with immutable data and mutable data equally well.

Sure it's highly encouraged to use immutable data - for any data which can be shared across threads/actors.

But it's also highly encouraged to use mutable data dedicated to individual threads/actors - i.e. in a context which can never be shared across threads/actors.

A simple high-level concurrency pattern is:

1. Make Mutable Data Private To Each Actor:
   • dedicate mutable types to individual actors
   • manipulate mutable data with your actor act/recieve/react methods
2. Only Share Immutable Data Between Actors
   • send messages to other actors - but convert the mutable data to immutable data and only send the latter between actors.
3. Never Send a reference to an Actor in a message to Another Actor
   • Due to (1), Actors themselves become mutable
   • Due to (2), they shouldn't be shared
   • This stops ActorA from invoking any methods on ActorB - all communication should be via messages (methods ! and !!)
4. Identify long-running operations and make them asynchronous. Even simple operations that go beyond the CPU + L1 Cache + L2 Cache can can be 100,000s to 10,000,000s times slower than faster in-CPU operations, E.g. reading volumous data from memory, reading/writing to disk, reading/writing to the network - see table at end: http://norvig.com/21-days.html
5. As a rule, don't block on asynchronous operations - it usually "takes out" a thread (consumes resources), if not carefully designed can lead to deadlock (runtime failure) and can always be handled in a non-blocking manner. If you need to process an asynchronous response, have the asynchronous request return a Future[T] as part of the initiation. Then specify response handling code via Future.onComplete (or even better via Future.flatMap/Map/withFilter/Recover and their for-comprehension equivalents - which are handy 'monadic'-wrappers around onComplete).