Functional core - Imperative shell: dealing with logic dependant on conditional & expensive IO operations

Since this is a frequently-asked question about functional programming, I wrote a 20-part article series to cover as many aspects of this problem as possible. Unfortunately, it's not in Kotlin, but rather has examples in C#, F#, and Haskell. Even so, I hope you can read enough of at least one of these languages to get something out of the articles.

In short, however, there are plenty of 'patterns' you can apply, each with their own limitations, but fetching data up-front is by far the simplest. As I write in one of those articles, data sets that may seem prohibitively large to humans turn out to be perhaps a few hundred megabytes, and often quite amenable to caching.

If all else fails, however, you can model 'effects' and 'continuations' as free monads, but I consider that a last resort in most languages, because in languages like C# or Java, at least, it would be so unidiomatic that it's hardly warranted. Even in F# I would consider it exotic.

As to the 'specific' example, it's hard to answer, exactly because the devil's in the details. How 'heavy' is heavyCheck? How much data is involved in pre-fetching findParticipantsMatching for both criteria? If not too much, the simplest design is still to load it all up front, but if not, you define a functor that describes the expensive actions, and then turn it into a monad using the free construction.

I'm sorry I don't know how to do this in Kotlin, but I can do it in either Haskell, F#, or C# if you'd like, although the C# version would be highly unidiomatic and complicated.

FWIW, here's a sketch of implementing the above 'procedural/quasi-OO example' in F# using a free monad. I'll mostly follow the free monad recipe, with a few additions to the computation expression builder taken from Song recommendations with F# free monads.

First, we need to define the instruction set. I didn't know exactly what to call it, so just settled on 'Q' for (Stack Overflow) Question.

type QInstruction<'a> =
    | HeavyDbCheck of (bool -> 'a)
    | FindParticipantsMatching of (Criterion * (Participant list -> 'a))
    | SendNotificationEmail of (Participant * (unit -> 'a))
    | SendRefusalEmail of (Participant * (unit -> 'a))

Because of the 'a type parameter in the covariant position, you can make this type a functor. Try it, it's a good exercise. (Hint: You'll need to define a map function with the type ('a -> 'b) -> QInstruction<'a> -> QInstruction<'b>.)

We now define the free construction and turn it into a monad:

type QProgram<'a> =
    | Free of QInstruction<QProgram<'a>>
    | Pure of 'a

let rec bind f = function
    | Free instr -> instr |> QI.map (bind f) |> Free
    | Pure x -> f x

where the QI.map function is the functor map left as an exercise, above.

Not strictly necessary, but nice for syntactic sugar, now define a computation expression builder:

type QBuilder () =
    member _.Bind (x, f) = bind f x
    member _.Return x = Pure x
    member _.ReturnFrom x = x
    member _.Zero () = Pure ()
    member this.While (guard, body) =
        if not (guard ())
        then this.Zero ()
        else this.Bind (body (), fun () -> this.While (guard, body))
    member this.TryFinally (body, compensation) =
        try this.ReturnFrom (body ())
        finally compensation ()
    member this.Using (disposable : #System.IDisposable, body) =
        let body' = fun () -> body disposable
        this.TryFinally (body', fun () ->
            match disposable with
            | null -> ()
            | disp -> disp.Dispose ())
    member this.For (sequence : seq<_>, body) =
        this.Using (sequence.GetEnumerator(), fun enum ->
            this.While (enum.MoveNext, fun () -> body enum.Current))

let q = QBuilder ()

Using a few helper functions, you can now write the desired program, which has the type QProgram<unit>:

let example allowed =
    q {
        if allowed
        then
            let! dbOk = heavyDbCheck
            if dbOk
            then
                let! participants = findParticipantsMatching CriterionA
                for p in participants do
                    do! sendNotificationEmail p
            else
                let! participants = findParticipantsMatching CriterionB
                for p in participants do
                    do! sendRefusalEmail p
    }

This programs produced by example true and example false are a pure values, and you can run various interpreters over them in order to 'run' them.

As you can tell, there's quite a bit of boilerplate involved, so even in F#, where the end result is nice and idiomatic, I'd usually look for other alternatives first. In Haskell, on the other hand, most of this automatically follows from type definitions, so it's much more lightweight, and therefore more idiomatic, there.

All that said, the problem with these kinds of 'what-if' principal questions is that the examples are often so vague that there's almost no logic left. I've previously demonstrated a realistic example of that in the article Refactoring registration flow to functional architecture. Once you allow the introduction of some standard combinators, there's not much logic left.

For instance, in Haskell, imagine that we have the four concrete (i.e. non-abstract) IO-bound actions heavyDbCheck, findParticipantsMatching, sendNotificationEmail, and sendRefusalEmail. In that case, you can compose the desired action entirely from combinators:

example :: Bool -> IO ()
example allowed = do
  when allowed $
    heavyDbCheck >>=
      bool
        (findParticipantsMatching CriterionB >>= mapM_ sendRefusalEmail)
        (findParticipantsMatching CriterionA >>= mapM_ sendNotificationEmail)

As I point out in Song recommendations from combinators, such a composition is an entirely impure action, so not strictly 'functional' , but on the other hand, simple enough that if pushed to the boundary of the application architecture, is that much of a problem?

To add another take, which I found immediately applicable, I found a little jewel of a presentation by Scott Wlaschin at NDC London: Moving IO to the edges of your app: Functional Core, Imperative Shell. In it, he addresses this in a very pragmatic way.

Now, people always say, "What if I really, really, really need to do I/O in the middle of my workflow? Please, please, I need to do this.” Well, yes, of course, sometimes you need to load something up. You make a decision, you need to load a particular record from the database, and then, based on that, you make another decision, and then you know, save something. That's a legitimate workflow. Sometimes. I think it's less common than people think, but yeah, it's legitimate. Your answer is, you just break it into pieces, the same way, right? You break it into little pieces, and each piece is is either pure or it's I/O, but not both, right? So each of the little pieces are testable. Each little piece can be unit tested.

I can live with that, if that allows the code to be simpler, even if not as elegant as some Lisp where even the lazily evaluated impure-commands would still just be data I could assert on.

The take-aways with regards to practical applicability is this

• Do try to encapsulate the outcome of the functional core in decisions, commands, effects or whatever you want to call it

• There is no need to build a general command processor: make the code as simple and testable as can be for the specific case and adjust as you go.

• If you have a workflow where it's infeasible to fetch everything up-front, that's fine. Break the workflow into pieces and unit test each piece.

With regards to several unit tests not giving you assurance that everything fits together functionally (hence the reference to this joke), you can always cover that entire workflow with a E2E test.