Serializing values to JSON array with pipes

Question

I'd like to serialize incoming values to JSON. Every value has a toJSON instance. The end result should be a list. The current code is the following:

import Pipes
import qualified Pipes.Prelude as P

-- assume a source of elements
main :: IO ()
main = runEffect $ source >-> P.map encode >-> P.stdoutLn

The problem is that in this way every line contains a valid JSON object, but I want the whole result to be parseable. I'd like that before the first object a [ character is outputted, then every element followed by a comma, and finally another ]. How can I do this with pipes?

Current output:

$ prog
{"key": "value"}
{"key": "value"}

Desired output:

$ prog
[{"key": "value"},
{"key": "value"}]

I found pipes-aeson but I don't understand how should I use the functions it provides.

EDIT: I modified ErikR's answer to get a Consumer, but it does not output the closing bracket:

jsonExporter :: Consumer (FilePath, AnalysisResult) IO ()
jsonExporter = do
    lift $ putStr "["
    P.map encode >-> insertCommas
    lift $ putStr "]"

I cannot understand why.

ErikR · Accepted Answer · 2015-11-01 13:54:59Z

1

This pipe segment:

for cat $ \x -> lift $ do { putStr ", "; putStrLn x }

will emit a comma before each element in the pipe.

To give the first element special treatment, we just unroll the loop once:

insertCommas = do
  x1 <- await
  lift $ putStrLn x1      -- print first element w/o a comma
  for cat $ \x -> lift $ do { putStr ", "; putStrLn x }

Now you can write your streaming JSON pipeline as:

putStr "["
runEffect $ source >-> P.map encode >-> insertCommas
putStrLn "]"

answered Nov 1, 2015 at 13:54

ErikR

52.2k9 gold badges78 silver badges136 bronze badges

Sign up to request clarification or add additional context in comments.

6 Comments

rubik Over a year ago

Great! I'm still getting used to lift and sometimes I find it hard to use it correctly.

rubik Over a year ago

Since Aeson's encode returns a ByteString I think I can use the intersperse function from the pipes-group package somehow? Instead of insertCommas I mean.

ErikR Over a year ago

You can try it, but I think that intersperses a character between every pair of characters in the Bytestring.

rubik Over a year ago

I needed a Consumer, so I modified your code (I put it in the question). But I cannot understand why the closing ] is never outputted.

ErikR Over a year ago

This should really be a new question. As a pipe segment insertCommas never returns, so that's why I print the brackets outside of the runEffect.

|

Collectives™ on Stack Overflow

Serializing values to JSON array with pipes

1 Answer 1

6 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

6 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related