Skip to main content
edited tags
Link
200_success
  • 145.7k
  • 22
  • 191
  • 481
Tweeted twitter.com/StackCodeReview/status/1522274934956961792
edited tags
Link
Peter Csala
  • 10.8k
  • 1
  • 16
  • 36
Source Link
Peter Csala
  • 10.8k
  • 1
  • 16
  • 36

Json Array to ND Json converter

Problem

  • We have a tool which produces output as json array
  • We have another tool which anticipates input as ndjson (newline delimited json)
  • I need to write a converter to transform json array to ndjson

Sample json array

[
  {
    "Property1": true,
    "Property2": 0.1,
    "Property3": "text"
  },
  {
    "Property1": false,
    "Property2": 0.2,
    "Property3": "not text"
  },
  {
    "Property1": true,
    "Property2": 3.14,
    "Property3": "sample"
  },
  {
    "Property1": false,
    "Property2": -51.0,
    "Property3": "Property3"
  }
]

Sample ndjson

{"Property1":true,"Property2":0.1,"Property3":"text"}
{"Property1":false,"Property2":0.2,"Property3":"not text"}
{"Property1":true,"Property2":3.14,"Property3":"sample"}
{"Property1":false,"Property2":-51.0,"Property3":"Property3"}

Design

  • Keep the converter as simple as possible
    • The converter should receive a source and a target file paths
    • The converter should not parse the data just reformat it
    • The converter should perform only some basic preliminary checks

Solution

  • I'm comfortable to implement this with Json.NET
    • But I wanted to practice with System.Text.Json
    • So, I've decided to implement it with the latter
  • I've added some comments to the code to help the reviewers
public static class JsonArrayToNDJsonConverter
{
    static JsonWriterOptions writerOptions = new() { Indented = false };
    public static void Convert(string sourcePath, string targetPath)
    {
        //Preliminary checks on source
        if (!File.Exists(sourcePath))
            throw new FileNotFoundException("Source file is not found");
        using var sourceFile = new FileStream(sourcePath, FileMode.Open);

        var jsonDocument = JsonDocument.Parse(sourceFile);
        if (jsonDocument.RootElement.ValueKind != JsonValueKind.Array)
            throw new InvalidOperationException("Json Array must be the source");

        //Not that graceful handling of file existence
        if (File.Exists(targetPath))
            File.Delete(targetPath);
        using var targetFile = new FileStream(targetPath, FileMode.CreateNew);

        using var targetFileWriter = new StreamWriter(targetFile);
        using var jsonObjectWithoutIndentation = new MemoryStream();
        foreach (var jsonObject in jsonDocument.RootElement.EnumerateArray())
        {
            //Write json object without indentation into a memorystream
            var jsonObjectStreamWriter = new Utf8JsonWriter(jsonObjectWithoutIndentation, writerOptions);
            jsonObject.WriteTo(jsonObjectStreamWriter);
            jsonObjectStreamWriter.Flush();

            //Write memorystream to target
            var singleLinedJson = Encoding.UTF8.GetString(jsonObjectWithoutIndentation.ToArray());
            targetFileWriter.WriteLine(singleLinedJson);

            //Reuse memory stream
            jsonObjectWithoutIndentation.Position = 0;
            jsonObjectWithoutIndentation.SetLength(0);
        }
    }
}