Merging / Adding 2 JSON File together using Powershell

Question

I want to combine following 2 JSON files below:

This is the first JSON File which is the original JSON file

{
    "toolcache": [
        {
            "name": "Python",
            "platform" : "linux",
            "platform_version": "22.04",
            "versions": [
                "3.7.*"
            ]
        }
    ],
    "android": {
        "ndk": {
            "default": "23",
            "versions": [
                "21", "23", "24"
            ]
        }
    },
    "powershellModules": [
        {"name": "Pester"},
        {"name": "PSScriptAnalyzer"}
    ],
    "docker": {
        "images": [
            "alpine:3.14",
            "alpine:3.15",
            "buildpack-deps:buster",
            "buildpack-deps:bullseye",
            "debian:10",
            "debian:11",
            "moby/buildkit:latest",
            "node:14",
            "node:16",
            "node:14-alpine",
            "node:16-alpine"
        ]
    },
    "postgresql": {
        "version": "14"
    }
}

This is the second JSON File which we can update and we expect it to merge / add to the original first JSON File

{
    "toolcache": [
        {
            "name": "node",
            "platform" : "linux",
            "versions": [
                "16.*"
            ]
        }
    ],
    "android": {
        "cmdline-tools": "latest",
        "platform_min_version": "27",
        "build_tools_min_version": "27.0.0",
        "extra_list": [
            "android;m2repository",
            "google;m2repository",
            "google;google_play_services"
        ],
        "addon_list": [
        ],
        "additional_tools": [
            "cmake;3.10.2.4988404",
            "cmake;3.18.1"
        ]
    },
    "powershellModules": [
        {"name": "MarkdownPS"},
        {"name": "Microsoft.Graph"}
    ],
    "docker": {
        "images": [
            "ubuntu:18.04",
            "ubuntu:20.04",
            "ubuntu:22.04"
        ]
    }
}

Here is the expected result:

{
    "toolcache": [
        {
            "name": "Python",
            "platform" : "linux",
            "platform_version": "22.04",
            "versions": [
                "3.7.*"
            ]
        },
        {
            "name": "node",
            "platform" : "linux",
            "versions": [
                "16.*"
            ]
        }
    ],
    "android": {
        "cmdline-tools": "latest",
        "platform_min_version": "27",
        "build_tools_min_version": "27.0.0",
        "extra_list": [
            "android;m2repository",
            "google;m2repository",
            "google;google_play_services"
        ],
        "addon_list": [
        ],
        "additional_tools": [
            "cmake;3.10.2.4988404",
            "cmake;3.18.1"
        ],
        "ndk": {
            "default": "23",
            "versions": [
                "21", "23", "24"
            ]
        }
    },
    "powershellModules": [
        {"name": "Pester"},
        {"name": "PSScriptAnalyzer"},
        {"name": "MarkdownPS"},
        {"name": "Microsoft.Graph"}
    ],
    "docker": {
        "images": [
            "alpine:3.14",
            "alpine:3.15",
            "buildpack-deps:buster",
            "buildpack-deps:bullseye",
            "debian:10",
            "debian:11",
            "moby/buildkit:latest",
            "node:14",
            "node:16",
            "node:14-alpine",
            "node:16-alpine",
            "ubuntu:18.04",
            "ubuntu:20.04",
            "ubuntu:22.04"
        ]
    },
    "postgresql": {
        "version": "14"
    }
}

I am not too sure if it is possible to add both JSON File together as I've only seen ways to merge while overwriting the existing JSON File. I have also tried Join Object and @($source; $extend) but it was not that simple as it would just create a duplicate.

I am try to experiment with Add-Member but I am now stuck. Any help would be much appreciated.

The expected JSON contains items that are in neither of the input files, e. g. toolcache -> versions -> 3.8.*. Is this a mistake? — zett42
– zett42, Commented Jul 26, 2022 at 13:37

zett42 · Accepted Answer · 2022-07-27 10:02:10Z

2

It looks like you want to merge objects and append arrays. For this I have slightly modified the code of another answer of mine:

function Merge-Json( $source, $extend ){
    if( $source -is [PSCustomObject] -and $extend -is [PSCustomObject] ){

        # Ordered hashtable for collecting properties
        $merged = [ordered] @{}

        # Copy $source properties or overwrite by $extend properties recursively
        foreach( $Property in $source.PSObject.Properties ){
            if( $null -eq $extend.$($Property.Name) ){
                $merged[ $Property.Name ] = $Property.Value
            }
            else {
                $merged[ $Property.Name ] = Merge-Json $Property.Value $extend.$($Property.Name)
            }
        }

        # Add $extend properties
        foreach( $Property in $extend.PSObject.Properties ){
            if( $null -eq $source.$($Property.Name) ) {
                $merged[ $Property.Name ] = $Property.Value
            }
        }

        # Convert hashtable into PSCustomObject and output
        [PSCustomObject] $merged
    }
    elseif( $source -is [Collections.IList] -and $extend -is [Collections.IList] ){

        # Output merged array, using comma operator to prevent enumeration 
        , ($source + $extend)

        # To remove duplicates, replace the previous line by:
        # , (($source + $extend) | Sort-Object -Unique)
    }
    else{
        # Output extend object (scalar or different types)
        $extend
    }
}

Usage:

$data1 = Get-Content .\data1.json -Raw | ConvertFrom-Json
$data2 = Get-Content .\data2.json -Raw | ConvertFrom-Json
Merge-Json $data1 $data2 | ConvertTo-Json -Depth 100 | Set-Content merged.json -Encoding UTF8

Output:

{
  "toolcache": [
    {
      "name": "Python",
      "platform": "linux",
      "platform_version": "22.04",
      "versions": [
        "3.7.*"
      ]
    },
    {
      "name": "node",
      "platform": "linux",
      "versions": [
        "16.*"
      ]
    }
  ],
  "android": {
    "ndk": {
      "default": "23",
      "versions": [
        "21",
        "23",
        "24"
      ]
    },
    "cmdline-tools": "latest",
    "platform_min_version": "27",
    "build_tools_min_version": "27.0.0",
    "extra_list": [
      "android;m2repository",
      "google;m2repository",
      "google;google_play_services"
    ],
    "addon_list": [],
    "additional_tools": [
      "cmake;3.10.2.4988404",
      "cmake;3.18.1"
    ]
  },
  "powershellModules": [
    {
      "name": "Pester"
    },
    {
      "name": "PSScriptAnalyzer"
    },
    {
      "name": "MarkdownPS"
    },
    {
      "name": "Microsoft.Graph"
    }
  ],
  "docker": {
    "images": [
      "alpine:3.14",
      "alpine:3.15",
      "buildpack-deps:buster",
      "buildpack-deps:bullseye",
      "debian:10",
      "debian:11",
      "moby/buildkit:latest",
      "node:14",
      "node:16",
      "node:14-alpine",
      "node:16-alpine",
      "ubuntu:18.04",
      "ubuntu:20.04",
      "ubuntu:22.04"
    ]
  },
  "postgresql": {
    "version": "14"
  }
}

The output slightly differs from your expected output, but it looks like these are just minor mistakes on your side. Also, my function puts android.ndk at the beginning, as it is from the first file. This shouldn't matter, because the JSON standard doesn't specify the ordering of properties to be significant.

For detailed explanation of the code, see the answer linked at the top. The only difference here is the elseif branch, which appends arrays.

edited Jul 27, 2022 at 10:02

answered Jul 26, 2022 at 14:13

zett42

28.3k3 gold badges44 silver badges97 bronze badges

Sign up to request clarification or add additional context in comments.

6 Comments

ChaMango Over a year ago

Thanks for your promp response zett42! That solves my question! Out of curiosity, is there a way to check if there is any duplicates which can be ignored / avoided? Also apology for my minor mistake on my result.json file. Thanks in advance!

ChaMango Over a year ago

An example would be: JSON1: ` "docker": { "images": [ "node:14", "node:16", "node:16-alpine" ] }` JSON2: ` "docker": { "images": [ "node:14", "node:16", "node:14-alpine" ] }` Expected Result: ` "docker": { "images": [ "node:14", "node:16", "node:16-alpine", "node:14-alpine" ] }`

zett42 Over a year ago

@ChaMango This is possible, let me find some time later this day.

zett42 Over a year ago

@ChaMango Replace the line , ($source + $extend) by , (($source + $extend) | Sort-Object -Unique). I've also added it as comment in the code sample.

ChaMango Over a year ago

Hi @zett42, after attempting to merge using (($source + $extend) | Sort-Object -Unique) I noticed that the merged file did not correctly append the arrays. Instead it removed the array.

{   "toolcache": {     "name": "Python",     "platform": "linux",     "platform_version": "22.04",     "versions": [       "3.7.*"     ]

The toolcache object is outside of the [ ] object and it is removed. I noticed that it did not merge with

"toolcache": [  {       "name": "node",             "platform" : "linux",             "versions": [                 "16.*"             ]         }

|

Collectives™ on Stack Overflow

Merging / Adding 2 JSON File together using Powershell

1 Answer 1

6 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

6 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related