Convert JSON to a PowerShell Object, then, replace that Object with a new Object and convert back to JSON

Question

Background

I have some JSON for a Packer build like so:

{
  "builders": [
    {
      "type": "azure-arm"
    }
  ],
  "provisioners": [
    {
      "type": "shell",
      "scripts": [
        "{{template_dir}}/scripts/installers/docker-compose.sh",
        "{{template_dir}}/scripts/installers/docker-moby.sh"
      ],
      "environment_vars": [
        "HELPER_SCRIPTS\u003d{{user `helper_script_folder`}}",
        "INSTALLER_SCRIPT_FOLDER\u003d{{user `installer_script_folder`}}",
        "DOCKERHUB_LOGIN\u003d{{user `dockerhub_login`}}",
        "DOCKERHUB_PASSWORD\u003d{{user `dockerhub_password`}}"
      ],
      "execute_command": "sudo sh -c \u0027{{ .Vars }} {{ .Path }}\u0027"
    },
    {
      "type": "shell",
      "scripts": [
        "{{template_dir}}/scripts/installers/azcopy.sh",
        "{{template_dir}}/scripts/installers/azure-cli.sh",
        "{{template_dir}}/scripts/installers/azure-devops-cli.sh",
        "{{template_dir}}/scripts/installers/basic.sh",
        "{{template_dir}}/scripts/installers/bicep.sh",
        "{{template_dir}}/scripts/installers/aliyun-cli.sh",
        "{{template_dir}}/scripts/installers/apache.sh",
        "{{template_dir}}/scripts/installers/aws.sh",
        "{{template_dir}}/scripts/installers/clang.sh",
        "{{template_dir}}/scripts/installers/swift.sh",
        "{{template_dir}}/scripts/installers/cmake.sh",
        "{{template_dir}}/scripts/installers/codeql-bundle.sh",
        "{{template_dir}}/scripts/installers/containers.sh",
        "{{template_dir}}/scripts/installers/dotnetcore-sdk.sh",
        "{{template_dir}}/scripts/installers/erlang.sh",
        "{{template_dir}}/scripts/installers/firefox.sh",
        "{{template_dir}}/scripts/installers/gcc.sh",
        "{{template_dir}}/scripts/installers/gfortran.sh",
        "{{template_dir}}/scripts/installers/git.sh",
        "{{template_dir}}/scripts/installers/github-cli.sh",
        "{{template_dir}}/scripts/installers/google-chrome.sh",
        "{{template_dir}}/scripts/installers/google-cloud-sdk.sh",
        "{{template_dir}}/scripts/installers/haskell.sh",
        "{{template_dir}}/scripts/installers/heroku.sh",
        "{{template_dir}}/scripts/installers/hhvm.sh",
        "{{template_dir}}/scripts/installers/java-tools.sh",
        "{{template_dir}}/scripts/installers/kubernetes-tools.sh",
        "{{template_dir}}/scripts/installers/oc.sh",
        "{{template_dir}}/scripts/installers/leiningen.sh",
        "{{template_dir}}/scripts/installers/miniconda.sh",
        "{{template_dir}}/scripts/installers/mono.sh",
        "{{template_dir}}/scripts/installers/kotlin.sh",
        "{{template_dir}}/scripts/installers/mysql.sh",
        "{{template_dir}}/scripts/installers/mssql-cmd-tools.sh",
        "{{template_dir}}/scripts/installers/sqlpackage.sh",
        "{{template_dir}}/scripts/installers/nginx.sh",
        "{{template_dir}}/scripts/installers/nvm.sh",
        "{{template_dir}}/scripts/installers/nodejs.sh",
        "{{template_dir}}/scripts/installers/bazel.sh",
        "{{template_dir}}/scripts/installers/oras-cli.sh",
        "{{template_dir}}/scripts/installers/phantomjs.sh",
        "{{template_dir}}/scripts/installers/php.sh",
        "{{template_dir}}/scripts/installers/postgresql.sh",
        "{{template_dir}}/scripts/installers/pulumi.sh",
        "{{template_dir}}/scripts/installers/ruby.sh",
        "{{template_dir}}/scripts/installers/r.sh",
        "{{template_dir}}/scripts/installers/rust.sh",
        "{{template_dir}}/scripts/installers/julia.sh",
        "{{template_dir}}/scripts/installers/sbt.sh",
        "{{template_dir}}/scripts/installers/selenium.sh",
        "{{template_dir}}/scripts/installers/smallstep-cli.sh",
        "{{template_dir}}/scripts/installers/terraform.sh",
        "{{template_dir}}/scripts/installers/packer.sh",
        "{{template_dir}}/scripts/installers/vcpkg.sh",
        "{{template_dir}}/scripts/installers/dpkg-config.sh",
        "{{template_dir}}/scripts/installers/mongodb.sh",
        "{{template_dir}}/scripts/installers/android.sh",
        "{{template_dir}}/scripts/installers/yq.sh",
        "{{template_dir}}/scripts/installers/pypy.sh",
        "{{template_dir}}/scripts/installers/python.sh",
        "{{template_dir}}/scripts/installers/graalvm.sh"
      ],
      "environment_vars": [
        "HELPER_SCRIPTS\u003d{{user `helper_script_folder`}}",
        "INSTALLER_SCRIPT_FOLDER\u003d{{user `installer_script_folder`}}",
        "DEBIAN_FRONTEND\u003dnoninteractive"
      ],
      "execute_command": "sudo sh -c \u0027{{ .Vars }} {{ .Path }}\u0027"
    },
    {
      "type": "shell",
      "scripts": [
        "{{template_dir}}/scripts/installers/Install-Toolset.ps1",
        "{{template_dir}}/scripts/installers/Configure-Toolset.ps1"
      ],
      "environment_vars": [
        "HELPER_SCRIPTS\u003d{{user `helper_script_folder`}}",
        "INSTALLER_SCRIPT_FOLDER\u003d{{user `installer_script_folder`}}"
      ],
      "execute_command": "sudo sh -c \u0027{{ .Vars }} pwsh -f {{ .Path }}\u0027"
    }
  ]
}

I am trying to add some items inside the provisioners list, but only inside the second index list called scripts, so in this case the scripts object. Ideally doing this all in PowerShell (I am using PowerShell 7 latest).

What I have tried

Doing some Googling around, it was suggested that I convert the JSON to a PowerShell Object, and then attempt to add to that instead:

$JsonFile = "build.json"
$JsonObject = $(Get-Content $JsonFile -Raw)
$SystemObject=$($JsonObject | ConvertFrom-Json)
$OldScriptsJson = $SystemObject.provisioners[1].scripts | ConvertTo-Json

Which gives me the old JSON which I want to add to:

[
  "{{template_dir}}/scripts/installers/azcopy.sh",
  "{{template_dir}}/scripts/installers/azure-cli.sh",
  "{{template_dir}}/scripts/installers/azure-devops-cli.sh",
  "{{template_dir}}/scripts/installers/basic.sh",
  "{{template_dir}}/scripts/installers/bicep.sh",
  "{{template_dir}}/scripts/installers/aliyun-cli.sh",
  "{{template_dir}}/scripts/installers/apache.sh",
  "{{template_dir}}/scripts/installers/aws.sh",
  "{{template_dir}}/scripts/installers/clang.sh",
  "{{template_dir}}/scripts/installers/swift.sh",
  "{{template_dir}}/scripts/installers/cmake.sh",
  "{{template_dir}}/scripts/installers/codeql-bundle.sh",
  "{{template_dir}}/scripts/installers/containers.sh",
  "{{template_dir}}/scripts/installers/dotnetcore-sdk.sh",
  "{{template_dir}}/scripts/installers/erlang.sh",
  "{{template_dir}}/scripts/installers/firefox.sh",
  "{{template_dir}}/scripts/installers/gcc.sh",
  "{{template_dir}}/scripts/installers/gfortran.sh",
  "{{template_dir}}/scripts/installers/git.sh",
  "{{template_dir}}/scripts/installers/github-cli.sh",
  "{{template_dir}}/scripts/installers/google-chrome.sh",
  "{{template_dir}}/scripts/installers/google-cloud-sdk.sh",
  "{{template_dir}}/scripts/installers/haskell.sh",
  "{{template_dir}}/scripts/installers/heroku.sh",
  "{{template_dir}}/scripts/installers/hhvm.sh",
  "{{template_dir}}/scripts/installers/java-tools.sh",
  "{{template_dir}}/scripts/installers/kubernetes-tools.sh",
  "{{template_dir}}/scripts/installers/oc.sh",
  "{{template_dir}}/scripts/installers/leiningen.sh",
  "{{template_dir}}/scripts/installers/miniconda.sh",
  "{{template_dir}}/scripts/installers/mono.sh",
  "{{template_dir}}/scripts/installers/kotlin.sh",
  "{{template_dir}}/scripts/installers/mysql.sh",
  "{{template_dir}}/scripts/installers/mssql-cmd-tools.sh",
  "{{template_dir}}/scripts/installers/sqlpackage.sh",
  "{{template_dir}}/scripts/installers/nginx.sh",
  "{{template_dir}}/scripts/installers/nvm.sh",
  "{{template_dir}}/scripts/installers/nodejs.sh",
  "{{template_dir}}/scripts/installers/bazel.sh",
  "{{template_dir}}/scripts/installers/oras-cli.sh",
  "{{template_dir}}/scripts/installers/phantomjs.sh",
  "{{template_dir}}/scripts/installers/php.sh",
  "{{template_dir}}/scripts/installers/postgresql.sh",
  "{{template_dir}}/scripts/installers/pulumi.sh",
  "{{template_dir}}/scripts/installers/ruby.sh",
  "{{template_dir}}/scripts/installers/r.sh",
  "{{template_dir}}/scripts/installers/rust.sh",
  "{{template_dir}}/scripts/installers/julia.sh",
  "{{template_dir}}/scripts/installers/sbt.sh",
  "{{template_dir}}/scripts/installers/selenium.sh",
  "{{template_dir}}/scripts/installers/smallstep-cli.sh",
  "{{template_dir}}/scripts/installers/terraform.sh",
  "{{template_dir}}/scripts/installers/packer.sh",
  "{{template_dir}}/scripts/installers/vcpkg.sh",
  "{{template_dir}}/scripts/installers/dpkg-config.sh",
  "{{template_dir}}/scripts/installers/mongodb.sh",
  "{{template_dir}}/scripts/installers/android.sh",
  "{{template_dir}}/scripts/installers/yq.sh",
  "{{template_dir}}/scripts/installers/pypy.sh",
  "{{template_dir}}/scripts/installers/python.sh",
  "{{template_dir}}/scripts/installers/graalvm.sh"
]

What I want to do

I basically want to add the following JSON to the old JSON

[
    "{{template_dir}}/scripts/installers/colordiff.sh",
    "{{template_dir}}/scripts/installers/kubelogin.sh",
    "{{template_dir}}/scripts/installers/kubeval.sh",
    "{{template_dir}}/scripts/installers/kubeaudit.sh",
    "{{template_dir}}/scripts/installers/smallstep-cli.sh"
  ]

So the final product of all of this can be saved back to the original JSON:

{
  "builders": [
    {
      "type": "azure-arm"
    }
  ],
  "provisioners": [
    {
      "type": "shell",
      "scripts": [
        "{{template_dir}}/scripts/installers/docker-compose.sh",
        "{{template_dir}}/scripts/installers/docker-moby.sh"
      ],
      "environment_vars": [
        "HELPER_SCRIPTS\u003d{{user `helper_script_folder`}}",
        "INSTALLER_SCRIPT_FOLDER\u003d{{user `installer_script_folder`}}",
        "DOCKERHUB_LOGIN\u003d{{user `dockerhub_login`}}",
        "DOCKERHUB_PASSWORD\u003d{{user `dockerhub_password`}}"
      ],
      "execute_command": "sudo sh -c \u0027{{ .Vars }} {{ .Path }}\u0027"
    },
    {
      "type": "shell",
      "scripts": [
        "{{template_dir}}/scripts/installers/azcopy.sh",
        "{{template_dir}}/scripts/installers/azure-cli.sh",
        "{{template_dir}}/scripts/installers/azure-devops-cli.sh",
        "{{template_dir}}/scripts/installers/basic.sh",
        "{{template_dir}}/scripts/installers/bicep.sh",
        "{{template_dir}}/scripts/installers/aliyun-cli.sh",
        "{{template_dir}}/scripts/installers/apache.sh",
        "{{template_dir}}/scripts/installers/aws.sh",
        "{{template_dir}}/scripts/installers/clang.sh",
        "{{template_dir}}/scripts/installers/swift.sh",
        "{{template_dir}}/scripts/installers/cmake.sh",
        "{{template_dir}}/scripts/installers/codeql-bundle.sh",
        "{{template_dir}}/scripts/installers/containers.sh",
        "{{template_dir}}/scripts/installers/dotnetcore-sdk.sh",
        "{{template_dir}}/scripts/installers/erlang.sh",
        "{{template_dir}}/scripts/installers/firefox.sh",
        "{{template_dir}}/scripts/installers/gcc.sh",
        "{{template_dir}}/scripts/installers/gfortran.sh",
        "{{template_dir}}/scripts/installers/git.sh",
        "{{template_dir}}/scripts/installers/github-cli.sh",
        "{{template_dir}}/scripts/installers/google-chrome.sh",
        "{{template_dir}}/scripts/installers/google-cloud-sdk.sh",
        "{{template_dir}}/scripts/installers/haskell.sh",
        "{{template_dir}}/scripts/installers/heroku.sh",
        "{{template_dir}}/scripts/installers/hhvm.sh",
        "{{template_dir}}/scripts/installers/java-tools.sh",
        "{{template_dir}}/scripts/installers/kubernetes-tools.sh",
        "{{template_dir}}/scripts/installers/oc.sh",
        "{{template_dir}}/scripts/installers/leiningen.sh",
        "{{template_dir}}/scripts/installers/miniconda.sh",
        "{{template_dir}}/scripts/installers/mono.sh",
        "{{template_dir}}/scripts/installers/kotlin.sh",
        "{{template_dir}}/scripts/installers/mysql.sh",
        "{{template_dir}}/scripts/installers/mssql-cmd-tools.sh",
        "{{template_dir}}/scripts/installers/sqlpackage.sh",
        "{{template_dir}}/scripts/installers/nginx.sh",
        "{{template_dir}}/scripts/installers/nvm.sh",
        "{{template_dir}}/scripts/installers/nodejs.sh",
        "{{template_dir}}/scripts/installers/bazel.sh",
        "{{template_dir}}/scripts/installers/oras-cli.sh",
        "{{template_dir}}/scripts/installers/phantomjs.sh",
        "{{template_dir}}/scripts/installers/php.sh",
        "{{template_dir}}/scripts/installers/postgresql.sh",
        "{{template_dir}}/scripts/installers/pulumi.sh",
        "{{template_dir}}/scripts/installers/ruby.sh",
        "{{template_dir}}/scripts/installers/r.sh",
        "{{template_dir}}/scripts/installers/rust.sh",
        "{{template_dir}}/scripts/installers/julia.sh",
        "{{template_dir}}/scripts/installers/sbt.sh",
        "{{template_dir}}/scripts/installers/selenium.sh",
        "{{template_dir}}/scripts/installers/smallstep-cli.sh",
        "{{template_dir}}/scripts/installers/terraform.sh",
        "{{template_dir}}/scripts/installers/packer.sh",
        "{{template_dir}}/scripts/installers/vcpkg.sh",
        "{{template_dir}}/scripts/installers/dpkg-config.sh",
        "{{template_dir}}/scripts/installers/mongodb.sh",
        "{{template_dir}}/scripts/installers/android.sh",
        "{{template_dir}}/scripts/installers/yq.sh",
        "{{template_dir}}/scripts/installers/pypy.sh",
        "{{template_dir}}/scripts/installers/python.sh",
        "{{template_dir}}/scripts/installers/graalvm.sh",
        "{{template_dir}}/scripts/installers/colordiff.sh",
        "{{template_dir}}/scripts/installers/kubelogin.sh",
        "{{template_dir}}/scripts/installers/kubeval.sh",
        "{{template_dir}}/scripts/installers/kubeaudit.sh",
        "{{template_dir}}/scripts/installers/smallstep-cli.sh"
      ],
      "environment_vars": [
        "HELPER_SCRIPTS\u003d{{user `helper_script_folder`}}",
        "INSTALLER_SCRIPT_FOLDER\u003d{{user `installer_script_folder`}}",
        "DEBIAN_FRONTEND\u003dnoninteractive"
      ],
      "execute_command": "sudo sh -c \u0027{{ .Vars }} {{ .Path }}\u0027"
    },
    {
      "type": "shell",
      "scripts": [
        "{{template_dir}}/scripts/installers/Install-Toolset.ps1",
        "{{template_dir}}/scripts/installers/Configure-Toolset.ps1"
      ],
      "environment_vars": [
        "HELPER_SCRIPTS\u003d{{user `helper_script_folder`}}",
        "INSTALLER_SCRIPT_FOLDER\u003d{{user `installer_script_folder`}}"
      ],
      "execute_command": "sudo sh -c \u0027{{ .Vars }} pwsh -f {{ .Path }}\u0027"
    }
  ]
}

How can I go about that using PowerShell?

zett42 · Accepted Answer · 2022-05-09 15:10:06Z

0

This should do it:

$json = Get-Content 'Build.json' -Raw | ConvertFrom-Json

# Define an array of strings to add.
$scriptsToAdd = @(
    '{{template_dir}}/scripts/installers/colordiff.sh'
    '{{template_dir}}/scripts/installers/kubelogin.sh'
    '{{template_dir}}/scripts/installers/kubeval.sh'
    '{{template_dir}}/scripts/installers/kubeaudit.sh'
    '{{template_dir}}/scripts/installers/smallstep-cli.sh'
)
# Alternatively read these from a JSON file as well:
# $scriptsToAdd = Get-Content 'ScriptsToAdd.json' -Raw | ConvertFrom-Json

# Append the $scriptsToAdd array to the scripts array of the JSON
$json.provisioners[1].scripts += $scriptsToAdd

# Make sure to specify a large enough argument for -Depth, which defaults to 2.
# Otherwise objects that are nested deeper will be truncated.
$json | ConvertTo-Json -Depth 100 | Set-Content 'BuildNew.json'

As for what you have tried:

$OldScriptsJson = $SystemObject.provisioners[1].scripts | ConvertTo-Json

You just converted part of the original JSON object back to a JSON string. When working with JSON, you should always operate on the object and only as a final step, convert back to the serialized JSON, which can be a string or a file.

edited May 9, 2022 at 15:10

answered May 9, 2022 at 14:33

zett42

28.3k3 gold badges44 silver badges97 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

craigthackerx Over a year ago

Works perfect! Thank you. You do not want to know how long I was stuck on this.

js2010 · Accepted Answer · 2022-05-09 14:40:44Z

0

$SystemObject = get-content build.json | convertfrom-json

# load changes into another object
$addobject = get-content add.json | convertfrom-json

# add 2 arrays together (+= kills puppies in bulk)
$SystemObject.provisioners[1].scripts += $addobject

# save, watch out for default depth of 2, the latest ps 7 gives a warning 
# "WARNING: Resulting JSON is truncated as serialization has exceeded the set depth
# of 2."
$SystemObject | ConvertTo-Json -Depth 3 | set-content build2.json

edited May 9, 2022 at 14:40

answered May 9, 2022 at 14:35

js2010

28.3k6 gold badges82 silver badges88 bronze badges

1 Comment

zett42 Over a year ago

Nice! We almost got the same solution. Note that ConvertFrom-Json may behave strangely when passing multiple strings (lines) as input, so it is recommended to always pass -Raw to Get-Content to pass a single (possibly multiline) string to ConvertFrom-Json.

Collectives™ on Stack Overflow

Convert JSON to a PowerShell Object, then, replace that Object with a new Object and convert back to JSON

Background

What I have tried

What I want to do

2 Answers 2

1 Comment

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

Background

What I have tried

What I want to do

2 Answers 2

1 Comment

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related