update one json file with a second json based on third json with jq

Question

I have three JSON files:

vault.json:

{
  "aws":
  {
    "access_key_id": "My-Key-id",
    "secret_access_key": "My-Access-Key"
  },
  "ssl":
  {
    "crt": "My-Crt",
    "key": "My-Key",
    "req": "My-Req"
  }
}

input.json:

{
  ".cloud_cpi_key": {
    "type": "wildcard_domain",
    "configurable": true,
    "credential": false,
    "value": "vault-supplied-value",
    "optional": false
  },
  ".cloud_cpi_secret": {
    "type": "wildcard_domain",
    "configurable": true,
    "credential": false,
    "value": "vault-supplied-value",
    "optional": false
  },
  ".properties.networking_point_of_entry": {
    "type": "selector",
    "configurable": true,
    "credential": false,
    "value": "external_ssl",
    "optional": false
  },
  ".properties.networking_point_of_entry.external_ssl.ssl_rsa_certificate": {
    "type": "rsa_cert_credentials",
    "configurable": true,
    "credential": true,
    "value": {
      "private_key_pem": "vault-supplied-value",
      "cert_pem": "vault-supplied-value"
    },
    "optional": false
  }
}

keyfile.json

{
  ".cloud_cpi_key.value": "aws.access_key_id",
  ".cloud_cpi_secret": "secret_access_key",
  ".properties.networking_point_of_entry.external_ssl.ssl_rsa_certificate.value.private_key_pem": "ssl.key",
  ".properties.networking_point_of_entry.external_ssl.ssl_rsa_certificate.value.cert_pem": "ssl.crt"
}

I'd like to update the second json file, with the values from the first json, based on the third json. can this be done via JQ to provide output.json?

output.json:

{
  ".cloud_cpi_key": {
    "type": "string",
    "configurable": true,
    "credential": true,
    "value": "My-Key-id",
    "optional": false
  },
  ".cloud_cpi_secret": {
    "type": "string",
    "configurable": true,
    "credential": true,
    "value": "My-Access-Key",
    "optional": false
  },
  ".properties.networking_point_of_entry": {
    "type": "selector",
    "configurable": true,
    "credential": false,
    "value": "external_ssl",
    "optional": false
  },

".properties.networking_point_of_entry.external_ssl.ssl_rsa_certificate": {
    "type": "rsa_cert_credentials",
    "configurable": true,
    "credential": true,
    "value": {
      "private_key_pem": "My-Key",
      "cert_pem": "My-Crt"
    },
    "optional": false
  }
}

I can modify keyfile.json any way I like to make things easier, like

{
    "fromkey": "aws.access_key_id"
    "tokey": ".cloud_cpi_key.value"
},
{   "fromkey": ....
}

But no values may be placed in keyfile.json, only key names.

And I can modify vault.json, to put things in arrays, or what-have-you, but I cannot change the lowest levels, i.e. I cannot change:

{
    "access_key_id": "My-Key-id",
    "secret_access_key": "My-Access-Key"
}

I cannot modify input.json. How can I accomplish this with JQ?

peak · Accepted Answer · 2017-07-07 03:36:59Z

2

You don't give details about the transformation you have in mind, but if you can specify the algorithm, then rest assured, it can be done in jq.

Perhaps the thing you're missing is how to get jq to read the three files. One way to proceed would be to use the invocation:

jq --argfile keyfile keyfile.json --argfile vault vault.json -f vault.jq input.json

where vault.jq is the file containing your jq program, in which you would refer to the contents of keyfile.json as $keyfile, and similarly for the contents of vault.json

Simplifying keyfile.json

Since you indicate you have some flexibility about the format of keyfile.json, and since it appears to hold path information, I would recommend considering adopting path specifications that can be used directly with the jq builtins getpath and setpath.

For example, consider this format for keyfile.json:

[ [<path in input.json>], [<path in vault.json> ], ... ]

The first two entries corresponding to your example would thus be:

[
  [ [".cloud_cpi_key","value"], ["aws","access_key_id"]],
  [ [".cloud_cpi_secret"], ["aws", "secret_access_key"]]
]

To use the "vault" as the basis for updates, your jq program would then be the one-liner:

reduce $keyfile[] as $p (.; setpath(($p|.[0]); $vault|getpath($p|.[1])))

edited Jul 7, 2017 at 3:36

answered Jul 6, 2017 at 18:31

peak

119k21 gold badges185 silver badges218 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

AG6HQ Over a year ago

this works perfectly, and I never lost faith in JQ. I craft the keyfile by hand, so reforming to the example you gave, will work swimmingly! Thanks again!

Collectives™ on Stack Overflow

update one json file with a second json based on third json with jq

1 Answer 1

Simplifying keyfile.json

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Simplifying keyfile.json

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related