bash var substitution inside json to use in curl

Question

Team, I am unable to substitute values inside json(myjson) to use in curl command. any hint?

JENKINS_USERNAME="svc-user"
JENKINS_USER_TOKEN="xxxx"
JENKINS_INSTANCE_FQDN="sham.blsm.comp.com"
INSTANCE="test-verify"
CREDENTIAL_ID_NAME="test"
USERNAME_KEY="tests"
USER_PASSWORD_KEY="testst"
ID_DESCRIPTION="asdfasdfasdf"

myjson=\''{ 
            "credentials": {
                "scope": "GLOBAL",
                "id": "'"${CREDENTIAL_ID_NAME}"'",
                "username": "'"${USERNAME_KEY}"'", 
                "password": "'"${USER_PASSWORD_KEY}"'",
                "description": "'"${ID_DESCRIPTION}"'",
                "\$class": "com.cloudbees.plugins.credentials.impl.UsernamePasswordCredentialsImpl"
            }
        }'\'
curl -X POST https://${JENKINS_USERNAME}:${JENKINS_USER_TOKEN}@${JENKINS_INSTANCE_FQDN}/${JENKINS_INSTANCE}/credentials/store/system/domain/_/createCredentials \
--data-urlencode ${myjson}

output

curl -X POST https://${JENKINS_USERNAME}:${JENKINS_USER_TOKEN}@${JENKINS_INSTANCE_FQDN}/${JENKINS_INSTANCE}/credentials/store/system/domain/_/createCredentials \
--data-urlencode ${myjson}
<html>
<head><title>404 Not Found</title></head>
<body>
<center><h1>404 Not Found</h1></center>
<hr><center>nginx</center>
</body>
</html>
curl: (3) URL using bad/illegal format or missing URL
curl: (3) unmatched brace in URL position 1:
{
 ^

below answer did not help me Bash variable substitution in a JSON string

choroba · Accepted Answer · 2023-12-20 22:36:54Z

1

The single quotes are part of the command line, not part of the value. They are removed in quote expansion before curl gets the value, so don't include them.

myjson='{ 
            "credentials": {
                "scope": "GLOBAL",
                "id": "'"${CREDENTIAL_ID_NAME}"'",
                "username": "'"${USERNAME_KEY}"'", 
                "password": "'"${USER_PASSWORD_KEY}"'",
                "description": "'"${ID_DESCRIPTION}"'",
                "$class": "com.cloudbees.plugins.credentials.impl.UsernamePasswordCredentialsImpl"
            }
        }'

(Note that the dollar sign needs no escaping in single quotes.)

You need to double quote the variable if you want to prevent word splitting:

curl -X POST https://${JENKINS_USERNAME}:${JENKINS_USER_TOKEN}@${JENKINS_INSTANCE_FQDN}/${JENKINS_INSTANCE}/credentials/store/system/domain/_/createCredentials \
--data-urlencode "${myjson}"

Note that this only works if no variable contains a double quote. It's better to use a JSON-aware tool to handle more complex cases, I'd recommend studying jq.

edited Dec 20, 2023 at 22:36

answered Dec 20, 2023 at 22:31

choroba

49.7k7 gold badges92 silver badges119 bronze badges

thanks.. at least am not getting the curl error now with your help but somehow the server is returning 404 where as it works perfectly when I use static values. so may be $class is not being intercepted?

AhmFM
– AhmFM

2023-12-20 22:39:59 +00:00
Commented Dec 20, 2023 at 22:39
Please update the question with the JSON you expect to get.

choroba
– choroba

2023-12-20 22:44:44 +00:00
Commented Dec 20, 2023 at 22:44
your answer worked on jenkins server pod. so am happy thanks.

AhmFM
– AhmFM

2023-12-20 23:23:14 +00:00
Commented Dec 20, 2023 at 23:23
1

This would unfortunately fail if the password contains double quotes or other characters that need to be encoded.

Kusalananda
– Kusalananda ♦

2023-12-21 08:12:38 +00:00
Commented Dec 21, 2023 at 8:12

Add a comment |

Kusalananda · Accepted Answer · 2023-12-21 08:49:44Z

The safest way to create a JSON document from a set of shell variables is by using jq. This ensures that each string value is encoded appropriately if needed (e.g. if the password includes double quotes or backslashes or other characters that need encoding in JSON).

payload=$(
    jq -n \
        --arg scope GLOBAL \
        --arg id "$CREDENTIAL_ID_NAME" \
        --arg username "$USERNAME_KEY" \
        --arg password "$USER_PASSWORD_KEY" \
        --arg description "$ID_DESCRIPTION" \
        --arg '$class' com.cloudbees.plugins.credentials.impl.UsernamePasswordCredentialsImpl \
        '{ credentials: $ARGS.named }'
)

$ARGS.named is a JSON object containing all the keys+values defined with --arg on the jq command line (use --argjson if a value is already an encoded JSON value).

Given the variables shown in the question, this would create a payload shell variable with the following contents (use jq with its -c option to create more compact output):

{
  "credentials": {
    "scope": "GLOBAL",
    "id": "test",
    "username": "tests",
    "password": "testst",
    "description": "asdfasdfasdf",
    "$class": "com.cloudbees.plugins.credentials.impl.UsernamePasswordCredentialsImpl"
  }
}

For a user password that is the string """''\\\!! followed by the invisible character produced by Ctrl+G, the password key's value would automatically be encoded as the JSON string \"\"\"''\\\\\\!!\u0007.

You would then use that with curl, just remember to double quote the expansion of the variable.

Stack Exchange Network

bash var substitution inside json to use in curl

2 Answers 2

You must log in to answer this question.

Linked

Hot Network Questions

bash var substitution inside json to use in curl

2 Answers 2

You must log in to answer this question.

Linked

Related

Hot Network Questions