Extract a specific value from an output stream in bash

Question

I have this command:

`curl -X POST https://server.domain.com/v2/jobs/$job_ID/runs?project_id=$project_ID -H "Content-Type: application/json" -H "Authorization: Bearer ${token}"  -d "{ \"job_run\": {} }"

and it gives this output

quote Response from job execution is {"metadata":{"rov":{"mode":0,"collaborator_ids":{}},"project_id":"aff59748-260a-476e-9578-b4f4a93e7a92","sandbox_id":"aff59748-260a-476e-9578-b4f4a93e7a92","usage":{"last_updated_at":"2021-02-12T00:31:19Z","last_updater_id":"1000331040","last_update_time":1613089879840,"last_accessed_at":"2021-02-12T00:31:19Z","last_access_time":1613089879840,"last_accessor_id":"1000331040","access_count":0},"name":"Notebook Job","description":"","tags":[],"asset_type":"job_run","origin_country":"us","rating":0,"total_ratings":0,"catalog_id":"d47cd45c-0161-4b89-ba4f-c0e48272f08e","created":1613089879840,"created_at":"2021-02-12T00:31:19Z","owner_id":"1000331040","size":0,"version":2,"asset_state":"available","asset_attributes":["job_run"],"asset_id":"5d9dae4e-0dfe-4e53-9e56-18a4bb44af","asset_category":"USER"},"entity":{"job_run":{"job_ref":"28723316-9373-44ba-9229-7c796f21b099","job_name":"Dummy Job for E2E Testing","job_type":"notebook","state":"Starting","isScheduledRun":false,"configuration":{"env_id":"jupconda37-aff59748-260a-476e-9578-b4f4a93e7a92","version":"ca0b9a58-ba0d-4d28-93d5-3cdc66a5b137","env_type":"notebook","env_variables":[],"job_manager_id":"9268a8a8-72e4-47a1-a851-65f38d8284ce","notebook_job_output":{},"command_line_arguments":[]},"project_name":"Tahoe Migration"}},"href":"/v2/assets/5d9dae4e-0dfe-4e53-9e56-ee18a4bb44af?project_id=aff59748-260a-476e-9578-b4f4a93e7a92"}

I want to extract the value of asset_id from the output stream and store it in a variable within shell script. I tried storing the response in a bash variable and tried to parse with awk, it didn't work. Could one of you please provide me an idea as to how to go about this?

Thank you

Gordon Davisson · Accepted Answer · 2021-02-12 06:13:06Z

2

jq is much better for processing json, although you need to strip off the "quote Response from job execution is " prefix first. Something like this:

response=$(curl ...)
asset_id=$(echo "${response#*Response from job execution is }" | jq -r '.metadata.asset_id')

answered Feb 12, 2021 at 6:13

Gordon Davisson

128k16 gold badges141 silver badges168 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

user3605317 Over a year ago

THanks a lot Gordon. It worked like a charm. You Sir are god sent to help folks like us. I couldn't figure this out after trying for hours. Hmm this jq tool is the magic, never knew about it. You really saved me from a lot of manual work. Thank you once again. God bless you

user3605317 Over a year ago

I accepted the answer now. Sorry about that

Mihai · Accepted Answer · 2021-02-12 06:26:10Z

0

If that's the only thing you need, you can solve it with sed:

<YOUR_CURL> | sed 's/.*"asset_id":"\([0-9a-z\-]*\)".*/\1/'

answered Feb 12, 2021 at 6:26

Mihai

2,1652 gold badges15 silver badges16 bronze badges

Collectives™ on Stack Overflow

Extract a specific value from an output stream in bash

2 Answers 2

2 Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

2 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related