Curl post with JSON data fails

Question

I'm trying to use a heredoc to pass in the ONE param I need to make a POST request via curl, this is the error I run into

./scripts/etcd.sh: line 10: warning: here-document at line 10 delimited by end-of-file (wanted `EOF{peerURLs:[http://etcd-$ordinal:2380]}EOF')
./scripts/etcd.sh: line 9: warning: here-document at line 9 delimited by end-of-file (wanted `EOF{peerURLs:[http://etcd-$ordinal:2380]}EOF')
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100    42  100    42    0     0  12639      0 --:--:-- --:--:-- --:--:-- 21000
{"message":"unexpected end of JSON input"}

And the 2 lines in the script that trigger it

request_body=$(cat <<EOF{\"peerURLs\":[\"http://etcd-$ordinal:2380\"]}EOF); 
curl http://etcd-0.etcd:2379/v2/members -XPOST -H \"Content-Type: application/json\" --data \"$request_body\";

I've tried all the answers here Using curl POST with variables defined in bash script functions before asking my own question.

Edit:

From the answers and comments below I've tried

curl http://etcd-0.etcd:2379/v2/members -XPOST -H \"Content-Type: application/json\" --data @<(cat <<EOF\n{\"peerURLs\":[\"http://etcd-$ordinal:2380\"]}\nEOF)

And it works, but with giving a similar error

./scripts/etcd.sh: line 12: warning: here-document at line 10 delimited by end-of-file (wanted `EOF')
./scripts/etcd.sh: line 12: warning: here-document at line 10 delimited by end-of-file (wanted `EOF')
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
{"id":"a473da4d8f77b2b0","name":"","peerURLs":["http://etcd-3.etcd:2380"],"clientURLs":[]}

Here-docs don't work like this. The contents of it must start on a new line, and the end marker has to be on a line on its own. Your script thinks that the whole thing (EOF{\"peerURLs\":[\"http://etcd-$ordinal:2380\"]}EOF);) is your marker. — Benjamin W.
– Benjamin W., Commented Mar 24, 2017 at 2:28
You don't want to escape the quotes, either. curl -H "Content-Type..." --data "$request_body". — chepner
– chepner, Commented Mar 24, 2017 at 4:09
Consider using a tool like jq to generate your JSON, especially when you are embedding the contents of a parameter. — chepner
– chepner, Commented Mar 24, 2017 at 12:16

Community · Accepted Answer · 2017-05-23 11:46:19Z

4

Try:

request_body="{\"peerURLs\":[\"http://etcd-$ordinal:2380\"]}"

Here-docs (<<EOF\n ... \nEOF):

are inherently multi-line constructs (because yours isn't, you're getting the warning)
send their contents to stdin.

Neither aspect is required in your case, where a single-line, expandable string ("...") is sufficient.

Additionally, the way you use quoting in your curl command is broken - see bottom.

If you did want to use a here-doc with a variable, this is the most efficient idiom:

read -d '' -r request_body <<EOF
{"peerURLs":["http://etcd-$ordinal:2380"]}
EOF

^{Note: The closing delimiter, EOF here, must be on its own line, at the very start of the line (no leading whitespace allowed),
and no other characters may follow, not even whitespace and comments.}

read is used to read the here-doc's contents, via stdin (-d '' ensures that the entire here-doc is read as a whole, and -r is used to ensure that the content is read without interpretation of embedded \ chars.)

As you can see, the advantage here is that there's no need to escape embedded " chars. as \".

chepner points out that often you can use here-docs directly, without the need for an auxiliary variable; in this case, @- as the --data argument tells curl to read from stdin:

curl http://etcd-0.etcd:2379/v2/members -XPOST -H 'Content-Type: application/json' \
  --data @- <<EOF
{"peerURLs":["http://etcd-$ordinal:2380"]}
EOF

The above command also shows how to quote the -H option-argument correctly: using unescaped single-quoting to produce a string literal. By contrast, \"Content-Type: application/json\" would create 2 arguments, "Content-Type:and application/json" - with \" turning into embedded " chars.

Similarly, to use your $request_body variable instead of a here-doc, use "$request_body" rather than \"$request_body\".

edited May 23, 2017 at 11:46

CommunityBot

11 silver badge

answered Mar 24, 2017 at 2:31

mklement0

453k68 gold badges728 silver badges988 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

mklement0 Over a year ago

@Jonathan: It sounds like you're seeing an unrelated error. If you use the here-doc as posted in this answer with bash, it should work. Also note that, as chepner has already pointed out, your curl command is broken with respect to quoting - see my update.

Jonathan Over a year ago

My quoting use is needed, because I'm writing bash commands to a file with cat, within a kubernetes yaml file annotation where the whole block is surrounded by double quotes already, other than thatI agree with you that I'm still very very unfamiliar with heredocs :)

mklement0 Over a year ago

@Jonathan: I see - you should have mentioned that in your question to avoid confusion.

Collectives™ on Stack Overflow

Curl post with JSON data fails

1 Answer 1

3 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

3 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related