Using Python Requests to send file and JSON in single request

Question

I'm trying to POST to an API (Build using SlimPHP) which accepts an image along with additional image meta data in the form of JSON.

I've verified the API works correctly using a REST client tool and can successfully POST to the service. All data is stored correctly.

I'm now trying to POST using Python - however my JSON data doesn't appear to be saving.

My code:

    data = {'key1': 'value1', 'key2': 'value2'}
    url = 'http://mydomain.com/api/endpoint'
    headers = {'Authorization': 'my-api-key'}
    files = {'file': (FILE, open(PATH, 'rb'), 'image/jpg', {'Expires': '0'})}
    r = requests.post(url, files=files, headers=headers, data=data)

--

I've attempted to set additional headers,

ie:/

headers = {'Authorization': 'unique-auth-key', 'Content-type': 'multipart/form-data'}

or

headers = {'Authorization': 'unique-auth-key', 'Content-type': 'application/json'}

These result in a 500 error.

UPDATE 14/07/2014:

Using a chrome extension (Advanced Rest Client) my POST is successful - here's what the console shows as the payload:

------WebKitFormBoundarysBpiwrA3hnGPUbMA
Content-Disposition: form-data; name="data"
test
------WebKitFormBoundarysBpiwrA3hnGPUbMA
Content-Disposition: form-data; name="file"; filename="image.jpg"
Content-Type: image/jpeg
------WebKitFormBoundarysBpiwrA3hnGPUbMA--

I'm not quite sure what this signifies...

You have to provide API specification for the call to run. A sample using curl, HTTPie or similir tool would help. Usually, the JSON data are sent in body as the only content. If you want to combine it with file upload, you are probably going to use sort of form, with one file being the image, and another text field containing the JSON. Anyway, data shall be a string, you provide a dictionary. Changing it to data=json.dumps(data) could help. — Jan Vlcinsky
– Jan Vlcinsky, Commented Jul 8, 2014 at 22:02
This cURL request works: curl -i -F data='{"key": "value", "key": "value", "key": "value"}' -F name=image.jpg -F [email protected] -H "Authorization: abcd-efgh-ijkl-mnop" my-url.com/api/endpoint — Simon
– Simon, Commented Jul 14, 2014 at 13:44

Jeremy Allen · Accepted Answer · 2014-07-15 01:33:06Z

22

+50

Your problem is that you are using the image metadata as the source of key/value pairs to be posted. Instead of sending it as the value of one of those key/value pairs.

The following code will send a request much like the curl statement you provided:

url = 'my-url.com/api/endpoint'
headers = {'Authorization': 'my-api-key'}
image_metadata = {'key1': 'value1', 'key2': 'value2'}
data = {'name': 'image.jpg', 'data': json.dumps(image_metadata)}
files = {'file': (FILE, open(PATH, 'rb'), 'image/jpg', {'Expires': '0'})}
r = requests.post(url, files=files, headers=headers, data=data)

edited Jul 15, 2014 at 1:33

answered Jul 15, 2014 at 1:23

Jeremy Allen

6,6642 gold badges28 silver badges31 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

John Hazen Over a year ago

As much as I'd like the bounty, after investigating this, I concur. After looking at docs.python-requests.org/en/latest/user/quickstart/… and posting test requests to 'httpbin.org/post', I replicated the curl output using python nearly identical to this answer. (Two caveats: the image_metadata dict obviously can't support multiple instances of 'key':'value', and FILE I guess is the filename.)

Pavel Zagalsky Over a year ago

Hi Jeremy. Can you explain please what the FILE variable stands for? Trying to use similar solution for my code

pymen · Accepted Answer · 2020-05-24 10:12:16Z

If you need to post nested JSON data and send files in single request, you need to use json.dumps for these elements (lists and mappings in the example below)

import requests

session = requests.Session()
session.post(login_url, json=your_credentials_dict)  # auth , not required
mapping = {"email": "Email", "First Name": "first_name"}
post_data = dict(
   mode=1,
   lists=json.dumps(["133", "899", "911"]),
   mapping=json.dumps(mapping),
)
files_data = {'file': ('your-file-name.csv', file_bytes_content)}
response = session.post(<your-url>, data=post_data, files=files_data)

And as a result will be sent request like this:

********** post file data with mapping **********
 --> REQUEST: POST /v3/contacts/import/file/ HTTP/1.1
 --> REQUEST: Host: <your-host>:4000
 --> REQUEST: User-Agent: python-requests/2.23.0
 --> REQUEST: Accept-Encoding: gzip, deflate
 --> REQUEST: Accept: */*
 --> REQUEST: Connection: keep-alive
 --> REQUEST: Cookie: jwt=eyJ0e***
 --> REQUEST: Content-Length: 654
 --> REQUEST: Content-Type: multipart/form-data; boundary=db850c3230988e010b1ebe21be3fb344
 --> REQUEST: 
 --> REQUEST: --db850c3230988e010b1ebe21be3fb344
Content-Disposition: form-data; name="lists"

["7syewz9qUz0CbaqhaGm", "LBfWKJwq4Q"]
--db850c3230988e010b1ebe21be3fb344
Content-Disposition: form-data; name="mode"

2
--db850c3230988e010b1ebe21be3fb344
Content-Disposition: form-data; name="mapping"

{"email": "Email", "First Name": "first_name"}
--db850c3230988e010b1ebe21be3fb344
Content-Disposition: form-data; name="file"; filename="good.csv"

first_name;last_name;email;age
John;Doe;[email protected];50
Mark;Twen;[email protected];20

--db850c3230988e010b1ebe21be3fb344--

requests-toolbelt package was used to generate the log above https://pypi.org/project/requests-toolbelt/

Collectives™ on Stack Overflow

Using Python Requests to send file and JSON in single request

2 Answers 2

2 Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

2 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related