15

I'm trying to send a multipart/related message using requests in Python. The script seems simple enough, except that requests only seems to allow multipart/form-data messages to be sent, though their documentation does not clearly state this one way or another.

My use case is sending soap with attachments. I can provide a dictionary with the two files whose contents are a test soap-message, and a test document that I'm trying to send. The first contains the soap message with all the instructions, the second is the actual document.

However, if I don't specify a headers value, requests only seems to use multipart/form-data when using the files option. But if I specify headers in an attempt to specify a different multipart type, requests does not seem to add in the mime boundary information.

url = 'http://10.10.10.90:8020/foo'
headers = {'content-type': 'multipart/related'}
files = {'submission': open('submission_set.xml', 'rb'), 'document': open('document.txt', 'rb')}
response = requests.post(url, data=data, headers=headers)
print response.text

Is there a way to get this done using requests? Or is there another tool that I should be looking at?

Improve this question
2
  • Have you checked these 22 questions which come up as a result of searching for [python] [python-requests] +multipart? Commented Apr 2, 2013 at 7:46
  • 5
    @PiotrDobrogost: Those are all about multipart/form-data, which requests handles for you. This is multipart/related, which is not a common encoding for POST and requests doesn't handle that automatically. Commented Apr 2, 2013 at 9:38

2 Answers 2

Reset to default
28

You'll have to create the MIME encoding yourself. You can do so with the email.mime package:

import requests
from email.mime.multipart import MIMEMultipart
from email.mime.text import MIMEText

related = MIMEMultipart('related')

submission = MIMEText('text', 'xml', 'utf8')
submission.set_payload(open('submission_set.xml', 'rb').read())
related.attach(submission)

document = MIMEText('text', 'plain')
document.set_payload(open('document.txt', 'rb').read())
related.attach(document)

body = related.as_string().split('\n\n', 1)[1]
headers = dict(related.items())

r = requests.post(url, data=body, headers=headers)

I presumed the XML file uses UTF-8, you probably want to set a character set for the document entry as well.

requests only knows how to create multipart/form-data post bodies; the multipart/related is not commonly used.

Improve this answer
Sign up to request clarification or add additional context in comments.

8 Comments

Thanks! This is very helpful. I actually tried using the email.mime package earlier, but couldn't figure out how to marry the two services together. You helped me merge my two alternate scripts into one! However there seems to be a small problem, and I can't tell if it's the combination of services or some other fault. I'm getting an Unexpected EOF in prolog at [row,col {unknown-source}]: [1,0] which makes it seem like the first character being sent is interoperated as an EOF. Could this be due to the tool chain/encodings?
I have absolutely no idea; sounds like a XML parsing problem, but the error is not familiar.
This looks fantastic, but my request seems to hang forever. Any clues?
@zapatilla: hard to say. Perhaps the content-length header is wrong and the server expects more data than you are sending?
There is a problem with the example - headers[Content-Type] actually has no boundary parameter. You should call as_string() before items() to let it generate it or provide boundary in MIMEMultipart constructor. Tested on python 3.5.0.
|
10

I'm working with requests and the Google Drive API "Multipart" upload.

The email.mime solution did not work with Google's API, so I dug into the requests source code to see how it implements multipart/form-data bodies.

requests uses the urllib3.filepost.encode_multipart_formdata() helper, which can be wrapped to provide multipart/related:

from urllib3.filepost import encode_multipart_formdata, choose_boundary

def encode_multipart_related(fields, boundary=None):
    if boundary is None:
        boundary = choose_boundary()

    body, _ = encode_multipart_formdata(fields, boundary)
    content_type = str('multipart/related; boundary=%s' % boundary)

    return body, content_type

Now we can use encode_multipart_related() to create a (body, content_type) tuple that matches Google's requirements:

import json
from urllib3.fields import RequestField

def encode_media_related(metadata, media, media_content_type):
    rf1 = RequestField(
        name='placeholder',
        data=json.dumps(metadata),
        headers={'Content-Type': 'application/json; charset=UTF-8'},
    )
    rf2 = RequestField(
        name='placeholder2',
        data=media,
        headers={'Content-Type': media_content_type},
    )
    return encode_multipart_related([rf1, rf2])

Here is a full example that uses our encode_media_related() to upload a hello world file to Google Drive, using the google_auth library.

from google.oauth2 import service_account
import google.auth.transport.requests

credentials = service_account.Credentials.from_service_account_file(
    PATH_TO_SERVICE_FILE,
    scopes=['https://www.googleapis.com/auth/drive.file'],
)
session = google.auth.transport.requests.AuthorizedSession(credentials)

metadata = {
    'mimeType': 'application/vnd.google-apps.document',
    'name': 'Test Upload',
}
body, content_type = encode_media_related(
    metadata,
    '<html><body><p>Hello World!</body></html>',
    'text/html; charset=UTF-8',
)
resp = session.post(
    'https://www.googleapis.com/upload/drive/v3/files',
    data=body,
    params={'uploadType': 'multipart'},
    headers={'Content-Type': content_type},
)

print 'Uploaded to file with id: %s' % resp.json()['id']
Improve this answer

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.