I have a server running flask and some api open for client to upload file.
The client is having problem when uploading big file(>100MB).
At first, I tried urllib2 using the following code at client:
def uploadFile(url, params, user, token):
try:
from poster.encode import multipart_encode
from poster.streaminghttp import register_openers
import urllib2
file_path = params["filepath"]
register_openers()
tempparams = {'file': open(file_path, "rb")}
f = open(file_path,'rb')
tempparams = {'file': open(file_path, "rb")}
finalparams = dict(params, **tempparams)
datagen, headers = multipart_encode(finalparams)
request = urllib2.Request(url, datagen, headers)
request.add_header("Authorization", "Basic %s" % base64string)
result = urllib2.urlopen(request)
resultData = result.read()
resultJson = json.loads(resultData)
return resultJson
except Exception as e:
return {"error": e, "error_code": 503}
At server:
def upload_file_old():
fileobj =request.files["file"]
try:
fileobj.save(webserver_filepath)
except Exception as e:
logger.info(e)
.....
return some_result, 201
But the above code will lead to the following error when uploading big file at client with server recording no log at all.
urlopen error [errno 10054]
Then I tried using requests toolbelt to send file since many post at stack overflow suggest using request other than using urllib.
At client:
def uploadFile(url, params, user, token):
try:
import requests
from requests_toolbelt import MultipartEncoder
file_path = params["filepath"]
m = MultipartEncoder(fields={
'file':(params["name"],open(file_path,'rb'),"application/octet-stream" )
})
r = requests.post(url, data=m, headers={'Content-Type': m.content_type}, auth=(user, token))
return r.json()
except Exception as e:
return {"error": e, "error_code": 503}
At server:
def upload_file_old():
fileobj =request.files["file"]
try:
fileobj.save(webserver_filepath)
except Exception as e:
logger.info(e)
.....
return some_result, 201
Again, sending small file works fine while big file failed for the above code, the error output is as foloowing:
('Connection aborted.', error(104, 'Connection reset by peer'))
What would be the correct way to upload big file in a restful http request using python?
