How do I (safely) send a Python object to my Flask API?

Question

I am currently trying to build a Flask Web API that is able to receive a python object in a POST-request.

I am using Python 3.7.1 for creating the request and Python 2.7 for running the API. The API is set up to run on my local machine. The object I am trying to send to my API is a RandomForestClassifier object from sklearn.ensemble, but this could be any of a wide variety of object types.

So far I have tried to json.dumps() my object, but this object is not JSON serializable. I have also tried to pickle.dumps() my object, but this caused an error when trying to load the object on the API side. Also, since the API will receive requests from anonymous users, I am worried about performing a pickle.loads() on a possibly malicious object.

Is this a grounded worry? And if so, what is the best way to send any python object in a POST request?

The script performing the POST request:

import requests
import pickle

url = "http://localhost:5000/flask-api-function"

# the object I want to send is the 'model' object
data = pickle.dumps(model)

r = requests.post(url,data=data)

The Flask API:

@app.route('/flask-api-function', methods=['POST'])
def flask_api_function():

  model = pickle.loads(request.get_data())

This setup actually causes an error when trying to decode the data with pickle:

Traceback (most recent call last):
  File "/Library/Python/2.7/site-packages/flask/app.py", line 2309, in __call__
return self.wsgi_app(environ, start_response)
  File "/Library/Python/2.7/site-packages/flask/app.py", line 2295, in wsgi_app
response = self.handle_exception(e)
  File "/Library/Python/2.7/site-packages/flask/app.py", line 1741, in handle_exception
reraise(exc_type, exc_value, tb)
  File "/Library/Python/2.7/site-packages/flask/app.py", line 2292, in wsgi_app
response = self.full_dispatch_request()
  File "/Library/Python/2.7/site-packages/flask/app.py", line 1815, in full_dispatch_request
rv = self.handle_user_exception(e)
  File "/Library/Python/2.7/site-packages/flask/app.py", line 1718, in handle_user_exception
reraise(exc_type, exc_value, tb)
  File "/Library/Python/2.7/site-packages/flask/app.py", line 1813, in full_dispatch_request
rv = self.dispatch_request()
  File "/Library/Python/2.7/site-packages/flask/app.py", line 1799, in dispatch_request
return self.view_functions[rule.endpoint](**req.view_args)
  File "/Users/larssuanet/Documents/enjins/dscs/flask_api.py", line 39, in store_model
model = pickle.loads(request.get_data())
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 1382, in loads
return Unpickler(file).load()
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 858, in load
dispatch[key](self)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 886, in load_proto
raise ValueError, "unsupported pickle protocol: %d" % proto
ValueError: unsupported pickle protocol: 3

Is there a nice and safe way to do is?

don't send the actual object, its bad practice, and probably will get you in a never ending maintenance fire fighting. see if you can send a representation of them, and reconstruct them on the other side.. also see: stackoverflow.com/questions/13031058/… — Urban48
– Urban48, Commented Jan 2, 2019 at 16:12
Any suggestions on how someone could go about doing that when they want to support a wide variety of objects? I feel like there should be an easier way to do this without having to write logic for reconstructing every single object-type you want to support. — Lars S
– Lars S, Commented Jan 2, 2019 at 16:17
in python everything is an object... can you be more specific? try to serialize your objects into JSON, thats a good place to start — Urban48
– Urban48, Commented Jan 2, 2019 at 16:21
As you can see I have tried that with the RandomForestClassifier object, and this object is not serializable. I don't need a solution for this specific object though, I am looking for a clean way to serialize a wide variety of objects (specifically ML models). — Lars S
– Lars S, Commented Jan 2, 2019 at 16:24

Dani G · Accepted Answer · 2019-01-02 16:45:20Z

3

When you pickle the object in python 3, pass the protocol keyword and set it to 2. This will ensure that it will work on python 2.

import requests
import pickle

url = "http://localhost:5000/flask-api-function"

# the object I want to send is the 'model' object
data = pickle.dumps(model,protocol=2)

r = requests.post(url,data=data)

Generally I would try to find a way to serialize the object into JSON since pickle has major security risks

answered Jan 2, 2019 at 16:45

Dani G

1,2409 silver badges16 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Lars S Over a year ago

Thanks, this will solve my error for now. But like you said, I will still be looking out for a more secure way to do this.

Collectives™ on Stack Overflow

How do I (safely) send a Python object to my Flask API?

1 Answer 1

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related