Convert curl command with form files into python requests

Question

I have the following curl command:

curl -X POST "_my_username_:[email protected]:5601/api/saved_objects/_import" -H "kbn-xsrf: true" --form file=@V:kibana\IndexPatterns\events.ndjson

Which works perfectly (import index pattern into elasticsearch), but I'm trying to convert it to Python requests. I tried several ways, including the following:

files = {'file': '@' + args.kibana_index_pattern_path}
res = requests.post("http://{0}:{1}@{2}:5601/api/saved_objects/_import".format(args.elastic_username, args.elastic_password, args.kibana_host),
                    headers={'kbn-xsrf': 'true'}, data=files)


files = {'file': '@' + args.kibana_index_pattern_path}
res = requests.post("http://{0}:{1}@{2}:5601/api/saved_objects/_import".format(args.elastic_username, args.elastic_password, args.kibana_host),
                    headers={'kbn-xsrf': 'true', 'Content-Type': 'text/plain'}, files=files)

With different combinations of with or without the @, files as a single string instead of dictionary, ect. I keep getting errors on bad requests and invalid content types (for example: {'message': 'Unsupported Media Type', 'error': 'Unsupported Media Type', 'statusCode': 415}).

Note that there are some tools to convert curl to requests, but all the ones I tried don't recognize the files param, either ignoring it or throwing exception. The command itself, however, works.

What am I doing wrong here?

One approach is to review the requests that are generated by the tools. Using curl -v will provde you with all the headers of the HTTP request. Consider reviewing the debug information of the request module (somehow) and comparing it to the working curl request. You can then concentrate your efforts on the missing (or different) HTTP headers. If you don't know how to get debug info from requests (I don't yet) you may consider doing a packet capture to review the HTTP headers. — Mark
– Mark, Commented Jan 6, 2020 at 16:55
@AMC got many errors depending on variations I tried, but most common one is: {'message': 'Unsupported Media Type', 'error': 'Unsupported Media Type', 'statusCode': 415} — Ronen Ness
– Ronen Ness, Commented Jan 7, 2020 at 11:12
@AMC unfortunately it did not work. The authentication is OK the problem is with the added file for some reason. Thanks. — Ronen Ness
– Ronen Ness, Commented Jan 8, 2020 at 8:06

score 1 · Accepted Answer · 2020-01-09 06:13:35Z

1

Try the following:

import requests
from requests.auth import HTTPBasicAuth

username = '_my_username_'
password = '_my_password_'
headers = {'kbn-xsrf': 'true'}
upload_url = "http://10.2.25.209:5601/api/saved_objects/_import"
files = {'file': open('V:\algotec\analytics\install\Kibana\IndexPatterns\events.ndjson', 'rb')}

r = requests.post(upload_url, headers=headers, auth=HTTPBasicAuth(username, password), files=files)
print(r.status_code)

If you receive a bad request with this error

error: 'Bad Request', message: 'Request must contain a kbn-xsrf header.'

Modify the header information as per the below and re-try.

headers = {
  'Content-Type': 'application/x-ndjson',
  'kbn-xsrf': 'anything',
  'Accept': 'application/x-ndjson'
}

edited Jan 9, 2020 at 6:13

answered Jan 6, 2020 at 16:32

user7548672

Sign up to request clarification or add additional context in comments.

7 Comments

Ronen Ness Over a year ago

Hi Victor, unfortunately it didn't work, ES expects a string file and not binary. PS. the authentication is OK the problem is with the added file for some reason.

user7548672 Over a year ago

@RonenNess what's the response code your getting and message?

Ronen Ness Over a year ago

{'message': 'Unsupported Media Type', 'error': 'Unsupported Media Type', 'statusCode': 415}). About auth - we use other apis the same way and auth works there, this is the only api that gives us problems, possible because the files. Thanks

user7548672 Over a year ago

@RonenNess are you using ES 6.0?

user7548672 Over a year ago

@RonenNess the problem is the Content-type: elastic.co/guide/en/elasticsearch/reference/current/… . I have updated the headers to include Newline delimited JSON (NDJSON), which is what is being sent. If it fails again, my suggestion would be to remove 'Accept', then 'kbn-xsrf' until your left with Content-Type. A matter of trial and error until.

|

Collectives™ on Stack Overflow

Convert curl command with form files into python requests

1 Answer 1

7 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

7 Comments

Your Answer

Sign up or log in

Post as a guest

Related