How can I get in python response for helpers.bulk function (elasticsearch API)?

Question

I found great example on helpers.bulk function but I cannot find a response code. Everything I get from

helpers.bulk(es, actions)

is

(1, [])

It is the same as presented in the example. I expected, that helpers.bulk(es, actions) returns something like Response [200] or {"acknowledged":true} — Pavol Travnik
– Pavol Travnik, Commented Sep 24, 2017 at 18:11
I am seeing the same issue, were you able to get the response? — user 923227
– user 923227, Commented Oct 31, 2018 at 0:42
What if we want to get the same response as when the bulk API is called from command line? — elachell
– elachell, Commented Feb 10, 2020 at 20:17

Lupanoide · Accepted Answer · 2017-09-25 14:21:22Z

2

As written in documentation:

It returns a tuple with summary information - number of successfully executed actions and either list of errors or number of errors if stats_only is set to True [...] If you need to process a lot of data and want to ignore/collect errors please consider using the streaming_bulk() helper which will just return the errors and not store them in memory.

with streaming_bulk() you have to use raise_on_error parameter for raise on error. if you want to collect a lot of data i suggest to use parallel_bulk() that is faster and more intuitive

edited Sep 25, 2017 at 14:21

answered Sep 25, 2017 at 13:20

Lupanoide

3,25225 silver badges38 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

user 923227 · Accepted Answer · 2018-11-02 21:52:17Z

I am doing the same way on AWS Lambda to push data to ElasticSearch:

print("Pushing data to ES <:", ESindex, ESmapping, ">", helpers.bulk(g.esClient, dataGenerator()))

Here the response

(1, [])

means 1 record pushed successfully to Elasticsearch. In case of a failure I got an error log like below and the Lambda Function Failed. (I had removed the [] array brackets from the Json Data.)

RequestError(400, 'action_request_validation_exception', 'Validation Failed: 1: index is missing;2: type is missing;3: index is missing;4: type is missing;5: index is missing;6: type is missing;7: index is missing;8: type is missing;9: index is missing;10: type is missing;11: index is missing;12: type is missing;13: index is missing;14: type is missing;15: index is missing;16: type is missing;17: index is missing;18: type is missing;19: index is missing;20: type is missing;21: index is missing;22: type is missing;23: index is missing;24: type is missing;25: index is missing;26: type is missing;27: index is missing;28: type is missing;29: index is missing;30: type is missing;'): RequestError

Collectives™ on Stack Overflow

How can I get in python response for helpers.bulk function (elasticsearch API)?

2 Answers 2

Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related