elasticsearch-py search using source filter

Question

I am using elasticsearch-py (es version is 2.3) and would like to return just the 'title' field from all documents in an index with the mapping: actors, director, genre, plot, title, year.

I'm currently trying messages = es.search(index="movies", _source=['hits.hits.title']) and the resulting response is:

{u'hits': {u'hits': [{u'_score': 1.0, u'_type': u'movie', u'_id': u'tt0116996', u'_source': {}, u'_index': u'movies'}, {u'_score': 1.0, u'_type': u'movie', u'_id': u'1', u'_source': {}, u'_index': u'movies'}], u'total': 2, u'max_score': 1.0}, u'_shards': {u'successful': 1, u'failed': 0, u'total': 1}, u'took': 2, u'timed_out': False}

I've tried different versions of filter paths and source field lists but can't seem to get it right.

Tried messages = es.search(index="movies", _source=['title']) which returned {u'hits': {u'hits': [{u'_score': 1.0, u'_type': u'movie', u'_id': u'tt0116996', u'_source': {u'title': u'Mars Attacks!'}, u'_index': u'movies'}, {u'_score': 1.0, u'_type': u'movie', u'_id': u'1', u'_source': {u'title': u'I saw a movie once a tale'}, u'_index': u'movies'}], u'total': 2, u'max_score': 1.0}, u'_shards': {u'successful': 1, u'failed': 0, u'total': 1}, u'took': 1, u'timed_out': False}so i'm still missing something, somewhere — nmacc
– nmacc, Commented Nov 29, 2016 at 16:52

Ryan Walker · Accepted Answer · 2016-11-29 16:56:12Z

4

You can apply source filtering with:

messages = es.search(index="movies", _source=["title"])

but you'll still need to parse the response. For this you can do something like:

titles = [hit["title"] for hit in messages["hits"]["hits"]["_source"]]]

There is nothing in the elasticsearch-py API (as far as I know) that will flatten down the rather verbose response you get from Elasticsearch.

answered Nov 29, 2016 at 16:56

Ryan Walker

3,2861 gold badge26 silver badges29 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

serpiko Over a year ago

wouldn't it be rather : titles = [hit['_source']['title'] for hit in msg['hits']['hits']]

Metropolis · Accepted Answer · 2018-03-20 16:57:14Z

1

You can now use the _source_exclued and _source_include kwargs in the search function to limit what fields are returned.

So something like:

messages = es.search(index="movies", _source=["title"], _source_include=['title'])

answered Mar 20, 2018 at 16:57

Metropolis

2,1282 gold badges20 silver badges39 bronze badges

Comments

Amela · Accepted Answer · 2018-05-30 12:00:49Z

1

I had similar problem and this is how I solved it. I needed it in a bit different context - I had to use information about the title later in the loop:

res = es.search(index="movies", body={"query": {"match_all": {}}})
for hit in res['hits']['hits']:
    title = hit['_source'].get('title')

answered May 30, 2018 at 12:00

Amela

236 bronze badges

Collectives™ on Stack Overflow

elasticsearch-py search using source filter

3 Answers 3

1 Comment

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

1 Comment

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related