How to iterate through every object in an ElasticSearch index?

Question

This should be easy.

I've inserted a number of records into ElasticSearch and have been scouring their documentation but I can't seem to find a way to simply take an index and iterate through it in Node.js

I don't have much experience with Elastic, so it's been frustrating.

Some of the items in that list need to be pruned more or less on a 10 minute basis, otherwise the DB just grows and grows. I'd like a separate task to do just that.

If you know which items need to be pruned, you don't need to iterate through the whole index, you can simply use the delete by query API — Val
– Val, Commented May 8, 2017 at 9:02
@Val How do I do this by date comparison? Anything with a date older than new Date() should get nuked. Seems like it's just a straight match in the API example - is there more to it? — dsp_099
– dsp_099, Commented May 8, 2017 at 9:05

Val · Accepted Answer · 2017-05-08 09:11:12Z

1

You can achieve what you want by using a range query in your delete by query call:

POST your_index/_delete_by_query
{
  "query": {
    "range" : {
        "date_field" : {
           "lt" : "2017-05-08T00:00:00.000Z"
        }
    }
  }
}

answered May 8, 2017 at 9:11

Val

218k14 gold badges377 silver badges384 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

dsp_099 Over a year ago

Great answer. Silly question: How do I turn the numeric UTC timestamp into "YYYY-MM-DDT00:00:00.000Z" format?

Val Over a year ago

You can also use the numeric timestamp instead of a formatted date.

Collectives™ on Stack Overflow

How to iterate through every object in an ElasticSearch index?

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related