ElasticSearch how display all documents matching date range aggregation

Question

Following elastic docs: https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations-bucket-datehistogram-aggregation.html

Question:

How to make date range aggregation and display all documents that match to relevant date bucket just not the doc_count.

The Aggregation :

{
    "aggs" : {
        "articles_over_time" : {
            "date_histogram" : {
                "field" : "date",
                "interval" : "1M",
                "format" : "yyyy-MM-dd" 
            }
        }
    }
}

Response:

        {
            "aggregations": {
                "articles_over_time": {
                    "buckets": [
                        {
                            "key_as_string": "2013-02-02",
                            "key": 1328140800000,
                            "doc_count": 1
                        },
                        {
                            "key_as_string": "2013-03-02",
                            "key": 1330646400000,
                            "doc_count": 2  //how display whole json ??
                 
                    [ .. Here i want to display 
                           all document with array based 
                           NOT only doc_count:2.......... ]

                        },
                        ...
                    ]
                }
            }
        }

Maybe I need to do some sub-aggregation or something else?

Any ideas?

Sumit · Accepted Answer · 2016-07-16 06:54:15Z

2

You have to perform top_hits sub-aggregation on date-histogram aggregation. All the options can be read from here.

Your final aggregation would look like this

{
  "aggs": {
    "articles_over_time": {
      "date_histogram": {
        "field": "date",
        "interval": "1M",
        "format": "yyyy-MM-dd"
      },
      "aggs": {
        "documents": {
          "top_hits": {
            "size": 10
          }
        }
      }
    }
  }
}

answered Jul 16, 2016 at 6:54

Sumit

2,31026 silver badges34 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Jettro Coenradie · Accepted Answer · 2016-07-16 07:31:05Z

0

Like what Sumit says, however, I think what you really want is to create a filter with a date range:

https://www.elastic.co/guide/en/elasticsearch/reference/2.3/query-dsl-range-query.html#ranges-on-dates

That way you filter out documents not in the date range and only keep the right documents. Than you can do everything you want with the results.

answered Jul 16, 2016 at 7:31

Jettro Coenradie

4,73325 silver badges31 bronze badges

7 Comments

VitalyT Over a year ago

Jettro Coenradie: Can filter give me an ability to map each document to their bucket range (for example .. 1st bucket: [1/1/2016/00:00 - 1/1/2016/01:00]--> {[all relevant documents]..},...) , etc?

Jettro Coenradie Over a year ago

Ah, if you want to have multiple time ranges, than you need the tops hits answer from Sumit. The date histogram together with the top hits. Beware there is no way to do pagination with top hits. It is also not suitable to get all documents if there are to many of them.

VitalyT Over a year ago

Jettro Coenradie: Ohh..Yes, it is exactly my case i have a thousand of documents on each bucket, my final purpose is iterate over some fields on each document in the bucket and execute JDBC (external mysql db call) , so return value from db will map back to the document .. or to new document :)

Jettro Coenradie Over a year ago

Depending on the amount of buckets or time ranges I would think about using multiple queries. The top hits was created to create buckets and show a few of the tops hits.

VitalyT Over a year ago

I have 24 buckets(hourly per day) ,on each bucket thousands of mapping documents , so do you suggest using date histogram aggregation + multiple queries ? instead of "top hits" (I must iterate over all documents in the bucket in order to make db call per document) , is it possible to do with elasticsearch aggregation ? maybe using elastic script like "map-reduce"?

|

Collectives™ on Stack Overflow

ElasticSearch how display all documents matching date range aggregation

Question:

2 Answers 2

Comments

7 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

Question:

2 Answers 2

Comments

7 Comments

Your Answer

Sign up or log in

Post as a guest

Related