How to get a count for each type of an index in elasticsearch?

Question

I have an index and want to get a count for the entries in every type of one particular index in elasticsearch, but might not know the types ahead of time.

So, for example, the index is

/events

and the types could be

/events/type1
/events/type2
...
/events/typeN

And I'd like to query the index and say "Give me the count of each of the types under index events", so maybe a result set like

/events/type1 : 40
/events/type2: 20
/events/typeN: 10

where /events/_count would give me

/events: 70

Edit:

imotov's answer is great. I'm having trouble figuring how to get it working in JavaScript/Ajax easily though. I have something like this right now:

$.ajax({
type: 'GET',
url: 'http://localhost:9200/events/_search?search_type=count',
data: '{ "facets" : { "count_by_type" : { "terms" : { "field": "_type" }}}}',
success: function(text) {
    console.log(text);
}
)}'

But am only getting the total count of elements in the ES, the facets portion of the answer seems to be missing.

imotov · Accepted Answer · 2015-09-30 13:00:59Z

39

You can use terms aggregations on the _type field to get this information:

curl "localhost:9200/test-idx/_search?search_type=count" -d '{
    "aggs": {
        "count_by_type": {
            "terms": {
                "field": "_type"
            }
        }
    }
}'

edited Sep 30, 2015 at 13:00

answered Jul 9, 2013 at 21:01

imotov

30.2k3 gold badges93 silver badges87 bronze badges

Sign up to request clarification or add additional context in comments.

5 Comments

cdietschrun Over a year ago

Thanks! That sure seems to make sense. I can't seem to get this call to my ES working from JavaScript/Ajax calls though...do you have any tips on how to do that? Now I have: $.ajax({ type: 'GET', url: 'localhost:9200/events/_search?search_type=count', data: '{ "facets" : { "count_by_type" : { "terms" : { "field": "_type" }}}}', success: function(text) { console.log(text); },

cdietschrun Over a year ago

Edit. Issue seemed to be needing to POST that request, now I understand more.

imotov Over a year ago

Some HTTP clients cannot send request body with GET requests. So, sometimes it's required to replace command with POST.

AlainIb Over a year ago

warning facet are deppreciated in the lastes version, use "aggs" instead

Code Guru Over a year ago

how can i get documents from each type under single index with pagination ?

A Coder Gamer · Accepted Answer · 2017-03-28 16:31:55Z

21

For Elasticsearch v5.0, search_type=count is removed. The same query from the above answers can be written as follows:

GET  indexname/_search
{
    "aggs": {
        "count_by_type": {
            "terms": {
                "field": "_type"
            }
        }
    },
    "size": 0
}

Ref: https://www.elastic.co/guide/en/elasticsearch/reference/5.0/breaking_50_search_changes.html#_literal_search_type_count_literal_removed

edited Mar 28, 2017 at 16:31

answered Nov 4, 2016 at 22:47

A Coder Gamer

8402 gold badges10 silver badges31 bronze badges

1 Comment

lfvv Over a year ago

Thanks, it works, but do you know how to unlimit the size of result? In my case I am getting the count fron only some _types

Roberto · Accepted Answer · 2015-07-28 16:11:19Z

The "facets" are deprecated in ES v. 1.5+ However you can use "aggregations", the use and results are quite similar:

curl "localhost:9200/events/_search?search_type=count" -d '{
    "aggregations": {
        "count_by_type": {
            "terms": {
                "field": "_type"
            }
        }
    },
    "size": 0
}'

You'll get something like:

{
   "took": 21,
   "timed_out": false,
   "_shards": {
      "total": 10,
      "successful": 10,
      "failed": 0
   },
   "hits": {
      "total": 150,
      "max_score": 0,
      "hits": []
   },
   "aggregations": {
      "count_by_type": {
         "doc_count_error_upper_bound": 0,
         "sum_other_doc_count": 0,
         "buckets": [
            {
               "key": "type1",
               "doc_count": 141
            },
            {
               "key": "type2",
               "doc_count": 6
            },
            {
               "key": "other_type",
               "doc_count": 3
            }
         ]
      }
   }
}

Srihari Sridharan · Accepted Answer · 2016-11-16 14:29:45Z

3

Answers by @Askshay and @Roberto highlight another important aspect. Setting size to 0 is really important, especially in low-bandwidth use-cases (say in mobile networks). It reduces the data payload size and that makes a huge difference when document size is large. Note "size": 0

GET  index/_search
{
    "aggs": {
        "countByType": {
            "terms": {
                "field": "_type"
            }
        }
    },
    "size": 0
}

answered Nov 16, 2016 at 14:29

Srihari Sridharan

4523 silver badges11 bronze badges

Collectives™ on Stack Overflow

How to get a count for each type of an index in elasticsearch?

4 Answers 4

5 Comments

1 Comment

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

4 Answers 4

5 Comments

1 Comment

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related