ElasticSearch: How to return only specific fields using ElasticSearch API?

Question

I am using ElasticSearch 7.3, to query some documents,
I want to return only specific fields of each document in the query response,
I found that _source can be used to achieve this,
Which I was able to do from Kibana using this query -

GET /test/_search?_source=id
{
  "query": {
    "match_all": {}
  }
}

Returns me the correct data -

{
  "took" : 1,
  "timed_out" : false,
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 6,
      "relation" : "eq"
    },
    "max_score" : 1.0,
    "hits" : [
      {
        "_index" : "test",
        "_type" : "_doc",
        "_id" : "3",
        "_score" : 1.0,
        "_source" : {
          "id" : 3
        }
      },
      {
        "_index" : "test",
        "_type" : "_doc",
        "_id" : "2",
        "_score" : 1.0,
        "_source" : {
          "id" : 2
        }
      },
      {
        "_index" : "test",
        "_type" : "_doc",
        "_id" : "1",
        "_score" : 1.0,
        "_source" : { }
      },
      {
        "_index" : "test",
        "_type" : "_doc",
        "_id" : "4",
        "_score" : 1.0,
        "_source" : {
          "id" : 4
        }
      },
      {
        "_index" : "test",
        "_type" : "_doc",
        "_id" : "5",
        "_score" : 1.0,
        "_source" : {
          "id" : 5
        }
      },
      {
        "_index" : "test",
        "_type" : "_doc",
        "_id" : "6",
        "_score" : 1.0,
        "_source" : {
          "id" : 6
        }
      }
    ]
  }
}

But I am not able to achieve the same using the node client of ElasticSearch -

const { Client } = require('@elastic/elasticsearch')
const client = new Client({ node: 'http://localhost:9200' })

let searchTest = async () => {
  let indexToQuery = 'test';
  let esQuery = {
    index: indexToQuery,
    //q: '_source:id',
    body: {
        "query": {
          "match_all": {}
        }
    }
  };
  console.log('esQuery =>\n', esQuery);

  const result = await client.search(esQuery);
  console.log("search resp => \n", result.body.hits.hits);
};

searchTest();

Can someone please help me find the correct way to achieve my use case?

References -
https://www.elastic.co/guide/en/elasticsearch/reference/7.3/docs-get.html#get-source-filtering
https://www.elastic.co/guide/en/elasticsearch/client/javascript-api/16.x/api-reference.html#api-search

Nishant · Accepted Answer · 2019-09-03 03:46:44Z

17

_source can also be used as a part of query. Add _source as sibling of query in the body block. Update as below:

{
  "query": {
    "match_all": {}
  },
  "_source": ["id"]
}

edited Sep 3, 2019 at 3:46

answered Sep 3, 2019 at 3:35

Nishant

7,9121 gold badge24 silver badges37 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Ak777 Over a year ago

this was a savior for me. :) and the question too. :D

apt-get_install_skill · Accepted Answer · 2019-09-03 07:08:41Z

1

You can control the fields that should be retrieved of the search hits with _source.

Note that if you use aggregations in the search query and you are only interested in their results (= buckets), you will find them outside of the _source-field. Therefore you can set

"_source": false

to avoid getting all fields under _source and slightly increase the performance of your query.

--- EDIT ---

Here's an example query which uses a terms aggregation on the kibana sample flight data to get the amount of hits for every destination airport id:

GET kibana_sample_data_flights/_search
{
  "_source": false, 
  "query": {
    "match_all": {}
  },
  "aggs": {
    "airport_id_agg": {
      "terms": {
        "field": "DestAirportID"
      }
    }
}

}

The response looks like this:

"hits" : {
"total" : {
  "value" : 10000,
  "relation" : "gte"
},
"max_score" : 1.0,
"hits" : [
  {
    "_index" : "kibana_sample_data_flights",
    "_type" : "_doc",
    "_id" : "K-Aj6mwBFbrhq0Rw_O-6",
    "_score" : 1.0
  },
  ...
  ]
  },
  "aggregations" : {
    "airport_id_agg" : {
      "doc_count_error_upper_bound" : 0,
      "sum_other_doc_count" : 8898,
      "buckets" : [
        {
          "key" : "ZRH",
          "doc_count" : 691
        },
        {
          "key" : "XIY",
          "doc_count" : 526
        },
        {
          "key" : "YWG",
          "doc_count" : 460
        },
        ...
      ]
    }
  }

As you can see, the response does not contain any data fields. Compare the results of this search query with one that does not set "_source": false on the Elastic Demo Page

edited Sep 3, 2019 at 7:08

answered Sep 3, 2019 at 6:12

apt-get_install_skill

2,95814 silver badges29 bronze badges

4 Comments

Dev1ce Over a year ago

Can you post an example query?

apt-get_install_skill Over a year ago

@AniruddhaRaje I updated my answer with an example query.

Nishant Over a year ago

@apt-get_install_skill It would be better to use "_source": false instead of "_source": [""] if you want that no field should be returned.

apt-get_install_skill Over a year ago

@NishantSaini thanks for that hint! Updated my answer and worked in your hint.

halfer · Accepted Answer · 2022-06-21 21:49:29Z

0

Here is an example of both to include and exclude fields in the response:

{
    "_source": {
        "include": [ "obj1.*", "obj2.*" ],
        "exclude": [ "*.description" ]
    },
    "query" : {
        "term" : { "user" : "kimchy" }
    }
}

edited Jun 21, 2022 at 21:49

halfer

20.2k20 gold badges110 silver badges207 bronze badges

answered Jun 9, 2022 at 12:56

Amulya Kashyap

2,3731 gold badge20 silver badges26 bronze badges

Collectives™ on Stack Overflow

ElasticSearch: How to return only specific fields using ElasticSearch API?

3 Answers 3

1 Comment

4 Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

1 Comment

4 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related