ElasticSearch 5: sorting by nested key and value results in "Fielddata is disabled on text fields by default"

Question

I'm trying to sort a search result in ElasticSearch 5 by a nested set of key value pairs. Ie.:

I have the following result (pseudo structure, for the sake of keeping it simple):

{
  "hit1": {
    "nested_objects": [
      {
        "Key": "abc",
        "Value": 0.1
      },
      {
        "Key": "def",
        "Value": 0.3
      }
    ]
  },
  "hit2": {
    "nested_objects": [
      {
        "Key": "abc",
        "Value": 0.9
      },
      {
        "Key": "def",
        "Value": 0.1
      }
    ]
  }
}

I'd like that to be sorted by "highest Value where Key = 'abc'", which would mean that "hit2" would come out on top.

My mapping is as follows:

{
  "test_index": {
    "mappings": {
      "test_type": {
        "properties": {
          "nested_objects": {
            "type": "nested",
            "properties": {
              "Key": {
                "type": "text",
                "fields": {
                  "keyword": {
                    "type": "keyword",
                    "ignore_above": 256
                  }
                }
              },
              "Value": {
                "type": "float"
              }
            }
          }
        }
      }
    }
  }
}

I've tried to follow the following suggestions:

Sort elastic search query based on a nested key value array

Sort nested object in Elasticsearch

...but I consistently get the following error:

"Fielddata is disabled on text fields by default"

An example of a sorting attempt that reproduces this:

{
  "sort": [
    {
      "nested_objects.Key": {
        "order": "desc",
        "nested_path": "nested_objects",
        "nested_filter": {
          "term": { "nested_objects.Key": "abc" }
        }
      }
    }
  ]
}

What is the best practice way of solving this? Is enabling field data (which uses a lot of ram) really the only option here?

The_Torst · Accepted Answer · 2017-10-04 12:55:05Z

1

In your example, you are sorting by the Key, but you say you want to sort by Value. Key is not sortable by default (as Taras points out, you'd have to use ".keyword" to sort on that field), which throws the error you encounter.

To solve your problem completely, sort by Value and filter by Key. You're already filtering by Key (you don't need to use ".keyword" to do so), so all you have to do is to sort by Value:

{
  "sort": [
    {
      "nested_objects.Value": {   <-- SOLUTION
        "order": "desc",
        "nested_path": "nested_objects",
        "nested_filter": {
          "term": { "nested_objects.Key": "abc" }
        }
      }
    }
  ]
}

answered Oct 4, 2017 at 12:55

The_Torst

4013 silver badges10 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

The_Torst · Accepted Answer · 2017-10-04 13:46:08Z

1

The Text datatype is analyzed (this type is used for full text search). Sorting on analyzed fields is expensive, that's why it is disabled by default.
Usually you don't need to sort by analyzed field, so try sorting on keyword field instead:

"term": { "nested_objects.Key.keyword": "abc" }

Read this doc to understand the problem more deeply.

edited Oct 4, 2017 at 13:46

The_Torst

4013 silver badges10 bronze badges

answered Oct 4, 2017 at 10:35

Taras Kohut

2,5753 gold badges21 silver badges43 bronze badges

1 Comment

Scarabas Over a year ago

Thanks Taras - this makes sense, and solves half of the problem (fielddata error). I still have to sort by the Value of "abc" - can I put more terms in: "term": { "nested_objects.Key.keyword": "abc" }?

Collectives™ on Stack Overflow

ElasticSearch 5: sorting by nested key and value results in "Fielddata is disabled on text fields by default"

2 Answers 2

Comments

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related