Elasticsearch 6.2: terms query require lowercase input when searching on keyword

Question

I've created an example index, with the following mapping:

{
    "_doc": {
        "_source": {
            "enabled": False
        },
        "properties": {
            "status": { "type": "keyword" }
        }
    }
}

And indexed a document:

{"status": "CMP"}

When searching the documents with this status with a terms query, I find no results:

{
    "query" : {
        "terms": { "status": ["CMP"]}
    }
}

However, if I make the same query by putting the input in lowercase, I will find my document:

{
    "query" : {
        "terms": { "status": ["cmp"]}
    }
}

Why is it? Since I'm searching on a keyword field, the indexed content should not be analyzed and should match an uppercase value...

I cannot reproduce this issue against a clean ES 6.2 instance. — Oliver Charlesworth
– Oliver Charlesworth, Commented Apr 10, 2018 at 10:57

Lupanoide · Accepted Answer · 2018-04-10 10:37:47Z

1

no more @Oliver Charlesworth Now - in Elastic 6.x - you could continue to use a keyword datatype, lowercasing your text with a normalizer,doc here. However in every cases you should change your index mapping and reindex your docs

answered Apr 10, 2018 at 10:37

Lupanoide

3,24225 silver badges38 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Heschoon Over a year ago

According to the documentation the normalizer should default to null... How is it possible that my keywords are lowercased?

Lupanoide Over a year ago

yes, by default is not enabled... I don't know why, You should check how your data are written before indexing, and how you ingest your data in ES

Heschoon · Accepted Answer · 2018-04-10 11:42:17Z

0

The index and mapping creation and the search were part of a test suite. It seems that the setup part of the test suite was not executed, and the mapping was not applied to the index.

The index was then using the default types instead of the mapping types, resulting of the use of string fields instead of keywords.

After changing the setup method of the automated tests, the mappings are well applied to the index, and the uppercase values for the status "CMP" are now matching documents.

answered Apr 10, 2018 at 11:42

Heschoon

3,0599 gold badges29 silver badges67 bronze badges

Comments

Oliver Charlesworth · Accepted Answer · 2018-04-10 22:00:38Z

0

The symptoms you're seeing shouldn't occur, unless something else is wrong.

A keyword index is not analysed, so your index should contain only CMP. A terms query is also not analysed, etc. so your index is searched only for CMP. Hence there should be a match.

edited Apr 10, 2018 at 22:00

answered Apr 10, 2018 at 10:32

Oliver Charlesworth

274k34 gold badges591 silver badges687 bronze badges

2 Comments

Heschoon Over a year ago

Thanks Olivier for your answer, but what happened is the opposite; I find results only when the search term is 'cmp' while my index contains only 'CMP'...

Oliver Charlesworth Over a year ago

@Heschoon - Hmm, yes, I misread that. I'll delete the answer while I think about this!

Collectives™ on Stack Overflow

Elasticsearch 6.2: terms query require lowercase input when searching on keyword

3 Answers 3

2 Comments

Comments

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

2 Comments

Comments

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related