Simple query not matching for one specific field

Question

I'm throwing some arbitrary data into elasticsearch (logs). The writes are happening fine, and most queries work properly, but I have a "reqId" field that never matches.

$ curl localhost:9200/log_general/log/c9811a1a-6710-424a-b67d-d02d6ad75c89 | jq .
{
  "_index": "log_general",
  "_type": "log",
  "_id": "c9811a1a-6710-424a-b67d-d02d6ad75c89",
  "_version": 1,
  "found": true,
  "_source": {
    "body": {
      "body": {
        "media": [],
        "parentId": "5a695c7bda3c26391649e332",
        "text": "Super bulk comment 25"
      },
      "method": "post",
      "url": "/addComment",
      "xuserid": "5a695c30da3c26391649e17f"
    },
    "logType": "request_start",
    "reqId": "5T42Q1AUmd9LS1E8Q",
    "reqUrl": "/addComment"
  }
}

I can try searching for it by reqId

curl -XPOST localhost:9200/_search -H 'content-type: application/json' --data-binary @sample-query

sample-query:

{
  "query": {
    "bool": {
      "must": [
      {
        "term": {
          "reqId": "5T42Q1AUmd9LS1E8Q"
        }
      }
      ],
      "filter": [],
      "should": []
    }
  }
}

Gives no hits, and no error.

If I try a different field, it returns results. Two of the results have the same reqId.

{
  "query": {
    "bool": {
      "must": [
      {
        "term": {
          "logType": "request_start"
        }
      }
      ],
      "filter": [],
      "should": []
    }
  }
}

This is the mapping elasticsearch generated for the two fields

      "logType": {
        "type": "text",
        "fields": {
          "keyword": {
            "type": "keyword",
            "ignore_above": 256
          }
        }
      },
      "reqId": {
        "type": "text",
        "fields": {
          "keyword": {
            "type": "keyword",
            "ignore_above": 256
          }
        }
      },

Really no idea what the issue could be here.

I just read that elasticsearch stores an original copy and a parsed copy. This might be what your hitting, try adding another prop to term: "index": "not_analyzed" — njsokol
– njsokol, Commented Jan 27, 2018 at 10:24
It didn't like that. From what I've read, it doesn't attempt to analyze if it's mapped as keyword. — Brigand
– Brigand, Commented Jan 27, 2018 at 11:31
After more testing, it was at least two fields not being processed. I reduced the number of things being indexed, which didn't seem to help. Then I dropped the index and set a mapping before writing any records to it with reqId: { type: 'keyword' }, and now it's working. Would still love any ideas as to what the problem could be. — Brigand
– Brigand, Commented Jan 27, 2018 at 16:00

gaurav9620 · Accepted Answer · 2018-01-27 17:32:48Z

2

In elastic search 6 for every string fields two type of mapping are generated. One is text and other is keyword.

Here reqId field is text type - >and defualt analyzer is standard.So the actual token that would be generated would be 5t42q1aumd9ls1e8q.
The query you are performing term query which finds exact term and doesn't analyze the search. So what is happening is token which was indexed was 5t42q1aumd9ls1e8q and you are searching 5T42Q1AUmd9LS1E8Q

There can be two solutions

You use match query on field reqId.This would analyze search string and which matches the exact indexed token.
Or else you search on reqId.keyword field which is not analyzed.

answered Jan 27, 2018 at 17:32

gaurav9620

1,1971 gold badge13 silver badges31 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Simple query not matching for one specific field

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related