Elasticsearch filtering with date range data and a must_not bool query

Question

I want to extract the following sql query to elasticsearch.

Select * 
from someTable
Where @timestamp < some_date and @timestamp >= some_other_date
and dst != '-'

And then do some aggregations on the returned documents. The aggregations part I have figured it out, and works perfectly. But I don't get the documents filtered properly. I tried the following query but I get doc's with dst = '-' which are in turns computed in the aggregations.

The query

"query": {                              
    "bool": {
        "must_not": {
            "term": {
                "dst": '-'
            }
        }, 
        "filter":{
            "range":{
                "@timestamp":{
                    "gte":"a date",
                    "lt": "another date"
                }
            }
        }
    }   
}

Am I doing something wrong. I set the size to 0 because I am using aggregations (not shown in here). Elasticsearch ver2.4 and python elasticsearch library.

I know it's not working because the aggregations results contain values from documents that dst is '-"

How are you sending your query? Also what is the mapping of your dst field? — Val
– Val, Commented Oct 5, 2016 at 9:58
sending the query via the python elasticsearch client with an aggregations query. the dst field is analzyed. Could that be the fault? Is the logic of the query correct? — Apostolos
– Apostolos, Commented Oct 5, 2016 at 10:03

Andrzej Wisłowski · Accepted Answer · 2016-10-05 10:18:11Z

1

This field should not be analyzed, as it analyzer removes - char from field in index.

If you need this field to be analyzed for an other reason use multi-fields. https://www.elastic.co/guide/en/elasticsearch/reference/current/multi-fields.html

answered Oct 5, 2016 at 10:18

Andrzej Wisłowski

1237 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Apostolos Over a year ago

I had a feeling this was the problem? Is the query correct? Does it correspond to the SQL query written on my post?

Andrzej Wisłowski Over a year ago

If you fix index mapping with not_analyzed it should work.

Collectives™ on Stack Overflow

Elasticsearch filtering with date range data and a must_not bool query

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related