How mapped the timestamp args in range filter using NEST for ElasticSearch?

Question

Attempt filtered data by range using timestamp field. Query is beelow:

        var scanResults = await _elasticClient.SearchAsync<OemCatalogModel>(s => s.Index(indexName)
                .Source(sf => sf
                    .Includes(i => i
                        .Fields(
                                    f => f.Event,
                                    f => f.MemberId,
                                    f => f.IsInternalUser,
                                    f => f.IndexName,
                                    f => f.IsMobile,
                                    f => f.VinNumber,
                                    f => f.Timestamp
                        )
                    )
                )
                .Query(q => q.Range(p => p.Field<Timestamp>(t=>t.Timestamp)
                             .GreaterThanOrEquals(sd)
                             .LessThanOrEquals(ed)
                    ))
                .Size(10000).Scroll("60s"));

startDate and endDate geted from DatePicker as DateTimeOffset and init sd and ed like beelow:

        var sd = startDate.Value.ToUnixTimeSeconds();
        var ed = endDate.Value.ToUnixTimeSeconds();

Timestamp field looks following in mapping model

    [Date(Name = "timestamp")]
    public Int64 Timestamp { get; set; }

This query raised "parse_exception" exception:

    "reason" : {
      "type" : "parse_exception",
      "reason" : "failed to parse date field [1.6540308E9] with format [epoch_second]: [failed to parse date field [1.6540308E9] with format [epoch_second]]",
      "caused_by" : {
        "type" : "illegal_argument_exception",
        "reason" : "failed to parse date field [1.6540308E9] with format [epoch_second]",
        "caused_by" : {
          "type" : "date_time_parse_exception",
          "reason" : "date_time_parse_exception: Failed to parse with all enclosed parsers"
        }
      }
    }

ElasticSearch query that generated by NEST looks following:

{"query":{"range":{"timestamp":{"gte":1654030800.0,"lte":1654203599.0}}},"size":10000,"_source":{"includes":["event","member_id","is_internal_user","indexName","is_mobile","vin_number","timestamp"]}}

This filter not working because to range arguments added .0. I checked this witout .0 and it is works. For example result for _source looks like:

    "_source" : {
      "member_id" : 69500,
      "is_mobile" : false,
      "event" : "close_unit_card",
      "is_internal_user" : false,
      "timestamp" : 1654066236
    }

How mapped correctly the range filter with timestamp? Maybee I must using DataRange? If yes, how cast sd and ed to timestamp that using in ElasticSearch?

Also trying to do the following but getting a different exception:

Query looks like:

.Query(q => q.DateRange(p => p.Field(t=>t.Timestamp)
.GreaterThanOrEquals(DateTime.Parse(startDate.Value.ToString()))
.LessThanOrEquals(DateTime.Parse(endDate.Value.ToString()))))

Rise following exception:

"reason" : {
          "type" : "parse_exception",
          "reason" : "failed to parse date field [2022-05-31T21:00:00Z] with format [epoch_second]: [failed to parse date field [2022-05-31T21:00:00Z] with format [epoch_second]]",
          "caused_by" : {
            "type" : "illegal_argument_exception",
            "reason" : "failed to parse date field [2022-05-31T21:00:00Z] with format [epoch_second]",
            "caused_by" : {
              "type" : "date_time_parse_exception",
              "reason" : "date_time_parse_exception: Failed to parse with all enclosed parsers"
            }
          }
        }

Can you please try with .GreaterThanOrEquals(DateTime.Parse(sd)) .LessThanOrEquals(DateTime.Parse(ed)) and see if it is working or not. — Sagar Patel
– Sagar Patel, Commented Sep 1, 2022 at 12:56
@SagarPatel Try change Range to DataRange but DateTime.Parse can't parse Int64. Also rise another exception if pass DateTime in DataRange because ElasticSearch using Timestamp... — user6408649
– user6408649, Commented Sep 1, 2022 at 13:30

user6408649 · Accepted Answer · 2022-09-01 16:08:01Z

2

Weird solution but working:

Itialize start and end dates:

    var sd = startDate.Value.ToUnixTimeSeconds();
    var ed = endDate.Value.ToUnixTimeSeconds();

DateRange looks like:

.Query(q => q.DateRange(p => p.Field(t=>t.Timestamp).Format("epoch_second")
                                 .GreaterThanOrEquals(sd)
                                 .LessThanOrEquals(ed)
                                 ))

and model look like:

    [Date(Format = DateFormat.epoch_second, Name = "timestamp")]
    public DateTime Timestamp { get; set; }

answered Sep 1, 2022 at 16:08

user6408649

1,3153 gold badges17 silver badges46 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

JPBlanc · Accepted Answer · 2023-11-18 16:05:56Z

With DateRange you can use DateTime structure :

var searchResponse = client.Search<_Source>(s => s
    .Query(q => q
        .Bool(b => b
            .Filter(fi => fi
                .DateRange(r => r
                    .Field(f => f.timestamp) // Replace with your actual timestamp field
                    .GreaterThanOrEquals(new DateTime(2023, 11, 17, 12, 05, 0)) // Start time
                    .LessThanOrEquals(new DateTime(2023, 11, 17, 12, 10, 0)) // End time
                )
            )
            .Must(fi => fi
                .Term(t => t
                    .Field(f => f.host.name) // Replace with your actual host name field
                    .Value("evp-lwsj101")
                )
            )
         )
    )
    .From(0)
    .Size(100) // adjust the size based on your needs
);

Collectives™ on Stack Overflow

How mapped the timestamp args in range filter using NEST for ElasticSearch?

2 Answers 2

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related