Elasticsearch cannot parse date using custom format

Question

According to the documentation (http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/mapping-date-format.html), it should be possible to parse dates using a Joda time compatible date pattern.

My data contains dates like this: "2015-02-09 02:10:05,245".

These can be parsed with Joda time using the following pattern: "yyyy-MM-dd HH:mm:ss,SSS".

However, when I tell my index to use this pattern to parse dates, loading data into Elasticsearch fails with the following error:

org.elasticsearch.index.mapper.MapperParsingException: failed to parse [mydate]
    at org.elasticsearch.index.mapper.core.AbstractFieldMapper.parse(AbstractFieldMapper.java:416)
    at org.elasticsearch.index.mapper.object.ObjectMapper.serializeValue(ObjectMapper.java:709)
    at org.elasticsearch.index.mapper.object.ObjectMapper.parse(ObjectMapper.java:500)
    at org.elasticsearch.index.mapper.DocumentMapper.parse(DocumentMapper.java:542)
    at org.elasticsearch.index.mapper.DocumentMapper.parse(DocumentMapper.java:491)
    at org.elasticsearch.index.shard.service.InternalIndexShard.prepareCreate(InternalIndexShard.java:392)
    at org.elasticsearch.action.bulk.TransportShardBulkAction.shardIndexOperation(TransportShardBulkAction.java:444)
    at org.elasticsearch.action.bulk.TransportShardBulkAction.shardOperationOnPrimary(TransportShardBulkAction.java:150)
    at org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$AsyncShardOperationAction.performOnPrimary(TransportShardReplicationOperationAction.java:512)
    at org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$AsyncShardOperationAction$1.run(TransportShardReplicationOperationAction.java:419)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)
Caused by: org.elasticsearch.index.mapper.MapperParsingException: failed to parse date field [2015-02-09 02:10:05,245], tried both date format [dateOptionalTime], and timestamp number with locale []
    at org.elasticsearch.index.mapper.core.DateFieldMapper.parseStringValue(DateFieldMapper.java:621)
    at org.elasticsearch.index.mapper.core.DateFieldMapper.innerParseCreateField(DateFieldMapper.java:549)
    at org.elasticsearch.index.mapper.core.NumberFieldMapper.parseCreateField(NumberFieldMapper.java:235)
    at org.elasticsearch.index.mapper.core.AbstractFieldMapper.parse(AbstractFieldMapper.java:406)
    ... 12 more
Caused by: java.lang.IllegalArgumentException: Invalid format: "2015-02-09 02:10:05,245" is malformed at " 02:10:05,245"
    at org.elasticsearch.common.joda.time.format.DateTimeFormatter.parseMillis(DateTimeFormatter.java:754)
    at org.elasticsearch.index.mapper.core.DateFieldMapper.parseStringValue(DateFieldMapper.java:615)
    ... 15 more

In the index template, 'mydate' is specified like this:

{
  "template": "te*",
  "mappings": {
    "_default_" : {
      "properties": {
        "mydate": {
          "index": "analyzed",
          "type": "date",
          "format": "yyyy-MM-dd HH:mm:ss,SSS"
        }
      }
    }
  }
}

That should do it, right?

jhilden · Accepted Answer · 2015-02-24 22:48:22Z

1

This works for me on ES 1.4.4

PUT hilden1

PUT hilden1/type1/_mapping
{
  "properties": {
    "dt": {"type": "date", "format": "yyyy-MM-dd HH:mm:ss,SSS"}
  }
}

POST hilden1/type1
{
  "dt": "2015-02-09 02:10:05,245"
}

GET hilden1/type1/_search

answered Feb 24, 2015 at 22:48

jhilden

12.5k5 gold badges56 silver badges77 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Rens Verhage Over a year ago

I don't really know why, but I've got things working now. I noticed you have explicitly created a type 'type1', where I rely on default. I changed my index, replaced default with type1 and told logstash to use type1 as index_type. Now it just works.

jhilden Over a year ago

You will most likely have lots of types within an index so its good practice to call them out

jerik Over a year ago

Help me to find my error. My mappings had the wrong type stated, so that's why elasticsearch could not map the specified date format. Using the correct name of the type solved the problem for me.

Collectives™ on Stack Overflow

Elasticsearch cannot parse date using custom format

1 Answer 1

3 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

3 Comments

Your Answer

Sign up or log in

Post as a guest

Related