ElasticSearch create an index with dynamic properties

Question

Is it possible to create an index, restricting indexing a parent property?

For example,

$ curl -XPOST 'http://localhost:9200/actions/action/' -d '{
  "user": "kimchy",
  "message": "trying out Elasticsearch",
  "actionHistory": [
    { "timestamp": 123456789, "action": "foo" },
    { "timestamp": 123456790, "action": "bar" },
    { "timestamp": 123456791, "action": "buz" },
    ...
  ]
}'

I don't want actionHistory to be indexed at all. How can this be done?

For the above document, I believe the index would be created as

$ curl -XPOST localhost:9200/actions -d '{
  "settings": {
    "number_of_shards": 1
  },
  "mappings": {
    "action": {
      "properties" : {
        "user": { "type": "string", "index" : "analyzed" },
        "message": { "type": "string": "index": "analyzed" },
        "actionHistory": {
          "properties": {
            "timestamp": { 
              "type": "date", 
              "format": "strict_date_optional_time||epoch_millis"
            },
            "action": { "type": "string", "index": "analyzed" }
          }
        }
      }
    }
  }
}'

Would removing properties from actionHistory and replace it with "index": "no" be the proper solution?

This is an example, however my actual situation are documents with dynamic properties (i.e. actionHistory contains various custom, non-repeating properties across all documents) and my mapping definition for this particular type has over 2000 different properties, making searches extremely slow (i.e. worst than full text search from the database).

Hm, how are your search queries built that having actionHistory so different make them have bad performance? Are you using _all in your searches or field names wildcards? — Andrei Stefan
– Andrei Stefan, Commented Jul 26, 2016 at 19:48
That's the downside of allowing user to do whatever they want with query_string. And the same goes with allowing users to create various custom, non-repeating properties across all documents. You can have all sorts of issues from these arbitrary query_strings and having whatever properties in your documents. I'd suggest restricting these freedoms for a more stable and predictable cluster. — Andrei Stefan
– Andrei Stefan, Commented Jul 26, 2016 at 23:07
Unfortunately, as the sole programmer, and with very limited resources (only 1 single physical server), this restriction is the kind of limitation I have to work with :/ — Yanick Rochon
– Yanick Rochon, Commented Jul 26, 2016 at 23:46

Val · Accepted Answer · 2016-07-26 19:00:03Z

1

You can probably get away by using dynamic templates, match on all actionHistory sub-fields and set "index": "no" for all of them.

PUT actions
{
  "mappings": {
    "action": {
      "dynamic_templates": [
        {
          "actionHistoryRule": {
            "path_match":   "actionHistory.*",
            "mapping": {
              "type": "{dynamic_type}",
              "index": "no"
            }
          }
        }
      ]
    }
  }
}

answered Jul 26, 2016 at 19:00

Val

218k14 gold badges377 silver badges384 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

ElasticSearch create an index with dynamic properties

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related