ElasticSearch: How to update a document type in an existing index

Question

Long story short:

I have created a strict template, created several daily indices and changes one of the document types (%_level type from float to integer) in the template.

After that, I've created the rest of the daily indices.

The problem:

I need to change the document type (%_level type from float to integer) in the old indices to be compatible with the new indices. How can I do this?

Now for the details...

I have the following strict template:

PUT _template/example-template
{
    "order":0,
    "version":200,
    "index_patterns":["example-*"],
    "settings":{
        "index":{
            "number_of_shards":4
         }
    },
    "mappings": {
        "iterations": {
            "dynamic":"strict",
            "properties": {
                "%_average1": {
                    "type":"float"
                },
                "%_average2": {
                    "type":"float"
                },
                "sum": {
                    "type":"integer"
                },
                "%_level": {
                    "type":"float"
                }
            }
        }
    }
}

Several indices were created with this template.

example-20191220
example-20191221
example-20191222
example-20191223

After a while I've realised that we need to change %_level type from float to integer, so I have changed the template into the following:

PUT _template/example-template
{
    "order":0,
    "version":200,
    "index_patterns":["example-*"],
    "settings":{
        "index":{
            "number_of_shards":4
         }
    },
    "mappings": {
        "iterations": {
            "dynamic":"strict",
            "properties": {
                "%_average1": {
                    "type":"float"
                },
                "%_average2": {
                    "type":"float"
                },
                "sum": {
                    "type":"integer"
                },
                "%_level": {
                    "type":"integer"
                }
            }
        }
    }
}

Now the the following indices were created where %_level type is an integer.

example-20192412
example-20192512
example-20192612

But the old indices contains indices with %_level that is float and the new indices are with %_level that is an integer.

I needed to convert the old indices %_level from float to integer so I can build a report with all the indices.

How can I change the %_level in the old indices from float to integer?

raven99 · Accepted Answer · 2019-12-30 09:46:36Z

Although you can add and remove document names from an exisitng index by providing the document ID, it is problematic to do so if you want to update the document type, and to apply it on all the index document.

A good solution for this problem will be reindex.

Important:

The reindex process can convert some types implicitly (Numeric Type Casting) and the others explicitly.

In case you need implicit conversion, you can skip section 2

Meaning, apply the following over all the old indices (e.g. example-20191220):

create a new index from example-20191220 that will be called example-20191220-new
add a painless script in the creation process that convert %_level type from float to integer
delete the old index example-20191220
reindex example-20191220-new into example-20191220

Now, the 'updated' index example-20191220 is with the correct %_level type.

The reindex in section 2 should look like the following:

(it is performed and tested on Kibana Dev Tools)

POST _reindex
{
    "source": {
        "index": "example-20191220"
    },
    "dest": {
        "index": "example-20191220-new"
    },
    "script": {
    "lang": "painless", 
    "source": 
        """
        Double num = ctx._source['%_level'];
        if(num != null)
        { ctx._source['%_level'] = num.intValue(); }
        """
    }
}

The reindex in section 4 should look like the following:

(it is performed and tested on Kibana Dev Tools)

POST _reindex
{
    "source": {
        "index": "example-20191220-new"
    },
    "dest": {
        "index": "example-20191220"
    }
}

Collectives™ on Stack Overflow

ElasticSearch: How to update a document type in an existing index

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related