How to _reindex elasticsearch data to new mapping (from flat fields to objects)?

Question

I have an old index (elasticsearch index) has more than 20K objects, this index has fields

{
    "title": "Test title",
    "title_ar": "عنوان تجريبي",
    "body": "<p>......</p>"
}

I want to _reindex them to convert all data to new mapping like this

{
    "title_1": {
        "en": "Test title",
        "ar": "عنوان تجريبي"
    },
    "body": "<p>......</p>"
}

What is the best elasticsearch pipeline processor to make this conversion available in _reindex API?

You will not be able to do that without recreating your index, because the title field already exists as text and you cannot change it to be an object — Val
– Val, Commented Jun 26, 2019 at 14:33
I create a new index with new mapping, I can name it title_1 if it is not possible. the most important thing is converting it to object — Abdallah Sabri
– Abdallah Sabri, Commented Jun 26, 2019 at 14:43
Then if you create a rand new index, it's definitely possible and easy to do — Val
– Val, Commented Jun 26, 2019 at 14:44

Val · Accepted Answer · 2019-06-26 14:47:06Z

2

I suggest to simply use the reindex API to do this:

POST _reindex
{
  "source": {
    "index": "old_index"
  },
  "dest": {
    "index": "new_index"
  },
  "script": {
    "source": "ctx._source.title = [ 'en' : ctx._source.title, 'ar': ctx._source.title_ar]",
    "lang": "painless"
  }
}

If in your old_index index you have this:

{
    "title": "Test title",
    "title_ar": "عنوان تجريبي",
    "body": "<p>......</p>"
}

In your new index, you'll have this:

{
    "title": {
        "en": "Test title",
        "ar": "عنوان تجريبي"
    },
    "body": "<p>......</p>"
}

answered Jun 26, 2019 at 14:47

Val

218k14 gold badges377 silver badges384 bronze badges

Sign up to request clarification or add additional context in comments.

8 Comments

Abdallah Sabri Over a year ago

I get the following error ` "script_stack": [ "ctx._source.title = [ 'en' : ctx._source.title, 'ar': ctx._source.title_ar]", " ^---- HERE" ],`

Abdallah Sabri Over a year ago

there is no .title field in ctx._source.title, but it is a mandatory field

Val Over a year ago

Not sure I understand your second comment

Abdallah Sabri Over a year ago

I get ScriptException[runtime error]; nested: NullPointerException; on the title_1 in the following statement "source": "ctx._source.title_1 = ....

Val Over a year ago

You don't need to create the title_1 field since you're creating a new index you can keep title

|

Collectives™ on Stack Overflow

How to _reindex elasticsearch data to new mapping (from flat fields to objects)?

1 Answer 1

8 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

8 Comments

Your Answer

Sign up or log in

Post as a guest

Related