elasticsearch script to check if field exists and create it

Question

I've created a script which records the history of the tags that are applied to my documents in elastic. The names of the tags are dynamic, so when I try to move the current tag to the history field, it fails for tags that do not already have a history field.

This is my script to copy the current tags, to the tag history field:

script:"ctx._source.tags[params.tagName.toString()].history.add(ctx._source.tags[params.tagName.toString()].current)"

This is what the documents look like:

"tags": {
                        "relevant": {
                            "current": {
                                "tagDate": 1501848372292,
                                "taggedByUser": "dev",
                                "tagActive": true
                            },
                            "history": [
                                {
                                    "tagDate": 1501841137822,
                                    "taggedByUser": "admin",
                                    "tagActive": true
                                },
                                {
                                    "tagDate": 1501841334127,
                                    "taggedByUser": "admin",
                                    "tagActive": true
                                },
                                }}}}

The users can add new tags dynamically, so what I want to do is create the history object if it does not exist and then I can populate it.

There is very little documentation available for the elasticsearch scripting, so I'm hoping someone wise will know the answer as I'm sure that checking for a field and creating it are fundamental things to the elastic scripting languages.

Update

So, having rethought the structure of this index, what I want to achieve is the following:

tags:[
   {hot:
    {current:{tagDate:1231231233, taggedbyUser: user1, tagStatus: true},
    history:[ {tagDate:123444433, taggedbyUser: user1, tagStatus: true},
         {tagDate:1234412433, taggedbyUser: user1, tagStatus: true}
   ]
 }

  {interesting:
     {current:{tagDate:1231231233, taggedbyUser: user1, tagStatus: true},
     history:[ {tagDate:123444433, taggedbyUser: user1, tagStatus: true},
           {tagDate:1234412433, taggedbyUser: user1, tagStatus: true}
     ]
} 
]

The tag names in this example are "hot" and "interesting", however the user will be able enter any tag name they want, so these are in no way predefined. When a user tags a document in elastic and the tag that is applied already exists in elastic, it should more the "current" tag to the "history" array and then overwrite the "current" tag with the new values.

Thank you for the responses to date, however the example code does not work for me.

The problem I think I'm having is that, first the code will need to loop through all of the tags and get the name. I then want to compare each of these to the name that I am supplying in the params. I think that this is where the first issue is arising.

I then need to move the "current" object to the "history" array. There also appears to be an issue here. I'm trying to use the "ctx._source.tags[i].history.add(params.param1), however nothing is added.

Any thoughts?

Thanks!

Joanna Mamczynska · Accepted Answer · 2017-08-07 16:31:58Z

It's a bit more complicated because you need to do three things in the script:

if history does not already exist, initialize the array
move current tag to history
replace old current tag with the new one

Assuming that your initial document looks like this (note no history yet):

{
    "_id": "AV2uvqCUfGXyNt1PjTbb",
    "tags": {
        "relevant": {
            "current": {
                "tagDate": 1501848372292,
                "taggedByUser": "dev",
                "tagActive": true
            }
        }
    }
}

to be able to execute these three steps, you need to run following script:

curl -X POST \
  http://127.0.0.1:9200/script/test/AV2uvqCUfGXyNt1PjTbb/_update \
  -d '{ 
    "script": {
        "inline": "if (ctx._source.tags.get(param2).history == null) ctx._source.tags.get(param2).history = new ArrayList();  ctx._source.tags.get(param2).history.add(ctx._source.tags.get(param2).current); ctx._source.tags.get(param2).current = param1;",
        "params" : {
            "param1" : {
                "tagDate": 1501848372292,
                "taggedByUser": "my_user",
                "tagActive": true
            },
            "param2": "relevant"
        }
    }
}'

And I get as a result:

{
    "_id": "AV2uvqCUfGXyNt1PjTbb",
    "_source": {
        "tags": {
            "relevant": {
                "current": {
                    "tagActive": true,
                    "tagDate": 1501848372292,
                    "taggedByUser": "my_user"
                },
                "history": [
                    {
                        "tagDate": 1501848372292,
                        "taggedByUser": "dev",
                        "tagActive": true
                    }
                ]
            }
        }
    }
}

Running the same script with a new content of parm1 (new tag) gives:

{
    "_id": "AV2uvqCUfGXyNt1PjTbb",
    "_source": {
        "tags": {
            "relevant": {
                "current": {
                    "tagActive": true,
                    "tagDate": 1501841334127,
                    "taggedByUser": "admin"
                },
                "history": [
                    {
                        "tagDate": 1501848372292,
                        "taggedByUser": "dev",
                        "tagActive": true
                    },
                    {
                        "tagActive": true,
                        "tagDate": 1501848372292,
                        "taggedByUser": "my_user"
                    }
                ]
            }
        }
    }
}

Update - if `tags` is a list

If tags is a list of "inner json objects", for example:

{
    "tags": [
        {
            "relevant": {
                "current": {
                    "tagDate": 1501841334127,
                    "taggedByUser": "dev",
                    "tagActive": true
                }
            }
        },
        {
            "new_tag": {
                "current": {
                    "tagDate": 1501848372292,
                    "taggedByUser": "admin",
                    "tagActive": true
                }
            }
        }
    ]
}

you have to iterate over the list to find the index of the right element. Let's say you want to update element new_tag. First, you need to check if this tag exists - if so, get its index, if not, return from the script. Having the index, just get the right element and you can go almost the same as before. The script looks like this:

int num = -1;
for (int i = 0; i < ctx._source.tags.size(); i++) {
    if (ctx._source.tags.get(i).get(param2) != null) {
        num = i;
        break;
    };
};
if (num == -1) {
    return;
};
if (ctx._source.tags.get(num).get(param2).history == null)
    ctx._source.tags.get(num).get(param2).history = new ArrayList();
ctx._source.tags.get(num).get(param2).history.add(ctx._source.tags.get(num).get(param2).current);
ctx._source.tags.get(num).get(param2).current = param1;

And the wole query:

curl -X POST \
  http://127.0.0.1:9200/script/test/AV29gAnpqbJMKVv3ij7U/_update \
  -d '{ 
    "script": {
        "inline": "int num = -1; for (int i = 0; i < ctx._source.tags.size(); i++) {if (ctx._source.tags.get(i).get(param2) != null) {num = i; break;};}; if (num == -1) {return;}; if (ctx._source.tags.get(num).get(param2).history == null) ctx._source.tags.get(num).get(param2).history = new ArrayList();  ctx._source.tags.get(num).get(param2).history.add(ctx._source.tags.get(num).get(param2).current); ctx._source.tags.get(num).get(param2).current = param1;",
        "params" : {
            "param1" : {
                "tagDate": 1501848372292,
                "taggedByUser": "my_user",
                "tagActive": true
            },
            "param2": "new_tag"
        }
    }
}
'

Result:

{
    "tags": [
        {
            "relevant": {
                "current": {
                    "tagDate": 1501841334127,
                    "taggedByUser": "dev",
                    "tagActive": true
                }
            }
        },
        {
            "new_tag": {
                "current": {
                    "tagActive": true,
                    "tagDate": 1501848372292,
                    "taggedByUser": "my_user"
                },
                "history": [
                    {
                        "tagDate": 1501848372292,
                        "taggedByUser": "admin",
                        "tagActive": true
                    }
                ]
            }
        }
    ]
}

Thanks, this is working great! If "tags" were an array, how would I modify the code to account for that? I've trued if (ctx._source.tags.get(param2).history == null) ctx._source.tags.get(param2).history = new ArrayList(); ctx._source.tags.get(param2).history.add(ctx._source.tags.get[param2].current); ctx._source.tags.get(param2).current = param1; Where I've replaced the () with [], however that doesnt appear to be working.
For the case when tags is a list, you need an additional step - iteration over the collection to find the right object to update. I updated my answer with details how to do it.
Thanks, I wasn't able to get this working. I've updated my post above. If you had any thoughts, it would be much appreciated. Painless is PAINFUL.

blackmamba · Accepted Answer · 2017-08-04 11:36:16Z

4

I think you can do something like this in groovy scripting

{
  "script": "if( ctx._source.containsKey(\"field_name\") ){ ctx.op = \"none\"} else{ctx._source.field_name= field_value;}"
}

answered Aug 4, 2017 at 11:36

blackmamba

5563 silver badges11 bronze badges

2 Comments

A Dev Over a year ago

Thanks, I tried that. The problem I have is that the "field_name" is a parameter and it doesn't work for me. Also, the field that I need to check is: ctx._source.tags[params.tagName.toString()].history, so I may be pointing this at the wrong place. I tried ctx._source.tags[params.tagName.toString()].containsKey(\"history\") but I just get a lot of errors.

blackmamba Over a year ago

can you update your question with the mapping and an example ?

Collectives™ on Stack Overflow

elasticsearch script to check if field exists and create it

2 Answers 2

Update - if `tags` is a list

3 Comments

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Update - if `tags` is a list

3 Comments

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related