42

I have documents which contains only "url"(analyzed) and "respsize"(not_analyzed) fields at first. I want to update documents that match the url and add new field "category" I mean; at first doc1:

{
 "url":"http://stackoverflow.com/users/4005632/mehmet-yener-yilmaz",
 "respsize":"500"
}

I have an external data and I know "stackoverflow.com" belongs to category 10, And I need to update the doc, and make it like:

{
 "url":"http://stackoverflow.com/users/4005632/mehmet-yener-yilmaz",
 "respsize":"500",
 "category":"10"
}

Of course I will do this all documents which url fields has "stackoverflow.com" and I need the update each doc oly once.. Because category data of url is not changeable, no need to update again. I need to use _update api with _version number to check it but cant compose the dsl query. EDIT I run this and looks works fine: enter image description here But documents not changed.. enter image description here

Although query result looks true, new field not added to docs, need refresh or etc?

Improve this question

2 Answers 2

Reset to default
81
+50

You could use the update by query plugin in order to do just that. The idea is to select all document without a category and whose url matches a certain string and add the category you wish.

curl -XPOST 'localhost:9200/webproxylog/_update_by_query' -H "Content-Type: application/json" -d '
{
  "query": {
    "filtered": {
      "filter": {
        "bool": {
          "must": [
            {
              "term": {
                "url": "stackoverflow.com"
              }
            },
            {
              "missing": {
                "field": "category"
              }
            }
          ]
        }
      }
    }
  },
  "script" : "ctx._source.category = \"10\";"
}'

After running this, all your documents with url: stackoverflow.com that don't have a category, will get category: 10. You can run the same query again later to fix new stackoverflow.com documents that have been indexed in the meantime.

Also make sure to enable scripting in elasticsearch.yml and restart ES: 

script.inline: on 
script.indexed: on

In the script, you're free to add as many fields as you want, e.g.

  ...
  "script" : "ctx._source.category1 = \"10\"; ctx._source.category2 = \"20\";"

UPDATE

ES 2.3 now features the update by query functionality. You can still use the above query exactly as is and it will work (except that filtered and missing are deprecated, but still working ;).

Improve this answer
Sign up to request clarification or add additional context in comments.

13 Comments

The first term in your update-by-query should be on field heroname not url. The screenshots don't seem to have anything in common with the initial documents in your question.
its analyzed, real field data Wrath King and SAND KING but I can get these docs with term:"king"
Yes, you can but in the same script. I'm updating the answer.
ES 2.3 now features the update by query functionality !!
I have been trying to do this but I keep getting the following error: {"error":{"root_cause":[{"type":"class_cast_exception","reason":"java.lang.String cannot be cast to java.util.Map"}],"type":"class_cast_exception","reason":"java.lang.String cannot be cast to java.util.Map"},"status":500} I don't understand why is this error being thrown. Can you please point me in the right direction?
|
1

That all sounds great but just to add to @Val answer, Update By Query is available form ElasticSearch 2.x but not for earlier versions. In our case we're using 1.4 for legacy reasons and there is no chance of upgrading in forseeable future so another solution is using the Update by query plugin provided here: https://github.com/yakaz/elasticsearch-action-updatebyquery

Improve this answer

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.