0

How to do an automated index creation at ElasticSearch?

Just like wordpress? See: http://gibrown.wordpress.com/2014/02/06/scaling-elasticsearch-part-2-indexing/

In our case we create one index for every 10 million blogs, with 25 shards per index.

Any light?

Thanks!

1 Answer 1

1

You do it in whatever your favorite scripting language is. You first run a query getting a count of the number of documents in the index. If it's beyond a certain amount you create a new one, either via an Elasticsearch API or a curl.

Here's the query to find the number of docs:

curl --XGET 'http://localhost:9200/youroldindex/_count'

Here's the index creation curl:

curl -XPUT 'http://localhost:9200/yournewindex/' -d '{
    "settings" : {
        "number_of_shards" : 25,
        "number_of_replicas" : 2
    }
}'

You will also probably want to create aliases so that your code can always point to a single index alias and then change the alias as you change your hot index:

http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/indices-aliases.html

You will probably want to predefine your mappings too:

curl -XPUT 'http://localhost:9200/yournewindex/yournewmapping/_mapping' -d '
{
    "document" : {
        "properties" : {
            "message" : {"type" : "string", "store" : true }
        }
    }
}
'

Elasticsearch has fairly complete documentation, a few good places to look:

http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/mapping.html http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/indices-create-index.html

Sign up to request clarification or add additional context in comments.

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.