how do you add multiple json files to elasticsearch

Question

I have a directory full of JSON files that I want to index within elastic search. I've looked into pyelastic, yet I'm quite new to both python and elastic search. I've pasted some code below.

from pyelasticsearch import ElasticSearch

# ElasticSearch settings
ES_CLUSTER = 'http://localhost:9200/'
ES_INDEX = 'test'
ES_TYPE = 'doc'
es = ElasticSearch(ES_CLUSTER)

es.bulk_index(ES_INDEX, ES_TYPE, ???)

Kutoff · Accepted Answer · 2015-02-23 14:03:10Z

2

The function is called bulk to bulk index documents.

Loading them into an array then using bulk index will work. Alternatively index them one at a time if there is a very large amount of docs ( > 1000 )

from pyelasticsearch import ElasticSearch
import json
import os

es = ElasticSearch(ES_CLUSTER)

json_docs = []
for filename in os.listdir(os.getcwd()):
    if filename.endswith('.json'):
        with open(filename) as open_file:
            json_docs.append(json.load(open_file))

es.bulk(ES_INDEX, ES_TYPE, json_docs)

edited Feb 23, 2015 at 14:03

answered Feb 23, 2015 at 13:57

Kutoff

3842 silver badges12 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

how do you add multiple json files to elasticsearch

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related