I'm trying to get all index document using python client but the result show me only the first document This is my python code :
res = es.search(index="92c603b3-8173-4d7a-9aca-f8c115ff5a18", doc_type="doc", body = {
'size' : 10000,
'query': {
'match_all' : {}
}
})
print("%d documents found" % res['hits']['total'])
data = [doc for doc in res['hits']['hits']]
for doc in data:
print(doc)
return "%s %s %s" % (doc['_id'], doc['_source']['0'], doc['_source']['5'])
try "_doc" instead of "doc"
res = es.search(index="92c603b3-8173-4d7a-9aca-f8c115ff5a18", doc_type="_doc", body = {
'size' : 100,
'query': {
'match_all' : {}
}
})
Elasticsearch by default retrieve only 10 documents. You could change this behaviour - doc here . The best practice for pagination are search after query and scroll query. It depends from your needs. Please read this answer Elastic search not giving data with big number for page size
To show all the results:
for doc in res['hits']['hits']:
print doc['_id'], doc['_source']
You can also use elasticsearch_dsl and its Search API which allows you to iterate over all your documents via the scan method.
import elasticsearch
from elasticsearch_dsl import Search
client = elasticsearch.Elasticsearch()
search = Search(using=client, index="92c603b3-8173-4d7a-9aca-f8c115ff5a18")
for hit in search.scan():
print(hit)
you can find info from here https://discuss.elastic.co/t/get-all-documents-from-an-index/86977/5
es = Elasticsearch(["http://x.x.x.x:9200/"])
doc = {"size": 10000, "query": {"match_all": {}}}
resObj = es.search(index="myIndex", doc_type="myType", body=doc, scroll="1m")
download_alldata = []
download_alldata.extend(resObj["hits"]["hits"])
total_data_size = resObj["hits"]["total"]["value"]
allbar = tqdm(total=total_data_size, desc="processing")
download_data = []
download_data.extend(resObj["hits"]["hits"])
allbar.update(len(resObj["hits"]["hits"]))
while True:
try:
i = es.scroll(scroll_id=resObj["_scroll_id"], scroll="1m")["hits"][
"hits"
]
download_data.extend(i)
# print(t)
allbar.update(len(i))
if len(i) == 0:
break
except Exception as e:
break
print(len(download_data))
You can try the following query. It will return all the documents.
result = es.search(index="index_name", body={"query":{"match_all":{}}})
I dont see mentioned that the index must be refreshed if you just added data. Use this:
es.indices.refresh(index="index_name")
doc_type=doc. can you recheck.