87 questions
0
votes
1
answer
287
views
How can I use kerberos under python elasticsearch?
I checked the documentation of elasticsearch:7.x but still didn't find the way how I can force the python ES package to use my kerberos token.
I can reach the ES through a browser, curl ( curl --...
- 178
2
votes
1
answer
649
views
How to search exact index (not index pattern) in elasticsearch eland - updated?
I am calling elasticsearch data using eland. The documentation is simple and I am able to implement it, but when searching the index it searches the index string using es_index_pattern which is ...
- 7,315
1
vote
1
answer
242
views
How to search exact index (not index pattern) in elasticsearch eland?
I am calling elasticsearch data using eland. The documentation is simple and I am able to implement it, but when searching the index it search using es_index_pattern which is basically a wildcard ...
- 7,315
4
votes
3
answers
6k
views
How to select or retrieve only specific fields after elasticsearch query?
I am able to query an index in elasticsearch. And, now I want to narrow down the data to some specific fields. But, I am keep getting errors.
Here is my query:
es = Elasticsearch(hosts="myhost&...
- 7,315
0
votes
1
answer
1k
views
Python Elasticsearch update_by_query not working
I want to update multiple docs using update_by_query in elasticsearch.py (version 7.13.1)
My query is:
es_query = {
"query": {
"match": {"user_id.keyword": ...
- 165
0
votes
1
answer
317
views
Python elasticsearch - GET finds the document but search does not
I am trying basic commands in elasticsearch and I am stuck with basic search.
My script:
from elasticsearch import Elasticsearch
INDEX_NAME = 'person'
es = Elasticsearch([{'host': 'localhost', 'port'...
- 444
0
votes
1
answer
358
views
Elasticsearch problem with pre-defined mapping and then index docs
I am trynig to index stackoverflow data. First of all I create an index with specified mapping and setting.
@classmethod
def create_index_with_set_map(cls, name, elasticsearch):
"&...
- 131
1
vote
2
answers
6k
views
Python Urllib ContextualVersionConflict
Till Oct 20, it was fine. Oct 21, it failed with the below messages. Can any one help here
command: /usr/local/bin/python -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/...
- 53
-1
votes
1
answer
233
views
python elasticsearch get field from given doc_id
My input is <index_name>, <doc_id>, <field_name>, i want the value of the field
I am looking for python-client equivalent of
GET <index_name>/_doc/<doc_id>/?...
- 1,082
0
votes
1
answer
63
views
Can't set types of fields when creating indexes
I am using Elasticsearch 7.6.2 (by running the official docker image with the default options) and py-elasticsearch 7.6.0 (installed with version >=7.6 <8)
I am trying to ...
- 6,252
3
votes
3
answers
710
views
How to update_index for haystack from a external script?
I am using Django Haystack with ElasticSearch backend for my search page.
I am using MongoDB as my Database.
Everything was working fine in my search page.
PROBLEM
My web-application uses an external ...
0
votes
0
answers
458
views
Painless script throwing connection error in py-elasticsearch
I have the following script search query:
bdy = {
"query": {
"function_score": {
"script_score": {
"script": {
"lang": "painless",
"inline": "def test = "+test1+"; ...
4
votes
1
answer
3k
views
Create custom analyzer filter for a index in py-elasticsearch-dsl
i'm working with py-elasticsearch-dsl for my master, i'm creating a index of title documents in a corpus of turkish titles, and i need implements a custom lowercase analyzer for turkish language: ...
- 2,310
4
votes
1
answer
8k
views
How to have Range and Match query in one elastic search query using python?
I have to find the matching documents which have the string, for example: "sky", within some "key" range. When I write separate match and range query, I get the output from the ES but it throws an ...
- 1,258
5
votes
2
answers
10k
views
Elastic search not giving data with big number for page size
Size of data to get: 20,000 approx
Issue: searching Elastic Search indexed data using below command in python
but not getting any results back.
from pyelasticsearch import ElasticSearch
es_repo = ...
- 2,678
3
votes
4
answers
3k
views
How to delete documents from multiple indices using particular field name in elasticsearch python?
This is sample document.
{
"_index": "mqtt-index-2018.01.23",
"_type": "iot_data",
"_id": "AWEjA7LNRU4cTUO-Lyj4",
"_score": null,
"_source": {
"message": "{\"datastream_name\": \"roshni\", \"value\":...
- 229
1
vote
1
answer
9k
views
Elasticsearch Python API results in "search() missing 1 required positional argument" on a simple query
I am desperately trying to perform a simple search operation on elasticsearch, but fail since hours. This is my code:
res = es.search(index="people", doc_type="test", body={"query":{"match":{"name": "...
- 13
1
vote
0
answers
253
views
flask-socketio / elasticsearch (in the context of)
I was wondering if anyone had actually used the official python elastic search ES client in the context of flask-socketio (with either eventlet / gevent + the monkey patches)?
The official python ES ...
- 43
0
votes
0
answers
1k
views
How to push data to elasticsearch from dataframe in python
I am trying to use elastic search libraries like pyelasticsearch and elasticsearch I am not getting any method where i can pass dataframe and that method will load data frame data to elastic search.
...
- 393
2
votes
0
answers
1k
views
Elasticsearch python bulking parallel issue
I am attempting to bulk index a generator using the parallel bulk method from elasticsearch helpers in python, however it seems that this method doesn't perform anything. If I use the regular bulk ...
- 460
5
votes
0
answers
575
views
pyelasticsearch bulk ValueError: too many values to unpack (expected 2)
I'm new in Python and Elasticsearch.
I'm trying to write code to insert documents in Elassticsearch using bulk. It is based on this description:
http://pyelasticsearch.readthedocs.io/en/latest/api/#...
- 51
2
votes
1
answer
385
views
elasticsearch aggregate unique entries of hashes and find all tags
I have two sets of hashes in single index with a different tag as an additional field like below:
Index: hashes-*
sha1-c tag
abcdefg12345 collect
abcdefg12345 collect
...
- 1,815
8
votes
5
answers
17k
views
ElasticSearch query_string fails to parse query with some characters
I'm using the ElasticSearch (2.4) and the official Python client to perform simple queries. My code:
from elasticsearch import Elasticsearch
es_client = Elasticsearch("localhost:9200")
index = "...
- 6,135
0
votes
1
answer
988
views
py-elasticsearch stop printing errors
I am using a py-elasticsearch to query elasticsearch:
try:
res = es.get(index='unique_names', doc_type='name', id=token, ignore=['404'])
except elasticsearch.exceptions.NotFoundError:
continue
As ...
- 969
1
vote
0
answers
162
views
Elasticsearch - Get unique records `msearch`
I'm using msearch to query ES. following is my query:-
[{
'index': u 'INDEX',
'type': u 'TYPE'
}, {
'query': {
'term': {
'FIELD1': u 'ID'
}
},
'from': ...
- 17.6k
0
votes
1
answer
2k
views
Error while loading bulk data into Elasticsearch
I am using Elasticsearch in python. I have data in pandas frame(3 columns), then I added two columns _index and _type and converted the data into json with each record using pandas inbuilt method.
...
- 2,621
2
votes
1
answer
6k
views
Python Elasticsearch DSL query value from a list
I am using Python's Elasticsearch DSL package:
http://elasticsearch-dsl.readthedocs.org/en/latest/search_dsl.html
My sample elasticsearch entry looks like:
{
"_index" : "users",
"_type" : "...
- 4,840
0
votes
1
answer
202
views
ElasticSearch: Access outer document fields from within an nested aggregated query
I have the following mapping:
{
"dynamic": "strict",
"properties": {
"id": {
"type": "string"
},
"title": {
"type": "string"
},
...
- 111
2
votes
1
answer
4k
views
Elasticsearch mapping empty string
In Elasticsearch, one of my field is a date which I define the mapping with custom date format matching my date.
However, some cases the value of my date field is just an empty string "LastUpdateDate"...
- 3,270
2
votes
2
answers
6k
views
Python elasticsearch dsl search multi fields and order result
How can i search in elasticsearch dsl python module on multi fields, example on title and body field and order it by created_at field DESC.
I have this example that search only on title field:
q = ...
- 4,381
1
vote
2
answers
12k
views
Change cluster name in elastic search
How to rename the current cluster in elasticsearch config?
i want to rename the cluster without it going down if possible.
- 11
1
vote
1
answer
95
views
search array value by index in elasticsearch
i have two documents stored in elasticsearch database
one is
{
q :["python" , "foo"]
}
and other is
{
q :["python" , "moo","foo"]
}
now my problem is to search document in which python is on ...
- 1,234
1
vote
1
answer
2k
views
How to do bucket aggregation on multi value field in elasticsearch
Say each document in my elasticsearch index is a blog post which consists of only two fields, title and tags. The title field is just a string while tags is a multi value field.
If I have three ...
user2767176
4
votes
1
answer
4k
views
highlighting based on term or bool query match in elasticsearch
I have two queries.
{'bool':
{'must':
{ 'terms': 'metadata.loc':['ten','twenty']}
{ 'terms': 'metadata.doc':['prince','queen']}
}
{'should':
{ 'match':...
- 1,575
8
votes
1
answer
5k
views
No handlers could be found for logger "elasticsearch.trace"
Updated:
Turns out, this is not a function of cron. I get the same behavior when running the script from the command line, if it in fact has a record to process and communicates with ElasticSearch.
I ...
- 6,210
2
votes
0
answers
297
views
elasticsearch-py driver is not filtering data properly when aggregating
I faced a weird with elasticsearch python drivers and would like if someone can explain it to me! The below code works directly from cURL but doesn't work with python-requests or elasticsearch-py, ...
- 2,221
1
vote
1
answer
784
views
Matching document by ElasticSearch's Percolate API always returns no matches if registered queries contain terms
I try to use Percolator by Elasticsearch and I have a minor issue.
Suppose our document looks like this:
{
"doc": {
"full_name": "Pacman"
"company": "Arcade Game LTD",
"...
- 895
48
votes
4
answers
65k
views
python elasticsearch client set mappings during create index
I can set mappings of index being created in curl command like this:
{
"mappings":{
"logs_june":{
"_timestamp":{
"enabled":"true"
},
"properties":{
"...
user4005632
1
vote
0
answers
713
views
python elasticsearch query for boolean
I have the following query:
Bool(must=[Terms(avail_req=['Available'], boost=2.0), Terms(boost=2.0,
sale_rent=['Sale']), Nested(path='listings', query=Bool(should=
[Terms(boost=2.0, listings....
- 1,575
1
vote
0
answers
370
views
ElasticSearch: minimum_should_match and length of terms list
Using ElasticSearch I'm trying to use the minimum_should_match option on a Terms Query to find documents that have a list of longs that is X% similar to the list of longs I'm querying with.
e.g:
{
...
- 3,784
-1
votes
2
answers
668
views
Any efficient way to get unique terms from Elasticsearch index
I aim is to store all unique term along with their md5 hashes in a database. I have a 1 million document index which has ~400000 unique terms. I got this figure from using aggregations in ...
- 6,028
-1
votes
1
answer
354
views
Elastic Search Index in python
I want to push my data from a dictionary to Elasticsearch. How can I create an index for the same?
I tried using the Curl commands as well on Linux server
curl -XPUT 'http://localhost:9200/osint/' -d ...
-1
votes
1
answer
352
views
In elasticsearch-py, how different should the search command be if I used custom tokenization during indexing?
I am using elasticsearch-py to index tweets (originally in JSON format). In order to preserve special characters like hashtags, user targets and emoticons, I specified a special mapping while creating ...
- 1,307
2
votes
1
answer
2k
views
elasticsearch-py can't query with script_score
I'm currently trying to query ES through Python, using elasticsearch-py but with no success... I'm testing the request on the "elasticsearch-head" plugin and it works fine (results come scored)
...
- 271
1
vote
0
answers
102
views
How to make filter aggregations using PyES
{
"aggs" : {
"in_stock_products" : {
"filter" : { "range" : { "stock" : { "gt" : 0 } } },
"aggs" : {
"avg_price" : { "avg" : { "field" : "price" } }
...
10
votes
1
answer
6k
views
How to disconnect from elasticsearch-py client/connection-pool
Whats the correct way to disconnect an elasticsearch-py (python) client rsp. close an existing connection pool? I cannot find any information in the docs.
- 163
0
votes
1
answer
2k
views
streaming and bulk update to elasticsearch
As part of data analysis, I collect records I need to store in Elasticsearch. As of now I gather the records in an intermediate list, which I then write via a bulk update.
While this works, it has ...
- 30.6k
8
votes
3
answers
17k
views
Elasticsearch Aggregation by Day of Week and Hour of Day
I have documents of type:
[{"msg":"hello", date: "some-date"},{"msg":"hi!", date: "some-date"}, ...
I want to have the count of documents by day of week. For example x messages were sent on Monday ...
- 5,869
1
vote
1
answer
190
views
How to use elasticsearch to search some words from group by result?
Here is the sample data:
In the type blog_comments, I have some comments data, whose structure like this:
{"blog_id": 1, "comments": "Apple", "comment_id": 1}
And for #1 and #2 blog, there are 6 ...
- 726
3
votes
1
answer
223
views
Specify search type in pyelasticsearch
Is there a way to specify the search type when using pyelasticsearch? There's nothing in the documentation for 'scan', 'scroll', 'query_and_fetch', etc. Running the following code:
from ...
- 105