getting document attribute values without scrolling - elastic search next

Question

I have indexed documents with this simplified structure:

public class Document
{
    public string Id { get; set; }
    public string x { get; set; }

}

I am searching them like this:

var initialResponse = client.Search<Document>(s => s.From(0)
    .Size(100).Scroll(scrollTimeout)
    .Query(q => q
        .MatchPhrase(c => c
            .Field(p => p.Attachment.Content)
            .Analyzer("standard")
            .Boost(1.1)
            .Query(searchTerm)
        )));

and then loop over the documents (more code here). This obviously take time and I wonder, if I could obtain all Ids or even better x values of the matched documents in one payload?

Thanks!

Evaldas Buinauskas · Accepted Answer · 2021-01-15 13:32:51Z

1

Elasticsearch has a index.max_result_window setting with a default value of 10,000.

That's the max number of documents you can retrieve in a single request. If you want to get more than that - you'll need to use scroll or search_after API, as you do already.

answered Jan 15, 2021 at 13:32

Evaldas Buinauskas

14.2k11 gold badges63 silver badges119 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

cs0815 Over a year ago

thanks. however, can one aggregate server side and just get the aggregated payload?

Evaldas Buinauskas Over a year ago

@cs0815 I believe you cannot do it. You'll have to do it on your client-side.

cs0815 Over a year ago

Thanks. wow, somehow I do not see the point of ES then. Let us say I just want the number of document containing a (fuzzy) term?

Evaldas Buinauskas Over a year ago

In that case, you could issue a count query and it would tell you the exact number of documents it found with that term. As for not seeing point for ES, ES being a search engine is optimized to return top-k documents, hence the limit :)

Collectives™ on Stack Overflow

getting document attribute values without scrolling - elastic search next

1 Answer 1

4 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

4 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related