Query ElasticSearch using Spring

Question

I am new to Logstash and ElasticSearch. I would like to read logs, which are saved in ElasticSearch using Spring Data (ElasticsearchRepository) or ElasticsearchTemplate.

A log entry in Elasticsearch looks like this:

{
    "_index": "logstash-2016.08.19",
    "_type": "logs",
    "_id": "AVagGzQ4GS5SuLe66qxR",
    "_score": 1.0,
    "_source": {
        "message": "Aug 19, 2016 12:02:32 AM com.example.server.MOM$2 handleDelivery\n",
        "@version": "1",
        "@timestamp": "2016-08-19T00:02:32.000Z",
        "host": "10.x.x.x",
        "priority": 11,
        "timestamp8601": "2016-08-19T00:02:32Z",
        "logsource": "a094b35d71da",
        "program": "xxx-router_1",
        "pid": "2424",
        "severity": 3,
        "facility": 1,
        "timestamp": "2016-08-19T00:02:32Z",
        "facility_label": "user-level",
        "severity_label": "Error"
    }
}

Based on the JSON data of the log entry, I created the following Java classes:

@Document(indexName = "logstash-2016.08.19")
public class Logstash {
    @Id
    private String _id;    
    private String _index;
    private String _type;
    private Integer _score;
    private Source _source;

/* getters & setters */
}

and

public class Source { 
    private String message;
    private String host;
    private Integer priority;
    private String timestamp8601;
    private String logsource;
    private String program;
    private String pid;
    private Integer severity;
    private Integer facility;
    private String facility_label;
    private String severity_label;

/* getters & setters */

}

Then I try to query the data... In this example I try to get the log with the specified ID. I tried various combinations of the query, but all I get is an empty object or array. Never achieved to get back an actual object... Furthermore I tried Spring's ElasticsearchRepository with no success.

@RestController
public class TestController {

    @Autowired
    ElasticsearchTemplate elt;

    @RequestMapping(value = "/e", method = RequestMethod.GET)
    public List<Logstash> elasticSearch() {

        SearchQuery searchQuery = new NativeSearchQueryBuilder().withQuery(matchAllQuery())
                .withFilter(QueryBuilders.boolQuery().should(QueryBuilders.termQuery("_id", "AVagGzQ4GS5SuLe66qxR")))
                .build();

        return elt.queryForList(searchQuery, Logstash.class);
    }

}

Any help is appreciated =)

Val · Accepted Answer · 2016-08-19 11:43:43Z

1

You should not map the fields starting with an underscore, but only what's in the source.

Your Logstash class should simply be like this

@Document(indexName = "logstash-2016.08.19")
public class Logstash {
    private String message;
    private String host;
    private Integer priority;
    private String timestamp8601;
    private String logsource;
    private String program;
    private String pid;
    private Integer severity;
    private Integer facility;
    private String facility_label;
    private String severity_label;

/* getters & setters */
}

answered Aug 19, 2016 at 11:43

Val

218k14 gold badges377 silver badges384 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

Daniel Over a year ago

Thanks for your answer, but I still get an empty array back. :-(

Daniel Over a year ago

If I can provide any further information to resolve this issue, please let me know. This is my second day I am using Elasticsearch. Therefore my overview is a bit limited.

Val Over a year ago

What if you remove the withFilter() call?

Daniel Over a year ago

Empty array. :( SearchQuery searchQuery = new NativeSearchQueryBuilder().withQuery(matchAllQuery()).build();

Collectives™ on Stack Overflow

Query ElasticSearch using Spring

1 Answer 1

4 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

4 Comments

Your Answer

Sign up or log in

Post as a guest

Related