parse deeply nested elasticsearch aggregations using it's client API

Question

I am using Java client API to get aggregations back. Following is the structure which I am dealing with.

aggregations
top_models
   buckets
       key : "BMW"
      doc_count : 3
      top_models
           buckets
               key : "X5"
               doc_count : 2
              top_hits
                   source
                      model : X5
                      color : Red
                  source
                     model:X5
                    color : White
           key : "X3"
               doc_count : 1
              top_hits
                   source
                      model : X3
                      color : Red
      key : "Mercedes"
      doc_count : 2
      top_models
           buckets
               key : "Benz"
               doc_count : 1
               top_hits
                   source
                      model : Benz
                      color : Red

              key : "ML"
              doc_count : 1
              top_hits
                   source
                      model : ML
                      color : Black

I am trying following (toy) code to retrieve all the results.

def getAggregations(aggres: Option[Aggregations]): Option[Iterable[Any]] = {


aggres map { agg =>


  val aggS = agg.asMap().asScala


  aggS map {


    case (name, termAgg: Terms) => getBuckets(Option(termAgg.getBuckets()))


    case (name, topHits: TopHits) =>


      val tHits = Option(topHits.getHits())


      tHits map { th => getTopHits(th.asScala)
    }


    case (h, a: InternalAvg) => println(h + "=>" + a.getValue());


  }





}


}





def getBuckets(buckets: Option[java.util.Collection[Bucket]]) = {


buckets map { bks =>


  val bksS = bks.asScala


  bksS map { b =>


    println("Bucket Key =>" + b.getKey())


    println("Doc count =>" + b.getDocCount())


    getAggregations(Option(b.getAggregations())


  }


  }

 }

need to populate final result to this class

case class FinalResponse(bucketName: String, count: Long, children: List[FinalResponse])

With nested relationship between Aggregations and Buckets it's becoming convoluted to retrieve all aggregation results. how do you approach this?

score 2 · Accepted Answer · 2015-06-08 11:39:46Z

2

+50

At my previous project we use that way to desirialize complex objects from elastic search:

Elastic search allow you to get a plain json view (as a string) of each element of result set. Than we simply use Jacson json library with scala module and make pojo-like classes to deserialize data.

Elastic search java api is horrible multi-laer set of nested maps. forget about that.

val hits = qb.execute().actionGet().getHits().getHits().asScala
hits.map { hit =>
  (hit.getId, hit.getSourceAsString, hit.getVersion)
}

For agregations it should be availible too

edited Jun 8, 2015 at 11:39

answered Jun 8, 2015 at 11:22

user2056986

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

parse deeply nested elasticsearch aggregations using it's client API

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related