Extract Options from potentially null JSON values using for expression

Question

I have a JSON document where some values can be null. Using for expressions in json4s, how I can yield None, instead of nothing?

The following will fail to yield when the value for either of the fields FormattedID or PlanEstimate is null.

val j: json4s.JValue = ...
for {
  JObject(list) <- j
  JField("FormattedID", JString(id)) <- list
  JField("PlanEstimate", JDouble(points)) <- list
} yield (id, points)

For example:

import org.json4s._
import org.json4s.jackson.JsonMethods._

scala> parse("""{
     |   "FormattedID" : "the id",
     |   "PlanEstimate" : null
     | }""")
res1: org.json4s.JValue = JObject(List((FormattedID,JString(the id)), 
    (PlanEstimate,JNull)))

scala> for {                                      
     | JObject(thing) <- res1                     
     | JField("FormattedID", JString(id)) <- thing
     | } yield id                                 
res2: List[String] = List(the id)

scala> for {                                      
     | JObject(thing) <- res1                     
     | JField("PlanEstimate", JDouble(points)) <- thing
     | } yield points
res3: List[Double] = List()
// Ideally res3 should be List[Option[Double]] = List(None)

thirstycrow · Accepted Answer · 2015-09-08 09:24:30Z

3

+300

scala> object OptExtractors {
     |
     |   // Define a custom extractor for using in for-comprehension.
     |   // It returns Some[Option[Double]], instead of Option[Double].
     |   object JDoubleOpt {
     |     def unapply(e: Any) = e match {
     |       case d: JDouble => Some(JDouble.unapply(d))
     |       case _ => Some(None)
     |     }
     |   }
     | }
defined object OptExtractors

scala>

scala> val j = parse("""{
     |   "FormattedID" : "the id",
     |   "PlanEstimate" : null
     | }""")
j: org.json4s.JValue = JObject(List((FormattedID,JString(the id)), (PlanEstimate,JNull)))

scala>

scala> import OptExtractors._
import OptExtractors._

scala>

scala> for {
     |   JObject(list) <- j
     |   JField("FormattedID", JString(id)) <- list
     |   JField("PlanEstimate", JDoubleOpt(points)) <- list
     | } yield (id, points)
res1: List[(String, Option[Double])] = List((the id,None))

answered Sep 8, 2015 at 9:24

thirstycrow

2,82417 silver badges19 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

WelcomeTo Over a year ago

Is it somehow possible to deal in same manner with missed JSON fields? I.e. I want to get None if field is absent or field value is null.

thirstycrow Over a year ago

I'm afraid it's not going to work if there is a missing field. In that case, I think it would be better to avoid using for-comprehension.

Mik378 · Accepted Answer · 2014-08-08 12:38:46Z

1

According to the documentation,

Any value can be optional. Field and value is completely removed when it doesn't have a value.

scala> val json = ("name" -> "joe") ~ ("age" -> (None: Option[Int]))

scala> compact(render(json))

res4: String = {"name":"joe"}

Explaining why your for comprehension doesn't yield anything.
Of course, a null value is mapped to None internally.

edited Aug 8, 2014 at 12:38

answered Aug 8, 2014 at 11:48

Mik378

22.2k15 gold badges95 silver badges191 bronze badges

1 Comment

Synesso Over a year ago

The JSON is fetched & parsed at runtime. The json4s object contains a JNull for this field, so it's not removed - it's just treated as empty by the for expression.

Oleg Rudenko · Accepted Answer · 2015-09-07 11:45:48Z

0

The last command should look like:

for {
  JObject(thing) <- res1
} yield thing.collectFirst{case JField("PlanEstimate", JDouble(points)) => points}

Or like

for {
  JObject(thing) <- res1
  points = thing.collectFirst{case JField("PlanEstimate", JDouble(p)) => p}
} yield points

answered Sep 7, 2015 at 11:45

Oleg Rudenko

7084 silver badges12 bronze badges

Comments

S.Karthik · Accepted Answer · 2015-09-08 10:39:06Z

0

What about this

 for {
      JObject(thing) <- res1      
      x = thing.find(_._1 == "PlanEstimate").flatMap(_._2.toOption.map(_.values))
     } yield x

edited Sep 8, 2015 at 10:39

answered Sep 8, 2015 at 10:24

S.Karthik

1,3899 silver badges22 bronze badges

Collectives™ on Stack Overflow

Extract Options from potentially null JSON values using for expression

4 Answers 4

2 Comments

1 Comment

Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

4 Answers 4

2 Comments

1 Comment

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related