Fast and safe conversion from string to numeric types

Question

What would be a fast a safe way to convert a String to a numeric type, while providing a default value when the conversion fails ?

I tried using the usually recommended way, i.e. using Exceptions:

implicit class StringConversion(val s: String) {

  private def toTypeOrElse[T](convert: String=>T, defaultVal: T) = try {
    convert(s)
  } catch {
    case _: NumberFormatException => defaultVal
  }

  def toShortOrElse(defaultVal: Short = 0) = toTypeOrElse[Short](_.toShort, defaultVal)
  def toByteOrElse(defaultVal: Byte = 0) = toTypeOrElse[Byte](_.toByte, defaultVal)
  def toIntOrElse(defaultVal: Int = 0) = toTypeOrElse[Int](_.toInt, defaultVal)
  def toDoubleOrElse(defaultVal: Double = 0D) = toTypeOrElse[Double](_.toDouble, defaultVal)
  def toLongOrElse(defaultVal: Long = 0L) = toTypeOrElse[Long](_.toLong, defaultVal)
  def toFloatOrElse(defaultVal: Float = 0F) = toTypeOrElse[Float](_.toFloat, defaultVal)
}

Using this utility class, I can now easily convert any String to a given numeric type, and provide a default value in case the String is not representing correctly the numeric type:

scala> "123".toIntOrElse()
res1: Int = 123
scala> "abc".toIntOrElse(-1)
res2: Int = -1
scala> "abc".toIntOrElse()
res3: Int = 0
scala> "3.14159".toDoubleOrElse()
res4: Double = 3.14159
...

While it works beautifully, this approach does not seem to scale well, probably because of the Exceptions mechanism:

scala> for (i<-1 to 10000000) "1234".toIntOrElse()

takes roughly 1 second to execute whereas

scala> for (i<-1 to 10000000) "abcd".toIntOrElse()

takes roughly 1 minute!

I guess another approach would be to avoid relying on exceptions being triggered by the toInt, toDouble, ... methods.

Could this be achieved by checking if a String "is of the given type" ? One could of course iterate through the String characters and check that they are digits (see e.g. this example), but then what about the other numeric formats (double, float, hex, octal, ...) ?

Regex is probably your best way to go here if you completely want to avoid the overhead of the try/catch semantic. You just need to come up with regexes for each of the possible numeric types you want to be able to convert from. But honestly, that is probably a premature optimization. How fast does this code need to be? How often is it hit? How often will it get invalid numbers this hitting the catch block? These are questions you need to ask yourself before optimizing as the code gets a bit more complex. — cmbaxter
– cmbaxter, Commented May 8, 2014 at 12:39
@cmbaxter I agree with you but I'm using this in a Big Data context, where I parse huge CSV files (Billions of rows), so it matters. — borck
– borck, Commented May 8, 2014 at 14:19
Fair enough. Then I would go with Regex to vet the string first. Will be much faster. — cmbaxter
– cmbaxter, Commented May 8, 2014 at 14:22
@pbr consider updated answer with enriched characters that may belong to a numeric value, yet to avoid performance penalty, no specialised parsing done. This may prove helpful for filtering out most non numeric values. — elm
– elm, Commented May 8, 2014 at 17:14
@pbr consider also stackoverflow.com/a/16699049/3189923 (and Apache Commons Lang). — elm
– elm, Commented May 8, 2014 at 20:53

Pratik Khadloya · Accepted Answer · 2015-10-13 21:39:14Z

1

As a first approach, filter out those input strings that do not contain any digit

private def toTypeOrElse[T](convert: String=>T, defaultVal: T) = try {
  if (s.contains("[0-9]")) convert(s) {
    else defaultVal
  } catch {
    case _: NumberFormatException => defaultVal
  }
}

Update

Enriched set of characters that may occur in a numeric value, yet no order of occurrence or limits in repetition considered,

private def toTypeOrElse[T](convert: String=>T, defaultVal: T) = try {
    if (s matches "[\\+\\-0-9.e]+") convert(s)
    else defaultVal
  } catch {
    case _: NumberFormatException => defaultVal
  }
}

edited Oct 13, 2015 at 21:39

Pratik Khadloya

12.9k12 gold badges86 silver badges106 bronze badges

answered May 8, 2014 at 12:29

elm

20.5k15 gold badges72 silver badges116 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Kigyo Over a year ago

Why not filter out the ones, that don't contain only digits or a -?

Collectives™ on Stack Overflow

Fast and safe conversion from string to numeric types

1 Answer 1

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related