Parse CSV and add only matching rows to List functionally in Scala

Question

I am reading csv scala. Person is a case class

Case class Person(name, address)

    def getData(path:String,existingName) : List[Person] = {

      Source.fromFile(“my_file_path”).getLines.drop(1).map(l => {
        val data = l.split("|", -1).map(_.trim).toList
        val personName  = data(0)

        if(personName.equalsIgnoreCase(existingName)) {
          val address=data(1)
          Person(personName,address)
           //here I want to add to list

        }
        else
          Nil
       ///here return empty list of zero length


     }).toList()

}

I want to achieve this functionally in scala.

jwvh · Accepted Answer · 2020-03-29 04:18:08Z

1

Here's the basic approach to what I think you're trying to do.

case class Person(name:String, address:String)

def getData(path:String, existingName:String) :List[Person] = {
  val recordPattern = raw"\s*(?i)($existingName)\s*\|\s*(.*)".r.unanchored

  io.Source.fromFile(path).getLines.drop(1).collect {
    case recordPattern(name,addr) => Person(name, addr.trim)
  }.toList
}

This doesn't close the file reader or report the error if the file can't be opened, which you really should do, but we'll leave that for a different day.

update: added file closing and error handling via Using (Scala 2.13)

import scala.util.{Using, Try}

case class Person(name:String, address:String)

def getData(path:String, existingName:String) :Try[List[Person]] =
  Using(io.Source.fromFile(path)){ file =>
    val recordPattern = raw"\s*(?i)($existingName)\s*\|\s*([^|]*)".r

    file.getLines.drop(1).collect {
      case recordPattern(name,addr) => Person(name, addr.trim)
    }.toList
  }

updated update

OK. Here's a version that:

reports the error if the file can't be opened
closes the file after it's been opened and read
ignores unwanted spaces and quote marks
is pre-2.13 compiler friendly

import scala.util.Try

case class Person(name:String, address:String)

def getData(path:String, existingName:String) :List[Person] = {
  val recordPattern =
    raw"""[\s"]*(?i)($existingName)["\s]*\|[\s"]*([^"|]*)*.""".r

  val file = Try(io.Source.fromFile(path))
  val res = file.fold(
    err => {println(err); List.empty[Person]},
    _.getLines.drop(1).collect {
      case recordPattern(name,addr) => Person(name, addr.trim)
    }.toList)
  file.map(_.close)
  res
}

And here's how the regex works:

[\s"]* there might be spaces or quote marks
(?i) matching is case-insensitive
($existingName) match and capture this string (1st capture group)
["\s]* there might be spaces or quote marks
\| there will be a bar character
[\s"]* there might be spaces or quote marks
([^"|]*) match and capture everything that isn't quote or bar
.* ignore anything that might come thereafter

edited Mar 29, 2020 at 4:18

answered Mar 28, 2020 at 6:59

jwvh

51.3k7 gold badges42 silver badges70 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

jwvh Over a year ago

A clean and efficient way to extract both name and address while doing a case-insensitive comparison at the same time.

HDev007 Over a year ago

Also pls suggest how to close the io conn.

HDev007 Over a year ago

regex not getting compile

HDev007 Over a year ago

There was some error related to the IDE..cant find declaration..can u pls explain the regex.i my csv string are in double quotes i am removing them via replaceall..... sample :- "SomeName"|"SomeAddress"| ....in future i may more col.

pedrorijo91 · Accepted Answer · 2020-03-28 11:33:27Z

1

you were not very clear on what was the problem on your approach, but this should do the trick (very close to what you have)

 def getData(path:String, existingName: String) : List[Person] = {

    val source = Source.fromFile("my_file_path")
    val lst = source.getLines.drop(1).flatMap(l => {
      val data = l.split("|", -1).map(_.trim).toList
      val personName = data.head

      if (personName.equalsIgnoreCase(existingName)) {
        val address = data(1)
        Option(Person(personName, address))
      }
      else
        Option.empty
    }).toList

    source.close()

    lst
  }

we read the file line per line, for each line we extract the personName from the first csv field, and if it's the one we are looking for we return an (Option) Person, otherwise none (Option.empty). By doing flatmap we discard the empty options (just to avoid using nils)

edited Mar 28, 2020 at 11:33

answered Mar 28, 2020 at 9:24

pedrorijo91

7,9559 gold badges48 silver badges89 bronze badges

5 Comments

HDev007 Over a year ago

Ok..with option it is working..can u suggest me on how to close connection after reading.

pedrorijo91 Over a year ago

updated the answer. but please look for yourself... if you google scala Source close it should be the first result...

HDev007 Over a year ago

yeah got it..i was looking for something equivalent of finally block in scala.I did not asked correctly.Thank you for the response.

pedrorijo91 Over a year ago

in java you have try with resources. In scala I don't think there's something equivalent. What you can is to create an Higher Order Function that receives a Source and a block. It will execute the block and close the Source at the end. Basically reproducing the java try-with-resource. See medium.com/@dkomanov/scala-try-with-resources-735baad0fd7d on tthe 'try-with-resources in Scala' section

HDev007 Over a year ago

I am getting output as List(None,None,......)....i wanted its length to be zero.

Collectives™ on Stack Overflow

Parse CSV and add only matching rows to List functionally in Scala

2 Answers 2

4 Comments

5 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

4 Comments

5 Comments

Your Answer

Sign up or log in

Post as a guest

Related