How do I split apart a CSV string in Ruby?

Question

I have this line as an example from a CSV file:

2412,21,"Which of the following is not found in all cells?","Curriculum","Life and Living Processes, Life Processes",,,1,0,"endofline"

I want to split it into an array. The immediate thought is to just split on commas, but some of the strings have commas in them, eg "Life and Living Processes, Life Processes", and these should stay as single elements in the array. Note also that there's two commas with nothing in between - i want to get these as empty strings.

In other words, the array i want to get is

[2412,21,"Which of the following is not found in all cells?","Curriculum","Life and Living Processes, Life Processes","","",1,0,"endofline"]

I can think of hacky ways involving eval but i'm hoping someone can come up with a clean regex to do it...

cheers, max

This is a perfect example of how not everything involved in extracting data from a string is a job for regexes. — Andy Lester
– Andy Lester, Commented Oct 14, 2010 at 14:53

user229044 · Accepted Answer · 2010-10-14 12:34:32Z

9

This is not a suitable task for regular expressions. You need a CSV parser, and Ruby has one built in:

http://ruby-doc.org/stdlib/libdoc/csv/rdoc/classes/CSV.html

And an arguably superior 3rd part library:

http://fastercsv.rubyforge.org/

answered Oct 14, 2010 at 12:34

user229044♦

241k41 gold badges347 silver badges350 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

willcodejavaforfood Over a year ago

I thought CSV could not cope with qualifiers?

willcodejavaforfood Over a year ago

FasterCSV is the default for Ruby 1.9.x, which allows you to specify a quote_char which might help in his case

glenn mcdonald Over a year ago

What are "qualifiers"? This is a stock CSV line. No need to mess with quote_chars.

Cary Swoveland Over a year ago

I agree that use of CSV methods is preferred, but that's not to say it can't be done with a regex.

steenslag · Accepted Answer · 2010-10-14 14:49:41Z

4

str=<<EOF
2412,21,"Which of the following is not found in all cells?","Curriculum","Life and Living Processes, Life Processes",,,1,0,"endofline"
EOF
require 'csv' # built in

p CSV.parse(str)
# That's it! However, empty fields appear as nil.
# Makes sense to me, but if you insist on empty strings then do something like:
parser = CSV.new(str)
parser.convert{|field| field.nil? ? "" : field}
p parser.readlines

answered Oct 14, 2010 at 14:49

steenslag

80.2k16 gold badges144 silver badges174 bronze badges

1 Comment

Max Williams Over a year ago

Thanks Steenslag, this is perfect. As it happens i don't mind the empty fields coming through as nil. Cheers, max

Dave · Accepted Answer · 2010-10-14 12:40:58Z

2

EDIT: I failed to read the Ruby tag. The good news is, the guide will explain the theory behind building this, even if the language specifics aren't right. Sorry.

Here is a fantastic guide to doing this:

http://knab.ws/blog/index.php?/archives/10-CSV-file-parser-and-writer-in-C-Part-2.html

and the csv writer is here:

http://knab.ws/blog/index.php?/archives/3-CSV-file-parser-and-writer-in-C-Part-1.html

These examples cover the case of having a quoted literal in a csv (which may or may not contain a comma).

answered Oct 14, 2010 at 12:40

Dave

7,0252 gold badges34 silver badges35 bronze badges

Comments

ghostdog74 · Accepted Answer · 2010-10-14 13:47:52Z

2

text=<<EOF
2412,21,"Which of the following is not found in all cells?","Curriculum","Life and Living Processes, Life Processes",,,1,0,"endofline"
EOF
x=[]
text.chomp.split("\042").each_with_index do |y,i|
  i%2==0 ?  x<< y.split(",") : x<<y
end
print x.flatten

output

$ ruby test.rb
["2412", "21", "Which of the following is not found in all cells?", "Curriculum", "Life and Living Processes, Life Processes", "", "", "", "1", "0", "endofline"]

answered Oct 14, 2010 at 13:47

ghostdog74

346k62 gold badges264 silver badges349 bronze badges

Comments

poseid · Accepted Answer · 2010-10-14 13:15:47Z

1

This morning I stumbled across a CSV Table Importer project for Ruby-on-Rails. Eventually you will find the code helpful:

Github TableImporter

answered Oct 14, 2010 at 13:15

poseid

7,23610 gold badges52 silver badges79 bronze badges

Comments

Cary Swoveland · Accepted Answer · 2020-07-09 07:01:56Z

My preference is @steenstag's solution, but an alternative is to use String#scan with the following regular expression.

r = /(?<![^,])(?:(?!")[^,\n]*(?<!")|"[^"\n]*")(?![^,])/

If the variable str holds the string given in the example, we obtain:

puts str.scan r

displays

2412
21
"Which of the following is not found in all cells?"
"Curriculum"
"Life and Living Processes, Life Processes"


1
0
"endofline"

Start your engine!

See also regex101 which provides a detailed explanation of each token of the regex. (Move your cursor across the regex.)

Ruby's regex engine performs the following operations.

(?<![^,]) : negative lookbehind assert current location is not preceded
            by a character other than a comma
(?:       : begin non-capture group
  (?!")   : negative lookahead asserts next char is not a double-quote
  [^,\n]* : match 0+ chars other than a comma and newline
  (?<!")  : negative lookbehind asserts preceding character is not a
            double-quote
  |       : or
  "       : match double-quote
  [^"\n]* : match 0+ chars other than double-quote and newline
  "       : match double-quote
)         : end of non-capture group
(?![^,])  : negative lookahead asserts current location is not followed
            by a character other than a comma

Note that (?<![^,]) is the same as (?<=,|^) and (?![^,]) is the same as (?=^|,).

Collectives™ on Stack Overflow

How do I split apart a CSV string in Ruby?

6 Answers 6

4 Comments

1 Comment

Comments

Comments

Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

6 Answers 6

4 Comments

1 Comment

Comments

Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related