encoding issue with CSV.parse

Question

My goal is to upload a file containing rows of firstname and lastname, parse it and create Person model in db for each row.

I do the following and it works fine

file = CSV.parse(the_file_to_parse)
file.each do |row|
  person = Person.new(:firstname => row[0], :lastname => row[1])
  person.save
end

until my file contains accents (french words), I get

Encoding::UndefinedConversionError: "\xC3" from ASCII-8BIT to UTF-8:
INSERT INTO "people" ("created_at", "firstname", "lastname",
"updated_at") VALUES (?, ?, ?, ?)

What is the best way to deal with this encoding issue?

Michael Slade · Accepted Answer · 2012-04-08 13:41:14Z

2

You need to open the csv file with the right encoding. For example:

require 'csv'
require 'pp'

encoding = "ISO-8859-1"

csv = CSV.open "names.csv", "rb:#{encoding}"
csv.each do |line|
    puts "#{line[0]} #{line[1]}"
end

Here's a hint: it's probably not UTF-8.

The list of encodings that your ruby supports can be viewed with this command in irb:

puts Encoding.list.map(&:to_s).sort

edited Apr 8, 2012 at 13:41

answered Apr 7, 2012 at 12:38

Michael Slade

13.9k2 gold badges41 silver badges44 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

kwaigon Over a year ago

I've got ArgumentError: 'mode' must be 'r', 'rb', 'w', or 'wb'

gayavat · Accepted Answer · 2012-04-07 11:23:44Z

0

try to set

# encoding: utf-8

on the top of ruby file. or

export RUBYOPT=-Ku

before rails s

answered Apr 7, 2012 at 11:23

gayavat

19.5k11 gold badges49 silver badges60 bronze badges

Comments

maerzbow · Accepted Answer · 2012-07-28 10:37:01Z

0

If you have no clue about the encoding of the input file you might try ensure_encoding.

It will try to guess the input encoding and try (does not work for all encodings) to convert it to your preferred target encoding.

answered Jul 28, 2012 at 10:37

maerzbow

1951 gold badge2 silver badges9 bronze badges

Collectives™ on Stack Overflow

encoding issue with CSV.parse

3 Answers 3

1 Comment

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

1 Comment

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related