Python CSV writer to quote strings with extra spaces

Question

My data looks something like this:

data = [
  [" trailing space", 19, 100],
  [" ", 19, 100],
]

writer = csv.writer(csv_filename, delimiter=',', quotechar='"', quoting=csv.QUOTE_MINIMAL)

Output

 trailing space,19,100
 ,19,100

What I want

" trailing space",19,100
" ",19,100

Python default CSV writer has the option to "QUOTE_MINIMAL" but it doesn't include quoting strings with extra spaces in it. In my case, those empty spaces are actually critical, but without quoting, the reader (like libre-office) strips the spaces if not quoted.

Is there any built in options or quick cheap way to tell the writer to quote empty strings with spaces?

Also, "QUOTE_NONNUMERIC" is quoting too much. The actual data is huge ( few hundred megabytes with 60% - 70% of strings). It may sounds silly, but I'm trying to reduce the csv size by minimizing the quotes.

ldonoso · Accepted Answer · 2019-03-05 09:50:26Z

5

It's a bit of a hack but one way of achiving this could be

df.to_csv(quoting=csv.QUOTE_MINIMAL, escapechar=' ')

It's not document but QUOTE_MINIMAL seems to quote fields containing escapechar although it has no effect (as quoting is not NONE and doublequote is True by default)

answered Mar 5, 2019 at 9:50

ldonoso

1712 silver badges6 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

DSM · Accepted Answer · 2016-04-20 03:30:13Z

2

Why not just use QUOTE_NONNUMERIC? That'll quote all strings, not just those with spaces, but it'll certainly quote those too.

with open("quote.csv", "w", newline="") as fp:
    writer = csv.writer(fp, quoting=csv.QUOTE_NONNUMERIC)
    writer.writerows(data)

gives me

(3.5.1) dsm@notebook:~/coding$ cat quote.csv 
" leading space",19,100
" ",19,100

answered Apr 20, 2016 at 3:30

DSM

355k67 gold badges606 silver badges504 bronze badges

3 Comments

Sufendy Over a year ago

It will be quoting too much. Not sure how much the difference the size will be, but my csv output can be quite big (updated the question with more details). And quoting can be unnecessary sometimes.

DSM Over a year ago

The change in file size due to removing some quotation marks will be trivial compared to pretty much anything else you could do (such as compressing the file and reading it through a decompression stream). It sounds like you've got a major case of the XY problem.

Sufendy Over a year ago

Good point. I don't know, I guess you're right. But it seems like the "QUOTE_MINIMAL" could have been more flexible. Even with custom quoting way from the other answer is too hacky.

Community · Accepted Answer · 2017-05-23 12:09:24Z

0

Have you tried csv writer in Python with custom quoting

Though make sure you know what you are quoting and take to manually escape stuff

edited May 23, 2017 at 12:09

CommunityBot

11 silver badge

answered Apr 20, 2016 at 5:12

karina

8055 silver badges10 bronze badges

Comments

Daniel Lee · Accepted Answer · 2016-08-10 14:58:45Z

0

Try removing the quoting altogether. Will keep all quote characters as required.

writer = csv.writer(csv_filename, delimiter=',', quoting=csv.QUOTE_NONE)

answered Aug 10, 2016 at 14:58

Daniel Lee

8,1974 gold badges31 silver badges47 bronze badges

1 Comment

Exploring Over a year ago

does not work. I get _csv.Error: need to escape, but no escapechar set

Collectives™ on Stack Overflow

Python CSV writer to quote strings with extra spaces

4 Answers 4

Comments

3 Comments

Comments

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

4 Answers 4

Comments

3 Comments

Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related