Combining csv with Python in new line

Question

Iam trying to concatenate multiple csv but the output file has all the data in a single row. How to add a new line character while giving a output. Here is a sample code of mine

from os import chdir
from glob import glob
import pandas as pdlib

# Produce a single CSV after combining all files
def produceOneCSV(list_of_files, file_out):
   # Consolidate all CSV files into one object
   result_obj = pdlib.concat([pdlib.read_csv(file) for file in list_of_files], sort=False)
   # Convert the above object into a csv file and export

   result_obj.to_csv(file_out, index=False, encoding="utf-8")

# Move to the path that holds our CSV files
csv_file_path = 'Path_for_all_the_csv/'
chdir(csv_file_path)

# List all CSV files in the working dir
file_pattern = ".csv"
list_of_files = [file for file in glob('*.csv')]
print(list_of_files)

file_out = "ConsolidateOutput.csv"
produceOneCSV(list_of_files, file_out)

I followed the document https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.concat.html. But didnt worked for me. Is there any change in document?

Does this answer your question? Import multiple csv files into pandas and concatenate into one DataFrame — Ron
– Ron, Commented Jun 2, 2020 at 12:29
If you want to create a consolidated csv, why are you involving pandas in this? Can't you just read each file one by one and write it out in to a larger file? — najeem
– najeem, Commented Jun 2, 2020 at 12:32

najeem · Accepted Answer · 2020-06-02 12:36:59Z

2

Do you need pandas to concatenate a bunch of csvs? If you're using linux, you can do it with one command.

$ cat *.csv > consolidated.csv

And if you want to use python

with open("consolidated.json", "w") as f:
    for ff in glob.glob("*.csv"):
        with open(ff) as fff:
            f.write(fff.read())

answered Jun 2, 2020 at 12:36

najeem

1,92116 silver badges32 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Ankzious Over a year ago

Well yes you are correct. I went too deep in thinking. Thanks Najeem :)

Collectives™ on Stack Overflow

Combining csv with Python in new line

1 Answer 1

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related