Problem to load .csv files into Apache Cassandra with Python

Question

I'm trying to load a .csv file with Python into Apache Cassandra database. The command "COPY" integrated with session.execute seems don't work. It gives an unexpected indent in correspondance of =',' but...I red something about and I found that the command COPY in this way is not supported.

In this script time_test and p are two float variables

from cassandra.cluster import Cluster

cluster = Cluster()

session = cluster.connect('myKEYSPACE')


rows = session.execute('COPY table_test (time_test, p) 
                        from'/home/mypc/Desktop/testfile.csv' with delimiter=',' and header=true;
                       ')
                                                                     

print('DONE')

Thank you for help!

Alex Ott · Accepted Answer · 2020-11-03 16:49:11Z

1

Main problem here is that COPY is not a CQL command, but a cqlsh command, so it couldn't be executed via session.execute.

I recommend to use DSBulk to load data into Cassandra - it's very flexible, performant, and doesn't require programming. For simplest case, when you have direct mapping of columns in header of CSV file into column names in database, then the command-line will be very simple:

dsbulk load -url file.csv -k keyspace -t table -header true

There is a series of blog posts about DSBulk that covers a lot of topics:

answered Nov 3, 2020 at 16:49

Alex Ott

88.1k10 gold badges110 silver badges157 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

wundolab Over a year ago

thank you...but...I need to use datastax? Cause I'm using APC on my ubuntu

Alex Ott Over a year ago

No, it should work with any supported Cassandra version

Alex Ott Over a year ago

and no, it's not only the one way, but it's the better way than manually written code. P.S. btw, COPY isn't very scalable & buggy - that's was one of the reasons for DSBulk creation

Collectives™ on Stack Overflow

Problem to load .csv files into Apache Cassandra with Python

1 Answer 1

3 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

3 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related