How to copy files using subprocess in python?

Question

I have a list of files:

file_list=['test1.txt','test2.txt','test3.txt']

I want to find and copy these files to a destination folder. I have the following code:

for files in file_list:
    subprocess.call(["find", "test_folder/",
                    "-iname", files,
                    "-exec", "cp", "{}",
                    "dest_folder/",
                    "\;"])

But, i keep getting the error:

find: missing argument to `-exec

The shell command looks something like this:

$find test_folder/ -iname 'test1.txt' -exec cp {} dest_folder/ \;

Anything i am doing wrong?

jfs · Accepted Answer · 2014-03-22 06:26:31Z

2

You don't need to escape the arguments; subprocess module calls find command directly without the shell. Replace "\;" with ";" and your command will work as is.

You could combine the search into a single command:

from subprocess import call

expr = [a for file in file_list for a in ['-iname', file, '-o']]
expr.pop() # remove last `-o`
rc = call(["find", "test_folder/", "("] + expr + [")", "-exec", 
           "cp", "-t", "dest_folder/", "--", "{}", "+"])

You could also combine expr list into a single -iregex argument if desired.

You don't need find command; you could implement the copying in pure Python using os.walk, re.match, and shutil.copy:

import os
import re
import shutil

found = re.compile('(?i)^(?:%s)$' % '|'.join(map(re.escape, file_list))).match
for root, dirs, files in os.walk('test_folder/'):
    for filename in files:
        if found(filename):
            shutil.copy(os.path.join(root, filename), "dest_folder/")

edited Mar 22, 2014 at 6:26

answered Mar 22, 2014 at 6:20

jfs

417k210 gold badges1k silver badges1.7k bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

msakya Over a year ago

yeah you are right on the first one, I got it to working after an inherent bug not related to the code. Is latter option any better than former? Thanks!

jfs Over a year ago

@msakya: the single find command should be faster: it copies several files at a time. The pure Python solution should be more portable (it doesn't require find, cp commands).

Community · Accepted Answer · 2017-05-23 11:44:27Z

1

You don't need to escape semi-colon. Here's what is working for me:

import shlex
import subprocess

file_list = ['test1.txt','test2.txt','test3.txt']

cmd = 'find test_folder -iname %s -exec cp {} dest_folder ;'
for files in file_list:
    subprocess.Popen(shlex.split(cmd % files))

Also see:

Hope that helps.

edited May 23, 2017 at 11:44

CommunityBot

11 silver badge

answered Mar 22, 2014 at 2:15

alecxe

476k127 gold badges1.1k silver badges1.2k bronze badges

11 Comments

msakya Over a year ago

I dont get any error now, but the destination folder doesnt have the copied files. Is there a way to look at whats being run? :)

msakya Over a year ago

I dont get any error now, but the destination folder doesnt have the copied files. Is there a way to print the subprocess being run? :)

alecxe Over a year ago

@msakya hm, it's working for me. You may find the way to log the commands here: stackoverflow.com/questions/14836947/…

alecxe Over a year ago

@msakya does adding shell=True argument to Popen make any difference?

msakya Over a year ago

This is weird, I am not really sure whats going on. putting shell=True didnt help.

|

Collectives™ on Stack Overflow

How to copy files using subprocess in python?

2 Answers 2

2 Comments

11 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

2 Comments

11 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related