Deleting large number of files

Question

I'm writing a script to delete large number of files in linux. The script will run every two minutes in crontab. I tried using the find /mnt/md0/capture/DCN/ -maxdepth 1 -type f -name "*.pcap" -print0 | xargs -0 ls -lt | tail -$del | awk '{print $8}'

and saving this to a variable and tried to delete using rm command, but dint run. I have found out that find . -type f -delete can be used to delete the files, but I don't know how to add this part to my command. How would I do this?

I assume want to delete all but n files (because of the tail -$del? If you could replace that with another condition that find supports (for example delete all files that are older than n minutes/hours/days old), this would be solvable with a pure find ... -delete. — Martin von Wittich
– Martin von Wittich, Commented Jan 30, 2014 at 15:31
Why are you doing this? If there are lots of garbage files, can't you inhibit their creating in the first place? If they are consumed by something else, arrange for the consumer to get rid of them when done? Use pipes to ship data around instead of files? — vonbrand
– vonbrand, Commented Jan 30, 2014 at 19:46
@vonbrand, good point. dumpcap also has provision to create a cyclic traffic dump. — Stéphane Chazelas
– Stéphane Chazelas, Commented Jan 30, 2014 at 23:22

Stéphane Chazelas · Accepted Answer · 2017-01-24 15:04:42Z

#! /bin/zsh -
del=10
rm -f /mnt/md0/capture/DCN/*.pcap(D.Om[1,$del])

That's using zsh globbing qualifiers:

D: includes hidden files (Dot files).
.: only regular files (like find's -type f)
Om: reverse Order on the age (based on modification time)
[1,$del]: only include the first $del files.

With GNU tools:

cd /mnt/md0/capture/DCN/ &&
  find . -maxdepth 1 -name '*.pcap' -type f -printf '%T@@%p\0' |
    sort -zn | sed -z "s/[^@]*@//;$del q" | xargs -r0 rm -f

find builds a NUL delimited list of filenames with a Unix timestamp prepended (like 1390682991.0859627500@./file) which is sorted by sort. sed removes the timestamp and quits after reading $del records. That's passed as arguments to rm using xargs -r0.

or:

cd /mnt/md0/capture/DCN/ &&
  find . -maxdepth 1 -name '*.pcap' -type f -printf '%T@@%p\0' |
    tr '\0\n' '\n\0' | sort -n | head -n "$del" |
    cut -d@ -f2- | tr  '\0\n' '\n\0' | xargs -r0 rm -f

Same, except that we're using cut to remove the timestamp and head to select the first $del lines. Because GNU cut and head don't support a -z to work on NUL delimited records, we use tr to swap the newline and NUL characters before and after feeding the data to them.

Mike Fairhurst · Accepted Answer · 2014-01-30 19:34:18Z

It could be done painlessly in bash by combining find -exec with a temporary file. Unfortunately this requires running find twice - once to count the files, and once again to remove all but n files. Note that $FIND | wc -l will not count properly for filenames including \n.

FIND="find /mnt/md0/capture/DCN/ -name "*.pcap" -type f -maxdepth 1"
$FIND | wc -l > delcount.tmp
$FIND -exec bash -c 'DELCOUNT=$(cat delcount.tmp) ; if [ $DELCOUNT -gt 5 ] ; then echo {}; echo $(( $DELCOUNT - 1 )) > delcount.tmp ; fi' ';'
rm delcount.tmp

Stack Exchange Network

Deleting large number of files

2 Answers 2

You must log in to answer this question.

Hot Network Questions

Deleting large number of files

2 Answers 2

You must log in to answer this question.

Related

Hot Network Questions