bash, list filenames, array

Question

I have a dir structure like

$ ls /comp/drive/
2009  2010  2011  2012  2013  2014

$ ls 2009
01  02  03  04  05  06  07  09  10  11  12

$ ls 2013
01  02  04  05  06  08  09  10  12

$ ls 2013/04/*.nc
file4.nc file44.nc file45.nc file49.nc

There are dirs like years and each year there are few months dirs and inside are .nc files.

What I want to do is get the array of filenames provided start and end years/months.

e.g. sYear=2011; eYear=2013; sMonth=03; eMonth=08

So, I want to get the array of all filenames from year 2011/03 to 2013/08 only without going inside the dirs.

Any bash trick?

Charles Duffy · Accepted Answer · 2015-01-04 21:27:44Z

4

sYear=2011; eYear=2013; sMonth=03; eMonth=08

# prevent bugs from interpreting numbers as hex
sMonth=$(( 10#$sMonth ))
eMonth=$(( 10#$eMonth ))

files=( )
for (( curYear=sYear; curYear <= eYear; curYear++ )); do
  # include only months after sMonth
  for monthDir in "$curYear"/*/; do
    [[ -e $monthDir ]] || continue # ignore years that don't exist
    curMonth=${monthDir##*/}
    (( curMonth )) || continue     # ignore non-numeric directory names
    (( curYear == sYear )) && (( 10#$curMonth < sMonth )) && continue
    (( curYear == eYear )) && (( 10#$curMonth > eMonth )) && continue
    files+=( "$monthDir"/*.nc )
  done
done

printf '%q\n' "${files[@]}"

edited Jan 4, 2015 at 21:27

answered Jan 3, 2015 at 18:52

Charles Duffy

299k43 gold badges441 silver badges497 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

user3652962 Over a year ago

Thanks Charles, I get this sMonth=(( 10#sMonth )) -bash: syntax error near unexpected token `(' Also, it would be better to have the full paths of the filenames in the final ${files[@]}. Sorry to bother again.

Charles Duffy Over a year ago

Needs to be sMonth=$((10#sMonth)) or ((sMonth = 10#sMonth)). Will answer more completely when at a real console.

Charles Duffy Over a year ago

@user3652962, by "full" paths do you mean absolute? The only change necessary for that is to change "$curYear"/*/ to "$PWD/$curYear"/*/.

Intelligent Shade of Blue · Accepted Answer · 2015-01-04 15:04:02Z

2

Try this:

sYear=2011
sMonth=03

eYear=2013
eMonth=08

shopt -s nullglob
declare -a files

for year in *; do
    (( ${year} < ${sYear} || ${year} > ${eYear} )) && continue

    for year_month in ${year}/*; do

        month=${year_month##*/}
        (( ${year} == ${sYear} && ${month##0} < ${sMonth##0} )) && continue;
        (( ${year} == ${eYear} && ${month##0} > ${eMonth##0} )) && continue;

        files+=(${year_month}/*.nc)
    done
done

echo "${files[@]}"
# printf "$(pwd)/%q\n" "${files[@]}" # for full path

edited Jan 4, 2015 at 15:04

answered Jan 3, 2015 at 18:52

Intelligent Shade of Blue

6,48828 silver badges56 bronze badges

9 Comments

Charles Duffy Over a year ago

First -- see mywiki.wooledge.org/ParsingLs. Second, 08 is treated as octal, not decimal, so this is buggy as-given with months numbered above 7 but below 10. Third, use of unquoted ${files[*]} is almost always the Wrong Thing -- if you had filenames with spaces, they'd be broken out into separate arguments, if you had a file with an * in its name surrounded by spaces, that glob would actually be expanded. Quoted "${files[@]}" is the Right Thing.

Charles Duffy Over a year ago

Not quite right here either -- the glob expression can't be in the quotes, even though the expansions should be. You want files+=( "${year}/${month}/"*.nc ), or somesuch. (The curly braces are strictly optional in this context).

Intelligent Shade of Blue Over a year ago

@CharlesDuffy, I just threw a quick example. Didn't actually test it. Thanks for catching the errors -- I fixed them.

Charles Duffy Over a year ago

nod. I think you posted your answer about 6 seconds before mine, and they certainly have a lot in common. Parallel development is funny like that.

Intelligent Shade of Blue Over a year ago

@user3652962, I tweaked it to return paths relative to the current directory. If you want the full path, uncomment the last line with printf

|

Collectives™ on Stack Overflow

bash, list filenames, array

2 Answers 2

3 Comments

9 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

3 Comments

9 Comments

Your Answer

Sign up or log in

Post as a guest

Related