3

Having some trouble here. I want to capture output from ls command into variable. Then later use that variable and count the number of lines in it. I've tried a few variations

This works, but then if there are NO .txt files, it says the count is 1:

testVar=`ls -1 *.txt`
count=`wc -l <<< $testVar`
echo '$count'

This works for when there are no .txt files, but the count comes up short by 1 when there are .txt files:

testVar=`ls -1 *.txt`
count=`printf '$testVar' | wc -l`
echo '$count'

This variation also says the count is 1 when NO .txt files exist:

testVar=`ls -1 *.txt`
count=`echo '$testVar' | wc -l`
echo '$count'

Edit: I should mention this is korn shell.

4 Answers 4

5

The correct approach is to use an array.

# Use ~(N) so that if the match fails, the array is empty instead
# of containing the pattern itself as the single entry.
testVar=( ~(N)*.txt )
count=${#testVar[@]}
Sign up to request clarification or add additional context in comments.

4 Comments

Can you explain the ~(N)* piece a little? What is this notation called? I have more complex file matching logic than *.txt to implement.
I think it's just called a sub-pattern; it's documented under "Filename Generation" in the man page, at the end of the section just before the section on "Quoting". (The documentation refers to it as making an unmatched pattern expand to the empty string, which I don't think is quite accurate. It appears to be identical to the bash shell option nullglob, which causes non-matching patterns to be simply ignored.)
Without the ~(N), the array would contain a single element *.txt in the event of no matching files, and ${#testVar[@]} would expand to 1, not 0.
@GillesQuenot shopt is a bash command, not part of ksh. (Which I discovered as I wrote this answer, resulting in me learning about the special ~-patterns.)
2

This little question actually includes the result of three standard shell gotchas (both bash and korn shell):

  1. Here-strings (<<<...) have a newline added to them if they don't end with a newline. That makes it impossible to send a completely empty input to a command with a here-string.

  2. All trailing newlines are removed from the output of a command used in command substitution (cmd or preferably $(cmd)). So you have no way to know how many blank lines were at the end of the output.

  3. (Not really a shell gotcha, but it comes up a lot). wc -l counts the number of newline characters, not the number of lines. So the last "line" is not counted if it is not terminated with a newline. (A non-empty file which does not end with a newline character is not a Posix-conformant text file. So weird results like this are not unexpected.)

So, when you do:

var=$(cmd)
utility <<<"$var"

The command substitution in the first line removes all trailing newlines, and then the here-string expansion in the second line puts exactly one trailing newline back. That converts an empty output to a single blank line, and otherwise removes blank lines from the end of the output.

So utility is wc -l, then you will get the correct count unless the output was empty, in which case it will be 1 instead of 0.

On the other hand, with

var=$(cmd)
printf %s "$cmd" | utility

The trailing newline(s) are removed, as before, by the command substitution, so the printf leaves the last line (if any) unterminated. Now if utility is wc -l, you'll end up with 0 if the output was empty, but for non-empty files, the count will not include the last line of the output.

One possible shell-independent work-around is to use the second option, but with grep '' as a filter:

var=$(cmd)
printf %s "${var}" | grep '' | utility

The empty pattern '' will match every line, and grep always terminates every line of output. (Of course, this still won't count blank lines at the end of the output.)


Having said all that, it is always a bad idea to try to parse the output of ls, even just to count the number of files. (A filename might include a newline character, for example.) So it would be better to use a glob expansion combined with some shell-specific way of counting the number of objects in the glob expansion (and some other shell-specific way of detecting when no file matches the glob).

Comments

1

I was going to suggest this, which is a construct I've used in bash:

f=($(</path/to/file))
echo ${#f[@]}

To handle multiple files, you'd just .. add files.

f=($(</path/to/file))
f=+($(</path/to/otherfile))

or

f=($(</path/to/file) $(</path/to/otherfile))

To handle lots of files, you could loop:

f=()
for file in *.txt; do
    f+=($(<$file))
done

Then I saw chepner's response, which I gather is more korn-y than mine.

NOTE: loops are better than parsing ls.

Comments

0

You can also use like this:

#!/bin/bash

testVar=`ls -1 *.txt`

if [ -z "$testVar" ]; then
        # Empty
        count=0
else
        # Not Empty
        count=`wc -l <<< "$testVar"`
fi

echo "Count : $count"

4 Comments

That is helpful, but I would like to understand why this is needed. What is the wc picking up on if the output is empty?
@user3349673, If the input is empty while passing as HERESTRING, wc is counting as one line. You can check it by using command: wc -l <<< "".
@user3349673, Unless it is EOF ( Ctrl + D), wc increases it's line count.
@user3349673: sat's explanation is not quite accurate, but the full explanation is too long for a comment so I made it an answer.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.