Why Regex doesn't work in bash/ksh while it works fine in command line

Question

I want to exclude any files ending with '.ses' or files with no extension using the following regex pattern. It works fine in command line but not in a shell (bash/ksh).

Regex pattern: "\.(?!ses\$)([^.]+\$)"

File name examples:

"/test/path/test file with spaces.__1" (expected true)
"/test/path/test file with spaces.ses" (expected false)
"/test/path/test file with spaces" (expected false)
"/test/path/test file with spaces.txt" (expected true)

FILE_NAME="/test/path/test file with spaces.__1"

PATTERN_STR="\.(?!ses\$)([^.]+\$)"

if [[ "${FILE_NAME}" =~ ${PATTERN_STR} ]]; then

        Match_Result="true"
else

        Match_Result="false"

fi

echo $Match_Result

it returns "true" but "false" in the shell. Anyone knows why?

Which shell are you using? Is your script and your cli using the same shell? — user9706
– user9706, Commented Nov 23, 2021 at 3:25
I tested your script in my shell and I got an error for using double quotes. The problem is the "!" sign in your string, this must be in single quotes. Try changing PATTERN_STR="\\\.(?!ses\$)([^.]+\$)" to PATTERN_STR='\\\.(?!ses\$)([^.]+\$)'. What I did was change the double quotes " " to single quotes ' '. — Edgar Magallon
– Edgar Magallon, Commented Nov 23, 2021 at 3:29
(?!...) is PCRE. Bash implements "extended regular expressions" like grep -E. What was you "command line" command you were using? — glenn jackman
– glenn jackman, Commented Nov 23, 2021 at 3:58
@Gill : I don't know what kind of regexp is accepted by ksh, but it does not look like a meaningful bash regexp. Can you demonstrate that it would work inside your script? For instance, in your question I don't see a complete script, nor do I see how you have invoked the script. — user1934428
– user1934428, Commented Nov 23, 2021 at 5:58
The correct regex would look like this: [[ "${FILE_NAME##*/}" =~ \.ses$|^[^.]+$ ]] && exclude=yes. You need to strip any leading path. But I would generally use a case statement, like Allan's answer. It's a lot clearer. — dan
– dan, Commented Nov 23, 2021 at 6:21

score 1 · Accepted Answer · 2021-11-23 22:18:22Z

1

I would just use a case statement with suitable globs:

case "${FILE_NAME##*/}" in
*.ses)
    Match_Result=false
    ;;
*.*)
    Match_Result=true
    ;;    
*)
    Match_Result=false
    ;;
esac

Consider using an array instead of doing whitespace gymnastics.

edited Nov 23, 2021 at 22:18

answered Nov 23, 2021 at 3:23

user9706

Sign up to request clarification or add additional context in comments.

5 Comments

user9706 Over a year ago

@JohnKugelman fixed with a hack.

dan Over a year ago

Good answer, but you should strip any leading path. FILE_NAME was a full path, and OP is interested in file names only. ${FILE_NAME##*/} or $(basename "$FILE_NAME")

user9706 Over a year ago

@dan updated as suggested.

Gill Over a year ago

Thanks @AllanWind for alternative solution. Is it possible to construct case statements using input parameters like using eval?

user9706 Over a year ago

Yes, both case word and the pattern are pattern expanded. See also shopt extglob which enables !(pattern-list) to matches anything except one of the given patterns.

Wiktor Stribiżew · Accepted Answer · 2021-11-25 07:53:17Z

0

You can reverse your logic and fail all strings that contain .ses at the end or do not contain a dot after the last /.

Then, you can use this script:

#!/bin/bash
declare -a arr=("/test/path/test file with spaces.__1"
"/test/path/test file with spaces.ses"
"/test/path/test file with spaces"
"/test/path/test file with spaces.txt")
# true false false true
PATTERN_STR='(/[^/.]+|\.ses)$'
for FILE_NAME in "${arr[@]}"; do
  if [[ "$FILE_NAME" =~ $PATTERN_STR ]]; then
    Match_Result="false"
  else
    Match_Result="true"
  fi
  echo $Match_Result
done;

Output:

true
false
false
true

Details:

( - start of a capturing group:
- /[^/.]+ - / and then one or more chars other than / and .
| - or
- \.ses - .ses
) - end of grouping
$ - end of a string.

A case insensitive version is enabled with shopt -s nocasematch/shopt -u nocasematch (see Case insensitive regex in Bash).

edited Nov 25, 2021 at 7:53

answered Nov 23, 2021 at 8:41

Wiktor Stribiżew

631k41 gold badges502 silver badges632 bronze badges

4 Comments

Gill Over a year ago

Thank you Wiktor, it solved my problem! Just a question though, is it hard to get it working without reversing?

user9706 Over a year ago

Good job including test data. Suggest removing the ! and swapping the two branches of the if statement.

Wiktor Stribiżew Over a year ago

@Gill It is hard to do with a single POSIX ERE regex though it is possible. It might look like /[^/.]+\.([^/s][^/][^/]|[^/][^/e][^/]|[^/][^/][^/s]|.{1,2}|.{4,})$, see demo.

Gill Over a year ago

Ahh, then it's no brainer, thanks! :)

Collectives™ on Stack Overflow

Why Regex doesn't work in bash/ksh while it works fine in command line

2 Answers 2

5 Comments

4 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

5 Comments

4 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related