1

I have a shell script on hdfs that accepts 8-9 parameters. Normally, I can carry it out as follows:

sh sample.sh -mode FULL -status DELETE -id 1456 -region AP -path </path/to/filepath>

I tried hadoop fs -cat /dev/test/sample.sh | exec bash -mode FULL -status DELETE -id 1456 -region AP -path /dev/resultsFolder Even though I pass these arguments they are not read and script executes without the arguments. Threw error as No such file or directory What is the best way to deal with this?

Improve this question
1
  • Please write your code exactly as you use it. You will for sure not write -cat <..>, i.e. using angle brackets. Also, explain what you mean by These may not work. Either it works, or it doesn't. You need to know, and if it doesn't work, you need to describe in what respect it doesn't work. Commented Jan 11, 2022 at 12:29

2 Answers 2

Reset to default
1 
bash -c "$(hadoop fs -cat /dev/test/sample.sh)" bash -mode FULL -status DELETE -id 1456 -region AP -path /dev/resultsFolder

would work also.

-c If the -c option is present, then commands are read from the first non-option argument command_string. If there are arguments after the command_string, the first argument is assigned to $0 and any remaining arguments are assigned to the positional parameters. The assignment to $0 sets the name of the shell, which is used in warning and error messages.

Perhaps more readably (although bash -s is rather elegant)

code=$(hadoop fs -cat /dev/test/sample.sh)
bash -c "$code" bash -mode FULL -status DELETE -id 1456 -region AP -path /dev/resultsFolder
Improve this answer
Sign up to request clarification or add additional context in comments.

Comments

0

bash -mode FULL passes the -mode and FULL parameter to bash, not to your script. From the bash man page, describing the options:

-- A -- signals the end of options and disables further option processing. Any arguments after the -- are treated as filenames and arguments. An argument of - is equivalent to --.

and:

-s If the -s option is present, or if no arguments remain after option processing, then commands are read from the standard input. This option allows the positional parameters to be set when invoking an interactive shell.

I would therefore invoke it as

.... | bash - --mode FULL ....

According to the comment given by Gordon Davisson, we need to use -s here:

.... | bash -s -- -mode FULL ....
Improve this answer

6 Comments

bash will still treat the first argument after -- as a filename to run as the command. You want bash -s -- --mode FULL ....
@GordonDavisson: I thought about -s first, but since according to the man-page, a lone - implies --, and -- signals the end of the options, I found - simpler and more logical. Is the man-page incorrect in this description? Otherwise, which point did I misunderstand?
-- signals the end of bash options, but "Any arguments after the -- are treated as filenames and arguments", and the first one after -- is taken as the name of the script. Try it: if you run bash - -mode FULL, you will get a "bash: -mode: No such file or directory" error because it's trying to run a script file named -mode. So you need both -s to run commands from stdin instead of a named file, and -- so that -mode isn't treated as bash options.
Ah, got it. But then, what is the purpose of a lone -? It is supposed to represent stdin as input file, isn't it?
In some situations, but apparently not this one. I get "bash: -: No such file or directory". I'd recommend testing your code to make sure it actually works.
|

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.