How can I feed this long line of arguments to an executable?

Question

Arguments file

On args.txt, there are long series of arguments for different call of an executable. Each line contain the arguments for a single call of the executable. One line of args.txt looks like

2 1 -A 7 -B true -C 0.0035 -D /path/to/somewhere ....

The line start by 2 1 as the first two arguments to be given to the executable are "unnamed" (do not come with a flag).

First try

I first tried

i=5
./myexec `sed "${i}q;d" args.txt`

it works most of the time. However, for some lines, the arguments are too long and I receive Error: Command Line Too long as I am overpassing getconf ARG_MAX. Note the software does not allow for specifying arguments other than through the command line.

Second try

So I tried

sed "${i}q;d" args.txt | xargs ./myexec

This second try causes the executable to return nothing.

Questions

Am I doing something wrong with sed "${i}q;d" args.txt | xargs ./myexec?
Once I fix the second try, will I encounter the same issue (Command Line Too long) as for the first try?
Could there be a quotation issue which causes ./myexec to consider the long string as a single argument or something similar?
Would you suggest me trying another way of feeding the arguments to myexec?

FYI

I am on Mac OS X 10.11.3 with Terminal 2.6.1

If it's too long to fit into ARG_MAX, it'll still be too long when you're using xargs -- xargs will just split it into multiple invocations of your executable, each of them given only a subset of the arguments you want (and thus, neither being correct). — Charles Duffy
– Charles Duffy, Commented Sep 22, 2016 at 21:00
Audit your set of defined environment variables, and unset any you can do without -- they use the same pool of space used for command-line arguments. Also adding an answer. — Charles Duffy
– Charles Duffy, Commented Sep 22, 2016 at 21:03
...otherwise, it's a matter of looking at whether the command you're running allows a configuration file or similar alternate source of options/arguments/&c. to be specified. — Charles Duffy
– Charles Duffy, Commented Sep 22, 2016 at 21:06
How long is the argument list? ARG_MAX is 262K on OS X. What are you doing that needs more arguments than this? — Barmar
– Barmar, Commented Sep 22, 2016 at 21:27

Charles Duffy · Accepted Answer · 2016-09-22 21:23:43Z

ARG_MAX is an operating-system constraint on the combined size of command-line arguments and environment variables.

Because it's an operating-system-enforced constraint, you can't bypass it without operating-system-level changes. xargs will split your code into multiple invocations, but each invocation is given only a subset of the arguments desired.

What you can do then, if you can't decrease the length of your argument list (and the program you're running can't read configuration by any means other than the command line), is unset any environment variables you don't need.

If you'd rather fail outright when your command-line argument list is too long to correctly execute, rather than (as xargs does) run two or more invocations each with a subset of the arguments given, I would suggest the following code be used for each line:

args=( )
while IFS= read -r -d '' arg; do
  args+=( "$arg")
done < <(xargs printf '%s\0' <<<"$line")

./yourprog "${args[@]}"

Collectives™ on Stack Overflow

How can I feed this long line of arguments to an executable?

1 Answer 1

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related