I'd like to set system property (not hadoop property) when running hadoop jobs. I found that it's not easy to setting system property. Even I set the property in shell
export HADOOP_OPTS="$HADOOP_OPTS:-Dproperty=value"
It's still not working. The "-D" option of hadoop command line is only for Configuration, not system property. So the "-D" option also not working
Anyone has ideas ? thanks
Why not simply use -Dfoo.bar=example in-line when starting the job via command line, like so:
hadoop jar example.jar com.example.ExampleTool -Dfoo.bar=example argument
In order to get at the property in code, use conf.get("foo.bar");
Now if you genuinely need it to be set as a system property, you can set it at the start of your code using the value you got from Hadoop config like so:
String property = conf.get("foo.bar");
System.setProperty("foo.bar", property);
The hadoop script invoke java class like this
exec "$JAVA" $JAVA_HEAP_MAX $HADOOP_OPTS $CLASS "$@"
so, we can pass system-wide property like this:
export HADOOP_OPTS="$HADOOP_OPTS -Dfoo=bar"
ToolRunnerorGenericOptionsParserat all?