Accessing hadoop config setting without having to pass Configuration object through many layers

Question

In the hadoop project I am working on, I need to access a setting in the hadoop Configuration object. However, I only need this setting many, many layers into the reducer, and I really would rather not pass the Context or Configuration objects through so many constructors just for this purpose.

Is there a way I can somehow get access to this setting without passing the configuration through everything between the reducer and the actual class that needs it?

One potential solution I have in mind is to set the configuration setting as a java system property at the start of the reducer, so that way I can access it using System.getProperty() but I'd like to know if there are any cleaner methods.

you can set some value in conf objet and retrive it in reducer — Sourav Gulati
– Sourav Gulati, Commented Apr 7, 2013 at 12:57

Chris Gerken · Accepted Answer · 2013-04-07 01:17:59Z

1

If it's all in the same JVM you could set a static class variable in the reducer's setup() method... or any other class as appropriate.

answered Apr 7, 2013 at 1:17

Chris Gerken

16.4k6 gold badges48 silver badges61 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Accessing hadoop config setting without having to pass Configuration object through many layers

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related