Passing Local Parameters to Hadoop script

Question

To my understanding, the following will result in passing a global Hive variable:

   hive -hiveconf DATE='01/01/2000' -f test_script.hql

That can be called with

   SELECT * FROM DATETABLE WHERE DATE = ${hiveconf:DATE}

And I know that local variables can be defined in the script and called by doing:

   set DATE='01/01/2000'
   SELECT * FROM DATETABLE WHERE DATE = ${DATE}

But say one wanted to submit many jobs with local parameters set for each script, how can we pass them from the command line?

The emphasis is avoiding one script picking up the hiveconf:DATE set by another script that was submitted in quick succession.

EDIT:

I guess this could work, creating a shell script and passing variables to the shell script and then passing those to the individual queries:

    #!/bin/bash

    FIRST_QUERY = "SELECT * FROM DATETABLE WHERE DATE = '$DATE'"

    hive -e "$FIRST_QUERY"

But this seems inefficient, I would still want to know if the option above is possible.

mlegge · Accepted Answer · 2014-09-02 14:56:06Z

1

I found the option -define here:

hive -e 'SELECT * FROM DATETABLE WHERE DATE = ${DATE}' -define DATE='01/01/2000'

answered Sep 2, 2014 at 14:56

mlegge

6,9233 gold badges44 silver badges70 bronze badges

Sign up to request clarification or add additional context in comments.

1 Answer 1