How to pass parameter to sql file with DatabricksSqlOperator in Airflow?

Question

The sample.sql file

INSERT INTO db.table
VALUES (
    '{{ ds }}',
    '${item1}',
    '{item2}'
)
;

The airflow operator

        task = DatabricksSqlOperator(
            task_id=task_id,
            sql_endpoint_name="my-serverless-endpoint",
            sql="sample.sql",
            parameters={
                "item1": "abc",
                "item2": "xyz"
            },
        )

The result db.table

Column A	Column B	Column C
2022-05-05	${item1}	{item2}

The marcro {{ ds }} works just fine. But I can't figure out how to get other parameter to work DatabricksSqlOperator is extending the SQLExecuteQueryOperator https://github.com/apache/airflow/blob/main/airflow/providers/databricks/operators/databricks_sql.py

I tried

'${item1}',
'{item1}
'{params["item1"]}',
'{param.item1}',
'{parameters["item1"]}',
'{parameters.item1}',

Hoping to get

Column A	Column B	Column C
2022-05-05	abc	xyz

ozs · Accepted Answer · 2023-05-06 10:04:29Z

0

You need to use it as Jinja Template : {{ params.item1 }}

DatabricksSqlOperator inherited from SQLExecuteQueryOperator that parse it that way.

answered May 6, 2023 at 10:04

ozs

3,7611 gold badge14 silver badges24 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

How to pass parameter to sql file with DatabricksSqlOperator in Airflow?

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related