Trigger airflow DAG using post request with parameters

Question

I'm trying to trigger Airflow's DAG and send parameters inside the post request.

Usage example - DAG has one task that only prints the number sent inside the trigger request.

Example to trigger dag:

import base64
import boto3
import requests

MWAA_ENVIRONMENT_NAME = 'production-mwaa'
dag_name = 'test_param_dag'
mwaa_cli_command = 'dags trigger'

client = boto3.client('mwaa')

mwaa_cli_token = client.create_cli_token(Name=MWAA_ENVIRONMENT_NAME)

mwaa_auth_token = 'Bearer ' + mwaa_cli_token['CliToken']
mwaa_webserver_hostname = f'https://{mwaa_cli_token["WebServerHostname"]}/aws_mwaa/cli'
raw_data = '{0} {1}'.format(mwaa_cli_command, dag_name)

mwaa_response = requests.post(url=mwaa_webserver_hostname,
                          headers=
                          {
                              'Authorization': mwaa_auth_token,
                              'Content-Type': 'text/plain'
                          },
                          params={"x": 11},
                          data=raw_data)

mwaa_std_err_message = base64.b64decode(mwaa_response.json()['stderr']).decode('utf8')
mwaa_std_out_message = base64.b64decode(mwaa_response.json()['stdout']).decode('utf8')

print(mwaa_response.status_code)
print('err:\n' + mwaa_std_err_message)
print('out:\n' + mwaa_std_out_message)

The dag should print the value of 'x' param --> 11.

hemant A · Accepted Answer · 2022-11-18 08:55:19Z

0

You can access the param 'x' inside the dag using "{{ dag_run.conf['x'] }}"

answered Nov 18, 2022 at 8:55

hemant A

1951 silver badge15 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Trigger airflow DAG using post request with parameters

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related