using airflow restapi to trigger dag doesn't work but clicking in UI does

Question

Hi I have a simple DAG that uses BashOperator and DockerOperator. I want to trigger from python script like so:

import requests
import json
from datetime import datetime
from pprint import pprint


headers = {
    'accept':'application/json',
    'content-type':'application/json',
}
auth = ('airflow','airflow')
body = {
    "conf": {},
    "dag_run_id":"testrun03",
    "execution_date":datetime.now().strftime("%y-%m-%dt%h:%m:%sz"),
}

result = requests.post(
  "http://localhost:8080/api/v1/dags/r_dag/dagruns",
    headers=headers,
    auth=auth,
  data=json.dumps(body)
)
pprint(result.content.decode('utf-8'))

it gives me this output:

('{\n'
 '  "conf": {},\n'
 '  "dag_id": "r_dag",\n'
 '  "dag_run_id": "testrun03",\n'
 '  "end_date": null,\n'
 '  "execution_date": "2021-05-22T23:59:34+00:00",\n'
 '  "external_trigger": true,\n'
 '  "start_date": "2021-05-23T06:59:34.746354+00:00",\n'
 '  "state": "running"\n'
 '}\n')

and when I GET the dag_run_id:

{
  "conf": {},
  "dag_id": "r_dag",
  "dag_run_id": "testrun03",
  "end_date": "2021-05-23T06:59:35.543319+00:00",
  "execution_date": "2021-05-22T23:59:34+00:00",
  "external_trigger": true,
  "start_date": "2021-05-23T06:59:34.746354+00:00",
  "state": "success"
}

But when I go into the UI and check logs, it is empty, and the graph view shows white outline of the graph. When I hover, it says DAG has yet to run. When I click run within the UI it works as expected. I thought triggering the DAG in programmatic way is the same? Am I missing another step?

Tomasz Urbaszek · Accepted Answer · 2021-05-23 10:01:11Z

1

I was able to reproduce the issue. Created a bug report on Apache Airflow.

EDIT: make sure you send the request with execution date in the same timezone as your instance is using. In my case request that I was sending were UTC+2 resulting in dag runs with execution date in "future".

edited May 23, 2021 at 10:01

answered May 23, 2021 at 9:51

Tomasz Urbaszek

8026 silver badges12 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Spencer Trinh Over a year ago

i changed my script to datetime.utcnow() but the same behaviour is encountered. when I go into Browse -> DAG runs the state is flagged as success, but when I click further into it, it shows that it has not run yet.

Spencer Trinh Over a year ago

hi i finally figured it out. It was indeed the utc timestamp issue. When I changed to utcnow() I made a mistake on the strftime() format. I didn't capitalise some of placeholders so it wasn't the expected timestamp. It works now. Thanks!

Collectives™ on Stack Overflow

using airflow restapi to trigger dag doesn't work but clicking in UI does

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related