Skip to content

Fix deferrable HttpSensor: HttpAsyncHook falls back to https when port == 443 #52585

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

nailo2c
Copy link
Contributor

@nailo2c nailo2c commented Jun 30, 2025

Related: #52319

Why

HttpSensor in deferrable mode fails after the task is resumed by the Triggerer: the schema field is missing, so HttpAsyncHook silently downgrades https://host to http://host.

schema = conn.schema if conn.schema else "http"

The root cause seems to be here. I guess Triggerer fetches the connection from the metadata DB on the first run and then serves subsequent requests from its cache. However, cache no longer contains schema_ in 3.0.0 somehow. In 2.11.0 the same DAG still works.

(The explanation above is my current understanding based on quick code-reading and logs. If I've misinterpreted how the Triggerer caches connections, please feel free to correct me, feedback is very welcome 😄)

msg = SUPERVISOR_COMMS.send(GetConnection(conn_id=conn_id))

# first run
SUPERVISOR_COMMS msg: conn_id='conn_id' conn_type='http' host='dummyjson.com' schema_='https' login=None password=None port=443 extra=None type='ConnectionResult'

# second ... Nth run
SUPERVISOR_COMMS msg: conn_id='conn_id' conn_type='http' host='dummyjson.com' schema_=None login=None password=None port=443 extra=None type='ConnectionResult

The example DAG in the issue works as expected on 2.11.0 but not on 3.0.0. Here is how the DAG runs on 2.11.0:
schema_success_in_2_11_0

How

  • Short Term Solution

conn.port is still present in every run, so fall back to https when port == 443.

  • Long Term Solution

Investigate why schema_ disappears in the Triggerer cache, maybe open a follow-up issue if needed.

What

With this patch the OP’s example DAG now uses https.
schema_can_get_https

@albertwangnz
Copy link

Thank you @nailo2c I tried but did not find the root cause of the missing schema value. You are so good!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants