Keep only the non-obvious constraints (unit contract, daemon no-keep-alive
assumption, ContextVar fail-open threading, replace-not-narrow intent) and
cut prose that restates the code. The module docstring in
first_token_timeout.py stays the single canonical description of the path.
Name the configured window in the mid-stream stall message so a read
timeout after the first token is traceable to first_token_timeout_ms; log
invalid values ignored at the pop point; state honestly in _read_timeout_for
that the budget replaces (not narrows) the operator read timeout; pin with a
test that FirstTokenTimeoutError passes graphon's _transform_invoke_error
unchanged, and cover the converter's defensive drop.
isInWorkflow was a proxy for 'this panel's model config reaches
fetch_model_config', which is false for the knowledge-retrieval metadata
filter and single-retrieval model, and for tool/trigger model-selector
params rendered through form-input-item — the field showed up there but the
backend never consumed it. Gate the rule on an explicit
supportFirstTokenTimeout prop passed only by the LLM, question classifier
and parameter extractor panels, and extend the help text with the
inter-token semantics of the read window.
Rename the completion_params key to first_token_timeout_ms so the unit is
explicit in the DSL, and lower the default from 60s to 10s with a 100ms-10min
range: users who enable this gate are latency-sensitive, and 60s rarely
matches that intent. The ms->s conversion happens exactly once, at the pop
point in _normalize_completion_params; everything downstream (ModelInstance
field, ContextVar, httpx read timeout) stays in seconds.
Append a synthetic FIRST_TOKEN_TIMEOUT_PARAMETER_RULE (int, seconds, opt-in)
to the model parameter panel, mirroring how the stop rule is injected. The
rule renders only in workflow advanced mode (isInWorkflow), covering the LLM,
question classifier and parameter extractor panels; easy-UI app orchestration
pages never show it. The value is stored in completion_params and consumed by
the Dify backend before the request reaches the provider.
The timeout travels the same channel as stop: configured per model in
completion_params.first_token_timeout (seconds), popped at the workflow
model-config boundary (_normalize_completion_params) into
ModelInstance.first_token_timeout, and read by DifyPreparedLLM to arm the
transport ContextVar around invoke_llm and invoke_llm_with_structured_output.
Invalid values (non-numeric, bool, non-positive) disable the gate.
Covers workflow and chatflow LLM-compatible nodes (llm, question classifier,
parameter extractor). The easy-UI model config converter drops the key
defensively so imported app configs never forward it to providers. No graphon
changes are required: the popped key never reaches graphon, and its retry
handler treats FirstTokenTimeoutError like any node failure.
The timeout is carried to the plugin-daemon transport through a ContextVar
(core.plugin.impl.first_token_timeout) and applied as httpx's per-request
read timeout in BasePluginClient._stream_request. The daemon withholds the
response headers until the model's first token, so the read timeout measures
time-to-first-token directly; a ReadTimeout before the first line surfaces as
FirstTokenTimeoutError (dify-local, subclassing graphon's InvokeError), while
a later stall stays a plain transport error. A non-positive budget disables
the gate.
L1nSn0w