Merge branch 'feat/agent-v2' into feat/agent-v2-e2e-test

Co-authored-by: CodingOnStar <hanxujiang@dify.com>

.agents/skills/how-to-write-component/SKILL.md

@@ -24,6 +24,7 @@ Use this as the component decision guide for Dify web. Existing code is referenc
 
 - Search before adding UI, hooks, helpers, query utilities, or styling patterns. Reuse existing base components, feature components, hooks, utilities, and design styles when they fit.
 - Follow Dify's CSS-first Tailwind v4 contract from `packages/dify-ui/README.md` and `packages/dify-ui/AGENTS.md`. Prefer design-system tokens, utilities, and radius mappings over generic Tailwind choices.
+- Preserve visible keyboard focus states on the final focusable element. Prefer styled `@langgenius/dify-ui/*` controls when available, because components such as `Button` and form/control primitives carry the standard Dify UI `focus-visible` styling. Do not assume every Dify UI export provides visual focus styles: headless anatomy parts and direct Base UI re-exports such as dialog/popover/tooltip/drawer triggers usually only provide behavior and semantics. When using native `button` / `a`, custom trigger `render` props, clickable rows, icon buttons, menu-like items, or direct trigger parts, verify the rendered focusable element has a visible focus state. If it does not, add the standard Dify UI focus style: `outline-hidden focus-visible:ring-2 focus-visible:ring-state-accent-solid`. Do not hide outlines without an equivalent visible `focus-visible` indicator. Component-specific focus styles should follow an existing styled primitive pattern or a concrete design constraint, not a new ad hoc style.
 - Group feature code by workflow, route, or ownership area with route-aligned names: components, hooks, local types, query helpers, atoms, constants, tests, and small utilities should live near the code that changes with them.
 - For each feature module, keep a module-local `README.md` as a boundary note. Start with the module name, a brief one-sentence description, then split dependencies into `Internal Modules` and `External Modules` sections; keep both sections and write `None.` when one category is empty. `Internal Modules` lists modules inside the same overall feature using paths from that feature root, such as `shared/domain/runtime-status`; `External Modules` lists project modules outside the feature using paths from the web root without a `web/` prefix, such as `app/components/base/skeleton`. Omit npm packages, workspace package dependencies, and whitelisted plumbing modules. Do not copy caller-relative import paths into the README.
 - Module README whitelist: `@/service/client`, `@/next/*`.
@@ -53,6 +54,7 @@ Use this as the component decision guide for Dify web. Existing code is referenc
 - Treat `useParams`, route args, and `nuqs` query state as framework-owned state. When atom logic needs those values, hydrate primitive atoms at the route or surface boundary, such as with `useHydrateAtoms(..., { dangerouslyForceHydrate: true })`; keep URL updates in the route/query-state APIs instead of write atoms.
 - Within a route-owned feature, choose one source for route identity. If route params are bridged into feature atoms, use that bridge consistently for route-derived queries and actions instead of also threading the same route id through page, tab, and section props.
 - For async work tied to atom state, use `atomWithQuery` or `atomWithMutation`; write atoms should update only the inputs that drive those atoms. This applies to pure frontend async work as well as network requests, so do not hand-roll loading/error/in-flight state with `useState` or `useRef` for atom-orchestrated async behavior. For component-owned remote work, use `useQuery` or `useMutation` directly.
+- `jotai-tanstack-query` query atoms do not support TanStack Query tracked properties. A component that reads `useAtomValue(queryAtom)` subscribes to the whole query result, even if it only accesses `data`, `isLoading`, or `isError`. Export field-specific derived atoms and have components read the exact fields they render; use `selectAtom(queryAtom, result => result.field)` for query-result fields so unchanged selections do not notify subscribers. Keep direct `useAtomValue(queryAtom)` only when the component or hook genuinely needs the full observer result.
 - Row-local async state belongs to the row owner unless it participates in a shared Jotai workflow or needs atom-scoped reset semantics.
 - Leave query and mutation atoms unscoped so they keep shared QueryClient cache and invalidation behavior. Scope resettable primitives and explicit hydration tuples; scope a derived atom only when every dependency should be private to that surface.
 - For scoped primitives that are always hydrated by `ScopeProvider`, prefer `atomWithLazy<T>(() => { throw new Error(...) })` when consumers should see a non-null type.

.github/workflows/main-ci.yml

@@ -53,6 +53,10 @@ jobs:
           filters: |
             api:
               - 'api/**'
+              - 'scripts/check_no_new_getattr.py'
+              - 'scripts/ast_grep_rules/no_new_getattr.yml'
+              - '.github/workflows/style.yml'
+              - '.github/workflows/main-ci.yml'
               - '.github/workflows/api-tests.yml'
               - 'docker/.env.example'
               - 'docker/envs/middleware.env.example'
@@ -380,6 +384,8 @@ jobs:
     needs: pre_job
     if: needs.pre_job.outputs.should_skip != 'true'
     uses: ./.github/workflows/style.yml
+    with:
+      base-rev: ${{ github.event.pull_request.base.sha || github.event.merge_group.base_sha }}
 
   vdb-tests-run:
     name: Run VDB Tests

.github/workflows/style.yml

@@ -2,6 +2,10 @@ name: Style check
 
 on:
   workflow_call:
+    inputs:
+      base-rev:
+        required: true
+        type: string
 
 concurrency:
   group: style-${{ github.head_ref || github.run_id }}
@@ -33,6 +37,7 @@ jobs:
             scripts/check_no_new_getattr.py
             scripts/ast_grep_rules/no_new_getattr.yml
             .github/workflows/style.yml
+            .github/workflows/main-ci.yml
 
       - name: Setup UV and Python
         if: steps.changed-files.outputs.any_changed == 'true'
@@ -54,17 +59,9 @@ jobs:
         if: steps.changed-files.outputs.any_changed == 'true'
         run: uv run --project api --dev python api/dev/lint_response_contracts.py --fail-on-mismatch
 
-      - name: Fetch merge target ref for getattr guard
-        if: steps.changed-files.outputs.any_changed == 'true'
-        run: git fetch --no-tags --depth=1 origin +refs/heads/main:refs/remotes/origin/main
-
-      - name: Bind merge target branch for getattr guard
-        if: steps.changed-files.outputs.any_changed == 'true'
-        run: git show-ref --verify --quiet refs/heads/main || git branch main origin/main
-
       - name: Run No New Getattr Guard
         if: steps.changed-files.outputs.any_changed == 'true'
-        run: uv run --project api python scripts/check_no_new_getattr.py --mode ci --merge-target main
+        run: uv run --project api python scripts/check_no_new_getattr.py --base-rev "${{ inputs.base-rev }}"
 
       - name: Run Type Checks
         if: steps.changed-files.outputs.any_changed == 'true'

.github/workflows/web-e2e.yml

@@ -1,6 +1,10 @@
 name: Web Full-Stack E2E
 
 on:
+  push:
+    branches:
+      - feat/agent-v2-e2e-test
+
   workflow_call:
     inputs:
       run-external-runtime:
@@ -59,7 +63,7 @@ jobs:
         run: vp run e2e:full
 
       - name: Run external runtime E2E tests
-        if: ${{ inputs.run-external-runtime }}
+        if: ${{ inputs.run-external-runtime || github.ref == 'refs/heads/feat/agent-v2-e2e-test' }}
         working-directory: ./e2e
         env:
           E2E_ADMIN_EMAIL: e2e-admin@example.com

.superpowers/sdd/hitl-timeout-semantics-impl-report.md

@@ -1,30 +0,0 @@
-# HITL timeout semantics implementation report
-
-## What changed
-
-- Updated `api/core/workflow/nodes/human_input/callback.py` so `DifyHITLCallback` now preserves Dify's timeout split at the boundary:
-  - `HumanInputFormStatus.TIMEOUT` returns the graphon timeout branch via `Expired(selected_handle="__timeout__", ...)`.
-  - `HumanInputFormStatus.EXPIRED` is treated as an invalid resume state and raises `AssertionError`.
-  - `HumanInputFormStatus.WAITING` with a past global deadline is treated as an invalid resume state and raises `AssertionError`.
-  - `HumanInputFormStatus.WAITING` with only the node-level deadline expired still returns the timeout branch.
-- Added `created_at` to `HumanInputFormEntity` and `_HumanInputFormEntityImpl` so the callback can compute the global deadline using Dify's shared `HUMAN_INPUT_GLOBAL_TIMEOUT_SECONDS` invariant.
-- Kept the submitted and pause flows unchanged.
-- Added focused unit coverage in `api/tests/unit_tests/core/workflow/test_human_input_callback.py` for:
-  - node timeout branch
-  - global expiration rejection
-  - waiting-form past node deadline timeout
-  - waiting-form past global deadline rejection
-
-## Verification
-
-- `uv run --project api pytest -o addopts='' api/tests/unit_tests/core/workflow/test_human_input_callback.py api/tests/unit_tests/core/workflow/nodes/human_input/test_human_input_form_filled_event.py -q`
-- `git diff --check`
-
-## Result
-
-- The focused test set is expected to pass with the new `created_at` boundary in place.
-- No unrelated files were modified.
-
-## Concerns
-
-- The callback now fails fast on invalid resume states by design. That is intentional, but any caller that previously relied on `EXPIRED` being mapped to the timeout branch will now see an assertion failure instead.

api/.env.example

@@ -663,6 +663,9 @@ PLUGIN_MODEL_SCHEMA_CACHE_TTL=3600
 PLUGIN_MODEL_PROVIDERS_CACHE_TTL=86400
 INNER_API_KEY_FOR_PLUGIN=QaHbTe77CtuXmsfyhR7+vRjI/+XbV1AaFy691iy+kGDv2Jvy0/eAh8Y1
 
+# Dify Agent backend
+AGENT_BACKEND_BASE_URL=http://localhost:5050
+
 # Marketplace configuration
 MARKETPLACE_ENABLED=true
 MARKETPLACE_API_URL=https://marketplace.dify.ai

api/clients/agent_backend/__init__.py

@@ -18,6 +18,7 @@ from clients.agent_backend.errors import (
     AgentBackendValidationError,
 )
 from clients.agent_backend.event_adapter import (
+    AgentBackendAgentMessageDeltaInternalEvent,
     AgentBackendDeferredToolCallInternalEvent,
     AgentBackendInternalEvent,
     AgentBackendInternalEventType,
@@ -46,6 +47,11 @@ from clients.agent_backend.request_builder import (
     AgentBackendWorkflowNodeRunInput,
     redact_for_agent_backend_log,
 )
+from clients.agent_backend.session_cleanup import (
+    AgentBackendSessionCleanupPayload,
+    AgentBackendSessionCleanupResult,
+    cleanup_agent_backend_session,
+)
 
 __all__ = [
     "AGENT_SOUL_PROMPT_LAYER_ID",
@@ -57,6 +63,7 @@ __all__ = [
     "WORKFLOW_NODE_JOB_PROMPT_LAYER_ID",
     "WORKFLOW_USER_PROMPT_LAYER_ID",
     "AgentBackendAgentAppRunInput",
+    "AgentBackendAgentMessageDeltaInternalEvent",
     "AgentBackendDeferredToolCallInternalEvent",
     "AgentBackendError",
     "AgentBackendHTTPError",
@@ -73,6 +80,8 @@ __all__ = [
     "AgentBackendRunRequestBuilder",
     "AgentBackendRunStartedInternalEvent",
     "AgentBackendRunSucceededInternalEvent",
+    "AgentBackendSessionCleanupPayload",
+    "AgentBackendSessionCleanupResult",
     "AgentBackendStreamError",
     "AgentBackendStreamInternalEvent",
     "AgentBackendTransportError",
@@ -82,6 +91,7 @@ __all__ = [
     "FakeAgentBackendRunClient",
     "FakeAgentBackendScenario",
     "RuntimeLayerSpec",
+    "cleanup_agent_backend_session",
     "create_agent_backend_run_client",
     "extract_runtime_layer_specs",
     "redact_for_agent_backend_log",

api/clients/agent_backend/event_adapter.py

@@ -5,6 +5,9 @@ The adapter does not define a new cross-service event contract. It consumes
 workflow Agent Node maps to Graphon/AppQueue events. Deferred external tool calls
 remain Dify Agent ``run_succeeded`` payloads on the wire; API code turns them
 into an internal event so workflow pause/session handling stays local to API.
+Agent-message deltas are exposed as annotations on ``PydanticAIStreamRunEvent``
+so API code does not have to parse Pydantic AI stream-event internals to
+preserve streaming. The terminal answer remains the ``run_succeeded`` output.
 """
 
 from __future__ import annotations
@@ -32,6 +35,7 @@ class AgentBackendInternalEventType(StrEnum):
 
     RUN_STARTED = "run_started"
     STREAM_EVENT = "stream_event"
+    AGENT_MESSAGE_DELTA = "agent_message_delta"
     DEFERRED_TOOL_CALL = "deferred_tool_call"
     RUN_SUCCEEDED = "run_succeeded"
     RUN_FAILED = "run_failed"
@@ -61,6 +65,13 @@ class AgentBackendStreamInternalEvent(AgentBackendInternalEventBase):
     data: JsonValue
 
 
+class AgentBackendAgentMessageDeltaInternalEvent(AgentBackendInternalEventBase):
+    """API-internal agent-message delta emitted independently from raw stream events."""
+
+    type: Literal[AgentBackendInternalEventType.AGENT_MESSAGE_DELTA] = AgentBackendInternalEventType.AGENT_MESSAGE_DELTA
+    delta: str
+
+
 class AgentBackendRunSucceededInternalEvent(AgentBackendInternalEventBase):
     """API-internal terminal success event carrying final output and session state."""
 
@@ -99,6 +110,7 @@ class AgentBackendRunCancelledInternalEvent(AgentBackendInternalEventBase):
 type AgentBackendInternalEvent = Annotated[
     AgentBackendRunStartedInternalEvent
     | AgentBackendStreamInternalEvent
+    | AgentBackendAgentMessageDeltaInternalEvent
     | AgentBackendDeferredToolCallInternalEvent
     | AgentBackendRunSucceededInternalEvent
     | AgentBackendRunFailedInternalEvent
@@ -121,6 +133,14 @@ class AgentBackendRunEventAdapter:
                     )
                 ]
             case PydanticAIStreamRunEvent():
+                if event.agent_message_delta:
+                    return [
+                        AgentBackendAgentMessageDeltaInternalEvent(
+                            run_id=event.run_id,
+                            source_event_id=event.id,
+                            delta=event.agent_message_delta,
+                        )
+                    ]
                 data = cast(JsonValue, _EVENT_DATA_ADAPTER.dump_python(event.data, mode="json"))
                 event_kind = data.get("event_kind") if isinstance(data, dict) else None
                 return [

api/clients/agent_backend/request_builder.py

@@ -11,8 +11,9 @@ composition-driven.
 
 from __future__ import annotations
 
+import re
 from collections.abc import Mapping
-from typing import ClassVar
+from typing import ClassVar, Literal
 
 from agenton.compositor import CompositorSessionSnapshot
 from agenton.compositor.schemas import LayerSessionSnapshot
@@ -46,7 +47,6 @@ from dify_agent.protocol import (
     LayerExitSignals,
     RunComposition,
     RunLayerSpec,
-    RunPurpose,
     RuntimeLayerSpec,
 )
 from pydantic import BaseModel, ConfigDict, Field, JsonValue, field_validator
@@ -63,6 +63,7 @@ DIFY_CORE_TOOLS_LAYER_ID = "core_tools"
 DIFY_KNOWLEDGE_BASE_LAYER_ID = "knowledge"
 DIFY_ASK_HUMAN_LAYER_ID = "ask_human"
 DIFY_SHELL_LAYER_ID = "shell"
+type AgentConfigVersionKind = Literal["snapshot", "draft", "build_draft"]
 
 
 def _filter_snapshot_to_specs(
@@ -104,6 +105,59 @@ def _shell_config_with_drive_ref(
     return config.model_copy(update={"agent_stub_drive_ref": drive_config.drive_ref})
 
 
+def _markdown_backtick_fence(text: str) -> str:
+    """Choose a fence that will not terminate inside the prompt body."""
+    longest_backtick_run = max((len(match.group(0)) for match in re.finditer(r"`+", text)), default=0)
+    return "`" * max(3, longest_backtick_run + 1)
+
+
+_BUILD_DRAFT_AGENT_SOUL_PROMPT = """You are running in build mode.
+
+Objective:
+- Improve this agent's working environment, configuration, tools, files, notes,
+  and context so it can handle the intended task well.
+
+Guidance:
+- Treat the intended task as context for setup work, validation, and configuration decisions.
+- Perform concrete investigative or setup steps when they help improve or verify the agent configuration.
+- Use the installed `dify-agent` CLI when you need to inspect or persist Agent configuration."""
+
+
+def _wrap_build_draft_agent_soul_prompt(prompt: str | None) -> str:
+    """Reframe build-draft Agent Soul prompts as preparation work for a future run."""
+    prompt_body = (prompt or "").strip()
+    if not prompt_body:
+        return _BUILD_DRAFT_AGENT_SOUL_PROMPT + "\n\nIntended task for later normal runs:\nNo task prompt was provided."
+    fence = _markdown_backtick_fence(prompt_body)
+    return (
+        _BUILD_DRAFT_AGENT_SOUL_PROMPT
+        + f"\n\nIntended task for later normal runs:\n{fence}text\n{prompt_body}\n{fence}"
+    )
+
+
+def _agent_soul_prompt_for_layer(
+    prompt: str | None,
+    *,
+    config_version_kind: AgentConfigVersionKind,
+) -> str | None:
+    """Preserve normal snapshot/draft prompts and only wrap build-draft prompts.
+
+    The API-side layer adapter is the product boundary where Agent Soul text
+    becomes the model-facing system-prompt layer. ``snapshot`` and normal
+    ``draft`` runs pass through the original effective prompt unchanged, while
+    ``build_draft`` always emits a setup prompt. When an original prompt is
+    present, it is reframed as future-run context and embedded in a fenced
+    block; when it is blank, the setup instruction is still kept.
+    """
+    if config_version_kind != "build_draft":
+        if prompt is None:
+            return None
+        if not prompt.strip():
+            return None
+        return prompt
+    return _wrap_build_draft_agent_soul_prompt(prompt)
+
+
 class AgentBackendModelConfig(BaseModel):
     """API-side model/plugin selection before it is converted to Dify Agent layers."""
 
@@ -163,7 +217,7 @@ class AgentBackendWorkflowNodeRunInput(BaseModel):
     workflow_node_job_prompt: str
     user_prompt: str
     agent_soul_prompt: str | None = None
-    purpose: RunPurpose = "workflow_node"
+    agent_config_version_kind: AgentConfigVersionKind = "snapshot"
     idempotency_key: str | None = None
     output: AgentBackendOutputConfig | None = None
     tools: DifyPluginToolsLayerConfig | None = None
@@ -212,7 +266,7 @@ class AgentBackendAgentAppRunInput(BaseModel):
     execution_context: DifyExecutionContextLayerConfig
     user_prompt: str
     agent_soul_prompt: str | None = None
-    purpose: RunPurpose = "agent_app"
+    agent_config_version_kind: AgentConfigVersionKind = "snapshot"
     idempotency_key: str | None = None
     output: AgentBackendOutputConfig | None = None
     tools: DifyPluginToolsLayerConfig | None = None
@@ -261,13 +315,17 @@ class AgentBackendRunRequestBuilder:
         prompt.
         """
         layers: list[RunLayerSpec] = []
-        if run_input.agent_soul_prompt:
+        agent_soul_prompt = _agent_soul_prompt_for_layer(
+            run_input.agent_soul_prompt,
+            config_version_kind=run_input.agent_config_version_kind,
+        )
+        if agent_soul_prompt:
             layers.append(
                 RunLayerSpec(
                     name=AGENT_SOUL_PROMPT_LAYER_ID,
                     type=PLAIN_PROMPT_LAYER_TYPE_ID,
                     metadata={**run_input.metadata, "origin": "agent_soul"},
-                    config=PromptLayerConfig(prefix=run_input.agent_soul_prompt),
+                    config=PromptLayerConfig(prefix=agent_soul_prompt),
                 )
             )
 
@@ -419,7 +477,6 @@ class AgentBackendRunRequestBuilder:
 
         return CreateRunRequest(
             composition=RunComposition(layers=layers),
-            purpose=run_input.purpose,
             idempotency_key=run_input.idempotency_key,
             metadata=run_input.metadata,
             session_snapshot=run_input.session_snapshot,
@@ -467,7 +524,6 @@ class AgentBackendRunRequestBuilder:
         filtered_snapshot = _filter_snapshot_to_specs(session_snapshot, runtime_layer_specs)
         return CreateRunRequest(
             composition=RunComposition(layers=layers),
-            purpose="workflow_node",
             idempotency_key=idempotency_key,
             metadata=request_metadata,
             session_snapshot=filtered_snapshot,
@@ -483,13 +539,17 @@ class AgentBackendRunRequestBuilder:
         ask_human / structured output.
         """
         layers: list[RunLayerSpec] = []
-        if run_input.agent_soul_prompt:
+        agent_soul_prompt = _agent_soul_prompt_for_layer(
+            run_input.agent_soul_prompt,
+            config_version_kind=run_input.agent_config_version_kind,
+        )
+        if agent_soul_prompt:
             layers.append(
                 RunLayerSpec(
                     name=AGENT_SOUL_PROMPT_LAYER_ID,
                     type=PLAIN_PROMPT_LAYER_TYPE_ID,
                     metadata={**run_input.metadata, "origin": "agent_soul"},
-                    config=PromptLayerConfig(prefix=run_input.agent_soul_prompt),
+                    config=PromptLayerConfig(prefix=agent_soul_prompt),
                 )
             )
 
@@ -649,7 +709,6 @@ class AgentBackendRunRequestBuilder:
 
         return CreateRunRequest(
             composition=RunComposition(layers=layers),
-            purpose=run_input.purpose,
             idempotency_key=run_input.idempotency_key,
             metadata=run_input.metadata,
             session_snapshot=run_input.session_snapshot,

api/clients/agent_backend/session_cleanup.py

@@ -0,0 +1,100 @@
+"""Shared API-side helper for Agent backend lifecycle-only session cleanup.
+
+Product code owns local row retirement and background-task dispatch. This module
+only adapts persisted cleanup inputs into the public ``dify-agent`` run
+protocol, performs the synchronous ``create_run + wait_run`` loop used by Celery
+workers, and reports whether the backend cleanup succeeded, was skipped, or
+failed.
+"""
+
+from __future__ import annotations
+
+from dataclasses import dataclass
+from typing import ClassVar, Literal
+
+from agenton.compositor import CompositorSessionSnapshot
+from dify_agent.protocol import RuntimeLayerSpec
+from pydantic import BaseModel, ConfigDict, Field, JsonValue
+
+from clients.agent_backend.client import AgentBackendRunClient
+from clients.agent_backend.errors import AgentBackendError
+from clients.agent_backend.request_builder import AgentBackendRunRequestBuilder
+
+
+class AgentBackendSessionCleanupPayload(BaseModel):
+    """Serialized cleanup inputs preserved across API and Celery boundaries."""
+
+    session_snapshot: CompositorSessionSnapshot | None = None
+    runtime_layer_specs: list[RuntimeLayerSpec] = Field(default_factory=list)
+    idempotency_key: str | None = None
+    metadata: dict[str, JsonValue] = Field(default_factory=dict)
+    timeout_seconds: float = 30.0
+
+    model_config: ClassVar[ConfigDict] = ConfigDict(extra="forbid")
+
+
+@dataclass(frozen=True, slots=True)
+class AgentBackendSessionCleanupResult:
+    """Terminal outcome of one backend cleanup attempt."""
+
+    status: Literal["succeeded", "skipped", "failed"]
+    reason: str | None = None
+    cleanup_run_id: str | None = None
+
+    @classmethod
+    def succeeded(cls, cleanup_run_id: str) -> AgentBackendSessionCleanupResult:
+        return cls(status="succeeded", cleanup_run_id=cleanup_run_id)
+
+    @classmethod
+    def skipped(cls, reason: str) -> AgentBackendSessionCleanupResult:
+        return cls(status="skipped", reason=reason)
+
+    @classmethod
+    def failed(cls, reason: str, cleanup_run_id: str | None = None) -> AgentBackendSessionCleanupResult:
+        return cls(status="failed", reason=reason, cleanup_run_id=cleanup_run_id)
+
+
+def cleanup_agent_backend_session(
+    *,
+    payload: AgentBackendSessionCleanupPayload,
+    client: AgentBackendRunClient | None,
+    request_builder: AgentBackendRunRequestBuilder | None = None,
+) -> AgentBackendSessionCleanupResult:
+    """Run lifecycle-only cleanup against the Agent backend and report status."""
+    if client is None:
+        return AgentBackendSessionCleanupResult.skipped("no_agent_backend_client")
+    if payload.session_snapshot is None:
+        return AgentBackendSessionCleanupResult.skipped("missing_session_snapshot")
+    if not payload.runtime_layer_specs:
+        return AgentBackendSessionCleanupResult.skipped("missing_runtime_layer_specs")
+
+    builder = request_builder or AgentBackendRunRequestBuilder()
+    request = builder.build_cleanup_request(
+        session_snapshot=payload.session_snapshot,
+        runtime_layer_specs=payload.runtime_layer_specs,
+        idempotency_key=payload.idempotency_key,
+        metadata=payload.metadata,
+    )
+
+    try:
+        response = client.create_run(request)
+    except AgentBackendError as exc:
+        return AgentBackendSessionCleanupResult.failed(str(exc))
+
+    try:
+        status_response = client.wait_run(response.run_id, timeout_seconds=payload.timeout_seconds)
+    except AgentBackendError as exc:
+        return AgentBackendSessionCleanupResult.failed(str(exc), cleanup_run_id=response.run_id)
+
+    if status_response.status != "succeeded":
+        reason = status_response.error or f"cleanup run ended with status {status_response.status}"
+        return AgentBackendSessionCleanupResult.failed(reason, cleanup_run_id=response.run_id)
+
+    return AgentBackendSessionCleanupResult.succeeded(response.run_id)
+
+
+__all__ = [
+    "AgentBackendSessionCleanupPayload",
+    "AgentBackendSessionCleanupResult",
+    "cleanup_agent_backend_session",
+]

api/commands/account.py

@@ -25,7 +25,7 @@ def reset_password(email, new_password, password_confirm):
         return
     normalized_email = email.strip().lower()
 
-    account = AccountService.get_account_by_email_with_case_fallback(db.session, email.strip())
+    account = AccountService.get_account_by_email_with_case_fallback(email.strip(), session=db.session())
 
     if not account:
         click.echo(click.style(f"Account not found for email: {email}", fg="red"))
@@ -67,7 +67,7 @@ def reset_email(email, new_email, email_confirm):
         return
     normalized_new_email = new_email.strip().lower()
 
-    account = AccountService.get_account_by_email_with_case_fallback(db.session, email.strip())
+    account = AccountService.get_account_by_email_with_case_fallback(email.strip(), session=db.session())
 
     if not account:
         click.echo(click.style(f"Account not found for email: {email}", fg="red"))
@@ -133,9 +133,9 @@ def create_tenant(email: str, language: str | None = None, name: str | None = No
         password=new_password,
         language=language,
         create_workspace_required=False,
-        session=db.session,
+        session=db.session(),
     )
-    TenantService.create_owner_tenant_if_not_exist(account, name, session=db.session)
+    TenantService.create_owner_tenant_if_not_exist(account, name, session=db.session())
 
     click.echo(
         click.style(

api/commands/data_migration.py

@@ -9,6 +9,7 @@ from uuid import UUID
 import click
 import sqlalchemy as sa
 import yaml
+from sqlalchemy.orm import Session
 
 from core.db.session_factory import session_factory
 from extensions.ext_database import db
@@ -108,7 +109,7 @@ def export_migration_data(input_file: str | None, output_file: str | None, overw
         raw_config = _load_json_object(input_file, "Export config")
         selection = ExportConfigParser().parse(raw_config)
         with session_factory.create_session() as session:
-            result = MigrationExportService().export(session, selection)
+            result = MigrationExportService().export(selection, session=session)
         MigrationPackageService().save_package(result.package, output_file, overwrite=overwrite)
         click.echo(click.style(f"Output written to {output_file}", fg="green"))
         _render_report(result.report_items, context=_with_output_path(result.report_context, output_file))
@@ -157,7 +158,6 @@ def import_migration_data(
         package = MigrationPackageService().load_package(input_file)
         with session_factory.create_session() as session:
             result = MigrationImportService().import_package(
-                session,
                 ImportRequest(
                     package=package,
                     cli_target_tenant=target_tenant,
@@ -169,6 +169,7 @@ def import_migration_data(
                         create_app_api_token_on_import=create_app_api_token_on_import,
                     ),
                 ),
+                session=session,
             )
         _render_report(result.report_items, context=result.report_context)
     except MigrationDataError as exc:
@@ -217,7 +218,9 @@ def migration_data_wizard() -> None:
             default=True,
             show_default=False,
         )
-        auto_tools = _discover_auto_tools([app for app in apps if app.id in set(app_ids)], include_referenced_tools)
+        auto_tools = _discover_auto_tools(
+            [app for app in apps if app.id in set(app_ids)], include_referenced_tools, session=db.session()
+        )
         auto_tools = _resolve_auto_tool_names(tenant.id, auto_tools)
         _print_auto_tools(auto_tools)
         additional_tools = _prompt_additional_tools(tenant.id, auto_tools)
@@ -253,7 +256,7 @@ def migration_data_wizard() -> None:
             output_file=output_file,
         )
         with session_factory.create_session() as session:
-            result = MigrationExportService().export(session, selection)
+            result = MigrationExportService().export(selection, session=session)
         MigrationPackageService().save_package(result.package, output_file, overwrite=overwrite)
         click.echo(click.style(f"Output written to {output_file}", fg="green"))
         _print_wizard_step("Report")
@@ -394,13 +397,13 @@ def _prompt_import_options() -> tuple[bool, bool, str, str]:
     return include_secrets, create_tokens, id_strategy, conflict_strategy
 
 
-def _discover_auto_tools(apps: list[App], include_referenced_tools: bool) -> WizardToolMap:
+def _discover_auto_tools(apps: list[App], include_referenced_tools: bool, *, session: Session) -> WizardToolMap:
     auto_tools: WizardToolMap = {"api_tools": {}, "workflow_tools": {}, "mcp_tools": {}}
     if not include_referenced_tools:
         return auto_tools
     discovery_service = DependencyDiscoveryService()
     for app in apps:
-        dsl_content = AppDslService.export_dsl(app_model=app, include_secret=False)
+        dsl_content = AppDslService.export_dsl(app_model=app, session=session, include_secret=False)
         raw_dsl = yaml.safe_load(dsl_content) if dsl_content else {}
         dsl = raw_dsl if isinstance(raw_dsl, dict) else {}
         for dependency in discovery_service.discover_from_dsl(dsl):

api/commands/plugin.py

@@ -472,6 +472,7 @@ def backfill_plugin_auto_upgrade(
         try:
             result = PluginAutoUpgradeService.backfill_strategy_categories(
                 current_tenant_id,
+                session=db.session(),
             )
         except Exception as e:
             failed_count += 1

api/commands/rbac.py

@@ -6,6 +6,7 @@ from concurrent.futures import ThreadPoolExecutor, as_completed
 
 import click
 from sqlalchemy import select
+from sqlalchemy.orm import Session
 
 from configs import dify_config
 from core.db.session_factory import session_factory
@@ -131,16 +132,35 @@ def _replace_member_role(
     operator_account_id: str,
     member_account_id: str,
     role_id: str,
+    *,
+    session: Session,
 ) -> str:
     RBACService.MemberRoles.replace(
         tenant_id=tenant_id,
         account_id=operator_account_id,
         member_account_id=member_account_id,
         role_ids=[role_id],
+        session=session,
     )
     return member_account_id
 
 
+def _replace_member_role_with_new_session(
+    tenant_id: str,
+    operator_account_id: str,
+    member_account_id: str,
+    role_id: str,
+) -> str:
+    with session_factory.create_session() as session:
+        return _replace_member_role(
+            tenant_id=tenant_id,
+            operator_account_id=operator_account_id,
+            member_account_id=member_account_id,
+            role_id=role_id,
+            session=session,
+        )
+
+
 @click.command(
     "rbac-migrate-member-roles", help="Migrate legacy workspace member roles into RBAC member-role bindings."
 )
@@ -217,14 +237,21 @@ def migrate_member_roles_to_rbac(
 
         if replace_jobs:
             if workers == 1:
-                for member_account_id, resolved_role_id in replace_jobs:
-                    _replace_member_role(workspace_id, owner_account_id, member_account_id, resolved_role_id)
-                    migrated_count += 1
+                with session_factory.create_session() as session:
+                    for member_account_id, resolved_role_id in replace_jobs:
+                        _replace_member_role(
+                            workspace_id,
+                            owner_account_id,
+                            member_account_id,
+                            resolved_role_id,
+                            session=session,
+                        )
+                        migrated_count += 1
             else:
                 with ThreadPoolExecutor(max_workers=workers) as executor:
                     futures = [
                         executor.submit(
-                            _replace_member_role,
+                            _replace_member_role_with_new_session,
                             workspace_id,
                             owner_account_id,
                             member_account_id,

api/configs/extra/agent_backend_config.py

@@ -25,11 +25,10 @@ class AgentBackendConfig(BaseSettings):
     AGENT_SHELL_ENABLED: bool = Field(
         description=(
             "Inject the dify.shell layer (sandboxed bash workspace) into Agent runs. "
-            "Requires the agent backend to be wired with a shellctl entrypoint; keep it "
-            "off until shellctl is deployed, otherwise every agent run that includes the "
-            "shell layer will fail."
+            "Requires the agent backend to be wired with a shellctl entrypoint before "
+            "shell-using Agent runs are executed."
         ),
-        default=False,
+        default=True,
     )
 
     AGENT_APP_TEXT_DELTA_DEBOUNCE_SECONDS: NonNegativeFloat = Field(

api/configs/feature/__init__.py

@@ -363,7 +363,10 @@ class FileAccessConfig(BaseSettings):
     INTERNAL_FILES_URL: str = Field(
         description="Internal base URL for file access within Docker network,"
         " used for plugin daemon and internal service communication."
-        " Falls back to FILES_URL if not specified.",
+        " Explicit INTERNAL_FILES_URL takes precedence; otherwise SERVER_CONSOLE_API_URL is used,"
+        " then FILES_URL.",
+        validation_alias=AliasChoices("INTERNAL_FILES_URL", "SERVER_CONSOLE_API_URL"),
+        alias_priority=1,
         default="",
     )
 

api/configs/packaging/pyproject.py

@@ -6,6 +6,17 @@ class PyProjectConfig(BaseModel):
     version: str = Field(description="Dify version", default="")
 
 
+class DifyToolConfig(BaseModel):
+    min_difyctl_version: str = Field(
+        description="Oldest difyctl version served on /openapi/v1",
+        default="0.0.0",
+    )
+
+
+class ToolConfig(BaseModel):
+    dify: DifyToolConfig = Field(default=DifyToolConfig())
+
+
 class PyProjectTomlConfig(BaseSettings):
     """
     configs in api/pyproject.toml
@@ -15,3 +26,8 @@ class PyProjectTomlConfig(BaseSettings):
         description="configs in the project section of pyproject.toml",
         default=PyProjectConfig(),
     )
+
+    tool: ToolConfig = Field(
+        description="configs in the [tool.*] section of pyproject.toml",
+        default=ToolConfig(),
+    )

api/controllers/common/app_access.py

@@ -4,6 +4,7 @@ from collections.abc import Sequence
 from dataclasses import dataclass
 from typing import TYPE_CHECKING
 
+from extensions.ext_database import db
 from services.enterprise import rbac_service as enterprise_rbac_service
 
 if TYPE_CHECKING:
@@ -76,7 +77,7 @@ def resolve_app_access_filter(
     inner-API round trip; otherwise it is fetched here.
     """
     if permissions is None:
-        permissions = enterprise_rbac_service.RBACService.MyPermissions.get(tenant_id, account_id)
+        permissions = enterprise_rbac_service.RBACService.MyPermissions.get(tenant_id, account_id, session=db.session())
     whitelist_scope = enterprise_rbac_service.RBACService.AppAccess.whitelist_resources(tenant_id, account_id)
 
     can_manage_own_apps = _MANAGE_OWN_APPS_PERMISSION_KEY in permissions.workspace.permission_keys

api/controllers/common/fields.py

@@ -1,6 +1,6 @@
 from __future__ import annotations
 
-from typing import Any
+from typing import Any, Literal
 
 from pydantic import BaseModel, ConfigDict, Field, RootModel, computed_field
 
@@ -52,6 +52,11 @@ class AudioTranscriptResponse(ResponseModel):
     text: str
 
 
+class ValidationResultResponse(ResponseModel):
+    result: Literal["success", "error"]
+    error: str | None = None
+
+
 class SimpleResultMessageResponse(ResponseModel):
     result: str
     message: str

api/controllers/console/agent/composer.py

@@ -16,6 +16,7 @@ from controllers.console.wraps import (
     with_current_tenant_id,
     with_current_user_id,
 )
+from extensions.ext_database import db
 from fields.agent_fields import (
     AgentAppComposerResponse,
     AgentComposerCandidatesResponse,
@@ -69,6 +70,7 @@ class WorkflowAgentComposerApi(Resource):
                 node_id=node_id,
                 account_id=account_id,
                 snapshot_id=query.snapshot_id,
+                session=db.session(),
             ),
         )
 
@@ -94,6 +96,7 @@ class WorkflowAgentComposerApi(Resource):
                 node_id=node_id,
                 account_id=account_id,
                 payload=payload,
+                session=db.session(),
             ),
         )
 
@@ -126,6 +129,7 @@ class WorkflowAgentComposerCopyFromRosterApi(Resource):
                 source_agent_id=payload.source_agent_id,
                 source_snapshot_id=payload.source_snapshot_id,
                 idempotency_key=payload.idempotency_key,
+                session=db.session(),
             ),
         )
 
@@ -149,8 +153,9 @@ class WorkflowAgentComposerValidateApi(Resource):
             tenant_id=tenant_id,
             payload=payload,
             agent_id=AgentComposerService.resolve_workflow_node_agent_id(
-                tenant_id=tenant_id, app_id=app_model.id, node_id=node_id
+                tenant_id=tenant_id, app_id=app_model.id, node_id=node_id, session=db.session()
             ),
+            session=db.session(),
         )
         return dump_response(AgentComposerValidateResponse, {"result": "success", "errors": [], **findings})
 
@@ -174,6 +179,7 @@ class WorkflowAgentComposerCandidatesApi(Resource):
                 app_id=app_model.id,
                 node_id=node_id,
                 user_id=current_user_id,
+                session=db.session(),
             ),
         )
 
@@ -196,7 +202,9 @@ class WorkflowAgentComposerImpactApi(Resource):
             )
         return dump_response(
             AgentComposerImpactResponse,
-            AgentComposerService.calculate_impact(tenant_id=tenant_id, current_snapshot_id=current_snapshot_id),
+            AgentComposerService.calculate_impact(
+                tenant_id=tenant_id, current_snapshot_id=current_snapshot_id, session=db.session()
+            ),
         )
 
 
@@ -224,6 +232,7 @@ class WorkflowAgentComposerSaveToRosterApi(Resource):
                 node_id=node_id,
                 account_id=account_id,
                 payload=payload,
+                session=db.session(),
             ),
         )
 
@@ -238,7 +247,7 @@ class AgentComposerApi(Resource):
     def get(self, tenant_id: str, agent_id: UUID):
         return dump_response(
             AgentAppComposerResponse,
-            AgentComposerService.load_agent_composer(tenant_id=tenant_id, agent_id=str(agent_id)),
+            AgentComposerService.load_agent_composer(tenant_id=tenant_id, agent_id=str(agent_id), session=db.session()),
         )
 
     @console_ns.expect(console_ns.models[ComposerSavePayload.__name__])
@@ -259,6 +268,7 @@ class AgentComposerApi(Resource):
                 agent_id=str(agent_id),
                 account_id=account_id,
                 payload=payload,
+                session=db.session(),
             ),
         )
 
@@ -274,7 +284,7 @@ class AgentComposerValidateApi(Resource):
     @account_initialization_required
     @with_current_tenant_id
     def post(self, tenant_id: str, agent_id: UUID):
-        AgentComposerService.load_agent_composer(tenant_id=tenant_id, agent_id=str(agent_id))
+        AgentComposerService.load_agent_composer(tenant_id=tenant_id, agent_id=str(agent_id), session=db.session())
         payload = ComposerSavePayload.model_validate(console_ns.payload or {})
         ComposerConfigValidator.validate_publish_payload(payload)
         AgentComposerService.validate_knowledge_datasets(tenant_id=tenant_id, agent_soul=payload.agent_soul)
@@ -282,6 +292,7 @@ class AgentComposerValidateApi(Resource):
             tenant_id=tenant_id,
             payload=payload,
             agent_id=str(agent_id),
+            session=db.session(),
         )
         return dump_response(AgentComposerValidateResponse, {"result": "success", "errors": [], **findings})
 
@@ -303,5 +314,6 @@ class AgentComposerCandidatesApi(Resource):
                 tenant_id=tenant_id,
                 agent_id=str(agent_id),
                 user_id=current_user_id,
+                session=db.session(),
             ),
         )

api/controllers/console/agent/roster.py

@@ -534,7 +534,7 @@ class AgentAppListApi(Resource):
             status="normal",
         )
 
-        app_pagination = AppService().get_paginate_apps(current_user.id, current_tenant_id, params, db.session)
+        app_pagination = AppService().get_paginate_apps(current_user.id, current_tenant_id, params, db.session())
         if app_pagination is None:
             empty = AgentAppPagination(page=args.page, limit=args.limit, total=0, has_more=False, data=[])
             return empty.model_dump(mode="json")
@@ -567,7 +567,7 @@ class AgentAppListApi(Resource):
             icon_background=args.icon_background,
         )
 
-        app = AppService().create_app(current_tenant_id, params, current_user)
+        app = AppService().create_app(current_tenant_id, params, current_user, session=db.session())
         return _serialize_agent_app_detail(app, current_user=current_user), 201
 
 
@@ -607,7 +607,7 @@ class AgentAppApi(Resource):
             "max_active_requests": args.max_active_requests or 0,
             "role": args.role,
         }
-        updated = AppService().update_app(app_model, args_dict)
+        updated = AppService().update_app(app_model, args_dict, session=db.session())
         return _serialize_agent_app_detail(updated, current_user=current_user)
 
     @console_ns.response(204, "Agent app deleted successfully")
@@ -619,7 +619,7 @@ class AgentAppApi(Resource):
     @with_current_tenant_id
     def delete(self, tenant_id: str, agent_id: UUID):
         app_model = _resolve_agent_app_model(tenant_id=tenant_id, agent_id=agent_id)
-        AppService().delete_app(app_model)
+        AppService().delete_app(app_model, session=db.session())
         return "", 204
 
 
@@ -668,6 +668,7 @@ class AgentPublishApi(Resource):
             agent_id=str(agent_id),
             account_id=current_user.id,
             version_note=args.version_note,
+            session=db.session(),
         )
 
 
@@ -688,6 +689,7 @@ class AgentBuildDraftCheckoutApi(Resource):
             agent_id=str(agent_id),
             account_id=current_user.id,
             force=args.force,
+            session=db.session(),
         )
 
 
@@ -705,6 +707,7 @@ class AgentBuildDraftApi(Resource):
             tenant_id=tenant_id,
             agent_id=str(agent_id),
             account_id=current_user.id,
+            session=db.session(),
         )
 
     @console_ns.expect(console_ns.models[ComposerSavePayload.__name__])
@@ -722,6 +725,7 @@ class AgentBuildDraftApi(Resource):
             agent_id=str(agent_id),
             account_id=current_user.id,
             payload=payload,
+            session=db.session(),
         )
 
     @console_ns.response(200, "Agent build draft discarded", console_ns.models[AgentSimpleResultResponse.__name__])
@@ -736,6 +740,7 @@ class AgentBuildDraftApi(Resource):
             tenant_id=tenant_id,
             agent_id=str(agent_id),
             account_id=current_user.id,
+            session=db.session(),
         )
 
 
@@ -753,6 +758,7 @@ class AgentBuildDraftApplyApi(Resource):
             tenant_id=tenant_id,
             agent_id=str(agent_id),
             account_id=current_user.id,
+            session=db.session(),
         )
 
 
@@ -810,7 +816,7 @@ class AgentApiStatusApi(Resource):
     def post(self, tenant_id: str, agent_id: UUID):
         app_model = _resolve_agent_app_model(tenant_id=tenant_id, agent_id=agent_id)
         args = AgentApiStatusPayload.model_validate(console_ns.payload)
-        app_model = AppService().update_app_api_status(app_model, args.enable_api)
+        app_model = AppService().update_app_api_status(app_model, args.enable_api, session=db.session())
         return _serialize_agent_api_access(app_model)
 
 

api/controllers/console/app/agent.py

@@ -172,7 +172,7 @@ register_response_schema_models(
 def _resolve_agent_id(app_model: App, node_id: str | None) -> str | None:
     if node_id and app_model.mode != AppMode.AGENT:
         return AgentComposerService.resolve_workflow_node_agent_id(
-            tenant_id=app_model.tenant_id, app_id=app_model.id, node_id=node_id
+            tenant_id=app_model.tenant_id, app_id=app_model.id, node_id=node_id, session=db.session()
         )
     return app_model.bound_agent_id
 
@@ -202,6 +202,7 @@ def _upload_skill_for_app(*, current_user: Account, app_model: App):
             tenant_id=app_model.tenant_id,
             user_id=current_user.id,
             agent_id=agent_id,
+            session=db.session(),
         )
     except (SkillPackageError, AgentDriveError) as exc:
         return {"code": exc.code, "message": exc.message}, exc.status_code
@@ -240,6 +241,7 @@ def _commit_drive_file_for_app(*, current_user: Account, app_model: App, allow_n
                     value_owned_by_drive=True,
                 )
             ],
+            session=db.session(),
         )
     except AgentDriveError as exc:
         return {"code": exc.code, "message": exc.message}, exc.status_code
@@ -273,6 +275,7 @@ def _delete_drive_file_for_app(*, current_user: Account, app_model: App, allow_n
             user_id=current_user.id,
             agent_id=agent_id,
             items=[DriveCommitItem(key=key, file_ref=None)],
+            session=db.session(),
         )
     except AgentDriveError as exc:
         return {"code": exc.code, "message": exc.message}, exc.status_code
@@ -298,6 +301,7 @@ def _delete_skill_for_app(*, current_user: Account, app_model: App, slug: str, a
                 DriveCommitItem(key=f"{slug}/SKILL.md", file_ref=None),
                 DriveCommitItem(key=f"{slug}/.DIFY-SKILL-FULL.zip", file_ref=None),
             ],
+            session=db.session(),
         )
     except AgentDriveError as exc:
         return {"code": exc.code, "message": exc.message}, exc.status_code
@@ -313,7 +317,9 @@ def _infer_skill_tools_for_app(*, app_model: App, slug: str):
     if "/" in slug or not slug.strip():
         return {"code": "drive_key_invalid", "message": "skill slug must be a single path segment"}, 400
     try:
-        return SkillToolInferenceService().infer(tenant_id=app_model.tenant_id, agent_id=agent_id, slug=slug)
+        return SkillToolInferenceService().infer(
+            tenant_id=app_model.tenant_id, agent_id=agent_id, slug=slug, session=db.session()
+        )
     except SkillToolInferenceError as exc:
         return {"code": exc.code, "message": exc.message}, exc.status_code
 
@@ -335,7 +341,7 @@ class AgentLogApi(Resource):
         """Get agent logs"""
         args = AgentLogQuery.model_validate(request.args.to_dict(flat=True))
 
-        return AgentService.get_agent_logs(app_model, args.conversation_id, args.message_id)
+        return AgentService.get_agent_logs(app_model, args.conversation_id, args.message_id, db.session())
 
 
 @console_ns.route("/agent/<uuid:agent_id>/skills/upload")

api/controllers/console/app/agent_app_feature.py

@@ -93,7 +93,7 @@ class AgentAppFeatureConfigResource(Resource):
             app_model=app_model,
             account=current_user,
             config=args.model_dump(exclude_none=True),
-            session=db.session,
+            session=db.session(),
         )
 
         app_model_config_was_updated.send(app_model, app_model_config=new_app_model_config)

api/controllers/console/app/agent_app_sandbox.py

@@ -25,6 +25,7 @@ from controllers.console import console_ns
 from controllers.console.agent.app_helpers import resolve_agent_runtime_app_model
 from controllers.console.app.wraps import get_app_model
 from controllers.console.wraps import account_initialization_required, setup_required, with_current_tenant_id
+from extensions.ext_database import db
 from fields.base import ResponseModel
 from libs.login import login_required
 from models.model import App, AppMode
@@ -269,6 +270,7 @@ class WorkflowAgentSandboxListResource(Resource):
                 node_id=node_id,
                 node_execution_id=query.node_execution_id,
                 path=query.path,
+                session=db.session(),
             )
         except Exception as exc:
             return _handle(exc)
@@ -305,6 +307,7 @@ class WorkflowAgentSandboxReadResource(Resource):
                 node_id=node_id,
                 node_execution_id=query.node_execution_id,
                 path=query.path,
+                session=db.session(),
             )
         except Exception as exc:
             return _handle(exc)
@@ -334,6 +337,7 @@ class WorkflowAgentSandboxUploadResource(Resource):
                 node_id=node_id,
                 node_execution_id=payload.node_execution_id,
                 path=payload.path,
+                session=db.session(),
             )
         except Exception as exc:
             return _handle(exc)

api/controllers/console/app/agent_config_inspector.py

@@ -253,6 +253,7 @@ def _resolve_agent_id(app_model: App, node_id: str | None) -> str | None:
             tenant_id=app_model.tenant_id,
             app_id=app_model.id,
             node_id=node_id,
+            session=db.session(),
         )
     return app_model.bound_agent_id
 
@@ -288,13 +289,16 @@ def _resolve_console_version(
                 tenant_id=tenant_id,
                 agent_id=agent_id,
                 account_id=account_id,
+                session=db.session(),
             )
             draft = state.get("draft") or {}
             draft_id = draft.get("id")
             if isinstance(draft_id, str) and draft_id:
                 return draft_id, AgentConfigVersionKind.BUILD_DRAFT
         else:
-            state = AgentComposerService.load_agent_composer(tenant_id=tenant_id, agent_id=agent_id)
+            state = AgentComposerService.load_agent_composer(
+                tenant_id=tenant_id, agent_id=agent_id, session=db.session()
+            )
             draft = state.get("draft") or {}
             draft_id = draft.get("id")
             if isinstance(draft_id, str) and draft_id:

api/controllers/console/app/agent_drive_inspector.py

@@ -28,6 +28,7 @@ from controllers.console import console_ns
 from controllers.console.agent.app_helpers import resolve_agent_runtime_app_model
 from controllers.console.app.wraps import get_app_model
 from controllers.console.wraps import account_initialization_required, setup_required, with_current_tenant_id
+from extensions.ext_database import db
 from fields.base import ResponseModel
 from libs.login import login_required
 from models.model import App, AppMode
@@ -147,7 +148,7 @@ def _resolve_agent_id(app_model: App, node_id: str | None) -> str | None:
     """Agent identity for the drive: app-bound agent, or the workflow node binding."""
     if node_id:
         return AgentComposerService.resolve_workflow_node_agent_id(
-            tenant_id=app_model.tenant_id, app_id=app_model.id, node_id=node_id
+            tenant_id=app_model.tenant_id, app_id=app_model.id, node_id=node_id, session=db.session()
         )
     return app_model.bound_agent_id
 
@@ -184,7 +185,9 @@ class AgentDriveListByAgentApi(Resource):
         query = query_params_from_request(AgentDriveListByAgentQuery)
         resolve_agent_runtime_app_model(tenant_id=tenant_id, agent_id=agent_id)
         try:
-            items = AgentDriveService().manifest(tenant_id=tenant_id, agent_id=str(agent_id), prefix=query.prefix)
+            items = AgentDriveService().manifest(
+                tenant_id=tenant_id, agent_id=str(agent_id), prefix=query.prefix, session=db.session()
+            )
         except AgentDriveError as exc:
             return _handle(exc)
         return {"items": [{k: v for k, v in item.items() if k != "file_id"} for item in items]}
@@ -203,7 +206,7 @@ class AgentDriveSkillListByAgentApi(Resource):
     def get(self, tenant_id: str, agent_id: UUID):
         resolve_agent_runtime_app_model(tenant_id=tenant_id, agent_id=agent_id)
         try:
-            items = AgentDriveService().list_skills(tenant_id=tenant_id, agent_id=str(agent_id))
+            items = AgentDriveService().list_skills(tenant_id=tenant_id, agent_id=str(agent_id), session=db.session())
         except AgentDriveError as exc:
             return _handle(exc)
         return {"items": items}
@@ -227,6 +230,7 @@ class AgentDriveSkillInspectByAgentApi(Resource):
                     tenant_id=tenant_id,
                     agent_id=str(agent_id),
                     skill_path=skill_path,
+                    session=db.session(),
                 )
             )
         except AgentDriveError as exc:
@@ -247,7 +251,9 @@ class AgentDrivePreviewByAgentApi(Resource):
         query = query_params_from_request(AgentDriveFileByAgentQuery)
         resolve_agent_runtime_app_model(tenant_id=tenant_id, agent_id=agent_id)
         try:
-            return AgentDriveService().preview(tenant_id=tenant_id, agent_id=str(agent_id), key=query.key)
+            return AgentDriveService().preview(
+                tenant_id=tenant_id, agent_id=str(agent_id), key=query.key, session=db.session()
+            )
         except AgentDriveError as exc:
             return _handle(exc)
 
@@ -266,7 +272,9 @@ class AgentDriveDownloadByAgentApi(Resource):
         query = query_params_from_request(AgentDriveFileByAgentQuery)
         resolve_agent_runtime_app_model(tenant_id=tenant_id, agent_id=agent_id)
         try:
-            url = AgentDriveService().download_url(tenant_id=tenant_id, agent_id=str(agent_id), key=query.key)
+            url = AgentDriveService().download_url(
+                tenant_id=tenant_id, agent_id=str(agent_id), key=query.key, session=db.session()
+            )
         except AgentDriveError as exc:
             return _handle(exc)
         return {"url": url}
@@ -288,7 +296,9 @@ class AgentDriveListApi(Resource):
         if not agent_id:
             return _agent_not_bound()
         try:
-            items = AgentDriveService().manifest(tenant_id=app_model.tenant_id, agent_id=agent_id, prefix=query.prefix)
+            items = AgentDriveService().manifest(
+                tenant_id=app_model.tenant_id, agent_id=agent_id, prefix=query.prefix, session=db.session()
+            )
         except AgentDriveError as exc:
             return _handle(exc)
         # the inner manifest exposes file_id for agent-side pulls; the console
@@ -312,7 +322,9 @@ class AgentDriveSkillListApi(Resource):
         if not agent_id:
             return _agent_not_bound()
         try:
-            items = AgentDriveService().list_skills(tenant_id=app_model.tenant_id, agent_id=agent_id)
+            items = AgentDriveService().list_skills(
+                tenant_id=app_model.tenant_id, agent_id=agent_id, session=db.session()
+            )
         except AgentDriveError as exc:
             return _handle(exc)
         return {"items": items}
@@ -345,6 +357,7 @@ class AgentDriveSkillInspectApi(Resource):
                     tenant_id=app_model.tenant_id,
                     agent_id=agent_id,
                     skill_path=skill_path,
+                    session=db.session(),
                 )
             )
         except AgentDriveError as exc:
@@ -367,7 +380,9 @@ class AgentDrivePreviewApi(Resource):
         if not agent_id:
             return _agent_not_bound()
         try:
-            return AgentDriveService().preview(tenant_id=app_model.tenant_id, agent_id=agent_id, key=query.key)
+            return AgentDriveService().preview(
+                tenant_id=app_model.tenant_id, agent_id=agent_id, key=query.key, session=db.session()
+            )
         except AgentDriveError as exc:
             return _handle(exc)
 
@@ -388,7 +403,9 @@ class AgentDriveDownloadApi(Resource):
         if not agent_id:
             return _agent_not_bound()
         try:
-            url = AgentDriveService().download_url(tenant_id=app_model.tenant_id, agent_id=agent_id, key=query.key)
+            url = AgentDriveService().download_url(
+                tenant_id=app_model.tenant_id, agent_id=agent_id, key=query.key, session=db.session()
+            )
         except AgentDriveError as exc:
             return _handle(exc)
         return {"url": url}

api/controllers/console/app/annotation.py

@@ -211,7 +211,7 @@ class AppAnnotationSettingDetailApi(Resource):
     @edit_permission_required
     @rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_VIEW_LAYOUT)
     def get(self, app_id: UUID):
-        result = AppAnnotationService.get_app_annotation_setting_by_app_id(str(app_id))
+        result = AppAnnotationService.get_app_annotation_setting_by_app_id(str(app_id), session=db.session())
         return dump_response(AnnotationSettingResponse, result), 200
 
 
@@ -235,7 +235,7 @@ class AppAnnotationSettingUpdateApi(Resource):
 
         setting_args: UpdateAnnotationSettingArgs = {"score_threshold": args.score_threshold}
         result = AppAnnotationService.update_app_annotation_setting(
-            str(app_id), annotation_setting_id_str, setting_args
+            str(app_id), annotation_setting_id_str, setting_args, session=db.session()
         )
         return dump_response(AnnotationSettingResponse, result), 200
 
@@ -292,7 +292,9 @@ class AnnotationApi(Resource):
         limit = args.limit
         keyword = args.keyword
 
-        annotation_list, total = AppAnnotationService.get_annotation_list_by_app_id(str(app_id), page, limit, keyword)
+        annotation_list, total = AppAnnotationService.get_annotation_list_by_app_id(
+            str(app_id), page, limit, keyword, session=db.session()
+        )
         annotation_models = TypeAdapter(list[Annotation]).validate_python(annotation_list, from_attributes=True)
         return AnnotationList(
             data=annotation_models, has_more=len(annotation_list) == limit, limit=limit, total=total, page=page
@@ -321,7 +323,9 @@ class AnnotationApi(Resource):
             upsert_args["message_id"] = args.message_id
         if args.question is not None:
             upsert_args["question"] = args.question
-        annotation = AppAnnotationService.up_insert_app_annotation_from_message(upsert_args, str(app_id))
+        annotation = AppAnnotationService.up_insert_app_annotation_from_message(
+            upsert_args, str(app_id), session=db.session()
+        )
         return dump_response(Annotation, annotation), 201
 
     @setup_required
@@ -345,11 +349,11 @@ class AnnotationApi(Resource):
                 }, 400
 
             app_ref = _get_app_ref(str(app_id))
-            AppAnnotationService.delete_app_annotations_in_batch(app_ref, annotation_ids)
+            AppAnnotationService.delete_app_annotations_in_batch(app_ref, annotation_ids, session=db.session())
             return "", 204
         # If no annotation_ids are provided, handle clearing all annotations
         else:
-            AppAnnotationService.clear_all_annotations(str(app_id))
+            AppAnnotationService.clear_all_annotations(str(app_id), session=db.session())
             return "", 204
 
 
@@ -370,7 +374,7 @@ class AnnotationExportApi(Resource):
     @edit_permission_required
     @rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_VIEW_LAYOUT)
     def get(self, app_id: UUID):
-        annotation_list = AppAnnotationService.export_annotation_list_by_app_id(str(app_id))
+        annotation_list = AppAnnotationService.export_annotation_list_by_app_id(str(app_id), session=db.session())
         annotation_models = TypeAdapter(list[Annotation]).validate_python(annotation_list, from_attributes=True)
         return (
             AnnotationExportList(data=annotation_models).model_dump(mode="json"),
@@ -406,7 +410,7 @@ class AnnotationUpdateDeleteApi(Resource):
             update_args["question"] = args.question
         app_ref = _get_app_ref(str(app_id))
         annotation_ref = AppRefService.create_annotation_ref(app_ref, str(annotation_id))
-        annotation = AppAnnotationService.update_app_annotation_directly(update_args, annotation_ref, db.session)
+        annotation = AppAnnotationService.update_app_annotation_directly(update_args, annotation_ref, db.session())
         return Annotation.model_validate(annotation, from_attributes=True).model_dump(mode="json")
 
     @setup_required
@@ -418,7 +422,7 @@ class AnnotationUpdateDeleteApi(Resource):
     def delete(self, app_id: UUID, annotation_id: UUID):
         app_ref = _get_app_ref(str(app_id))
         annotation_ref = AppRefService.create_annotation_ref(app_ref, str(annotation_id))
-        AppAnnotationService.delete_app_annotation(annotation_ref, db.session)
+        AppAnnotationService.delete_app_annotation(annotation_ref, db.session())
         return "", 204
 
 
@@ -477,7 +481,7 @@ class AnnotationBatchImportApi(Resource):
 
         return dump_response(
             AnnotationBatchImportResponse,
-            AppAnnotationService.batch_import_app_annotations(str(app_id), file),
+            AppAnnotationService.batch_import_app_annotations(str(app_id), file, session=db.session()),
         )
 
 
@@ -538,6 +542,7 @@ class AnnotationHitHistoryListApi(Resource):
             annotation_ref,
             page,
             limit,
+            session=db.session(),
         )
         history_models = TypeAdapter(list[AnnotationHitHistory]).validate_python(
             annotation_hit_history_list, from_attributes=True

api/controllers/console/app/app.py

@@ -584,6 +584,7 @@ class AppListApi(Resource):
         permissions = enterprise_rbac_service.RBACService.MyPermissions.get(
             str(current_tenant_id),
             current_user_id,
+            session=db.session(),
         )
         if dify_config.RBAC_ENABLED:
             access_filter = resolve_app_access_filter(
@@ -595,7 +596,7 @@ class AppListApi(Resource):
 
         # get app list
         app_service = AppService()
-        app_pagination = app_service.get_paginate_apps(current_user_id, current_tenant_id, params, db.session)
+        app_pagination = app_service.get_paginate_apps(current_user_id, current_tenant_id, params, session)
         if not app_pagination:
             response = AppPagination(page=args.page, limit=args.limit, total=0, has_more=False, data=[])
             return response.model_dump(mode="json"), 200
@@ -643,11 +644,12 @@ class AppListApi(Resource):
         )
 
         app_service = AppService()
-        app = app_service.create_app(current_tenant_id, params, current_user)
+        app = app_service.create_app(current_tenant_id, params, current_user, session=db.session())
         permission_keys_map = enterprise_rbac_service.RBACService.AppPermissions.batch_get(
             str(current_tenant_id),
             current_user.id,
             [str(app.id)],
+            session=db.session(),
         )
         app_detail = AppDetailWithSite.model_validate(app, from_attributes=True).model_copy(
             update={"permission_keys": permission_keys_map.get(str(app.id), [])}
@@ -681,7 +683,7 @@ class StarredAppListApi(Resource):
             is_created_by_me=args.is_created_by_me,
         )
 
-        app_pagination = AppService().get_paginate_starred_apps(current_user_id, current_tenant_id, params, db.session)
+        app_pagination = AppService().get_paginate_starred_apps(current_user_id, current_tenant_id, params, session)
         if not app_pagination:
             empty = AppPagination(page=args.page, limit=args.limit, total=0, has_more=False, data=[])
             return empty.model_dump(mode="json"), 200
@@ -705,7 +707,7 @@ class AppStarApi(Resource):
     @with_session
     @get_app_model(mode=None)
     def post(self, session: Session, current_user_id: str, app_model: App):
-        AppService.star_app(session, app=app_model, account_id=current_user_id)
+        AppService.star_app(app=app_model, account_id=current_user_id, session=session)
         return SimpleResultResponse(result="success").model_dump(mode="json")
 
     @console_ns.doc("unstar_app")
@@ -721,7 +723,7 @@ class AppStarApi(Resource):
     @with_session
     @get_app_model(mode=None)
     def delete(self, session: Session, current_user_id: str, app_model: App):
-        AppService.unstar_app(session, app=app_model, account_id=current_user_id)
+        AppService.unstar_app(app=app_model, account_id=current_user_id, session=session)
         return SimpleResultResponse(result="success").model_dump(mode="json")
 
 
@@ -753,6 +755,7 @@ class AppApi(Resource):
             str(current_tenant_id),
             current_user.id,
             app_id=str(app_model.id),
+            session=db.session(),
         )
         permission_keys_map = permissions.app.permission_keys_by_resource_ids([str(app_model.id)])
 
@@ -789,7 +792,7 @@ class AppApi(Resource):
             "use_icon_as_answer_icon": args.use_icon_as_answer_icon or False,
             "max_active_requests": args.max_active_requests or 0,
         }
-        app_model = app_service.update_app(app_model, args_dict)
+        app_model = app_service.update_app(app_model, args_dict, session=db.session())
         return dump_response(AppDetailWithSite, app_model)
 
     @console_ns.doc("delete_app")
@@ -806,7 +809,7 @@ class AppApi(Resource):
     def delete(self, app_model: App):
         """Delete app"""
         app_service = AppService()
-        app_service.delete_app(app_model)
+        app_service.delete_app(app_model, session=db.session())
 
         return "", 204
 
@@ -835,7 +838,7 @@ class AppCopyApi(Resource):
 
         with Session(db.engine, expire_on_commit=False) as session:
             import_service = AppDslService(session)
-            yaml_content = import_service.export_dsl(app_model=app_model, include_secret=True)
+            yaml_content = import_service.export_dsl(app_model=app_model, session=session, include_secret=True)
             result = import_service.import_app(
                 account=current_user,
                 import_mode=ImportMode.YAML_CONTENT,
@@ -877,6 +880,7 @@ class AppCopyApi(Resource):
             str(current_tenant_id),
             current_user.id,
             [str(app.id)],
+            session=db.session(),
         )
         response_model = AppDetailWithSite.model_validate(app, from_attributes=True).model_copy(
             update={"permission_keys": permission_keys_map.get(str(app.id), [])}
@@ -905,6 +909,7 @@ class AppExportApi(Resource):
         response = AppExportResponse(
             data=AppDslService.export_dsl(
                 app_model=app_model,
+                session=db.session(),
                 include_secret=args.include_secret,
                 workflow_id=args.workflow_id,
             )
@@ -929,7 +934,7 @@ class AppPublishToCreatorsPlatformApi(Resource):
         if not dify_config.CREATORS_PLATFORM_FEATURES_ENABLED:
             return {"error": "Creators Platform features are not enabled"}, 403
 
-        dsl_content = AppDslService.export_dsl(app_model=app_model, include_secret=False)
+        dsl_content = AppDslService.export_dsl(app_model=app_model, session=db.session(), include_secret=False)
         dsl_bytes = dsl_content.encode("utf-8")
 
         claim_code = upload_dsl(dsl_bytes)
@@ -955,7 +960,7 @@ class AppNameApi(Resource):
         args = AppNamePayload.model_validate(console_ns.payload)
 
         app_service = AppService()
-        app_model = app_service.update_app_name(app_model, args.name)
+        app_model = app_service.update_app_name(app_model, args.name, session=db.session())
         return dump_response(AppDetail, app_model)
 
 
@@ -982,6 +987,7 @@ class AppIconApi(Resource):
             args.icon or "",
             args.icon_background or "",
             args.icon_type,
+            session=db.session(),
         )
         return dump_response(AppDetail, app_model)
 
@@ -1004,7 +1010,7 @@ class AppSiteStatus(Resource):
         args = AppSiteStatusPayload.model_validate(console_ns.payload)
 
         app_service = AppService()
-        app_model = app_service.update_app_site_status(app_model, args.enable_site)
+        app_model = app_service.update_app_site_status(app_model, args.enable_site, session=db.session())
         return dump_response(AppDetail, app_model)
 
 
@@ -1026,7 +1032,7 @@ class AppApiStatus(Resource):
         args = AppApiStatusPayload.model_validate(console_ns.payload)
 
         app_service = AppService()
-        app_model = app_service.update_app_api_status(app_model, args.enable_api)
+        app_model = app_service.update_app_api_status(app_model, args.enable_api, session=db.session())
         return dump_response(AppDetail, app_model)
 
 

api/controllers/console/app/audio.py

@@ -161,7 +161,7 @@ class ChatMessageTextApi(Resource):
             # response-contract:ignore
             return AudioService.transcript_tts(
                 app_model=app_model,
-                session=db.session,
+                session=db.session(),
                 text=payload.text,
                 voice=payload.voice,
                 message_ref=message_ref,

api/controllers/console/app/completion.py

@@ -36,7 +36,7 @@ from controllers.console.wraps import (
     with_current_user_id,
 )
 from controllers.web.error import InvokeRateLimitError as InvokeRateLimitHttpError
-from core.app.entities.app_invoke_entities import InvokeFrom
+from core.app.entities.app_invoke_entities import AGENT_RUNTIME_EXIT_INTENT_ARG, InvokeFrom
 from core.app.features.rate_limiting.rate_limit import RateLimitGenerator
 from core.errors.error import (
     ModelCurrentlyNotSupportError,
@@ -416,6 +416,7 @@ def _create_build_chat_finalization_message(
         "draft_type": "debug_build",
         "conversation_id": debug_conversation_id,
         "auto_generate_name": False,
+        AGENT_RUNTIME_EXIT_INTENT_ARG: "delete",
     }
     external_trace_id = get_external_trace_id(request)
     if external_trace_id:

api/controllers/console/app/conversation.py

@@ -200,7 +200,7 @@ class CompletionConversationDetailApi(Resource):
         conversation_id_str = str(conversation_id)
 
         try:
-            ConversationService.delete(app_model, conversation_id_str, current_user)
+            ConversationService.delete(app_model, conversation_id_str, current_user, session=db.session())
         except ConversationNotExistsError:
             raise NotFound("Conversation Not Exists.")
 
@@ -354,7 +354,7 @@ class ChatConversationDetailApi(Resource):
         conversation_id_str = str(conversation_id)
 
         try:
-            ConversationService.delete(app_model, conversation_id_str, current_user)
+            ConversationService.delete(app_model, conversation_id_str, current_user, session=db.session())
         except ConversationNotExistsError:
             raise NotFound("Conversation Not Exists.")
 

api/controllers/console/app/message.py

@@ -363,6 +363,7 @@ def _list_chat_messages(*, app_model: App, current_user: Account | None = None):
                 app_model=app_model,
                 conversation_id=args.conversation_id,
                 user=current_user,
+                session=db.session(),
             )
         except ConversationNotExistsError:
             raise NotFound("Conversation Not Exists.")
@@ -474,7 +475,11 @@ def _get_message_suggested_questions(*, current_user: Account, app_model: App, m
 
     try:
         questions = MessageService.get_suggested_questions_after_answer(
-            app_model=app_model, message_id=message_id_str, user=current_user, invoke_from=InvokeFrom.DEBUGGER
+            app_model=app_model,
+            message_id=message_id_str,
+            user=current_user,
+            invoke_from=InvokeFrom.DEBUGGER,
+            session=db.session(),
         )
     except MessageNotExistsError:
         raise NotFound("Message not found")

api/controllers/console/app/ops_trace.py

@@ -17,6 +17,7 @@ from controllers.console.wraps import (
     rbac_permission_required,
     setup_required,
 )
+from extensions.ext_database import db
 from fields.base import ResponseModel
 from libs.login import login_required
 from models import App
@@ -78,7 +79,7 @@ class TraceAppConfigApi(Resource):
 
         try:
             trace_config = OpsService.get_tracing_app_config(
-                app_id=app_model.id, tracing_provider=args.tracing_provider
+                app_id=app_model.id, tracing_provider=args.tracing_provider, session=db.session()
             )
             if not trace_config:
                 return {"has_not_configured": True}
@@ -109,7 +110,10 @@ class TraceAppConfigApi(Resource):
 
         try:
             result = OpsService.create_tracing_app_config(
-                app_id=app_model.id, tracing_provider=args.tracing_provider, tracing_config=args.tracing_config
+                app_id=app_model.id,
+                tracing_provider=args.tracing_provider,
+                tracing_config=args.tracing_config,
+                session=db.session(),
             )
             if not result:
                 raise TracingConfigIsExist()
@@ -142,7 +146,10 @@ class TraceAppConfigApi(Resource):
 
         try:
             result = OpsService.update_tracing_app_config(
-                app_id=app_model.id, tracing_provider=args.tracing_provider, tracing_config=args.tracing_config
+                app_id=app_model.id,
+                tracing_provider=args.tracing_provider,
+                tracing_config=args.tracing_config,
+                session=db.session(),
             )
             if not result:
                 raise TracingConfigNotExist()
@@ -168,7 +175,9 @@ class TraceAppConfigApi(Resource):
         args = TraceProviderQuery.model_validate(request.args.to_dict(flat=True))
 
         try:
-            result = OpsService.delete_tracing_app_config(app_id=app_model.id, tracing_provider=args.tracing_provider)
+            result = OpsService.delete_tracing_app_config(
+                app_id=app_model.id, tracing_provider=args.tracing_provider, session=db.session()
+            )
             if not result:
                 raise TracingConfigNotExist()
             return "", 204

api/controllers/console/app/permission_keys.py

@@ -1,6 +1,9 @@
+from extensions.ext_database import db
 from services.enterprise import rbac_service as enterprise_rbac_service
 
 
 def get_app_permission_keys(tenant_id: str, account_id: str | None, app_id: str) -> list[str]:
-    permission_keys_map = enterprise_rbac_service.RBACService.AppPermissions.batch_get(tenant_id, account_id, [app_id])
+    permission_keys_map = enterprise_rbac_service.RBACService.AppPermissions.batch_get(
+        tenant_id, account_id, [app_id], session=db.session()
+    )
     return permission_keys_map.get(app_id, [])

api/controllers/console/app/workflow.py

@@ -2,7 +2,7 @@ import json
 import logging
 from collections.abc import Sequence
 from datetime import datetime
-from typing import Any, NotRequired, TypedDict, cast
+from typing import Any, NotRequired, TypedDict
 
 from flask import abort, request
 from flask_restx import Resource, fields
@@ -522,7 +522,7 @@ class DraftWorkflowApi(Resource):
         """
         # fetch draft workflow by app_model
         workflow_service = WorkflowService()
-        workflow = workflow_service.get_draft_workflow(app_model=app_model)
+        workflow = workflow_service.get_draft_workflow(app_model=app_model, session=db.session())
 
         if not workflow:
             raise DraftWorkflowNotExist()
@@ -533,7 +533,7 @@ class DraftWorkflowApi(Resource):
         # front-end can treat draft graph node data as the editing source.
         response = WorkflowResponse.model_validate(workflow, from_attributes=True).model_dump(mode="json")
         response["graph"] = WorkflowAgentPublishService.project_draft_bindings_to_graph(
-            session=cast(Session, db.session),
+            session=db.session(),
             draft_workflow=workflow,
         )
         return response
@@ -602,6 +602,7 @@ class DraftWorkflowApi(Resource):
                 account=current_user,
                 environment_variables=environment_variables,
                 conversation_variables=conversation_variables,
+                session=db.session(),
             )
         except WorkflowHashNotEqualError:
             raise DraftWorkflowNotSync()
@@ -695,7 +696,12 @@ class AdvancedChatDraftRunIterationNodeApi(Resource):
 
         try:
             response = AppGenerateService.generate_single_iteration(
-                app_model=app_model, user=current_user, node_id=node_id, args=args, streaming=True
+                app_model=app_model,
+                user=current_user,
+                node_id=node_id,
+                args=args,
+                session=db.session(),
+                streaming=True,
             )
 
             return helper.compact_generate_response(response)
@@ -738,7 +744,12 @@ class WorkflowDraftRunIterationNodeApi(Resource):
 
         try:
             response = AppGenerateService.generate_single_iteration(
-                app_model=app_model, user=current_user, node_id=node_id, args=args, streaming=True
+                app_model=app_model,
+                user=current_user,
+                node_id=node_id,
+                args=args,
+                session=db.session(),
+                streaming=True,
             )
 
             return helper.compact_generate_response(response)
@@ -777,7 +788,12 @@ class AdvancedChatDraftRunLoopNodeApi(Resource):
 
         try:
             response = AppGenerateService.generate_single_loop(
-                app_model=app_model, user=current_user, node_id=node_id, args=args, streaming=True
+                app_model=app_model,
+                user=current_user,
+                node_id=node_id,
+                args=args,
+                session=db.session(),
+                streaming=True,
             )
 
             return helper.compact_generate_response(response)
@@ -820,7 +836,12 @@ class WorkflowDraftRunLoopNodeApi(Resource):
 
         try:
             response = AppGenerateService.generate_single_loop(
-                app_model=app_model, user=current_user, node_id=node_id, args=args, streaming=True
+                app_model=app_model,
+                user=current_user,
+                node_id=node_id,
+                args=args,
+                session=db.session(),
+                streaming=True,
             )
 
             return helper.compact_generate_response(response)
@@ -897,6 +918,7 @@ class AdvancedChatDraftHumanInputFormPreviewApi(Resource):
             account=current_user,
             node_id=node_id,
             inputs=inputs,
+            session=db.session(),
         )
         return jsonable_encoder(preview)
 
@@ -932,6 +954,7 @@ class AdvancedChatDraftHumanInputFormRunApi(Resource):
             form_inputs=args.form_inputs,
             inputs=args.inputs,
             action=args.action,
+            session=db.session(),
         )
         return jsonable_encoder(result)
 
@@ -963,6 +986,7 @@ class WorkflowDraftHumanInputFormPreviewApi(Resource):
             account=current_user,
             node_id=node_id,
             inputs=inputs,
+            session=db.session(),
         )
         return jsonable_encoder(preview)
 
@@ -998,6 +1022,7 @@ class WorkflowDraftHumanInputFormRunApi(Resource):
             form_inputs=args.form_inputs,
             inputs=args.inputs,
             action=args.action,
+            session=db.session(),
         )
         return jsonable_encoder(result)
 
@@ -1028,6 +1053,7 @@ class WorkflowDraftHumanInputDeliveryTestApi(Resource):
             node_id=node_id,
             delivery_method_id=args.delivery_method_id,
             inputs=args.inputs,
+            session=db.session(),
         )
         return jsonable_encoder({})
 
@@ -1138,7 +1164,7 @@ class DraftWorkflowNodeRunApi(Resource):
 
         workflow_srv = WorkflowService()
         # fetch draft workflow by app_model
-        draft_workflow = workflow_srv.get_draft_workflow(app_model=app_model)
+        draft_workflow = workflow_srv.get_draft_workflow(app_model=app_model, session=db.session())
         if not draft_workflow:
             raise ValueError("Workflow not initialized")
         files = _parse_file(draft_workflow, args.get("files"))
@@ -1181,7 +1207,7 @@ class PublishedWorkflowApi(Resource):
         """
         # fetch published workflow by app_model
         workflow_service = WorkflowService()
-        workflow = workflow_service.get_published_workflow(app_model=app_model)
+        workflow = workflow_service.get_published_workflow(app_model=app_model, session=db.session())
 
         # return workflow, if not found, return None
         if workflow is None:
@@ -1323,7 +1349,9 @@ class ConvertToWorkflowApi(Resource):
 
         # convert to workflow mode
         workflow_service = WorkflowService()
-        new_app_model = workflow_service.convert_to_workflow(app_model=app_model, account=current_user, args=args)
+        new_app_model = workflow_service.convert_to_workflow(
+            app_model=app_model, account=current_user, args=args, session=db.session()
+        )
 
         # return app id
         return {
@@ -1358,7 +1386,9 @@ class WorkflowFeaturesApi(Resource):
         features = args.features.model_dump(mode="json", exclude_unset=True)
 
         workflow_service = WorkflowService()
-        workflow_service.update_draft_workflow_features(app_model=app_model, features=features, account=current_user)
+        workflow_service.update_draft_workflow_features(
+            app_model=app_model, features=features, account=current_user, session=db.session()
+        )
 
         return {"result": "success"}
 
@@ -1439,6 +1469,7 @@ class DraftWorkflowRestoreApi(Resource):
                 app_model=app_model,
                 workflow_id=workflow_id,
                 account=current_user,
+                session=db.session(),
             )
         except IsDraftWorkflowError as exc:
             raise BadRequest(RESTORE_SOURCE_WORKFLOW_MUST_BE_PUBLISHED_MESSAGE) from exc
@@ -1553,7 +1584,7 @@ class DraftWorkflowNodeLastRunApi(Resource):
     @get_app_model(mode=[AppMode.ADVANCED_CHAT, AppMode.WORKFLOW])
     def get(self, app_model: App, node_id: str):
         srv = WorkflowService()
-        workflow = srv.get_draft_workflow(app_model)
+        workflow = srv.get_draft_workflow(app_model, session=db.session())
         if not workflow:
             raise NotFound("Workflow not found")
         node_exec = srv.get_node_last_run(
@@ -1606,7 +1637,7 @@ class DraftWorkflowTriggerRunApi(Resource):
         args = DraftWorkflowTriggerRunPayload.model_validate(console_ns.payload or {})
         node_id = args.node_id
         workflow_service = WorkflowService()
-        draft_workflow = workflow_service.get_draft_workflow(app_model)
+        draft_workflow = workflow_service.get_draft_workflow(app_model, session=db.session())
         if not draft_workflow:
             raise ValueError("Workflow not found")
 
@@ -1675,7 +1706,7 @@ class DraftWorkflowTriggerNodeApi(Resource):
         """
 
         workflow_service = WorkflowService()
-        draft_workflow = workflow_service.get_draft_workflow(app_model)
+        draft_workflow = workflow_service.get_draft_workflow(app_model, session=db.session())
         if not draft_workflow:
             raise ValueError("Workflow not found")
 
@@ -1759,7 +1790,7 @@ class DraftWorkflowTriggerRunAllApi(Resource):
         args = DraftWorkflowTriggerRunAllPayload.model_validate(console_ns.payload or {})
         node_ids = args.node_ids
         workflow_service = WorkflowService()
-        draft_workflow = workflow_service.get_draft_workflow(app_model)
+        draft_workflow = workflow_service.get_draft_workflow(app_model, session=db.session())
         if not draft_workflow:
             raise ValueError("Workflow not found")
 
@@ -1828,7 +1859,7 @@ class WorkflowOnlineUsersApi(Resource):
             return {"data": []}
 
         workflow_service = WorkflowService()
-        accessible_app_ids = workflow_service.get_accessible_app_ids(app_ids, current_tenant_id)
+        accessible_app_ids = workflow_service.get_accessible_app_ids(app_ids, current_tenant_id, session=db.session())
         ordered_accessible_app_ids = [app_id for app_id in app_ids if app_id in accessible_app_ids]
 
         users_json_by_app_id: dict[str, Any] = {}

api/controllers/console/app/workflow_comment.py

@@ -490,7 +490,7 @@ class WorkflowCommentMentionUsersApi(Resource):
         current_tenant = current_user.current_tenant  # need the tenant object here
         if current_tenant is None:
             raise ValueError("current tenant is required")
-        members = TenantService.get_tenant_members(current_tenant, session=db.session)
+        members = TenantService.get_tenant_members(current_tenant, session=db.session())
         users = TypeAdapter(list[AccountWithRole]).validate_python(members, from_attributes=True)
         response = WorkflowCommentMentionUsersPayload(users=users)
         return response.model_dump(mode="json"), 200

api/controllers/console/app/workflow_draft_variable.py

@@ -337,7 +337,7 @@ class WorkflowVariableCollectionApi(Resource):
 
         # fetch draft workflow by app_model
         workflow_service = WorkflowService()
-        workflow_exist = workflow_service.is_workflow_exist(app_model=app_model)
+        workflow_exist = workflow_service.is_workflow_exist(app_model=app_model, session=db.session())
         if not workflow_exist:
             raise DraftWorkflowNotExist()
 
@@ -553,7 +553,7 @@ class VariableResetApi(Resource):
         )
 
         workflow_srv = WorkflowService()
-        draft_workflow = workflow_srv.get_draft_workflow(app_model)
+        draft_workflow = workflow_srv.get_draft_workflow(app_model, session=db.session())
         if draft_workflow is None:
             raise NotFoundError(
                 f"Draft workflow not found, app_id={app_model.id}",
@@ -606,7 +606,7 @@ class ConversationVariableCollectionApi(Resource):
         # NOTE(QuantumGhost): Prefill conversation variables into the draft variables table
         # so their IDs can be returned to the caller.
         workflow_srv = WorkflowService()
-        draft_workflow = workflow_srv.get_draft_workflow(app_model)
+        draft_workflow = workflow_srv.get_draft_workflow(app_model, session=db.session())
         if draft_workflow is None:
             raise NotFoundError(description=f"draft workflow not found, id={app_model.id}")
         draft_var_srv = WorkflowDraftVariableService(db.session())
@@ -646,6 +646,7 @@ class ConversationVariableCollectionApi(Resource):
             app_model=app_model,
             account=current_user,
             conversation_variables=conversation_variables,
+            session=db.session(),
         )
 
         return {"result": "success"}
@@ -683,7 +684,7 @@ class EnvironmentVariableCollectionApi(Resource):
         """
         # fetch draft workflow by app_model
         workflow_service = WorkflowService()
-        workflow = workflow_service.get_draft_workflow(app_model=app_model)
+        workflow = workflow_service.get_draft_workflow(app_model=app_model, session=db.session())
         if workflow is None:
             raise DraftWorkflowNotExist()
 
@@ -740,6 +741,7 @@ class EnvironmentVariableCollectionApi(Resource):
             app_model=app_model,
             account=current_user,
             environment_variables=environment_variables,
+            session=db.session(),
         )
 
         return {"result": "success"}

api/controllers/console/app/workflow_node_output_inspector.py

@@ -41,6 +41,7 @@ from controllers.console.wraps import (
     rbac_permission_required,
     setup_required,
 )
+from extensions.ext_database import db
 from libs.exception import BaseHTTPException
 from libs.login import login_required
 from models import App, AppMode
@@ -92,7 +93,9 @@ def _serve_snapshot(app_model: App, run_id: UUID) -> dict:
     Flask request context.
     """
     try:
-        snapshot = _service().snapshot_workflow_run(app_model=app_model, workflow_run_id=str(run_id))
+        snapshot = _service().snapshot_workflow_run(
+            app_model=app_model, workflow_run_id=str(run_id), session=db.session()
+        )
     except NodeOutputInspectorError as error:
         raise _InspectorNotFound(error) from error
     return snapshot.model_dump(mode="json")
@@ -105,6 +108,7 @@ def _serve_node_detail(app_model: App, run_id: UUID, node_id: str) -> dict:
             app_model=app_model,
             workflow_run_id=str(run_id),
             node_id=node_id,
+            session=db.session(),
         )
     except NodeOutputInspectorError as error:
         raise _InspectorNotFound(error) from error
@@ -119,6 +123,7 @@ def _serve_output_preview(app_model: App, run_id: UUID, node_id: str, output_nam
             workflow_run_id=str(run_id),
             node_id=node_id,
             output_name=output_name,
+            session=db.session(),
         )
     except NodeOutputInspectorError as error:
         raise _InspectorNotFound(error) from error
@@ -245,7 +250,7 @@ def _stream_inspector_events(app_model: App, run_id: UUID) -> Iterator[str]:
     # if the run is gone (raised before yielding any bytes, so Flask turns it
     # into the normal HTTP 404 path).
     try:
-        snapshot = service.snapshot_workflow_run(app_model=app_model, workflow_run_id=run_id_str)
+        snapshot = service.snapshot_workflow_run(app_model=app_model, workflow_run_id=run_id_str, session=db.session())
     except NodeOutputInspectorError as error:
         raise _InspectorNotFound(error) from error
 
@@ -308,6 +313,7 @@ def _stream_inspector_events(app_model: App, run_id: UUID) -> Iterator[str]:
                 app_model=app_model,
                 workflow_run_id=run_id_str,
                 node_id=message.node_id,
+                session=db.session(),
             )
         except NodeOutputInspectorError:
             # Node may not appear in the graph yet (race with persistence); skip.

api/controllers/console/auth/activate.py

@@ -90,7 +90,7 @@ class ActivateCheckApi(Resource):
         token = args.token
 
         invitation = RegisterService.get_invitation_with_case_fallback(
-            workspaceId, args.email, token, session=db.session
+            workspaceId, args.email, token, session=db.session()
         )
         if invitation:
             data = invitation.get("data", {})
@@ -140,7 +140,7 @@ class ActivateApi(Resource):
 
         normalized_request_email = args.email.lower() if args.email else None
         invitation = RegisterService.get_invitation_with_case_fallback(
-            args.workspace_id, args.email, args.token, session=db.session
+            args.workspace_id, args.email, args.token, session=db.session()
         )
         if invitation is None:
             raise AlreadyActivateError()
@@ -178,7 +178,7 @@ class ActivateApi(Resource):
         RegisterService.revoke_token(args.workspace_id, normalized_request_email, args.token)
 
         if membership_id is None:
-            TenantService.create_tenant_member(tenant, account, db.session, role=role)
+            TenantService.create_tenant_member(tenant, account, db.session(), role=role)
 
         if setup_fields:
             account.name = setup_fields[0]
@@ -188,6 +188,6 @@ class ActivateApi(Resource):
             account.status = AccountStatus.ACTIVE
             account.initialized_at = naive_utc_now()
 
-        TenantService.switch_tenant(account, tenant.id, session=db.session)
+        TenantService.switch_tenant(account, tenant.id, session=db.session())
 
         return {"result": "success"}

api/controllers/console/auth/data_source_bearer_auth.py

@@ -59,7 +59,7 @@ class ApiKeyAuthDataSource(Resource):
     @account_initialization_required
     @with_current_tenant_id
     def get(self, current_tenant_id: str):
-        data_source_api_key_bindings = ApiKeyAuthService.get_provider_auth_list(db.session(), current_tenant_id)
+        data_source_api_key_bindings = ApiKeyAuthService.get_provider_auth_list(current_tenant_id, session=db.session())
         if data_source_api_key_bindings:
             return {
                 "sources": [
@@ -93,7 +93,7 @@ class ApiKeyAuthDataSourceBinding(Resource):
         data = payload.model_dump()
         ApiKeyAuthService.validate_api_key_auth_args(data)
         try:
-            ApiKeyAuthService.create_provider_auth(db.session(), current_tenant_id, data)
+            ApiKeyAuthService.create_provider_auth(current_tenant_id, data, session=db.session())
         except Exception as e:
             raise ApiKeyAuthFailedError(str(e))
         return {"result": "success"}, 200
@@ -110,6 +110,6 @@ class ApiKeyAuthDataSourceBindingDelete(Resource):
     @with_current_tenant_id
     def delete(self, current_tenant_id: str, binding_id: UUID):
         # The role of the current user in the table must be admin or owner
-        ApiKeyAuthService.delete_provider_auth(db.session(), current_tenant_id, str(binding_id))
+        ApiKeyAuthService.delete_provider_auth(current_tenant_id, str(binding_id), session=db.session())
 
         return "", 204

api/controllers/console/auth/email_register.py

@@ -101,7 +101,7 @@ class EmailRegisterSendEmailApi(Resource):
         if dify_config.BILLING_ENABLED and BillingService.is_email_in_freeze(normalized_email):
             raise AccountInFreezeError()
 
-        account = AccountService.get_account_by_email_with_case_fallback(db.session, args.email)
+        account = AccountService.get_account_by_email_with_case_fallback(args.email, session=db.session())
         token = AccountService.send_email_register_email(email=normalized_email, account=account, language=language)
         return {"result": "success", "data": token}
 
@@ -176,7 +176,7 @@ class EmailRegisterResetApi(Resource):
         email = register_data.get("email", "")
         normalized_email = email.lower()
 
-        account = AccountService.get_account_by_email_with_case_fallback(db.session, email)
+        account = AccountService.get_account_by_email_with_case_fallback(email, session=db.session())
 
         if account:
             raise EmailAlreadyInUseError()
@@ -187,7 +187,7 @@ class EmailRegisterResetApi(Resource):
             timezone=args.timezone,
             language=args.language,
         )
-        token_pair = AccountService.login(account=account, session=db.session, ip_address=extract_remote_ip(request))
+        token_pair = AccountService.login(account=account, session=db.session(), ip_address=extract_remote_ip(request))
         AccountService.reset_login_error_rate_limit(normalized_email)
 
         return {"result": "success", "data": token_pair.model_dump()}
@@ -206,7 +206,7 @@ class EmailRegisterResetApi(Resource):
                 password=password,
                 interface_language=get_valid_language(language),
                 timezone=timezone,
-                session=db.session,
+                session=db.session(),
             )
         except AccountRegisterError:
             raise AccountInFreezeError()

api/controllers/console/auth/forgot_password.py

@@ -82,7 +82,7 @@ class ForgotPasswordSendEmailApi(Resource):
         else:
             language = "en-US"
 
-        account = AccountService.get_account_by_email_with_case_fallback(db.session, args.email)
+        account = AccountService.get_account_by_email_with_case_fallback(args.email, session=db.session())
 
         token = AccountService.send_reset_password_email(
             account=account,
@@ -180,7 +180,7 @@ class ForgotPasswordResetApi(Resource):
         password_hashed = hash_password(args.new_password, salt)
 
         email = reset_data.get("email", "")
-        account = AccountService.get_account_by_email_with_case_fallback(db.session, email)
+        account = AccountService.get_account_by_email_with_case_fallback(email, session=db.session())
 
         if account:
             account = db.session.merge(account)
@@ -198,10 +198,10 @@ class ForgotPasswordResetApi(Resource):
 
         # Create workspace if needed
         if (
-            not TenantService.get_join_tenants(account, session=db.session)
+            not TenantService.get_join_tenants(account, session=db.session())
             and FeatureService.get_system_features().is_allow_create_workspace
         ):
-            tenant = TenantService.create_tenant(f"{account.name}'s Workspace", session=db.session)
-            TenantService.create_tenant_member(tenant, account, db.session, role="owner")
+            tenant = TenantService.create_tenant(f"{account.name}'s Workspace", session=db.session())
+            TenantService.create_tenant_member(tenant, account, db.session(), role="owner")
             account.current_tenant = tenant
             tenant_was_created.send(tenant)

api/controllers/console/auth/login.py

@@ -126,7 +126,7 @@ class LoginApi(Resource):
         invitation_data: InvitationDetailDict | None = None
         if invite_token:
             invitation_data = RegisterService.get_invitation_with_case_fallback(
-                None, request_email, invite_token, session=db.session
+                None, request_email, invite_token, session=db.session()
             )
             if invitation_data is None:
                 invite_token = None
@@ -153,7 +153,7 @@ class LoginApi(Resource):
             _log_console_login_failure(email=normalized_email, reason=LoginFailureReason.INVALID_CREDENTIALS)
             raise AuthenticationFailedError() from exc
         # SELF_HOSTED only have one workspace
-        tenants = TenantService.get_join_tenants(account, session=db.session)
+        tenants = TenantService.get_join_tenants(account, session=db.session())
         if len(tenants) == 0:
             system_features = FeatureService.get_system_features()
 
@@ -165,7 +165,7 @@ class LoginApi(Resource):
                     data="workspace not found, please contact system admin to invite you to join in a workspace",
                 ).model_dump(mode="json")
 
-        token_pair = AccountService.login(account=account, session=db.session, ip_address=extract_remote_ip(request))
+        token_pair = AccountService.login(account=account, session=db.session(), ip_address=extract_remote_ip(request))
         AccountService.reset_login_error_rate_limit(normalized_email)
 
         # Create response with cookies instead of returning tokens in body
@@ -301,7 +301,7 @@ class EmailCodeLoginApi(Resource):
             _log_console_login_failure(email=user_email, reason=LoginFailureReason.ACCOUNT_IN_FREEZE)
             raise AccountInFreezeError()
         if account:
-            tenants = TenantService.get_join_tenants(account, session=db.session)
+            tenants = TenantService.get_join_tenants(account, session=db.session())
             if not tenants:
                 workspaces = FeatureService.get_system_features().license.workspaces
                 if not workspaces.is_available():
@@ -309,8 +309,8 @@ class EmailCodeLoginApi(Resource):
                 if not FeatureService.get_system_features().is_allow_create_workspace:
                     raise NotAllowedCreateWorkspace()
                 else:
-                    new_tenant = TenantService.create_tenant(f"{account.name}'s Workspace", session=db.session)
-                    TenantService.create_tenant_member(new_tenant, account, db.session, role="owner")
+                    new_tenant = TenantService.create_tenant(f"{account.name}'s Workspace", session=db.session())
+                    TenantService.create_tenant_member(new_tenant, account, db.session(), role="owner")
                     account.current_tenant = new_tenant
                     tenant_was_created.send(new_tenant)
 
@@ -321,7 +321,7 @@ class EmailCodeLoginApi(Resource):
                     name=user_email,
                     interface_language=get_valid_language(language),
                     timezone=args.timezone,
-                    session=db.session,
+                    session=db.session(),
                 )
             except WorkSpaceNotAllowedCreateError:
                 raise NotAllowedCreateWorkspace()
@@ -330,7 +330,7 @@ class EmailCodeLoginApi(Resource):
                 raise AccountInFreezeError()
             except WorkspacesLimitExceededError:
                 raise WorkspacesLimitExceeded()
-        token_pair = AccountService.login(account, session=db.session, ip_address=extract_remote_ip(request))
+        token_pair = AccountService.login(account, session=db.session(), ip_address=extract_remote_ip(request))
         AccountService.reset_login_error_rate_limit(user_email)
 
         # Create response with cookies instead of returning tokens in body
@@ -358,7 +358,7 @@ class RefreshTokenApi(Resource):
             ), 401
 
         try:
-            new_token_pair = AccountService.refresh_token(refresh_token, session=db.session)
+            new_token_pair = AccountService.refresh_token(refresh_token, session=db.session())
         except Unauthorized as exc:
             return SimpleResultMessageResponse(result="fail", message=exc.description or "Unauthorized.").model_dump(
                 mode="json"
@@ -378,22 +378,22 @@ class RefreshTokenApi(Resource):
 
 
 def _get_account_with_case_fallback(email: str):
-    account = AccountService.get_user_through_email(email, session=db.session)
+    account = AccountService.get_user_through_email(email, session=db.session())
     if account or email == email.lower():
         return account
 
-    return AccountService.get_user_through_email(email.lower(), session=db.session)
+    return AccountService.get_user_through_email(email.lower(), session=db.session())
 
 
 def _authenticate_account_with_case_fallback(
     original_email: str, normalized_email: str, password: str, invite_token: str | None
 ):
     try:
-        return AccountService.authenticate(original_email, password, invite_token, session=db.session)
+        return AccountService.authenticate(original_email, password, invite_token, session=db.session())
     except services.errors.account.AccountPasswordError:
         if original_email == normalized_email:
             raise
-        return AccountService.authenticate(normalized_email, password, invite_token, session=db.session)
+        return AccountService.authenticate(normalized_email, password, invite_token, session=db.session())
 
 
 def _log_console_login_failure(*, email: str, reason: LoginFailureReason) -> None:

api/controllers/console/auth/oauth.py

@@ -195,7 +195,7 @@ class OAuthCallback(Resource):
             db.session.commit()
 
         try:
-            TenantService.create_owner_tenant_if_not_exist(account, session=db.session)
+            TenantService.create_owner_tenant_if_not_exist(account, session=db.session())
         except Unauthorized:
             return redirect(f"{dify_config.CONSOLE_WEB_URL}/signin?message=Workspace not found.")
         except WorkSpaceNotAllowedCreateError:
@@ -206,7 +206,7 @@ class OAuthCallback(Resource):
 
         token_pair = AccountService.login(
             account=account,
-            session=db.session,
+            session=db.session(),
             ip_address=extract_remote_ip(request),
         )
 
@@ -225,7 +225,7 @@ def _get_account_by_openid_or_email(provider: str, user_info: OAuthUserInfo) ->
     account: Account | None = Account.get_by_openid(provider, user_info.id)
 
     if not account:
-        account = AccountService.get_account_by_email_with_case_fallback(db.session, user_info.email)
+        account = AccountService.get_account_by_email_with_case_fallback(user_info.email, session=db.session())
 
     return account
 
@@ -241,13 +241,13 @@ def _generate_account(
     oauth_new_user = False
 
     if account:
-        tenants = TenantService.get_join_tenants(account, session=db.session)
+        tenants = TenantService.get_join_tenants(account, session=db.session())
         if not tenants:
             if not FeatureService.get_system_features().is_allow_create_workspace:
                 raise WorkSpaceNotAllowedCreateError()
             else:
-                new_tenant = TenantService.create_tenant(f"{account.name}'s Workspace", session=db.session)
-                TenantService.create_tenant_member(new_tenant, account, db.session, role="owner")
+                new_tenant = TenantService.create_tenant(f"{account.name}'s Workspace", session=db.session())
+                TenantService.create_tenant_member(new_tenant, account, db.session(), role="owner")
                 account.current_tenant = new_tenant
                 tenant_was_created.send(new_tenant)
 
@@ -273,10 +273,10 @@ def _generate_account(
             provider=provider,
             language=interface_language,
             timezone=timezone,
-            session=db.session,
+            session=db.session(),
         )
 
     # Link account
-    AccountService.link_account_integrate(provider, user_info.id, account, session=db.session)
+    AccountService.link_account_integrate(provider, user_info.id, account, session=db.session())
 
     return account, oauth_new_user

api/controllers/console/auth/oauth_server.py

@@ -10,6 +10,7 @@ from werkzeug.exceptions import BadRequest, NotFound
 
 from controllers.common.schema import register_response_schema_models, register_schema_models
 from controllers.console.wraps import account_initialization_required, setup_required, with_current_user
+from extensions.ext_database import db
 from graphon.model_runtime.utils.encoders import jsonable_encoder
 from libs.login import login_required
 from models import Account
@@ -131,7 +132,9 @@ def oauth_server_access_token_required[T, **P, R](
             response.headers["WWW-Authenticate"] = "Bearer"
             return response
 
-        account = OAuthServerService.validate_oauth_access_token(oauth_provider_app.client_id, access_token)
+        account = OAuthServerService.validate_oauth_access_token(
+            oauth_provider_app.client_id, access_token, db.session()
+        )
         if not account:
             response = jsonify({"error": "access_token or client_id is invalid"})
             response.status_code = 401

api/controllers/console/billing/billing.py

@@ -56,7 +56,7 @@ class Subscription(Resource):
     @with_current_tenant_id
     def get(self, current_tenant_id: str, current_user: Account):
         args = SubscriptionQuery.model_validate(request.args.to_dict(flat=True))
-        BillingService.is_tenant_owner_or_admin(db.session, current_user)
+        BillingService.is_tenant_owner_or_admin(current_user, session=db.session())
         return BillingService.get_subscription(args.plan, args.interval, current_user.email, current_tenant_id)
 
 
@@ -70,7 +70,7 @@ class Invoices(Resource):
     @with_current_user
     @with_current_tenant_id
     def get(self, current_tenant_id: str, current_user: Account):
-        BillingService.is_tenant_owner_or_admin(db.session, current_user)
+        BillingService.is_tenant_owner_or_admin(current_user, session=db.session())
         return BillingService.get_invoices(current_user.email, current_tenant_id)
 
 

api/controllers/console/datasets/data_source.py

@@ -245,7 +245,7 @@ class DataSourceNotionListApi(Resource):
         exist_page_ids = []
         # import notion in the exist dataset
         if query.dataset_id:
-            dataset = DatasetService.get_dataset(query.dataset_id, db.session)
+            dataset = DatasetService.get_dataset(query.dataset_id, db.session())
             if not dataset:
                 raise NotFound("Dataset not found.")
             if dataset.data_source_type != "notion_import":
@@ -400,11 +400,11 @@ class DataSourceNotionDatasetSyncApi(Resource):
     @rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_CREATE_AND_MANAGEMENT)
     def get(self, dataset_id: UUID) -> tuple[dict[str, str], int]:
         dataset_id_str = str(dataset_id)
-        dataset = DatasetService.get_dataset(dataset_id_str, db.session)
+        dataset = DatasetService.get_dataset(dataset_id_str, db.session())
         if dataset is None:
             raise NotFound("Dataset not found.")
 
-        documents = DocumentService.get_document_by_dataset_id(dataset_id_str, db.session)
+        documents = DocumentService.get_document_by_dataset_id(dataset_id_str, db.session())
         for document in documents:
             document_indexing_sync_task.delay(dataset_id_str, document.id)
         return {"result": "success"}, 200
@@ -420,11 +420,11 @@ class DataSourceNotionDocumentSyncApi(Resource):
     def get(self, dataset_id: UUID, document_id: UUID) -> tuple[dict[str, str], int]:
         dataset_id_str = str(dataset_id)
         document_id_str = str(document_id)
-        dataset = DatasetService.get_dataset(dataset_id_str, db.session)
+        dataset = DatasetService.get_dataset(dataset_id_str, db.session())
         if dataset is None:
             raise NotFound("Dataset not found.")
 
-        document = DocumentService.get_document(dataset_id_str, document_id_str, session=db.session)
+        document = DocumentService.get_document(dataset_id_str, document_id_str, session=db.session())
         if document is None:
             raise NotFound("Document not found.")
         document_indexing_sync_task.delay(dataset_id_str, document_id_str)

api/controllers/console/datasets/datasets.py

@@ -30,6 +30,7 @@ from controllers.console.wraps import (
     with_current_tenant_id,
     with_current_user,
 )
+from core.entities.knowledge_entities import IndexingEstimate
 from core.errors.error import LLMBadRequestError, ProviderTokenNotInitError
 from core.indexing_runner import IndexingRunner
 from core.plugin.impl.model_runtime_factory import create_plugin_provider_manager
@@ -268,21 +269,10 @@ class ErrorDocsResponse(DocumentStatusListResponse):
     total: int
 
 
-class IndexingEstimatePreviewItemResponse(ResponseModel):
-    content: str
-    child_chunks: list[str] | None = None
-    summary: str | None = None
-
-
-class IndexingEstimateQaPreviewItemResponse(ResponseModel):
-    question: str
-    answer: str
-
-
-class IndexingEstimateResponse(ResponseModel):
-    total_segments: int
-    preview: list[IndexingEstimatePreviewItemResponse]
-    qa_preview: list[IndexingEstimateQaPreviewItemResponse] | None = None
+class IndexingEstimateResponse(IndexingEstimate):
+    tokens: int
+    total_price: float | int
+    currency: str
 
 
 class RetrievalSettingResponse(ResponseModel):
@@ -418,6 +408,7 @@ class DatasetListApi(Resource):
         permissions = enterprise_rbac_service.RBACService.MyPermissions.get(
             str(current_tenant_id),
             current_user.id,
+            session=db.session(),
         )
 
         accessible_dataset_ids: list[str] | None = None
@@ -461,7 +452,7 @@ class DatasetListApi(Resource):
             datasets, total = DatasetService.get_datasets(
                 query.page,
                 query.limit,
-                db.session,
+                db.session(),
                 current_tenant_id,
                 current_user,
                 query.keyword,
@@ -573,6 +564,7 @@ class DatasetListApi(Resource):
             current_tenant_id,
             current_user.id,
             [dataset.id],
+            session=session,
         )
 
         item = DatasetDetailWithPartialMembersResponse.model_validate(dataset, from_attributes=True).model_dump(
@@ -602,17 +594,18 @@ class DatasetApi(Resource):
     @with_current_tenant_id
     def get(self, current_tenant_id: str, current_user: Account, dataset_id: UUID):
         dataset_id_str = str(dataset_id)
-        dataset = DatasetService.get_dataset(dataset_id_str, db.session)
+        dataset = DatasetService.get_dataset(dataset_id_str, db.session())
         if dataset is None:
             raise NotFound("Dataset not found.")
         try:
-            DatasetService.check_dataset_permission(dataset, current_user, db.session)
+            DatasetService.check_dataset_permission(dataset, current_user, db.session())
         except services.errors.account.NoPermissionError as e:
             raise Forbidden(str(e))
         permissions = enterprise_rbac_service.RBACService.MyPermissions.get(
             current_tenant_id,
             current_user.id,
             dataset_id=dataset_id_str,
+            session=db.session(),
         )
         permission_keys_map = permissions.dataset.permission_keys_by_resource_ids([dataset_id_str])
         data = dump_response(DatasetDetailResponse, dataset)
@@ -622,7 +615,7 @@ class DatasetApi(Resource):
                 provider_id = ModelProviderID(dataset.embedding_model_provider)
                 data["embedding_model_provider"] = str(provider_id)
         if data.get("permission") == "partial_members":
-            part_users_list = DatasetPermissionService.get_dataset_partial_member_list(dataset_id_str, db.session)
+            part_users_list = DatasetPermissionService.get_dataset_partial_member_list(dataset_id_str, db.session())
             data.update({"partial_member_list": part_users_list})
 
         # check embedding setting
@@ -644,7 +637,7 @@ class DatasetApi(Resource):
         else:
             data["embedding_available"] = True
 
-        return data, 200
+        return dump_response(DatasetDetailWithPartialMembersResponse, data), 200
 
     @console_ns.doc("update_dataset")
     @console_ns.doc(description="Update dataset details")
@@ -666,7 +659,7 @@ class DatasetApi(Resource):
     @with_session
     def patch(self, session: Session, current_tenant_id: str, current_user: Account, dataset_id: UUID):
         dataset_id_str = str(dataset_id)
-        dataset = DatasetService.get_dataset(dataset_id_str, db.session)
+        dataset = DatasetService.get_dataset(dataset_id_str, db.session())
         if dataset is None:
             raise NotFound("Dataset not found.")
 
@@ -685,10 +678,10 @@ class DatasetApi(Resource):
         # The role of the current user in the ta table must be admin, owner, editor, or dataset_operator
         if not dify_config.RBAC_ENABLED:
             DatasetPermissionService.check_permission(
-                session, current_user, dataset, payload.permission, payload.partial_member_list
+                current_user, dataset, payload.permission, payload.partial_member_list, session=session
             )
 
-        dataset = DatasetService.update_dataset(session, dataset_id_str, payload_data, current_user)
+        dataset = DatasetService.update_dataset(dataset_id_str, payload_data, current_user, session=session)
 
         if dataset is None:
             raise NotFound("Dataset not found.")
@@ -697,6 +690,7 @@ class DatasetApi(Resource):
             current_tenant_id,
             current_user.id,
             [dataset_id_str],
+            session=session,
         )
         result_data = dump_response(DatasetDetailResponse, dataset)
         result_data["permission_keys"] = permission_keys_map.get(dataset_id_str, [])
@@ -704,16 +698,16 @@ class DatasetApi(Resource):
 
         if payload.partial_member_list is not None and payload.permission == DatasetPermissionEnum.PARTIAL_TEAM:
             DatasetPermissionService.update_partial_member_list(
-                tenant_id, dataset_id_str, payload.partial_member_list, db.session
+                tenant_id, dataset_id_str, payload.partial_member_list, db.session()
             )
         # clear partial member list when permission is only_me or all_team_members
         elif payload.permission in {DatasetPermissionEnum.ONLY_ME, DatasetPermissionEnum.ALL_TEAM}:
-            DatasetPermissionService.clear_partial_member_list(dataset_id_str, db.session)
+            DatasetPermissionService.clear_partial_member_list(dataset_id_str, db.session())
 
-        partial_member_list = DatasetPermissionService.get_dataset_partial_member_list(dataset_id_str, db.session)
+        partial_member_list = DatasetPermissionService.get_dataset_partial_member_list(dataset_id_str, db.session())
         result_data.update({"partial_member_list": partial_member_list})
 
-        return result_data, 200
+        return dump_response(DatasetDetailWithPartialMembersResponse, result_data), 200
 
     @setup_required
     @login_required
@@ -729,8 +723,8 @@ class DatasetApi(Resource):
             raise Forbidden()
 
         try:
-            if DatasetService.delete_dataset(dataset_id_str, current_user, db.session):
-                DatasetPermissionService.clear_partial_member_list(dataset_id_str, db.session)
+            if DatasetService.delete_dataset(dataset_id_str, current_user, db.session()):
+                DatasetPermissionService.clear_partial_member_list(dataset_id_str, db.session())
                 return "", 204
             else:
                 raise NotFound("Dataset not found.")
@@ -755,8 +749,8 @@ class DatasetUseCheckApi(Resource):
     def get(self, dataset_id: UUID):
         dataset_id_str = str(dataset_id)
 
-        dataset_is_using = DatasetService.dataset_use_check(dataset_id_str, db.session)
-        return {"is_using": dataset_is_using}, 200
+        dataset_is_using = DatasetService.dataset_use_check(dataset_id_str, db.session())
+        return UsageCheckResponse(is_using=dataset_is_using).model_dump(mode="json"), 200
 
 
 @console_ns.route("/datasets/<uuid:dataset_id>/queries")
@@ -776,12 +770,12 @@ class DatasetQueryApi(Resource):
     @rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_READONLY)
     def get(self, current_user: Account, dataset_id: UUID):
         dataset_id_str = str(dataset_id)
-        dataset = DatasetService.get_dataset(dataset_id_str, db.session)
+        dataset = DatasetService.get_dataset(dataset_id_str, db.session())
         if dataset is None:
             raise NotFound("Dataset not found.")
 
         try:
-            DatasetService.check_dataset_permission(dataset, current_user, db.session)
+            DatasetService.check_dataset_permission(dataset, current_user, db.session())
         except services.errors.account.NoPermissionError as e:
             raise Forbidden(str(e))
 
@@ -897,7 +891,17 @@ class DatasetIndexingEstimateApi(Resource):
         except Exception as e:
             raise IndexingEstimateError(str(e))
 
-        return response.model_dump(), 200
+        return (
+            IndexingEstimateResponse(
+                tokens=0,
+                total_price=0,
+                currency="USD",
+                total_segments=response.total_segments,
+                preview=response.preview,
+                qa_preview=response.qa_preview,
+            ).model_dump(mode="json", exclude_none=True),
+            200,
+        )
 
 
 @console_ns.route("/datasets/<uuid:dataset_id>/related-apps")
@@ -917,16 +921,16 @@ class DatasetRelatedAppListApi(Resource):
     @rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_READONLY)
     def get(self, current_user: Account, dataset_id: UUID):
         dataset_id_str = str(dataset_id)
-        dataset = DatasetService.get_dataset(dataset_id_str, db.session)
+        dataset = DatasetService.get_dataset(dataset_id_str, db.session())
         if dataset is None:
             raise NotFound("Dataset not found.")
 
         try:
-            DatasetService.check_dataset_permission(dataset, current_user, db.session)
+            DatasetService.check_dataset_permission(dataset, current_user, db.session())
         except services.errors.account.NoPermissionError as e:
             raise Forbidden(str(e))
 
-        app_dataset_joins = DatasetService.get_related_apps(dataset.id, db.session)
+        app_dataset_joins = DatasetService.get_related_apps(dataset.id, db.session())
 
         related_apps = []
         for app_dataset_join in app_dataset_joins:
@@ -1014,7 +1018,7 @@ class DatasetApiKeyApi(Resource):
         keys = db.session.scalars(
             select(ApiToken).where(ApiToken.type == self.resource_type, ApiToken.tenant_id == current_tenant_id)
         ).all()
-        return ApiKeyList.model_validate({"data": keys}, from_attributes=True).model_dump(mode="json")
+        return dump_response(ApiKeyList, {"data": keys})
 
     @console_ns.response(200, "API key created successfully", console_ns.models[ApiKeyItem.__name__])
     @console_ns.response(400, "Maximum keys exceeded")
@@ -1048,7 +1052,7 @@ class DatasetApiKeyApi(Resource):
         api_token.type = self.resource_type
         db.session.add(api_token)
         db.session.commit()
-        return ApiKeyItem.model_validate(api_token, from_attributes=True).model_dump(mode="json"), 200
+        return dump_response(ApiKeyItem, api_token), 200
 
 
 @console_ns.route("/datasets/api-keys/<uuid:api_key_id>")
@@ -1101,9 +1105,9 @@ class DatasetEnableApiApi(Resource):
     def post(self, dataset_id: UUID, status: str):
         dataset_id_str = str(dataset_id)
 
-        DatasetService.update_dataset_api_status(dataset_id_str, status == "enable", db.session)
+        DatasetService.update_dataset_api_status(dataset_id_str, status == "enable", db.session())
 
-        return {"result": "success"}, 200
+        return SimpleResultResponse(result="success").model_dump(mode="json"), 200
 
 
 @console_ns.route("/datasets/api-base-info")
@@ -1116,7 +1120,7 @@ class DatasetApiBaseUrlApi(Resource):
     @account_initialization_required
     def get(self):
         base = dify_config.SERVICE_API_URL or request.host_url.rstrip("/")
-        return {"api_base_url": normalize_api_base_url(base)}
+        return ApiBaseUrlResponse(api_base_url=normalize_api_base_url(base)).model_dump(mode="json")
 
 
 @console_ns.route("/datasets/retrieval-setting")
@@ -1170,10 +1174,10 @@ class DatasetErrorDocs(Resource):
     @rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_READONLY)
     def get(self, dataset_id: UUID):
         dataset_id_str = str(dataset_id)
-        dataset = DatasetService.get_dataset(dataset_id_str, db.session)
+        dataset = DatasetService.get_dataset(dataset_id_str, db.session())
         if dataset is None:
             raise NotFound("Dataset not found.")
-        results = DocumentService.get_error_documents_by_dataset_id(dataset_id_str, db.session)
+        results = DocumentService.get_error_documents_by_dataset_id(dataset_id_str, db.session())
 
         return dump_response(ErrorDocsResponse, {"data": results, "total": len(results)}), 200
 
@@ -1197,15 +1201,15 @@ class DatasetPermissionUserListApi(Resource):
     @rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_READONLY)
     def get(self, current_user: Account, dataset_id: UUID):
         dataset_id_str = str(dataset_id)
-        dataset = DatasetService.get_dataset(dataset_id_str, db.session)
+        dataset = DatasetService.get_dataset(dataset_id_str, db.session())
         if dataset is None:
             raise NotFound("Dataset not found.")
         try:
-            DatasetService.check_dataset_permission(dataset, current_user, db.session)
+            DatasetService.check_dataset_permission(dataset, current_user, db.session())
         except services.errors.account.NoPermissionError as e:
             raise Forbidden(str(e))
 
-        partial_members_list = DatasetPermissionService.get_dataset_partial_member_list(dataset_id_str, db.session)
+        partial_members_list = DatasetPermissionService.get_dataset_partial_member_list(dataset_id_str, db.session())
 
         return dump_response(PartialMemberListResponse, {"data": partial_members_list}), 200
 
@@ -1227,8 +1231,8 @@ class DatasetAutoDisableLogApi(Resource):
     @rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_READONLY)
     def get(self, dataset_id: UUID):
         dataset_id_str = str(dataset_id)
-        dataset = DatasetService.get_dataset(dataset_id_str, db.session)
+        dataset = DatasetService.get_dataset(dataset_id_str, db.session())
         if dataset is None:
             raise NotFound("Dataset not found.")
-        auto_disable_logs = DatasetService.get_dataset_auto_disable_logs(dataset_id_str, db.session)
+        auto_disable_logs = DatasetService.get_dataset_auto_disable_logs(dataset_id_str, db.session())
         return dump_response(AutoDisableLogsResponse, auto_disable_logs), 200

api/controllers/console/datasets/datasets_document.py

@@ -10,16 +10,17 @@ from uuid import UUID
 import sqlalchemy as sa
 from flask import request, send_file
 from flask_restx import Resource
-from pydantic import BaseModel, Field, RootModel, field_validator
+from pydantic import BaseModel, Field, JsonValue, field_validator
 from sqlalchemy import asc, desc, func, select
 from werkzeug.exceptions import Forbidden, NotFound
 
 import services
 from controllers.common.controller_schemas import DocumentBatchDownloadZipPayload
-from controllers.common.fields import BinaryFileResponse, SimpleResultMessageResponse, SimpleResultResponse, UrlResponse
+from controllers.common.fields import SimpleResultMessageResponse, SimpleResultResponse, UrlResponse
 from controllers.common.schema import register_response_schema_models, register_schema_models
 from controllers.console import console_ns
 from controllers.console.wraps import RBACPermission, RBACResourceScope, rbac_permission_required
+from core.entities.knowledge_entities import IndexingEstimate
 from core.errors.error import (
     LLMBadRequestError,
     ModelCurrentlyNotSupportError,
@@ -29,6 +30,7 @@ from core.errors.error import (
 from core.indexing_runner import IndexingRunner
 from core.model_manager import ModelManager
 from core.plugin.impl.exc import PluginDaemonClientSideError
+from core.rag.entities import Rule
 from core.rag.extractor.entity.datasource_type import DatasourceType
 from core.rag.extractor.entity.extract_setting import ExtractSetting, NotionInfo, WebsiteInfo
 from core.rag.index_processor.constant.index_type import IndexTechniqueType
@@ -49,7 +51,7 @@ from libs.login import login_required
 from libs.pagination import paginate_query
 from models import Account, DatasetProcessRule, Document, DocumentSegment, UploadFile
 from models.dataset import DocumentPipelineExecutionLog
-from models.enums import IndexingStatus, SegmentStatus
+from models.enums import IndexingStatus, ProcessRuleMode, SegmentStatus
 from services.dataset_ref_service import DatasetRefService
 from services.dataset_service import DatasetService, DocumentService
 from services.entities.knowledge_entities.knowledge_entities import KnowledgeConfig, ProcessRule, RetrievalModel
@@ -148,8 +150,91 @@ class DocumentWithSegmentsListResponse(ResponseModel):
     page: int
 
 
-class OpaqueObjectResponse(RootModel[dict[str, Any]]):
-    root: dict[str, Any]
+class IndexingEstimateResponse(IndexingEstimate):
+    tokens: int
+    total_price: float | int
+    currency: str
+
+
+class DocumentDetailResponse(ResponseModel):
+    id: str
+    position: int | None = None
+    data_source_type: str | None = None
+    data_source_info: Any = None
+    data_source_detail_dict: Any = None
+    dataset_process_rule_id: str | None = None
+    dataset_process_rule: Any = None
+    document_process_rule: Any = None
+    name: str | None = None
+    created_from: str | None = None
+    created_by: str | None = None
+    created_at: int | None = None
+    tokens: int | None = None
+    indexing_status: str | None = None
+    completed_at: int | None = None
+    updated_at: int | None = None
+    indexing_latency: float | None = None
+    error: str | None = None
+    enabled: bool | None = None
+    disabled_at: int | None = None
+    disabled_by: str | None = None
+    archived: bool | None = None
+    doc_type: str | None = None
+    doc_metadata: list[DocumentMetadataResponse] | None = None
+    segment_count: int | None = None
+    average_segment_length: float | None = None
+    hit_count: int | None = None
+    display_status: str | None = None
+    doc_form: str | None = None
+    doc_language: str | None = None
+    need_summary: bool | None = None
+
+    @field_validator("data_source_type", "indexing_status", "display_status", "doc_form", mode="before")
+    @classmethod
+    def _normalize_enum_fields(cls, value: Any) -> Any:
+        return normalize_enum(value)
+
+
+class SummaryStatusResponse(ResponseModel):
+    completed: int = 0
+    generating: int = 0
+    error: int = 0
+    not_started: int = 0
+    timeout: int = 0
+
+
+class SummaryEntryResponse(ResponseModel):
+    segment_id: str
+    segment_position: int
+    status: str
+    summary_preview: str | None = None
+    error: str | None = None
+    created_at: int | None = None
+    updated_at: int | None = None
+
+    @field_validator("status", mode="before")
+    @classmethod
+    def _normalize_status(cls, value: Any) -> Any:
+        return normalize_enum(value)
+
+
+class DocumentSummaryStatusResponse(ResponseModel):
+    total_segments: int
+    summary_status: SummaryStatusResponse
+    summaries: list[SummaryEntryResponse]
+
+
+class ProcessRuleResponse(ResponseModel):
+    mode: ProcessRuleMode
+    rules: Rule | None = None
+    limits: dict[str, Any]
+
+
+class DocumentPipelineExecutionLogResponse(ResponseModel):
+    datasource_info: JsonValue | None = None
+    datasource_type: str | None = None
+    input_data: JsonValue | None = None
+    datasource_node_id: str | None = None
 
 
 register_schema_models(
@@ -165,7 +250,6 @@ register_schema_models(
 )
 register_response_schema_models(
     console_ns,
-    BinaryFileResponse,
     SimpleResultMessageResponse,
     SimpleResultResponse,
     UrlResponse,
@@ -175,7 +259,11 @@ register_response_schema_models(
     DocumentWithSegmentsResponse,
     DatasetAndDocumentResponse,
     DocumentWithSegmentsListResponse,
-    OpaqueObjectResponse,
+    IndexingEstimateResponse,
+    DocumentDetailResponse,
+    DocumentSummaryStatusResponse,
+    ProcessRuleResponse,
+    DocumentPipelineExecutionLogResponse,
 )
 
 
@@ -183,16 +271,16 @@ class DocumentResource(Resource):
     def get_document(
         self, dataset_id: str, document_id: str, current_user: Account, current_tenant_id: str
     ) -> Document:
-        dataset = DatasetService.get_dataset(dataset_id, db.session)
+        dataset = DatasetService.get_dataset(dataset_id, db.session())
         if not dataset:
             raise NotFound("Dataset not found.")
 
         try:
-            DatasetService.check_dataset_permission(dataset, current_user, db.session)
+            DatasetService.check_dataset_permission(dataset, current_user, db.session())
         except services.errors.account.NoPermissionError as e:
             raise Forbidden(str(e))
 
-        document = DocumentService.get_document(dataset_id, document_id, session=db.session)
+        document = DocumentService.get_document(dataset_id, document_id, session=db.session())
 
         if not document:
             raise NotFound("Document not found.")
@@ -203,16 +291,16 @@ class DocumentResource(Resource):
         return document
 
     def get_batch_documents(self, dataset_id: str, batch: str, current_user: Account) -> Sequence[Document]:
-        dataset = DatasetService.get_dataset(dataset_id, db.session)
+        dataset = DatasetService.get_dataset(dataset_id, db.session())
         if not dataset:
             raise NotFound("Dataset not found.")
 
         try:
-            DatasetService.check_dataset_permission(dataset, current_user, db.session)
+            DatasetService.check_dataset_permission(dataset, current_user, db.session())
         except services.errors.account.NoPermissionError as e:
             raise Forbidden(str(e))
 
-        documents = DocumentService.get_batch_documents(dataset_id, batch, db.session)
+        documents = DocumentService.get_batch_documents(dataset_id, batch, db.session())
 
         if not documents:
             raise NotFound("Documents not found.")
@@ -225,7 +313,7 @@ class GetProcessRuleApi(Resource):
     @console_ns.doc("get_process_rule")
     @console_ns.doc(description="Get dataset document processing rules")
     @console_ns.doc(params={"document_id": "Document ID (optional)"})
-    @console_ns.response(200, "Process rules retrieved successfully", console_ns.models[OpaqueObjectResponse.__name__])
+    @console_ns.response(200, "Process rules retrieved successfully", console_ns.models[ProcessRuleResponse.__name__])
     @setup_required
     @login_required
     @account_initialization_required
@@ -243,13 +331,13 @@ class GetProcessRuleApi(Resource):
             # get the latest process rule
             document = db.get_or_404(Document, document_id)
 
-            dataset = DatasetService.get_dataset(document.dataset_id, db.session)
+            dataset = DatasetService.get_dataset(document.dataset_id, db.session())
 
             if not dataset:
                 raise NotFound("Dataset not found.")
 
             try:
-                DatasetService.check_dataset_permission(dataset, current_user, db.session)
+                DatasetService.check_dataset_permission(dataset, current_user, db.session())
             except services.errors.account.NoPermissionError as e:
                 raise Forbidden(str(e))
 
@@ -264,7 +352,7 @@ class GetProcessRuleApi(Resource):
                 mode = dataset_process_rule.mode
                 rules = dataset_process_rule.rules_dict
 
-        return {"mode": mode, "rules": rules, "limits": limits}
+        return dump_response(ProcessRuleResponse, {"mode": mode, "rules": rules, "limits": limits})
 
 
 @console_ns.route("/datasets/<uuid:dataset_id>/documents")
@@ -319,12 +407,12 @@ class DatasetDocumentListApi(Resource):
                     )
         except (ArgumentTypeError, ValueError, Exception):
             fetch = False
-        dataset = DatasetService.get_dataset(dataset_id_str, db.session)
+        dataset = DatasetService.get_dataset(dataset_id_str, db.session())
         if not dataset:
             raise NotFound("Dataset not found.")
 
         try:
-            DatasetService.check_dataset_permission(dataset, current_user, db.session)
+            DatasetService.check_dataset_permission(dataset, current_user, db.session())
         except services.errors.account.NoPermissionError as e:
             raise Forbidden(str(e))
 
@@ -376,6 +464,7 @@ class DatasetDocumentListApi(Resource):
             documents=documents,
             dataset=dataset,
             tenant_id=current_tenant_id,
+            session=db.session(),
         )
 
         if fetch:
@@ -423,7 +512,7 @@ class DatasetDocumentListApi(Resource):
     def post(self, current_user: Account, dataset_id: UUID):
         dataset_id_str = str(dataset_id)
 
-        dataset = DatasetService.get_dataset(dataset_id_str, db.session)
+        dataset = DatasetService.get_dataset(dataset_id_str, db.session())
 
         if not dataset:
             raise NotFound("Dataset not found.")
@@ -433,7 +522,7 @@ class DatasetDocumentListApi(Resource):
             raise Forbidden()
 
         try:
-            DatasetService.check_dataset_permission(dataset, current_user, db.session)
+            DatasetService.check_dataset_permission(dataset, current_user, db.session())
         except services.errors.account.NoPermissionError as e:
             raise Forbidden(str(e))
 
@@ -447,9 +536,9 @@ class DatasetDocumentListApi(Resource):
 
         try:
             documents, batch = DocumentService.save_document_with_dataset_id(
-                dataset, knowledge_config, current_user, session=db.session
+                dataset, knowledge_config, current_user, session=db.session()
             )
-            dataset = DatasetService.get_dataset(dataset_id_str, db.session)
+            dataset = DatasetService.get_dataset(dataset_id_str, db.session())
 
         except ProviderTokenNotInitError as ex:
             raise ProviderNotInitializeError(ex.description)
@@ -468,7 +557,7 @@ class DatasetDocumentListApi(Resource):
     @rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_EDIT)
     def delete(self, dataset_id: UUID):
         dataset_id_str = str(dataset_id)
-        dataset = DatasetService.get_dataset(dataset_id_str, db.session)
+        dataset = DatasetService.get_dataset(dataset_id_str, db.session())
         if dataset is None:
             raise NotFound("Dataset not found.")
         # check user's model setting
@@ -477,7 +566,7 @@ class DatasetDocumentListApi(Resource):
         try:
             document_ids = request.args.getlist("document_id")
             dataset_ref = DatasetRefService.create_dataset_ref(dataset)
-            DocumentService.delete_documents(dataset_ref, document_ids, dataset.doc_form, db.session)
+            DocumentService.delete_documents(dataset_ref, document_ids, dataset.doc_form, db.session())
         except services.errors.document.DocumentIndexingError:
             raise DocumentIndexingError("Cannot delete document during indexing.")
 
@@ -490,7 +579,7 @@ class DatasetInitApi(Resource):
     @console_ns.doc(description="Initialize dataset with documents")
     @console_ns.expect(console_ns.models[KnowledgeConfig.__name__])
     @console_ns.response(
-        201, "Dataset initialized successfully", console_ns.models[DatasetAndDocumentResponse.__name__]
+        200, "Dataset initialized successfully", console_ns.models[DatasetAndDocumentResponse.__name__]
     )
     @console_ns.response(400, "Invalid request parameters")
     @setup_required
@@ -536,7 +625,7 @@ class DatasetInitApi(Resource):
                 tenant_id=current_tenant_id,
                 knowledge_config=knowledge_config,
                 account=current_user,
-                session=db.session,
+                session=db.session(),
             )
         except ProviderTokenNotInitError as ex:
             raise ProviderNotInitializeError(ex.description)
@@ -556,7 +645,7 @@ class DocumentIndexingEstimateApi(DocumentResource):
     @console_ns.response(
         200,
         "Indexing estimate calculated successfully",
-        console_ns.models[OpaqueObjectResponse.__name__],
+        console_ns.models[IndexingEstimateResponse.__name__],
     )
     @console_ns.response(404, "Document not found")
     @console_ns.response(400, "Document already finished")
@@ -577,8 +666,6 @@ class DocumentIndexingEstimateApi(DocumentResource):
         data_process_rule = document.dataset_process_rule
         data_process_rule_dict = data_process_rule.to_dict() if data_process_rule else {}
 
-        response = {"tokens": 0, "total_price": 0, "currency": "USD", "total_segments": 0, "preview": []}
-
         if document.data_source_type == "upload_file":
             data_source_info = document.data_source_info_dict
             if data_source_info and "upload_file_id" in data_source_info:
@@ -609,7 +696,18 @@ class DocumentIndexingEstimateApi(DocumentResource):
                         "English",
                         dataset_id_str,
                     )
-                    return estimate_response.model_dump(), 200
+                    return (
+                        # TODO: why using zero here? the same for the below endpoint
+                        IndexingEstimateResponse(
+                            tokens=0,
+                            total_price=0,
+                            currency="USD",
+                            total_segments=estimate_response.total_segments,
+                            preview=estimate_response.preview,
+                            qa_preview=estimate_response.qa_preview,
+                        ).model_dump(mode="json", exclude_none=True),
+                        200,
+                    )
                 except LLMBadRequestError:
                     raise ProviderNotInitializeError(
                         "No Embedding Model available. Please configure a valid provider "
@@ -622,15 +720,24 @@ class DocumentIndexingEstimateApi(DocumentResource):
                 except Exception as e:
                     raise IndexingEstimateError(str(e))
 
-        return response, 200
+        return (
+            IndexingEstimateResponse(
+                tokens=0,
+                total_price=0,
+                currency="USD",
+                total_segments=0,
+                preview=[],
+            ).model_dump(mode="json", exclude_none=True),
+            200,
+        )
 
 
 @console_ns.route("/datasets/<uuid:dataset_id>/batch/<string:batch>/indexing-estimate")
 class DocumentBatchIndexingEstimateApi(DocumentResource):
     @console_ns.response(
         200,
-        "Batch indexing estimate calculated successfully",
-        console_ns.models[OpaqueObjectResponse.__name__],
+        "Indexing estimate calculated successfully",
+        console_ns.models[IndexingEstimateResponse.__name__],
     )
     @setup_required
     @login_required
@@ -642,7 +749,16 @@ class DocumentBatchIndexingEstimateApi(DocumentResource):
         dataset_id_str = str(dataset_id)
         documents = self.get_batch_documents(dataset_id_str, batch, current_user)
         if not documents:
-            return {"tokens": 0, "total_price": 0, "currency": "USD", "total_segments": 0, "preview": []}, 200
+            return (
+                IndexingEstimateResponse(
+                    tokens=0,
+                    total_price=0,
+                    currency="USD",
+                    total_segments=0,
+                    preview=[],
+                ).model_dump(mode="json", exclude_none=True),
+                200,
+            )
         data_process_rule = documents[0].dataset_process_rule
         data_process_rule_dict = data_process_rule.to_dict() if data_process_rule else {}
         extract_settings = []
@@ -716,7 +832,17 @@ class DocumentBatchIndexingEstimateApi(DocumentResource):
                     "English",
                     dataset_id_str,
                 )
-                return response.model_dump(), 200
+                return (
+                    IndexingEstimateResponse(
+                        tokens=0,
+                        total_price=0,
+                        currency="USD",
+                        total_segments=response.total_segments,
+                        preview=response.preview,
+                        qa_preview=response.qa_preview,
+                    ).model_dump(mode="json", exclude_none=True),
+                    200,
+                )
             except LLMBadRequestError:
                 raise ProviderNotInitializeError(
                     "No Embedding Model available. Please configure a valid provider in the Settings -> Model Provider."
@@ -853,7 +979,7 @@ class DocumentApi(DocumentResource):
             "metadata": "Metadata inclusion (all/only/without)",
         }
     )
-    @console_ns.response(200, "Document retrieved successfully", console_ns.models[OpaqueObjectResponse.__name__])
+    @console_ns.response(200, "Document retrieved successfully", console_ns.models[DocumentDetailResponse.__name__])
     @console_ns.response(404, "Document not found")
     @setup_required
     @login_required
@@ -870,46 +996,21 @@ class DocumentApi(DocumentResource):
         if metadata not in self.METADATA_CHOICES:
             raise InvalidMetadataError(f"Invalid metadata value: {metadata}")
 
+        metadata_fields = {"doc_type", "doc_metadata"}
         if metadata == "only":
-            response = {"id": document.id, "doc_type": document.doc_type, "doc_metadata": document.doc_metadata_details}
-        elif metadata == "without":
-            dataset_process_rules = DatasetService.get_process_rules(dataset_id_str, db.session)
-            document_process_rules = document.dataset_process_rule.to_dict() if document.dataset_process_rule else {}
-            response = {
-                "id": document.id,
-                "position": document.position,
-                "data_source_type": document.data_source_type,
-                "data_source_info": document.data_source_info_dict,
-                "data_source_detail_dict": document.data_source_detail_dict,
-                "dataset_process_rule_id": document.dataset_process_rule_id,
-                "dataset_process_rule": dataset_process_rules,
-                "document_process_rule": document_process_rules,
-                "name": document.name,
-                "created_from": document.created_from,
-                "created_by": document.created_by,
-                "created_at": int(document.created_at.timestamp()),
-                "tokens": document.tokens,
-                "indexing_status": document.indexing_status,
-                "completed_at": int(document.completed_at.timestamp()) if document.completed_at else None,
-                "updated_at": int(document.updated_at.timestamp()) if document.updated_at else None,
-                "indexing_latency": document.indexing_latency,
-                "error": document.error,
-                "enabled": document.enabled,
-                "disabled_at": int(document.disabled_at.timestamp()) if document.disabled_at else None,
-                "disabled_by": document.disabled_by,
-                "archived": document.archived,
-                "segment_count": document.segment_count,
-                "average_segment_length": document.average_segment_length,
-                "hit_count": document.hit_count,
-                "display_status": document.display_status,
-                "doc_form": document.doc_form,
-                "doc_language": document.doc_language,
-                "need_summary": document.need_summary if document.need_summary is not None else False,
-            }
-        else:
-            dataset_process_rules = DatasetService.get_process_rules(dataset_id_str, db.session)
-            document_process_rules = document.dataset_process_rule.to_dict() if document.dataset_process_rule else {}
-            response = {
+            response = DocumentDetailResponse.model_validate(
+                {
+                    "id": document.id,
+                    "doc_type": document.doc_type,
+                    "doc_metadata": document.doc_metadata_details,
+                }
+            )
+            return response.model_dump(mode="json", include={"id", *metadata_fields}, exclude_unset=True), 200
+
+        dataset_process_rules = DatasetService.get_process_rules(dataset_id_str, db.session())
+        document_process_rules = document.dataset_process_rule.to_dict() if document.dataset_process_rule else {}
+        response = DocumentDetailResponse.model_validate(
+            {
                 "id": document.id,
                 "position": document.position,
                 "data_source_type": document.data_source_type,
@@ -942,8 +1043,9 @@ class DocumentApi(DocumentResource):
                 "doc_language": document.doc_language,
                 "need_summary": document.need_summary if document.need_summary is not None else False,
             }
-
-        return response, 200
+        )
+        exclude = metadata_fields if metadata == "without" else None
+        return response.model_dump(mode="json", exclude=exclude, exclude_unset=True), 200
 
     @setup_required
     @login_required
@@ -956,7 +1058,7 @@ class DocumentApi(DocumentResource):
     def delete(self, current_tenant_id: str, current_user: Account, dataset_id: UUID, document_id: UUID):
         dataset_id_str = str(dataset_id)
         document_id_str = str(document_id)
-        dataset = DatasetService.get_dataset(dataset_id_str, db.session)
+        dataset = DatasetService.get_dataset(dataset_id_str, db.session())
         if dataset is None:
             raise NotFound("Dataset not found.")
         # check user's model setting
@@ -965,7 +1067,7 @@ class DocumentApi(DocumentResource):
         document = self.get_document(dataset_id_str, document_id_str, current_user, current_tenant_id)
 
         try:
-            DocumentService.delete_document(document, db.session)
+            DocumentService.delete_document(document, db.session())
         except services.errors.document.DocumentIndexingError:
             raise DocumentIndexingError("Cannot delete document during indexing.")
 
@@ -989,7 +1091,9 @@ class DocumentDownloadApi(DocumentResource):
     def get(self, current_tenant_id: str, current_user: Account, dataset_id: UUID, document_id: UUID) -> dict[str, Any]:
         # Reuse the shared permission/tenant checks implemented in DocumentResource.
         document = self.get_document(str(dataset_id), str(document_id), current_user, current_tenant_id)
-        return {"url": DocumentService.get_document_download_url(document, db.session)}
+        return UrlResponse(url=DocumentService.get_document_download_url(document, db.session())).model_dump(
+            mode="json"
+        )
 
 
 @console_ns.route("/datasets/<uuid:dataset_id>/documents/download-zip")
@@ -998,7 +1102,7 @@ class DocumentBatchDownloadZipApi(DocumentResource):
 
     @console_ns.doc("download_dataset_documents_as_zip")
     @console_ns.doc(description="Download selected dataset documents as a single ZIP archive (upload-file only)")
-    @console_ns.response(200, "ZIP archive generated successfully", console_ns.models[BinaryFileResponse.__name__])
+    @console_ns.response(200, "ZIP archive downloaded successfully")
     @setup_required
     @login_required
     @account_initialization_required
@@ -1019,7 +1123,7 @@ class DocumentBatchDownloadZipApi(DocumentResource):
             document_ids=document_ids,
             tenant_id=current_tenant_id,
             current_user=current_user,
-            session=db.session,
+            session=db.session(),
         )
 
         # Delegate ZIP packing to FileService, but keep Flask response+cleanup in the route.
@@ -1033,6 +1137,7 @@ class DocumentBatchDownloadZipApi(DocumentResource):
             )
             cleanup = stack.pop_all()
             response.call_on_close(cleanup.close)
+        # response-contract:ignore binary ZIP download response
         return response
 
 
@@ -1092,7 +1197,7 @@ class DocumentProcessingApi(DocumentResource):
                 document.is_paused = False
                 db.session.commit()
 
-        return {"result": "success"}, 200
+        return SimpleResultResponse(result="success").model_dump(mode="json"), 200
 
 
 @console_ns.route("/datasets/<uuid:dataset_id>/documents/<uuid:document_id>/metadata")
@@ -1151,7 +1256,9 @@ class DocumentMetadataApi(DocumentResource):
         document.updated_at = naive_utc_now()
         db.session.commit()
 
-        return {"result": "success", "message": "Document metadata updated."}, 200
+        return SimpleResultMessageResponse(result="success", message="Document metadata updated.").model_dump(
+            mode="json"
+        ), 200
 
 
 @console_ns.route("/datasets/<uuid:dataset_id>/documents/status/<string:action>/batch")
@@ -1168,7 +1275,7 @@ class DocumentStatusApi(DocumentResource):
         self, current_user: Account, dataset_id: UUID, action: Literal["enable", "disable", "archive", "un_archive"]
     ):
         dataset_id_str = str(dataset_id)
-        dataset = DatasetService.get_dataset(dataset_id_str, db.session)
+        dataset = DatasetService.get_dataset(dataset_id_str, db.session())
         if dataset is None:
             raise NotFound("Dataset not found.")
 
@@ -1180,12 +1287,12 @@ class DocumentStatusApi(DocumentResource):
         DatasetService.check_dataset_model_setting(dataset)
 
         # check user's permission
-        DatasetService.check_dataset_permission(dataset, current_user, db.session)
+        DatasetService.check_dataset_permission(dataset, current_user, db.session())
 
         document_ids = request.args.getlist("document_id")
 
         try:
-            DocumentService.batch_update_document_status(dataset, document_ids, action, current_user, db.session)
+            DocumentService.batch_update_document_status(dataset, document_ids, action, current_user, db.session())
         except services.errors.document.DocumentIndexingError as e:
             raise InvalidActionError(str(e))
         except ValueError as e:
@@ -1193,7 +1300,7 @@ class DocumentStatusApi(DocumentResource):
         except NotFound as e:
             raise NotFound(str(e))
 
-        return {"result": "success"}, 200
+        return SimpleResultResponse(result="success").model_dump(mode="json"), 200
 
 
 @console_ns.route("/datasets/<uuid:dataset_id>/documents/<uuid:document_id>/processing/pause")
@@ -1209,11 +1316,11 @@ class DocumentPauseApi(DocumentResource):
         dataset_id_str = str(dataset_id)
         document_id_str = str(document_id)
 
-        dataset = DatasetService.get_dataset(dataset_id_str, db.session)
+        dataset = DatasetService.get_dataset(dataset_id_str, db.session())
         if not dataset:
             raise NotFound("Dataset not found.")
 
-        document = DocumentService.get_document(dataset.id, document_id_str, session=db.session)
+        document = DocumentService.get_document(dataset.id, document_id_str, session=db.session())
 
         # 404 if document not found
         if document is None:
@@ -1225,7 +1332,7 @@ class DocumentPauseApi(DocumentResource):
 
         try:
             # pause document
-            DocumentService.pause_document(document, db.session)
+            DocumentService.pause_document(document, db.session())
         except services.errors.document.DocumentIndexingError:
             raise DocumentIndexingError("Cannot pause completed document.")
 
@@ -1244,10 +1351,10 @@ class DocumentRecoverApi(DocumentResource):
         """recover document."""
         dataset_id_str = str(dataset_id)
         document_id_str = str(document_id)
-        dataset = DatasetService.get_dataset(dataset_id_str, db.session)
+        dataset = DatasetService.get_dataset(dataset_id_str, db.session())
         if not dataset:
             raise NotFound("Dataset not found.")
-        document = DocumentService.get_document(dataset.id, document_id_str, session=db.session)
+        document = DocumentService.get_document(dataset.id, document_id_str, session=db.session())
 
         # 404 if document not found
         if document is None:
@@ -1258,7 +1365,7 @@ class DocumentRecoverApi(DocumentResource):
             raise ArchivedDocumentImmutableError()
         try:
             # pause document
-            DocumentService.recover_document(document, db.session)
+            DocumentService.recover_document(document, db.session())
         except services.errors.document.DocumentIndexingError:
             raise DocumentIndexingError("Document is not in paused status.")
 
@@ -1278,13 +1385,13 @@ class DocumentRetryApi(DocumentResource):
         """retry document."""
         payload = DocumentRetryPayload.model_validate(console_ns.payload or {})
         dataset_id_str = str(dataset_id)
-        dataset = DatasetService.get_dataset(dataset_id_str, db.session)
+        dataset = DatasetService.get_dataset(dataset_id_str, db.session())
         retry_documents = []
         if not dataset:
             raise NotFound("Dataset not found.")
         for document_id in payload.document_ids:
             try:
-                document = DocumentService.get_document(dataset.id, document_id, session=db.session)
+                document = DocumentService.get_document(dataset.id, document_id, session=db.session())
 
                 # 404 if document not found
                 if document is None:
@@ -1302,7 +1409,7 @@ class DocumentRetryApi(DocumentResource):
                 logger.exception("Failed to retry document, document id: %s", document_id)
                 continue
         # retry document
-        DocumentService.retry_document(dataset_id_str, retry_documents, db.session)
+        DocumentService.retry_document(dataset_id_str, retry_documents, db.session())
 
         return "", 204
 
@@ -1320,14 +1427,14 @@ class DocumentRenameApi(DocumentResource):
         # The role of the current user in the ta table must be admin, owner, editor, or dataset_operator
         if not current_user.is_dataset_editor:
             raise Forbidden()
-        dataset = DatasetService.get_dataset(dataset_id, db.session)
+        dataset = DatasetService.get_dataset(str(dataset_id), db.session())
         if not dataset:
             raise NotFound("Dataset not found.")
-        DatasetService.check_dataset_operator_permission(current_user, dataset, session=db.session)
+        DatasetService.check_dataset_operator_permission(current_user, dataset, session=db.session())
         payload = DocumentRenamePayload.model_validate(console_ns.payload or {})
 
         try:
-            document = DocumentService.rename_document(str(dataset_id), str(document_id), payload.name, db.session)
+            document = DocumentService.rename_document(str(dataset_id), str(document_id), payload.name, db.session())
         except services.errors.document.DocumentIndexingError:
             raise DocumentIndexingError("Cannot delete document during indexing.")
 
@@ -1345,11 +1452,11 @@ class WebsiteDocumentSyncApi(DocumentResource):
     def get(self, current_tenant_id: str, dataset_id: UUID, document_id: UUID):
         """sync website document."""
         dataset_id_str = str(dataset_id)
-        dataset = DatasetService.get_dataset(dataset_id_str, db.session)
+        dataset = DatasetService.get_dataset(dataset_id_str, db.session())
         if not dataset:
             raise NotFound("Dataset not found.")
         document_id_str = str(document_id)
-        document = DocumentService.get_document(dataset.id, document_id_str, session=db.session)
+        document = DocumentService.get_document(dataset.id, document_id_str, session=db.session())
         if not document:
             raise NotFound("Document not found.")
         if document.tenant_id != current_tenant_id:
@@ -1360,17 +1467,17 @@ class WebsiteDocumentSyncApi(DocumentResource):
         if DocumentService.check_archived(document):
             raise ArchivedDocumentImmutableError()
         # sync document
-        DocumentService.sync_website_document(dataset_id_str, document, db.session)
+        DocumentService.sync_website_document(dataset_id_str, document, db.session())
 
-        return {"result": "success"}, 200
+        return SimpleResultResponse(result="success").model_dump(mode="json"), 200
 
 
 @console_ns.route("/datasets/<uuid:dataset_id>/documents/<uuid:document_id>/pipeline-execution-log")
 class DocumentPipelineExecutionLogApi(DocumentResource):
     @console_ns.response(
         200,
-        "Document pipeline execution log retrieved successfully",
-        console_ns.models[OpaqueObjectResponse.__name__],
+        "Pipeline execution log retrieved successfully",
+        console_ns.models[DocumentPipelineExecutionLogResponse.__name__],
     )
     @setup_required
     @login_required
@@ -1380,10 +1487,10 @@ class DocumentPipelineExecutionLogApi(DocumentResource):
         dataset_id_str = str(dataset_id)
         document_id_str = str(document_id)
 
-        dataset = DatasetService.get_dataset(dataset_id_str, db.session)
+        dataset = DatasetService.get_dataset(dataset_id_str, db.session())
         if not dataset:
             raise NotFound("Dataset not found.")
-        document = DocumentService.get_document(dataset.id, document_id_str, session=db.session)
+        document = DocumentService.get_document(dataset.id, document_id_str, session=db.session())
         if not document:
             raise NotFound("Document not found.")
         log = db.session.scalar(
@@ -1393,18 +1500,16 @@ class DocumentPipelineExecutionLogApi(DocumentResource):
             .limit(1)
         )
         if not log:
-            return {
-                "datasource_info": None,
-                "datasource_type": None,
-                "input_data": None,
-                "datasource_node_id": None,
-            }, 200
-        return {
-            "datasource_info": json.loads(log.datasource_info),
-            "datasource_type": log.datasource_type,
-            "input_data": log.input_data,
-            "datasource_node_id": log.datasource_node_id,
-        }, 200
+            return DocumentPipelineExecutionLogResponse().model_dump(mode="json"), 200
+        return dump_response(
+            DocumentPipelineExecutionLogResponse,
+            {
+                "datasource_info": json.loads(log.datasource_info),
+                "datasource_type": log.datasource_type,
+                "input_data": log.input_data,
+                "datasource_node_id": log.datasource_node_id,
+            },
+        ), 200
 
 
 @console_ns.route("/datasets/<uuid:dataset_id>/documents/generate-summary")
@@ -1438,7 +1543,7 @@ class DocumentGenerateSummaryApi(Resource):
         dataset_id_str = str(dataset_id)
 
         # Get dataset
-        dataset = DatasetService.get_dataset(dataset_id_str, db.session)
+        dataset = DatasetService.get_dataset(dataset_id_str, db.session())
         if not dataset:
             raise NotFound("Dataset not found.")
 
@@ -1447,7 +1552,7 @@ class DocumentGenerateSummaryApi(Resource):
             raise Forbidden()
 
         try:
-            DatasetService.check_dataset_permission(dataset, current_user, db.session)
+            DatasetService.check_dataset_permission(dataset, current_user, db.session())
         except services.errors.account.NoPermissionError as e:
             raise Forbidden(str(e))
 
@@ -1472,7 +1577,7 @@ class DocumentGenerateSummaryApi(Resource):
             raise ValueError("Summary index is not enabled for this dataset. Please enable it in the dataset settings.")
 
         # Verify all documents exist and belong to the dataset
-        documents = DocumentService.get_documents_by_ids(dataset_id_str, document_list, db.session)
+        documents = DocumentService.get_documents_by_ids(dataset_id_str, document_list, db.session())
 
         if len(documents) != len(document_list):
             found_ids = {doc.id for doc in documents}
@@ -1488,7 +1593,7 @@ class DocumentGenerateSummaryApi(Resource):
             DocumentService.update_documents_need_summary(
                 dataset_id=dataset_id_str,
                 document_ids=document_ids_to_update,
-                session=db.session,
+                session=db.session(),
                 need_summary=True,
             )
 
@@ -1507,7 +1612,7 @@ class DocumentGenerateSummaryApi(Resource):
                 dataset_id_str,
             )
 
-        return {"result": "success"}, 200
+        return SimpleResultResponse(result="success").model_dump(mode="json"), 200
 
 
 @console_ns.route("/datasets/<uuid:dataset_id>/documents/<uuid:document_id>/summary-status")
@@ -1515,7 +1620,11 @@ class DocumentSummaryStatusApi(DocumentResource):
     @console_ns.doc("get_document_summary_status")
     @console_ns.doc(description="Get summary index generation status for a document")
     @console_ns.doc(params={"dataset_id": "Dataset ID", "document_id": "Document ID"})
-    @console_ns.response(200, "Summary status retrieved successfully", console_ns.models[OpaqueObjectResponse.__name__])
+    @console_ns.response(
+        200,
+        "Summary status retrieved successfully",
+        console_ns.models[DocumentSummaryStatusResponse.__name__],
+    )
     @console_ns.response(404, "Document not found")
     @setup_required
     @login_required
@@ -1533,19 +1642,20 @@ class DocumentSummaryStatusApi(DocumentResource):
           - generating: Number of summaries being generated
           - error: Number of summaries with errors
           - not_started: Number of segments without summary records
+          - timeout: Number of summaries that timed out
         - summaries: List of summary records with status and content preview
         """
         dataset_id_str = str(dataset_id)
         document_id_str = str(document_id)
 
         # Get dataset
-        dataset = DatasetService.get_dataset(dataset_id_str, db.session)
+        dataset = DatasetService.get_dataset(dataset_id_str, db.session())
         if not dataset:
             raise NotFound("Dataset not found.")
 
         # Check permissions
         try:
-            DatasetService.check_dataset_permission(dataset, current_user, db.session)
+            DatasetService.check_dataset_permission(dataset, current_user, db.session())
         except services.errors.account.NoPermissionError as e:
             raise Forbidden(str(e))
 
@@ -1555,7 +1665,7 @@ class DocumentSummaryStatusApi(DocumentResource):
         result = SummaryIndexService.get_document_summary_status_detail(
             document_id=document_id_str,
             dataset_id=dataset_id_str,
-            session=db.session,
+            session=db.session(),
         )
 
-        return result, 200
+        return dump_response(DocumentSummaryStatusResponse, result), 200

api/controllers/console/datasets/datasets_segments.py

@@ -173,7 +173,7 @@ def _get_segment_for_document(
         raise NotFound("Document not found.")
 
     segment_ref = DatasetRefService.create_segment_ref(document_ref, segment_id)
-    segment = SegmentService.get_segment_by_ref(segment_ref)
+    segment = SegmentService.get_segment_by_ref(segment_ref, db.session())
     if not segment:
         raise NotFound("Segment not found.")
     return segment_ref, segment
@@ -193,16 +193,16 @@ class DatasetDocumentSegmentListApi(Resource):
     def get(self, current_tenant_id: str, current_user: Account, dataset_id: UUID, document_id: UUID):
         dataset_id_str = str(dataset_id)
         document_id_str = str(document_id)
-        dataset = DatasetService.get_dataset(dataset_id_str, db.session)
+        dataset = DatasetService.get_dataset(dataset_id_str, db.session())
         if not dataset:
             raise NotFound("Dataset not found.")
 
         try:
-            DatasetService.check_dataset_permission(dataset, current_user, db.session)
+            DatasetService.check_dataset_permission(dataset, current_user, db.session())
         except services.errors.account.NoPermissionError as e:
             raise Forbidden(str(e))
 
-        document = DocumentService.get_document(dataset_id_str, document_id_str, session=db.session)
+        document = DocumentService.get_document(dataset_id_str, document_id_str, session=db.session())
 
         if not document:
             raise NotFound("Document not found.")
@@ -278,7 +278,7 @@ class DatasetDocumentSegmentListApi(Resource):
         summaries: dict[str, str | None] = {}
         if segment_ids:
             summary_records = SummaryIndexService.get_segments_summaries(
-                segment_ids=segment_ids, dataset_id=dataset_id_str
+                segment_ids=segment_ids, dataset_id=dataset_id_str, session=db.session()
             )
             summaries = {chunk_id: summary.summary_content for chunk_id, summary in summary_records.items()}
 
@@ -303,14 +303,14 @@ class DatasetDocumentSegmentListApi(Resource):
     def delete(self, current_user: Account, dataset_id: UUID, document_id: UUID):
         # check dataset
         dataset_id_str = str(dataset_id)
-        dataset = DatasetService.get_dataset(dataset_id_str, db.session)
+        dataset = DatasetService.get_dataset(dataset_id_str, db.session())
         if not dataset:
             raise NotFound("Dataset not found.")
         # check user's model setting
         DatasetService.check_dataset_model_setting(dataset)
         # check document
         document_id_str = str(document_id)
-        document = DocumentService.get_document(dataset_id_str, document_id_str, session=db.session)
+        document = DocumentService.get_document(dataset_id_str, document_id_str, session=db.session())
         if not document:
             raise NotFound("Document not found.")
         segment_ids = request.args.getlist("segment_id")
@@ -319,10 +319,10 @@ class DatasetDocumentSegmentListApi(Resource):
         if not current_user.is_dataset_editor:
             raise Forbidden()
         try:
-            DatasetService.check_dataset_permission(dataset, current_user, db.session)
+            DatasetService.check_dataset_permission(dataset, current_user, db.session())
         except services.errors.account.NoPermissionError as e:
             raise Forbidden(str(e))
-        SegmentService.delete_segments(segment_ids, document, dataset, db.session)
+        SegmentService.delete_segments(segment_ids, document, dataset, db.session())
         return "", 204
 
 
@@ -348,11 +348,11 @@ class DatasetDocumentSegmentApi(Resource):
         action: Literal["enable", "disable"],
     ):
         dataset_id_str = str(dataset_id)
-        dataset = DatasetService.get_dataset(dataset_id_str, db.session)
+        dataset = DatasetService.get_dataset(dataset_id_str, db.session())
         if not dataset:
             raise NotFound("Dataset not found.")
         document_id_str = str(document_id)
-        document = DocumentService.get_document(dataset_id_str, document_id_str, session=db.session)
+        document = DocumentService.get_document(dataset_id_str, document_id_str, session=db.session())
         if not document:
             raise NotFound("Document not found.")
         # check user's model setting
@@ -362,7 +362,7 @@ class DatasetDocumentSegmentApi(Resource):
             raise Forbidden()
 
         try:
-            DatasetService.check_dataset_permission(dataset, current_user, db.session)
+            DatasetService.check_dataset_permission(dataset, current_user, db.session())
         except services.errors.account.NoPermissionError as e:
             raise Forbidden(str(e))
         if dataset.indexing_technique == IndexTechniqueType.HIGH_QUALITY:
@@ -388,10 +388,10 @@ class DatasetDocumentSegmentApi(Resource):
         if cache_result is not None:
             raise InvalidActionError("Document is being indexed, please try again later")
         try:
-            SegmentService.update_segments_status(segment_ids, action, dataset, document, db.session)
+            SegmentService.update_segments_status(segment_ids, action, dataset, document, db.session())
         except Exception as e:
             raise InvalidActionError(str(e))
-        return dump_response(SimpleResultResponse, {"result": "success"}), 200
+        return SimpleResultResponse(result="success").model_dump(mode="json"), 200
 
 
 @console_ns.route("/datasets/<uuid:dataset_id>/documents/<uuid:document_id>/segment")
@@ -411,12 +411,12 @@ class DatasetDocumentSegmentAddApi(Resource):
     def post(self, current_tenant_id: str, current_user: Account, dataset_id: UUID, document_id: UUID):
         # check dataset
         dataset_id_str = str(dataset_id)
-        dataset = DatasetService.get_dataset(dataset_id_str, db.session)
+        dataset = DatasetService.get_dataset(dataset_id_str, db.session())
         if not dataset:
             raise NotFound("Dataset not found.")
         # check document
         document_id_str = str(document_id)
-        document = DocumentService.get_document(dataset_id_str, document_id_str, session=db.session)
+        document = DocumentService.get_document(dataset_id_str, document_id_str, session=db.session())
         if not document:
             raise NotFound("Document not found.")
         if not current_user.is_dataset_editor:
@@ -438,15 +438,20 @@ class DatasetDocumentSegmentAddApi(Resource):
             except ProviderTokenNotInitError as ex:
                 raise ProviderNotInitializeError(ex.description)
         try:
-            DatasetService.check_dataset_permission(dataset, current_user, db.session)
+            DatasetService.check_dataset_permission(dataset, current_user, db.session())
         except services.errors.account.NoPermissionError as e:
             raise Forbidden(str(e))
         # validate args
         payload = SegmentCreatePayload.model_validate(console_ns.payload or {})
         payload_dict = payload.model_dump(exclude_none=True)
         SegmentService.segment_create_args_validate(payload_dict, document)
-        segment = type_cast(DocumentSegment, SegmentService.create_segment(payload_dict, document, dataset, db.session))
-        summary = SummaryIndexService.get_segment_summary(segment_id=segment.id, dataset_id=dataset_id_str)
+        segment = type_cast(
+            DocumentSegment,
+            SegmentService.create_segment(payload_dict, document, dataset, db.session()),
+        )
+        summary = SummaryIndexService.get_segment_summary(
+            segment_id=segment.id, dataset_id=dataset_id_str, session=db.session()
+        )
         response = {
             "data": segment_response_with_summary(segment, summary.summary_content if summary else None),
             "doc_form": document.doc_form,
@@ -472,21 +477,21 @@ class DatasetDocumentSegmentUpdateApi(Resource):
     ):
         # check dataset
         dataset_id_str = str(dataset_id)
-        dataset = DatasetService.get_dataset(dataset_id_str, db.session)
+        dataset = DatasetService.get_dataset(dataset_id_str, db.session())
         if not dataset:
             raise NotFound("Dataset not found.")
         # check user's model setting
         DatasetService.check_dataset_model_setting(dataset)
         # check document
         document_id_str = str(document_id)
-        document = DocumentService.get_document(dataset_id_str, document_id_str, session=db.session)
+        document = DocumentService.get_document(dataset_id_str, document_id_str, session=db.session())
         if not document:
             raise NotFound("Document not found.")
         # The role of the current user in the ta table must be admin, owner, dataset_operator, or editor
         if not current_user.is_dataset_editor:
             raise Forbidden()
         try:
-            DatasetService.check_dataset_permission(dataset, current_user, db.session)
+            DatasetService.check_dataset_permission(dataset, current_user, db.session())
         except services.errors.account.NoPermissionError as e:
             raise Forbidden(str(e))
         if dataset.indexing_technique == IndexTechniqueType.HIGH_QUALITY:
@@ -518,9 +523,11 @@ class DatasetDocumentSegmentUpdateApi(Resource):
             segment,
             document,
             dataset,
-            db.session,
+            db.session(),
+        )
+        summary = SummaryIndexService.get_segment_summary(
+            segment_id=segment.id, dataset_id=dataset_id_str, session=db.session()
         )
-        summary = SummaryIndexService.get_segment_summary(segment_id=segment.id, dataset_id=dataset_id_str)
         response = {
             "data": segment_response_with_summary(segment, summary.summary_content if summary else None),
             "doc_form": document.doc_form,
@@ -541,26 +548,26 @@ class DatasetDocumentSegmentUpdateApi(Resource):
     ):
         # check dataset
         dataset_id_str = str(dataset_id)
-        dataset = DatasetService.get_dataset(dataset_id_str, db.session)
+        dataset = DatasetService.get_dataset(dataset_id_str, db.session())
         if not dataset:
             raise NotFound("Dataset not found.")
         # check user's model setting
         DatasetService.check_dataset_model_setting(dataset)
         # check document
         document_id_str = str(document_id)
-        document = DocumentService.get_document(dataset_id_str, document_id_str, session=db.session)
+        document = DocumentService.get_document(dataset_id_str, document_id_str, session=db.session())
         if not document:
             raise NotFound("Document not found.")
         # The role of the current user in the ta table must be admin, owner, dataset_operator, or editor
         if not current_user.is_dataset_editor:
             raise Forbidden()
         try:
-            DatasetService.check_dataset_permission(dataset, current_user, db.session)
+            DatasetService.check_dataset_permission(dataset, current_user, db.session())
         except services.errors.account.NoPermissionError as e:
             raise Forbidden(str(e))
         segment_id_str = str(segment_id)
         _, segment = _get_segment_for_document(dataset, document, segment_id_str)
-        SegmentService.delete_segment(segment, document, dataset, db.session)
+        SegmentService.delete_segment(segment, document, dataset, db.session())
         return "", 204
 
 
@@ -583,12 +590,12 @@ class DatasetDocumentSegmentBatchImportApi(Resource):
     def post(self, current_tenant_id: str, current_user: Account, dataset_id: UUID, document_id: UUID):
         # check dataset
         dataset_id_str = str(dataset_id)
-        dataset = DatasetService.get_dataset(dataset_id_str, db.session)
+        dataset = DatasetService.get_dataset(dataset_id_str, db.session())
         if not dataset:
             raise NotFound("Dataset not found.")
         # check document
         document_id_str = str(document_id)
-        document = DocumentService.get_document(dataset_id_str, document_id_str, session=db.session)
+        document = DocumentService.get_document(dataset_id_str, document_id_str, session=db.session())
         if not document:
             raise NotFound("Document not found.")
 
@@ -658,18 +665,18 @@ class ChildChunkAddApi(Resource):
     ):
         # check dataset
         dataset_id_str = str(dataset_id)
-        dataset = DatasetService.get_dataset(dataset_id_str, db.session)
+        dataset = DatasetService.get_dataset(dataset_id_str, db.session())
         if not dataset:
             raise NotFound("Dataset not found.")
         # check document
         document_id_str = str(document_id)
-        document = DocumentService.get_document(dataset_id_str, document_id_str, session=db.session)
+        document = DocumentService.get_document(dataset_id_str, document_id_str, session=db.session())
         if not document:
             raise NotFound("Document not found.")
         if not current_user.is_dataset_editor:
             raise Forbidden()
         try:
-            DatasetService.check_dataset_permission(dataset, current_user, db.session)
+            DatasetService.check_dataset_permission(dataset, current_user, db.session())
         except services.errors.account.NoPermissionError as e:
             raise Forbidden(str(e))
         # check embedding model setting
@@ -693,7 +700,7 @@ class ChildChunkAddApi(Resource):
         # validate args
         try:
             payload = ChildChunkCreatePayload.model_validate(console_ns.payload or {})
-            child_chunk = SegmentService.create_child_chunk(payload.content, segment, document, dataset, db.session)
+            child_chunk = SegmentService.create_child_chunk(payload.content, segment, document, dataset, db.session())
         except ChildChunkIndexingServiceError as e:
             raise ChildChunkIndexingError(str(e))
         return dump_response(ChildChunkDetailResponse, {"data": child_chunk}), 200
@@ -709,14 +716,14 @@ class ChildChunkAddApi(Resource):
     def get(self, current_tenant_id: str, dataset_id: UUID, document_id: UUID, segment_id: UUID):
         # check dataset
         dataset_id_str = str(dataset_id)
-        dataset = DatasetService.get_dataset(dataset_id_str, db.session)
+        dataset = DatasetService.get_dataset(dataset_id_str, db.session())
         if not dataset:
             raise NotFound("Dataset not found.")
         # check user's model setting
         DatasetService.check_dataset_model_setting(dataset)
         # check document
         document_id_str = str(document_id)
-        document = DocumentService.get_document(dataset_id_str, document_id_str, session=db.session)
+        document = DocumentService.get_document(dataset_id_str, document_id_str, session=db.session())
         if not document:
             raise NotFound("Document not found.")
         segment_id_str = str(segment_id)
@@ -759,21 +766,21 @@ class ChildChunkAddApi(Resource):
     ):
         # check dataset
         dataset_id_str = str(dataset_id)
-        dataset = DatasetService.get_dataset(dataset_id_str, db.session)
+        dataset = DatasetService.get_dataset(dataset_id_str, db.session())
         if not dataset:
             raise NotFound("Dataset not found.")
         # check user's model setting
         DatasetService.check_dataset_model_setting(dataset)
         # check document
         document_id_str = str(document_id)
-        document = DocumentService.get_document(dataset_id_str, document_id_str, session=db.session)
+        document = DocumentService.get_document(dataset_id_str, document_id_str, session=db.session())
         if not document:
             raise NotFound("Document not found.")
         # The role of the current user in the ta table must be admin, owner, dataset_operator, or editor
         if not current_user.is_dataset_editor:
             raise Forbidden()
         try:
-            DatasetService.check_dataset_permission(dataset, current_user, db.session)
+            DatasetService.check_dataset_permission(dataset, current_user, db.session())
         except services.errors.account.NoPermissionError as e:
             raise Forbidden(str(e))
         segment_id_str = str(segment_id)
@@ -781,7 +788,7 @@ class ChildChunkAddApi(Resource):
         # validate args
         payload = ChildChunkBatchUpdatePayload.model_validate(console_ns.payload or {})
         try:
-            child_chunks = SegmentService.update_child_chunks(payload.chunks, segment, document, dataset, db.session)
+            child_chunks = SegmentService.update_child_chunks(payload.chunks, segment, document, dataset, db.session())
         except ChildChunkIndexingServiceError as e:
             raise ChildChunkIndexingError(str(e))
         return dump_response(ChildChunkBatchUpdateResponse, {"data": child_chunks}), 200
@@ -811,31 +818,31 @@ class ChildChunkUpdateApi(Resource):
     ):
         # check dataset
         dataset_id_str = str(dataset_id)
-        dataset = DatasetService.get_dataset(dataset_id_str, db.session)
+        dataset = DatasetService.get_dataset(dataset_id_str, db.session())
         if not dataset:
             raise NotFound("Dataset not found.")
         # check user's model setting
         DatasetService.check_dataset_model_setting(dataset)
         # check document
         document_id_str = str(document_id)
-        document = DocumentService.get_document(dataset_id_str, document_id_str, session=db.session)
+        document = DocumentService.get_document(dataset_id_str, document_id_str, session=db.session())
         if not document:
             raise NotFound("Document not found.")
         # The role of the current user in the ta table must be admin, owner, dataset_operator, or editor
         if not current_user.is_dataset_editor:
             raise Forbidden()
         try:
-            DatasetService.check_dataset_permission(dataset, current_user, db.session)
+            DatasetService.check_dataset_permission(dataset, current_user, db.session())
         except services.errors.account.NoPermissionError as e:
             raise Forbidden(str(e))
         segment_id_str = str(segment_id)
         segment_ref, _ = _get_segment_for_document(dataset, document, segment_id_str)
         child_chunk_id_str = str(child_chunk_id)
-        child_chunk = SegmentService.get_child_chunk_by_segment_ref(child_chunk_id_str, segment_ref)
+        child_chunk = SegmentService.get_child_chunk_by_segment_ref(child_chunk_id_str, segment_ref, db.session())
         if not child_chunk:
             raise NotFound("Child chunk not found.")
         try:
-            SegmentService.delete_child_chunk(child_chunk, dataset, db.session)
+            SegmentService.delete_child_chunk(child_chunk, dataset, db.session())
         except ChildChunkDeleteIndexServiceError as e:
             raise ChildChunkDeleteIndexError(str(e))
         return "", 204
@@ -862,34 +869,34 @@ class ChildChunkUpdateApi(Resource):
     ):
         # check dataset
         dataset_id_str = str(dataset_id)
-        dataset = DatasetService.get_dataset(dataset_id_str, db.session)
+        dataset = DatasetService.get_dataset(dataset_id_str, db.session())
         if not dataset:
             raise NotFound("Dataset not found.")
         # check user's model setting
         DatasetService.check_dataset_model_setting(dataset)
         # check document
         document_id_str = str(document_id)
-        document = DocumentService.get_document(dataset_id_str, document_id_str, session=db.session)
+        document = DocumentService.get_document(dataset_id_str, document_id_str, session=db.session())
         if not document:
             raise NotFound("Document not found.")
         # The role of the current user in the ta table must be admin, owner, dataset_operator, or editor
         if not current_user.is_dataset_editor:
             raise Forbidden()
         try:
-            DatasetService.check_dataset_permission(dataset, current_user, db.session)
+            DatasetService.check_dataset_permission(dataset, current_user, db.session())
         except services.errors.account.NoPermissionError as e:
             raise Forbidden(str(e))
         segment_id_str = str(segment_id)
         segment_ref, segment = _get_segment_for_document(dataset, document, segment_id_str)
         child_chunk_id_str = str(child_chunk_id)
-        child_chunk = SegmentService.get_child_chunk_by_segment_ref(child_chunk_id_str, segment_ref)
+        child_chunk = SegmentService.get_child_chunk_by_segment_ref(child_chunk_id_str, segment_ref, db.session())
         if not child_chunk:
             raise NotFound("Child chunk not found.")
         # validate args
         try:
             payload = ChildChunkUpdatePayload.model_validate(console_ns.payload or {})
             child_chunk = SegmentService.update_child_chunk(
-                payload.content, child_chunk, segment, document, dataset, db.session
+                payload.content, child_chunk, segment, document, dataset, db.session()
             )
         except ChildChunkIndexingServiceError as e:
             raise ChildChunkIndexingError(str(e))

api/controllers/console/datasets/external.py

@@ -1,20 +1,16 @@
+from datetime import datetime
 from typing import Any
 from uuid import UUID
 
 from flask import request
-from flask_restx import Resource, fields, marshal
-from pydantic import BaseModel, Field, RootModel
+from flask_restx import Resource
+from pydantic import AliasChoices, BaseModel, Field, field_validator
 from sqlalchemy.orm import Session
 from werkzeug.exceptions import Forbidden, InternalServerError, NotFound
 
 import services
 from controllers.common.fields import UsageCountResponse
-from controllers.common.schema import (
-    get_or_create_model,
-    query_params_from_model,
-    register_response_schema_models,
-    register_schema_models,
-)
+from controllers.common.schema import query_params_from_model, register_response_schema_models, register_schema_models
 from controllers.console import console_ns
 from controllers.console.app.wraps import with_session
 from controllers.console.datasets.error import DatasetNameDuplicateError
@@ -28,21 +24,9 @@ from controllers.console.wraps import (
     with_current_tenant_id,
     with_current_user,
 )
-from extensions.ext_database import db
 from fields.base import ResponseModel
-from fields.dataset_fields import (
-    dataset_detail_fields,
-    dataset_retrieval_model_fields,
-    doc_metadata_fields,
-    external_knowledge_info_fields,
-    external_retrieval_model_fields,
-    icon_info_fields,
-    keyword_setting_fields,
-    reranking_model_fields,
-    tag_fields,
-    vector_setting_fields,
-    weighted_score_fields,
-)
+from fields.dataset_fields import DatasetDetailResponse
+from libs.helper import dump_response
 from libs.login import login_required
 from models import Account
 from services.dataset_service import DatasetService
@@ -51,50 +35,10 @@ from services.external_knowledge_service import ExternalDatasetService
 from services.hit_testing_service import HitTestingService
 from services.knowledge_service import BedrockRetrievalSetting, ExternalDatasetTestService
 
-register_response_schema_models(console_ns, UsageCountResponse)
-
-
-def _build_dataset_detail_model():
-    keyword_setting_model = get_or_create_model("DatasetKeywordSetting", keyword_setting_fields)
-    vector_setting_model = get_or_create_model("DatasetVectorSetting", vector_setting_fields)
-
-    weighted_score_fields_copy = weighted_score_fields.copy()
-    weighted_score_fields_copy["keyword_setting"] = fields.Nested(keyword_setting_model)
-    weighted_score_fields_copy["vector_setting"] = fields.Nested(vector_setting_model)
-    weighted_score_model = get_or_create_model("DatasetWeightedScore", weighted_score_fields_copy)
-
-    reranking_model = get_or_create_model("DatasetRerankingModel", reranking_model_fields)
-
-    dataset_retrieval_model_fields_copy = dataset_retrieval_model_fields.copy()
-    dataset_retrieval_model_fields_copy["reranking_model"] = fields.Nested(reranking_model)
-    dataset_retrieval_model_fields_copy["weights"] = fields.Nested(weighted_score_model, allow_null=True)
-    dataset_retrieval_model = get_or_create_model("DatasetRetrievalModel", dataset_retrieval_model_fields_copy)
-
-    tag_model = get_or_create_model("Tag", tag_fields)
-    doc_metadata_model = get_or_create_model("DatasetDocMetadata", doc_metadata_fields)
-    external_knowledge_info_model = get_or_create_model("ExternalKnowledgeInfo", external_knowledge_info_fields)
-    external_retrieval_model = get_or_create_model("ExternalRetrievalModel", external_retrieval_model_fields)
-    icon_info_model = get_or_create_model("DatasetIconInfo", icon_info_fields)
-
-    dataset_detail_fields_copy = dataset_detail_fields.copy()
-    dataset_detail_fields_copy["retrieval_model_dict"] = fields.Nested(dataset_retrieval_model)
-    dataset_detail_fields_copy["tags"] = fields.List(fields.Nested(tag_model))
-    dataset_detail_fields_copy["external_knowledge_info"] = fields.Nested(external_knowledge_info_model)
-    dataset_detail_fields_copy["external_retrieval_model"] = fields.Nested(external_retrieval_model, allow_null=True)
-    dataset_detail_fields_copy["doc_metadata"] = fields.List(fields.Nested(doc_metadata_model))
-    dataset_detail_fields_copy["icon_info"] = fields.Nested(icon_info_model)
-    return get_or_create_model("DatasetDetail", dataset_detail_fields_copy)
-
-
-try:
-    dataset_detail_model = console_ns.models["DatasetDetail"]
-except KeyError:
-    dataset_detail_model = _build_dataset_detail_model()
-
 
 class ExternalKnowledgeApiPayload(BaseModel):
     name: str = Field(..., min_length=1, max_length=40)
-    settings: dict[str, object]
+    settings: dict[str, Any]
 
 
 class ExternalDatasetCreatePayload(BaseModel):
@@ -102,15 +46,13 @@ class ExternalDatasetCreatePayload(BaseModel):
     external_knowledge_id: str
     name: str = Field(..., min_length=1, max_length=100)
     description: str | None = Field(None, max_length=400)
-    external_retrieval_model: dict[str, object] | None = Field(default=None)
+    external_retrieval_model: dict[str, Any] | None = None
 
 
 class ExternalHitTestingPayload(BaseModel):
     query: str
-    external_retrieval_model: dict[str, object] | None = Field(default=None)
-    metadata_filtering_conditions: dict[str, object] | None = Field(
-        default=None,
-    )
+    external_retrieval_model: dict[str, Any] | None = None
+    metadata_filtering_conditions: dict[str, Any] | None = None
 
 
 class BedrockRetrievalPayload(BaseModel):
@@ -125,7 +67,7 @@ class ExternalApiTemplateListQuery(BaseModel):
     keyword: str | None = Field(default=None, description="Search keyword")
 
 
-class ExternalKnowledgeDatasetBindingResponse(ResponseModel):
+class ExternalKnowledgeApiBindingResponse(ResponseModel):
     id: str
     name: str
 
@@ -135,22 +77,52 @@ class ExternalKnowledgeApiResponse(ResponseModel):
     tenant_id: str
     name: str
     description: str
-    settings: dict[str, Any] | None = Field(default=None)
-    dataset_bindings: list[ExternalKnowledgeDatasetBindingResponse] = Field(default_factory=list)
+    settings: dict[str, Any] | None = Field(validation_alias=AliasChoices("settings_dict", "settings"))
+    dataset_bindings: list[ExternalKnowledgeApiBindingResponse]
     created_by: str
     created_at: str
 
+    @field_validator("created_at", mode="before")
+    @classmethod
+    def _normalize_created_at(cls, value: datetime | str) -> str:
+        if isinstance(value, datetime):
+            return value.isoformat()
+        return value
+
 
 class ExternalKnowledgeApiListResponse(ResponseModel):
     data: list[ExternalKnowledgeApiResponse]
     has_more: bool
     limit: int
-    total: int
+    total: int | None
     page: int
 
 
-class ExternalRetrievalTestResponse(RootModel[dict[str, Any] | list[dict[str, Any]]]):
-    root: dict[str, Any] | list[dict[str, Any]]
+class ExternalHitTestingQueryResponse(ResponseModel):
+    content: str
+
+
+class ExternalHitTestingRecordResponse(ResponseModel):
+    content: str | None = None
+    title: str | None = None
+    score: float | None = None
+    metadata: dict[str, Any] | None = None
+
+
+class ExternalHitTestingResponse(ResponseModel):
+    query: ExternalHitTestingQueryResponse
+    records: list[ExternalHitTestingRecordResponse]
+
+
+class BedrockRetrievalRecordResponse(ResponseModel):
+    metadata: dict[str, Any] | None = None
+    score: float
+    title: str | None = None
+    content: str | None = None
+
+
+class BedrockRetrievalResponse(ResponseModel):
+    records: list[BedrockRetrievalRecordResponse]
 
 
 register_schema_models(
@@ -163,9 +135,16 @@ register_schema_models(
 )
 register_response_schema_models(
     console_ns,
+    UsageCountResponse,
+    DatasetDetailResponse,
+    ExternalKnowledgeApiBindingResponse,
     ExternalKnowledgeApiResponse,
     ExternalKnowledgeApiListResponse,
-    ExternalRetrievalTestResponse,
+    ExternalHitTestingQueryResponse,
+    ExternalHitTestingRecordResponse,
+    ExternalHitTestingResponse,
+    BedrockRetrievalRecordResponse,
+    BedrockRetrievalResponse,
 )
 
 
@@ -189,24 +168,26 @@ class ExternalApiTemplateListApi(Resource):
         external_knowledge_apis, total = ExternalDatasetService.get_external_knowledge_apis(
             query.page, query.limit, current_tenant_id, query.keyword
         )
-        response = {
-            "data": [item.to_dict() for item in external_knowledge_apis],
-            "has_more": len(external_knowledge_apis) == query.limit,
-            "limit": query.limit,
-            "total": total,
-            "page": query.page,
-        }
-        return response, 200
+        return ExternalKnowledgeApiListResponse(
+            data=[ExternalKnowledgeApiResponse.model_validate(item) for item in external_knowledge_apis],
+            has_more=len(external_knowledge_apis) == query.limit,
+            limit=query.limit,
+            total=total,
+            page=query.page,
+        ).model_dump(mode="json"), 200
 
-    @setup_required
-    @login_required
-    @account_initialization_required
+    @console_ns.doc("create_external_api_template")
+    @console_ns.doc(description="Create external knowledge API template")
     @console_ns.expect(console_ns.models[ExternalKnowledgeApiPayload.__name__])
     @console_ns.response(
         201,
         "External API template created successfully",
         console_ns.models[ExternalKnowledgeApiResponse.__name__],
     )
+    @console_ns.response(403, "Permission denied")
+    @setup_required
+    @login_required
+    @account_initialization_required
     @with_current_user
     @with_current_tenant_id
     @with_session
@@ -229,7 +210,7 @@ class ExternalApiTemplateListApi(Resource):
         except services.errors.dataset.DatasetNameDuplicateError:
             raise DatasetNameDuplicateError()
 
-        return external_knowledge_api.to_dict(), 201
+        return dump_response(ExternalKnowledgeApiResponse, external_knowledge_api), 201
 
 
 @console_ns.route("/datasets/external-knowledge-api/<uuid:external_knowledge_api_id>")
@@ -256,17 +237,21 @@ class ExternalApiTemplateApi(Resource):
         if external_knowledge_api is None:
             raise NotFound("API template not found.")
 
-        return external_knowledge_api.to_dict(), 200
+        return dump_response(ExternalKnowledgeApiResponse, external_knowledge_api), 200
 
+    @console_ns.doc("update_external_api_template")
+    @console_ns.doc(description="Update external knowledge API template")
+    @console_ns.doc(params={"external_knowledge_api_id": "External knowledge API ID"})
+    @console_ns.expect(console_ns.models[ExternalKnowledgeApiPayload.__name__])
     @console_ns.response(
         200,
         "External API template updated successfully",
         console_ns.models[ExternalKnowledgeApiResponse.__name__],
     )
+    @console_ns.response(404, "Template not found")
     @setup_required
     @login_required
     @account_initialization_required
-    @console_ns.expect(console_ns.models[ExternalKnowledgeApiPayload.__name__])
     @with_current_user
     @with_current_tenant_id
     @with_session
@@ -284,7 +269,7 @@ class ExternalApiTemplateApi(Resource):
             session=session,
         )
 
-        return external_knowledge_api.to_dict(), 200
+        return dump_response(ExternalKnowledgeApiResponse, external_knowledge_api), 200
 
     @setup_required
     @login_required
@@ -299,7 +284,9 @@ class ExternalApiTemplateApi(Resource):
         if not (current_user.has_edit_permission or current_user.is_dataset_operator):
             raise Forbidden()
 
-        ExternalDatasetService.delete_external_knowledge_api(session, current_tenant_id, external_knowledge_api_id_str)
+        ExternalDatasetService.delete_external_knowledge_api(
+            current_tenant_id, external_knowledge_api_id_str, session=session
+        )
         return "", 204
 
 
@@ -318,11 +305,9 @@ class ExternalApiUseCheckApi(Resource):
         external_knowledge_api_id_str = str(external_knowledge_api_id)
 
         external_knowledge_api_is_using, count = ExternalDatasetService.external_knowledge_api_use_check(
-            session,
-            external_knowledge_api_id_str,
-            current_tenant_id,
+            external_knowledge_api_id_str, current_tenant_id, session=session
         )
-        return {"is_using": external_knowledge_api_is_using, "count": count}, 200
+        return UsageCountResponse(is_using=external_knowledge_api_is_using, count=count).model_dump(mode="json"), 200
 
 
 @console_ns.route("/datasets/external")
@@ -330,7 +315,9 @@ class ExternalDatasetCreateApi(Resource):
     @console_ns.doc("create_external_dataset")
     @console_ns.doc(description="Create external knowledge dataset")
     @console_ns.expect(console_ns.models[ExternalDatasetCreatePayload.__name__])
-    @console_ns.response(201, "External dataset created successfully", dataset_detail_model)
+    @console_ns.response(
+        201, "External dataset created successfully", console_ns.models[DatasetDetailResponse.__name__]
+    )
     @console_ns.response(400, "Invalid parameters")
     @console_ns.response(403, "Permission denied")
     @setup_required
@@ -360,16 +347,16 @@ class ExternalDatasetCreateApi(Resource):
         except services.errors.dataset.DatasetNameDuplicateError:
             raise DatasetNameDuplicateError()
 
-        item = marshal(dataset, dataset_detail_fields)
-        dataset_id_str = item["id"]
+        dataset_id_str = str(dataset.id)
         permission_keys_map = enterprise_rbac_service.RBACService.DatasetPermissions.batch_get(
             str(current_tenant_id),
             current_user.id,
             [dataset_id_str],
+            session=session,
         )
-        item["permission_keys"] = permission_keys_map.get(dataset_id_str, [])
-
-        return item, 201
+        data = DatasetDetailResponse.model_validate(dataset).model_dump(mode="json")
+        data["permission_keys"] = permission_keys_map.get(dataset_id_str, [])
+        return data, 201
 
 
 @console_ns.route("/datasets/<uuid:dataset_id>/external-hit-testing")
@@ -381,7 +368,7 @@ class ExternalKnowledgeHitTestingApi(Resource):
     @console_ns.response(
         200,
         "External hit testing completed successfully",
-        console_ns.models[ExternalRetrievalTestResponse.__name__],
+        console_ns.models[ExternalHitTestingResponse.__name__],
     )
     @console_ns.response(404, "Dataset not found")
     @console_ns.response(400, "Invalid parameters")
@@ -393,12 +380,12 @@ class ExternalKnowledgeHitTestingApi(Resource):
     @with_session
     def post(self, session: Session, current_user: Account, dataset_id: UUID):
         dataset_id_str = str(dataset_id)
-        dataset = DatasetService.get_dataset(dataset_id_str, db.session)
+        dataset = DatasetService.get_dataset(dataset_id_str, session)
         if dataset is None:
             raise NotFound("Dataset not found.")
 
         try:
-            DatasetService.check_dataset_permission(dataset, current_user, db.session)
+            DatasetService.check_dataset_permission(dataset, current_user, session)
         except services.errors.account.NoPermissionError as e:
             raise Forbidden(str(e))
 
@@ -415,7 +402,7 @@ class ExternalKnowledgeHitTestingApi(Resource):
                 metadata_filtering_conditions=payload.metadata_filtering_conditions,
             )
 
-            return response
+            return dump_response(ExternalHitTestingResponse, response)
         except Exception as e:
             raise InternalServerError(str(e))
 
@@ -426,11 +413,7 @@ class BedrockRetrievalApi(Resource):
     @console_ns.doc("bedrock_retrieval_test")
     @console_ns.doc(description="Bedrock retrieval test (internal use only)")
     @console_ns.expect(console_ns.models[BedrockRetrievalPayload.__name__])
-    @console_ns.response(
-        200,
-        "Bedrock retrieval test completed",
-        console_ns.models[ExternalRetrievalTestResponse.__name__],
-    )
+    @console_ns.response(200, "Bedrock retrieval test completed", console_ns.models[BedrockRetrievalResponse.__name__])
     def post(self):
         payload = BedrockRetrievalPayload.model_validate(console_ns.payload or {})
 
@@ -438,4 +421,4 @@ class BedrockRetrievalApi(Resource):
         result = ExternalDatasetTestService.knowledge_retrieval(
             payload.retrieval_setting, payload.query, payload.knowledge_id
         )
-        return result, 200
+        return dump_response(BedrockRetrievalResponse, result), 200

api/controllers/console/datasets/hit_testing_base.py

@@ -86,12 +86,12 @@ class DatasetsHitTestingBase:
         dataset_id: str, current_user: Account | None = None, current_tenant_id: str | None = None
     ) -> Dataset:
         current_user, _ = resolve_account_fallback(current_user, current_tenant_id)
-        dataset = DatasetService.get_dataset(dataset_id, db.session)
+        dataset = DatasetService.get_dataset(dataset_id, db.session())
         if dataset is None:
             raise NotFound("Dataset not found.")
 
         try:
-            DatasetService.check_dataset_permission(dataset, current_user, db.session)
+            DatasetService.check_dataset_permission(dataset, current_user, db.session())
         except services.errors.account.NoPermissionError as e:
             raise Forbidden(str(e))
 

api/controllers/console/datasets/metadata.py

@@ -61,13 +61,13 @@ class DatasetMetadataCreateApi(Resource):
         metadata_args = MetadataArgs.model_validate(console_ns.payload or {})
 
         dataset_id_str = str(dataset_id)
-        dataset = DatasetService.get_dataset(dataset_id_str, db.session)
+        dataset = DatasetService.get_dataset(dataset_id_str, db.session())
         if dataset is None:
             raise NotFound("Dataset not found.")
-        DatasetService.check_dataset_permission(dataset, current_user, db.session)
+        DatasetService.check_dataset_permission(dataset, current_user, db.session())
 
         metadata = MetadataService.create_metadata(
-            db.session(), dataset_id_str, metadata_args, current_user, current_tenant_id
+            dataset_id_str, metadata_args, current_user, current_tenant_id, session=db.session()
         )
         return dump_response(DatasetMetadataResponse, metadata), 201
 
@@ -81,10 +81,10 @@ class DatasetMetadataCreateApi(Resource):
     @rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_CREATE_AND_MANAGEMENT)
     def get(self, dataset_id: UUID):
         dataset_id_str = str(dataset_id)
-        dataset = DatasetService.get_dataset(dataset_id_str, db.session)
+        dataset = DatasetService.get_dataset(dataset_id_str, db.session())
         if dataset is None:
             raise NotFound("Dataset not found.")
-        metadata = MetadataService.get_dataset_metadatas(db.session(), dataset)
+        metadata = MetadataService.get_dataset_metadatas(dataset, session=db.session())
         return dump_response(DatasetMetadataListResponse, metadata), 200
 
 
@@ -105,13 +105,13 @@ class DatasetMetadataApi(Resource):
 
         dataset_id_str = str(dataset_id)
         metadata_id_str = str(metadata_id)
-        dataset = DatasetService.get_dataset(dataset_id_str, db.session)
+        dataset = DatasetService.get_dataset(dataset_id_str, db.session())
         if dataset is None:
             raise NotFound("Dataset not found.")
-        DatasetService.check_dataset_permission(dataset, current_user, db.session)
+        DatasetService.check_dataset_permission(dataset, current_user, db.session())
 
         metadata = MetadataService.update_metadata_name(
-            db.session(), dataset_id_str, metadata_id_str, name, current_user, current_tenant_id
+            dataset_id_str, metadata_id_str, name, current_user, current_tenant_id, session=db.session()
         )
         return dump_response(DatasetMetadataResponse, metadata), 200
 
@@ -125,12 +125,12 @@ class DatasetMetadataApi(Resource):
     def delete(self, current_user: Account, dataset_id: UUID, metadata_id: UUID):
         dataset_id_str = str(dataset_id)
         metadata_id_str = str(metadata_id)
-        dataset = DatasetService.get_dataset(dataset_id_str, db.session)
+        dataset = DatasetService.get_dataset(dataset_id_str, db.session())
         if dataset is None:
             raise NotFound("Dataset not found.")
-        DatasetService.check_dataset_permission(dataset, current_user, db.session)
+        DatasetService.check_dataset_permission(dataset, current_user, db.session())
 
-        MetadataService.delete_metadata(db.session(), dataset_id_str, metadata_id_str)
+        MetadataService.delete_metadata(dataset_id_str, metadata_id_str, session=db.session())
         # Frontend callers only await success and invalidate metadata caches; no response body is consumed.
         return "", 204
 
@@ -162,16 +162,16 @@ class DatasetMetadataBuiltInFieldActionApi(Resource):
     @rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_EDIT)
     def post(self, current_user: Account, dataset_id: UUID, action: Literal["enable", "disable"]):
         dataset_id_str = str(dataset_id)
-        dataset = DatasetService.get_dataset(dataset_id_str, db.session)
+        dataset = DatasetService.get_dataset(dataset_id_str, db.session())
         if dataset is None:
             raise NotFound("Dataset not found.")
-        DatasetService.check_dataset_permission(dataset, current_user, db.session)
+        DatasetService.check_dataset_permission(dataset, current_user, db.session())
 
         match action:
             case "enable":
-                MetadataService.enable_built_in_field(db.session(), dataset)
+                MetadataService.enable_built_in_field(dataset, session=db.session())
             case "disable":
-                MetadataService.disable_built_in_field(db.session(), dataset)
+                MetadataService.disable_built_in_field(dataset, session=db.session())
         # Frontend callers only await success and invalidate metadata caches; no response body is consumed.
         return "", 204
 
@@ -191,14 +191,14 @@ class DocumentMetadataEditApi(Resource):
     @rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_EDIT)
     def post(self, current_user: Account, dataset_id: UUID):
         dataset_id_str = str(dataset_id)
-        dataset = DatasetService.get_dataset(dataset_id_str, db.session)
+        dataset = DatasetService.get_dataset(dataset_id_str, db.session())
         if dataset is None:
             raise NotFound("Dataset not found.")
-        DatasetService.check_dataset_permission(dataset, current_user, db.session)
+        DatasetService.check_dataset_permission(dataset, current_user, db.session())
 
         metadata_args = MetadataOperationData.model_validate(console_ns.payload or {})
 
-        MetadataService.update_documents_metadata(db.session(), dataset, metadata_args, current_user)
+        MetadataService.update_documents_metadata(dataset, metadata_args, current_user, session=db.session())
 
         # Frontend callers only await success and invalidate caches; no response body is consumed.
         return "", 204

api/controllers/console/datasets/rag_pipeline/datasource_auth.py

@@ -23,6 +23,7 @@ from core.entities.provider_entities import ProviderConfig
 from core.plugin.entities.plugin_daemon import PluginOAuthAuthorizationUrlResponse
 from core.plugin.impl.oauth import OAuthHandler
 from core.tools.entities.common_entities import I18nObject
+from extensions.ext_database import db
 from fields.base import ResponseModel
 from graphon.model_runtime.errors.validate import CredentialsValidateFailedError
 from libs.helper import dump_response
@@ -309,6 +310,7 @@ class DatasourceAuth(Resource):
             provider=datasource_provider_id.provider_name,
             plugin_id=datasource_provider_id.plugin_id,
             user=user,
+            session=db.session(),
         )
         return dump_response(DatasourceCredentialListResponse, {"result": datasources}), 200
 
@@ -335,6 +337,7 @@ class DatasourceAuthDeleteApi(Resource):
             auth_id=payload.credential_id,
             provider=provider_name,
             plugin_id=plugin_id,
+            session=db.session(),
         )
         return SimpleResultResponse(result="success").model_dump(mode="json"), 200
 
@@ -380,7 +383,9 @@ class DatasourceAuthListApi(Resource):
     @with_current_tenant_id
     def get(self, current_tenant_id: str):
         datasource_provider_service = DatasourceProviderService()
-        datasources = datasource_provider_service.get_all_datasource_credentials(tenant_id=current_tenant_id)
+        datasources = datasource_provider_service.get_all_datasource_credentials(
+            tenant_id=current_tenant_id, session=db.session()
+        )
         return dump_response(DatasourceProviderAuthListResponse, {"result": datasources}), 200
 
 
@@ -397,7 +402,9 @@ class DatasourceHardCodeAuthListApi(Resource):
     @with_current_tenant_id
     def get(self, current_tenant_id: str):
         datasource_provider_service = DatasourceProviderService()
-        datasources = datasource_provider_service.get_hard_code_datasource_credentials(tenant_id=current_tenant_id)
+        datasources = datasource_provider_service.get_hard_code_datasource_credentials(
+            tenant_id=current_tenant_id, session=db.session()
+        )
         return dump_response(DatasourceProviderAuthListResponse, {"result": datasources}), 200
 
 

api/controllers/console/datasets/rag_pipeline/datasource_content_preview.py

@@ -9,6 +9,7 @@ from controllers.common.schema import register_schema_models
 from controllers.console import console_ns
 from controllers.console.datasets.wraps import get_rag_pipeline
 from controllers.console.wraps import account_initialization_required, setup_required, with_current_user
+from extensions.ext_database import db
 from libs.login import login_required
 from models import Account
 from models.dataset import Pipeline
@@ -41,7 +42,7 @@ class DataSourceContentPreviewApi(Resource):
 
         inputs = args.inputs
         datasource_type = args.datasource_type
-        rag_pipeline_service = RagPipelineService()
+        rag_pipeline_service = RagPipelineService(db.session())
         preview_content = rag_pipeline_service.run_datasource_node_preview(
             pipeline=pipeline,
             node_id=node_id,

api/controllers/console/datasets/rag_pipeline/rag_pipeline.py

@@ -108,7 +108,10 @@ class PipelineTemplateListApi(Resource):
         query = PipelineTemplateListQuery.model_validate(request.args.to_dict(flat=True))
         # get pipeline templates
         pipeline_templates = RagPipelineService.get_pipeline_templates(
-            session, query.type, query.language, current_tenant_id
+            type=query.type,
+            language=query.language,
+            current_tenant_id=current_tenant_id,
+            session=session,
         )
         return dump_response(PipelineTemplateListResponse, pipeline_templates), 200
 
@@ -124,8 +127,11 @@ class PipelineTemplateDetailApi(Resource):
     @with_session
     def get(self, session: Session, template_id: str) -> JsonResponseWithStatus:
         query = PipelineTemplateDetailQuery.model_validate(request.args.to_dict(flat=True))
-        rag_pipeline_service = RagPipelineService()
-        pipeline_template = rag_pipeline_service.get_pipeline_template_detail(session, template_id, query.type)
+        pipeline_template = RagPipelineService.get_pipeline_template_detail(
+            template_id,
+            type=query.type,
+            session=session,
+        )
         if pipeline_template is None:
             raise NotFound("Pipeline template not found from upstream service.")
         return dump_response(PipelineTemplateDetailResponse, pipeline_template), 200
@@ -145,7 +151,7 @@ class CustomizedPipelineTemplateApi(Resource):
         payload = CustomizedPipelineTemplatePayload.model_validate(console_ns.payload or {})
         pipeline_template_info = PipelineTemplateInfoEntity.model_validate(payload.model_dump())
         RagPipelineService.update_customized_pipeline_template(
-            template_id, pipeline_template_info, current_user, current_tenant_id
+            template_id, pipeline_template_info, current_user, current_tenant_id, session=db.session()
         )
         return "", 204
 
@@ -156,7 +162,7 @@ class CustomizedPipelineTemplateApi(Resource):
     @enterprise_license_required
     @with_current_tenant_id
     def delete(self, current_tenant_id: str, template_id: str) -> tuple[str, int]:
-        RagPipelineService.delete_customized_pipeline_template(template_id, current_tenant_id)
+        RagPipelineService.delete_customized_pipeline_template(template_id, current_tenant_id, session=db.session())
         return "", 204
 
     @setup_required
@@ -188,8 +194,8 @@ class PublishCustomizedPipelineTemplateApi(Resource):
     @with_current_tenant_id
     def post(self, current_tenant_id: str, current_user: Account, pipeline_id: str) -> tuple[str, int]:
         payload = CustomizedPipelineTemplatePayload.model_validate(console_ns.payload or {})
-        rag_pipeline_service = RagPipelineService()
+        rag_pipeline_service = RagPipelineService(db.session())
         rag_pipeline_service.publish_customized_pipeline_template(
-            pipeline_id, payload.model_dump(), current_user, current_tenant_id
+            pipeline_id, payload.model_dump(), current_user, current_tenant_id, session=db.session()
         )
         return "", 204

api/controllers/console/datasets/rag_pipeline/rag_pipeline_datasets.py

@@ -65,7 +65,7 @@ class CreateRagPipelineDatasetApi(Resource):
             yaml_content=payload.yaml_content,
         )
         try:
-            rag_pipeline_dsl_service = RagPipelineDslService(db.session)
+            rag_pipeline_dsl_service = RagPipelineDslService(db.session())
             import_info = rag_pipeline_dsl_service.create_rag_pipeline_dataset(
                 tenant_id=current_tenant_id,
                 rag_pipeline_dataset_create_entity=rag_pipeline_dataset_create_entity,
@@ -75,7 +75,7 @@ class CreateRagPipelineDatasetApi(Resource):
                     current_tenant_id,
                     import_info["dataset_id"],
                     rag_pipeline_dataset_create_entity.partial_member_list,
-                    db.session,
+                    db.session(),
                 )
             db.session.commit()
         except services.errors.dataset.DatasetNameDuplicateError:
@@ -110,6 +110,6 @@ class CreateEmptyRagPipelineDatasetApi(Resource):
                 permission=DatasetPermissionEnum.ONLY_ME,
                 partial_member_list=None,
             ),
-            session=db.session,
+            session=db.session(),
         )
         return dump_response(DatasetDetailResponse, dataset), 201

api/controllers/console/datasets/rag_pipeline/rag_pipeline_draft_variable.py

@@ -98,7 +98,7 @@ class RagPipelineVariableCollectionApi(Resource):
         query = PaginationQuery.model_validate(request.args.to_dict())
 
         # fetch draft workflow by app_model
-        rag_pipeline_service = RagPipelineService()
+        rag_pipeline_service = RagPipelineService(db.session())
         workflow_exist = rag_pipeline_service.is_workflow_exist(pipeline=pipeline)
         if not workflow_exist:
             raise DraftWorkflowNotExist()
@@ -290,7 +290,7 @@ class RagPipelineVariableResetApi(Resource):
             session=db.session(),
         )
 
-        rag_pipeline_service = RagPipelineService()
+        rag_pipeline_service = RagPipelineService(db.session())
         draft_workflow = rag_pipeline_service.get_draft_workflow(pipeline=pipeline)
         if draft_workflow is None:
             raise NotFoundError(
@@ -347,7 +347,7 @@ class RagPipelineEnvironmentVariableCollectionApi(Resource):
         Get draft workflow
         """
         # fetch draft workflow by app_model
-        rag_pipeline_service = RagPipelineService()
+        rag_pipeline_service = RagPipelineService(db.session())
         workflow = rag_pipeline_service.get_draft_workflow(pipeline=pipeline)
         if workflow is None:
             raise DraftWorkflowNotExist()

api/controllers/console/datasets/rag_pipeline/rag_pipeline_workflow.py

@@ -197,7 +197,7 @@ class DraftRagPipelineApi(Resource):
         Get draft rag pipeline's workflow
         """
         # fetch draft workflow by app_model
-        rag_pipeline_service = RagPipelineService()
+        rag_pipeline_service = RagPipelineService(db.session())
         workflow = rag_pipeline_service.get_draft_workflow(pipeline=pipeline)
 
         if not workflow:
@@ -231,7 +231,7 @@ class DraftRagPipelineApi(Resource):
                 return {"message": "Invalid JSON data"}, 400
         else:
             abort(415)
-        rag_pipeline_service = RagPipelineService()
+        rag_pipeline_service = RagPipelineService(db.session())
 
         try:
             environment_variables_list = Workflow.normalize_environment_variable_mappings(
@@ -283,7 +283,7 @@ class RagPipelineDraftRunIterationNodeApi(Resource):
 
         try:
             response = PipelineGenerateService.generate_single_iteration(
-                pipeline=pipeline, user=current_user, node_id=node_id, args=args, streaming=True
+                pipeline=pipeline, user=current_user, node_id=node_id, args=args, session=db.session(), streaming=True
             )
 
             return helper.compact_generate_response(response)
@@ -318,7 +318,7 @@ class RagPipelineDraftRunLoopNodeApi(Resource):
 
         try:
             response = PipelineGenerateService.generate_single_loop(
-                pipeline=pipeline, user=current_user, node_id=node_id, args=args, streaming=True
+                pipeline=pipeline, user=current_user, node_id=node_id, args=args, session=db.session(), streaming=True
             )
 
             return helper.compact_generate_response(response)
@@ -343,8 +343,8 @@ class DraftRagPipelineRunApi(Resource):
     @edit_permission_required
     @rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_EDIT)
     @with_current_user
-    @get_rag_pipeline
     @with_session
+    @get_rag_pipeline
     def post(self, session: Session, current_user: Account, pipeline: Pipeline):
         """
         Run draft workflow
@@ -377,8 +377,8 @@ class PublishedRagPipelineRunApi(Resource):
     @edit_permission_required
     @rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_EDIT)
     @with_current_user
-    @get_rag_pipeline
     @with_session
+    @get_rag_pipeline
     def post(self, session: Session, current_user: Account, pipeline: Pipeline):
         """
         Run published workflow
@@ -419,7 +419,7 @@ class RagPipelinePublishedDatasourceNodeRunApi(Resource):
         """
         payload = DatasourceNodeRunPayload.model_validate(console_ns.payload or {})
 
-        rag_pipeline_service = RagPipelineService()
+        rag_pipeline_service = RagPipelineService(db.session())
         return helper.compact_generate_response(
             PipelineGenerator.convert_to_event_stream(
                 rag_pipeline_service.run_datasource_workflow_node(
@@ -452,7 +452,7 @@ class RagPipelineDraftDatasourceNodeRunApi(Resource):
         """
         payload = DatasourceNodeRunPayload.model_validate(console_ns.payload or {})
 
-        rag_pipeline_service = RagPipelineService()
+        rag_pipeline_service = RagPipelineService(db.session())
         return helper.compact_generate_response(
             PipelineGenerator.convert_to_event_stream(
                 rag_pipeline_service.run_datasource_workflow_node(
@@ -490,7 +490,7 @@ class RagPipelineDraftNodeRunApi(Resource):
         payload = NodeRunRequiredPayload.model_validate(console_ns.payload or {})
         inputs = payload.inputs
 
-        rag_pipeline_service = RagPipelineService()
+        rag_pipeline_service = RagPipelineService(db.session())
         workflow_node_execution = rag_pipeline_service.run_draft_workflow_node(
             pipeline=pipeline, node_id=node_id, user_inputs=inputs, account=current_user
         )
@@ -543,7 +543,7 @@ class PublishedRagPipelineApi(Resource):
         if not pipeline.is_published:
             return None
         # fetch published workflow by pipeline
-        rag_pipeline_service = RagPipelineService()
+        rag_pipeline_service = RagPipelineService(db.session())
         workflow = rag_pipeline_service.get_published_workflow(pipeline=pipeline)
 
         # return workflow, if not found, return None
@@ -564,9 +564,9 @@ class PublishedRagPipelineApi(Resource):
         """
         Publish workflow
         """
-        rag_pipeline_service = RagPipelineService()
+        rag_pipeline_service = RagPipelineService(db.session())
         workflow = rag_pipeline_service.publish_workflow(
-            session=db.session,  # type: ignore[reportArgumentType,arg-type]
+            session=db.session(),
             pipeline=pipeline,
             account=current_user,
         )
@@ -599,7 +599,7 @@ class DefaultRagPipelineBlockConfigsApi(Resource):
         Get default block config
         """
         # Get default block configs
-        rag_pipeline_service = RagPipelineService()
+        rag_pipeline_service = RagPipelineService(db.session())
         return rag_pipeline_service.get_default_block_configs()
 
 
@@ -631,7 +631,7 @@ class DefaultRagPipelineBlockConfigApi(Resource):
                 raise ValueError("Invalid filters")
 
         # Get default block configs
-        rag_pipeline_service = RagPipelineService()
+        rag_pipeline_service = RagPipelineService(db.session())
         return rag_pipeline_service.get_default_block_config(node_type=block_type, filters=filters)
 
 
@@ -666,7 +666,7 @@ class PublishedAllRagPipelineApi(Resource):
             if user_id != current_user.id:
                 raise Forbidden()
 
-        rag_pipeline_service = RagPipelineService()
+        rag_pipeline_service = RagPipelineService(db.session())
         with sessionmaker(db.engine).begin() as session:
             workflows, has_more = rag_pipeline_service.get_all_published_workflow(
                 session=session,
@@ -698,7 +698,7 @@ class RagPipelineDraftWorkflowRestoreApi(Resource):
     @with_current_user
     @get_rag_pipeline
     def post(self, current_user: Account, pipeline: Pipeline, workflow_id: str):
-        rag_pipeline_service = RagPipelineService()
+        rag_pipeline_service = RagPipelineService(db.session())
 
         try:
             workflow = rag_pipeline_service.restore_published_workflow_to_draft(
@@ -743,7 +743,7 @@ class RagPipelineByIdApi(Resource):
         if not update_data:
             return {"message": "No valid fields to update"}, 400
 
-        rag_pipeline_service = RagPipelineService()
+        rag_pipeline_service = RagPipelineService(db.session())
         workflow_ref = WorkflowRefService.create_pipeline_workflow_ref(pipeline, workflow_id)
 
         # Create a session and manage the transaction
@@ -809,7 +809,7 @@ class PublishedRagPipelineSecondStepApi(Resource):
         """
         query = NodeIdQuery.model_validate(request.args.to_dict())
         node_id = query.node_id
-        rag_pipeline_service = RagPipelineService()
+        rag_pipeline_service = RagPipelineService(db.session())
         variables = rag_pipeline_service.get_second_step_parameters(pipeline=pipeline, node_id=node_id, is_draft=False)
         return {
             "variables": variables,
@@ -832,7 +832,7 @@ class PublishedRagPipelineFirstStepApi(Resource):
         """
         query = NodeIdQuery.model_validate(request.args.to_dict())
         node_id = query.node_id
-        rag_pipeline_service = RagPipelineService()
+        rag_pipeline_service = RagPipelineService(db.session())
         variables = rag_pipeline_service.get_first_step_parameters(pipeline=pipeline, node_id=node_id, is_draft=False)
         return {
             "variables": variables,
@@ -855,7 +855,7 @@ class DraftRagPipelineFirstStepApi(Resource):
         """
         query = NodeIdQuery.model_validate(request.args.to_dict())
         node_id = query.node_id
-        rag_pipeline_service = RagPipelineService()
+        rag_pipeline_service = RagPipelineService(db.session())
         variables = rag_pipeline_service.get_first_step_parameters(pipeline=pipeline, node_id=node_id, is_draft=True)
         return {
             "variables": variables,
@@ -879,7 +879,7 @@ class DraftRagPipelineSecondStepApi(Resource):
         query = NodeIdQuery.model_validate(request.args.to_dict())
         node_id = query.node_id
 
-        rag_pipeline_service = RagPipelineService()
+        rag_pipeline_service = RagPipelineService(db.session())
         variables = rag_pipeline_service.get_second_step_parameters(pipeline=pipeline, node_id=node_id, is_draft=True)
         return {
             "variables": variables,
@@ -913,7 +913,7 @@ class RagPipelineWorkflowRunListApi(Resource):
             "limit": query.limit,
         }
 
-        rag_pipeline_service = RagPipelineService()
+        rag_pipeline_service = RagPipelineService(db.session())
         result = rag_pipeline_service.get_rag_pipeline_paginate_workflow_runs(pipeline=pipeline, args=args)
 
         return WorkflowRunPaginationResponse.model_validate(result, from_attributes=True).model_dump(mode="json")
@@ -936,7 +936,7 @@ class RagPipelineWorkflowRunDetailApi(Resource):
         """
         run_id_str = str(run_id)
 
-        rag_pipeline_service = RagPipelineService()
+        rag_pipeline_service = RagPipelineService(db.session())
         workflow_run = rag_pipeline_service.get_rag_pipeline_workflow_run(pipeline=pipeline, run_id=run_id_str)
         if workflow_run is None:
             raise NotFound("Workflow run not found")
@@ -962,7 +962,7 @@ class RagPipelineWorkflowRunNodeExecutionListApi(Resource):
         """
         run_id_str = str(run_id)
 
-        rag_pipeline_service = RagPipelineService()
+        rag_pipeline_service = RagPipelineService(db.session())
         user = cast("Account | EndUser", current_user)
         node_executions = rag_pipeline_service.get_rag_pipeline_workflow_run_node_executions(
             pipeline=pipeline,
@@ -998,7 +998,7 @@ class RagPipelineWorkflowLastRunApi(Resource):
     @account_initialization_required
     @get_rag_pipeline
     def get(self, pipeline: Pipeline, node_id: str):
-        rag_pipeline_service = RagPipelineService()
+        rag_pipeline_service = RagPipelineService(db.session())
         workflow = rag_pipeline_service.get_draft_workflow(pipeline=pipeline)
         if not workflow:
             raise NotFound("Workflow not found")
@@ -1051,7 +1051,7 @@ class RagPipelineDatasourceVariableApi(Resource):
         """
         args = DatasourceVariablesPayload.model_validate(console_ns.payload or {}).model_dump()
 
-        rag_pipeline_service = RagPipelineService()
+        rag_pipeline_service = RagPipelineService(db.session())
         workflow_node_execution = rag_pipeline_service.set_datasource_variables(
             pipeline=pipeline,
             args=args,
@@ -1074,6 +1074,6 @@ class RagPipelineRecommendedPluginApi(Resource):
     def get(self, current_tenant_id: str, current_user: Account):
         query = RagPipelineRecommendedPluginQuery.model_validate(request.args.to_dict())
 
-        rag_pipeline_service = RagPipelineService()
+        rag_pipeline_service = RagPipelineService(db.session())
         recommended_plugins = rag_pipeline_service.get_recommended_plugins(query.type, current_user, current_tenant_id)
         return recommended_plugins

api/controllers/console/datasets/wraps.py

@@ -2,6 +2,7 @@ from collections.abc import Callable
 from functools import wraps
 
 from sqlalchemy import select
+from sqlalchemy.orm import Session
 
 from controllers.console.datasets.error import PipelineNotFoundError
 from extensions.ext_database import db
@@ -22,9 +23,10 @@ def get_rag_pipeline[**P, R](view_func: Callable[P, R]) -> Callable[P, R]:
 
         del kwargs["pipeline_id"]
 
-        pipeline = db.session.scalar(
-            select(Pipeline).where(Pipeline.id == pipeline_id, Pipeline.tenant_id == current_tenant_id).limit(1)
-        )
+        stmt = select(Pipeline).where(Pipeline.id == pipeline_id, Pipeline.tenant_id == current_tenant_id).limit(1)
+        # Migrated handlers pass the request Session as args[1]; legacy handlers still use db.session.
+        session = args[1] if len(args) > 1 and isinstance(args[1], Session) else db.session
+        pipeline = session.scalar(stmt)
 
         if not pipeline:
             raise PipelineNotFoundError()

api/controllers/console/explore/audio.py

@@ -113,7 +113,7 @@ class ChatTextApi(InstalledAppResource):
 
             response = AudioService.transcript_tts(
                 app_model=app_model,
-                session=db.session,
+                session=db.session(),
                 text=text,
                 voice=voice,
                 message_ref=message_ref,

api/controllers/console/explore/conversation.py

@@ -111,7 +111,7 @@ class ConversationApi(InstalledAppResource):
 
         conversation_id = str(c_id)
         try:
-            ConversationService.delete(app_model, conversation_id, current_user)
+            ConversationService.delete(app_model, conversation_id, current_user, session=db.session())
         except ConversationNotExistsError:
             raise NotFound("Conversation Not Exists.")
 
@@ -140,7 +140,7 @@ class ConversationRenameApi(InstalledAppResource):
 
         try:
             conversation = ConversationService.rename(
-                app_model, conversation_id, current_user, payload.name, payload.auto_generate
+                app_model, conversation_id, current_user, payload.name, payload.auto_generate, session=db.session()
             )
             return (
                 TypeAdapter(SimpleConversation)
@@ -169,7 +169,7 @@ class ConversationPinApi(InstalledAppResource):
         conversation_id = str(c_id)
 
         try:
-            WebConversationService.pin(app_model, conversation_id, current_user)
+            WebConversationService.pin(app_model, conversation_id, current_user, db.session())
         except ConversationNotExistsError:
             raise NotFound("Conversation Not Exists.")
 
@@ -192,6 +192,6 @@ class ConversationUnPinApi(InstalledAppResource):
             raise NotChatAppError()
 
         conversation_id = str(c_id)
-        WebConversationService.unpin(app_model, conversation_id, current_user)
+        WebConversationService.unpin(app_model, conversation_id, current_user, db.session())
 
         return ResultResponse(result="success").model_dump(mode="json")

api/controllers/console/explore/installed_app.py

@@ -181,7 +181,7 @@ class InstalledAppsListApi(Resource):
 
         if current_user.current_tenant is None:
             raise ValueError("current_user.current_tenant must not be None")
-        current_user.role = TenantService.get_user_role(current_user, current_user.current_tenant, session=db.session)
+        current_user.role = TenantService.get_user_role(current_user, current_user.current_tenant, session=db.session())
         installed_app_list: list[dict[str, Any]] = []
         for installed_app, app_model in installed_apps:
             installed_app_list.append(

api/controllers/console/explore/message.py

@@ -27,6 +27,7 @@ from controllers.console.explore.wraps import InstalledAppResource
 from controllers.console.wraps import with_current_user
 from core.app.entities.app_invoke_entities import InvokeFrom
 from core.errors.error import ModelCurrentlyNotSupportError, ProviderTokenNotInitError, QuotaExceededError
+from extensions.ext_database import db
 from fields.conversation_fields import ResultResponse
 from fields.message_fields import (
     ExploreMessageInfiniteScrollPagination,
@@ -91,6 +92,7 @@ class MessageListApi(InstalledAppResource):
                 args.conversation_id,
                 args.first_id or None,
                 args.limit,
+                session=db.session(),
             )
             adapter = TypeAdapter(ExploreMessageListItem)
             items = [adapter.validate_python(message, from_attributes=True) for message in pagination.data]
@@ -129,6 +131,7 @@ class MessageFeedbackApi(InstalledAppResource):
                 user=current_user,
                 rating=FeedbackRating(payload.rating) if payload.rating else None,
                 content=payload.content,
+                session=db.session(),
             )
         except MessageNotExistsError:
             raise NotFound("Message Not Exists.")
@@ -207,7 +210,11 @@ class MessageSuggestedQuestionApi(InstalledAppResource):
 
         try:
             questions = MessageService.get_suggested_questions_after_answer(
-                app_model=app_model, user=current_user, message_id=message_id_str, invoke_from=InvokeFrom.EXPLORE
+                app_model=app_model,
+                user=current_user,
+                message_id=message_id_str,
+                invoke_from=InvokeFrom.EXPLORE,
+                session=db.session(),
             )
         except MessageNotExistsError:
             raise NotFound("Message not found")

api/controllers/console/explore/parameter.py

@@ -8,6 +8,7 @@ from controllers.console import console_ns
 from controllers.console.app.error import AppUnavailableError
 from controllers.console.explore.wraps import InstalledAppResource
 from core.app.app_config.common.parameters_mapping import get_parameters_from_feature_dict
+from extensions.ext_database import db
 from models.model import AppMode, InstalledApp
 from services.app_service import AppService
 
@@ -64,4 +65,4 @@ class ExploreAppMetaApi(InstalledAppResource):
         app_model = installed_app.app
         if not app_model:
             raise ValueError("App not found")
-        return AppService().get_app_meta(app_model)
+        return AppService().get_app_meta(app_model, session=db.session())

api/controllers/console/explore/recommended_app.py

@@ -120,7 +120,7 @@ class RecommendedAppListApi(Resource):
         language_prefix = _resolve_language(args.language, current_user)
 
         return RecommendedAppListResponse.model_validate(
-            RecommendedAppService.get_recommended_apps_and_categories(db.session, language_prefix),
+            RecommendedAppService.get_recommended_apps_and_categories(language_prefix, session=db.session()),
             from_attributes=True,
         ).model_dump(mode="json")
 
@@ -137,7 +137,7 @@ class LearnDifyAppListApi(Resource):
         language_prefix = _resolve_language(args.language, current_user)
 
         return LearnDifyAppListResponse.model_validate(
-            RecommendedAppService.get_learn_dify_apps(db.session, language_prefix),
+            RecommendedAppService.get_learn_dify_apps(language_prefix, session=db.session()),
             from_attributes=True,
         ).model_dump(mode="json")
 
@@ -148,4 +148,4 @@ class RecommendedAppApi(Resource):
     @login_required
     @account_initialization_required
     def get(self, app_id: UUID):
-        return RecommendedAppService.get_recommend_app_detail(db.session, str(app_id))
+        return RecommendedAppService.get_recommend_app_detail(str(app_id), session=db.session())

api/controllers/console/explore/saved_message.py

@@ -38,11 +38,7 @@ class SavedMessageListApi(InstalledAppResource):
         args = SavedMessageListQuery.model_validate(request.args.to_dict())
 
         pagination = SavedMessageService.pagination_by_last_id(
-            db.session(),
-            app_model,
-            current_user,
-            str(args.last_id) if args.last_id else None,
-            args.limit,
+            app_model, current_user, str(args.last_id) if args.last_id else None, args.limit, session=db.session()
         )
         adapter = TypeAdapter(SavedMessageItem)
         items = [adapter.validate_python(message, from_attributes=True) for message in pagination.data]
@@ -65,7 +61,7 @@ class SavedMessageListApi(InstalledAppResource):
         payload = SavedMessageCreatePayload.model_validate(console_ns.payload or {})
 
         try:
-            SavedMessageService.save(db.session(), app_model, current_user, str(payload.message_id))
+            SavedMessageService.save(app_model, current_user, str(payload.message_id), session=db.session())
         except MessageNotExistsError:
             raise NotFound("Message Not Exists.")
 
@@ -88,6 +84,6 @@ class SavedMessageApi(InstalledAppResource):
         if app_model.mode != "completion":
             raise NotCompletionAppError()
 
-        SavedMessageService.delete(db.session(), app_model, current_user, message_id_str)
+        SavedMessageService.delete(app_model, current_user, message_id_str, session=db.session())
 
         return "", 204

api/controllers/console/explore/trial.py

@@ -431,7 +431,7 @@ class TrialAppWorkflowRunApi(TrialAppResource):
                 invoke_from=InvokeFrom.EXPLORE,
                 streaming=True,
             )
-            RecommendedAppService.add_trial_app_record(db.session, app_id, user_id)
+            RecommendedAppService.add_trial_app_record(app_id, user_id, session=session)
             # response-contract:ignore compact_generate_response
             return helper.compact_generate_response(response)
         except ProviderTokenNotInitError as ex:
@@ -511,7 +511,7 @@ class TrialChatApi(TrialAppResource):
                 invoke_from=InvokeFrom.EXPLORE,
                 streaming=True,
             )
-            RecommendedAppService.add_trial_app_record(db.session, app_id, user_id)
+            RecommendedAppService.add_trial_app_record(app_id, user_id, session=session)
             # response-contract:ignore compact_generate_response
             return helper.compact_generate_response(response)
         except services.errors.conversation.ConversationNotExistsError:
@@ -551,7 +551,11 @@ class TrialMessageSuggestedQuestionApi(TrialAppResource):
 
         try:
             questions = MessageService.get_suggested_questions_after_answer(
-                app_model=app_model, user=current_user, message_id=message_id, invoke_from=InvokeFrom.EXPLORE
+                app_model=app_model,
+                user=current_user,
+                message_id=message_id,
+                invoke_from=InvokeFrom.EXPLORE,
+                session=db.session(),
             )
         except MessageNotExistsError:
             raise NotFound("Message not found")
@@ -589,7 +593,7 @@ class TrialChatAudioApi(TrialAppResource):
             user_id = current_user.id
 
             response = AudioService.transcript_asr(app_model=app_model, file=file, end_user=None)
-            RecommendedAppService.add_trial_app_record(db.session, app_id, user_id)
+            RecommendedAppService.add_trial_app_record(app_id, user_id, session=db.session())
             return response
         except services.errors.app_model_config.AppModelConfigBrokenError:
             logger.exception("App model config broken.")
@@ -645,12 +649,12 @@ class TrialChatTextApi(TrialAppResource):
 
             response = AudioService.transcript_tts(
                 app_model=app_model,
-                session=db.session,
+                session=db.session(),
                 text=text,
                 voice=voice,
                 message_ref=message_ref,
             )
-            RecommendedAppService.add_trial_app_record(db.session, app_id, user_id)
+            RecommendedAppService.add_trial_app_record(app_id, user_id, session=db.session())
             return response
         except services.errors.app_model_config.AppModelConfigBrokenError:
             logger.exception("App model config broken.")
@@ -709,7 +713,7 @@ class TrialCompletionApi(TrialAppResource):
                 streaming=streaming,
             )
 
-            RecommendedAppService.add_trial_app_record(db.session, app_id, user_id)
+            RecommendedAppService.add_trial_app_record(app_id, user_id, session=session)
             # response-contract:ignore compact_generate_response
             return helper.compact_generate_response(response)
         except services.errors.conversation.ConversationNotExistsError:

api/controllers/console/extension.py

@@ -112,7 +112,7 @@ class APIBasedExtensionAPI(Resource):
     def get(self, current_tenant_id: str):
         return dump_response(
             APIBasedExtensionListResponse,
-            APIBasedExtensionService.get_all_by_tenant_id(db.session(), current_tenant_id),
+            APIBasedExtensionService.get_all_by_tenant_id(current_tenant_id, session=db.session()),
         )
 
     @console_ns.doc("create_api_based_extension")
@@ -133,7 +133,7 @@ class APIBasedExtensionAPI(Resource):
             api_key=payload.api_key,
         )
 
-        extension = APIBasedExtensionService.save(db.session(), extension_data)
+        extension = APIBasedExtensionService.save(extension_data, session=db.session())
         return APIBasedExtensionResponse(
             id=extension.id,
             name=extension.name,
@@ -158,7 +158,9 @@ class APIBasedExtensionDetailAPI(Resource):
 
         return dump_response(
             APIBasedExtensionResponse,
-            APIBasedExtensionService.get_with_tenant_id(db.session(), current_tenant_id, api_based_extension_id),
+            APIBasedExtensionService.get_with_tenant_id(
+                current_tenant_id, api_based_extension_id, session=db.session()
+            ),
         )
 
     @console_ns.doc("update_api_based_extension")
@@ -174,7 +176,7 @@ class APIBasedExtensionDetailAPI(Resource):
         api_based_extension_id = str(id)
 
         extension_data_from_db = APIBasedExtensionService.get_with_tenant_id(
-            db.session(), current_tenant_id, api_based_extension_id
+            current_tenant_id, api_based_extension_id, session=db.session()
         )
 
         payload = APIBasedExtensionPayload.model_validate(console_ns.payload or {})
@@ -187,7 +189,7 @@ class APIBasedExtensionDetailAPI(Resource):
             extension_data_from_db.api_key = payload.api_key
             api_key_for_response = payload.api_key
 
-        APIBasedExtensionService.save(db.session(), extension_data_from_db)
+        APIBasedExtensionService.save(extension_data_from_db, session=db.session())
         return APIBasedExtensionResponse(
             id=extension_data_from_db.id,
             name=extension_data_from_db.name,
@@ -208,9 +210,9 @@ class APIBasedExtensionDetailAPI(Resource):
         api_based_extension_id = str(id)
 
         extension_data_from_db = APIBasedExtensionService.get_with_tenant_id(
-            db.session(), current_tenant_id, api_based_extension_id
+            current_tenant_id, api_based_extension_id, session=db.session()
         )
 
-        APIBasedExtensionService.delete(db.session(), extension_data_from_db)
+        APIBasedExtensionService.delete(extension_data_from_db, session=db.session())
 
         return "", 204

api/controllers/console/files.py

@@ -27,6 +27,7 @@ from controllers.console.wraps import (
 )
 from extensions.ext_database import db
 from fields.file_fields import FileResponse, UploadConfig
+from libs.helper import dump_response
 from libs.login import login_required
 from models import Account
 from services.file_service import FileService
@@ -117,8 +118,7 @@ class FileApi(Resource):
         except services.errors.file.BlockedFileExtensionError as blocked_extension_error:
             raise BlockedFileExtensionError(blocked_extension_error.description)
 
-        response = FileResponse.model_validate(upload_file, from_attributes=True)
-        return response.model_dump(mode="json"), 201
+        return dump_response(FileResponse, upload_file), 201
 
 
 @console_ns.route("/files/<uuid:file_id>/preview")
@@ -131,7 +131,7 @@ class FilePreviewApi(Resource):
     def get(self, current_tenant_id: str, file_id: UUID):
         file_id_str = str(file_id)
         text = FileService(db.engine).get_file_preview(file_id_str, current_tenant_id)
-        return {"content": text}
+        return TextContentResponse(content=text).model_dump(mode="json")
 
 
 @console_ns.route("/files/support-type")
@@ -141,4 +141,4 @@ class FileSupportTypeApi(Resource):
     @account_initialization_required
     @console_ns.response(200, "Success", console_ns.models[AllowedExtensionsResponse.__name__])
     def get(self):
-        return {"allowed_extensions": list(DOCUMENT_EXTENSIONS)}
+        return AllowedExtensionsResponse(allowed_extensions=list(DOCUMENT_EXTENSIONS)).model_dump(mode="json")

api/controllers/console/init_validate.py

@@ -50,7 +50,7 @@ def get_init_status() -> InitStatusResponse:
 @only_edition_self_hosted
 def validate_init_password(payload: InitValidatePayload) -> InitValidateResponse:
     """Validate initialization password."""
-    tenant_count = TenantService.get_tenant_count(session=db.session)
+    tenant_count = TenantService.get_tenant_count(session=db.session())
     if tenant_count > 0:
         raise AlreadySetupError()
 

api/controllers/console/setup.py

@@ -13,7 +13,7 @@ from services.account_service import RegisterService, TenantService
 
 from .error import AlreadySetupError, NotInitValidateError
 from .init_validate import get_init_validate_status
-from .wraps import only_edition_self_hosted
+from .wraps import mark_setup_completed, only_edition_self_hosted
 
 
 class SetupRequestPayload(BaseModel):
@@ -79,7 +79,7 @@ def setup_system(payload: SetupRequestPayload) -> SetupResponse:
     if get_setup_status():
         raise AlreadySetupError()
 
-    tenant_count = TenantService.get_tenant_count(session=db.session)
+    tenant_count = TenantService.get_tenant_count(session=db.session())
     if tenant_count > 0:
         raise AlreadySetupError()
 
@@ -94,8 +94,9 @@ def setup_system(payload: SetupRequestPayload) -> SetupResponse:
         password=payload.password,
         ip_address=extract_remote_ip(request),
         language=payload.language,
-        session=db.session,
+        session=db.session(),
     )
+    mark_setup_completed()
 
     return SetupResponse(result="success")
 

api/controllers/console/snippets/payloads.py

@@ -148,6 +148,8 @@ class PublishWorkflowPayload(BaseModel):
     """Payload for publishing snippet workflow."""
 
     knowledge_base_setting: dict[str, Any] | None = Field(default=None)
+    marked_name: str | None = Field(default=None, max_length=20)
+    marked_comment: str | None = Field(default=None, max_length=100)
 
 
 class SnippetImportPayload(BaseModel):

api/controllers/console/socketio/workflow.py

@@ -44,7 +44,7 @@ def socket_connect(sid, environ, auth):
             return False
 
         with sio.app.app_context():
-            user = AccountService.load_logged_in_account(account_id=user_id, session=db.session)
+            user = AccountService.load_logged_in_account(account_id=user_id, session=db.session())
             if not user:
                 logging.warning("Socket connect rejected: user not found (user_id=%s, sid=%s)", user_id, sid)
                 return False
@@ -69,7 +69,8 @@ def handle_user_connect(sid, data):
     if not workflow_id:
         return {"msg": "workflow_id is required"}, 400
 
-    result = collaboration_service.authorize_and_join_workflow_room(workflow_id, sid)
+    with sio.app.app_context():
+        result = collaboration_service.authorize_and_join_workflow_room(workflow_id, sid, session=db.session())
     if not result:
         return {"msg": "unauthorized"}, 401
 

api/controllers/console/spec.py

@@ -1,8 +1,9 @@
 import logging
+from collections.abc import Mapping
 from typing import Any
 
 from flask_restx import Resource
-from pydantic import RootModel
+from pydantic import Field, RootModel
 
 from controllers.common.schema import register_response_schema_models
 from controllers.console.wraps import (
@@ -10,6 +11,7 @@ from controllers.console.wraps import (
     setup_required,
 )
 from core.schemas.schema_manager import SchemaManager
+from fields.base import ResponseModel
 from libs.login import login_required
 
 from . import console_ns
@@ -17,11 +19,17 @@ from . import console_ns
 logger = logging.getLogger(__name__)
 
 
-class SchemaDefinitionsResponse(RootModel[Any]):
-    root: Any
+class SchemaDefinitionItemResponse(ResponseModel):
+    name: str
+    label: str
+    schema_: Mapping[str, Any] = Field(alias="schema")
 
 
-register_response_schema_models(console_ns, SchemaDefinitionsResponse)
+class SchemaDefinitionsResponse(RootModel[list[SchemaDefinitionItemResponse]]):
+    pass
+
+
+register_response_schema_models(console_ns, SchemaDefinitionItemResponse, SchemaDefinitionsResponse)
 
 
 @console_ns.route("/spec/schema-definitions")

api/controllers/console/tag/tags.py

@@ -137,7 +137,7 @@ class TagListApi(Resource):
     def get(self, current_tenant_id: str):
         raw_args = request.args.to_dict()
         param = TagListQueryParam.model_validate(raw_args)
-        tags = TagService.get_tags(db.session(), param.type, current_tenant_id, param.keyword)
+        tags = TagService.get_tags(param.type, current_tenant_id, param.keyword, session=db.session())
 
         return dump_response(TagListResponse, tags), 200
 
@@ -154,7 +154,7 @@ class TagListApi(Resource):
 
         payload = TagBasePayload.model_validate(console_ns.payload or {})
         _enforce_snippet_tag_rbac_if_needed(payload.type)
-        tag = TagService.save_tags(SaveTagPayload(name=payload.name, type=payload.type), db.session)
+        tag = TagService.save_tags(SaveTagPayload(name=payload.name, type=payload.type), db.session())
 
         return dump_response(TagResponse, {"id": tag.id, "name": tag.name, "type": tag.type, "binding_count": 0}), 200
 
@@ -175,9 +175,9 @@ class TagUpdateDeleteApi(Resource):
 
         payload = TagUpdateRequestPayload.model_validate(console_ns.payload or {})
         _enforce_snippet_tag_rbac_by_tag_id(tag_id_str)
-        tag = TagService.update_tags(UpdateTagPayload(name=payload.name), tag_id_str, db.session)
+        tag = TagService.update_tags(UpdateTagPayload(name=payload.name), tag_id_str, db.session())
 
-        binding_count = TagService.get_tag_binding_count(tag_id_str, db.session)
+        binding_count = TagService.get_tag_binding_count(tag_id_str, db.session())
 
         return (
             dump_response(
@@ -196,7 +196,7 @@ class TagUpdateDeleteApi(Resource):
         tag_id_str = str(tag_id)
 
         _enforce_snippet_tag_rbac_by_tag_id(tag_id_str)
-        TagService.delete_tag(tag_id_str, db.session)
+        TagService.delete_tag(tag_id_str, db.session())
 
         return "", 204
 
@@ -223,7 +223,7 @@ def _create_tag_bindings(current_user: Account) -> tuple[dict[str, str], int]:
             target_id=payload.target_id,
             type=payload.type,
         ),
-        db.session,
+        db.session(),
     )
     return {"result": "success"}, 200
 
@@ -239,7 +239,7 @@ def _remove_tag_bindings(current_user: Account) -> tuple[dict[str, str], int]:
             target_id=payload.target_id,
             type=payload.type,
         ),
-        db.session,
+        db.session(),
     )
     return {"result": "success"}, 200
 

api/controllers/console/workspace/account.py

@@ -317,7 +317,7 @@ class AccountNameApi(Resource):
     def post(self, current_user: Account):
         payload = console_ns.payload or {}
         args = AccountNamePayload.model_validate(payload)
-        updated_account = AccountService.update_account(current_user, session=db.session, name=args.name)
+        updated_account = AccountService.update_account(current_user, session=db.session(), name=args.name)
 
         return dump_response(AccountResponse, updated_account)
 
@@ -363,7 +363,7 @@ class AccountAvatarApi(Resource):
         payload = console_ns.payload or {}
         args = AccountAvatarPayload.model_validate(payload)
 
-        updated_account = AccountService.update_account(current_user, session=db.session, avatar=args.avatar)
+        updated_account = AccountService.update_account(current_user, session=db.session(), avatar=args.avatar)
 
         return dump_response(AccountResponse, updated_account)
 
@@ -381,7 +381,7 @@ class AccountInterfaceLanguageApi(Resource):
         args = AccountInterfaceLanguagePayload.model_validate(payload)
 
         updated_account = AccountService.update_account(
-            current_user, session=db.session, interface_language=args.interface_language
+            current_user, session=db.session(), interface_language=args.interface_language
         )
 
         return dump_response(AccountResponse, updated_account)
@@ -400,7 +400,7 @@ class AccountInterfaceThemeApi(Resource):
         args = AccountInterfaceThemePayload.model_validate(payload)
 
         updated_account = AccountService.update_account(
-            current_user, session=db.session, interface_theme=args.interface_theme
+            current_user, session=db.session(), interface_theme=args.interface_theme
         )
 
         return dump_response(AccountResponse, updated_account)
@@ -418,7 +418,7 @@ class AccountTimezoneApi(Resource):
         payload = console_ns.payload or {}
         args = AccountTimezonePayload.model_validate(payload)
 
-        updated_account = AccountService.update_account(current_user, session=db.session, timezone=args.timezone)
+        updated_account = AccountService.update_account(current_user, session=db.session(), timezone=args.timezone)
 
         return dump_response(AccountResponse, updated_account)
 
@@ -437,7 +437,7 @@ class AccountPasswordApi(Resource):
 
         try:
             assert args.password is not None
-            AccountService.update_account_password(current_user, args.password, args.new_password, session=db.session)
+            AccountService.update_account_password(current_user, args.password, args.new_password, session=db.session())
         except ServiceCurrentPasswordIncorrectError:
             raise CurrentPasswordIncorrectError()
 
@@ -514,7 +514,7 @@ class AccountDeleteApi(Resource):
         if not AccountService.verify_account_deletion_code(args.token, args.code):
             raise InvalidAccountDeletionCodeError()
 
-        AccountService.delete_account(account)
+        AccountService.delete_account(account, session=db.session())
 
         return SimpleResultResponse(result="success").model_dump(mode="json")
 
@@ -726,7 +726,7 @@ class ChangeEmailResetApi(Resource):
         if AccountService.is_account_in_freeze(normalized_new_email):
             raise AccountInFreezeError()
 
-        if not AccountService.check_email_unique(normalized_new_email, session=db.session):
+        if not AccountService.check_email_unique(normalized_new_email, session=db.session()):
             raise EmailAlreadyInUseError()
 
         reset_data = AccountService.get_change_email_data(args.token)
@@ -751,7 +751,7 @@ class ChangeEmailResetApi(Resource):
         AccountService.revoke_change_email_token(args.token)
 
         updated_account = AccountService.update_account_email(
-            current_user, email=normalized_new_email, session=db.session
+            current_user, email=normalized_new_email, session=db.session()
         )
 
         AccountService.send_change_email_completed_notify_email(
@@ -772,6 +772,6 @@ class CheckEmailUnique(Resource):
         normalized_email = args.email.lower()
         if AccountService.is_account_in_freeze(normalized_email):
             raise AccountInFreezeError()
-        if not AccountService.check_email_unique(normalized_email, session=db.session):
+        if not AccountService.check_email_unique(normalized_email, session=db.session()):
             raise EmailAlreadyInUseError()
         return SimpleResultResponse(result="success").model_dump(mode="json")

api/controllers/console/workspace/load_balancing_config.py

@@ -10,6 +10,7 @@ from controllers.console.wraps import (
     with_current_tenant_id,
     with_current_user,
 )
+from extensions.ext_database import db
 from fields.base import ResponseModel
 from graphon.model_runtime.entities.model_entities import ModelType
 from graphon.model_runtime.errors.validate import CredentialsValidateFailedError
@@ -69,6 +70,7 @@ class LoadBalancingCredentialsValidateApi(Resource):
                 model=payload.model,
                 model_type=payload.model_type,
                 credentials=payload.credentials,
+                session=db.session(),
             )
         except CredentialsValidateFailedError as ex:
             result = False
@@ -118,6 +120,7 @@ class LoadBalancingConfigCredentialsValidateApi(Resource):
                 model=payload.model,
                 model_type=payload.model_type,
                 credentials=payload.credentials,
+                session=db.session(),
                 config_id=config_id,
             )
         except CredentialsValidateFailedError as ex:

api/controllers/console/workspace/members.py

@@ -135,7 +135,7 @@ def _normalize_enum_value(value: object) -> str:
 def _count_new_member_invites(tenant_id: str, emails: list[str]) -> int:
     new_member_count = 0
     for email in emails:
-        account = AccountService.get_account_by_email_with_case_fallback(db.session, email)
+        account = AccountService.get_account_by_email_with_case_fallback(email, session=db.session())
         if not account:
             new_member_count += 1
             continue
@@ -190,7 +190,7 @@ class MemberListApi(Resource):
             current_user, _ = current_account_with_tenant()
         if not current_user.current_tenant:
             raise ValueError("No current tenant")
-        members = TenantService.get_tenant_members(current_user.current_tenant, session=db.session)
+        members = TenantService.get_tenant_members(current_user.current_tenant, session=db.session())
         if dify_config.RBAC_ENABLED:
             member_ids = [member.id for member in members]
             member_roles = enterprise_rbac_service.RBACService.MemberRoles.batch_get(
@@ -275,7 +275,7 @@ class MemberInviteEmailApi(Resource):
                         language=interface_language,
                         role=invitee_role,
                         inviter=inviter,
-                        session=db.session,
+                        session=db.session(),
                     )
                     encoded_invitee_email = parse.quote(invitee_email)
                     invitation_results.append(
@@ -323,7 +323,7 @@ class MemberCancelInviteApi(Resource):
         else:
             try:
                 TenantService.remove_member_from_tenant(
-                    current_user.current_tenant, member, current_user, session=db.session
+                    current_user.current_tenant, member, current_user, session=db.session()
                 )
             except services.errors.account.CannotOperateSelfError as e:
                 return {"code": "cannot-operate-self", "message": str(e)}, HTTPStatus.BAD_REQUEST
@@ -368,7 +368,7 @@ class MemberUpdateRoleApi(Resource):
         try:
             assert member is not None, "Member not found"
             TenantService.update_member_role(
-                current_user.current_tenant, member, new_role, current_user, session=db.session
+                current_user.current_tenant, member, new_role, current_user, session=db.session()
             )
         except services.errors.account.CannotOperateSelfError as e:
             return {"code": "cannot-operate-self", "message": str(e)}, HTTPStatus.BAD_REQUEST
@@ -396,7 +396,7 @@ class DatasetOperatorMemberListApi(Resource):
     def get(self, current_user: Account):
         if not current_user.current_tenant:
             raise ValueError("No current tenant")
-        members = TenantService.get_dataset_operator_members(current_user.current_tenant, session=db.session)
+        members = TenantService.get_dataset_operator_members(current_user.current_tenant, session=db.session())
         return dump_response(AccountWithRoleListResponse, {"accounts": members}), HTTPStatus.OK
 
 
@@ -420,7 +420,7 @@ class SendOwnerTransferEmailApi(Resource):
         # check if the current user is the owner of the workspace
         if not current_user.current_tenant:
             raise ValueError("No current tenant")
-        if not TenantService.is_owner(current_user, current_user.current_tenant, session=db.session):
+        if not TenantService.is_owner(current_user, current_user.current_tenant, session=db.session()):
             raise NotOwnerError()
 
         if args.language is not None and args.language == "zh-Hans":
@@ -455,7 +455,7 @@ class OwnerTransferCheckApi(Resource):
         # check if the current user is the owner of the workspace
         if not current_user.current_tenant:
             raise ValueError("No current tenant")
-        if not TenantService.is_owner(current_user, current_user.current_tenant, session=db.session):
+        if not TenantService.is_owner(current_user, current_user.current_tenant, session=db.session()):
             raise NotOwnerError()
 
         user_email = current_user.email
@@ -501,7 +501,7 @@ class OwnerTransfer(Resource):
         # check if the current user is the owner of the workspace
         if not current_user.current_tenant:
             raise ValueError("No current tenant")
-        if not TenantService.is_owner(current_user, current_user.current_tenant, session=db.session):
+        if not TenantService.is_owner(current_user, current_user.current_tenant, session=db.session()):
             raise NotOwnerError()
 
         if current_user.id == str(member_id):
@@ -522,13 +522,13 @@ class OwnerTransfer(Resource):
 
         if not current_user.current_tenant:
             raise ValueError("No current tenant")
-        if not TenantService.is_member(member, current_user.current_tenant, session=db.session):
+        if not TenantService.is_member(member, current_user.current_tenant, session=db.session()):
             raise MemberNotInTenantError()
 
         try:
             assert member is not None, "Member not found"
             TenantService.update_member_role(
-                current_user.current_tenant, member, "owner", current_user, session=db.session
+                current_user.current_tenant, member, "owner", current_user, session=db.session()
             )
 
             AccountService.send_new_owner_transfer_notify_email(

api/controllers/console/workspace/model_providers.py

@@ -5,7 +5,7 @@ from flask import request, send_file
 from flask_restx import Resource
 from pydantic import BaseModel, Field, field_validator
 
-from controllers.common.fields import BinaryFileResponse, SimpleResultResponse
+from controllers.common.fields import SimpleResultResponse, ValidationResultResponse
 from controllers.common.schema import query_params_from_model, register_response_schema_models, register_schema_models
 from controllers.console import console_ns
 from controllers.console.wraps import (
@@ -22,8 +22,7 @@ from extensions.ext_database import db
 from fields.base import ResponseModel
 from graphon.model_runtime.entities.model_entities import ModelType
 from graphon.model_runtime.errors.validate import CredentialsValidateFailedError
-from graphon.model_runtime.utils.encoders import jsonable_encoder
-from libs.helper import uuid_value
+from libs.helper import dump_response, uuid_value
 from libs.login import login_required
 from models import Account
 from services.billing_service import BillingService
@@ -92,13 +91,8 @@ class ModelProviderListResponse(ResponseModel):
     data: list[ProviderResponse]
 
 
-class ProviderCredentialResponse(ResponseModel):
-    credentials: dict[str, Any] | None = Field(default=None)
-
-
-class ProviderCredentialValidateResponse(ResponseModel):
-    result: Literal["success", "error"]
-    error: str | None = None
+class ProviderCredentialsResponse(ResponseModel):
+    credentials: dict[str, Any] | None = None
 
 
 class ModelProviderPaymentCheckoutUrlResponse(ResponseModel):
@@ -118,19 +112,20 @@ register_schema_models(
 )
 register_response_schema_models(
     console_ns,
-    BinaryFileResponse,
     SimpleResultResponse,
     ModelProviderListResponse,
+    ProviderCredentialsResponse,
+    ValidationResultResponse,
     ModelProviderPaymentCheckoutUrlResponse,
-    ProviderCredentialResponse,
-    ProviderCredentialValidateResponse,
 )
 
 
 @console_ns.route("/workspaces/current/model-providers")
 class ModelProviderListApi(Resource):
     @console_ns.doc(params=query_params_from_model(ParserModelList))
-    @console_ns.response(200, "Success", console_ns.models[ModelProviderListResponse.__name__])
+    @console_ns.response(
+        200, "Model providers retrieved successfully", console_ns.models[ModelProviderListResponse.__name__]
+    )
     @setup_required
     @login_required
     @account_initialization_required
@@ -142,13 +137,17 @@ class ModelProviderListApi(Resource):
         model_provider_service = ModelProviderService()
         provider_list = model_provider_service.get_provider_list(tenant_id=tenant_id, model_type=args.model_type)
 
-        return jsonable_encoder({"data": provider_list})
+        return ModelProviderListResponse(data=provider_list).model_dump(mode="json")
 
 
 @console_ns.route("/workspaces/current/model-providers/<path:provider>/credentials")
 class ModelProviderCredentialApi(Resource):
     @console_ns.doc(params=query_params_from_model(ParserCredentialId))
-    @console_ns.response(200, "Success", console_ns.models[ProviderCredentialResponse.__name__])
+    @console_ns.response(
+        200,
+        "Provider credentials retrieved successfully",
+        console_ns.models[ProviderCredentialsResponse.__name__],
+    )
     @setup_required
     @login_required
     @account_initialization_required
@@ -163,7 +162,7 @@ class ModelProviderCredentialApi(Resource):
             tenant_id=tenant_id, provider=provider, credential_id=args.credential_id
         )
 
-        return {"credentials": credentials}
+        return ProviderCredentialsResponse(credentials=credentials).model_dump(mode="json")
 
     @console_ns.expect(console_ns.models[ParserCredentialCreate.__name__])
     @console_ns.response(201, "Credential created successfully", console_ns.models[SimpleResultResponse.__name__])
@@ -189,7 +188,7 @@ class ModelProviderCredentialApi(Resource):
         except CredentialsValidateFailedError as ex:
             raise ValueError(str(ex))
 
-        return {"result": "success"}, 201
+        return SimpleResultResponse(result="success").model_dump(mode="json"), 201
 
     @console_ns.expect(console_ns.models[ParserCredentialUpdate.__name__])
     @console_ns.response(200, "Credential updated successfully", console_ns.models[SimpleResultResponse.__name__])
@@ -216,7 +215,7 @@ class ModelProviderCredentialApi(Resource):
         except CredentialsValidateFailedError as ex:
             raise ValueError(str(ex))
 
-        return {"result": "success"}
+        return SimpleResultResponse(result="success").model_dump(mode="json")
 
     @console_ns.expect(console_ns.models[ParserCredentialDelete.__name__])
     @console_ns.response(204, "Credential deleted successfully")
@@ -258,7 +257,7 @@ class ModelProviderCredentialSwitchApi(Resource):
             provider=provider,
             credential_id=args.credential_id,
         )
-        return {"result": "success"}
+        return SimpleResultResponse(result="success").model_dump(mode="json")
 
 
 @console_ns.route("/workspaces/current/model-providers/<path:provider>/credentials/validate")
@@ -266,8 +265,8 @@ class ModelProviderValidateApi(Resource):
     @console_ns.expect(console_ns.models[ParserCredentialValidate.__name__])
     @console_ns.response(
         200,
-        "Credential validation result",
-        console_ns.models[ProviderCredentialValidateResponse.__name__],
+        "Provider credentials validated successfully",
+        console_ns.models[ValidationResultResponse.__name__],
     )
     @setup_required
     @login_required
@@ -292,12 +291,10 @@ class ModelProviderValidateApi(Resource):
             result = False
             error = str(ex)
 
-        response = {"result": "success" if result else "error"}
-
         if not result:
-            response["error"] = error or "Unknown error"
+            return ValidationResultResponse(result="error", error=error or "Unknown error").model_dump(mode="json")
 
-        return response
+        return ValidationResultResponse(result="success").model_dump(mode="json")
 
 
 @console_ns.route("/workspaces/<string:tenant_id>/model-providers/<path:provider>/<string:icon_type>/<string:lang>")
@@ -306,8 +303,9 @@ class ModelProviderIconApi(Resource):
     Get model provider icon
     """
 
-    @console_ns.response(200, "Success", console_ns.models[BinaryFileResponse.__name__])
+    @console_ns.response(200, "Model provider icon")
     def get(self, tenant_id: str, provider: str, icon_type: str, lang: str):
+        # response-contract:ignore binary send_file response
         model_provider_service = ModelProviderService()
         icon, mimetype = model_provider_service.get_model_provider_icon(
             tenant_id=tenant_id,
@@ -339,12 +337,16 @@ class PreferredProviderTypeUpdateApi(Resource):
             tenant_id=tenant_id, provider=provider, preferred_provider_type=args.preferred_provider_type
         )
 
-        return {"result": "success"}
+        return SimpleResultResponse(result="success").model_dump(mode="json")
 
 
 @console_ns.route("/workspaces/current/model-providers/<path:provider>/checkout-url")
 class ModelProviderPaymentCheckoutUrlApi(Resource):
-    @console_ns.response(200, "Success", console_ns.models[ModelProviderPaymentCheckoutUrlResponse.__name__])
+    @console_ns.response(
+        200,
+        "Model provider checkout URL retrieved successfully",
+        console_ns.models[ModelProviderPaymentCheckoutUrlResponse.__name__],
+    )
     @setup_required
     @login_required
     @account_initialization_required
@@ -353,11 +355,11 @@ class ModelProviderPaymentCheckoutUrlApi(Resource):
     def get(self, current_tenant_id: str, current_user: Account, provider: str):
         if provider != "anthropic":
             raise ValueError(f"provider name {provider} is invalid")
-        BillingService.is_tenant_owner_or_admin(db.session, current_user)
+        BillingService.is_tenant_owner_or_admin(current_user, session=db.session())
         data = BillingService.get_model_provider_payment_link(
             provider_name=provider,
             tenant_id=current_tenant_id,
             account_id=current_user.id,
             prefilled_email=current_user.email,
         )
-        return data
+        return dump_response(ModelProviderPaymentCheckoutUrlResponse, data)

api/controllers/console/workspace/models.py

@@ -5,7 +5,7 @@ from flask import request
 from flask_restx import Resource
 from pydantic import BaseModel, Field, field_validator
 
-from controllers.common.fields import SimpleResultResponse
+from controllers.common.fields import SimpleResultResponse, ValidationResultResponse
 from controllers.common.schema import (
     query_params_from_model,
     register_enum_models,
@@ -24,10 +24,10 @@ from controllers.console.wraps import (
     with_current_user,
 )
 from core.entities.provider_entities import CredentialConfiguration
+from extensions.ext_database import db
 from fields.base import ResponseModel
 from graphon.model_runtime.entities.model_entities import ModelType, ParameterRule
 from graphon.model_runtime.errors.validate import CredentialsValidateFailedError
-from graphon.model_runtime.utils.encoders import jsonable_encoder
 from libs.helper import uuid_value
 from libs.login import login_required
 from models import Account
@@ -62,7 +62,7 @@ class ParserDeleteModels(BaseModel):
 
 
 class LoadBalancingPayload(BaseModel):
-    configs: list[dict[str, Any]] | None = Field(default=None)
+    configs: list[dict[str, Any]] | None = None
     enabled: bool | None = None
 
 
@@ -139,33 +139,38 @@ class DefaultModelDataResponse(ResponseModel):
     data: DefaultModelResponse | None = None
 
 
-class ModelWithProviderListResponse(ResponseModel):
+class ProviderModelListResponse(ResponseModel):
     data: list[ModelWithProviderEntityResponse]
 
 
-class ProviderWithModelsDataResponse(ResponseModel):
+class AvailableModelListResponse(ResponseModel):
     data: list[ProviderWithModelsResponse]
 
 
-class ModelCredentialLoadBalancingResponse(ResponseModel):
+class ModelLoadBalancingConfigResponse(ResponseModel):
+    id: str
+    name: str
+    credentials: dict[str, Any]
+    credential_id: str | None = None
     enabled: bool
-    configs: list[dict[str, Any]] = Field(default_factory=list)
+    in_cooldown: bool
+    ttl: int
+
+
+class ModelLoadBalancingResponse(ResponseModel):
+    enabled: bool
+    configs: list[ModelLoadBalancingConfigResponse]
 
 
 class ModelCredentialResponse(ResponseModel):
-    credentials: dict[str, Any] = Field(default_factory=dict)
+    credentials: dict[str, Any]
     current_credential_id: str | None = None
     current_credential_name: str | None = None
-    load_balancing: ModelCredentialLoadBalancingResponse
+    load_balancing: ModelLoadBalancingResponse
     available_credentials: list[CredentialConfiguration]
 
 
-class ModelCredentialValidateResponse(ResponseModel):
-    result: str
-    error: str | None = None
-
-
-class ModelParameterRulesResponse(ResponseModel):
+class ModelParameterRuleListResponse(ResponseModel):
     data: list[ParameterRule]
 
 
@@ -186,12 +191,12 @@ register_schema_models(
 register_response_schema_models(
     console_ns,
     SimpleResultResponse,
+    ValidationResultResponse,
     DefaultModelDataResponse,
-    ModelWithProviderListResponse,
-    ProviderWithModelsDataResponse,
+    ProviderModelListResponse,
     ModelCredentialResponse,
-    ModelCredentialValidateResponse,
-    ModelParameterRulesResponse,
+    ModelParameterRuleListResponse,
+    AvailableModelListResponse,
 )
 
 register_enum_models(console_ns, ModelType)
@@ -200,7 +205,9 @@ register_enum_models(console_ns, ModelType)
 @console_ns.route("/workspaces/current/default-model")
 class DefaultModelApi(Resource):
     @console_ns.doc(params=query_params_from_model(ParserGetDefault))
-    @console_ns.response(200, "Success", console_ns.models[DefaultModelDataResponse.__name__])
+    @console_ns.response(
+        200, "Default model retrieved successfully", console_ns.models[DefaultModelDataResponse.__name__]
+    )
     @setup_required
     @login_required
     @account_initialization_required
@@ -213,7 +220,7 @@ class DefaultModelApi(Resource):
             tenant_id=tenant_id, model_type=args.model_type
         )
 
-        return jsonable_encoder({"data": default_model_entity})
+        return DefaultModelDataResponse(data=default_model_entity).model_dump(mode="json")
 
     @console_ns.expect(console_ns.models[ParserPostDefault.__name__])
     @console_ns.response(200, "Success", console_ns.models[SimpleResultResponse.__name__])
@@ -246,12 +253,14 @@ class DefaultModelApi(Resource):
                 )
                 raise ex
 
-        return {"result": "success"}
+        return SimpleResultResponse(result="success").model_dump(mode="json")
 
 
 @console_ns.route("/workspaces/current/model-providers/<path:provider>/models")
 class ModelProviderModelApi(Resource):
-    @console_ns.response(200, "Success", console_ns.models[ModelWithProviderListResponse.__name__])
+    @console_ns.response(
+        200, "Provider models retrieved successfully", console_ns.models[ProviderModelListResponse.__name__]
+    )
     @setup_required
     @login_required
     @account_initialization_required
@@ -260,10 +269,10 @@ class ModelProviderModelApi(Resource):
         model_provider_service = ModelProviderService()
         models = model_provider_service.get_models_by_provider(tenant_id=tenant_id, provider=provider)
 
-        return jsonable_encoder({"data": models})
+        return ProviderModelListResponse(data=models).model_dump(mode="json")
 
     @console_ns.expect(console_ns.models[ParserPostModels.__name__])
-    @console_ns.response(200, "Success", console_ns.models[SimpleResultResponse.__name__])
+    @console_ns.response(200, "Model updated successfully", console_ns.models[SimpleResultResponse.__name__])
     @setup_required
     @login_required
     @is_admin_or_owner_required
@@ -297,6 +306,7 @@ class ModelProviderModelApi(Resource):
                 model_type=args.model_type,
                 configs=args.load_balancing.configs,
                 config_from=args.config_from or "",
+                session=db.session(),
             )
 
             if args.load_balancing.enabled:
@@ -308,7 +318,7 @@ class ModelProviderModelApi(Resource):
                     tenant_id=tenant_id, provider=provider, model=args.model, model_type=args.model_type
                 )
 
-        return {"result": "success"}, 200
+        return SimpleResultResponse(result="success").model_dump(mode="json"), 200
 
     @console_ns.expect(console_ns.models[ParserDeleteModels.__name__])
     @console_ns.response(204, "Model deleted successfully")
@@ -332,7 +342,11 @@ class ModelProviderModelApi(Resource):
 @console_ns.route("/workspaces/current/model-providers/<path:provider>/models/credentials")
 class ModelProviderModelCredentialApi(Resource):
     @console_ns.doc(params=query_params_from_model(ParserGetCredentials))
-    @console_ns.response(200, "Success", console_ns.models[ModelCredentialResponse.__name__])
+    @console_ns.response(
+        200,
+        "Model credentials retrieved successfully",
+        console_ns.models[ModelCredentialResponse.__name__],
+    )
     @setup_required
     @login_required
     @account_initialization_required
@@ -356,6 +370,7 @@ class ModelProviderModelCredentialApi(Resource):
             provider=provider,
             model=args.model,
             model_type=args.model_type,
+            session=db.session(),
             config_from=args.config_from or "",
         )
 
@@ -376,22 +391,23 @@ class ModelProviderModelCredentialApi(Resource):
                 model=args.model,
             )
 
-        return jsonable_encoder(
-            {
-                "credentials": current_credential.get("credentials") if current_credential else {},
-                "current_credential_id": current_credential.get("current_credential_id")
-                if current_credential
-                else None,
-                "current_credential_name": current_credential.get("current_credential_name")
-                if current_credential
-                else None,
-                "load_balancing": {"enabled": is_load_balancing_enabled, "configs": load_balancing_configs},
-                "available_credentials": available_credentials,
-            }
-        )
+        credentials: dict[str, Any] = {}
+        # TODO: make this throw error when type mismatches?
+        if current_credential and isinstance(current_credential.get("credentials"), dict):
+            credentials = cast(dict[str, Any], current_credential["credentials"])
+
+        return ModelCredentialResponse(
+            credentials=credentials,
+            current_credential_id=current_credential.get("current_credential_id") if current_credential else None,
+            current_credential_name=current_credential.get("current_credential_name") if current_credential else None,
+            load_balancing=ModelLoadBalancingResponse.model_validate(
+                {"enabled": is_load_balancing_enabled, "configs": load_balancing_configs}
+            ),
+            available_credentials=available_credentials,
+        ).model_dump(mode="json")
 
     @console_ns.expect(console_ns.models[ParserCreateCredential.__name__])
-    @console_ns.response(201, "Credential created successfully", console_ns.models[SimpleResultResponse.__name__])
+    @console_ns.response(201, "Model credential created successfully", console_ns.models[SimpleResultResponse.__name__])
     @setup_required
     @login_required
     @is_admin_or_owner_required
@@ -421,10 +437,10 @@ class ModelProviderModelCredentialApi(Resource):
             )
             raise ValueError(str(ex))
 
-        return {"result": "success"}, 201
+        return SimpleResultResponse(result="success").model_dump(mode="json"), 201
 
     @console_ns.expect(console_ns.models[ParserUpdateCredential.__name__])
-    @console_ns.response(200, "Credential updated successfully", console_ns.models[SimpleResultResponse.__name__])
+    @console_ns.response(200, "Model credential updated successfully", console_ns.models[SimpleResultResponse.__name__])
     @setup_required
     @login_required
     @is_admin_or_owner_required
@@ -449,7 +465,7 @@ class ModelProviderModelCredentialApi(Resource):
         except CredentialsValidateFailedError as ex:
             raise ValueError(str(ex))
 
-        return {"result": "success"}
+        return SimpleResultResponse(result="success").model_dump(mode="json")
 
     @console_ns.expect(console_ns.models[ParserDeleteCredential.__name__])
     @console_ns.response(204, "Credential deleted successfully")
@@ -495,7 +511,7 @@ class ModelProviderModelCredentialSwitchApi(Resource):
             model=args.model,
             credential_id=args.credential_id,
         )
-        return {"result": "success"}
+        return SimpleResultResponse(result="success").model_dump(mode="json")
 
 
 @console_ns.route(
@@ -517,7 +533,7 @@ class ModelProviderModelEnableApi(Resource):
             tenant_id=tenant_id, provider=provider, model=args.model, model_type=args.model_type
         )
 
-        return {"result": "success"}
+        return SimpleResultResponse(result="success").model_dump(mode="json")
 
 
 @console_ns.route(
@@ -539,7 +555,7 @@ class ModelProviderModelDisableApi(Resource):
             tenant_id=tenant_id, provider=provider, model=args.model, model_type=args.model_type
         )
 
-        return {"result": "success"}
+        return SimpleResultResponse(result="success").model_dump(mode="json")
 
 
 class ParserValidate(BaseModel):
@@ -556,8 +572,8 @@ class ModelProviderModelValidateApi(Resource):
     @console_ns.expect(console_ns.models[ParserValidate.__name__])
     @console_ns.response(
         200,
-        "Credential validation result",
-        console_ns.models[ModelCredentialValidateResponse.__name__],
+        "Model credentials validated successfully",
+        console_ns.models[ValidationResultResponse.__name__],
     )
     @setup_required
     @login_required
@@ -583,18 +599,20 @@ class ModelProviderModelValidateApi(Resource):
             result = False
             error = str(ex)
 
-        response = {"result": "success" if result else "error"}
-
         if not result:
-            response["error"] = error or ""
+            return ValidationResultResponse(result="error", error=error or "").model_dump(mode="json")
 
-        return response
+        return ValidationResultResponse(result="success").model_dump(mode="json")
 
 
 @console_ns.route("/workspaces/current/model-providers/<path:provider>/models/parameter-rules")
 class ModelProviderModelParameterRuleApi(Resource):
     @console_ns.doc(params=query_params_from_model(ParserParameter))
-    @console_ns.response(200, "Success", console_ns.models[ModelParameterRulesResponse.__name__])
+    @console_ns.response(
+        200,
+        "Model parameter rules retrieved successfully",
+        console_ns.models[ModelParameterRuleListResponse.__name__],
+    )
     @setup_required
     @login_required
     @account_initialization_required
@@ -607,12 +625,14 @@ class ModelProviderModelParameterRuleApi(Resource):
             tenant_id=tenant_id, provider=provider, model=args.model
         )
 
-        return jsonable_encoder({"data": parameter_rules})
+        return ModelParameterRuleListResponse(data=parameter_rules).model_dump(mode="json")
 
 
 @console_ns.route("/workspaces/current/models/model-types/<string:model_type>")
 class ModelProviderAvailableModelApi(Resource):
-    @console_ns.response(200, "Success", console_ns.models[ProviderWithModelsDataResponse.__name__])
+    @console_ns.response(
+        200, "Available models retrieved successfully", console_ns.models[AvailableModelListResponse.__name__]
+    )
     @setup_required
     @login_required
     @account_initialization_required
@@ -621,4 +641,4 @@ class ModelProviderAvailableModelApi(Resource):
         model_provider_service = ModelProviderService()
         models = model_provider_service.get_models_by_model_type(tenant_id=tenant_id, model_type=model_type)
 
-        return jsonable_encoder({"data": models})
+        return AvailableModelListResponse(data=models).model_dump(mode="json")

api/controllers/console/workspace/plugin.py

@@ -31,13 +31,22 @@ from controllers.console.wraps import (
     with_current_user_id,
 )
 from core.helper.position_helper import is_filtered
-from core.plugin.entities.plugin import PluginCategory, PluginInstallationSource
+from core.plugin.entities.bundle import PluginBundleDependency
+from core.plugin.entities.parameters import PluginParameterOption
+from core.plugin.entities.plugin import (
+    PluginCategory,
+    PluginDeclaration,
+    PluginEntity,
+    PluginInstallationSource,
+)
+from core.plugin.entities.plugin_daemon import PluginDecodeResponse, PluginInstallTask, PluginInstallTaskStartResponse
 from core.plugin.impl.exc import PluginDaemonClientSideError
 from core.plugin.plugin_service import PluginService
 from core.tools.builtin_tool.providers._positions import BuiltinToolProviderSort
 from core.tools.entities.common_entities import I18nObject
 from core.tools.entities.tool_entities import ToolProviderType
 from core.tools.tool_manager import ToolManager
+from extensions.ext_database import db
 from fields.base import ResponseModel
 from graphon.model_runtime.utils.encoders import jsonable_encoder
 from libs.helper import dump_response
@@ -298,12 +307,12 @@ class PluginCategoryListResponse(ResponseModel):
     has_more: bool
 
 
-class PluginDaemonOperationResponse(RootModel[Any]):
-    root: Any
+class PluginBundleUploadResponse(RootModel[list[PluginBundleDependency]]):
+    pass
 
 
 class PluginListResponse(ResponseModel):
-    plugins: Any
+    plugins: list[PluginEntity]
     total: int
 
 
@@ -333,15 +342,15 @@ class PluginInstallationsResponse(ResponseModel):
 
 
 class PluginManifestResponse(ResponseModel):
-    manifest: Any
+    manifest: PluginDeclaration
 
 
 class PluginTasksResponse(ResponseModel):
-    tasks: Any
+    tasks: list[PluginInstallTask]
 
 
 class PluginTaskResponse(ResponseModel):
-    task: Any
+    task: PluginInstallTask
 
 
 class PluginPermissionResponse(ResponseModel):
@@ -350,7 +359,7 @@ class PluginPermissionResponse(ResponseModel):
 
 
 class PluginDynamicOptionsResponse(ResponseModel):
-    options: Any
+    options: list[PluginParameterOption]
 
 
 class PluginOperationSuccessResponse(ResponseModel):
@@ -397,10 +406,12 @@ register_response_schema_models(
     PluginCategoryBuiltinToolResponse,
     PluginCategoryInstalledPluginResponse,
     PluginCategoryListResponse,
-    PluginDaemonOperationResponse,
+    PluginBundleUploadResponse,
+    PluginDecodeResponse,
     PluginDebuggingKeyResponse,
     PluginDynamicOptionsResponse,
     PluginInstallationsResponse,
+    PluginInstallTaskStartResponse,
     PluginListResponse,
     PluginManifestResponse,
     PluginOperationSuccessResponse,
@@ -634,7 +645,7 @@ class PluginAssetApi(Resource):
 
 @console_ns.route("/workspaces/current/plugin/upload/pkg")
 class PluginUploadFromPkgApi(Resource):
-    @console_ns.response(200, "Success", console_ns.models[PluginDaemonOperationResponse.__name__])
+    @console_ns.response(200, "Success", console_ns.models[PluginDecodeResponse.__name__])
     @setup_required
     @login_required
     @account_initialization_required
@@ -655,7 +666,7 @@ class PluginUploadFromPkgApi(Resource):
 @console_ns.route("/workspaces/current/plugin/upload/github")
 class PluginUploadFromGithubApi(Resource):
     @console_ns.expect(console_ns.models[ParserGithubUpload.__name__])
-    @console_ns.response(200, "Success", console_ns.models[PluginDaemonOperationResponse.__name__])
+    @console_ns.response(200, "Success", console_ns.models[PluginDecodeResponse.__name__])
     @setup_required
     @login_required
     @account_initialization_required
@@ -675,7 +686,7 @@ class PluginUploadFromGithubApi(Resource):
 
 @console_ns.route("/workspaces/current/plugin/upload/bundle")
 class PluginUploadFromBundleApi(Resource):
-    @console_ns.response(200, "Success", console_ns.models[PluginDaemonOperationResponse.__name__])
+    @console_ns.response(200, "Success", console_ns.models[PluginBundleUploadResponse.__name__])
     @setup_required
     @login_required
     @account_initialization_required
@@ -696,7 +707,7 @@ class PluginUploadFromBundleApi(Resource):
 @console_ns.route("/workspaces/current/plugin/install/pkg")
 class PluginInstallFromPkgApi(Resource):
     @console_ns.expect(console_ns.models[ParserPluginIdentifiers.__name__])
-    @console_ns.response(200, "Success", console_ns.models[PluginDaemonOperationResponse.__name__])
+    @console_ns.response(200, "Success", console_ns.models[PluginInstallTaskStartResponse.__name__])
     @setup_required
     @login_required
     @account_initialization_required
@@ -717,7 +728,7 @@ class PluginInstallFromPkgApi(Resource):
 @console_ns.route("/workspaces/current/plugin/install/github")
 class PluginInstallFromGithubApi(Resource):
     @console_ns.expect(console_ns.models[ParserGithubInstall.__name__])
-    @console_ns.response(200, "Success", console_ns.models[PluginDaemonOperationResponse.__name__])
+    @console_ns.response(200, "Success", console_ns.models[PluginInstallTaskStartResponse.__name__])
     @setup_required
     @login_required
     @account_initialization_required
@@ -744,7 +755,7 @@ class PluginInstallFromGithubApi(Resource):
 @console_ns.route("/workspaces/current/plugin/install/marketplace")
 class PluginInstallFromMarketplaceApi(Resource):
     @console_ns.expect(console_ns.models[ParserPluginIdentifiers.__name__])
-    @console_ns.response(200, "Success", console_ns.models[PluginDaemonOperationResponse.__name__])
+    @console_ns.response(200, "Success", console_ns.models[PluginInstallTaskStartResponse.__name__])
     @setup_required
     @login_required
     @account_initialization_required
@@ -890,7 +901,7 @@ class PluginDeleteInstallTaskItemApi(Resource):
 @console_ns.route("/workspaces/current/plugin/upgrade/marketplace")
 class PluginUpgradeFromMarketplaceApi(Resource):
     @console_ns.expect(console_ns.models[ParserMarketplaceUpgrade.__name__])
-    @console_ns.response(200, "Success", console_ns.models[PluginDaemonOperationResponse.__name__])
+    @console_ns.response(200, "Success", console_ns.models[PluginInstallTaskStartResponse.__name__])
     @setup_required
     @login_required
     @account_initialization_required
@@ -913,7 +924,7 @@ class PluginUpgradeFromMarketplaceApi(Resource):
 @console_ns.route("/workspaces/current/plugin/upgrade/github")
 class PluginUpgradeFromGithubApi(Resource):
     @console_ns.expect(console_ns.models[ParserGithubUpgrade.__name__])
-    @console_ns.response(200, "Success", console_ns.models[PluginDaemonOperationResponse.__name__])
+    @console_ns.response(200, "Success", console_ns.models[PluginInstallTaskStartResponse.__name__])
     @setup_required
     @login_required
     @account_initialization_required
@@ -973,7 +984,7 @@ class PluginChangePermissionApi(Resource):
         args = ParserPermissionChange.model_validate(console_ns.payload)
 
         set_permission_result = PluginPermissionService.change_permission(
-            tenant_id, args.install_permission, args.debug_permission
+            tenant_id, args.install_permission, args.debug_permission, session=db.session()
         )
         if not set_permission_result:
             return jsonable_encoder({"success": False, "message": "Failed to set permission"})
@@ -989,7 +1000,7 @@ class PluginFetchPermissionApi(Resource):
     @account_initialization_required
     @with_current_tenant_id
     def get(self, tenant_id: str):
-        permission = PluginPermissionService.get_permission(tenant_id)
+        permission = PluginPermissionService.get_permission(tenant_id, session=db.session())
         if not permission:
             return jsonable_encoder(
                 {
@@ -1094,6 +1105,7 @@ class PluginChangeAutoUpgradeApi(Resource):
             auto_upgrade.exclude_plugins,
             auto_upgrade.include_plugins,
             category=args.category,
+            session=db.session(),
         )
         if not set_auto_upgrade_strategy_result:
             return jsonable_encoder({"success": False, "message": "Failed to set auto upgrade strategy"})
@@ -1111,7 +1123,7 @@ class PluginFetchAutoUpgradeApi(Resource):
     @with_current_tenant_id
     def get(self, tenant_id: str):
         args = ParserAutoUpgradeFetch.model_validate(request.args.to_dict(flat=True))
-        auto_upgrade = PluginAutoUpgradeService.get_strategy(tenant_id, args.category)
+        auto_upgrade = PluginAutoUpgradeService.get_strategy(tenant_id, args.category, session=db.session())
         auto_upgrade_dict = (
             _auto_upgrade_settings_to_dict(auto_upgrade)
             if auto_upgrade
@@ -1140,7 +1152,11 @@ class PluginAutoUpgradeExcludePluginApi(Resource):
         args = ParserExcludePlugin.model_validate(console_ns.payload)
 
         return jsonable_encoder(
-            {"success": PluginAutoUpgradeService.exclude_plugin(tenant_id, args.plugin_id, args.category)}
+            {
+                "success": PluginAutoUpgradeService.exclude_plugin(
+                    tenant_id, args.plugin_id, args.category, session=db.session()
+                )
+            }
         )
 
 

api/controllers/console/workspace/rbac.py

@@ -14,6 +14,7 @@ from controllers.console import console_ns
 from controllers.console.wraps import RBACPermission, RBACResourceScope, rbac_permission_required
 from core.db.session_factory import session_factory
 from core.rbac import RBACResourceWhitelistScope
+from extensions.ext_database import db
 from libs.login import current_account_with_tenant, login_required
 from models import Account
 from services.enterprise import rbac_service as svc
@@ -564,6 +565,7 @@ class RBACMyPermissionsApi(Resource):
                 account_id,
                 app_id=request.args.get("app_id") or None,
                 dataset_id=request.args.get("dataset_id") or None,
+                session=db.session(),
             )
         )
 
@@ -902,7 +904,7 @@ class RBACMemberRolesApi(Resource):
     @console_ns.response(200, "Success", console_ns.models[svc.MemberRolesResponse.__name__])
     def get(self, member_id):
         tenant_id, account_id = _current_ids()
-        return _dump(svc.RBACService.MemberRoles.get(tenant_id, account_id, str(member_id)))
+        return _dump(svc.RBACService.MemberRoles.get(tenant_id, account_id, str(member_id), session=db.session()))
 
     @login_required
     @console_ns.expect(console_ns.models[_ReplaceMemberRolesRequest.__name__])
@@ -916,6 +918,7 @@ class RBACMemberRolesApi(Resource):
                 account_id,
                 str(member_id),
                 role_ids=list(request.role_ids),
+                session=db.session(),
             )
         )
 

api/controllers/console/workspace/snippets.py

@@ -1,12 +1,9 @@
 import logging
-from datetime import datetime
-from typing import Any
 from urllib.parse import quote
+from uuid import UUID
 
 from flask import Response, request
 from flask_restx import Resource
-from pydantic import Field as PydanticField
-from pydantic import field_validator
 from sqlalchemy.orm import Session, sessionmaker
 from werkzeug.exceptions import NotFound
 
@@ -37,7 +34,8 @@ from controllers.console.wraps import (
 from core.plugin.entities.plugin import PluginDependency
 from extensions.ext_database import db
 from fields.base import ResponseModel
-from libs.helper import dump_response, to_timestamp
+from fields.snippet_fields import SnippetListItemResponse, SnippetPaginationResponse, SnippetResponse
+from libs.helper import dump_response
 from libs.login import login_required
 from models import Account
 from models.snippet import SnippetType
@@ -65,77 +63,6 @@ class SnippetUseCountResponse(ResponseModel):
     use_count: int
 
 
-class SnippetTagResponse(ResponseModel):
-    id: str
-    name: str
-    type: str
-
-
-class SnippetAccountResponse(ResponseModel):
-    id: str
-    name: str
-    email: str
-
-
-class SnippetListItemResponse(ResponseModel):
-    id: str
-    name: str
-    description: str | None
-    type: SnippetType
-    version: int
-    use_count: int
-    is_published: bool
-    icon_info: dict[str, Any] | None
-    tags: list[SnippetTagResponse]
-    created_by: str | None
-    author_name: str | None
-    created_at: int
-    updated_by: str | None
-    updated_at: int
-
-    @field_validator("created_at", "updated_at", mode="before")
-    @classmethod
-    def _normalize_timestamp(cls, value: datetime | int | None) -> int:
-        timestamp = to_timestamp(value)
-        if timestamp is None:
-            raise ValueError("timestamp is required")
-        return timestamp
-
-
-class SnippetResponse(ResponseModel):
-    id: str
-    name: str
-    description: str | None
-    type: SnippetType
-    version: int
-    use_count: int
-    is_published: bool
-    icon_info: dict[str, Any] | None
-    graph: dict[str, Any] = PydanticField(validation_alias="graph_dict")
-    input_fields: list[dict[str, Any]] = PydanticField(validation_alias="input_fields_list")
-    tags: list[SnippetTagResponse]
-    created_by: SnippetAccountResponse | None = PydanticField(validation_alias="created_by_account")
-    created_at: int
-    updated_by: SnippetAccountResponse | None = PydanticField(validation_alias="updated_by_account")
-    updated_at: int
-
-    @field_validator("created_at", "updated_at", mode="before")
-    @classmethod
-    def _normalize_timestamp(cls, value: datetime | int | None) -> int:
-        timestamp = to_timestamp(value)
-        if timestamp is None:
-            raise ValueError("timestamp is required")
-        return timestamp
-
-
-class SnippetPaginationResponse(ResponseModel):
-    data: list[SnippetListItemResponse]
-    page: int
-    limit: int
-    total: int
-    has_more: bool
-
-
 def _snippet_service() -> SnippetService:
     return SnippetService(sessionmaker(bind=db.engine, expire_on_commit=False))
 
@@ -188,7 +115,7 @@ class CustomizedSnippetsApi(Resource):
         snippet_service = _snippet_service()
         snippets, total, has_more = snippet_service.get_snippets(
             tenant_id=current_tenant_id,
-            session=db.session,
+            session=db.session(),
             page=query.page,
             limit=query.limit,
             keyword=query.keyword,
@@ -259,7 +186,7 @@ class CustomizedSnippetDetailApi(Resource):
     @login_required
     @account_initialization_required
     @with_current_tenant_id
-    def get(self, current_tenant_id: str, snippet_id: str):
+    def get(self, current_tenant_id: str, snippet_id: UUID):
         """Get customized snippet details."""
         snippet_service = _snippet_service()
         snippet = snippet_service.get_snippet_by_id(
@@ -535,4 +462,4 @@ class CustomizedSnippetUseCountIncrementApi(Resource):
             session.commit()
             session.refresh(snippet)
 
-        return SnippetUseCountResponse(result="success", use_count=snippet.use_count).model_dump(mode="json"), 200
+        return {"result": "success", "use_count": snippet.use_count}, 200

api/controllers/console/workspace/trigger_providers.py

@@ -1,20 +1,19 @@
 import logging
-from typing import Any, Literal
+from typing import Any
 
 from flask import make_response, redirect, request
 from flask_restx import Resource
-from pydantic import BaseModel, Field, RootModel, model_validator
+from pydantic import BaseModel, RootModel, model_validator
 from sqlalchemy.orm import sessionmaker
 from werkzeug.exceptions import BadRequest, Forbidden
 
 from configs import dify_config
 from controllers.common.errors import NotFoundError
-from controllers.common.fields import BinaryFileResponse, RedirectResponse, SimpleResultResponse
+from controllers.common.fields import SimpleResultResponse
 from controllers.common.schema import register_response_schema_models, register_schema_models
-from core.entities.parameter_entities import AppSelectorScope, ModelSelectorScope, ToolSelectorScope
+from core.entities.provider_entities import ProviderConfig
 from core.plugin.entities.plugin_daemon import CredentialType
 from core.plugin.impl.oauth import OAuthHandler
-from core.tools.entities.common_entities import I18nObject
 from core.trigger.entities.api_entities import (
     SubscriptionBuilderApiEntity,
     TriggerProviderApiEntity,
@@ -24,7 +23,7 @@ from core.trigger.entities.entities import RequestLog, SubscriptionBuilderUpdate
 from core.trigger.trigger_manager import TriggerManager
 from extensions.ext_database import db
 from fields.base import ResponseModel
-from graphon.model_runtime.utils.encoders import jsonable_encoder
+from libs.helper import dump_response
 from libs.login import login_required
 from models.account import Account
 from models.provider_ids import TriggerProviderID
@@ -59,9 +58,9 @@ class TriggerSubscriptionBuilderVerifyPayload(BaseModel):
 
 class TriggerSubscriptionBuilderUpdatePayload(BaseModel):
     name: str | None = None
-    parameters: dict[str, Any] | None = Field(default=None)
-    properties: dict[str, Any] | None = Field(default=None)
-    credentials: dict[str, Any] | None = Field(default=None)
+    parameters: dict[str, Any] | None = None
+    properties: dict[str, Any] | None = None
+    credentials: dict[str, Any] | None = None
 
     @model_validator(mode="after")
     def check_at_least_one_field(self):
@@ -71,70 +70,23 @@ class TriggerSubscriptionBuilderUpdatePayload(BaseModel):
 
 
 class TriggerOAuthClientPayload(BaseModel):
-    client_params: dict[str, Any] | None = Field(default=None)
+    client_params: dict[str, Any] | None = None
     enabled: bool | None = None
 
 
-class TriggerOAuthAuthorizeResponse(BaseModel):
-    authorization_url: str
-    subscription_builder_id: str
-    subscription_builder: SubscriptionBuilderApiEntity
-
-
-class TriggerProviderConfigOptionResponse(BaseModel):
-    value: str = Field(..., description="The value of the option")
-    label: I18nObject = Field(..., description="The label of the option")
-
-
-class TriggerProviderConfigResponse(BaseModel):
-    type: Literal[
-        "secret-input",
-        "text-input",
-        "select",
-        "boolean",
-        "app-selector",
-        "model-selector",
-        "array[tools]",
-    ] = Field(..., description="The type of the credentials")
-    name: str = Field(..., description="The name of the credentials")
-    scope: AppSelectorScope | ModelSelectorScope | ToolSelectorScope | None = None
-    required: bool = False
-    default: int | str | float | bool | None = None
-    options: list[TriggerProviderConfigOptionResponse] | None = None
-    multiple: bool = False
-    label: I18nObject | None = None
-    help: I18nObject | None = None
-    url: str | None = None
-    placeholder: I18nObject | None = None
-
-
-class TriggerOAuthClientResponse(BaseModel):
-    configured: bool
-    system_configured: bool
-    custom_configured: bool
-    oauth_client_schema: list[TriggerProviderConfigResponse]
-    custom_enabled: bool
-    redirect_uri: str
-    params: dict[str, Any]
-
-
-class TriggerProviderOpaqueResponse(RootModel[Any]):
-    root: Any
-
-
 class TriggerProviderListResponse(RootModel[list[TriggerProviderApiEntity]]):
-    root: list[TriggerProviderApiEntity]
+    pass
 
 
-class TriggerSubscriptionListResponse(RootModel[list[TriggerProviderSubscriptionApiEntity]]):
-    root: list[TriggerProviderSubscriptionApiEntity]
+class TriggerProviderSubscriptionListResponse(RootModel[list[TriggerProviderSubscriptionApiEntity]]):
+    pass
 
 
 class TriggerSubscriptionBuilderCreateResponse(ResponseModel):
     subscription_builder: SubscriptionBuilderApiEntity
 
 
-class TriggerSubscriptionBuilderVerifyResponse(ResponseModel):
+class TriggerVerificationResponse(ResponseModel):
     verified: bool
 
 
@@ -142,6 +94,26 @@ class TriggerSubscriptionBuilderLogsResponse(ResponseModel):
     logs: list[RequestLog]
 
 
+class TriggerOAuthAuthorizeResponse(ResponseModel):
+    authorization_url: str
+    subscription_builder_id: str
+    subscription_builder: SubscriptionBuilderApiEntity
+
+
+class TriggerOAuthClientResponse(ResponseModel):
+    configured: bool
+    system_configured: bool
+    custom_configured: bool
+    oauth_client_schema: list[ProviderConfig]
+    custom_enabled: bool
+    redirect_uri: str
+    params: dict[str, Any]
+
+
+class TriggerProviderErrorResponse(ResponseModel):
+    error: str
+
+
 register_schema_models(
     console_ns,
     TriggerSubscriptionBuilderCreatePayload,
@@ -151,27 +123,24 @@ register_schema_models(
 )
 register_response_schema_models(
     console_ns,
-    BinaryFileResponse,
-    RedirectResponse,
     SimpleResultResponse,
     TriggerOAuthAuthorizeResponse,
     TriggerOAuthClientResponse,
-    TriggerProviderOpaqueResponse,
     TriggerProviderApiEntity,
+    TriggerProviderErrorResponse,
     TriggerProviderListResponse,
-    TriggerProviderSubscriptionApiEntity,
-    TriggerSubscriptionListResponse,
-    SubscriptionBuilderApiEntity,
+    TriggerProviderSubscriptionListResponse,
     TriggerSubscriptionBuilderCreateResponse,
-    TriggerSubscriptionBuilderVerifyResponse,
-    RequestLog,
     TriggerSubscriptionBuilderLogsResponse,
+    SubscriptionBuilderApiEntity,
+    TriggerVerificationResponse,
 )
 
 
 @console_ns.route("/workspaces/current/trigger-provider/<path:provider>/icon")
 class TriggerProviderIconApi(Resource):
-    @console_ns.response(200, "Success", console_ns.models[BinaryFileResponse.__name__])
+    # response-contract:ignore binary trigger provider icon
+    @console_ns.response(200, "Trigger provider icon")
     @setup_required
     @login_required
     @account_initialization_required
@@ -182,31 +151,45 @@ class TriggerProviderIconApi(Resource):
 
 @console_ns.route("/workspaces/current/triggers")
 class TriggerProviderListApi(Resource):
-    @console_ns.response(200, "Success", console_ns.models[TriggerProviderListResponse.__name__])
+    @console_ns.response(
+        200,
+        "Trigger providers retrieved successfully",
+        console_ns.models[TriggerProviderListResponse.__name__],
+    )
     @setup_required
     @login_required
     @account_initialization_required
     @with_current_tenant_id
     def get(self, tenant_id: str):
         """List all trigger providers for the current tenant"""
-        return jsonable_encoder(TriggerProviderService.list_trigger_providers(tenant_id))
+        return dump_response(TriggerProviderListResponse, TriggerProviderService.list_trigger_providers(tenant_id))
 
 
 @console_ns.route("/workspaces/current/trigger-provider/<path:provider>/info")
 class TriggerProviderInfoApi(Resource):
-    @console_ns.response(200, "Success", console_ns.models[TriggerProviderApiEntity.__name__])
+    @console_ns.response(
+        200,
+        "Trigger provider retrieved successfully",
+        console_ns.models[TriggerProviderApiEntity.__name__],
+    )
     @setup_required
     @login_required
     @account_initialization_required
     @with_current_tenant_id
     def get(self, tenant_id: str, provider: str):
         """Get info for a trigger provider"""
-        return jsonable_encoder(TriggerProviderService.get_trigger_provider(tenant_id, TriggerProviderID(provider)))
+        provider_entity = TriggerProviderService.get_trigger_provider(tenant_id, TriggerProviderID(provider))
+        return provider_entity.model_dump(mode="json")
 
 
 @console_ns.route("/workspaces/current/trigger-provider/<path:provider>/subscriptions/list")
 class TriggerSubscriptionListApi(Resource):
-    @console_ns.response(200, "Success", console_ns.models[TriggerSubscriptionListResponse.__name__])
+    @console_ns.response(
+        200,
+        "Trigger subscriptions retrieved successfully",
+        console_ns.models[TriggerProviderSubscriptionListResponse.__name__],
+    )
+    @console_ns.response(404, "Trigger provider not found", console_ns.models[TriggerProviderErrorResponse.__name__])
     @setup_required
     @login_required
     @edit_permission_required
@@ -216,16 +199,18 @@ class TriggerSubscriptionListApi(Resource):
     @with_current_tenant_id
     def get(self, tenant_id: str, user: Account, provider: str):
         """List all trigger subscriptions for the current tenant's provider"""
+
         try:
-            return jsonable_encoder(
+            return dump_response(
+                TriggerProviderSubscriptionListResponse,
                 TriggerProviderService.list_trigger_provider_subscriptions(
                     tenant_id=tenant_id,
                     provider_id=TriggerProviderID(provider),
                     user=user,
-                )
+                ),
             )
         except ValueError as e:
-            return jsonable_encoder({"error": str(e)}), 404
+            return TriggerProviderErrorResponse(error=str(e)).model_dump(mode="json"), 404
         except Exception as e:
             logger.exception("Error listing trigger providers", exc_info=e)
             raise
@@ -236,7 +221,11 @@ class TriggerSubscriptionListApi(Resource):
 )
 class TriggerSubscriptionBuilderCreateApi(Resource):
     @console_ns.expect(console_ns.models[TriggerSubscriptionBuilderCreatePayload.__name__])
-    @console_ns.response(200, "Success", console_ns.models[TriggerSubscriptionBuilderCreateResponse.__name__])
+    @console_ns.response(
+        200,
+        "Trigger subscription builder created successfully",
+        console_ns.models[TriggerSubscriptionBuilderCreateResponse.__name__],
+    )
     @setup_required
     @login_required
     @edit_permission_required
@@ -246,6 +235,7 @@ class TriggerSubscriptionBuilderCreateApi(Resource):
     @with_current_tenant_id
     def post(self, tenant_id: str, user: Account, provider: str):
         """Add a new subscription instance for a trigger provider"""
+
         payload = TriggerSubscriptionBuilderCreatePayload.model_validate(console_ns.payload or {})
 
         try:
@@ -256,7 +246,9 @@ class TriggerSubscriptionBuilderCreateApi(Resource):
                 provider_id=TriggerProviderID(provider),
                 credential_type=credential_type,
             )
-            return jsonable_encoder({"subscription_builder": subscription_builder})
+            return TriggerSubscriptionBuilderCreateResponse(subscription_builder=subscription_builder).model_dump(
+                mode="json"
+            )
         except Exception as e:
             logger.exception("Error adding provider credential", exc_info=e)
             raise
@@ -266,7 +258,11 @@ class TriggerSubscriptionBuilderCreateApi(Resource):
     "/workspaces/current/trigger-provider/<path:provider>/subscriptions/builder/<path:subscription_builder_id>",
 )
 class TriggerSubscriptionBuilderGetApi(Resource):
-    @console_ns.response(200, "Success", console_ns.models[SubscriptionBuilderApiEntity.__name__])
+    @console_ns.response(
+        200,
+        "Trigger subscription builder retrieved successfully",
+        console_ns.models[SubscriptionBuilderApiEntity.__name__],
+    )
     @setup_required
     @login_required
     @edit_permission_required
@@ -274,9 +270,8 @@ class TriggerSubscriptionBuilderGetApi(Resource):
     @account_initialization_required
     def get(self, provider: str, subscription_builder_id: str):
         """Get a subscription instance for a trigger provider"""
-        return jsonable_encoder(
-            TriggerSubscriptionBuilderService.get_subscription_builder_by_id(subscription_builder_id)
-        )
+        subscription_builder = TriggerSubscriptionBuilderService.get_subscription_builder_by_id(subscription_builder_id)
+        return subscription_builder.model_dump(mode="json")
 
 
 @console_ns.route(
@@ -284,7 +279,11 @@ class TriggerSubscriptionBuilderGetApi(Resource):
 )
 class TriggerSubscriptionBuilderVerifyApi(Resource):
     @console_ns.expect(console_ns.models[TriggerSubscriptionBuilderVerifyPayload.__name__])
-    @console_ns.response(200, "Success", console_ns.models[TriggerSubscriptionBuilderVerifyResponse.__name__])
+    @console_ns.response(
+        200,
+        "Trigger subscription builder verified successfully",
+        console_ns.models[TriggerVerificationResponse.__name__],
+    )
     @setup_required
     @login_required
     @edit_permission_required
@@ -294,11 +293,12 @@ class TriggerSubscriptionBuilderVerifyApi(Resource):
     @with_current_tenant_id
     def post(self, tenant_id: str, user: Account, provider: str, subscription_builder_id: str):
         """Verify and update a subscription instance for a trigger provider"""
+
         payload = TriggerSubscriptionBuilderVerifyPayload.model_validate(console_ns.payload or {})
 
         try:
             # Use atomic update_and_verify to prevent race conditions
-            return TriggerSubscriptionBuilderService.update_and_verify_builder(
+            result = TriggerSubscriptionBuilderService.update_and_verify_builder(
                 tenant_id=tenant_id,
                 user_id=user.id,
                 provider_id=TriggerProviderID(provider),
@@ -307,6 +307,7 @@ class TriggerSubscriptionBuilderVerifyApi(Resource):
                     credentials=payload.credentials,
                 ),
             )
+            return dump_response(TriggerVerificationResponse, result)
         except Exception as e:
             logger.exception("Error verifying provider credential", exc_info=e)
             raise ValueError(str(e)) from e
@@ -317,7 +318,11 @@ class TriggerSubscriptionBuilderVerifyApi(Resource):
 )
 class TriggerSubscriptionBuilderUpdateApi(Resource):
     @console_ns.expect(console_ns.models[TriggerSubscriptionBuilderUpdatePayload.__name__])
-    @console_ns.response(200, "Success", console_ns.models[SubscriptionBuilderApiEntity.__name__])
+    @console_ns.response(
+        200,
+        "Trigger subscription builder updated successfully",
+        console_ns.models[SubscriptionBuilderApiEntity.__name__],
+    )
     @setup_required
     @login_required
     @edit_permission_required
@@ -326,21 +331,20 @@ class TriggerSubscriptionBuilderUpdateApi(Resource):
     @with_current_tenant_id
     def post(self, tenant_id: str, provider: str, subscription_builder_id: str):
         """Update a subscription instance for a trigger provider"""
+
         payload = TriggerSubscriptionBuilderUpdatePayload.model_validate(console_ns.payload or {})
         try:
-            return jsonable_encoder(
-                TriggerSubscriptionBuilderService.update_trigger_subscription_builder(
-                    tenant_id=tenant_id,
-                    provider_id=TriggerProviderID(provider),
-                    subscription_builder_id=subscription_builder_id,
-                    subscription_builder_updater=SubscriptionBuilderUpdater(
-                        name=payload.name,
-                        parameters=payload.parameters,
-                        properties=payload.properties,
-                        credentials=payload.credentials,
-                    ),
-                )
-            )
+            return TriggerSubscriptionBuilderService.update_trigger_subscription_builder(
+                tenant_id=tenant_id,
+                provider_id=TriggerProviderID(provider),
+                subscription_builder_id=subscription_builder_id,
+                subscription_builder_updater=SubscriptionBuilderUpdater(
+                    name=payload.name,
+                    parameters=payload.parameters,
+                    properties=payload.properties,
+                    credentials=payload.credentials,
+                ),
+            ).model_dump(mode="json")
         except Exception as e:
             logger.exception("Error updating provider credential", exc_info=e)
             raise
@@ -350,7 +354,11 @@ class TriggerSubscriptionBuilderUpdateApi(Resource):
     "/workspaces/current/trigger-provider/<path:provider>/subscriptions/builder/logs/<path:subscription_builder_id>",
 )
 class TriggerSubscriptionBuilderLogsApi(Resource):
-    @console_ns.response(200, "Success", console_ns.models[TriggerSubscriptionBuilderLogsResponse.__name__])
+    @console_ns.response(
+        200,
+        "Trigger subscription builder logs retrieved successfully",
+        console_ns.models[TriggerSubscriptionBuilderLogsResponse.__name__],
+    )
     @setup_required
     @login_required
     @edit_permission_required
@@ -358,9 +366,10 @@ class TriggerSubscriptionBuilderLogsApi(Resource):
     @account_initialization_required
     def get(self, provider: str, subscription_builder_id: str):
         """Get the request logs for a subscription instance for a trigger provider"""
+
         try:
             logs = TriggerSubscriptionBuilderService.list_logs(subscription_builder_id)
-            return jsonable_encoder({"logs": [log.model_dump(mode="json") for log in logs]})
+            return dump_response(TriggerSubscriptionBuilderLogsResponse, {"logs": logs})
         except Exception as e:
             logger.exception("Error getting request logs for subscription builder", exc_info=e)
             raise
@@ -371,7 +380,9 @@ class TriggerSubscriptionBuilderLogsApi(Resource):
 )
 class TriggerSubscriptionBuilderBuildApi(Resource):
     @console_ns.expect(console_ns.models[TriggerSubscriptionBuilderUpdatePayload.__name__])
-    @console_ns.response(200, "Success", console_ns.models[TriggerProviderOpaqueResponse.__name__])
+    @console_ns.response(
+        200, "Trigger subscription builder built successfully", console_ns.models[SimpleResultResponse.__name__]
+    )
     @setup_required
     @login_required
     @edit_permission_required
@@ -395,7 +406,7 @@ class TriggerSubscriptionBuilderBuildApi(Resource):
                     properties=payload.properties,
                 ),
             )
-            return 200
+            return SimpleResultResponse(result="success").model_dump(mode="json")
         except Exception as e:
             logger.exception("Error building provider credential", exc_info=e)
             raise ValueError(str(e)) from e
@@ -406,7 +417,9 @@ class TriggerSubscriptionBuilderBuildApi(Resource):
 )
 class TriggerSubscriptionUpdateApi(Resource):
     @console_ns.expect(console_ns.models[TriggerSubscriptionBuilderUpdatePayload.__name__])
-    @console_ns.response(200, "Success", console_ns.models[TriggerProviderOpaqueResponse.__name__])
+    @console_ns.response(
+        200, "Trigger subscription updated successfully", console_ns.models[SimpleResultResponse.__name__]
+    )
     @setup_required
     @login_required
     @edit_permission_required
@@ -415,6 +428,7 @@ class TriggerSubscriptionUpdateApi(Resource):
     @with_current_tenant_id
     def post(self, tenant_id: str, subscription_id: str):
         """Update a subscription instance"""
+
         request = TriggerSubscriptionBuilderUpdatePayload.model_validate(console_ns.payload or {})
 
         subscription = TriggerProviderService.get_subscription_by_id(
@@ -440,7 +454,7 @@ class TriggerSubscriptionUpdateApi(Resource):
                     name=request.name,
                     properties=request.properties,
                 )
-                return 200
+                return SimpleResultResponse(result="success").model_dump(mode="json")
 
             # For the rest cases(API_KEY, OAUTH2)
             # we need to call third party provider(e.g. GitHub) to rebuild the subscription
@@ -452,7 +466,7 @@ class TriggerSubscriptionUpdateApi(Resource):
                 credentials=request.credentials or subscription.credentials,
                 parameters=request.parameters or subscription.parameters,
             )
-            return 200
+            return SimpleResultResponse(result="success").model_dump(mode="json")
         except ValueError as e:
             raise BadRequest(str(e))
         except Exception as e:
@@ -473,6 +487,7 @@ class TriggerSubscriptionDeleteApi(Resource):
     @with_current_tenant_id
     def post(self, tenant_id: str, subscription_id: str):
         """Delete a subscription instance"""
+
         try:
             with sessionmaker(db.engine).begin() as session:
                 # Delete trigger provider subscription
@@ -487,7 +502,7 @@ class TriggerSubscriptionDeleteApi(Resource):
                     tenant_id=tenant_id,
                     subscription_id=subscription_id,
                 )
-            return {"result": "success"}
+            return SimpleResultResponse(result="success").model_dump(mode="json")
         except ValueError as e:
             raise BadRequest(str(e))
         except Exception as e:
@@ -497,9 +512,10 @@ class TriggerSubscriptionDeleteApi(Resource):
 
 @console_ns.route("/workspaces/current/trigger-provider/<path:provider>/subscriptions/oauth/authorize")
 class TriggerOAuthAuthorizeApi(Resource):
+    # response-contract:ignore cookie-bearing Flask response
     @console_ns.response(
         200,
-        "Authorization URL retrieved successfully",
+        "Trigger OAuth authorization URL generated successfully",
         console_ns.models[TriggerOAuthAuthorizeResponse.__name__],
     )
     @setup_required
@@ -509,10 +525,12 @@ class TriggerOAuthAuthorizeApi(Resource):
     @with_current_tenant_id
     def get(self, tenant_id: str, user: Account, provider: str):
         """Initiate OAuth authorization flow for a trigger provider"""
+
         try:
             provider_id = TriggerProviderID(provider)
             plugin_id = provider_id.plugin_id
             provider_name = provider_id.provider_name
+            tenant_id = tenant_id
 
             # Get OAuth client configuration
             oauth_client_params = TriggerProviderService.get_oauth_client(
@@ -556,15 +574,12 @@ class TriggerOAuthAuthorizeApi(Resource):
                 system_credentials=oauth_client_params,
             )
 
-            # Create response with cookie
             response = make_response(
-                jsonable_encoder(
-                    {
-                        "authorization_url": authorization_url_response.authorization_url,
-                        "subscription_builder_id": subscription_builder.id,
-                        "subscription_builder": subscription_builder,
-                    }
-                )
+                TriggerOAuthAuthorizeResponse(
+                    authorization_url=authorization_url_response.authorization_url,
+                    subscription_builder_id=subscription_builder.id,
+                    subscription_builder=subscription_builder,
+                ).model_dump(mode="json")
             )
             response.set_cookie(
                 "context_id",
@@ -583,11 +598,8 @@ class TriggerOAuthAuthorizeApi(Resource):
 
 @console_ns.route("/oauth/plugin/<path:provider>/trigger/callback")
 class TriggerOAuthCallbackApi(Resource):
-    @console_ns.response(
-        302,
-        "Redirect to console OAuth callback page",
-        console_ns.models[RedirectResponse.__name__],
-    )
+    # response-contract:ignore redirect response
+    @console_ns.response(302, "Redirect to OAuth callback page")
     @setup_required
     def get(self, provider: str):
         """Handle OAuth callback for trigger provider"""
@@ -653,7 +665,11 @@ class TriggerOAuthCallbackApi(Resource):
 
 @console_ns.route("/workspaces/current/trigger-provider/<path:provider>/oauth/client")
 class TriggerOAuthClientManageApi(Resource):
-    @console_ns.response(200, "Success", console_ns.models[TriggerOAuthClientResponse.__name__])
+    @console_ns.response(
+        200,
+        "Trigger OAuth client retrieved successfully",
+        console_ns.models[TriggerOAuthClientResponse.__name__],
+    )
     @setup_required
     @login_required
     @is_admin_or_owner_required
@@ -662,6 +678,7 @@ class TriggerOAuthClientManageApi(Resource):
     @with_current_tenant_id
     def get(self, tenant_id: str, provider: str):
         """Get OAuth client configuration for a provider"""
+
         try:
             provider_id = TriggerProviderID(provider)
 
@@ -682,24 +699,24 @@ class TriggerOAuthClientManageApi(Resource):
             )
             provider_controller = TriggerManager.get_trigger_provider(tenant_id, provider_id)
             redirect_uri = f"{dify_config.CONSOLE_API_URL}/console/api/oauth/plugin/{provider}/trigger/callback"
-            return jsonable_encoder(
-                {
-                    "configured": bool(custom_params or system_client_exists),
-                    "system_configured": system_client_exists,
-                    "custom_configured": bool(custom_params),
-                    "oauth_client_schema": provider_controller.get_oauth_client_schema(),
-                    "custom_enabled": is_custom_enabled,
-                    "redirect_uri": redirect_uri,
-                    "params": custom_params or {},
-                }
-            )
+            return TriggerOAuthClientResponse(
+                configured=bool(custom_params or system_client_exists),
+                system_configured=system_client_exists,
+                custom_configured=bool(custom_params),
+                oauth_client_schema=provider_controller.get_oauth_client_schema(),
+                custom_enabled=is_custom_enabled,
+                redirect_uri=redirect_uri,
+                params=dict(custom_params or {}),
+            ).model_dump(mode="json")
 
         except Exception as e:
             logger.exception("Error getting OAuth client", exc_info=e)
             raise
 
     @console_ns.expect(console_ns.models[TriggerOAuthClientPayload.__name__])
-    @console_ns.response(200, "Success", console_ns.models[SimpleResultResponse.__name__])
+    @console_ns.response(
+        200, "Trigger OAuth client saved successfully", console_ns.models[SimpleResultResponse.__name__]
+    )
     @setup_required
     @login_required
     @is_admin_or_owner_required
@@ -708,16 +725,18 @@ class TriggerOAuthClientManageApi(Resource):
     @with_current_tenant_id
     def post(self, tenant_id: str, provider: str):
         """Configure custom OAuth client for a provider"""
+
         payload = TriggerOAuthClientPayload.model_validate(console_ns.payload or {})
 
         try:
             provider_id = TriggerProviderID(provider)
-            return TriggerProviderService.save_custom_oauth_client_params(
+            result = TriggerProviderService.save_custom_oauth_client_params(
                 tenant_id=tenant_id,
                 provider_id=provider_id,
                 client_params=payload.client_params,
                 enabled=payload.enabled,
             )
+            return dump_response(SimpleResultResponse, result)
 
         except ValueError as e:
             raise BadRequest(str(e))
@@ -725,22 +744,26 @@ class TriggerOAuthClientManageApi(Resource):
             logger.exception("Error configuring OAuth client", exc_info=e)
             raise
 
+    @console_ns.response(
+        200, "Trigger OAuth client deleted successfully", console_ns.models[SimpleResultResponse.__name__]
+    )
     @setup_required
     @login_required
     @is_admin_or_owner_required
     @rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.PLUGIN_PREFERENCES, resource_required=False)
     @account_initialization_required
-    @console_ns.response(200, "Success", console_ns.models[SimpleResultResponse.__name__])
     @with_current_tenant_id
     def delete(self, tenant_id: str, provider: str):
         """Remove custom OAuth client configuration"""
+
         try:
             provider_id = TriggerProviderID(provider)
 
-            return TriggerProviderService.delete_custom_oauth_client_params(
+            result = TriggerProviderService.delete_custom_oauth_client_params(
                 tenant_id=tenant_id,
                 provider_id=provider_id,
             )
+            return dump_response(SimpleResultResponse, result)
         except ValueError as e:
             raise BadRequest(str(e))
         except Exception as e:
@@ -753,7 +776,11 @@ class TriggerOAuthClientManageApi(Resource):
 )
 class TriggerSubscriptionVerifyApi(Resource):
     @console_ns.expect(console_ns.models[TriggerSubscriptionBuilderVerifyPayload.__name__])
-    @console_ns.response(200, "Success", console_ns.models[TriggerSubscriptionBuilderVerifyResponse.__name__])
+    @console_ns.response(
+        200,
+        "Trigger subscription verified successfully",
+        console_ns.models[TriggerVerificationResponse.__name__],
+    )
     @setup_required
     @login_required
     @edit_permission_required
@@ -763,6 +790,7 @@ class TriggerSubscriptionVerifyApi(Resource):
     @with_current_tenant_id
     def post(self, tenant_id: str, user: Account, provider: str, subscription_id: str):
         """Verify credentials for an existing subscription (edit mode only)"""
+
         verify_request = TriggerSubscriptionBuilderVerifyPayload.model_validate(console_ns.payload or {})
 
         try:
@@ -773,7 +801,7 @@ class TriggerSubscriptionVerifyApi(Resource):
                 subscription_id=subscription_id,
                 credentials=verify_request.credentials,
             )
-            return result
+            return dump_response(TriggerVerificationResponse, result)
         except ValueError as e:
             logger.warning("Credential verification failed", exc_info=e)
             raise BadRequest(str(e)) from e

api/controllers/console/workspace/workspace.py

@@ -223,7 +223,7 @@ class TenantListApi(Resource):
     def get(self, current_tenant_id: str, current_user: Account):
         tenant_rows: list[tuple[Tenant, TenantAccountJoin]] = [
             (tenant, membership)
-            for tenant, membership in TenantService.get_workspaces_for_account(db.session, current_user.id)
+            for tenant, membership in TenantService.get_workspaces_for_account(current_user.id, session=db.session())
             if tenant.status == TenantStatus.NORMAL
         ]
         tenants = [tenant for tenant, _ in tenant_rows]
@@ -306,16 +306,19 @@ class TenantApi(Resource):
             raise ValueError("No current tenant")
 
         if tenant.status == TenantStatus.ARCHIVE:
-            tenants = TenantService.get_join_tenants(current_user, session=db.session)
+            tenants = TenantService.get_join_tenants(current_user, session=db.session())
             # if there is any tenant, switch to the first one
             if len(tenants) > 0:
-                TenantService.switch_tenant(current_user, tenants[0].id, session=db.session)
+                TenantService.switch_tenant(current_user, tenants[0].id, session=db.session())
                 tenant = tenants[0]
             # else, raise Unauthorized
             else:
                 raise Unauthorized("workspace is archived")
 
-        return dump_response(TenantInfoResponse, WorkspaceService.get_tenant_info(tenant)), HTTPStatus.OK
+        return (
+            dump_response(TenantInfoResponse, WorkspaceService.get_tenant_info(tenant, session=db.session())),
+            HTTPStatus.OK,
+        )
 
 
 @console_ns.route("/workspaces/switch")
@@ -332,7 +335,7 @@ class SwitchWorkspaceApi(Resource):
 
         # Check whether the tenant_id belongs to the current account.
         try:
-            TenantService.switch_tenant(current_user, args.tenant_id, session=db.session)
+            TenantService.switch_tenant(current_user, args.tenant_id, session=db.session())
         except Exception:
             raise AccountNotLinkTenantError("Account not link tenant")
 
@@ -341,7 +344,7 @@ class SwitchWorkspaceApi(Resource):
             raise ValueError("Tenant not found")
 
         return SwitchWorkspaceResponse(
-            result="success", new_tenant=WorkspaceService.get_tenant_info(new_tenant)
+            result="success", new_tenant=WorkspaceService.get_tenant_info(new_tenant, session=db.session())
         ).model_dump(mode="json")
 
 
@@ -372,7 +375,7 @@ class CustomConfigWorkspaceApi(Resource):
         db.session.commit()
 
         return WorkspaceTenantResultResponse(
-            result="success", tenant=WorkspaceService.get_tenant_info(tenant)
+            result="success", tenant=WorkspaceService.get_tenant_info(tenant, session=db.session())
         ).model_dump(mode="json")
 
 
@@ -438,7 +441,7 @@ class WorkspaceInfoApi(Resource):
         db.session.commit()
 
         return WorkspaceTenantResultResponse(
-            result="success", tenant=WorkspaceService.get_tenant_info(tenant)
+            result="success", tenant=WorkspaceService.get_tenant_info(tenant, session=db.session())
         ).model_dump(mode="json")
 
 

api/controllers/console/wraps.py

@@ -4,7 +4,7 @@ import os
 import time
 from collections.abc import Callable
 from functools import wraps
-from typing import Any, Concatenate, overload
+from typing import Any, Concatenate, Protocol, cast, overload
 
 from flask import abort, request
 from pydantic import BaseModel, ValidationError
@@ -46,6 +46,60 @@ ERROR_MSG_INVALID_ENCRYPTED_DATA = "Invalid encrypted data"
 ERROR_MSG_INVALID_ENCRYPTED_CODE = "Invalid encrypted code"
 
 
+class OnceTrueCallable[**P](Protocol):
+    def __call__(self, *args: P.args, **kwargs: P.kwargs) -> bool: ...
+
+    def mark_success(self) -> None: ...
+
+    def reset_success(self) -> None: ...
+
+
+def once_true[**P](func: Callable[P, bool]) -> OnceTrueCallable[P]:
+    """Wrap a predicate so only a strict True result is memoized."""
+    has_success = False
+
+    def mark_success() -> None:
+        nonlocal has_success
+
+        has_success = True
+
+    def reset_success() -> None:
+        nonlocal has_success
+
+        has_success = False
+
+    @wraps(func)
+    def wrapper(*args: P.args, **kwargs: P.kwargs) -> bool:
+        nonlocal has_success
+
+        if has_success:
+            return True
+
+        result = func(*args, **kwargs)
+        if result is True:
+            has_success = True
+
+        return result
+
+    wrapper.mark_success = mark_success  # type: ignore[attr-defined]
+    wrapper.reset_success = reset_success  # type: ignore[attr-defined]
+    return cast(OnceTrueCallable[P], wrapper)
+
+
+def mark_setup_completed() -> None:
+    """Remember in this process that one-time self-hosted setup has completed."""
+    _is_setup_completed.mark_success()
+
+
+@once_true
+def _is_setup_completed() -> bool:
+    """Check whether setup exists, caching only successful observations.
+
+    Use `once_true` instead of `@cache` because a pre-setup False result must not be memoized.
+    """
+    return db.session.scalar(select(DifySetup).limit(1)) is not None
+
+
 @overload
 def account_initialization_required[T, **P, R](
     view: Callable[Concatenate[T, P], R],
@@ -246,7 +300,9 @@ def setup_required[T, **P, R](
 
 
 @overload
-def setup_required[**P, R](view: Callable[P, R]) -> Callable[P, R]: ...
+def setup_required[**P, R](view: Callable[P, R]) -> Callable[P, R]:
+    """Require self-hosted bootstrap setup before serving protected routes."""
+    ...
 
 
 def setup_required[R](view: Callable[..., R]) -> Callable[..., R]:
@@ -255,7 +311,7 @@ def setup_required[R](view: Callable[..., R]) -> Callable[..., R]:
         # The overloads keep Resource methods method-aware for pyrefly while
         # preserving support for plain functions used in tests and utilities.
         # check setup
-        if dify_config.EDITION == "SELF_HOSTED" and not db.session.scalar(select(DifySetup).limit(1)):
+        if dify_config.EDITION == "SELF_HOSTED" and not _is_setup_completed():
             if os.environ.get("INIT_PASSWORD"):
                 raise NotInitValidateError()
             raise NotSetupError()

api/controllers/files/agent_drive_archive.py

@@ -8,6 +8,7 @@ from werkzeug.exceptions import Forbidden, NotFound
 from controllers.common.file_response import enforce_download_for_html
 from controllers.common.schema import register_schema_models
 from controllers.files import files_ns
+from extensions.ext_database import db
 from models.agent import AgentDriveFileKind
 from services.agent_drive_service import AgentDriveError, AgentDriveService
 
@@ -54,6 +55,7 @@ class AgentDriveArchiveMemberApi(Resource):
                 archive_file_kind=args.archive_file_kind,
                 archive_file_id=args.archive_file_id,
                 member_path=args.member_path,
+                session=db.session(),
             )
         except AgentDriveError as exc:
             raise NotFound(exc.message) from exc

api/controllers/files/upload.py

@@ -1,5 +1,3 @@
-from mimetypes import guess_extension
-
 from flask import request
 from flask_restx import Resource
 from flask_restx.api import HTTPStatus
@@ -8,7 +6,7 @@ from werkzeug.exceptions import Forbidden
 
 import services
 from core.tools.signature import verify_plugin_file_signature
-from core.tools.tool_file_manager import ToolFileManager
+from core.tools.tool_file_manager import ToolFileManager, resolve_extension
 from core.workflow.file_reference import build_file_reference
 from fields.file_fields import FileResponse
 
@@ -110,7 +108,7 @@ class PluginUploadFileApi(Resource):
                 conversation_id=args.conversation_id,
             )
 
-            extension = guess_extension(tool_file.mimetype) or ".bin"
+            extension = resolve_extension(filename=tool_file.name, mimetype=tool_file.mimetype)
             preview_url = ToolFileManager.sign_file(tool_file_id=tool_file.id, extension=extension)
 
             # Create a dictionary with all the necessary attributes

api/controllers/inner_api/app/dsl.py

@@ -98,6 +98,7 @@ class EnterpriseAppDSLExport(Resource):
 
         data = AppDslService.export_dsl(
             app_model=app_model,
+            session=db.session(),
             include_secret=include_secret,
         )
 

api/controllers/inner_api/plugin/agent_drive.py

@@ -17,6 +17,7 @@ from controllers.console.wraps import setup_required
 from controllers.inner_api import inner_api_ns
 from controllers.inner_api.plugin.wraps import get_user
 from controllers.inner_api.wraps import plugin_inner_api_only
+from extensions.ext_database import db
 from services.agent_drive_service import (
     AgentDriveError,
     AgentDriveService,
@@ -53,6 +54,7 @@ class AgentDriveManifestApi(Resource):
                 agent_id=agent_id,
                 prefix=request.args.get("prefix", ""),
                 include_download_url=include_download_url,
+                session=db.session(),
             )
         except AgentDriveError as exc:
             return _error_response(exc)
@@ -71,7 +73,7 @@ class AgentDriveSkillsApi(Resource):
             tenant_id = (request.args.get("tenant_id") or "").strip()
             if not tenant_id:
                 raise AgentDriveError("missing_tenant_id", "tenant_id is required", status_code=400)
-            items = AgentDriveService().list_skills(tenant_id=tenant_id, agent_id=agent_id)
+            items = AgentDriveService().list_skills(tenant_id=tenant_id, agent_id=agent_id, session=db.session())
         except AgentDriveError as exc:
             return _error_response(exc)
         return {"items": items}
@@ -96,6 +98,7 @@ class AgentDriveCommitApi(Resource):
                 user_id=user.id,
                 agent_id=agent_id,
                 items=body.items,
+                session=db.session(),
             )
         except AgentDriveError as exc:
             return _error_response(exc)

api/controllers/inner_api/plugin/plugin.py

@@ -476,6 +476,7 @@ class PluginDownloadFileRequestApi(Resource):
             user_from=payload.user_from,
             invoke_from=payload.invoke_from,
             file_mapping=payload.file.model_dump(mode="python", exclude_none=True),
+            for_external=payload.for_external,
         )
         return BaseBackwardsInvocationResponse(
             data={

api/controllers/inner_api/workspace/workspace.py

@@ -47,8 +47,8 @@ class EnterpriseWorkspace(Resource):
         if account is None:
             return {"message": "owner account not found."}, 404
 
-        tenant = TenantService.create_tenant(args.name, is_from_dashboard=True, session=db.session)
-        TenantService.create_tenant_member(tenant, account, db.session, role="owner")
+        tenant = TenantService.create_tenant(args.name, is_from_dashboard=True, session=db.session())
+        TenantService.create_tenant_member(tenant, account, db.session(), role="owner")
 
         tenant_was_created.send(tenant)
 
@@ -84,7 +84,7 @@ class EnterpriseWorkspaceNoOwnerEmail(Resource):
     def post(self):
         args = WorkspaceOwnerlessPayload.model_validate(inner_api_ns.payload or {})
 
-        tenant = TenantService.create_tenant(args.name, is_from_dashboard=True, session=db.session)
+        tenant = TenantService.create_tenant(args.name, is_from_dashboard=True, session=db.session())
 
         tenant_was_created.send(tenant)
 

api/controllers/mcp/mcp.py

@@ -2,11 +2,11 @@ from typing import Any, Union
 
 from flask import Response, request
 from flask_restx import Resource
-from pydantic import BaseModel, Field, ValidationError
+from pydantic import BaseModel, Field, RootModel, ValidationError
 from sqlalchemy import select
 from sqlalchemy.orm import Session, sessionmaker
 
-from controllers.common.schema import register_schema_model
+from controllers.common.schema import register_response_schema_models, register_schema_model
 from controllers.mcp import mcp_ns
 from core.mcp import types as mcp_types
 from core.mcp.server.streamable_http import handle_mcp_request, negotiate_protocol_version
@@ -33,7 +33,12 @@ class MCPRequestPayload(BaseModel):
     id: int | str | None = Field(default=None, description="Request ID for tracking responses")
 
 
+class MCPJSONRPCResponse(RootModel[mcp_types.JSONRPCResponse | mcp_types.JSONRPCError]):
+    pass
+
+
 register_schema_model(mcp_ns, MCPRequestPayload)
+register_response_schema_models(mcp_ns, MCPJSONRPCResponse)
 
 
 @mcp_ns.route("/server/<string:server_code>/mcp")
@@ -42,13 +47,10 @@ class MCPAppApi(Resource):
     @mcp_ns.doc("handle_mcp_request")
     @mcp_ns.doc(description="Handle Model Context Protocol (MCP) requests for a specific server")
     @mcp_ns.doc(params={"server_code": "Unique identifier for the MCP server"})
-    @mcp_ns.doc(
-        responses={
-            200: "MCP response successfully processed",
-            400: "Invalid MCP request or parameters",
-            404: "Server or app not found",
-        }
-    )
+    @mcp_ns.response(200, "MCP JSON-RPC response", mcp_ns.models[MCPJSONRPCResponse.__name__])
+    @mcp_ns.response(202, "MCP notification accepted")
+    @mcp_ns.response(400, "Invalid MCP request or parameters")
+    @mcp_ns.response(404, "Server or app not found")
     def post(self, server_code: str):
         """Handle MCP requests for a specific server.
 
@@ -64,6 +66,7 @@ class MCPAppApi(Resource):
         Raises:
             ValidationError: Invalid request format or parameters
         """
+        # response-contract:ignore MCP route returns Flask Response from JSON-RPC handler
         args = MCPRequestPayload.model_validate(mcp_ns.payload or {})
         request_id: Union[int, str] | None = args.id
         mcp_request = self._parse_mcp_request(args.model_dump(exclude_none=True))

api/controllers/openapi/__init__.py

@@ -2,11 +2,13 @@ from flask import Blueprint
 from flask_restx import Namespace
 
 from controllers.openapi._errors import ErrorBody, OpenApiErrorCode, OpenApiErrorFormatter
+from controllers.openapi._version_gate import attach_version_gate
 from libs.device_flow_security import attach_anti_framing
 from libs.external_api import ExternalApi
 
 bp = Blueprint("openapi", __name__, url_prefix="/openapi/v1")
 attach_anti_framing(bp)
+attach_version_gate(bp)
 
 api = ExternalApi(
     bp,

api/controllers/openapi/_errors.py

@@ -45,6 +45,7 @@ class OpenApiErrorCode(StrEnum):
     TOO_MANY_REQUESTS = "too_many_requests"
     INTERNAL_ERROR = "internal_server_error"
     BAD_GATEWAY = "bad_gateway"
+    UPGRADE_REQUIRED = "upgrade_required"
     UNKNOWN = "unknown"
     # domain codes (must match the error_code attribute of the exception
     # classes raised on the openapi surface)

api/controllers/openapi/_models.py

@@ -279,7 +279,7 @@ def _csv_string_query_schema(schema: dict[str, Any]) -> None:
 
 
 class AppDescribeQuery(BaseModel):
-    """`?fields=` allow-list for GET /apps/<id>/describe.
+    """`?fields=` allow-list for GET /apps/<id>.
 
     Empty / omitted → all blocks. Unknown member → ValidationError → 422.
     """
@@ -441,7 +441,7 @@ class MemberActionResponse(BaseModel):
 
 
 class TaskStopResponse(BaseModel):
-    """200 body for POST /apps/<id>/tasks/<task_id>/stop. The handler always returns
+    """200 body for POST /apps/<id>/tasks/<task_id>:stop. The handler always returns
     {"result": "success"}, so `result` is required (no default) — the generated contract
     types it as a required `'success'` rather than an optional field."""
 
@@ -473,7 +473,7 @@ class AppDslImportPayload(BaseModel):
 
 
 class AppDslExportQuery(BaseModel):
-    """Query parameters for GET /apps/<app_id>/export."""
+    """Query parameters for GET /apps/<app_id>/dsl."""
 
     include_secret: bool = Field(False, description="Include encrypted secret values in the exported DSL")
     workflow_id: UUIDStr | None = Field(
@@ -488,7 +488,7 @@ class AppDslExportResponse(BaseModel):
 
 
 class FormSubmitResponse(BaseModel):
-    """Empty 200 body for POST /apps/<id>/form/human_input/<token>. `extra='forbid'`
+    """Empty 200 body for POST /apps/<id>/human-input-forms/<token>:submit. `extra='forbid'`
     pins `additionalProperties: false` so the generated contract is an exact `{}` rather
     than an under-annotated open object."""
 

api/controllers/openapi/_version_gate.py

@@ -0,0 +1,69 @@
+"""Version gate: reject outdated difyctl clients on /openapi/v1 with HTTP 426.
+
+difyctl and the ``/openapi/v1`` surface ship in lockstep. A breaking path change
+(resource-oriented paths) means an outdated difyctl would call removed paths and
+get a bare 404; this gate returns ``426 Upgrade Required`` with an upgrade hint
+instead.
+"""
+
+from __future__ import annotations
+
+import re
+from typing import Final
+
+from flask import Blueprint, Response, request
+from packaging.version import InvalidVersion, Version
+
+from configs import dify_config
+from controllers.openapi._errors import ErrorBody, OpenApiErrorCode
+
+_UPGRADE_HINT: Final = "Upgrade difyctl: https://docs.dify.ai/en/cli/install"
+
+# difyctl sends `User-Agent: difyctl/<semver> (<os>; <arch>; <channel>)`.
+_DIFYCTL_UA_RE = re.compile(r"^difyctl/(\d+\.\d+\.\d+(?:-[\w.]+)?)")
+
+_PREFIX: Final = "/openapi/v1/"
+
+# Paths a too-old client must still reach to discover that it is outdated.
+_ALLOWLIST: Final = frozenset({"/openapi/v1/_version", "/openapi/v1/_health"})
+
+
+def _upgrade_required_response(client_version: str, min_version: str) -> Response:
+    body = ErrorBody(
+        code=OpenApiErrorCode.UPGRADE_REQUIRED,
+        message=f"difyctl {client_version} is no longer supported; upgrade to >= {min_version}.",
+        status=426,
+        hint=_UPGRADE_HINT,
+    )
+    return Response(body.model_dump_json(exclude_none=True), status=426, mimetype="application/json")
+
+
+def attach_version_gate(bp: Blueprint) -> None:
+    """Reject difyctl clients older than ``[tool.dify] min_difyctl_version`` with 426.
+
+    Registered app-wide (``before_app_request``) rather than blueprint-scoped so it
+    also fires for requests to *removed* paths — those no longer match an openapi
+    route and would 404 before a blueprint-scoped ``before_request`` ever runs. The
+    prefix guard scopes it back to ``/openapi/v1``. Fails open for non-difyctl or
+    unparseable User-Agents (only a confidently-too-old difyctl is blocked).
+    """
+
+    @bp.before_app_request
+    def _enforce_min_client_version() -> Response | None:  # pyright: ignore[reportUnusedFunction]
+        if not request.path.startswith(_PREFIX):
+            return None
+        if request.path in _ALLOWLIST:
+            return None
+        match = _DIFYCTL_UA_RE.match(request.headers.get("User-Agent", ""))
+        if match is None:
+            return None
+        try:
+            client_version = Version(match.group(1))
+        except InvalidVersion:
+            return None
+        # Compare the numeric core (major.minor.patch) only — a pre-release build
+        # like 0.2.0-rc.1 must not sort below the 0.2.0 floor.
+        min_version = dify_config.tool.dify.min_difyctl_version
+        if client_version.release[:3] < Version(min_version).release[:3]:
+            return _upgrade_required_response(match.group(1), min_version)
+        return None

api/controllers/openapi/account.py

@@ -45,8 +45,10 @@ class AccountApi(Resource):
         enforce(LIMIT_ME_PER_ACCOUNT, key=f"account:{auth_data.account_id}")
 
         account_id_str = str(auth_data.account_id) if auth_data.account_id else None
-        account = AccountService.get_account_by_id(db.session, account_id_str) if account_id_str else None
-        memberships = TenantService.get_account_memberships(db.session, account_id_str) if account_id_str else []
+        account = AccountService.get_account_by_id(account_id_str, session=db.session()) if account_id_str else None
+        memberships = (
+            TenantService.get_account_memberships(account_id_str, session=db.session()) if account_id_str else []
+        )
         default_ws_id = _pick_default_workspace(memberships)
 
         return AccountResponse(
@@ -63,7 +65,7 @@ class AccountSessionsSelfApi(Resource):
     @auth_router.guard(scope=Scope.FULL, allowed_token_types=frozenset({TokenType.OAUTH_ACCOUNT}))
     @returns(200, RevokeResponse, description="Session revoked")
     def delete(self, *, auth_data: AuthData):
-        revoke_oauth_token(db.session, redis_client, str(auth_data.token_id))
+        revoke_oauth_token(redis_client, str(auth_data.token_id), session=db.session())
         return RevokeResponse(status="revoked")
 
 
@@ -81,7 +83,7 @@ class AccountSessionsApi(Resource):
         page = query.page
         limit = query.limit
 
-        all_rows = list_active_sessions(db.session, ctx, now)
+        all_rows = list_active_sessions(ctx, now, session=db.session())
 
         total = len(all_rows)
         sliced = all_rows[(page - 1) * limit : page * limit]
@@ -117,10 +119,10 @@ class AccountSessionByIdApi(Resource):
 
         # 404 (not 403) on cross-subject so the endpoint doesn't leak
         # token IDs that belong to other subjects.
-        if not token_belongs_to_subject(db.session, session_id, ctx):
+        if not token_belongs_to_subject(session_id, ctx, session=db.session()):
             raise NotFound("session not found")
 
-        revoke_oauth_token(db.session, redis_client, session_id)
+        revoke_oauth_token(redis_client, session_id, session=db.session())
         return RevokeResponse(status="revoked")