feat(openapi): distinguish expired OAuth bearer from invalid token

Previously an expired OAuth bearer and an unknown/invalid one both
surfaced as an indistinguishable generic 401 (and an invalid token
actually leaked a 500), so a client could not tell "session expired,
re-authenticate" apart from "never authenticated."

The resolver now raises a distinct TokenExpiredError for expired DB
rows and records a separate `expired` negative-cache marker, so a
retry within the negative-cache TTL still reports expiry instead of
collapsing into a generic miss. The auth pipeline maps the two domain
errors to unified OpenApiError responses: SessionExpired (code
`token_expired`) and InvalidBearer (code `unauthorized`), both 401.
This also fixes the latent 500 on invalid bearers.

The `token_expired` code is synced through the contract codegen into
the generated types/zod, and the difyctl error mapper branches the
401 on it. The CLI `expired_token` taxonomy member (RFC 8628
device-flow code expiry) is merged into `token_expired`; the RFC 8628
wire value is unchanged.

Closes WTA-1062

.agents/skills/e2e-cucumber-playwright/SKILL.md

@@ -5,7 +5,7 @@ description: Write, update, or review Dify end-to-end tests under `e2e/` that us
 
 # Dify E2E Cucumber + Playwright
 
-Use this skill for Dify's repository-level E2E suite in `e2e/`. Use [`e2e/AGENTS.md`](../../../e2e/AGENTS.md) as the canonical package guide for local architecture and conventions, then read any feature-scoped `AGENTS.md` that owns the target area. Apply Playwright/Cucumber best practices only where they fit the current suite.
+Use this skill for Dify's repository-level E2E suite in `e2e/`. Use [`e2e/AGENTS.md`](../../../e2e/AGENTS.md) as the canonical guide for local architecture and conventions, then apply Playwright/Cucumber best practices only where they fit the current suite.
 
 ## Scope
 
@@ -25,8 +25,6 @@ Use this skill for Dify's repository-level E2E suite in `e2e/`. Use [`e2e/AGENTS
 4. Read [`references/cucumber-best-practices.md`](references/cucumber-best-practices.md) only when scenario wording, step granularity, tags, or expression design are involved.
 5. Re-check official Playwright or Cucumber docs with the available documentation tools before introducing a new framework pattern.
 
-Keep this skill focused on Cucumber, Playwright, and package-level E2E guidance. Put feature-specific conventions in the owning feature's `AGENTS.md` instead of adding them here.
-
 ## Local Rules
 
 - `e2e/` uses Cucumber for scenarios and Playwright as the browser layer.
@@ -45,7 +43,6 @@ Keep this skill focused on Cucumber, Playwright, and package-level E2E guidance.
    - Inspect the target feature area.
    - Reuse an existing step when wording and behavior already match.
    - Add a new step only for a genuinely new user action or assertion.
-   - Before adding several similar steps, scan the target capability for an existing domain noun that can be parameterized without hiding behavior.
    - Keep edits close to the current capability folder unless the step is broadly reusable.
 2. Write behavior-first scenarios.
    - Describe user-observable behavior, not DOM mechanics.
@@ -54,19 +51,15 @@ Keep this skill focused on Cucumber, Playwright, and package-level E2E guidance.
 3. Write step definitions in the local style.
    - Keep one step to one user-visible action or one assertion.
    - Prefer Cucumber Expressions such as `{string}` and `{int}`.
-   - Use a bounded regex only when the accepted values are a small explicit domain set and Cucumber Expressions would make the Gherkin less natural.
-   - Do not create one-off steps for each case variant when the same domain action or outcome applies to named surfaces, modes, or resources.
    - Scope locators to stable containers when the page has repeated elements.
    - Avoid page-object layers or extra helper abstractions unless repeated complexity clearly justifies them.
 4. Use Playwright in the local style.
    - Prefer user-facing locators: `getByRole`, `getByLabel`, `getByPlaceholder`, `getByText`, then `getByTestId` for explicit contracts.
    - Use web-first `expect(...)` assertions.
    - Do not use `waitForTimeout`, manual polling, or raw visibility checks when a locator action or retrying assertion already expresses the behavior.
-   - Use `expect.poll` for API persistence, backend eventual consistency, captured browser events, or other non-DOM state; prefer locator assertions for DOM readiness and visible UI state.
-   - If a product element has real user-facing semantics but no accessible name, prefer fixing that accessible contract over adding a test id.
 5. Validate narrowly.
    - Run the narrowest tagged scenario or flow that exercises the change.
-   - Run `vpr lint --fix --quiet` from the repository root and `pnpm -C e2e type-check`.
+   - Run `pnpm -C e2e check`.
    - Broaden verification only when the change affects hooks, tags, setup, or shared step semantics.
 
 ## Review Checklist

.agents/skills/e2e-cucumber-playwright/references/cucumber-best-practices.md

@@ -39,7 +39,6 @@ Prefer reuse when:
 - the user action is genuinely the same
 - the expected outcome is genuinely the same
 - the wording stays natural across features
-- the parameter is a real product domain value such as a named surface, mode, resource, or status
 
 Write a new step when:
 
@@ -47,8 +46,6 @@ Write a new step when:
 - reusing the old wording would make the scenario misleading
 - a supposedly generic step would become an implementation-detail wrapper
 
-Do not optimize for a low step count by making vague steps. Optimize for a small set of truthful, domain-owned steps.
-
 ### 4. Prefer Cucumber Expressions
 
 Use Cucumber Expressions for parameters unless regex is clearly necessary.
@@ -62,8 +59,6 @@ Common examples:
 
 Keep expressions readable. If a step needs complicated parsing logic, first ask whether the scenario wording should be simpler.
 
-Use regex for a bounded natural-language alternative only when it keeps Gherkin readable, for example `/(Web app|Backend service API)/`. Avoid broad regexes that accept unowned language.
-
 ### 5. Keep step definitions thin and meaningful
 
 Step definitions are glue between Gherkin and automation, not a second abstraction language.

.agents/skills/e2e-cucumber-playwright/references/playwright-best-practices.md

@@ -41,7 +41,6 @@ Also remember:
 - repeated content usually needs scoping to a stable container
 - exact text matching is often too brittle when role/name or label already exists
 - `getByTestId` is acceptable when semantics are weak but the contract is intentional
-- when a real UI region, card, status, or icon lacks an accessible name, prefer adding that semantic contract in product code before falling back to `getByTestId`
 
 ### 3. Use web-first assertions
 
@@ -63,8 +62,6 @@ Avoid:
 
 If a condition genuinely needs custom retry logic, use Playwright's polling/assertion tools deliberately and keep that choice local and explicit.
 
-Use `expect.poll` for non-DOM truth such as API state, backend eventual consistency, generated resources, or captured browser events. For DOM state, use locator assertions so Playwright can apply actionability and web-first retry semantics.
-
 ### 4. Let actions wait for actionability
 
 Locator actions already wait for the element to be actionable. Do not preface every click/fill with extra timing logic unless the action needs a specific visible/ready assertion for clarity.

.agents/skills/frontend-code-review/references/dify-ui.md

@@ -40,13 +40,9 @@ Flag:
 - JavaScript conditional class logic for visual states that the Dify UI/Base UI primitive already exposes through `data-*` attributes or CSS variables.
 - Controlled props added when uncontrolled DOM state or CSS variables would be enough.
 - Thin wrappers that rename Base UI parts without adding semantics.
-- Generic Base UI selection primitives wrapped without preserving their value generics, such as `Select.Root<Value, Multiple>`, `RadioGroup<Value>`, or `Radio.Root<Value>`.
-- Shared select/radio option components that type selected values as `string` while callers pass enums, unions, booleans, numbers, objects, or nullable placeholder values.
 
 Prefer Base UI/Dify UI data attributes and CSS variables for visual state: `data-open`, `data-checked`, `data-disabled`, `data-highlighted`, `data-popup-open`, `group-data-*`, `peer-data-*`, `has-[:focus-visible]`, and primitive CSS variables such as anchor width or transform origin. Use JS conditional classes for product/business state that the primitive does not expose.
 
-For non-string `Select` and `RadioGroup` values, prefer explicit domain generics at the root and at child value carriers. JSX children do not inherit the parent generic, so `RadioGroup<PromptMode>` should compose with `Radio<PromptMode>`, `RadioRoot<PromptMode>`, or option values from a typed collection. For `Select`, prefer the Base UI `items` collection pattern for typed value-to-label rendering, and flag string coercion helpers used only to recover display labels.
-
 ## Forms
 
 Flag:

.agents/skills/how-to-write-component/SKILL.md

@@ -25,8 +25,6 @@ Use this as the component decision guide for Dify web. Existing code is referenc
 - Search before adding UI, hooks, helpers, query utilities, or styling patterns. Reuse existing base components, feature components, hooks, utilities, and design styles when they fit.
 - Follow Dify's CSS-first Tailwind v4 contract from `packages/dify-ui/README.md` and `packages/dify-ui/AGENTS.md`. Prefer design-system tokens, utilities, and radius mappings over generic Tailwind choices.
 - Group feature code by workflow, route, or ownership area with route-aligned names: components, hooks, local types, query helpers, atoms, constants, tests, and small utilities should live near the code that changes with them.
-- For each feature module, keep a module-local `README.md` as a boundary note. Start with the module name, a brief one-sentence description, then split dependencies into `Internal Modules` and `External Modules` sections; keep both sections and write `None.` when one category is empty. `Internal Modules` lists modules inside the same overall feature using paths from that feature root, such as `shared/domain/runtime-status`; `External Modules` lists project modules outside the feature using paths from the web root without a `web/` prefix, such as `app/components/base/skeleton`. Omit npm packages, workspace package dependencies, and whitelisted plumbing modules. Do not copy caller-relative import paths into the README.
-- Module README whitelist: `@/service/client`, `@/next/*`.
 - Keep source/default selection, validation, dirty checks, and payload shaping close to the workflow that owns submit behavior. Do not hide flow-specific priority order, fallback behavior, or submit semantics in generic utilities.
 - Prefer direct conditionals for small branch-specific decisions, especially form source selection and request payload assembly.
 - Loading states for page sections, cards, lists, tables, forms, and drawers should be skeletons scoped to the content being loaded. Use spinners only for small inline busy indicators.
@@ -69,7 +67,6 @@ Use this as the component decision guide for Dify web. Existing code is referenc
 - Avoid barrel files that only re-export secondary owners. `index.tsx` is acceptable for a route/tab entry component; import header controls, switches, sections, and row owners from their concrete owner files.
 - Type simple one-off props inline. Use a named `Props` type only when reused, exported, complex, or clearer.
 - Use API-generated or API-returned types at component boundaries. Keep small UI conversion helpers and one-off UI extensions beside the component that needs them.
-- Preserve domain value types for selection components. Do not widen enum, union, boolean, numeric, object, or nullable select/radio values to `string`; keep wrappers and option value carriers typed from their feature option collection.
 - Avoid `common.tsx` buckets for shared UI. Use a feature-local `components/` folder with concrete filenames that describe the shared role.
 - Do not create type aliases that only rename another type. Use aliases only for real UI concepts, refinements, or reusable local contracts.
 - Name values by their domain role and backend API contract, especially persistent IDs and route params. Normalize framework or route params at the boundary.

.github/CODEOWNERS

@@ -26,9 +26,6 @@
 /cli/ @GareArc
 /.github/workflows/cli-tests.yml @GareArc
 
-# E2E
-/e2e/ @lyzno1
-
 # Backend (default owner, more specific rules below will override)
 /api/ @QuantumGhost
 

.github/labeler.yml

@@ -7,9 +7,3 @@ web:
           - 'pnpm-lock.yaml'
           - 'pnpm-workspace.yaml'
           - '.nvmrc'
-
-e2e:
-  - changed-files:
-      - any-glob-to-any-file:
-          - 'e2e/**'
-          - '.github/workflows/web-e2e.yml'

.github/scripts/check-hotfix-cherry-picks.sh

@@ -45,7 +45,7 @@ while IFS= read -r commit_sha; do
   )
 
   if [[ -z "$source_sha" ]]; then
-    error "Commit $commit_sha ($subject) is missing cherry-pick provenance. $REMEDIATION_HINT If version differences prevent using git cherry-pick -x, manually add '(cherry picked from commit <sha>)' to the commit message."
+    error "Commit $commit_sha ($subject) is missing cherry-pick provenance. $REMEDIATION_HINT"
     failed=1
     continue
   fi

.github/workflows/api-tests.yml

@@ -29,13 +29,13 @@ jobs:
 
     steps:
       - name: Checkout code
-        uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
+        uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
         with:
           fetch-depth: 0
           persist-credentials: false
 
       - name: Setup UV and Python
-        uses: astral-sh/setup-uv@d31148d669074a8d0a63714ba94f3201e7020bc3 # v8.3.0
+        uses: astral-sh/setup-uv@fac544c07dec837d0ccb6301d7b5580bf5edae39 # v8.2.0
         with:
           enable-cache: true
           python-version: ${{ matrix.python-version }}
@@ -91,13 +91,13 @@ jobs:
 
     steps:
       - name: Checkout code
-        uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
+        uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
         with:
           fetch-depth: 0
           persist-credentials: false
 
       - name: Setup UV and Python
-        uses: astral-sh/setup-uv@d31148d669074a8d0a63714ba94f3201e7020bc3 # v8.3.0
+        uses: astral-sh/setup-uv@fac544c07dec837d0ccb6301d7b5580bf5edae39 # v8.2.0
         with:
           enable-cache: true
           python-version: ${{ matrix.python-version }}
@@ -142,13 +142,13 @@ jobs:
 
     steps:
       - name: Checkout code
-        uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
+        uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
         with:
           fetch-depth: 0
           persist-credentials: false
 
       - name: Setup UV and Python
-        uses: astral-sh/setup-uv@d31148d669074a8d0a63714ba94f3201e7020bc3 # v8.3.0
+        uses: astral-sh/setup-uv@fac544c07dec837d0ccb6301d7b5580bf5edae39 # v8.2.0
         with:
           enable-cache: true
           python-version: "3.12"

.github/workflows/autofix.yml

@@ -20,7 +20,7 @@ jobs:
         run: echo "autofix.ci updates pull request branches, not merge group refs."
 
       - if: github.event_name != 'merge_group'
-        uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
+        uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
 
       - name: Check Docker Compose inputs
         if: github.event_name != 'merge_group'
@@ -51,18 +51,6 @@ jobs:
         with:
           files: |
             api/**
-      - name: Check frontend contract inputs
-        if: github.event_name != 'merge_group'
-        id: frontend-contract-changes
-        uses: tj-actions/changed-files@9426d40962ed5378910ee2e21d5f8c6fcbf2dd96 # v47.0.6
-        with:
-          files: |
-            api/**
-            packages/contracts/openapi-ts.api.config.ts
-            packages/contracts/package.json
-            packages/contracts/openapi/**
-            pnpm-lock.yaml
-            pnpm-workspace.yaml
       - name: Check dify-agent inputs
         if: github.event_name != 'merge_group'
         id: dify-agent-changes
@@ -73,12 +61,12 @@ jobs:
             dify-agent/pyproject.toml
             dify-agent/uv.lock
       - if: github.event_name != 'merge_group'
-        uses: actions/setup-python@ece7cb06caefa5fff74198d8649806c4678c61a1 # v6.3.0
+        uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
         with:
           python-version: "3.11"
 
       - if: github.event_name != 'merge_group'
-        uses: astral-sh/setup-uv@d31148d669074a8d0a63714ba94f3201e7020bc3 # v8.3.0
+        uses: astral-sh/setup-uv@fac544c07dec837d0ccb6301d7b5580bf5edae39 # v8.2.0
 
       - name: Generate Docker Compose
         if: github.event_name != 'merge_group' && steps.docker-compose-changes.outputs.any_changed == 'true'
@@ -155,8 +143,8 @@ jobs:
           uv run dev/generate_swagger_markdown_docs.py --swagger-dir ../packages/contracts/openapi --markdown-dir openapi/markdown --keep-swagger-json
 
       - name: Generate frontend contracts
-        if: github.event_name != 'merge_group' && steps.frontend-contract-changes.outputs.any_changed == 'true'
-        run: pnpm --dir packages/contracts gen-api-contract
+        if: github.event_name != 'merge_group' && steps.api-changes.outputs.any_changed == 'true'
+        run: pnpm --dir packages/contracts gen-api-contract-from-openapi
 
       - name: ESLint autofix
         if: github.event_name != 'merge_group' && steps.web-changes.outputs.any_changed == 'true'

.github/workflows/build-push.yml

@@ -97,7 +97,7 @@ jobs:
           echo "PLATFORM_PAIR=${platform//\//-}" >> $GITHUB_ENV
 
       - name: Login to Docker Hub
-        uses: docker/login-action@af1e73f918a031802d376d3c8bbc3fe56130a9b0 # v4.4.0
+        uses: docker/login-action@650006c6eb7dba73a995cc03b0b2d7f5ca915bee # v4.2.0
         with:
           username: ${{ env.DOCKERHUB_USER }}
           password: ${{ env.DOCKERHUB_TOKEN }}
@@ -107,7 +107,7 @@ jobs:
 
       - name: Extract metadata for Docker
         id: meta
-        uses: docker/metadata-action@dc802804100637a589fabce1cb79ff13a1411302 # v6.2.0
+        uses: docker/metadata-action@80c7e94dd9b9319bd5eb7a0e0fe9291e23a2a2e9 # v6.1.0
         with:
           images: ${{ env[matrix.image_name_env] }}
 
@@ -159,10 +159,10 @@ jobs:
             file: "docker/local-sandbox/Dockerfile"
     steps:
       - name: Set up Docker Buildx
-        uses: docker/setup-buildx-action@bb05f3f5519dd87d3ba754cc423b652a5edd6d2c # v4.2.0
+        uses: docker/setup-buildx-action@d7f5e7f509e45cec5c76c4d5afdd7de93d0b3df5 # v4.1.0
 
       - name: Validate Docker image
-        uses: docker/build-push-action@53b7df96c91f9c12dcc8a07bcb9ccacbed38856a # v7.3.0
+        uses: docker/build-push-action@f9f3042f7e2789586610d6e8b85c8f03e5195baf # v7.2.0
         with:
           push: false
           context: ${{ matrix.build_context }}
@@ -197,14 +197,14 @@ jobs:
           merge-multiple: true
 
       - name: Login to Docker Hub
-        uses: docker/login-action@af1e73f918a031802d376d3c8bbc3fe56130a9b0 # v4.4.0
+        uses: docker/login-action@650006c6eb7dba73a995cc03b0b2d7f5ca915bee # v4.2.0
         with:
           username: ${{ env.DOCKERHUB_USER }}
           password: ${{ env.DOCKERHUB_TOKEN }}
 
       - name: Extract metadata for Docker
         id: meta
-        uses: docker/metadata-action@dc802804100637a589fabce1cb79ff13a1411302 # v6.2.0
+        uses: docker/metadata-action@80c7e94dd9b9319bd5eb7a0e0fe9291e23a2a2e9 # v6.1.0
         with:
           images: ${{ env[matrix.image_name_env] }}
           tags: |

.github/workflows/cli-e2e.yml

@@ -79,7 +79,7 @@ jobs:
       ws2_app_id:                 ${{ steps.out.outputs.DIFY_E2E_WS2_APP_ID }}
 
     steps:
-      - uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v4
+      - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v4
         with:
           ref: ${{ inputs.cli_ref || github.ref }}
           persist-credentials: false
@@ -123,7 +123,7 @@ jobs:
         shell: bash
 
     steps:
-      - uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v4
+      - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v4
         with:
           ref: ${{ inputs.cli_ref || github.ref }}
           persist-credentials: false
@@ -170,7 +170,7 @@ jobs:
         shell: bash
 
     steps:
-      - uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v4
+      - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v4
         with:
           ref: ${{ inputs.cli_ref || github.ref }}
           persist-credentials: false
@@ -233,7 +233,7 @@ jobs:
         shell: bash
 
     steps:
-      - uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v4
+      - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v4
         with:
           ref: ${{ inputs.cli_ref || github.ref }}
           persist-credentials: false
@@ -295,7 +295,7 @@ jobs:
         shell: bash
 
     steps:
-      - uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v4
+      - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v4
         with:
           ref: ${{ inputs.cli_ref || github.ref }}
           persist-credentials: false
@@ -351,7 +351,7 @@ jobs:
         shell: bash
 
     steps:
-      - uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v4
+      - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v4
         with:
           ref: ${{ inputs.cli_ref || github.ref }}
           persist-credentials: false

.github/workflows/cli-edge.yml

@@ -23,7 +23,7 @@ jobs:
         working-directory: ./cli
     steps:
       - name: Checkout
-        uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
+        uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
         with:
           persist-credentials: false
           fetch-depth: 0

.github/workflows/cli-release.yml

@@ -35,7 +35,7 @@ jobs:
       dify_tag: ${{ steps.resolve.outputs.dify_tag }}
     steps:
       - name: Checkout
-        uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
+        uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
         with:
           persist-credentials: false
 
@@ -98,7 +98,7 @@ jobs:
       DIFY_TAG: ${{ needs.validate.outputs.dify_tag }}
     steps:
       - name: Checkout
-        uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
+        uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
         with:
           persist-credentials: false
           fetch-depth: 1

.github/workflows/cli-smoke.yml

@@ -24,7 +24,7 @@ jobs:
         shell: bash
     steps:
       - name: Checkout cli ref
-        uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
+        uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
         with:
           ref: ${{ inputs.cli_ref || github.ref }}
           persist-credentials: false

.github/workflows/cli-tests.yml

@@ -30,7 +30,7 @@ jobs:
 
     steps:
       - name: Checkout code
-        uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
+        uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
         with:
           persist-credentials: false
 

.github/workflows/db-migration-test.yml

@@ -13,13 +13,13 @@ jobs:
 
     steps:
       - name: Checkout code
-        uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
+        uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
         with:
           fetch-depth: 0
           persist-credentials: false
 
       - name: Setup UV and Python
-        uses: astral-sh/setup-uv@d31148d669074a8d0a63714ba94f3201e7020bc3 # v8.3.0
+        uses: astral-sh/setup-uv@fac544c07dec837d0ccb6301d7b5580bf5edae39 # v8.2.0
         with:
           enable-cache: true
           python-version: "3.12"
@@ -63,13 +63,13 @@ jobs:
 
     steps:
       - name: Checkout code
-        uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
+        uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
         with:
           fetch-depth: 0
           persist-credentials: false
 
       - name: Setup UV and Python
-        uses: astral-sh/setup-uv@d31148d669074a8d0a63714ba94f3201e7020bc3 # v8.3.0
+        uses: astral-sh/setup-uv@fac544c07dec837d0ccb6301d7b5580bf5edae39 # v8.2.0
         with:
           enable-cache: true
           python-version: "3.12"

.github/workflows/docker-build.yml

@@ -77,10 +77,10 @@ jobs:
             file: "web/Dockerfile"
     steps:
       - name: Set up Docker Buildx
-        uses: docker/setup-buildx-action@bb05f3f5519dd87d3ba754cc423b652a5edd6d2c # v4.2.0
+        uses: docker/setup-buildx-action@d7f5e7f509e45cec5c76c4d5afdd7de93d0b3df5 # v4.1.0
 
       - name: Build Docker Image
-        uses: docker/build-push-action@53b7df96c91f9c12dcc8a07bcb9ccacbed38856a # v7.3.0
+        uses: docker/build-push-action@f9f3042f7e2789586610d6e8b85c8f03e5195baf # v7.2.0
         with:
           push: false
           context: ${{ matrix.context }}

.github/workflows/hotfix-cherry-pick.yml

@@ -24,7 +24,7 @@ jobs:
     name: Require cherry-pick provenance
     runs-on: depot-ubuntu-24.04
     steps:
-      - uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
+      - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
         with:
           fetch-depth: 0
 

.github/workflows/main-ci.yml

@@ -6,6 +6,8 @@ on:
   merge_group:
     branches: ["main"]
     types: [checks_requested]
+  push:
+    branches: ["main"]
 
 permissions:
   actions: write
@@ -46,8 +48,8 @@ jobs:
       vdb-changed: ${{ steps.changes.outputs.vdb }}
       migration-changed: ${{ steps.changes.outputs.migration }}
     steps:
-      - uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
-      - uses: dorny/paths-filter@7b450fff21473bca461d4b92ce414b9d0420d706 # v4.0.2
+      - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
+      - uses: dorny/paths-filter@fbd0ab8f3e69293af611ebaee6363fc25e6d187d # v4.0.1
         id: changes
         with:
           filters: |
@@ -322,7 +324,6 @@ jobs:
       - check-changes
     if: needs.pre_job.outputs.should_skip != 'true' && needs.check-changes.outputs.e2e-changed == 'true'
     uses: ./.github/workflows/web-e2e.yml
-    secrets: inherit
 
   web-e2e-skip:
     name: Skip Web Full-Stack E2E

.github/workflows/post-merge.yml

@@ -1,54 +0,0 @@
-name: Post-Merge Checks
-
-on:
-  push:
-    branches: ["main"]
-
-permissions:
-  contents: read
-
-concurrency:
-  group: post-merge-${{ github.sha }}
-  cancel-in-progress: false
-
-jobs:
-  check-changes:
-    name: Check Changed Files
-    runs-on: depot-ubuntu-24.04
-    outputs:
-      external-e2e-changed: ${{ steps.changes.outputs.external_e2e }}
-    steps:
-      - uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
-      - uses: dorny/paths-filter@7b450fff21473bca461d4b92ce414b9d0420d706 # v4.0.2
-        id: changes
-        with:
-          filters: |
-            external_e2e:
-              - 'e2e/features/agent-v2/**'
-              - 'e2e/scripts/**'
-              - 'e2e/support/**'
-              - '.github/workflows/post-merge.yml'
-              - '.github/workflows/web-e2e.yml'
-              - '.github/actions/setup-web/**'
-              - 'dify-agent/**'
-              - 'api/clients/agent_backend/**'
-              - 'api/core/app/apps/agent_app/**'
-              - 'api/core/workflow/nodes/agent_v2/**'
-              - 'api/controllers/console/agent/**'
-              - 'api/services/agent/**'
-              - 'api/core/plugin/**'
-              - 'api/services/plugin/**'
-              - 'api/core/tools/**'
-              - 'api/services/tools/**'
-              - 'web/features/agent-v2/**'
-              - 'web/app/(commonLayout)/roster/**'
-              - 'web/app/components/workflow/nodes/agent-v2/**'
-
-  external-e2e:
-    name: External Runtime E2E
-    needs: check-changes
-    if: needs.check-changes.outputs.external-e2e-changed == 'true'
-    uses: ./.github/workflows/web-e2e.yml
-    with:
-      run-external-runtime: true
-    secrets: inherit

.github/workflows/pyrefly-diff.yml

@@ -17,12 +17,12 @@ jobs:
       pull-requests: write
     steps:
       - name: Checkout PR branch
-        uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
+        uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
         with:
           fetch-depth: 0
 
       - name: Setup Python & UV
-        uses: astral-sh/setup-uv@d31148d669074a8d0a63714ba94f3201e7020bc3 # v8.3.0
+        uses: astral-sh/setup-uv@fac544c07dec837d0ccb6301d7b5580bf5edae39 # v8.2.0
         with:
           enable-cache: true
 

.github/workflows/pyrefly-type-coverage-comment.yml

@@ -21,10 +21,10 @@ jobs:
     if: ${{ github.event.workflow_run.conclusion == 'success' && github.event.workflow_run.pull_requests[0].head.repo.full_name != github.repository }}
     steps:
       - name: Checkout default branch (trusted code)
-        uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
+        uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
 
       - name: Setup Python & UV
-        uses: astral-sh/setup-uv@d31148d669074a8d0a63714ba94f3201e7020bc3 # v8.3.0
+        uses: astral-sh/setup-uv@fac544c07dec837d0ccb6301d7b5580bf5edae39 # v8.2.0
         with:
           enable-cache: true
 

.github/workflows/pyrefly-type-coverage.yml

@@ -17,12 +17,12 @@ jobs:
       pull-requests: write
     steps:
       - name: Checkout PR branch
-        uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
+        uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
         with:
           fetch-depth: 0
 
       - name: Setup Python & UV
-        uses: astral-sh/setup-uv@d31148d669074a8d0a63714ba94f3201e7020bc3 # v8.3.0
+        uses: astral-sh/setup-uv@fac544c07dec837d0ccb6301d7b5580bf5edae39 # v8.2.0
         with:
           enable-cache: true
 

.github/workflows/style.yml

@@ -19,10 +19,9 @@ jobs:
 
     steps:
       - name: Checkout code
-        uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
+        uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
         with:
           persist-credentials: false
-          fetch-depth: 0
 
       - name: Check changed files
         id: changed-files
@@ -30,13 +29,11 @@ jobs:
         with:
           files: |
             api/**
-            scripts/check_no_new_getattr.py
-            scripts/ast_grep_rules/no_new_getattr.yml
             .github/workflows/style.yml
 
       - name: Setup UV and Python
         if: steps.changed-files.outputs.any_changed == 'true'
-        uses: astral-sh/setup-uv@d31148d669074a8d0a63714ba94f3201e7020bc3 # v8.3.0
+        uses: astral-sh/setup-uv@fac544c07dec837d0ccb6301d7b5580bf5edae39 # v8.2.0
         with:
           enable-cache: false
           python-version: "3.12"
@@ -54,18 +51,6 @@ jobs:
         if: steps.changed-files.outputs.any_changed == 'true'
         run: uv run --project api --dev python api/dev/lint_response_contracts.py --fail-on-mismatch
 
-      - name: Fetch merge target ref for getattr guard
-        if: steps.changed-files.outputs.any_changed == 'true'
-        run: git fetch --no-tags --depth=1 origin +refs/heads/main:refs/remotes/origin/main
-
-      - name: Bind merge target branch for getattr guard
-        if: steps.changed-files.outputs.any_changed == 'true'
-        run: git show-ref --verify --quiet refs/heads/main || git branch main origin/main
-
-      - name: Run No New Getattr Guard
-        if: steps.changed-files.outputs.any_changed == 'true'
-        run: uv run --project api python scripts/check_no_new_getattr.py --mode ci --merge-target main
-
       - name: Run Type Checks
         if: steps.changed-files.outputs.any_changed == 'true'
         env:
@@ -88,7 +73,7 @@ jobs:
 
     steps:
       - name: Checkout code
-        uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
+        uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
         with:
           persist-credentials: false
 
@@ -139,7 +124,7 @@ jobs:
 
     steps:
       - name: Checkout code
-        uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
+        uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
         with:
           persist-credentials: false
 
@@ -168,7 +153,7 @@ jobs:
       - name: Restore ESLint cache
         if: steps.changed-files.outputs.any_changed == 'true'
         id: eslint-cache-restore
-        uses: actions/cache/restore@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
+        uses: actions/cache/restore@27d5ce7f107fe9357f9df03efb73ab90386fccae # v5.0.5
         with:
           path: .eslintcache
           key: ${{ runner.os }}-eslint-${{ hashFiles('pnpm-lock.yaml', 'eslint.config.mjs', 'web/eslint.config.mjs', 'web/eslint.constants.mjs', 'web/plugins/eslint/**') }}-${{ github.sha }}
@@ -185,7 +170,7 @@ jobs:
 
       - name: Save ESLint cache
         if: steps.changed-files.outputs.any_changed == 'true' && success() && steps.eslint-cache-restore.outputs.cache-hit != 'true'
-        uses: actions/cache/save@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
+        uses: actions/cache/save@27d5ce7f107fe9357f9df03efb73ab90386fccae # v5.0.5
         with:
           path: .eslintcache
           key: ${{ steps.eslint-cache-restore.outputs.cache-primary-key }}
@@ -196,7 +181,7 @@ jobs:
 
     steps:
       - name: Checkout code
-        uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
+        uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
         with:
           fetch-depth: 0
           persist-credentials: false
@@ -214,7 +199,7 @@ jobs:
             .editorconfig
 
       - name: Super-linter
-        uses: super-linter/super-linter/slim@4ce20838b8ab83717e78138c5b3a1407148e0918 # v8.7.0
+        uses: super-linter/super-linter/slim@9e863354e3ff62e0727d37183162c4a88873df41 # v8.6.0
         if: steps.changed-files.outputs.any_changed == 'true'
         env:
           BASH_SEVERITY: warning

.github/workflows/tool-test-sdks.yaml

@@ -24,7 +24,7 @@ jobs:
         working-directory: sdks/nodejs-client
 
     steps:
-      - uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
+      - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
         with:
           persist-credentials: false
 

.github/workflows/translate-i18n-claude.yml

@@ -40,7 +40,7 @@ jobs:
 
     steps:
       - name: Checkout repository
-        uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
+        uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
         with:
           fetch-depth: 0
           token: ${{ secrets.GITHUB_TOKEN }}
@@ -158,7 +158,7 @@ jobs:
 
       - name: Run Claude Code for Translation Sync
         if: steps.context.outputs.CHANGED_FILES != ''
-        uses: anthropics/claude-code-action@558b1d6cab4085c7753fe402c10bef0fbb92ac7a # v1.0.165
+        uses: anthropics/claude-code-action@806af32823ef69c8ef357086c573a902af641307 # v1.0.151
         with:
           anthropic_api_key: ${{ secrets.ANTHROPIC_API_KEY }}
           github_token: ${{ secrets.GITHUB_TOKEN }}

.github/workflows/trigger-i18n-sync.yml

@@ -21,7 +21,7 @@ jobs:
 
     steps:
       - name: Checkout repository
-        uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
+        uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
         with:
           fetch-depth: 0
 

.github/workflows/vdb-tests-full.yml

@@ -24,7 +24,7 @@ jobs:
 
     steps:
       - name: Checkout code
-        uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
+        uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
         with:
           persist-credentials: false
 
@@ -36,7 +36,7 @@ jobs:
           remove_tool_cache: true
 
       - name: Setup UV and Python
-        uses: astral-sh/setup-uv@d31148d669074a8d0a63714ba94f3201e7020bc3 # v8.3.0
+        uses: astral-sh/setup-uv@fac544c07dec837d0ccb6301d7b5580bf5edae39 # v8.2.0
         with:
           enable-cache: true
           python-version: ${{ matrix.python-version }}

.github/workflows/vdb-tests.yml

@@ -21,7 +21,7 @@ jobs:
 
     steps:
       - name: Checkout code
-        uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
+        uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
         with:
           persist-credentials: false
 
@@ -33,7 +33,7 @@ jobs:
           remove_tool_cache: true
 
       - name: Setup UV and Python
-        uses: astral-sh/setup-uv@d31148d669074a8d0a63714ba94f3201e7020bc3 # v8.3.0
+        uses: astral-sh/setup-uv@fac544c07dec837d0ccb6301d7b5580bf5edae39 # v8.2.0
         with:
           enable-cache: true
           python-version: ${{ matrix.python-version }}

.github/workflows/web-e2e.yml

@@ -2,11 +2,6 @@ name: Web Full-Stack E2E
 
 on:
   workflow_call:
-    inputs:
-      run-external-runtime:
-        required: false
-        type: boolean
-        default: false
 
 permissions:
   contents: read
@@ -25,7 +20,7 @@ jobs:
 
     steps:
       - name: Checkout code
-        uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
+        uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
         with:
           persist-credentials: false
 
@@ -33,13 +28,11 @@ jobs:
         uses: ./.github/actions/setup-web
 
       - name: Setup UV and Python
-        uses: astral-sh/setup-uv@d31148d669074a8d0a63714ba94f3201e7020bc3 # v8.3.0
+        uses: astral-sh/setup-uv@fac544c07dec837d0ccb6301d7b5580bf5edae39 # v8.2.0
         with:
           enable-cache: true
           python-version: "3.12"
-          cache-dependency-glob: |
-            api/uv.lock
-            dify-agent/uv.lock
+          cache-dependency-glob: api/uv.lock
 
       - name: Install API dependencies
         run: uv sync --project api --dev
@@ -58,52 +51,12 @@ jobs:
           E2E_INIT_PASSWORD: E2eInit12345
         run: vp run e2e:full
 
-      - name: Run external runtime E2E tests
-        if: ${{ inputs.run-external-runtime }}
-        working-directory: ./e2e
-        env:
-          E2E_ADMIN_EMAIL: e2e-admin@example.com
-          E2E_ADMIN_NAME: E2E Admin
-          E2E_ADMIN_PASSWORD: E2eAdmin12345
-          E2E_AGENT_DECISION_MODEL_NAME: ${{ vars.E2E_AGENT_DECISION_MODEL_NAME || 'gpt-5.5' }}
-          E2E_AGENT_DECISION_MODEL_PROVIDER: ${{ vars.E2E_AGENT_DECISION_MODEL_PROVIDER || 'openai' }}
-          E2E_AGENT_DECISION_MODEL_TYPE: ${{ vars.E2E_AGENT_DECISION_MODEL_TYPE || 'llm' }}
-          E2E_EXTERNAL_RUNTIME_SEED_SPECS: ${{ vars.E2E_EXTERNAL_RUNTIME_SEED_SPECS }}
-          E2E_EXTERNAL_RUNTIME_TAGS: ${{ vars.E2E_EXTERNAL_RUNTIME_TAGS }}
-          E2E_FORCE_WEB_BUILD: "1"
-          E2E_INIT_PASSWORD: E2eInit12345
-          E2E_MARKETPLACE_API_URL: ${{ vars.E2E_MARKETPLACE_API_URL }}
-          E2E_MARKETPLACE_PLUGIN_IDS: ${{ vars.E2E_MARKETPLACE_PLUGIN_IDS }}
-          E2E_MARKETPLACE_PLUGIN_UNIQUE_IDENTIFIERS: ${{ vars.E2E_MARKETPLACE_PLUGIN_UNIQUE_IDENTIFIERS }}
-          E2E_MODEL_PROVIDER_CREDENTIALS_JSON: ${{ secrets.E2E_MODEL_PROVIDER_CREDENTIALS_JSON }}
-          E2E_START_AGENT_BACKEND: ${{ vars.E2E_START_AGENT_BACKEND || '1' }}
-          E2E_STABLE_MODEL_NAME: ${{ vars.E2E_STABLE_MODEL_NAME || 'gpt-5-nano' }}
-          E2E_STABLE_MODEL_PROVIDER: ${{ vars.E2E_STABLE_MODEL_PROVIDER || 'openai' }}
-          E2E_STABLE_MODEL_TYPE: ${{ vars.E2E_STABLE_MODEL_TYPE || 'llm' }}
-        run: |
-          if [[ -z "${E2E_MODEL_PROVIDER_CREDENTIALS_JSON}" ]]; then
-            echo "E2E_MODEL_PROVIDER_CREDENTIALS_JSON is required for external runtime E2E." >&2
-            exit 1
-          fi
-
-          if [[ -d cucumber-report ]]; then
-            rm -rf cucumber-report-non-external
-            mv cucumber-report cucumber-report-non-external
-          fi
-
-          trap 'vp run e2e:middleware:down' EXIT
-          vp run e2e:middleware:up
-          vp run e2e:external:prepare
-          vp run e2e:external
-
       - name: Upload Cucumber report
         if: ${{ !cancelled() }}
         uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
         with:
           name: cucumber-report
-          path: |
-            e2e/cucumber-report
-            e2e/cucumber-report-non-external
+          path: e2e/cucumber-report
           retention-days: 7
 
       - name: Upload E2E logs

.github/workflows/web-tests.yml

@@ -31,7 +31,7 @@ jobs:
 
     steps:
       - name: Checkout code
-        uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
+        uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
         with:
           persist-credentials: false
 
@@ -64,7 +64,7 @@ jobs:
 
     steps:
       - name: Checkout code
-        uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
+        uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
         with:
           persist-credentials: false
 
@@ -102,7 +102,7 @@ jobs:
 
     steps:
       - name: Checkout code
-        uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
+        uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
         with:
           persist-credentials: false
 
@@ -134,7 +134,7 @@ jobs:
 
     steps:
       - name: Checkout code
-        uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
+        uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
         with:
           persist-credentials: false
 

.superpowers/sdd/hitl-timeout-semantics-impl-report.md

@@ -1,30 +0,0 @@
-# HITL timeout semantics implementation report
-
-## What changed
-
-- Updated `api/core/workflow/nodes/human_input/callback.py` so `DifyHITLCallback` now preserves Dify's timeout split at the boundary:
-  - `HumanInputFormStatus.TIMEOUT` returns the graphon timeout branch via `Expired(selected_handle="__timeout__", ...)`.
-  - `HumanInputFormStatus.EXPIRED` is treated as an invalid resume state and raises `AssertionError`.
-  - `HumanInputFormStatus.WAITING` with a past global deadline is treated as an invalid resume state and raises `AssertionError`.
-  - `HumanInputFormStatus.WAITING` with only the node-level deadline expired still returns the timeout branch.
-- Added `created_at` to `HumanInputFormEntity` and `_HumanInputFormEntityImpl` so the callback can compute the global deadline using Dify's shared `HUMAN_INPUT_GLOBAL_TIMEOUT_SECONDS` invariant.
-- Kept the submitted and pause flows unchanged.
-- Added focused unit coverage in `api/tests/unit_tests/core/workflow/test_human_input_callback.py` for:
-  - node timeout branch
-  - global expiration rejection
-  - waiting-form past node deadline timeout
-  - waiting-form past global deadline rejection
-
-## Verification
-
-- `uv run --project api pytest -o addopts='' api/tests/unit_tests/core/workflow/test_human_input_callback.py api/tests/unit_tests/core/workflow/nodes/human_input/test_human_input_form_filled_event.py -q`
-- `git diff --check`
-
-## Result
-
-- The focused test set is expected to pass with the new `created_at` boundary in place.
-- No unrelated files were modified.
-
-## Concerns
-
-- The callback now fails fast on invalid resume states by design. That is intentional, but any caller that previously relied on `EXPIRED` being mapped to the timeout branch will now see an assertion failure instead.

api/clients/agent_backend/__init__.py

@@ -32,8 +32,6 @@ from clients.agent_backend.factory import create_agent_backend_run_client
 from clients.agent_backend.fake_client import FakeAgentBackendRunClient, FakeAgentBackendScenario
 from clients.agent_backend.request_builder import (
     AGENT_SOUL_PROMPT_LAYER_ID,
-    DIFY_CONFIG_LAYER_ID,
-    DIFY_CORE_TOOLS_LAYER_ID,
     DIFY_EXECUTION_CONTEXT_LAYER_ID,
     DIFY_KNOWLEDGE_BASE_LAYER_ID,
     DIFY_PLUGIN_TOOLS_LAYER_ID,
@@ -49,8 +47,6 @@ from clients.agent_backend.request_builder import (
 
 __all__ = [
     "AGENT_SOUL_PROMPT_LAYER_ID",
-    "DIFY_CONFIG_LAYER_ID",
-    "DIFY_CORE_TOOLS_LAYER_ID",
     "DIFY_EXECUTION_CONTEXT_LAYER_ID",
     "DIFY_KNOWLEDGE_BASE_LAYER_ID",
     "DIFY_PLUGIN_TOOLS_LAYER_ID",

api/clients/agent_backend/event_adapter.py

@@ -67,7 +67,6 @@ class AgentBackendRunSucceededInternalEvent(AgentBackendInternalEventBase):
     type: Literal[AgentBackendInternalEventType.RUN_SUCCEEDED] = AgentBackendInternalEventType.RUN_SUCCEEDED
     output: JsonValue
     session_snapshot: CompositorSessionSnapshot
-    usage: dict[str, JsonValue] | None = None
 
 
 class AgentBackendDeferredToolCallInternalEvent(AgentBackendInternalEventBase):
@@ -77,7 +76,6 @@ class AgentBackendDeferredToolCallInternalEvent(AgentBackendInternalEventBase):
     deferred_tool_call: DeferredToolCallPayload
     message: str | None = None
     session_snapshot: CompositorSessionSnapshot
-    usage: dict[str, JsonValue] | None = None
 
 
 class AgentBackendRunFailedInternalEvent(AgentBackendInternalEventBase):
@@ -142,7 +140,6 @@ class AgentBackendRunEventAdapter:
                             deferred_tool_call=event.data.deferred_tool_call,
                             message=_deferred_tool_call_message(event.data.deferred_tool_call),
                             session_snapshot=event.data.session_snapshot,
-                            usage=_agent_run_usage(event.data.usage),
                         )
                     ]
                 return [
@@ -151,7 +148,6 @@ class AgentBackendRunEventAdapter:
                         source_event_id=event.id,
                         output=event.data.output,
                         session_snapshot=event.data.session_snapshot,
-                        usage=_agent_run_usage(event.data.usage),
                     )
                 ]
             case RunFailedEvent():
@@ -188,13 +184,3 @@ def _deferred_tool_call_message(payload: DeferredToolCallPayload) -> str:
             return title
 
     return f"Agent backend requested external input via deferred tool '{payload.tool_name}'."
-
-
-def _agent_run_usage(usage: object | None) -> dict[str, JsonValue] | None:
-    """Return JSON-safe usage metadata from optional Agent backend usage."""
-    if usage is None:
-        return None
-    dumped = _EVENT_DATA_ADAPTER.dump_python(usage, mode="json")
-    if not isinstance(dumped, dict):
-        return None
-    return cast(dict[str, JsonValue], dumped)

api/clients/agent_backend/request_builder.py

@@ -20,8 +20,6 @@ from agenton.layers import ExitIntent
 from agenton_collections.layers.plain import PLAIN_PROMPT_LAYER_TYPE_ID, PromptLayerConfig
 from agenton_collections.layers.pydantic_ai import PYDANTIC_AI_HISTORY_LAYER_TYPE_ID
 from dify_agent.layers.ask_human import DIFY_ASK_HUMAN_LAYER_TYPE_ID, DifyAskHumanLayerConfig
-from dify_agent.layers.config import DIFY_CONFIG_LAYER_TYPE_ID, DifyConfigLayerConfig
-from dify_agent.layers.dify_core_tools import DIFY_CORE_TOOLS_LAYER_TYPE_ID, DifyCoreToolsLayerConfig
 from dify_agent.layers.dify_plugin import (
     DIFY_PLUGIN_LLM_LAYER_TYPE_ID,
     DIFY_PLUGIN_TOOLS_LAYER_TYPE_ID,
@@ -56,10 +54,8 @@ WORKFLOW_NODE_JOB_PROMPT_LAYER_ID = "workflow_node_job_prompt"
 WORKFLOW_USER_PROMPT_LAYER_ID = "workflow_user_prompt"
 AGENT_APP_USER_PROMPT_LAYER_ID = "agent_app_user_prompt"
 DIFY_EXECUTION_CONTEXT_LAYER_ID = "execution_context"
-DIFY_CONFIG_LAYER_ID = "config"
 DIFY_DRIVE_LAYER_ID = "drive"
 DIFY_PLUGIN_TOOLS_LAYER_ID = "tools"
-DIFY_CORE_TOOLS_LAYER_ID = "core_tools"
 DIFY_KNOWLEDGE_BASE_LAYER_ID = "knowledge"
 DIFY_ASK_HUMAN_LAYER_ID = "ask_human"
 DIFY_SHELL_LAYER_ID = "shell"
@@ -82,26 +78,11 @@ def _filter_snapshot_to_specs(
     return CompositorSessionSnapshot(schema_version=snapshot.schema_version, layers=filtered_layers)
 
 
-def _shell_layer_deps() -> dict[str, str]:
-    return {"execution_context": DIFY_EXECUTION_CONTEXT_LAYER_ID}
-
-
-def _drive_layer_deps() -> dict[str, str]:
-    return {"shell": DIFY_SHELL_LAYER_ID}
-
-
-def _config_layer_deps() -> dict[str, str]:
-    return {"shell": DIFY_SHELL_LAYER_ID}
-
-
-def _shell_config_with_drive_ref(
-    shell_config: DifyShellLayerConfig | None,
-    drive_config: DifyDriveLayerConfig | None,
-) -> DifyShellLayerConfig:
-    config = shell_config or DifyShellLayerConfig()
-    if drive_config is None:
-        return config
-    return config.model_copy(update={"agent_stub_drive_ref": drive_config.drive_ref})
+def _shell_layer_deps(*, include_drive: bool) -> dict[str, str]:
+    deps = {"execution_context": DIFY_EXECUTION_CONTEXT_LAYER_ID}
+    if include_drive:
+        deps["drive"] = DIFY_DRIVE_LAYER_ID
+    return deps
 
 
 class AgentBackendModelConfig(BaseModel):
@@ -167,11 +148,9 @@ class AgentBackendWorkflowNodeRunInput(BaseModel):
     idempotency_key: str | None = None
     output: AgentBackendOutputConfig | None = None
     tools: DifyPluginToolsLayerConfig | None = None
-    core_tools: DifyCoreToolsLayerConfig | None = None
     knowledge: DifyKnowledgeBaseLayerConfig | None = None
-    config_layer_config: DifyConfigLayerConfig | None = None
     # Drive Skills & Files declaration (dify.drive) — an index the agent pulls
-    # through the back proxy, never inline content.
+    # through the back proxy, never inline content; see AGENT_DRIVE_MANIFEST_ENABLED.
     drive_config: DifyDriveLayerConfig | None = None
     # Human-in-the-loop ask_human deferred tool (dify.ask_human). Present only when
     # the Agent Soul configures human involvement; a deferred call ends the run and
@@ -216,11 +195,9 @@ class AgentBackendAgentAppRunInput(BaseModel):
     idempotency_key: str | None = None
     output: AgentBackendOutputConfig | None = None
     tools: DifyPluginToolsLayerConfig | None = None
-    core_tools: DifyCoreToolsLayerConfig | None = None
     knowledge: DifyKnowledgeBaseLayerConfig | None = None
-    config_layer_config: DifyConfigLayerConfig | None = None
     # Drive Skills & Files declaration (dify.drive) — an index the agent pulls
-    # through the back proxy, never inline content.
+    # through the back proxy, never inline content; see AGENT_DRIVE_MANIFEST_ENABLED.
     drive_config: DifyDriveLayerConfig | None = None
     # Human-in-the-loop ask_human deferred tool (dify.ask_human). Present only when
     # the Agent Soul configures human involvement (ENG-635).
@@ -254,11 +231,9 @@ class AgentBackendRunRequestBuilder:
         """Build an Agent App conversation-turn run request.
 
         Layer graph: optional Agent Soul system prompt → user prompt →
-        execution context → optional shell / config / drive / history
-        (multi-turn) → LLM → optional plugin-direct tools / core-routed tools /
-        knowledge search / ask_human / structured output. Mirrors the
-        workflow-node layer ordering minus the workflow-job / previous-node
-        prompt.
+        execution context → optional history (multi-turn) → LLM → optional
+        plugin tools / knowledge search → optional structured output. Mirrors the workflow-node
+        layer ordering minus the workflow-job / previous-node prompt.
         """
         layers: list[RunLayerSpec] = []
         if run_input.agent_soul_prompt:
@@ -288,42 +263,14 @@ class AgentBackendRunRequestBuilder:
             ]
         )
 
-        include_shell = (
-            run_input.include_shell or run_input.config_layer_config is not None or run_input.drive_config is not None
-        )
-        if include_shell:
-            # Sandboxed bash workspace (dify.shell). It enters before config/drive
-            # so eager pulls materialize content in the same filesystem used by
-            # model commands.
-            layers.append(
-                RunLayerSpec(
-                    name=DIFY_SHELL_LAYER_ID,
-                    type=DIFY_SHELL_LAYER_TYPE_ID,
-                    deps=_shell_layer_deps(),
-                    metadata=run_input.metadata,
-                    config=_shell_config_with_drive_ref(run_input.shell_config, run_input.drive_config),
-                )
-            )
-
-        if run_input.config_layer_config is not None:
-            layers.append(
-                RunLayerSpec(
-                    name=DIFY_CONFIG_LAYER_ID,
-                    type=DIFY_CONFIG_LAYER_TYPE_ID,
-                    deps=_config_layer_deps(),
-                    metadata=run_input.metadata,
-                    config=run_input.config_layer_config,
-                )
-            )
-
         if run_input.drive_config is not None:
-            # Drive Skills & Files declaration (dify.drive): the catalog plus
-            # prompt-mentioned entries eagerly pulled through the shell layer.
+            # Drive Skills & Files declaration (dify.drive): a config-only index;
+            # the agent pulls listed entries through the back proxy by drive_ref.
             layers.append(
                 RunLayerSpec(
                     name=DIFY_DRIVE_LAYER_ID,
                     type=DIFY_DRIVE_LAYER_TYPE_ID,
-                    deps=_drive_layer_deps(),
+                    deps={"execution_context": DIFY_EXECUTION_CONTEXT_LAYER_ID},
                     metadata=run_input.metadata,
                     config=run_input.drive_config,
                 )
@@ -355,31 +302,17 @@ class AgentBackendRunRequestBuilder:
         )
 
         if run_input.tools is not None and run_input.tools.tools:
-            plugin_tool_deps = {"execution_context": DIFY_EXECUTION_CONTEXT_LAYER_ID}
-            if include_shell:
-                plugin_tool_deps["shell"] = DIFY_SHELL_LAYER_ID
             layers.append(
                 RunLayerSpec(
                     name=DIFY_PLUGIN_TOOLS_LAYER_ID,
                     type=DIFY_PLUGIN_TOOLS_LAYER_TYPE_ID,
-                    deps=plugin_tool_deps,
+                    deps={"execution_context": DIFY_EXECUTION_CONTEXT_LAYER_ID},
                     metadata=run_input.metadata,
                     config=run_input.tools,
                 )
             )
 
-        if run_input.core_tools is not None and run_input.core_tools.tools:
-            layers.append(
-                RunLayerSpec(
-                    name=DIFY_CORE_TOOLS_LAYER_ID,
-                    type=DIFY_CORE_TOOLS_LAYER_TYPE_ID,
-                    deps={"execution_context": DIFY_EXECUTION_CONTEXT_LAYER_ID},
-                    metadata=run_input.metadata,
-                    config=run_input.core_tools,
-                )
-            )
-
-        if run_input.knowledge is not None and run_input.knowledge.sets:
+        if run_input.knowledge is not None and run_input.knowledge.dataset_ids:
             layers.append(
                 RunLayerSpec(
                     name=DIFY_KNOWLEDGE_BASE_LAYER_ID,
@@ -403,6 +336,21 @@ class AgentBackendRunRequestBuilder:
                 )
             )
 
+        if run_input.include_shell:
+            # Sandboxed bash workspace (dify.shell). Depends on execution_context
+            # so the agent server can mint per-command Agent Stub env, and on
+            # drive when present so that env points at /mnt/drive/<drive_ref>.
+            # shellctl connection itself is server-injected.
+            layers.append(
+                RunLayerSpec(
+                    name=DIFY_SHELL_LAYER_ID,
+                    type=DIFY_SHELL_LAYER_TYPE_ID,
+                    deps=_shell_layer_deps(include_drive=run_input.drive_config is not None),
+                    metadata=run_input.metadata,
+                    config=run_input.shell_config or DifyShellLayerConfig(),
+                )
+            )
+
         if run_input.output is not None:
             layers.append(
                 RunLayerSpec(
@@ -444,8 +392,7 @@ class AgentBackendRunRequestBuilder:
         non-plugin layer graph that produced the snapshot. Plugin layers
         (``dify.plugin.llm``, ``dify.plugin.tools``) are excluded from both the
         composition and the snapshot before submission because their configs
-        may carry credentials or runtime-only declarations that are not
-        persisted between runs.
+        require credentials that are not persisted between runs.
         """
         if not runtime_layer_specs:
             raise ValueError(
@@ -478,9 +425,8 @@ class AgentBackendRunRequestBuilder:
         """Build a workflow Agent Node run request without defining another wire schema.
 
         Layer graph mirrors the workflow surface: prompts → execution context →
-        optional shell / config / drive / history → LLM → optional
-        plugin-direct tools / core-routed tools / knowledge search /
-        ask_human / structured output.
+        optional drive/history → LLM → optional plugin tools / knowledge search
+        → optional auxiliary layers such as ask_human, shell, and structured output.
         """
         layers: list[RunLayerSpec] = []
         if run_input.agent_soul_prompt:
@@ -499,7 +445,7 @@ class AgentBackendRunRequestBuilder:
                     name=WORKFLOW_NODE_JOB_PROMPT_LAYER_ID,
                     type=PLAIN_PROMPT_LAYER_TYPE_ID,
                     metadata={**run_input.metadata, "origin": "workflow_node_job"},
-                    config=PromptLayerConfig(user=run_input.workflow_node_job_prompt),
+                    config=PromptLayerConfig(prefix=run_input.workflow_node_job_prompt),
                 ),
                 RunLayerSpec(
                     name=WORKFLOW_USER_PROMPT_LAYER_ID,
@@ -516,42 +462,14 @@ class AgentBackendRunRequestBuilder:
             ]
         )
 
-        include_shell = (
-            run_input.include_shell or run_input.config_layer_config is not None or run_input.drive_config is not None
-        )
-        if include_shell:
-            # Sandboxed bash workspace (dify.shell). It enters before drive so
-            # drive can materialize mentioned targets with `dify-agent drive pull`
-            # in the same shell-visible filesystem used by model commands.
-            layers.append(
-                RunLayerSpec(
-                    name=DIFY_SHELL_LAYER_ID,
-                    type=DIFY_SHELL_LAYER_TYPE_ID,
-                    deps=_shell_layer_deps(),
-                    metadata=run_input.metadata,
-                    config=_shell_config_with_drive_ref(run_input.shell_config, run_input.drive_config),
-                )
-            )
-
-        if run_input.config_layer_config is not None:
-            layers.append(
-                RunLayerSpec(
-                    name=DIFY_CONFIG_LAYER_ID,
-                    type=DIFY_CONFIG_LAYER_TYPE_ID,
-                    deps=_config_layer_deps(),
-                    metadata=run_input.metadata,
-                    config=run_input.config_layer_config,
-                )
-            )
-
         if run_input.drive_config is not None:
-            # Drive Skills & Files declaration (dify.drive): the catalog plus
-            # prompt-mentioned entries eagerly pulled through the shell layer.
+            # Drive Skills & Files declaration (dify.drive): a config-only index;
+            # the agent pulls listed entries through the back proxy by drive_ref.
             layers.append(
                 RunLayerSpec(
                     name=DIFY_DRIVE_LAYER_ID,
                     type=DIFY_DRIVE_LAYER_TYPE_ID,
-                    deps=_drive_layer_deps(),
+                    deps={"execution_context": DIFY_EXECUTION_CONTEXT_LAYER_ID},
                     metadata=run_input.metadata,
                     config=run_input.drive_config,
                 )
@@ -585,31 +503,17 @@ class AgentBackendRunRequestBuilder:
         )
 
         if run_input.tools is not None and run_input.tools.tools:
-            plugin_tool_deps = {"execution_context": DIFY_EXECUTION_CONTEXT_LAYER_ID}
-            if include_shell:
-                plugin_tool_deps["shell"] = DIFY_SHELL_LAYER_ID
             layers.append(
                 RunLayerSpec(
                     name=DIFY_PLUGIN_TOOLS_LAYER_ID,
                     type=DIFY_PLUGIN_TOOLS_LAYER_TYPE_ID,
-                    deps=plugin_tool_deps,
+                    deps={"execution_context": DIFY_EXECUTION_CONTEXT_LAYER_ID},
                     metadata=run_input.metadata,
                     config=run_input.tools,
                 )
             )
 
-        if run_input.core_tools is not None and run_input.core_tools.tools:
-            layers.append(
-                RunLayerSpec(
-                    name=DIFY_CORE_TOOLS_LAYER_ID,
-                    type=DIFY_CORE_TOOLS_LAYER_TYPE_ID,
-                    deps={"execution_context": DIFY_EXECUTION_CONTEXT_LAYER_ID},
-                    metadata=run_input.metadata,
-                    config=run_input.core_tools,
-                )
-            )
-
-        if run_input.knowledge is not None and run_input.knowledge.sets:
+        if run_input.knowledge is not None and run_input.knowledge.dataset_ids:
             layers.append(
                 RunLayerSpec(
                     name=DIFY_KNOWLEDGE_BASE_LAYER_ID,
@@ -633,6 +537,21 @@ class AgentBackendRunRequestBuilder:
                 )
             )
 
+        if run_input.include_shell:
+            # Sandboxed bash workspace (dify.shell). Depends on execution_context
+            # so the agent server can mint per-command Agent Stub env, and on
+            # drive when present so that env points at /mnt/drive/<drive_ref>.
+            # shellctl connection itself is server-injected.
+            layers.append(
+                RunLayerSpec(
+                    name=DIFY_SHELL_LAYER_ID,
+                    type=DIFY_SHELL_LAYER_TYPE_ID,
+                    deps=_shell_layer_deps(include_drive=run_input.drive_config is not None),
+                    metadata=run_input.metadata,
+                    config=run_input.shell_config or DifyShellLayerConfig(),
+                )
+            )
+
         if run_input.output is not None:
             layers.append(
                 RunLayerSpec(

api/commands/__init__.py

@@ -22,7 +22,7 @@ from .plugin import (
     setup_system_trigger_oauth_client,
     transform_datasource_credentials,
 )
-from .rbac import migrate_dataset_permissions_to_rbac, migrate_member_roles_to_rbac
+from .rbac import migrate_member_roles_to_rbac
 from .retention import (
     archive_workflow_runs,
     archive_workflow_runs_plan,
@@ -76,7 +76,6 @@ __all__ = [
     "legacy_model_types",
     "migrate_annotation_vector_database",
     "migrate_data_for_plugin",
-    "migrate_dataset_permissions_to_rbac",
     "migrate_knowledge_vector_database",
     "migrate_member_roles_to_rbac",
     "migrate_oss",

api/commands/account.py

@@ -25,7 +25,7 @@ def reset_password(email, new_password, password_confirm):
         return
     normalized_email = email.strip().lower()
 
-    account = AccountService.get_account_by_email_with_case_fallback(db.session(), email.strip())
+    account = AccountService.get_account_by_email_with_case_fallback(db.session, email.strip())
 
     if not account:
         click.echo(click.style(f"Account not found for email: {email}", fg="red"))
@@ -67,7 +67,7 @@ def reset_email(email, new_email, email_confirm):
         return
     normalized_new_email = new_email.strip().lower()
 
-    account = AccountService.get_account_by_email_with_case_fallback(db.session(), email.strip())
+    account = AccountService.get_account_by_email_with_case_fallback(db.session, email.strip())
 
     if not account:
         click.echo(click.style(f"Account not found for email: {email}", fg="red"))
@@ -133,9 +133,9 @@ def create_tenant(email: str, language: str | None = None, name: str | None = No
         password=new_password,
         language=language,
         create_workspace_required=False,
-        session=db.session(),
+        session=db.session,
     )
-    TenantService.create_owner_tenant_if_not_exist(account, name, session=db.session())
+    TenantService.create_owner_tenant_if_not_exist(account, name, session=db.session)
 
     click.echo(
         click.style(

api/commands/data_migrate.py

@@ -7,7 +7,6 @@ from typing import cast
 
 import click
 
-from commands.rbac import migrate_dataset_permissions_to_rbac
 from extensions.ext_database import db
 from graphon.model_runtime.entities.model_entities import ModelType
 from services.legacy_model_type_migration import (
@@ -178,4 +177,3 @@ def legacy_model_types(
 
 
 data_migrate.add_command(legacy_model_types)
-data_migrate.add_command(migrate_dataset_permissions_to_rbac)

api/commands/data_migration.py

@@ -9,9 +9,7 @@ from uuid import UUID
 import click
 import sqlalchemy as sa
 import yaml
-from sqlalchemy.orm import Session
 
-from core.db.session_factory import session_factory
 from extensions.ext_database import db
 from models import Tenant
 from models.model import App
@@ -108,8 +106,7 @@ def export_migration_data(input_file: str | None, output_file: str | None, overw
         assert output_file is not None
         raw_config = _load_json_object(input_file, "Export config")
         selection = ExportConfigParser().parse(raw_config)
-        with session_factory.create_session() as session:
-            result = MigrationExportService().export(selection, session=session)
+        result = MigrationExportService().export(selection)
         MigrationPackageService().save_package(result.package, output_file, overwrite=overwrite)
         click.echo(click.style(f"Output written to {output_file}", fg="green"))
         _render_report(result.report_items, context=_with_output_path(result.report_context, output_file))
@@ -156,21 +153,19 @@ def import_migration_data(
         _require_options(("--input", input_file))
         assert input_file is not None
         package = MigrationPackageService().load_package(input_file)
-        with session_factory.create_session() as session:
-            result = MigrationImportService().import_package(
-                ImportRequest(
-                    package=package,
-                    cli_target_tenant=target_tenant,
-                    operator_email=operator_email,
-                    options_override=_build_options_override(
-                        package.metadata.import_options,
-                        id_strategy=id_strategy,
-                        conflict_strategy=conflict_strategy,
-                        create_app_api_token_on_import=create_app_api_token_on_import,
-                    ),
+        result = MigrationImportService().import_package(
+            ImportRequest(
+                package=package,
+                cli_target_tenant=target_tenant,
+                operator_email=operator_email,
+                options_override=_build_options_override(
+                    package.metadata.import_options,
+                    id_strategy=id_strategy,
+                    conflict_strategy=conflict_strategy,
+                    create_app_api_token_on_import=create_app_api_token_on_import,
                 ),
-                session=session,
             )
+        )
         _render_report(result.report_items, context=result.report_context)
     except MigrationDataError as exc:
         raise click.ClickException(str(exc)) from exc
@@ -218,9 +213,7 @@ def migration_data_wizard() -> None:
             default=True,
             show_default=False,
         )
-        auto_tools = _discover_auto_tools(
-            [app for app in apps if app.id in set(app_ids)], include_referenced_tools, session=db.session()
-        )
+        auto_tools = _discover_auto_tools([app for app in apps if app.id in set(app_ids)], include_referenced_tools)
         auto_tools = _resolve_auto_tool_names(tenant.id, auto_tools)
         _print_auto_tools(auto_tools)
         additional_tools = _prompt_additional_tools(tenant.id, auto_tools)
@@ -255,8 +248,7 @@ def migration_data_wizard() -> None:
             conflict_strategy=conflict_strategy,
             output_file=output_file,
         )
-        with session_factory.create_session() as session:
-            result = MigrationExportService().export(selection, session=session)
+        result = MigrationExportService().export(selection)
         MigrationPackageService().save_package(result.package, output_file, overwrite=overwrite)
         click.echo(click.style(f"Output written to {output_file}", fg="green"))
         _print_wizard_step("Report")
@@ -397,13 +389,13 @@ def _prompt_import_options() -> tuple[bool, bool, str, str]:
     return include_secrets, create_tokens, id_strategy, conflict_strategy
 
 
-def _discover_auto_tools(apps: list[App], include_referenced_tools: bool, *, session: Session) -> WizardToolMap:
+def _discover_auto_tools(apps: list[App], include_referenced_tools: bool) -> WizardToolMap:
     auto_tools: WizardToolMap = {"api_tools": {}, "workflow_tools": {}, "mcp_tools": {}}
     if not include_referenced_tools:
         return auto_tools
     discovery_service = DependencyDiscoveryService()
     for app in apps:
-        dsl_content = AppDslService.export_dsl(app_model=app, session=session, include_secret=False)
+        dsl_content = AppDslService.export_dsl(app_model=app, include_secret=False)
         raw_dsl = yaml.safe_load(dsl_content) if dsl_content else {}
         dsl = raw_dsl if isinstance(raw_dsl, dict) else {}
         for dependency in discovery_service.discover_from_dsl(dsl):

api/commands/plugin.py

@@ -16,7 +16,7 @@ from core.plugin.plugin_service import PluginService
 from core.tools.utils.system_encryption import encrypt_system_params
 from extensions.ext_database import db
 from models import Tenant
-from models.account import TenantPluginAutoUpgradeCategory, TenantPluginAutoUpgradeStrategy
+from models.account import TenantPluginAutoUpgradeStrategy
 from models.oauth import DatasourceOauthParamConfig, DatasourceProvider
 from models.provider_ids import DatasourceProviderID, ToolProviderID
 from models.source import DataSourceApiKeyAuthBinding, DataSourceOauthBinding
@@ -188,13 +188,13 @@ def transform_datasource_credentials(environment: str):
         firecrawl_plugin_id = "langgenius/firecrawl_datasource"
         jina_plugin_id = "langgenius/jina_datasource"
         if environment == "online":
-            notion_package_identifier = plugin_migration._fetch_latest_package_identifier(notion_plugin_id)
-            firecrawl_package_identifier = plugin_migration._fetch_latest_package_identifier(firecrawl_plugin_id)
-            jina_package_identifier = plugin_migration._fetch_latest_package_identifier(jina_plugin_id)
+            notion_plugin_unique_identifier = plugin_migration._fetch_plugin_unique_identifier(notion_plugin_id)
+            firecrawl_plugin_unique_identifier = plugin_migration._fetch_plugin_unique_identifier(firecrawl_plugin_id)
+            jina_plugin_unique_identifier = plugin_migration._fetch_plugin_unique_identifier(jina_plugin_id)
         else:
-            notion_package_identifier = None
-            firecrawl_package_identifier = None
-            jina_package_identifier = None
+            notion_plugin_unique_identifier = None
+            firecrawl_plugin_unique_identifier = None
+            jina_plugin_unique_identifier = None
         oauth_credential_type = CredentialType.OAUTH2
         api_key_credential_type = CredentialType.API_KEY
 
@@ -219,9 +219,9 @@ def transform_datasource_credentials(environment: str):
                     installed_plugins = installer_manager.list_plugins(tenant_id)
                     installed_plugins_ids = [plugin.plugin_id for plugin in installed_plugins]
                     if notion_plugin_id not in installed_plugins_ids:
-                        if notion_package_identifier:
+                        if notion_plugin_unique_identifier:
                             # install notion plugin
-                            PluginService.install_from_marketplace_pkg(tenant_id, [notion_package_identifier])
+                            PluginService.install_from_marketplace_pkg(tenant_id, [notion_plugin_unique_identifier])
                     auth_count = 0
                     for notion_tenant_credential in notion_tenant_credentials:
                         auth_count += 1
@@ -279,9 +279,9 @@ def transform_datasource_credentials(environment: str):
                     installed_plugins = installer_manager.list_plugins(tenant_id)
                     installed_plugins_ids = [plugin.plugin_id for plugin in installed_plugins]
                     if firecrawl_plugin_id not in installed_plugins_ids:
-                        if firecrawl_package_identifier:
+                        if firecrawl_plugin_unique_identifier:
                             # install firecrawl plugin
-                            PluginService.install_from_marketplace_pkg(tenant_id, [firecrawl_package_identifier])
+                            PluginService.install_from_marketplace_pkg(tenant_id, [firecrawl_plugin_unique_identifier])
 
                     auth_count = 0
                     for firecrawl_tenant_credential in firecrawl_tenant_credentials:
@@ -343,10 +343,10 @@ def transform_datasource_credentials(environment: str):
                     installed_plugins = installer_manager.list_plugins(tenant_id)
                     installed_plugins_ids = [plugin.plugin_id for plugin in installed_plugins]
                     if jina_plugin_id not in installed_plugins_ids:
-                        if jina_package_identifier:
+                        if jina_plugin_unique_identifier:
                             # install jina plugin
-                            logger.debug("Installing Jina plugin %s", jina_package_identifier)
-                            PluginService.install_from_marketplace_pkg(tenant_id, [jina_package_identifier])
+                            logger.debug("Installing Jina plugin %s", jina_plugin_unique_identifier)
+                            PluginService.install_from_marketplace_pkg(tenant_id, [jina_plugin_unique_identifier])
 
                     auth_count = 0
                     for jina_tenant_credential in jina_tenant_credentials:
@@ -406,7 +406,7 @@ def migrate_data_for_plugin():
 
 
 def _candidate_auto_upgrade_strategy_tenant_ids_stmt(limit: int | None = None):
-    category_count = len(TenantPluginAutoUpgradeCategory)
+    category_count = len(TenantPluginAutoUpgradeStrategy.PluginCategory)
     stmt = (
         select(TenantPluginAutoUpgradeStrategy.tenant_id)
         .group_by(TenantPluginAutoUpgradeStrategy.tenant_id)
@@ -472,7 +472,6 @@ def backfill_plugin_auto_upgrade(
         try:
             result = PluginAutoUpgradeService.backfill_strategy_categories(
                 current_tenant_id,
-                session=db.session(),
             )
         except Exception as e:
             failed_count += 1

api/commands/rbac.py

@@ -1,56 +1,11 @@
 from __future__ import annotations
 
-import json
-from collections.abc import Iterator
-from concurrent.futures import ThreadPoolExecutor, as_completed
-
 import click
 from sqlalchemy import select
-from sqlalchemy.orm import Session
 
-from configs import dify_config
 from core.db.session_factory import session_factory
-from core.rbac import RBACResourceWhitelistScope
-from models import Dataset, DatasetPermission, DatasetPermissionEnum, TenantAccountJoin, TenantAccountRole
-from services.enterprise.rbac_service import ListOption, RBACService, ReplaceMemberBindings, ReplaceUserAccessPolicies
-
-_RBAC_DEFAULT_ACCESS_POLICY_ID = "default"
-
-_LEGACY_ROLE_TO_BUILTIN_TAG = {
-    TenantAccountRole.OWNER.value: "owner",
-    TenantAccountRole.ADMIN.value: "admin",
-    TenantAccountRole.EDITOR.value: "editor",
-    TenantAccountRole.NORMAL.value: "normal",
-    TenantAccountRole.DATASET_OPERATOR.value: "dataset_operator",
-}
-
-
-def _resolve_builtin_role_ids(tenant_id: str, operator_account_id: str) -> dict[str, str]:
-    """Resolve every legacy workspace role to the current tenant's builtin RBAC role id.
-
-    The migration replays the old `TenantAccountJoin.role` values onto the
-    RBAC member-role binding API. Builtin RBAC roles are tenant-scoped and
-    identified by runtime ids, so the command must look them up per tenant.
-    """
-    roles = RBACService.Roles.list(
-        tenant_id=tenant_id,
-        account_id=operator_account_id,
-        options=ListOption(page_number=1, results_per_page=100),
-    ).data
-    role_id_by_tag = {
-        role.role_tag: role.id
-        for role in roles
-        if role.is_builtin and role.category == "global_system_default" and role.role_tag
-    }
-    resolved: dict[str, str] = {}
-    for legacy_role, expected_builtin_tag in _LEGACY_ROLE_TO_BUILTIN_TAG.items():
-        role_id = role_id_by_tag.get(expected_builtin_tag)
-        if expected_builtin_tag == "dataset_operator" and not dify_config.DATASET_OPERATOR_ENABLED:
-            continue
-        if not role_id:
-            raise ValueError(f"Builtin RBAC role not found for tenant={tenant_id}, legacy_role={legacy_role}")
-        resolved[legacy_role] = role_id
-    return resolved
+from models import TenantAccountJoin, TenantAccountRole
+from services.enterprise.rbac_service import ListOption, RBACService
 
 
 def _resolve_builtin_role_id(tenant_id: str, operator_account_id: str, legacy_role: str) -> str:
@@ -60,105 +15,26 @@ def _resolve_builtin_role_id(tenant_id: str, operator_account_id: str, legacy_ro
     RBAC member-role binding API. Builtin RBAC roles are tenant-scoped and
     identified by runtime ids, so the command must look them up per tenant.
     """
-    if legacy_role not in _LEGACY_ROLE_TO_BUILTIN_TAG:
+    expected_builtin_tag = {
+        TenantAccountRole.OWNER.value: "owner",
+        TenantAccountRole.ADMIN.value: "admin",
+        TenantAccountRole.EDITOR.value: "editor",
+        TenantAccountRole.NORMAL.value: "normal",
+        TenantAccountRole.DATASET_OPERATOR.value: "dataset_operator",
+    }.get(legacy_role)
+    if not expected_builtin_tag:
         raise ValueError(f"Unsupported legacy workspace role: {legacy_role}")
 
-    return _resolve_builtin_role_ids(tenant_id, operator_account_id)[legacy_role]
-
-
-def _iter_tenant_member_batches(
-    tenant_id: str | None,
-    *,
-    db_batch_size: int,
-    api_batch_size: int,
-) -> Iterator[tuple[str, str, list[tuple[str, str]]]]:
-    """Yield legacy member roles in tenant-scoped API-sized batches.
-
-    Rows are projected to primitive values and streamed from the database, so
-    the command never materializes every TenantAccountJoin ORM object. The
-    iterator only keeps one tenant's API-sized batches in memory while it
-    finds that tenant's owner account.
-    """
-    with session_factory.create_session() as session:
-        stmt = (
-            select(TenantAccountJoin.tenant_id, TenantAccountJoin.account_id, TenantAccountJoin.role)
-            .order_by(TenantAccountJoin.tenant_id.asc(), TenantAccountJoin.id.asc())
-            .execution_options(yield_per=db_batch_size)
-        )
-        if tenant_id:
-            stmt = stmt.where(TenantAccountJoin.tenant_id == tenant_id)
-
-        current_tenant_id: str | None = None
-        owner_account_id: str | None = None
-        batches: list[list[tuple[str, str]]] = []
-        batch: list[tuple[str, str]] = []
-
-        def flush_current_tenant() -> Iterator[tuple[str, str, list[tuple[str, str]]]]:
-            if current_tenant_id is None:
-                return
-            if batch:
-                batches.append(batch.copy())
-            if not owner_account_id:
-                raise ValueError(f"Workspace owner not found for tenant={current_tenant_id}")
-            for item in batches:
-                yield current_tenant_id, owner_account_id, item
-
-        for row in session.execute(stmt):
-            workspace_id = str(row.tenant_id)
-            if current_tenant_id is not None and workspace_id != current_tenant_id:
-                yield from flush_current_tenant()
-                owner_account_id = None
-                batches = []
-                batch = []
-            current_tenant_id = workspace_id
-            account_id = str(row.account_id)
-            role = str(row.role)
-            if role == TenantAccountRole.OWNER.value:
-                owner_account_id = account_id
-            batch.append((account_id, role))
-            if len(batch) >= api_batch_size:
-                batches.append(batch)
-                batch = []
-
-        yield from flush_current_tenant()
-
-
-def _member_already_has_role(current_roles_by_account_id: dict[str, set[str]], account_id: str, role_id: str) -> bool:
-    return current_roles_by_account_id.get(account_id) == {role_id}
-
-
-def _replace_member_role(
-    tenant_id: str,
-    operator_account_id: str,
-    member_account_id: str,
-    role_id: str,
-    *,
-    session: Session,
-) -> str:
-    RBACService.MemberRoles.replace(
+    roles = RBACService.Roles.list(
         tenant_id=tenant_id,
         account_id=operator_account_id,
-        member_account_id=member_account_id,
-        role_ids=[role_id],
-        session=session,
-    )
-    return member_account_id
+        options=ListOption(page_number=1, results_per_page=100),
+    ).data
+    for role in roles:
+        if role.is_builtin and role.category == "global_system_default" and role.role_tag == expected_builtin_tag:
+            return str(role.id)
 
-
-def _replace_member_role_with_new_session(
-    tenant_id: str,
-    operator_account_id: str,
-    member_account_id: str,
-    role_id: str,
-) -> str:
-    with session_factory.create_session() as session:
-        return _replace_member_role(
-            tenant_id=tenant_id,
-            operator_account_id=operator_account_id,
-            member_account_id=member_account_id,
-            role_id=role_id,
-            session=session,
-        )
+    raise ValueError(f"Builtin RBAC role not found for tenant={tenant_id}, legacy_role={legacy_role}")
 
 
 @click.command(
@@ -166,16 +42,7 @@ def _replace_member_role_with_new_session(
 )
 @click.option("--tenant-id", help="Only migrate a single workspace.")
 @click.option("--dry-run", is_flag=True, default=False, help="Preview the migration without writing RBAC bindings.")
-@click.option("--db-batch-size", default=5000, show_default=True, help="Rows fetched per database batch.")
-@click.option("--api-batch-size", default=200, show_default=True, help="Members checked per RBAC batch_get call.")
-@click.option("--workers", default=1, show_default=True, help="Concurrent member role replace calls per tenant batch.")
-def migrate_member_roles_to_rbac(
-    tenant_id: str | None,
-    dry_run: bool,
-    db_batch_size: int,
-    api_batch_size: int,
-    workers: int,
-) -> None:
+def migrate_member_roles_to_rbac(tenant_id: str | None, dry_run: bool) -> None:
     """Backfill RBAC member-role bindings from legacy `TenantAccountJoin.role` data.
 
     This is an offline migration command for workspaces that already have
@@ -183,329 +50,63 @@ def migrate_member_roles_to_rbac(
     member-role binding store.
     """
     click.echo(click.style("Starting RBAC member-role migration.", fg="green"))
-    if workers < 1:
-        raise click.BadParameter("workers must be >= 1", param_hint="--workers")
 
-    tenant_count = 0
-    scanned_count = 0
-    skipped_count = 0
-    migrated_count = 0
-    current_tenant_id: str | None = None
-    role_ids_by_legacy_role: dict[str, str] = {}
+    with session_factory.create_session() as session:
+        stmt = select(TenantAccountJoin).order_by(TenantAccountJoin.tenant_id.asc(), TenantAccountJoin.id.asc())
+        if tenant_id:
+            stmt = stmt.where(TenantAccountJoin.tenant_id == tenant_id)
 
-    for workspace_id, owner_account_id, batch in _iter_tenant_member_batches(
-        tenant_id,
-        db_batch_size=db_batch_size,
-        api_batch_size=api_batch_size,
-    ):
-        scanned_count += len(batch)
-        if workspace_id != current_tenant_id:
-            tenant_count += 1
-            current_tenant_id = workspace_id
-            role_ids_by_legacy_role = _resolve_builtin_role_ids(workspace_id, owner_account_id)
-            click.echo(f"tenant={workspace_id}")
+        joins = list(session.scalars(stmt).all())
 
-        current_roles_by_account_id: dict[str, set[str]] = {}
-        if not dry_run:
-            current_roles = RBACService.MemberRoles.batch_get(
-                tenant_id=workspace_id,
-                account_id=owner_account_id,
-                member_account_ids=[account_id for account_id, _ in batch],
-            )
-            current_roles_by_account_id = {
-                item.account_id: {str(role.id) for role in item.roles} for item in current_roles
-            }
-
-        replace_jobs: list[tuple[str, str]] = []
-        for member_account_id, legacy_role in batch:
-            resolved_role_id = role_ids_by_legacy_role.get(legacy_role)
-            if not resolved_role_id:
-                raise ValueError(f"Unsupported legacy workspace role: {legacy_role}")
-
-            if dry_run:
-                click.echo(
-                    f"tenant={workspace_id} member={member_account_id} "
-                    f"legacy_role={legacy_role} -> rbac_role_id={resolved_role_id}"
-                )
-                continue
-
-            if _member_already_has_role(current_roles_by_account_id, member_account_id, resolved_role_id):
-                skipped_count += 1
-                continue
-
-            replace_jobs.append((member_account_id, resolved_role_id))
-
-        if replace_jobs:
-            if workers == 1:
-                with session_factory.create_session() as session:
-                    for member_account_id, resolved_role_id in replace_jobs:
-                        _replace_member_role(
-                            workspace_id,
-                            owner_account_id,
-                            member_account_id,
-                            resolved_role_id,
-                            session=session,
-                        )
-                        migrated_count += 1
-            else:
-                with ThreadPoolExecutor(max_workers=workers) as executor:
-                    futures = [
-                        executor.submit(
-                            _replace_member_role_with_new_session,
-                            workspace_id,
-                            owner_account_id,
-                            member_account_id,
-                            resolved_role_id,
-                        )
-                        for member_account_id, resolved_role_id in replace_jobs
-                    ]
-                    for future in as_completed(futures):
-                        future.result()
-                        migrated_count += 1
-
-        if scanned_count % 10000 == 0:
-            click.echo(
-                f"progress scanned={scanned_count} migrated={migrated_count} skipped={skipped_count}",
-                err=True,
-            )
-
-    if scanned_count == 0:
+    if not joins:
         click.echo(click.style("No workspace members found for migration.", fg="yellow"))
         return
 
-    if dry_run:
-        click.echo(
-            click.style(
-                f"Dry run completed. Scanned {scanned_count} members across {tenant_count} tenants. "
-                "No RBAC bindings were written.",
-                fg="yellow",
+    owner_account_by_tenant: dict[str, str] = {}
+    resolved_role_ids: dict[tuple[str, str], str] = {}
+    migrated_count = 0
+
+    for join in joins:
+        workspace_id = str(join.tenant_id)
+        member_account_id = str(join.account_id)
+        legacy_role = str(join.role)
+
+        if workspace_id not in owner_account_by_tenant:
+            owner_join = next(
+                (
+                    item
+                    for item in joins
+                    if str(item.tenant_id) == workspace_id and str(item.role) == TenantAccountRole.OWNER.value
+                ),
+                None,
             )
-        )
-    else:
+            if not owner_join:
+                raise ValueError(f"Workspace owner not found for tenant={workspace_id}")
+            owner_account_by_tenant[workspace_id] = str(owner_join.account_id)
+
+        operator_account_id = owner_account_by_tenant[workspace_id]
+        cache_key = (workspace_id, legacy_role)
+        if cache_key not in resolved_role_ids:
+            resolved_role_ids[cache_key] = _resolve_builtin_role_id(workspace_id, operator_account_id, legacy_role)
+
+        resolved_role_id = resolved_role_ids[cache_key]
         click.echo(
-            click.style(
-                f"RBAC member-role migration completed. Scanned {scanned_count} members across {tenant_count} tenants, "
-                f"migrated {migrated_count}, skipped {skipped_count} already up-to-date.",
-                fg="green",
-            )
+            f"tenant={workspace_id} member={member_account_id} "
+            f"legacy_role={legacy_role} -> rbac_role_id={resolved_role_id}"
         )
 
+        if dry_run:
+            continue
 
-def _dataset_permission_enum(permission: DatasetPermissionEnum | str | None) -> DatasetPermissionEnum:
-    if permission is None:
-        return DatasetPermissionEnum.ONLY_ME
-    try:
-        return DatasetPermissionEnum(permission)
-    except ValueError as exc:
-        raise ValueError(f"Unsupported legacy dataset permission: {permission}") from exc
-
-
-def _rbac_dataset_scope_for_legacy_permission(permission: DatasetPermissionEnum) -> RBACResourceWhitelistScope:
-    if permission is DatasetPermissionEnum.ALL_TEAM:
-        return RBACResourceWhitelistScope.ALL
-    if permission in {DatasetPermissionEnum.ONLY_ME, DatasetPermissionEnum.PARTIAL_TEAM}:
-        return RBACResourceWhitelistScope.SPECIFIC
-    raise ValueError(f"Unsupported legacy dataset permission: {permission}")
-
-
-def _emit_dataset_permission_migration_event(payload: dict[str, object]) -> None:
-    click.echo(json.dumps(payload, sort_keys=True))
-
-
-@click.command(
-    "rbac-migrate-dataset-permissions",
-    help=(
-        "Migrate legacy dataset permission scopes and partial members into RBAC dataset access bindings. "
-        "Side effect: replacing each dataset whitelist clears existing per-user policy bindings; "
-        "the command then recreates legacy partial-member default bindings."
-    ),
-)
-@click.option("--tenant-id", help="Only migrate datasets in a single workspace.")
-@click.option("--dataset-id", help="Only migrate a single dataset.")
-@click.option("--batch-size", default=500, show_default=True, type=click.IntRange(min=1))
-@click.option(
-    "--dry-run/--apply",
-    default=True,
-    show_default=True,
-    help="Preview the migration without writing RBAC bindings. Use --apply to write changes.",
-)
-def migrate_dataset_permissions_to_rbac(
-    tenant_id: str | None,
-    dataset_id: str | None,
-    batch_size: int,
-    dry_run: bool,
-) -> None:
-    """Backfill RBAC dataset access config from legacy `Dataset.permission`.
-
-    Legacy mapping:
-    - all_team_members -> RBAC dataset whitelist scope "all"
-    - partial_members  -> RBAC dataset whitelist scope "specific" plus each partial member gets the
-      virtual default policy
-    - only_me          -> RBAC dataset whitelist scope "specific" with no member policy bindings
-
-    The command replaces each dataset's RBAC whitelist scope first. RBAC clears
-    existing per-user policy bindings during that replace, then this command
-    recreates the legacy partial-member default bindings. Re-running it is
-    therefore idempotent for a dataset's current legacy configuration.
-    """
-    click.echo(click.style("Starting RBAC dataset permission migration.", fg="green"))
-
-    scanned_count = 0
-    scope_migrated_count = 0
-    user_policy_migrated_count = 0
-    partial_dataset_count = 0
-
-    last_dataset_id: str | None = None
-    while True:
-        with session_factory.create_session() as session:
-            stmt = (
-                select(Dataset.id, Dataset.tenant_id, Dataset.permission, Dataset.created_by)
-                .order_by(Dataset.id.asc())
-                .limit(batch_size)
-            )
-            if tenant_id:
-                stmt = stmt.where(Dataset.tenant_id == tenant_id)
-            if dataset_id:
-                stmt = stmt.where(Dataset.id == dataset_id)
-            if last_dataset_id:
-                stmt = stmt.where(Dataset.id > last_dataset_id)
-
-            dataset_rows = list(session.execute(stmt).all())
-            if not dataset_rows:
-                break
-
-            dataset_ids = [str(row.id) for row in dataset_rows]
-            partial_members_by_dataset_id: dict[str, list[str]] = {item: [] for item in dataset_ids}
-            permission_rows = session.execute(
-                select(DatasetPermission.dataset_id, DatasetPermission.account_id).where(
-                    DatasetPermission.dataset_id.in_(dataset_ids)
-                )
-            ).all()
-            for row in permission_rows:
-                partial_members_by_dataset_id[str(row.dataset_id)].append(str(row.account_id))
-
-        for dataset in dataset_rows:
-            workspace_id = str(dataset.tenant_id)
-            current_dataset_id = str(dataset.id)
-            operator_account_id = str(dataset.created_by)
-            permission_value = _dataset_permission_enum(dataset.permission)
-            scope = _rbac_dataset_scope_for_legacy_permission(permission_value)
-            partial_member_ids = sorted(set(partial_members_by_dataset_id[current_dataset_id]))
-            should_bind_partial_members = permission_value is DatasetPermissionEnum.PARTIAL_TEAM
-
-            click.echo(
-                f"tenant={workspace_id} dataset={current_dataset_id} "
-                f"operator={operator_account_id} "
-                f"legacy_permission={permission_value} -> rbac_scope={scope} "
-                f"partial_members={len(partial_member_ids) if should_bind_partial_members else 0}"
-            )
-
-            scanned_count += 1
-            replace_whitelist_payload = ReplaceMemberBindings(scope=scope)
-            if dry_run:
-                _emit_dataset_permission_migration_event(
-                    {
-                        "event": "dataset_permission_migration_proposed_change",
-                        "action": "replace_whitelist",
-                        "dry_run": True,
-                        "tenant_id": workspace_id,
-                        "dataset_id": current_dataset_id,
-                        "operator_account_id": operator_account_id,
-                        "before": {
-                            "legacy_dataset_permission": permission_value.value,
-                            "legacy_partial_member_ids": partial_member_ids if should_bind_partial_members else [],
-                        },
-                        "after": {
-                            "rbac_whitelist_scope": scope.value,
-                        },
-                        "call": {
-                            "method": "RBACService.DatasetAccess.replace_whitelist",
-                            "kwargs": {
-                                "tenant_id": workspace_id,
-                                "account_id": operator_account_id,
-                                "dataset_id": current_dataset_id,
-                                "payload": replace_whitelist_payload.model_dump(mode="json"),
-                            },
-                        },
-                    }
-                )
-            if not dry_run:
-                RBACService.DatasetAccess.replace_whitelist(
-                    tenant_id=workspace_id,
-                    account_id=operator_account_id,
-                    dataset_id=current_dataset_id,
-                    payload=replace_whitelist_payload,
-                )
-                scope_migrated_count += 1
-
-            if should_bind_partial_members:
-                partial_dataset_count += 1
-                for member_account_id in partial_member_ids:
-                    replace_user_access_policies_payload = ReplaceUserAccessPolicies(
-                        access_policy_ids=[_RBAC_DEFAULT_ACCESS_POLICY_ID],
-                    )
-                    if dry_run:
-                        _emit_dataset_permission_migration_event(
-                            {
-                                "event": "dataset_permission_migration_proposed_change",
-                                "action": "replace_user_access_policies",
-                                "dry_run": True,
-                                "tenant_id": workspace_id,
-                                "dataset_id": current_dataset_id,
-                                "operator_account_id": operator_account_id,
-                                "target_account_id": member_account_id,
-                                "before": {
-                                    "legacy_dataset_permission": permission_value.value,
-                                    "legacy_partial_member_id": member_account_id,
-                                },
-                                "after": {
-                                    "rbac_user_access_policy_ids": [_RBAC_DEFAULT_ACCESS_POLICY_ID],
-                                },
-                                "call": {
-                                    "method": "RBACService.DatasetAccess.replace_user_access_policies",
-                                    "kwargs": {
-                                        "tenant_id": workspace_id,
-                                        "account_id": operator_account_id,
-                                        "dataset_id": current_dataset_id,
-                                        "target_account_id": member_account_id,
-                                        "payload": replace_user_access_policies_payload.model_dump(mode="json"),
-                                    },
-                                },
-                            }
-                        )
-                        continue
-                    RBACService.DatasetAccess.replace_user_access_policies(
-                        tenant_id=workspace_id,
-                        account_id=operator_account_id,
-                        dataset_id=current_dataset_id,
-                        target_account_id=member_account_id,
-                        payload=replace_user_access_policies_payload,
-                    )
-                    user_policy_migrated_count += 1
-
-        last_dataset_id = dataset_ids[-1]
-
-        if dataset_id:
-            break
-
-    if scanned_count == 0:
-        click.echo(click.style("No datasets found for migration.", fg="yellow"))
-        return
+        RBACService.MemberRoles.replace(
+            tenant_id=workspace_id,
+            account_id=operator_account_id,
+            member_account_id=member_account_id,
+            role_ids=[resolved_role_id],
+        )
+        migrated_count += 1
 
     if dry_run:
-        click.echo(
-            click.style(
-                f"Dry run completed. Scanned {scanned_count} datasets; "
-                f"{partial_dataset_count} partial-member datasets would be migrated.",
-                fg="yellow",
-            )
-        )
+        click.echo(click.style("Dry run completed. No RBAC bindings were written.", fg="yellow"))
     else:
-        click.echo(
-            click.style(
-                "RBAC dataset permission migration completed. "
-                f"Scanned {scanned_count} datasets, migrated {scope_migrated_count} scopes, "
-                f"wrote {user_policy_migrated_count} user default-policy bindings.",
-                fg="green",
-            )
-        )
+        click.echo(click.style(f"RBAC member-role migration completed. Migrated {migrated_count} members.", fg="green"))

api/commands/retention.py

@@ -35,12 +35,6 @@ class WorkflowRunArchiveTenantPlan(TypedDict):
     unpaid_tenant_ids: list[str]
 
 
-def _normalize_utc_datetime(value: datetime.datetime) -> datetime.datetime:
-    if value.tzinfo is None:
-        return value.replace(tzinfo=datetime.UTC)
-    return value.astimezone(datetime.UTC)
-
-
 def _parse_tenant_prefixes(prefixes: str | None) -> list[str]:
     if not prefixes:
         return []
@@ -162,16 +156,11 @@ def _resolve_archive_time_range(
             raise click.UsageError("Choose either day offsets or explicit dates, not both.")
         if from_days_ago <= to_days_ago:
             raise click.UsageError("--from-days-ago must be greater than --to-days-ago.")
-        now = datetime.datetime.now(datetime.UTC)
+        now = datetime.datetime.now()
         start_from = now - datetime.timedelta(days=from_days_ago)
         end_before = now - datetime.timedelta(days=to_days_ago)
         before_days = 0
 
-    if start_from is not None:
-        start_from = _normalize_utc_datetime(start_from)
-    if end_before is not None:
-        end_before = _normalize_utc_datetime(end_before)
-
     if start_from and end_before and start_from >= end_before:
         raise click.UsageError("--start-from must be earlier than --end-before.")
 
@@ -413,13 +402,6 @@ def archive_workflow_runs_plan(
             fg="white",
         )
     )
-    click.echo(
-        click.style(
-            "fixed_archive_window="
-            f"{start_from.isoformat() if start_from else 'unbounded'},{plan_end_before.isoformat()}",
-            fg="white",
-        )
-    )
     click.echo("tenant_prefix,total_tenants,workflow_runs,workflow_node_executions,paid_tenants,unpaid_tenants")
     for row in rows:
         click.echo(
@@ -469,7 +451,7 @@ def archive_workflow_runs_plan(
     default=None,
     help="Archive runs created before this timestamp (UTC if no timezone).",
 )
-@click.option("--batch-size", default=10000, show_default=True, help="Maximum workflow runs per archive bundle.")
+@click.option("--batch-size", default=100, show_default=True, help="Maximum workflow runs per archive bundle.")
 @click.option(
     "--workers",
     default=1,
@@ -539,7 +521,6 @@ def archive_workflow_runs(
         )
     )
 
-    uses_relative_window = start_from is None and end_before is None
     try:
         before_days, start_from, end_before = _resolve_archive_time_range(
             before_days=before_days,
@@ -565,14 +546,6 @@ def archive_workflow_runs(
     if delete_after_archive:
         click.echo(click.style("delete-after-archive is not supported by bundle archive.", fg="red"))
         return
-    if uses_relative_window:
-        click.echo(
-            click.style(
-                "Relative archive windows are evaluated at command start. For multi-day prefix/shard rollout, "
-                "reuse absolute --start-from/--end-before values from archive-workflow-runs-plan.",
-                fg="yellow",
-            )
-        )
 
     try:
         tenant_plan = _resolve_archive_tenant_ids_from_plan(

api/commands/vector.py

@@ -13,7 +13,6 @@ from core.rag.index_processor.constant.built_in_field import BuiltInField
 from core.rag.index_processor.constant.index_type import IndexStructureType, IndexTechniqueType
 from core.rag.models.document import ChildDocument, Document
 from extensions.ext_database import db
-from libs.pagination import paginate_query
 from models.dataset import Dataset, DatasetCollectionBinding, DatasetMetadata, DatasetMetadataBinding, DocumentSegment
 from models.dataset import Document as DatasetDocument
 from models.enums import DatasetMetadataType, IndexingStatus, SegmentStatus
@@ -184,7 +183,7 @@ def migrate_knowledge_vector_database():
                 .order_by(Dataset.created_at.desc())
             )
 
-            datasets = paginate_query(stmt, page=page, per_page=50, max_per_page=50)
+            datasets = db.paginate(select=stmt, page=page, per_page=50, max_per_page=50, error_out=False)
             if not datasets.items:
                 break
         except SQLAlchemyError:
@@ -410,7 +409,7 @@ def old_metadata_migration():
                 .where(DatasetDocument.doc_metadata.is_not(None))
                 .order_by(DatasetDocument.created_at.desc())
             )
-            documents = paginate_query(stmt, page=page, per_page=50, max_per_page=50)
+            documents = db.paginate(select=stmt, page=page, per_page=50, max_per_page=50, error_out=False)
         except SQLAlchemyError:
             raise
         if not documents:

api/configs/enterprise/__init__.py

@@ -16,7 +16,7 @@ class EnterpriseFeatureConfig(BaseSettings):
 
     CAN_REPLACE_LOGO: bool = Field(
         description="Allow customization of the enterprise logo.",
-        default=False,
+        default=True,
     )
 
     ENTERPRISE_REQUEST_TIMEOUT: int = Field(
@@ -34,12 +34,6 @@ class EnterpriseFeatureConfig(BaseSettings):
         default=False,
     )
 
-    ENTERPRISE_RBAC_REQUEST_TIMEOUT: int = Field(
-        ge=1,
-        description="Maximum timeout in seconds for inner RBAC requests.",
-        default=30,
-    )
-
 
 class EnterpriseTelemetryConfig(BaseSettings):
     """

api/configs/extra/agent_backend_config.py

@@ -1,4 +1,4 @@
-from pydantic import Field, NonNegativeFloat
+from pydantic import Field
 from pydantic_settings import BaseSettings
 
 
@@ -32,10 +32,12 @@ class AgentBackendConfig(BaseSettings):
         default=False,
     )
 
-    AGENT_APP_TEXT_DELTA_DEBOUNCE_SECONDS: NonNegativeFloat = Field(
+    AGENT_DRIVE_MANIFEST_ENABLED: bool = Field(
         description=(
-            "Buffer Agent App assistant text deltas for up to this many seconds before "
-            "publishing SSE chunks. Set to 0 to publish each delta immediately."
+            "Inject the dify.drive layer (Skills & Files drive manifest declaration) "
+            "into Agent runs. The declaration is an index only — the agent backend "
+            "pulls the actual SKILL.md / files through the back proxy. Keep it off "
+            "until the agent backend registers the dify.drive layer type."
         ),
-        default=0.5,
+        default=False,
     )

api/controllers/common/agent_app_parameters.py

@@ -1,54 +0,0 @@
-from typing import Any
-
-from sqlalchemy import select
-
-from core.app.apps.agent_app.app_feature_projection import merge_agent_app_features
-from core.app.apps.agent_app.app_variable_projection import agent_app_variables_to_user_input_form
-from core.app.apps.agent_app.errors import AgentAppGeneratorError, AgentAppNotPublishedError
-from extensions.ext_database import db
-from models.agent import Agent, AgentConfigSnapshot, AgentStatus
-from models.agent_config_entities import AgentSoulConfig
-from models.model import App
-
-
-def get_published_agent_app_feature_dict_and_user_input_form(
-    app_model: App,
-) -> tuple[dict[str, Any], list[dict[str, Any]]]:
-    """Return public Agent App parameters backed by the published Agent Soul."""
-    app_model_config = app_model.app_model_config
-
-    agent_id = app_model.bound_agent_id
-    if not agent_id:
-        raise AgentAppGeneratorError("Agent App has no bound Agent")
-
-    agent = db.session.scalar(
-        select(Agent)
-        .where(
-            Agent.tenant_id == app_model.tenant_id,
-            Agent.id == agent_id,
-            Agent.status == AgentStatus.ACTIVE,
-        )
-        .limit(1)
-    )
-    if agent is None:
-        raise AgentAppGeneratorError("Agent App has no bound Agent")
-    # active_config_is_published means the draft has no unpublished edits; the public app
-    # can still read parameters from the active snapshot while a newer draft is pending.
-    if not agent.active_config_snapshot_id:
-        raise AgentAppNotPublishedError("Agent has not been published")
-
-    snapshot = db.session.scalar(
-        select(AgentConfigSnapshot)
-        .where(
-            AgentConfigSnapshot.tenant_id == app_model.tenant_id,
-            AgentConfigSnapshot.agent_id == agent.id,
-            AgentConfigSnapshot.id == agent.active_config_snapshot_id,
-        )
-        .limit(1)
-    )
-    if snapshot is None:
-        raise AgentAppGeneratorError("Agent published version not found")
-
-    agent_soul = AgentSoulConfig.model_validate(snapshot.config_snapshot_dict)
-    features_dict = merge_agent_app_features(agent_soul=agent_soul, app_model_config=app_model_config)
-    return features_dict, agent_app_variables_to_user_input_form(agent_soul.app_variables)

api/controllers/common/app_access.py

@@ -4,7 +4,6 @@ from collections.abc import Sequence
 from dataclasses import dataclass
 from typing import TYPE_CHECKING
 
-from extensions.ext_database import db
 from services.enterprise import rbac_service as enterprise_rbac_service
 
 if TYPE_CHECKING:
@@ -77,7 +76,7 @@ def resolve_app_access_filter(
     inner-API round trip; otherwise it is fetched here.
     """
     if permissions is None:
-        permissions = enterprise_rbac_service.RBACService.MyPermissions.get(tenant_id, account_id, session=db.session())
+        permissions = enterprise_rbac_service.RBACService.MyPermissions.get(tenant_id, account_id)
     whitelist_scope = enterprise_rbac_service.RBACService.AppAccess.whitelist_resources(tenant_id, account_id)
 
     can_manage_own_apps = _MANAGE_OWN_APPS_PERMISSION_KEY in permissions.workspace.permission_keys

api/controllers/common/fields.py

@@ -183,7 +183,6 @@ class Site(BaseModel):
     description: str | None = None
     copyright: str | None = None
     privacy_policy: str | None = None
-    input_placeholder: str | None = None
     custom_disclaimer: str | None = None
     default_language: str
     show_workflow_steps: bool

api/controllers/common/session.py

@@ -1,57 +0,0 @@
-"""Controller session decorators.
-
-`with_session` is an HTTP controller helper: it opens one SQLAlchemy session
-for a Resource handler and injects it as the first argument after `self`.
-Handlers use a transaction by default so migrated write paths keep
-commit/rollback handling; pure read handlers may opt out with `write=False`.
-"""
-
-from collections.abc import Callable
-from functools import wraps
-from typing import Concatenate, overload
-
-from sqlalchemy.orm import Session
-
-from core.db.session_factory import session_factory
-
-
-@overload
-def with_session[T, **P, R](
-    view: Callable[Concatenate[T, Session, P], R],
-    *,
-    write: bool = True,
-) -> Callable[Concatenate[T, P], R]: ...
-
-
-@overload
-def with_session[T, **P, R](
-    view: None = None,
-    *,
-    write: bool = True,
-) -> Callable[[Callable[Concatenate[T, Session, P], R]], Callable[Concatenate[T, P], R]]: ...
-
-
-def with_session[T, **P, R](
-    view: Callable[Concatenate[T, Session, P], R] | None = None,
-    *,
-    write: bool = True,
-) -> (
-    Callable[Concatenate[T, P], R] | Callable[[Callable[Concatenate[T, Session, P], R]], Callable[Concatenate[T, P], R]]
-):
-    """Inject a request-scoped session, using a transaction only for write handlers."""
-
-    def decorator(view: Callable[Concatenate[T, Session, P], R]) -> Callable[Concatenate[T, P], R]:
-        @wraps(view)
-        def wrapper(self: T, *args: P.args, **kwargs: P.kwargs) -> R:
-            if write:
-                with session_factory.get_session_maker().begin() as session:
-                    return view(self, session, *args, **kwargs)
-
-            with session_factory.create_session() as session:
-                return view(self, session, *args, **kwargs)
-
-        return wrapper
-
-    if view is None:
-        return decorator
-    return decorator(view)

api/controllers/console/__init__.py

@@ -54,7 +54,6 @@ from .app import (
     agent_app_access,
     agent_app_feature,
     agent_app_sandbox,
-    agent_config_inspector,
     agent_drive_inspector,
     annotation,
     app,
@@ -158,7 +157,6 @@ __all__ = [
     "agent_app_feature",
     "agent_app_sandbox",
     "agent_composer",
-    "agent_config_inspector",
     "agent_drive_inspector",
     "agent_providers",
     "agent_roster",

api/controllers/console/agent/app_helpers.py

@@ -6,15 +6,5 @@ from services.agent.roster_service import AgentRosterService
 
 
 def resolve_agent_app_model(*, tenant_id: str, agent_id: UUID) -> App:
-    """Resolve a roster Agent's public Agent App."""
+    """Resolve the hidden Agent App backing an Agent Console resource."""
     return AgentRosterService(db.session).get_agent_app_model(tenant_id=tenant_id, agent_id=str(agent_id))
-
-
-def resolve_agent_runtime_app_model(*, tenant_id: str, agent_id: UUID) -> App:
-    """Resolve the App that backs an Agent runtime surface.
-
-    This accepts both roster Agent Apps and workflow-only inline Agents with a
-    hidden backing App.
-    """
-
-    return AgentRosterService(db.session).get_agent_runtime_app_model(tenant_id=tenant_id, agent_id=str(agent_id))

api/controllers/console/agent/composer.py

@@ -1,10 +1,10 @@
 from uuid import UUID
 
-from flask import request
 from flask_restx import Resource
 
-from controllers.common.schema import query_params_from_model, register_response_schema_models, register_schema_models
+from controllers.common.schema import register_response_schema_models, register_schema_models
 from controllers.console import console_ns
+from controllers.console.agent.app_helpers import resolve_agent_app_model
 from controllers.console.app.wraps import get_app_model
 from controllers.console.wraps import (
     RBACPermission,
@@ -16,7 +16,6 @@ from controllers.console.wraps import (
     with_current_tenant_id,
     with_current_user_id,
 )
-from extensions.ext_database import db
 from fields.agent_fields import (
     AgentAppComposerResponse,
     AgentComposerCandidatesResponse,
@@ -29,15 +28,9 @@ from libs.login import login_required
 from models.model import App, AppMode
 from services.agent.composer_service import AgentComposerService
 from services.agent.composer_validator import ComposerConfigValidator
-from services.entities.agent_entities import (
-    ComposerSavePayload,
-    WorkflowAgentComposerQuery,
-    WorkflowComposerCopyFromRosterPayload,
-)
+from services.entities.agent_entities import ComposerSavePayload, WorkflowComposerCopyFromRosterPayload
 
-register_schema_models(
-    console_ns, ComposerSavePayload, WorkflowAgentComposerQuery, WorkflowComposerCopyFromRosterPayload
-)
+register_schema_models(console_ns, ComposerSavePayload, WorkflowComposerCopyFromRosterPayload)
 register_response_schema_models(
     console_ns,
     AgentAppComposerResponse,
@@ -48,29 +41,27 @@ register_response_schema_models(
 )
 
 
+def _resolve_agent_app_id(*, tenant_id: str, agent_id: UUID) -> str:
+    return resolve_agent_app_model(tenant_id=tenant_id, agent_id=agent_id).id
+
+
 @console_ns.route("/apps/<uuid:app_id>/workflows/draft/nodes/<string:node_id>/agent-composer")
 class WorkflowAgentComposerApi(Resource):
     @console_ns.response(
         200, "Workflow agent composer state", console_ns.models[WorkflowAgentComposerResponse.__name__]
     )
-    @console_ns.doc(params=query_params_from_model(WorkflowAgentComposerQuery))
     @setup_required
     @login_required
     @account_initialization_required
     @get_app_model(mode=[AppMode.WORKFLOW, AppMode.ADVANCED_CHAT])
-    @with_current_user_id
     @with_current_tenant_id
-    def get(self, tenant_id: str, account_id: str, app_model: App, node_id: str):
-        query = WorkflowAgentComposerQuery.model_validate(request.args.to_dict(flat=True))
+    def get(self, tenant_id: str, app_model: App, node_id: str):
         return dump_response(
             WorkflowAgentComposerResponse,
             AgentComposerService.load_workflow_composer(
                 tenant_id=tenant_id,
                 app_id=app_model.id,
                 node_id=node_id,
-                account_id=account_id,
-                snapshot_id=query.snapshot_id,
-                session=db.session(),
             ),
         )
 
@@ -96,7 +87,6 @@ class WorkflowAgentComposerApi(Resource):
                 node_id=node_id,
                 account_id=account_id,
                 payload=payload,
-                session=db.session(),
             ),
         )
 
@@ -129,7 +119,6 @@ class WorkflowAgentComposerCopyFromRosterApi(Resource):
                 source_agent_id=payload.source_agent_id,
                 source_snapshot_id=payload.source_snapshot_id,
                 idempotency_key=payload.idempotency_key,
-                session=db.session(),
             ),
         )
 
@@ -148,14 +137,12 @@ class WorkflowAgentComposerValidateApi(Resource):
     def post(self, tenant_id: str, app_model: App, node_id: str):
         payload = ComposerSavePayload.model_validate(console_ns.payload or {})
         ComposerConfigValidator.validate_publish_payload(payload)
-        AgentComposerService.validate_knowledge_datasets(tenant_id=tenant_id, agent_soul=payload.agent_soul)
         findings = AgentComposerService.collect_validation_findings(
             tenant_id=tenant_id,
             payload=payload,
             agent_id=AgentComposerService.resolve_workflow_node_agent_id(
-                tenant_id=tenant_id, app_id=app_model.id, node_id=node_id, session=db.session()
+                tenant_id=tenant_id, app_id=app_model.id, node_id=node_id
             ),
-            session=db.session(),
         )
         return dump_response(AgentComposerValidateResponse, {"result": "success", "errors": [], **findings})
 
@@ -179,7 +166,6 @@ class WorkflowAgentComposerCandidatesApi(Resource):
                 app_id=app_model.id,
                 node_id=node_id,
                 user_id=current_user_id,
-                session=db.session(),
             ),
         )
 
@@ -202,9 +188,7 @@ class WorkflowAgentComposerImpactApi(Resource):
             )
         return dump_response(
             AgentComposerImpactResponse,
-            AgentComposerService.calculate_impact(
-                tenant_id=tenant_id, current_snapshot_id=current_snapshot_id, session=db.session()
-            ),
+            AgentComposerService.calculate_impact(tenant_id=tenant_id, current_snapshot_id=current_snapshot_id),
         )
 
 
@@ -232,7 +216,6 @@ class WorkflowAgentComposerSaveToRosterApi(Resource):
                 node_id=node_id,
                 account_id=account_id,
                 payload=payload,
-                session=db.session(),
             ),
         )
 
@@ -245,9 +228,10 @@ class AgentComposerApi(Resource):
     @account_initialization_required
     @with_current_tenant_id
     def get(self, tenant_id: str, agent_id: UUID):
+        app_id = _resolve_agent_app_id(tenant_id=tenant_id, agent_id=agent_id)
         return dump_response(
             AgentAppComposerResponse,
-            AgentComposerService.load_agent_composer(tenant_id=tenant_id, agent_id=str(agent_id), session=db.session()),
+            AgentComposerService.load_agent_app_composer(tenant_id=tenant_id, app_id=app_id),
         )
 
     @console_ns.expect(console_ns.models[ComposerSavePayload.__name__])
@@ -260,15 +244,15 @@ class AgentComposerApi(Resource):
     @with_current_user_id
     @with_current_tenant_id
     def put(self, tenant_id: str, account_id: str, agent_id: UUID):
+        app_id = _resolve_agent_app_id(tenant_id=tenant_id, agent_id=agent_id)
         payload = ComposerSavePayload.model_validate(console_ns.payload or {})
         return dump_response(
             AgentAppComposerResponse,
-            AgentComposerService.save_agent_composer(
+            AgentComposerService.save_agent_app_composer(
                 tenant_id=tenant_id,
-                agent_id=str(agent_id),
+                app_id=app_id,
                 account_id=account_id,
                 payload=payload,
-                session=db.session(),
             ),
         )
 
@@ -284,15 +268,13 @@ class AgentComposerValidateApi(Resource):
     @account_initialization_required
     @with_current_tenant_id
     def post(self, tenant_id: str, agent_id: UUID):
-        AgentComposerService.load_agent_composer(tenant_id=tenant_id, agent_id=str(agent_id), session=db.session())
+        _resolve_agent_app_id(tenant_id=tenant_id, agent_id=agent_id)
         payload = ComposerSavePayload.model_validate(console_ns.payload or {})
         ComposerConfigValidator.validate_publish_payload(payload)
-        AgentComposerService.validate_knowledge_datasets(tenant_id=tenant_id, agent_soul=payload.agent_soul)
         findings = AgentComposerService.collect_validation_findings(
             tenant_id=tenant_id,
             payload=payload,
             agent_id=str(agent_id),
-            session=db.session(),
         )
         return dump_response(AgentComposerValidateResponse, {"result": "success", "errors": [], **findings})
 
@@ -308,12 +290,12 @@ class AgentComposerCandidatesApi(Resource):
     @with_current_user_id
     @with_current_tenant_id
     def get(self, tenant_id: str, current_user_id: str, agent_id: UUID):
+        app_id = _resolve_agent_app_id(tenant_id=tenant_id, agent_id=agent_id)
         return dump_response(
             AgentComposerCandidatesResponse,
             AgentComposerService.get_agent_app_candidates(
                 tenant_id=tenant_id,
-                agent_id=str(agent_id),
+                app_id=app_id,
                 user_id=current_user_id,
-                session=db.session(),
             ),
         )

api/controllers/console/agent/roster.py

@@ -5,21 +5,16 @@ from flask_restx import Resource
 from pydantic import AliasChoices, BaseModel, Field, field_validator
 from sqlalchemy import func, select
 
-from controllers.common.schema import (
-    query_params_from_model,
-    query_params_from_request,
-    register_response_schema_models,
-    register_schema_models,
-)
+from controllers.common.schema import query_params_from_model, register_response_schema_models, register_schema_models
 from controllers.console import console_ns
-from controllers.console.agent.app_helpers import resolve_agent_app_model, resolve_agent_runtime_app_model
+from controllers.console.agent.app_helpers import resolve_agent_app_model
 from controllers.console.apikey import ApiKeyItem, ApiKeyList, BaseApiKeyListResource, BaseApiKeyResource
 from controllers.console.app.app import (
-    APP_LIST_QUERY_ARRAY_FIELDS,
-    AppListQuery,
+    AppDetailWithSite as GenericAppDetailWithSite,
 )
 from controllers.console.app.app import (
-    AppDetailWithSite as GenericAppDetailWithSite,
+    AppListQuery,
+    _normalize_app_list_query_args,
 )
 from controllers.console.app.app import (
     AppPagination as GenericAppPagination,
@@ -44,11 +39,9 @@ from controllers.console.wraps import (
 )
 from extensions.ext_database import db
 from fields.agent_fields import (
-    AgentConfigDraftSummaryResponse,
     AgentConfigSnapshotDetailResponse,
     AgentConfigSnapshotListResponse,
     AgentConfigSnapshotRestoreResponse,
-    AgentConfigSnapshotSummaryResponse,
     AgentInviteOptionsResponse,
     AgentLogListResponse,
     AgentLogMessageListResponse,
@@ -57,16 +50,12 @@ from fields.agent_fields import (
     AgentRosterListResponse,
     AgentStatisticSummaryEnvelopeResponse,
 )
-from fields.base import ResponseModel
 from libs.datetime_utils import parse_time_range
 from libs.helper import dump_response
 from libs.login import login_required
 from models import Account
-from models.agent import Agent, AgentStatus
-from models.agent_config_entities import AgentSoulConfig
 from models.enums import ApiTokenType
 from models.model import ApiToken, App, IconType
-from services.agent.composer_service import AgentComposerService
 from services.agent.errors import AgentNotFoundError
 from services.agent.observability_service import (
     AgentLogQueryParams,
@@ -76,7 +65,7 @@ from services.agent.observability_service import (
 from services.agent.roster_service import AgentRosterService
 from services.app_service import AppListParams, AppService, CreateAppParams
 from services.enterprise.enterprise_service import EnterpriseService
-from services.entities.agent_entities import ComposerSavePayload, RosterListQuery
+from services.entities.agent_entities import RosterListQuery
 from services.feature_service import FeatureService
 
 
@@ -91,31 +80,33 @@ class AgentIdPath(BaseModel):
 class AgentAppCreatePayload(BaseModel):
     name: str = Field(..., min_length=1, description="Agent name")
     description: str | None = Field(default=None, description="Agent description (max 400 chars)", max_length=400)
-    role: str | None = Field(default=None, description="Agent role", max_length=255)
+    role: str = Field(..., min_length=1, description="Agent role", max_length=255)
     icon_type: IconType | None = Field(default=None, description="Icon type")
     icon: str | None = Field(default=None, description="Icon")
     icon_background: str | None = Field(default=None, description="Icon background color")
 
     @field_validator("role")
     @classmethod
-    def validate_role(cls, value: str | None) -> str | None:
-        if value is None:
-            return None
-        return value.strip()
+    def validate_role(cls, value: str) -> str:
+        role = value.strip()
+        if not role:
+            raise ValueError("Agent role is required.")
+        return role
 
 
 # Keep agent-app roster DTOs agent-specific instead of reusing the shared
 # /apps response/request models. The roster surface needs Agent-only fields such
 # as `role`, while the generic console/apps contracts must stay unchanged.
 class AgentAppUpdatePayload(GenericUpdateAppPayload):
-    role: str | None = Field(default=None, description="Agent role", max_length=255)
+    role: str = Field(..., min_length=1, description="Agent role", max_length=255)
 
     @field_validator("role")
     @classmethod
-    def validate_role(cls, value: str | None) -> str | None:
-        if value is None:
-            return None
-        return value.strip()
+    def validate_role(cls, value: str) -> str:
+        role = value.strip()
+        if not role:
+            raise ValueError("Agent role is required.")
+        return role
 
 
 class AgentAppCopyPayload(BaseModel):
@@ -131,7 +122,10 @@ class AgentAppCopyPayload(BaseModel):
     def validate_role(cls, value: str | None) -> str | None:
         if value is None:
             return None
-        return value.strip()
+        role = value.strip()
+        if not role:
+            raise ValueError("Agent role is required when provided.")
+        return role
 
 
 class AgentApiStatusPayload(BaseModel):
@@ -197,9 +191,11 @@ class AgentLogsQuery(BaseModel):
     def empty_list_values_to_list(cls, value: object) -> list[str]:
         if value in (None, ""):
             return []
+        if isinstance(value, str):
+            return [value]
         if isinstance(value, list):
-            return [str(item).strip() for item in value if str(item).strip()]
-        raise ValueError("Unsupported query list type.")
+            return [item for item in value if item]
+        return []
 
     @field_validator("sort_by")
     @classmethod
@@ -236,8 +232,6 @@ class AgentStatisticsQuery(BaseModel):
 
 class AgentAppPartial(GenericAppPartial):
     app_id: str | None = None
-    backing_app_id: str | None = None
-    hidden_app_backed: bool = False
     debug_conversation_id: str | None = None
     role: str | None = None
     active_config_is_published: bool = False
@@ -247,49 +241,13 @@ class AgentAppPartial(GenericAppPartial):
 
 class AgentAppDetailWithSite(GenericAppDetailWithSite):
     app_id: str | None = None
-    backing_app_id: str | None = None
-    hidden_app_backed: bool = False
     debug_conversation_id: str | None = None
-    debug_conversation_has_messages: bool = False
-    debug_conversation_message_count: int = 0
     role: str | None = None
     active_config_is_published: bool = False
 
 
 class AgentDebugConversationRefreshResponse(BaseModel):
     debug_conversation_id: str
-    debug_conversation_has_messages: bool = False
-    debug_conversation_message_count: int = 0
-
-
-class AgentPublishPayload(BaseModel):
-    version_note: str | None = Field(default=None, description="Optional note for this published Agent version")
-
-
-class AgentPublishResponse(ResponseModel):
-    result: str
-    active_config_snapshot_id: str
-    active_config_snapshot: AgentConfigSnapshotSummaryResponse | None = None
-    draft: AgentConfigDraftSummaryResponse | None = None
-
-
-class AgentBuildDraftCheckoutPayload(BaseModel):
-    force: bool = Field(default=False, description="Overwrite the existing current-user build draft")
-
-
-class AgentBuildDraftResponse(ResponseModel):
-    variant: str
-    draft: AgentConfigDraftSummaryResponse
-    agent_soul: AgentSoulConfig
-
-
-class AgentBuildDraftApplyResponse(BaseModel):
-    result: str
-    draft: dict[str, object]
-
-
-class AgentSimpleResultResponse(BaseModel):
-    result: str
 
 
 class AgentAppPagination(GenericAppPagination):
@@ -303,9 +261,6 @@ register_schema_models(
     AgentAppCreatePayload,
     AgentAppUpdatePayload,
     AgentAppCopyPayload,
-    AgentPublishPayload,
-    AgentBuildDraftCheckoutPayload,
-    ComposerSavePayload,
     AgentApiStatusPayload,
     AgentInviteOptionsQuery,
     AgentLogsQuery,
@@ -322,10 +277,6 @@ register_response_schema_models(
     AgentAppDetailWithSite,
     AgentAppPartial,
     AgentDebugConversationRefreshResponse,
-    AgentPublishResponse,
-    AgentBuildDraftResponse,
-    AgentBuildDraftApplyResponse,
-    AgentSimpleResultResponse,
     AgentConfigSnapshotDetailResponse,
     AgentConfigSnapshotListResponse,
     AgentConfigSnapshotRestoreResponse,
@@ -343,7 +294,7 @@ def _agent_roster_service() -> AgentRosterService:
     return AgentRosterService(db.session)
 
 
-def _serialize_agent_app_detail(app_model, *, current_user: Account, agent_id: str | None = None) -> dict:
+def _serialize_agent_app_detail(app_model, *, current_user: Account) -> dict:
     """Serialize an Agent App detail using roster-only DTOs.
 
     `/agent` responses are roster-shaped rather than raw app-shaped: `id`
@@ -360,35 +311,17 @@ def _serialize_agent_app_detail(app_model, *, current_user: Account, agent_id: s
 
     roster_service = _agent_roster_service()
     payload = AgentAppDetailWithSite.model_validate(app_model, from_attributes=True).model_dump(mode="json")
-    agent = (
-        db.session.scalar(
-            select(Agent).where(
-                Agent.tenant_id == app_model.tenant_id,
-                Agent.id == agent_id,
-                Agent.status == AgentStatus.ACTIVE,
-            )
-        )
-        if agent_id
-        else roster_service.get_app_backing_agent(tenant_id=app_model.tenant_id, app_id=str(app_model.id))
-    )
+    agent = roster_service.get_app_backing_agent(tenant_id=app_model.tenant_id, app_id=str(app_model.id))
     if not agent:
         raise AgentNotFoundError()
     payload.pop("bound_agent_id", None)
-    payload["app_id"] = agent.app_id
-    payload["backing_app_id"] = roster_service.runtime_backing_app_id(agent)
-    payload["hidden_app_backed"] = bool(agent.backing_app_id and agent.backing_app_id != agent.app_id)
+    payload["app_id"] = str(app_model.id)
     payload["id"] = agent.id
-    debug_conversation_id = roster_service.get_or_create_agent_app_debug_conversation_id(
+    payload["debug_conversation_id"] = roster_service.get_or_create_agent_app_debug_conversation_id(
         tenant_id=app_model.tenant_id,
         agent_id=agent.id,
         account_id=current_user.id,
     )
-    message_count = roster_service.count_agent_app_debug_conversation_messages(
-        conversation_id=debug_conversation_id,
-    )
-    payload["debug_conversation_id"] = debug_conversation_id
-    payload["debug_conversation_has_messages"] = message_count > 0
-    payload["debug_conversation_message_count"] = message_count
     payload["role"] = agent.role or ""
     payload["active_config_is_published"] = roster_service.active_config_is_published(
         tenant_id=app_model.tenant_id,
@@ -432,8 +365,6 @@ def _serialize_agent_app_pagination(app_pagination, *, tenant_id: str, current_u
         agent = agents_by_app_id.get(app_id)
         if agent:
             item["app_id"] = app_id
-            item["backing_app_id"] = agent.backing_app_id or app_id
-            item["hidden_app_backed"] = False
             item["id"] = agent.id
             item["debug_conversation_id"] = debug_conversation_ids_by_agent_id.get(agent.id)
             item["role"] = agent.role or ""
@@ -504,11 +435,16 @@ def _parse_observability_time_range(start: str | None, end: str | None, account:
         abort(400, description=str(exc))
 
 
-def _query_values(name: str, alias_name: str | None = None) -> list[str]:
-    values = request.args.getlist(name)
-    if alias_name:
-        values.extend(request.args.getlist(alias_name))
-    return [value.strip() for value in values if value.strip()]
+def _multi_query_values(name: str, legacy_name: str | None = None) -> list[str]:
+    values: list[str] = []
+    for query_name in (name, f"{name}[]"):
+        values.extend(request.args.getlist(query_name))
+    if legacy_name:
+        values.extend(request.args.getlist(legacy_name))
+    parsed: list[str] = []
+    for value in values:
+        parsed.extend(item.strip() for item in value.split(",") if item.strip())
+    return parsed
 
 
 @console_ns.route("/agent")
@@ -521,12 +457,11 @@ class AgentAppListApi(Resource):
     @with_current_user
     @with_current_tenant_id
     def get(self, current_tenant_id: str, current_user: Account):
-        args = query_params_from_request(AppListQuery, list_fields=APP_LIST_QUERY_ARRAY_FIELDS)
+        args = AppListQuery.model_validate(_normalize_app_list_query_args(request.args))
         params = AppListParams(
             page=args.page,
             limit=args.limit,
             mode="agent",
-            sort_by=args.sort_by,
             name=args.name,
             tag_ids=args.tag_ids,
             creator_ids=args.creator_ids,
@@ -534,7 +469,7 @@ class AgentAppListApi(Resource):
             status="normal",
         )
 
-        app_pagination = AppService().get_paginate_apps(current_user.id, current_tenant_id, params, db.session())
+        app_pagination = AppService().get_paginate_apps(current_user.id, current_tenant_id, params, db.session)
         if app_pagination is None:
             empty = AgentAppPagination(page=args.page, limit=args.limit, total=0, has_more=False, data=[])
             return empty.model_dump(mode="json")
@@ -561,13 +496,13 @@ class AgentAppListApi(Resource):
             name=args.name,
             description=args.description,
             mode="agent",
-            agent_role=args.role or "",
+            agent_role=args.role,
             icon_type=args.icon_type,
             icon=args.icon,
             icon_background=args.icon_background,
         )
 
-        app = AppService().create_app(current_tenant_id, params, current_user, session=db.session())
+        app = AppService().create_app(current_tenant_id, params, current_user)
         return _serialize_agent_app_detail(app, current_user=current_user), 201
 
 
@@ -581,8 +516,8 @@ class AgentAppApi(Resource):
     @with_current_user
     @with_current_tenant_id
     def get(self, tenant_id: str, current_user: Account, agent_id: UUID):
-        app_model = resolve_agent_runtime_app_model(tenant_id=tenant_id, agent_id=agent_id)
-        return _serialize_agent_app_detail(app_model, current_user=current_user, agent_id=str(agent_id))
+        app_model = _resolve_agent_app_model(tenant_id=tenant_id, agent_id=agent_id)
+        return _serialize_agent_app_detail(app_model, current_user=current_user)
 
     @console_ns.expect(console_ns.models[AgentAppUpdatePayload.__name__])
     @console_ns.response(200, "Agent app updated successfully", console_ns.models[AgentAppDetailWithSite.__name__])
@@ -607,7 +542,7 @@ class AgentAppApi(Resource):
             "max_active_requests": args.max_active_requests or 0,
             "role": args.role,
         }
-        updated = AppService().update_app(app_model, args_dict, session=db.session())
+        updated = AppService().update_app(app_model, args_dict)
         return _serialize_agent_app_detail(updated, current_user=current_user)
 
     @console_ns.response(204, "Agent app deleted successfully")
@@ -619,7 +554,7 @@ class AgentAppApi(Resource):
     @with_current_tenant_id
     def delete(self, tenant_id: str, agent_id: UUID):
         app_model = _resolve_agent_app_model(tenant_id=tenant_id, agent_id=agent_id)
-        AppService().delete_app(app_model, session=db.session())
+        AppService().delete_app(app_model)
         return "", 204
 
 
@@ -643,122 +578,8 @@ class AgentDebugConversationRefreshApi(Resource):
             agent_id=str(agent_id),
             account_id=current_user.id,
         )
-        return AgentDebugConversationRefreshResponse(
-            debug_conversation_id=debug_conversation_id,
-            debug_conversation_has_messages=False,
-            debug_conversation_message_count=0,
-        ).model_dump(mode="json")
-
-
-@console_ns.route("/agent/<uuid:agent_id>/publish")
-class AgentPublishApi(Resource):
-    @console_ns.expect(console_ns.models[AgentPublishPayload.__name__])
-    @console_ns.response(200, "Agent draft published", console_ns.models[AgentPublishResponse.__name__])
-    @console_ns.response(403, "Insufficient permissions")
-    @setup_required
-    @login_required
-    @account_initialization_required
-    @edit_permission_required
-    @with_current_user
-    @with_current_tenant_id
-    def post(self, tenant_id: str, current_user: Account, agent_id: UUID):
-        args = AgentPublishPayload.model_validate(console_ns.payload or {})
-        return AgentComposerService.publish_agent_app_draft(
-            tenant_id=tenant_id,
-            agent_id=str(agent_id),
-            account_id=current_user.id,
-            version_note=args.version_note,
-            session=db.session(),
-        )
-
-
-@console_ns.route("/agent/<uuid:agent_id>/build-draft/checkout")
-class AgentBuildDraftCheckoutApi(Resource):
-    @console_ns.expect(console_ns.models[AgentBuildDraftCheckoutPayload.__name__])
-    @console_ns.response(200, "Agent build draft checked out", console_ns.models[AgentBuildDraftResponse.__name__])
-    @setup_required
-    @login_required
-    @account_initialization_required
-    @edit_permission_required
-    @with_current_user
-    @with_current_tenant_id
-    def post(self, tenant_id: str, current_user: Account, agent_id: UUID):
-        args = AgentBuildDraftCheckoutPayload.model_validate(console_ns.payload or {})
-        return AgentComposerService.checkout_agent_app_build_draft(
-            tenant_id=tenant_id,
-            agent_id=str(agent_id),
-            account_id=current_user.id,
-            force=args.force,
-            session=db.session(),
-        )
-
-
-@console_ns.route("/agent/<uuid:agent_id>/build-draft")
-class AgentBuildDraftApi(Resource):
-    @console_ns.response(200, "Agent build draft", console_ns.models[AgentBuildDraftResponse.__name__])
-    @setup_required
-    @login_required
-    @account_initialization_required
-    @edit_permission_required
-    @with_current_user
-    @with_current_tenant_id
-    def get(self, tenant_id: str, current_user: Account, agent_id: UUID):
-        return AgentComposerService.load_agent_app_build_draft(
-            tenant_id=tenant_id,
-            agent_id=str(agent_id),
-            account_id=current_user.id,
-            session=db.session(),
-        )
-
-    @console_ns.expect(console_ns.models[ComposerSavePayload.__name__])
-    @console_ns.response(200, "Agent build draft saved", console_ns.models[AgentBuildDraftResponse.__name__])
-    @setup_required
-    @login_required
-    @account_initialization_required
-    @edit_permission_required
-    @with_current_user
-    @with_current_tenant_id
-    def put(self, tenant_id: str, current_user: Account, agent_id: UUID):
-        payload = ComposerSavePayload.model_validate(console_ns.payload or {})
-        return AgentComposerService.save_agent_app_build_draft(
-            tenant_id=tenant_id,
-            agent_id=str(agent_id),
-            account_id=current_user.id,
-            payload=payload,
-            session=db.session(),
-        )
-
-    @console_ns.response(200, "Agent build draft discarded", console_ns.models[AgentSimpleResultResponse.__name__])
-    @setup_required
-    @login_required
-    @account_initialization_required
-    @edit_permission_required
-    @with_current_user
-    @with_current_tenant_id
-    def delete(self, tenant_id: str, current_user: Account, agent_id: UUID):
-        return AgentComposerService.discard_agent_app_build_draft(
-            tenant_id=tenant_id,
-            agent_id=str(agent_id),
-            account_id=current_user.id,
-            session=db.session(),
-        )
-
-
-@console_ns.route("/agent/<uuid:agent_id>/build-draft/apply")
-class AgentBuildDraftApplyApi(Resource):
-    @console_ns.response(200, "Agent build draft applied", console_ns.models[AgentBuildDraftApplyResponse.__name__])
-    @setup_required
-    @login_required
-    @account_initialization_required
-    @edit_permission_required
-    @with_current_user
-    @with_current_tenant_id
-    def post(self, tenant_id: str, current_user: Account, agent_id: UUID):
-        return AgentComposerService.apply_agent_app_build_draft(
-            tenant_id=tenant_id,
-            agent_id=str(agent_id),
-            account_id=current_user.id,
-            session=db.session(),
+        return AgentDebugConversationRefreshResponse(debug_conversation_id=debug_conversation_id).model_dump(
+            mode="json"
         )
 
 
@@ -816,7 +637,7 @@ class AgentApiStatusApi(Resource):
     def post(self, tenant_id: str, agent_id: UUID):
         app_model = _resolve_agent_app_model(tenant_id=tenant_id, agent_id=agent_id)
         args = AgentApiStatusPayload.model_validate(console_ns.payload)
-        app_model = AppService().update_app_api_status(app_model, args.enable_api, session=db.session())
+        app_model = AppService().update_app_api_status(app_model, args.enable_api)
         return _serialize_agent_api_access(app_model)
 
 
@@ -891,10 +712,10 @@ class AgentLogsApi(Resource):
     @with_current_user
     @with_current_tenant_id
     def get(self, tenant_id: str, current_user: Account, agent_id: UUID):
-        app_model = resolve_agent_runtime_app_model(tenant_id=tenant_id, agent_id=agent_id)
+        app_model = _resolve_agent_app_model(tenant_id=tenant_id, agent_id=agent_id)
         query_data: dict[str, object] = dict(request.args.to_dict(flat=True))
-        query_data["sources"] = _query_values("sources", "source")
-        query_data["statuses"] = _query_values("statuses", "status")
+        query_data["sources"] = _multi_query_values("sources", "source")
+        query_data["statuses"] = _multi_query_values("statuses", "status")
         query = AgentLogsQuery.model_validate(query_data)
         start, end = _parse_observability_time_range(query.start, query.end, current_user)
         try:
@@ -928,10 +749,10 @@ class AgentLogMessagesApi(Resource):
     @with_current_user
     @with_current_tenant_id
     def get(self, tenant_id: str, current_user: Account, agent_id: UUID, conversation_id: UUID):
-        app_model = resolve_agent_runtime_app_model(tenant_id=tenant_id, agent_id=agent_id)
+        app_model = _resolve_agent_app_model(tenant_id=tenant_id, agent_id=agent_id)
         query_data: dict[str, object] = dict(request.args.to_dict(flat=True))
-        query_data["sources"] = _query_values("sources", "source")
-        query_data["statuses"] = _query_values("statuses", "status")
+        query_data["sources"] = _multi_query_values("sources", "source")
+        query_data["statuses"] = _multi_query_values("statuses", "status")
         query = AgentLogsQuery.model_validate(query_data)
         start, end = _parse_observability_time_range(query.start, query.end, current_user)
         try:
@@ -965,7 +786,7 @@ class AgentLogSourcesApi(Resource):
     @with_current_user
     @with_current_tenant_id
     def get(self, tenant_id: str, current_user: Account, agent_id: UUID):
-        app_model = resolve_agent_runtime_app_model(tenant_id=tenant_id, agent_id=agent_id)
+        app_model = _resolve_agent_app_model(tenant_id=tenant_id, agent_id=agent_id)
         payload = _agent_observability_service().list_log_sources(app=app_model, agent_id=str(agent_id))
         return dump_response(AgentLogSourceListResponse, payload)
 
@@ -984,7 +805,7 @@ class AgentStatisticsSummaryApi(Resource):
     @with_current_user
     @with_current_tenant_id
     def get(self, tenant_id: str, current_user: Account, agent_id: UUID):
-        app_model = resolve_agent_runtime_app_model(tenant_id=tenant_id, agent_id=agent_id)
+        app_model = _resolve_agent_app_model(tenant_id=tenant_id, agent_id=agent_id)
         query = AgentStatisticsQuery.model_validate(request.args.to_dict(flat=True))
         timezone = current_user.timezone or "UTC"
         start, end = _parse_observability_time_range(query.start, query.end, current_user)

api/controllers/console/app/agent.py

@@ -13,7 +13,7 @@ from controllers.common.schema import (
     register_schema_models,
 )
 from controllers.console import console_ns
-from controllers.console.agent.app_helpers import resolve_agent_runtime_app_model
+from controllers.console.agent.app_helpers import resolve_agent_app_model
 from controllers.console.app.wraps import get_app_model
 from controllers.console.wraps import (
     RBACPermission,
@@ -172,7 +172,7 @@ register_response_schema_models(
 def _resolve_agent_id(app_model: App, node_id: str | None) -> str | None:
     if node_id and app_model.mode != AppMode.AGENT:
         return AgentComposerService.resolve_workflow_node_agent_id(
-            tenant_id=app_model.tenant_id, app_id=app_model.id, node_id=node_id, session=db.session()
+            tenant_id=app_model.tenant_id, app_id=app_model.id, node_id=node_id
         )
     return app_model.bound_agent_id
 
@@ -202,7 +202,6 @@ def _upload_skill_for_app(*, current_user: Account, app_model: App):
             tenant_id=app_model.tenant_id,
             user_id=current_user.id,
             agent_id=agent_id,
-            session=db.session(),
         )
     except (SkillPackageError, AgentDriveError) as exc:
         return {"code": exc.code, "message": exc.message}, exc.status_code
@@ -241,7 +240,6 @@ def _commit_drive_file_for_app(*, current_user: Account, app_model: App, allow_n
                     value_owned_by_drive=True,
                 )
             ],
-            session=db.session(),
         )
     except AgentDriveError as exc:
         return {"code": exc.code, "message": exc.message}, exc.status_code
@@ -275,7 +273,6 @@ def _delete_drive_file_for_app(*, current_user: Account, app_model: App, allow_n
             user_id=current_user.id,
             agent_id=agent_id,
             items=[DriveCommitItem(key=key, file_ref=None)],
-            session=db.session(),
         )
     except AgentDriveError as exc:
         return {"code": exc.code, "message": exc.message}, exc.status_code
@@ -301,7 +298,6 @@ def _delete_skill_for_app(*, current_user: Account, app_model: App, slug: str, a
                 DriveCommitItem(key=f"{slug}/SKILL.md", file_ref=None),
                 DriveCommitItem(key=f"{slug}/.DIFY-SKILL-FULL.zip", file_ref=None),
             ],
-            session=db.session(),
         )
     except AgentDriveError as exc:
         return {"code": exc.code, "message": exc.message}, exc.status_code
@@ -317,9 +313,7 @@ def _infer_skill_tools_for_app(*, app_model: App, slug: str):
     if "/" in slug or not slug.strip():
         return {"code": "drive_key_invalid", "message": "skill slug must be a single path segment"}, 400
     try:
-        return SkillToolInferenceService().infer(
-            tenant_id=app_model.tenant_id, agent_id=agent_id, slug=slug, session=db.session()
-        )
+        return SkillToolInferenceService().infer(tenant_id=app_model.tenant_id, agent_id=agent_id, slug=slug)
     except SkillToolInferenceError as exc:
         return {"code": exc.code, "message": exc.message}, exc.status_code
 
@@ -341,7 +335,7 @@ class AgentLogApi(Resource):
         """Get agent logs"""
         args = AgentLogQuery.model_validate(request.args.to_dict(flat=True))
 
-        return AgentService.get_agent_logs(app_model, args.conversation_id, args.message_id, db.session())
+        return AgentService.get_agent_logs(app_model, args.conversation_id, args.message_id)
 
 
 @console_ns.route("/agent/<uuid:agent_id>/skills/upload")
@@ -357,7 +351,7 @@ class AgentSkillUploadByAgentApi(Resource):
     @with_current_user
     @with_current_tenant_id
     def post(self, tenant_id: str, current_user: Account, agent_id: UUID):
-        app_model = resolve_agent_runtime_app_model(tenant_id=tenant_id, agent_id=agent_id)
+        app_model = resolve_agent_app_model(tenant_id=tenant_id, agent_id=agent_id)
         return _upload_skill_for_app(current_user=current_user, app_model=app_model)
 
 
@@ -400,7 +394,7 @@ class AgentDriveFilesByAgentApi(Resource):
     @with_current_user
     @with_current_tenant_id
     def post(self, tenant_id: str, current_user: Account, agent_id: UUID):
-        app_model = resolve_agent_runtime_app_model(tenant_id=tenant_id, agent_id=agent_id)
+        app_model = resolve_agent_app_model(tenant_id=tenant_id, agent_id=agent_id)
         return _commit_drive_file_for_app(current_user=current_user, app_model=app_model, allow_node_id=False)
 
     @console_ns.doc("delete_agent_drive_file_by_agent")
@@ -413,7 +407,7 @@ class AgentDriveFilesByAgentApi(Resource):
     @with_current_user
     @with_current_tenant_id
     def delete(self, tenant_id: str, current_user: Account, agent_id: UUID):
-        app_model = resolve_agent_runtime_app_model(tenant_id=tenant_id, agent_id=agent_id)
+        app_model = resolve_agent_app_model(tenant_id=tenant_id, agent_id=agent_id)
         return _delete_drive_file_for_app(current_user=current_user, app_model=app_model, allow_node_id=False)
 
 
@@ -460,7 +454,7 @@ class AgentSkillByAgentApi(Resource):
     @with_current_user
     @with_current_tenant_id
     def delete(self, tenant_id: str, current_user: Account, agent_id: UUID, slug: str):
-        app_model = resolve_agent_runtime_app_model(tenant_id=tenant_id, agent_id=agent_id)
+        app_model = resolve_agent_app_model(tenant_id=tenant_id, agent_id=agent_id)
         return _delete_skill_for_app(current_user=current_user, app_model=app_model, slug=slug, allow_node_id=False)
 
 
@@ -500,7 +494,7 @@ class AgentSkillInferToolsByAgentApi(Resource):
     @account_initialization_required
     @with_current_tenant_id
     def post(self, tenant_id: str, agent_id: UUID, slug: str):
-        app_model = resolve_agent_runtime_app_model(tenant_id=tenant_id, agent_id=agent_id)
+        app_model = resolve_agent_app_model(tenant_id=tenant_id, agent_id=agent_id)
         return _infer_skill_tools_for_app(app_model=app_model, slug=slug)
 
 

api/controllers/console/app/agent_app_feature.py

@@ -17,7 +17,7 @@ from pydantic import BaseModel, Field
 from controllers.common.fields import SimpleResultResponse
 from controllers.common.schema import register_response_schema_models, register_schema_models
 from controllers.console import console_ns
-from controllers.console.agent.app_helpers import resolve_agent_runtime_app_model
+from controllers.console.agent.app_helpers import resolve_agent_app_model
 from controllers.console.wraps import (
     RBACPermission,
     RBACResourceScope,
@@ -30,6 +30,7 @@ from controllers.console.wraps import (
 )
 from events.app_event import app_model_config_was_updated
 from extensions.ext_database import db
+from libs.helper import dump_response
 from libs.login import login_required
 from models import Account
 from models.agent_config_entities import (
@@ -86,16 +87,16 @@ class AgentAppFeatureConfigResource(Resource):
     @with_current_user
     @with_current_tenant_id
     def post(self, tenant_id: str, current_user: Account, agent_id: UUID):
-        app_model = resolve_agent_runtime_app_model(tenant_id=tenant_id, agent_id=agent_id)
+        app_model = resolve_agent_app_model(tenant_id=tenant_id, agent_id=agent_id)
         args = AgentAppFeaturesPayload.model_validate(console_ns.payload or {})
 
         new_app_model_config = AgentAppFeatureConfigService.update_features(
             app_model=app_model,
             account=current_user,
             config=args.model_dump(exclude_none=True),
-            session=db.session(),
+            session=db.session,
         )
 
         app_model_config_was_updated.send(app_model, app_model_config=new_app_model_config)
 
-        return SimpleResultResponse(result="success").model_dump(mode="json")
+        return dump_response(SimpleResultResponse, {"result": "success"})

api/controllers/console/app/agent_app_sandbox.py

@@ -22,10 +22,9 @@ from controllers.common.schema import (
     register_schema_models,
 )
 from controllers.console import console_ns
-from controllers.console.agent.app_helpers import resolve_agent_runtime_app_model
+from controllers.console.agent.app_helpers import resolve_agent_app_model
 from controllers.console.app.wraps import get_app_model
 from controllers.console.wraps import account_initialization_required, setup_required, with_current_tenant_id
-from extensions.ext_database import db
 from fields.base import ResponseModel
 from libs.login import login_required
 from models.model import App, AppMode
@@ -45,10 +44,6 @@ class AgentSandboxListQuery(BaseModel):
     path: str = Field(default=".", description="Directory path relative to the sandbox workspace")
 
 
-class AgentSandboxInfoQuery(BaseModel):
-    conversation_id: str = Field(min_length=1, description="Agent App conversation ID")
-
-
 class AgentSandboxFileQuery(BaseModel):
     conversation_id: str = Field(min_length=1, description="Agent App conversation ID")
     path: str = Field(min_length=1, description="File path relative to the sandbox workspace")
@@ -96,11 +91,6 @@ class SandboxListResponse(ResponseModel):
     truncated: bool = False
 
 
-class SandboxInfoResponse(ResponseModel):
-    session_id: str
-    workspace_cwd: str
-
-
 class SandboxReadResponse(ResponseModel):
     path: str
     size: int | None = None
@@ -109,8 +99,14 @@ class SandboxReadResponse(ResponseModel):
     text: str | None = None
 
 
+class SandboxToolFileResponse(ResponseModel):
+    transfer_method: Literal["tool_file"] = "tool_file"
+    reference: str
+
+
 class SandboxUploadResponse(ResponseModel):
-    url: str
+    path: str
+    file: SandboxToolFileResponse
 
 
 register_schema_models(
@@ -118,13 +114,7 @@ register_schema_models(
     AgentSandboxUploadPayload,
     WorkflowAgentSandboxUploadPayload,
 )
-register_response_schema_models(
-    console_ns,
-    SandboxInfoResponse,
-    SandboxListResponse,
-    SandboxReadResponse,
-    SandboxUploadResponse,
-)
+register_response_schema_models(console_ns, SandboxListResponse, SandboxReadResponse, SandboxUploadResponse)
 
 
 def _handle(exc: Exception) -> tuple[dict[str, object], int]:
@@ -143,30 +133,6 @@ def _handle(exc: Exception) -> tuple[dict[str, object], int]:
     raise exc
 
 
-@console_ns.route("/agent/<uuid:agent_id>/sandbox")
-class AgentAppSandboxInfoResource(Resource):
-    @console_ns.doc("get_agent_app_sandbox_info")
-    @console_ns.doc(description="Get basic information for an Agent App conversation sandbox")
-    @console_ns.doc(params={"agent_id": "Agent ID", **query_params_from_model(AgentSandboxInfoQuery)})
-    @console_ns.response(200, "Sandbox information returned", console_ns.models[SandboxInfoResponse.__name__])
-    @setup_required
-    @login_required
-    @account_initialization_required
-    @with_current_tenant_id
-    def get(self, tenant_id: str, agent_id: UUID):
-        app_model = resolve_agent_runtime_app_model(tenant_id=tenant_id, agent_id=agent_id)
-        query = query_params_from_request(AgentSandboxInfoQuery)
-        try:
-            result = AgentAppSandboxService().get_info(
-                tenant_id=tenant_id,
-                app_id=app_model.id,
-                conversation_id=query.conversation_id,
-            )
-        except Exception as exc:
-            return _handle(exc)
-        return result.model_dump()
-
-
 @console_ns.route("/agent/<uuid:agent_id>/sandbox/files")
 class AgentAppSandboxListResource(Resource):
     @console_ns.doc("list_agent_app_sandbox_files")
@@ -178,7 +144,7 @@ class AgentAppSandboxListResource(Resource):
     @account_initialization_required
     @with_current_tenant_id
     def get(self, tenant_id: str, agent_id: UUID):
-        app_model = resolve_agent_runtime_app_model(tenant_id=tenant_id, agent_id=agent_id)
+        app_model = resolve_agent_app_model(tenant_id=tenant_id, agent_id=agent_id)
         query = query_params_from_request(AgentSandboxListQuery)
         try:
             result = AgentAppSandboxService().list_files(
@@ -203,7 +169,7 @@ class AgentAppSandboxReadResource(Resource):
     @account_initialization_required
     @with_current_tenant_id
     def get(self, tenant_id: str, agent_id: UUID):
-        app_model = resolve_agent_runtime_app_model(tenant_id=tenant_id, agent_id=agent_id)
+        app_model = resolve_agent_app_model(tenant_id=tenant_id, agent_id=agent_id)
         query = query_params_from_request(AgentSandboxFileQuery)
         try:
             result = AgentAppSandboxService().read_file(
@@ -220,7 +186,7 @@ class AgentAppSandboxReadResource(Resource):
 @console_ns.route("/agent/<uuid:agent_id>/sandbox/files/upload")
 class AgentAppSandboxUploadResource(Resource):
     @console_ns.doc("upload_agent_app_sandbox_file")
-    @console_ns.doc(description="Upload one Agent App sandbox file and return a signed download URL")
+    @console_ns.doc(description="Upload one Agent App sandbox file as a Dify ToolFile mapping")
     @console_ns.expect(console_ns.models[AgentSandboxUploadPayload.__name__])
     @console_ns.response(200, "Uploaded", console_ns.models[SandboxUploadResponse.__name__])
     @setup_required
@@ -228,7 +194,7 @@ class AgentAppSandboxUploadResource(Resource):
     @account_initialization_required
     @with_current_tenant_id
     def post(self, tenant_id: str, agent_id: UUID):
-        app_model = resolve_agent_runtime_app_model(tenant_id=tenant_id, agent_id=agent_id)
+        app_model = resolve_agent_app_model(tenant_id=tenant_id, agent_id=agent_id)
         payload = AgentSandboxUploadPayload.model_validate(request.get_json(silent=True) or {})
         try:
             result = AgentAppSandboxService().upload_file(
@@ -270,7 +236,6 @@ class WorkflowAgentSandboxListResource(Resource):
                 node_id=node_id,
                 node_execution_id=query.node_execution_id,
                 path=query.path,
-                session=db.session(),
             )
         except Exception as exc:
             return _handle(exc)
@@ -307,7 +272,6 @@ class WorkflowAgentSandboxReadResource(Resource):
                 node_id=node_id,
                 node_execution_id=query.node_execution_id,
                 path=query.path,
-                session=db.session(),
             )
         except Exception as exc:
             return _handle(exc)
@@ -319,7 +283,7 @@ class WorkflowAgentSandboxReadResource(Resource):
 )
 class WorkflowAgentSandboxUploadResource(Resource):
     @console_ns.doc("upload_workflow_agent_sandbox_file")
-    @console_ns.doc(description="Upload one workflow Agent sandbox file and return a signed download URL")
+    @console_ns.doc(description="Upload one workflow Agent sandbox file as a Dify ToolFile mapping")
     @console_ns.expect(console_ns.models[WorkflowAgentSandboxUploadPayload.__name__])
     @console_ns.response(200, "Uploaded", console_ns.models[SandboxUploadResponse.__name__])
     @setup_required
@@ -337,7 +301,6 @@ class WorkflowAgentSandboxUploadResource(Resource):
                 node_id=node_id,
                 node_execution_id=payload.node_execution_id,
                 path=payload.path,
-                session=db.session(),
             )
         except Exception as exc:
             return _handle(exc)

api/controllers/console/app/agent_drive_inspector.py

@@ -25,10 +25,9 @@ from controllers.common.schema import (
     register_response_schema_models,
 )
 from controllers.console import console_ns
-from controllers.console.agent.app_helpers import resolve_agent_runtime_app_model
+from controllers.console.agent.app_helpers import resolve_agent_app_model
 from controllers.console.app.wraps import get_app_model
 from controllers.console.wraps import account_initialization_required, setup_required, with_current_tenant_id
-from extensions.ext_database import db
 from fields.base import ResponseModel
 from libs.login import login_required
 from models.model import App, AppMode
@@ -148,7 +147,7 @@ def _resolve_agent_id(app_model: App, node_id: str | None) -> str | None:
     """Agent identity for the drive: app-bound agent, or the workflow node binding."""
     if node_id:
         return AgentComposerService.resolve_workflow_node_agent_id(
-            tenant_id=app_model.tenant_id, app_id=app_model.id, node_id=node_id, session=db.session()
+            tenant_id=app_model.tenant_id, app_id=app_model.id, node_id=node_id
         )
     return app_model.bound_agent_id
 
@@ -183,11 +182,9 @@ class AgentDriveListByAgentApi(Resource):
     @with_current_tenant_id
     def get(self, tenant_id: str, agent_id: UUID):
         query = query_params_from_request(AgentDriveListByAgentQuery)
-        resolve_agent_runtime_app_model(tenant_id=tenant_id, agent_id=agent_id)
+        resolve_agent_app_model(tenant_id=tenant_id, agent_id=agent_id)
         try:
-            items = AgentDriveService().manifest(
-                tenant_id=tenant_id, agent_id=str(agent_id), prefix=query.prefix, session=db.session()
-            )
+            items = AgentDriveService().manifest(tenant_id=tenant_id, agent_id=str(agent_id), prefix=query.prefix)
         except AgentDriveError as exc:
             return _handle(exc)
         return {"items": [{k: v for k, v in item.items() if k != "file_id"} for item in items]}
@@ -204,9 +201,9 @@ class AgentDriveSkillListByAgentApi(Resource):
     @account_initialization_required
     @with_current_tenant_id
     def get(self, tenant_id: str, agent_id: UUID):
-        resolve_agent_runtime_app_model(tenant_id=tenant_id, agent_id=agent_id)
+        resolve_agent_app_model(tenant_id=tenant_id, agent_id=agent_id)
         try:
-            items = AgentDriveService().list_skills(tenant_id=tenant_id, agent_id=str(agent_id), session=db.session())
+            items = AgentDriveService().list_skills(tenant_id=tenant_id, agent_id=str(agent_id))
         except AgentDriveError as exc:
             return _handle(exc)
         return {"items": items}
@@ -223,14 +220,13 @@ class AgentDriveSkillInspectByAgentApi(Resource):
     @account_initialization_required
     @with_current_tenant_id
     def get(self, tenant_id: str, agent_id: UUID, skill_path: str):
-        resolve_agent_runtime_app_model(tenant_id=tenant_id, agent_id=agent_id)
+        resolve_agent_app_model(tenant_id=tenant_id, agent_id=agent_id)
         try:
             return _json_response(
                 AgentDriveService().inspect_skill(
                     tenant_id=tenant_id,
                     agent_id=str(agent_id),
                     skill_path=skill_path,
-                    session=db.session(),
                 )
             )
         except AgentDriveError as exc:
@@ -249,11 +245,9 @@ class AgentDrivePreviewByAgentApi(Resource):
     @with_current_tenant_id
     def get(self, tenant_id: str, agent_id: UUID):
         query = query_params_from_request(AgentDriveFileByAgentQuery)
-        resolve_agent_runtime_app_model(tenant_id=tenant_id, agent_id=agent_id)
+        resolve_agent_app_model(tenant_id=tenant_id, agent_id=agent_id)
         try:
-            return AgentDriveService().preview(
-                tenant_id=tenant_id, agent_id=str(agent_id), key=query.key, session=db.session()
-            )
+            return AgentDriveService().preview(tenant_id=tenant_id, agent_id=str(agent_id), key=query.key)
         except AgentDriveError as exc:
             return _handle(exc)
 
@@ -270,11 +264,9 @@ class AgentDriveDownloadByAgentApi(Resource):
     @with_current_tenant_id
     def get(self, tenant_id: str, agent_id: UUID):
         query = query_params_from_request(AgentDriveFileByAgentQuery)
-        resolve_agent_runtime_app_model(tenant_id=tenant_id, agent_id=agent_id)
+        resolve_agent_app_model(tenant_id=tenant_id, agent_id=agent_id)
         try:
-            url = AgentDriveService().download_url(
-                tenant_id=tenant_id, agent_id=str(agent_id), key=query.key, session=db.session()
-            )
+            url = AgentDriveService().download_url(tenant_id=tenant_id, agent_id=str(agent_id), key=query.key)
         except AgentDriveError as exc:
             return _handle(exc)
         return {"url": url}
@@ -296,9 +288,7 @@ class AgentDriveListApi(Resource):
         if not agent_id:
             return _agent_not_bound()
         try:
-            items = AgentDriveService().manifest(
-                tenant_id=app_model.tenant_id, agent_id=agent_id, prefix=query.prefix, session=db.session()
-            )
+            items = AgentDriveService().manifest(tenant_id=app_model.tenant_id, agent_id=agent_id, prefix=query.prefix)
         except AgentDriveError as exc:
             return _handle(exc)
         # the inner manifest exposes file_id for agent-side pulls; the console
@@ -322,9 +312,7 @@ class AgentDriveSkillListApi(Resource):
         if not agent_id:
             return _agent_not_bound()
         try:
-            items = AgentDriveService().list_skills(
-                tenant_id=app_model.tenant_id, agent_id=agent_id, session=db.session()
-            )
+            items = AgentDriveService().list_skills(tenant_id=app_model.tenant_id, agent_id=agent_id)
         except AgentDriveError as exc:
             return _handle(exc)
         return {"items": items}
@@ -357,7 +345,6 @@ class AgentDriveSkillInspectApi(Resource):
                     tenant_id=app_model.tenant_id,
                     agent_id=agent_id,
                     skill_path=skill_path,
-                    session=db.session(),
                 )
             )
         except AgentDriveError as exc:
@@ -380,9 +367,7 @@ class AgentDrivePreviewApi(Resource):
         if not agent_id:
             return _agent_not_bound()
         try:
-            return AgentDriveService().preview(
-                tenant_id=app_model.tenant_id, agent_id=agent_id, key=query.key, session=db.session()
-            )
+            return AgentDriveService().preview(tenant_id=app_model.tenant_id, agent_id=agent_id, key=query.key)
         except AgentDriveError as exc:
             return _handle(exc)
 
@@ -403,9 +388,7 @@ class AgentDriveDownloadApi(Resource):
         if not agent_id:
             return _agent_not_bound()
         try:
-            url = AgentDriveService().download_url(
-                tenant_id=app_model.tenant_id, agent_id=agent_id, key=query.key, session=db.session()
-            )
+            url = AgentDriveService().download_url(tenant_id=app_model.tenant_id, agent_id=agent_id, key=query.key)
         except AgentDriveError as exc:
             return _handle(exc)
         return {"url": url}

api/controllers/console/app/annotation.py

@@ -1,11 +1,9 @@
 from typing import Any, Literal
 from uuid import UUID
 
-from flask import abort, request
+from flask import abort, make_response, request
 from flask_restx import Resource
 from pydantic import BaseModel, Field, TypeAdapter, field_validator
-from sqlalchemy import select
-from werkzeug.exceptions import NotFound
 
 from controllers.common.errors import NoFileUploadedError, TooManyFilesError
 from controllers.common.schema import query_params_from_model, register_response_schema_models, register_schema_models
@@ -28,14 +26,11 @@ from fields.annotation_fields import (
     AnnotationExportList,
     AnnotationHitHistory,
     AnnotationHitHistoryList,
-    AnnotationJobStatusDetailResponse,
-    AnnotationJobStatusResponse,
     AnnotationList,
 )
 from fields.base import ResponseModel
-from libs.helper import dump_response, uuid_value
-from libs.login import current_account_with_tenant, login_required
-from models.model import App
+from libs.helper import uuid_value
+from libs.login import login_required
 from services.annotation_service import (
     AppAnnotationService,
     EnableAnnotationArgs,
@@ -43,17 +38,6 @@ from services.annotation_service import (
     UpdateAnnotationSettingArgs,
     UpsertAnnotationArgs,
 )
-from services.app_ref_service import AppRef, AppRefService
-
-
-def _get_app_ref(app_id: str) -> AppRef:
-    _, current_tenant_id = current_account_with_tenant()
-    app = db.session.scalar(
-        select(App).where(App.id == app_id, App.tenant_id == current_tenant_id, App.status == "normal").limit(1)
-    )
-    if app is None:
-        raise NotFound("App not found")
-    return AppRefService.create_app_ref(app)
 
 
 class AnnotationReplyPayload(BaseModel):
@@ -115,23 +99,23 @@ class AnnotationFilePayload(BaseModel):
         return uuid_value(value)
 
 
-class AnnotationSettingEmbeddingModelResponse(ResponseModel):
+class AnnotationJobStatusResponse(ResponseModel):
+    job_id: str | None = None
+    job_status: str | None = None
+    error_msg: str | None = None
+    record_count: int | None = None
+
+
+class AnnotationEmbeddingModelResponse(ResponseModel):
     embedding_provider_name: str | None = None
     embedding_model_name: str | None = None
 
 
 class AnnotationSettingResponse(ResponseModel):
-    enabled: bool
     id: str | None = None
+    enabled: bool
     score_threshold: float | None = None
-    embedding_model: AnnotationSettingEmbeddingModelResponse | None = None
-
-
-class AnnotationBatchImportResponse(ResponseModel):
-    job_id: str | None = None
-    job_status: str | None = None
-    record_count: int | None = None
-    error_msg: str | None = None
+    embedding_model: AnnotationEmbeddingModelResponse | None = None
 
 
 register_schema_models(
@@ -158,10 +142,7 @@ register_response_schema_models(
     AnnotationHitHistory,
     AnnotationHitHistoryList,
     AnnotationJobStatusResponse,
-    AnnotationJobStatusDetailResponse,
-    AnnotationSettingEmbeddingModelResponse,
     AnnotationSettingResponse,
-    AnnotationBatchImportResponse,
 )
 
 
@@ -191,7 +172,7 @@ class AnnotationReplyActionApi(Resource):
                 result = AppAnnotationService.enable_app_annotation(enable_args, str(app_id))
             case "disable":
                 result = AppAnnotationService.disable_app_annotation(str(app_id))
-        return dump_response(AnnotationJobStatusResponse, result), 200
+        return result, 200
 
 
 @console_ns.route("/apps/<uuid:app_id>/annotation-setting")
@@ -211,8 +192,8 @@ class AppAnnotationSettingDetailApi(Resource):
     @edit_permission_required
     @rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_VIEW_LAYOUT)
     def get(self, app_id: UUID):
-        result = AppAnnotationService.get_app_annotation_setting_by_app_id(str(app_id), session=db.session())
-        return dump_response(AnnotationSettingResponse, result), 200
+        result = AppAnnotationService.get_app_annotation_setting_by_app_id(str(app_id))
+        return result, 200
 
 
 @console_ns.route("/apps/<uuid:app_id>/annotation-settings/<uuid:annotation_setting_id>")
@@ -235,9 +216,9 @@ class AppAnnotationSettingUpdateApi(Resource):
 
         setting_args: UpdateAnnotationSettingArgs = {"score_threshold": args.score_threshold}
         result = AppAnnotationService.update_app_annotation_setting(
-            str(app_id), annotation_setting_id_str, setting_args, session=db.session()
+            str(app_id), annotation_setting_id_str, setting_args
         )
-        return dump_response(AnnotationSettingResponse, result), 200
+        return result, 200
 
 
 @console_ns.route("/apps/<uuid:app_id>/annotation-reply/<string:action>/status/<uuid:job_id>")
@@ -246,7 +227,9 @@ class AnnotationReplyActionStatusApi(Resource):
     @console_ns.doc(description="Get status of annotation reply action job")
     @console_ns.doc(params={"app_id": "Application ID", "job_id": "Job ID", "action": "Action type"})
     @console_ns.response(
-        200, "Job status retrieved successfully", console_ns.models[AnnotationJobStatusDetailResponse.__name__]
+        200,
+        "Job status retrieved successfully",
+        console_ns.models[AnnotationJobStatusResponse.__name__],
     )
     @console_ns.response(403, "Insufficient permissions")
     @setup_required
@@ -268,9 +251,7 @@ class AnnotationReplyActionStatusApi(Resource):
             app_annotation_error_key = f"{action}_app_annotation_error_{job_id_str}"
             error_msg = redis_client.get(app_annotation_error_key).decode()
 
-        return AnnotationJobStatusDetailResponse(
-            job_id=job_id_str, job_status=job_status, error_msg=error_msg
-        ).model_dump(mode="json"), 200
+        return {"job_id": job_id_str, "job_status": job_status, "error_msg": error_msg}, 200
 
 
 @console_ns.route("/apps/<uuid:app_id>/annotations")
@@ -292,13 +273,16 @@ class AnnotationApi(Resource):
         limit = args.limit
         keyword = args.keyword
 
-        annotation_list, total = AppAnnotationService.get_annotation_list_by_app_id(
-            str(app_id), page, limit, keyword, session=db.session()
-        )
+        annotation_list, total = AppAnnotationService.get_annotation_list_by_app_id(str(app_id), page, limit, keyword)
         annotation_models = TypeAdapter(list[Annotation]).validate_python(annotation_list, from_attributes=True)
-        return AnnotationList(
-            data=annotation_models, has_more=len(annotation_list) == limit, limit=limit, total=total, page=page
-        ).model_dump(mode="json"), 200
+        response = AnnotationList(
+            data=annotation_models,
+            has_more=len(annotation_list) == limit,
+            limit=limit,
+            total=total,
+            page=page,
+        )
+        return response.model_dump(mode="json"), 200
 
     @console_ns.doc("create_annotation")
     @console_ns.doc(description="Create a new annotation for an app")
@@ -323,10 +307,8 @@ class AnnotationApi(Resource):
             upsert_args["message_id"] = args.message_id
         if args.question is not None:
             upsert_args["question"] = args.question
-        annotation = AppAnnotationService.up_insert_app_annotation_from_message(
-            upsert_args, str(app_id), session=db.session()
-        )
-        return dump_response(Annotation, annotation), 201
+        annotation = AppAnnotationService.up_insert_app_annotation_from_message(upsert_args, str(app_id))
+        return Annotation.model_validate(annotation, from_attributes=True).model_dump(mode="json")
 
     @setup_required
     @login_required
@@ -348,12 +330,11 @@ class AnnotationApi(Resource):
                     "message": "annotation_ids are required if the parameter is provided.",
                 }, 400
 
-            app_ref = _get_app_ref(str(app_id))
-            AppAnnotationService.delete_app_annotations_in_batch(app_ref, annotation_ids, session=db.session())
+            AppAnnotationService.delete_app_annotations_in_batch(str(app_id), annotation_ids)
             return "", 204
         # If no annotation_ids are provided, handle clearing all annotations
         else:
-            AppAnnotationService.clear_all_annotations(str(app_id), session=db.session())
+            AppAnnotationService.clear_all_annotations(str(app_id))
             return "", 204
 
 
@@ -374,16 +355,16 @@ class AnnotationExportApi(Resource):
     @edit_permission_required
     @rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_VIEW_LAYOUT)
     def get(self, app_id: UUID):
-        annotation_list = AppAnnotationService.export_annotation_list_by_app_id(str(app_id), session=db.session())
+        annotation_list = AppAnnotationService.export_annotation_list_by_app_id(str(app_id))
         annotation_models = TypeAdapter(list[Annotation]).validate_python(annotation_list, from_attributes=True)
-        return (
-            AnnotationExportList(data=annotation_models).model_dump(mode="json"),
-            200,
-            {
-                "Content-Type": "application/json; charset=utf-8",
-                "X-Content-Type-Options": "nosniff",
-            },
-        )
+        response_data = AnnotationExportList(data=annotation_models).model_dump(mode="json")
+
+        # Create response with secure headers for CSV export
+        response = make_response(response_data, 200)
+        response.headers["Content-Type"] = "application/json; charset=utf-8"
+        response.headers["X-Content-Type-Options"] = "nosniff"
+
+        return response
 
 
 @console_ns.route("/apps/<uuid:app_id>/annotations/<uuid:annotation_id>")
@@ -408,9 +389,9 @@ class AnnotationUpdateDeleteApi(Resource):
             update_args["answer"] = args.answer
         if args.question is not None:
             update_args["question"] = args.question
-        app_ref = _get_app_ref(str(app_id))
-        annotation_ref = AppRefService.create_annotation_ref(app_ref, str(annotation_id))
-        annotation = AppAnnotationService.update_app_annotation_directly(update_args, annotation_ref, db.session())
+        annotation = AppAnnotationService.update_app_annotation_directly(
+            update_args, str(app_id), str(annotation_id), db.session
+        )
         return Annotation.model_validate(annotation, from_attributes=True).model_dump(mode="json")
 
     @setup_required
@@ -420,9 +401,7 @@ class AnnotationUpdateDeleteApi(Resource):
     @rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_EDIT)
     @console_ns.response(204, "Annotation deleted successfully")
     def delete(self, app_id: UUID, annotation_id: UUID):
-        app_ref = _get_app_ref(str(app_id))
-        annotation_ref = AppRefService.create_annotation_ref(app_ref, str(annotation_id))
-        AppAnnotationService.delete_app_annotation(annotation_ref, db.session())
+        AppAnnotationService.delete_app_annotation(str(app_id), str(annotation_id), db.session)
         return "", 204
 
 
@@ -432,7 +411,9 @@ class AnnotationBatchImportApi(Resource):
     @console_ns.doc(description="Batch import annotations from CSV file with rate limiting and security checks")
     @console_ns.doc(params={"app_id": "Application ID"})
     @console_ns.response(
-        200, "Batch import started successfully", console_ns.models[AnnotationBatchImportResponse.__name__]
+        200,
+        "Batch import started successfully",
+        console_ns.models[AnnotationJobStatusResponse.__name__],
     )
     @console_ns.response(403, "Insufficient permissions")
     @console_ns.response(400, "No file uploaded or too many files")
@@ -479,10 +460,7 @@ class AnnotationBatchImportApi(Resource):
         if file_size == 0:
             raise ValueError("The uploaded file is empty")
 
-        return dump_response(
-            AnnotationBatchImportResponse,
-            AppAnnotationService.batch_import_app_annotations(str(app_id), file, session=db.session()),
-        )
+        return AppAnnotationService.batch_import_app_annotations(str(app_id), file)
 
 
 @console_ns.route("/apps/<uuid:app_id>/annotations/batch-import-status/<uuid:job_id>")
@@ -491,7 +469,9 @@ class AnnotationBatchImportStatusApi(Resource):
     @console_ns.doc(description="Get status of batch import job")
     @console_ns.doc(params={"app_id": "Application ID", "job_id": "Job ID"})
     @console_ns.response(
-        200, "Job status retrieved successfully", console_ns.models[AnnotationJobStatusDetailResponse.__name__]
+        200,
+        "Job status retrieved successfully",
+        console_ns.models[AnnotationJobStatusResponse.__name__],
     )
     @console_ns.response(403, "Insufficient permissions")
     @setup_required
@@ -511,9 +491,7 @@ class AnnotationBatchImportStatusApi(Resource):
             indexing_error_msg_key = f"app_annotation_batch_import_error_msg_{str(job_id)}"
             error_msg = redis_client.get(indexing_error_msg_key).decode()
 
-        return AnnotationJobStatusDetailResponse(
-            job_id=str(job_id), job_status=job_status, error_msg=error_msg
-        ).model_dump(mode="json"), 200
+        return {"job_id": job_id, "job_status": job_status, "error_msg": error_msg}, 200
 
 
 @console_ns.route("/apps/<uuid:app_id>/annotations/<uuid:annotation_id>/hit-histories")
@@ -536,17 +514,17 @@ class AnnotationHitHistoryListApi(Resource):
     def get(self, app_id: UUID, annotation_id: UUID):
         page = request.args.get("page", default=1, type=int)
         limit = request.args.get("limit", default=20, type=int)
-        app_ref = _get_app_ref(str(app_id))
-        annotation_ref = AppRefService.create_annotation_ref(app_ref, str(annotation_id))
         annotation_hit_history_list, total = AppAnnotationService.get_annotation_hit_histories(
-            annotation_ref,
-            page,
-            limit,
-            session=db.session(),
+            str(app_id), str(annotation_id), page, limit
         )
         history_models = TypeAdapter(list[AnnotationHitHistory]).validate_python(
             annotation_hit_history_list, from_attributes=True
         )
-        return AnnotationHitHistoryList(
-            data=history_models, has_more=len(annotation_hit_history_list) == limit, limit=limit, total=total, page=page
-        ).model_dump(mode="json")
+        response = AnnotationHitHistoryList(
+            data=history_models,
+            has_more=len(annotation_hit_history_list) == limit,
+            limit=limit,
+            total=total,
+            page=page,
+        )
+        return response.model_dump(mode="json")

api/controllers/console/app/app.py

@@ -1,4 +1,5 @@
 import logging
+import re
 import uuid
 from collections.abc import Sequence
 from datetime import datetime
@@ -9,6 +10,7 @@ from flask_restx import Resource
 from pydantic import AliasChoices, BaseModel, Field, computed_field, field_validator
 from sqlalchemy import select
 from sqlalchemy.orm import Session
+from werkzeug.datastructures import MultiDict
 from werkzeug.exceptions import BadRequest, NotFound
 
 from configs import dify_config
@@ -17,7 +19,6 @@ from controllers.common.fields import RedirectUrlResponse, SimpleResultResponse
 from controllers.common.helpers import FileInfo
 from controllers.common.schema import (
     query_params_from_model,
-    query_params_from_request,
     register_enum_models,
     register_response_schema_models,
     register_schema_models,
@@ -74,9 +75,10 @@ ALLOW_CREATE_APP_MODES = ["chat", "agent-chat", "advanced-chat", "workflow", "co
 register_enum_models(console_ns, IconType)
 
 _logger = logging.getLogger(__name__)
+_TAG_IDS_BRACKET_PATTERN = re.compile(r"^tag_ids\[(\d+)\]$")
+_CREATOR_IDS_BRACKET_PATTERN = re.compile(r"^creator_ids\[(\d+)\]$")
 AppListMode = Literal["completion", "chat", "advanced-chat", "workflow", "agent-chat", "agent", "channel", "all"]
 DEFAULT_APP_LIST_MODE: AppListMode = "all"
-APP_LIST_QUERY_ARRAY_FIELDS = ("tag_ids", "creator_ids")
 
 
 class AppListBaseQuery(BaseModel):
@@ -137,6 +139,34 @@ class StarredAppListQuery(AppListBaseQuery):
     pass
 
 
+def _normalize_app_list_query_args(query_args: MultiDict[str, str]) -> dict[str, str | list[str]]:
+    normalized: dict[str, str | list[str]] = {}
+    indexed_tag_ids: list[tuple[int, str]] = []
+    indexed_creator_ids: list[tuple[int, str]] = []
+
+    for key in query_args:
+        match = _TAG_IDS_BRACKET_PATTERN.fullmatch(key)
+        if match:
+            indexed_tag_ids.extend((int(match.group(1)), value) for value in query_args.getlist(key))
+            continue
+
+        match = _CREATOR_IDS_BRACKET_PATTERN.fullmatch(key)
+        if match:
+            indexed_creator_ids.extend((int(match.group(1)), value) for value in query_args.getlist(key))
+            continue
+
+        value = query_args.get(key)
+        if value is not None:
+            normalized[key] = value
+
+    if indexed_tag_ids:
+        normalized["tag_ids"] = [value for _, value in sorted(indexed_tag_ids)]
+    if indexed_creator_ids:
+        normalized["creator_ids"] = [value for _, value in sorted(indexed_creator_ids)]
+
+    return normalized
+
+
 class CreateAppPayload(BaseModel):
     name: str = Field(..., min_length=1, description="App name")
     description: str | None = Field(default=None, description="App description (max 400 chars)", max_length=400)
@@ -200,10 +230,13 @@ class AppTracePayload(BaseModel):
 
 
 class AppTraceResponse(ResponseModel):
-    enabled: bool = False
+    enabled: bool
     tracing_provider: str | None = None
 
 
+type JSONValue = Any
+
+
 class Tag(ResponseModel):
     id: str
     name: str
@@ -224,7 +257,7 @@ class WorkflowPartial(ResponseModel):
 
 
 class ModelConfigPartial(ResponseModel):
-    model: Any | None = Field(default=None, validation_alias=AliasChoices("model_dict", "model"))
+    model: JSONValue | None = Field(default=None, validation_alias=AliasChoices("model_dict", "model"))
     pre_prompt: str | None = None
     created_by: str | None = None
     created_at: int | None = None
@@ -239,52 +272,54 @@ class ModelConfigPartial(ResponseModel):
 
 class ModelConfig(ResponseModel):
     opening_statement: str | None = None
-    suggested_questions: Any | None = Field(
+    suggested_questions: JSONValue | None = Field(
         default=None, validation_alias=AliasChoices("suggested_questions_list", "suggested_questions")
     )
-    suggested_questions_after_answer: Any | None = Field(
+    suggested_questions_after_answer: JSONValue | None = Field(
         default=None,
         validation_alias=AliasChoices("suggested_questions_after_answer_dict", "suggested_questions_after_answer"),
     )
-    speech_to_text: Any | None = Field(
+    speech_to_text: JSONValue | None = Field(
         default=None, validation_alias=AliasChoices("speech_to_text_dict", "speech_to_text")
     )
-    text_to_speech: Any | None = Field(
+    text_to_speech: JSONValue | None = Field(
         default=None, validation_alias=AliasChoices("text_to_speech_dict", "text_to_speech")
     )
-    retriever_resource: Any | None = Field(
+    retriever_resource: JSONValue | None = Field(
         default=None, validation_alias=AliasChoices("retriever_resource_dict", "retriever_resource")
     )
-    annotation_reply: Any | None = Field(
+    annotation_reply: JSONValue | None = Field(
         default=None, validation_alias=AliasChoices("annotation_reply_dict", "annotation_reply")
     )
-    more_like_this: Any | None = Field(
+    more_like_this: JSONValue | None = Field(
         default=None, validation_alias=AliasChoices("more_like_this_dict", "more_like_this")
     )
-    sensitive_word_avoidance: Any | None = Field(
+    sensitive_word_avoidance: JSONValue | None = Field(
         default=None, validation_alias=AliasChoices("sensitive_word_avoidance_dict", "sensitive_word_avoidance")
     )
-    external_data_tools: Any | None = Field(
+    external_data_tools: JSONValue | None = Field(
         default=None, validation_alias=AliasChoices("external_data_tools_list", "external_data_tools")
     )
-    model: Any | None = Field(default=None, validation_alias=AliasChoices("model_dict", "model"))
-    user_input_form: Any | None = Field(
+    model: JSONValue | None = Field(default=None, validation_alias=AliasChoices("model_dict", "model"))
+    user_input_form: JSONValue | None = Field(
         default=None, validation_alias=AliasChoices("user_input_form_list", "user_input_form")
     )
     dataset_query_variable: str | None = None
     pre_prompt: str | None = None
-    agent_mode: Any | None = Field(default=None, validation_alias=AliasChoices("agent_mode_dict", "agent_mode"))
+    agent_mode: JSONValue | None = Field(default=None, validation_alias=AliasChoices("agent_mode_dict", "agent_mode"))
     prompt_type: str | None = None
-    chat_prompt_config: Any | None = Field(
+    chat_prompt_config: JSONValue | None = Field(
         default=None, validation_alias=AliasChoices("chat_prompt_config_dict", "chat_prompt_config")
     )
-    completion_prompt_config: Any | None = Field(
+    completion_prompt_config: JSONValue | None = Field(
         default=None, validation_alias=AliasChoices("completion_prompt_config_dict", "completion_prompt_config")
     )
-    dataset_configs: Any | None = Field(
+    dataset_configs: JSONValue | None = Field(
         default=None, validation_alias=AliasChoices("dataset_configs_dict", "dataset_configs")
     )
-    file_upload: Any | None = Field(default=None, validation_alias=AliasChoices("file_upload_dict", "file_upload"))
+    file_upload: JSONValue | None = Field(
+        default=None, validation_alias=AliasChoices("file_upload_dict", "file_upload")
+    )
     created_by: str | None = None
     created_at: int | None = None
     updated_by: str | None = None
@@ -296,7 +331,7 @@ class ModelConfig(ResponseModel):
         return to_timestamp(value)
 
 
-class AppDetailSiteResponse(ResponseModel):
+class Site(ResponseModel):
     access_token: str | None = Field(default=None, validation_alias="code")
     code: str | None = None
     title: str | None = None
@@ -310,7 +345,6 @@ class AppDetailSiteResponse(ResponseModel):
     customize_domain: str | None = None
     copyright: str | None = None
     privacy_policy: str | None = None
-    input_placeholder: str | None = None
     custom_disclaimer: str | None = None
     customize_token_strategy: str | None = None
     prompt_public: bool | None = None
@@ -405,7 +439,7 @@ class AppDetail(ResponseModel):
         alias="model_config",
     )
     workflow: WorkflowPartial | None = None
-    tracing: Any | None = None
+    tracing: JSONValue | None = None
     use_icon_as_answer_icon: bool | None = None
     created_by: str | None = None
     created_at: int | None = None
@@ -427,7 +461,7 @@ class AppDetailWithSite(AppDetail):
     api_base_url: str | None = None
     max_active_requests: int | None = None
     deleted_tools: list[DeletedTool] = Field(default_factory=list)
-    site: AppDetailSiteResponse | None = None
+    site: Site | None = None
     # For Agent App type: the roster Agent backing this app (None otherwise).
     bound_agent_id: str | None = None
     # For Agent App responses exposed through /agent.
@@ -451,16 +485,6 @@ class AppExportResponse(ResponseModel):
     data: str
 
 
-class AppImportResponse(ResponseModel):
-    id: str
-    status: ImportStatus
-    app_id: str | None = None
-    app_mode: str | None = None
-    current_dsl_version: str
-    imported_dsl_version: str = ""
-    error: str = ""
-
-
 def _enrich_app_list_items(session: Session, *, apps: Sequence[App], tenant_id: str) -> None:
     if FeatureService.get_system_features().webapp_auth.enabled:
         app_ids = [str(app.id) for app in apps]
@@ -503,9 +527,7 @@ def _enrich_app_list_items(session: Session, *, apps: Sequence[App], tenant_id:
 
 
 register_enum_models(console_ns, RetrievalMethod, WorkflowExecutionStatus, DatasetPermissionEnum)
-register_response_schema_models(
-    console_ns, RedirectUrlResponse, SimpleResultResponse, AppImportResponse, AppTraceResponse
-)
+register_response_schema_models(console_ns, AppTraceResponse, RedirectUrlResponse, SimpleResultResponse)
 
 register_schema_models(
     console_ns,
@@ -524,7 +546,7 @@ register_schema_models(
     WorkflowPartial,
     ModelConfigPartial,
     ModelConfig,
-    AppDetailSiteResponse,
+    Site,
     DeletedTool,
     AppDetail,
     AppExportResponse,
@@ -569,7 +591,7 @@ class AppListApi(Resource):
     @with_current_tenant_id
     def get(self, current_tenant_id: str, current_user_id: str, session: Session):
         """Get app list"""
-        args = query_params_from_request(AppListQuery, list_fields=APP_LIST_QUERY_ARRAY_FIELDS)
+        args = AppListQuery.model_validate(_normalize_app_list_query_args(request.args))
         params = AppListParams(
             page=args.page,
             limit=args.limit,
@@ -584,7 +606,6 @@ class AppListApi(Resource):
         permissions = enterprise_rbac_service.RBACService.MyPermissions.get(
             str(current_tenant_id),
             current_user_id,
-            session=db.session(),
         )
         if dify_config.RBAC_ENABLED:
             access_filter = resolve_app_access_filter(
@@ -596,10 +617,10 @@ class AppListApi(Resource):
 
         # get app list
         app_service = AppService()
-        app_pagination = app_service.get_paginate_apps(current_user_id, current_tenant_id, params, session)
+        app_pagination = app_service.get_paginate_apps(current_user_id, current_tenant_id, params, db.session)
         if not app_pagination:
-            response = AppPagination(page=args.page, limit=args.limit, total=0, has_more=False, data=[])
-            return response.model_dump(mode="json"), 200
+            empty = AppPagination(page=args.page, limit=args.limit, total=0, has_more=False, data=[])
+            return empty.model_dump(mode="json"), 200
 
         app_ids = [str(app.id) for app in app_pagination.items]
         permission_keys_map = permissions.app.permission_keys_by_resource_ids(app_ids)
@@ -644,12 +665,11 @@ class AppListApi(Resource):
         )
 
         app_service = AppService()
-        app = app_service.create_app(current_tenant_id, params, current_user, session=db.session())
+        app = app_service.create_app(current_tenant_id, params, current_user)
         permission_keys_map = enterprise_rbac_service.RBACService.AppPermissions.batch_get(
             str(current_tenant_id),
             current_user.id,
             [str(app.id)],
-            session=db.session(),
         )
         app_detail = AppDetailWithSite.model_validate(app, from_attributes=True).model_copy(
             update={"permission_keys": permission_keys_map.get(str(app.id), [])}
@@ -671,7 +691,7 @@ class StarredAppListApi(Resource):
     @with_current_user_id
     @with_current_tenant_id
     def get(self, current_tenant_id: str, current_user_id: str, session: Session):
-        args = query_params_from_request(StarredAppListQuery, list_fields=APP_LIST_QUERY_ARRAY_FIELDS)
+        args = StarredAppListQuery.model_validate(_normalize_app_list_query_args(request.args))
         params = StarredAppListParams(
             page=args.page,
             limit=args.limit,
@@ -683,13 +703,15 @@ class StarredAppListApi(Resource):
             is_created_by_me=args.is_created_by_me,
         )
 
-        app_pagination = AppService().get_paginate_starred_apps(current_user_id, current_tenant_id, params, session)
+        app_pagination = AppService().get_paginate_starred_apps(current_user_id, current_tenant_id, params, db.session)
         if not app_pagination:
             empty = AppPagination(page=args.page, limit=args.limit, total=0, has_more=False, data=[])
             return empty.model_dump(mode="json"), 200
 
         _enrich_app_list_items(session, apps=app_pagination.items, tenant_id=current_tenant_id)
-        return AppPagination.model_validate(app_pagination, from_attributes=True).model_dump(mode="json"), 200
+
+        pagination_model = AppPagination.model_validate(app_pagination, from_attributes=True)
+        return pagination_model.model_dump(mode="json"), 200
 
 
 @console_ns.route("/apps/<uuid:app_id>/star")
@@ -708,7 +730,7 @@ class AppStarApi(Resource):
     @get_app_model(mode=None)
     def post(self, session: Session, current_user_id: str, app_model: App):
         AppService.star_app(session, app=app_model, account_id=current_user_id)
-        return SimpleResultResponse(result="success").model_dump(mode="json")
+        return dump_response(SimpleResultResponse, {"result": "success"})
 
     @console_ns.doc("unstar_app")
     @console_ns.doc(description="Remove the current account's star from an application")
@@ -724,7 +746,7 @@ class AppStarApi(Resource):
     @get_app_model(mode=None)
     def delete(self, session: Session, current_user_id: str, app_model: App):
         AppService.unstar_app(session, app=app_model, account_id=current_user_id)
-        return SimpleResultResponse(result="success").model_dump(mode="json")
+        return dump_response(SimpleResultResponse, {"result": "success"})
 
 
 @console_ns.route("/apps/<uuid:app_id>")
@@ -755,7 +777,6 @@ class AppApi(Resource):
             str(current_tenant_id),
             current_user.id,
             app_id=str(app_model.id),
-            session=db.session(),
         )
         permission_keys_map = permissions.app.permission_keys_by_resource_ids([str(app_model.id)])
 
@@ -792,8 +813,9 @@ class AppApi(Resource):
             "use_icon_as_answer_icon": args.use_icon_as_answer_icon or False,
             "max_active_requests": args.max_active_requests or 0,
         }
-        app_model = app_service.update_app(app_model, args_dict, session=db.session())
-        return dump_response(AppDetailWithSite, app_model)
+        app_model = app_service.update_app(app_model, args_dict)
+        response_model = AppDetailWithSite.model_validate(app_model, from_attributes=True)
+        return response_model.model_dump(mode="json")
 
     @console_ns.doc("delete_app")
     @console_ns.doc(description="Delete application")
@@ -809,7 +831,7 @@ class AppApi(Resource):
     def delete(self, app_model: App):
         """Delete app"""
         app_service = AppService()
-        app_service.delete_app(app_model, session=db.session())
+        app_service.delete_app(app_model)
 
         return "", 204
 
@@ -821,7 +843,6 @@ class AppCopyApi(Resource):
     @console_ns.doc(params={"app_id": "Application ID to copy"})
     @console_ns.expect(console_ns.models[CopyAppPayload.__name__])
     @console_ns.response(201, "App copied successfully", console_ns.models[AppDetailWithSite.__name__])
-    @console_ns.response(202, "App copy requires confirmation", console_ns.models[AppImportResponse.__name__])
     @console_ns.response(403, "Insufficient permissions")
     @setup_required
     @login_required
@@ -838,7 +859,7 @@ class AppCopyApi(Resource):
 
         with Session(db.engine, expire_on_commit=False) as session:
             import_service = AppDslService(session)
-            yaml_content = import_service.export_dsl(app_model=app_model, session=session, include_secret=True)
+            yaml_content = import_service.export_dsl(app_model=app_model, include_secret=True)
             result = import_service.import_app(
                 account=current_user,
                 import_mode=ImportMode.YAML_CONTENT,
@@ -851,10 +872,10 @@ class AppCopyApi(Resource):
             )
             if result.status == ImportStatus.FAILED:
                 session.rollback()
-                return dump_response(AppImportResponse, result), 400
+                return result.model_dump(mode="json"), 400
             if result.status == ImportStatus.PENDING:
                 session.rollback()
-                return dump_response(AppImportResponse, result), 202
+                return result.model_dump(mode="json"), 202
             session.commit()
 
             # Inherit web app permission from original app
@@ -880,7 +901,6 @@ class AppCopyApi(Resource):
             str(current_tenant_id),
             current_user.id,
             [str(app.id)],
-            session=db.session(),
         )
         response_model = AppDetailWithSite.model_validate(app, from_attributes=True).model_copy(
             update={"permission_keys": permission_keys_map.get(str(app.id), [])}
@@ -906,15 +926,14 @@ class AppExportApi(Resource):
         """Export app"""
         args = AppExportQuery.model_validate(request.args.to_dict(flat=True))
 
-        response = AppExportResponse(
+        payload = AppExportResponse(
             data=AppDslService.export_dsl(
                 app_model=app_model,
-                session=db.session(),
                 include_secret=args.include_secret,
                 workflow_id=args.workflow_id,
             )
         )
-        return response.model_dump(mode="json")
+        return payload.model_dump(mode="json")
 
 
 @console_ns.route("/apps/<uuid:app_id>/publish-to-creators-platform")
@@ -934,13 +953,13 @@ class AppPublishToCreatorsPlatformApi(Resource):
         if not dify_config.CREATORS_PLATFORM_FEATURES_ENABLED:
             return {"error": "Creators Platform features are not enabled"}, 403
 
-        dsl_content = AppDslService.export_dsl(app_model=app_model, session=db.session(), include_secret=False)
+        dsl_content = AppDslService.export_dsl(app_model=app_model, include_secret=False)
         dsl_bytes = dsl_content.encode("utf-8")
 
         claim_code = upload_dsl(dsl_bytes)
         redirect_url = get_redirect_url(current_user_id, claim_code)
 
-        return RedirectUrlResponse(redirect_url=redirect_url).model_dump(mode="json")
+        return {"redirect_url": redirect_url}
 
 
 @console_ns.route("/apps/<uuid:app_id>/name")
@@ -960,8 +979,9 @@ class AppNameApi(Resource):
         args = AppNamePayload.model_validate(console_ns.payload)
 
         app_service = AppService()
-        app_model = app_service.update_app_name(app_model, args.name, session=db.session())
-        return dump_response(AppDetail, app_model)
+        app_model = app_service.update_app_name(app_model, args.name)
+        response_model = AppDetail.model_validate(app_model, from_attributes=True)
+        return response_model.model_dump(mode="json")
 
 
 @console_ns.route("/apps/<uuid:app_id>/icon")
@@ -987,9 +1007,9 @@ class AppIconApi(Resource):
             args.icon or "",
             args.icon_background or "",
             args.icon_type,
-            session=db.session(),
         )
-        return dump_response(AppDetail, app_model)
+        response_model = AppDetail.model_validate(app_model, from_attributes=True)
+        return response_model.model_dump(mode="json")
 
 
 @console_ns.route("/apps/<uuid:app_id>/site-enable")
@@ -1010,8 +1030,9 @@ class AppSiteStatus(Resource):
         args = AppSiteStatusPayload.model_validate(console_ns.payload)
 
         app_service = AppService()
-        app_model = app_service.update_app_site_status(app_model, args.enable_site, session=db.session())
-        return dump_response(AppDetail, app_model)
+        app_model = app_service.update_app_site_status(app_model, args.enable_site)
+        response_model = AppDetail.model_validate(app_model, from_attributes=True)
+        return response_model.model_dump(mode="json")
 
 
 @console_ns.route("/apps/<uuid:app_id>/api-enable")
@@ -1032,8 +1053,9 @@ class AppApiStatus(Resource):
         args = AppApiStatusPayload.model_validate(console_ns.payload)
 
         app_service = AppService()
-        app_model = app_service.update_app_api_status(app_model, args.enable_api, session=db.session())
-        return dump_response(AppDetail, app_model)
+        app_model = app_service.update_app_api_status(app_model, args.enable_api)
+        response_model = AppDetail.model_validate(app_model, from_attributes=True)
+        return response_model.model_dump(mode="json")
 
 
 @console_ns.route("/apps/<uuid:app_id>/trace")
@@ -1056,7 +1078,7 @@ class AppTraceApi(Resource):
         """Get app trace"""
         app_trace_config = OpsTraceManager.get_app_tracing_config(app_model.id, session)
 
-        return dump_response(AppTraceResponse, app_trace_config)
+        return app_trace_config
 
     @console_ns.doc("update_app_trace")
     @console_ns.doc(description="Update app tracing configuration")
@@ -1084,4 +1106,4 @@ class AppTraceApi(Resource):
             tracing_provider=args.tracing_provider,
         )
 
-        return SimpleResultResponse(result="success").model_dump(mode="json")
+        return {"result": "success"}

api/controllers/console/app/audio.py

@@ -1,4 +1,5 @@
 import logging
+from typing import Any
 
 from flask import request
 from flask_restx import Resource
@@ -6,6 +7,7 @@ from pydantic import BaseModel, Field, RootModel
 from werkzeug.exceptions import InternalServerError
 
 import services
+from controllers.common.fields import AudioBinaryResponse
 from controllers.common.schema import query_params_from_model, register_response_schema_models, register_schema_models
 from controllers.console import console_ns
 from controllers.console.app.error import (
@@ -29,12 +31,9 @@ from controllers.console.wraps import (
 )
 from core.errors.error import ModelCurrentlyNotSupportError, ProviderTokenNotInitError, QuotaExceededError
 from extensions.ext_database import db
-from fields.base import ResponseModel
 from graphon.model_runtime.errors.invoke import InvokeError
-from libs.helper import dump_response
-from libs.login import current_user, login_required
+from libs.login import login_required
 from models import App, AppMode
-from services.app_ref_service import AppRefService
 from services.audio_service import AudioService
 from services.errors.audio import (
     AudioTooLargeServiceError,
@@ -57,27 +56,16 @@ class TextToSpeechVoiceQuery(BaseModel):
     language: str = Field(..., description="Language code")
 
 
-class AudioTranscriptResponse(ResponseModel):
+class AudioTranscriptResponse(BaseModel):
     text: str = Field(description="Transcribed text from audio")
 
 
-class TextToSpeechVoiceResponse(ResponseModel):
-    # see api/core/plugin/impl/model.py
-    name: str = Field(description="Voice display name")
-    value: str = Field(description="Voice identifier")
+class TextToSpeechVoiceListResponse(RootModel[list[dict[str, Any]]]):
+    root: list[dict[str, Any]]
 
 
-class TextToSpeechVoiceListResponse(RootModel[list[TextToSpeechVoiceResponse]]):
-    root: list[TextToSpeechVoiceResponse] = Field(description="Available voices")
-
-
-register_schema_models(console_ns, TextToSpeechPayload, TextToSpeechVoiceQuery)
-register_response_schema_models(
-    console_ns,
-    AudioTranscriptResponse,
-    TextToSpeechVoiceResponse,
-    TextToSpeechVoiceListResponse,
-)
+register_schema_models(console_ns, AudioTranscriptResponse, TextToSpeechPayload, TextToSpeechVoiceQuery)
+register_response_schema_models(console_ns, AudioBinaryResponse, TextToSpeechVoiceListResponse)
 
 
 @console_ns.route("/apps/<uuid:app_id>/audio-to-text")
@@ -106,7 +94,7 @@ class ChatMessageAudioApi(Resource):
                 end_user=None,
             )
 
-            return dump_response(AudioTranscriptResponse, response)
+            return response
         except services.errors.app_model_config.AppModelConfigBrokenError:
             logger.exception("App model config broken.")
             raise AppUnavailableError()
@@ -139,8 +127,11 @@ class ChatMessageTextApi(Resource):
     @console_ns.doc(description="Convert text to speech for chat messages")
     @console_ns.doc(params={"app_id": "App ID"})
     @console_ns.expect(console_ns.models[TextToSpeechPayload.__name__])
-    # TTS returns provider audio bytes, so the success response is intentionally schema-less.
-    @console_ns.response(200, "Text to speech conversion successful")
+    @console_ns.response(
+        200,
+        "Text to speech conversion successful",
+        console_ns.models[AudioBinaryResponse.__name__],
+    )
     @console_ns.response(400, "Bad request - Invalid parameters")
     @setup_required
     @login_required
@@ -149,24 +140,16 @@ class ChatMessageTextApi(Resource):
     def post(self, app_model: App):
         try:
             payload = TextToSpeechPayload.model_validate(console_ns.payload)
-            message_ref = None
-            if payload.message_id:
-                app_ref = AppRefService.create_app_ref(app_model)
-                message_ref = AppRefService.create_message_ref(
-                    app_ref,
-                    payload.message_id,
-                    account_id=current_user.id,
-                )
 
-            # response-contract:ignore
-            return AudioService.transcript_tts(
+            response = AudioService.transcript_tts(
                 app_model=app_model,
-                session=db.session(),
+                session=db.session,
                 text=payload.text,
                 voice=payload.voice,
-                message_ref=message_ref,
+                message_id=payload.message_id,
                 is_draft=True,
             )
+            return response
         except services.errors.app_model_config.AppModelConfigBrokenError:
             logger.exception("App model config broken.")
             raise AppUnavailableError()
@@ -197,7 +180,8 @@ class ChatMessageTextApi(Resource):
 class TextModesApi(Resource):
     @console_ns.doc("get_text_to_speech_voices")
     @console_ns.doc(description="Get available TTS voices for a specific language")
-    @console_ns.doc(params={"app_id": "App ID", **query_params_from_model(TextToSpeechVoiceQuery)})
+    @console_ns.doc(params={"app_id": "App ID"})
+    @console_ns.doc(params=query_params_from_model(TextToSpeechVoiceQuery))
     @console_ns.response(
         200,
         "TTS voices retrieved successfully",
@@ -218,7 +202,7 @@ class TextModesApi(Resource):
                 language=args.language,
             )
 
-            return dump_response(TextToSpeechVoiceListResponse, response)
+            return response
         except services.errors.audio.ProviderNotSupportTextToSpeechLanageServiceError:
             raise AppUnavailableError("Text to audio voices language parameter loss.")
         except NoAudioUploadedServiceError:

api/controllers/console/app/completion.py

@@ -1,20 +1,17 @@
-import json
 import logging
-from collections.abc import Generator, Iterator, Mapping
-from typing import Any, Literal, Protocol, runtime_checkable
+from typing import Any, Literal
 from uuid import UUID
 
 from flask import request
 from flask_restx import Resource
 from pydantic import BaseModel, Field, field_validator
-from sqlalchemy.orm import Session
 from werkzeug.exceptions import BadRequest, InternalServerError, NotFound
 
 import services
-from controllers.common.fields import SimpleResultResponse
+from controllers.common.fields import GeneratedAppResponse, SimpleResultResponse
 from controllers.common.schema import register_response_schema_models, register_schema_models
 from controllers.console import console_ns
-from controllers.console.agent.app_helpers import resolve_agent_runtime_app_model
+from controllers.console.agent.app_helpers import resolve_agent_app_model
 from controllers.console.app.error import (
     AppUnavailableError,
     CompletionRequestError,
@@ -23,7 +20,7 @@ from controllers.console.app.error import (
     ProviderNotInitializeError,
     ProviderQuotaExceededError,
 )
-from controllers.console.app.wraps import get_app_model, with_session
+from controllers.console.app.wraps import get_app_model
 from controllers.console.wraps import (
     RBACPermission,
     RBACResourceScope,
@@ -37,7 +34,6 @@ from controllers.console.wraps import (
 )
 from controllers.web.error import InvokeRateLimitError as InvokeRateLimitHttpError
 from core.app.entities.app_invoke_entities import InvokeFrom
-from core.app.features.rate_limiting.rate_limit import RateLimitGenerator
 from core.errors.error import (
     ModelCurrentlyNotSupportError,
     ProviderTokenNotInitError,
@@ -60,11 +56,6 @@ from services.errors.llm import InvokeRateLimitError
 logger = logging.getLogger(__name__)
 
 
-@runtime_checkable
-class _ClosableStream(Protocol):
-    def close(self) -> None: ...
-
-
 def _resolve_debugger_chat_streaming(
     *, app_mode: AppMode, response_mode: str, response_mode_provided: bool = True
 ) -> bool:
@@ -102,10 +93,6 @@ class ChatMessagePayload(BaseMessagePayload):
     query: str = Field(..., description="User query")
     conversation_id: str | None = Field(default=None, description="Conversation ID")
     parent_message_id: str | None = Field(default=None, description="Parent message ID")
-    draft_type: Literal["draft", "debug_build"] = Field(
-        default="draft",
-        description="Agent App debug config source. Use debug_build while the Agent is in build mode.",
-    )
 
     @field_validator("conversation_id", "parent_message_id")
     @classmethod
@@ -115,32 +102,8 @@ class ChatMessagePayload(BaseMessagePayload):
         return uuid_value(value)
 
 
-_BUILD_CHAT_FINALIZATION_QUERY = """Finalize this Build chat configuration for the agent.
-
-This step is only for persisting Agent config changes discovered in the current Build chat. Do not install packages,
-edit workspace files, run validation or debugging commands, make exploratory checks, or perform other work.
-
-Use only the current Build chat message history to identify changes that need to be persisted. Do not inspect, test, or
-validate old config unless the message history already shows that the old config is invalid.
-
-Only update the build-draft config note when the current Build chat contains durable context that later runs need.
-Write the config note in the language used by the message history.
-Do not create, update, delete, inspect, or fill gaps in other Agent config resources, including config files, config
-skills, config env, tools, models, knowledge, or prompt settings.
-
-When updating the config note with the Agent config CLI usage provided in the runtime prompt, record only durable
-context needed by later runs, such as:
-
-- what you installed or configured outside the workspace for this agent,
-- where those external updates live, including CLI tools, packages, and persistent $HOME paths,
-- how the agent should use it in later runs,
-- any setup, authentication, or user action still required.
-
-After config persistence completes, respond FINISHED."""
-
-
 register_schema_models(console_ns, CompletionMessagePayload, ChatMessagePayload)
-register_response_schema_models(console_ns, SimpleResultResponse)
+register_response_schema_models(console_ns, GeneratedAppResponse, SimpleResultResponse)
 
 
 # define completion message api for user
@@ -150,7 +113,7 @@ class CompletionMessageApi(Resource):
     @console_ns.doc(description="Generate completion message for debugging")
     @console_ns.doc(params={"app_id": "Application ID"})
     @console_ns.expect(console_ns.models[CompletionMessagePayload.__name__])
-    @console_ns.response(200, "Completion generated successfully")
+    @console_ns.response(200, "Completion generated successfully", console_ns.models[GeneratedAppResponse.__name__])
     @console_ns.response(400, "Invalid request parameters")
     @console_ns.response(404, "App not found")
     @setup_required
@@ -159,8 +122,7 @@ class CompletionMessageApi(Resource):
     @with_current_user
     @rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_TEST_AND_RUN)
     @get_app_model(mode=AppMode.COMPLETION)
-    @with_session
-    def post(self, session: Session, current_user: Account, app_model: App):
+    def post(self, current_user: Account, app_model: App):
         args_model = CompletionMessagePayload.model_validate(console_ns.payload)
         args = args_model.model_dump(exclude_none=True, by_alias=True)
 
@@ -169,15 +131,9 @@ class CompletionMessageApi(Resource):
 
         try:
             response = AppGenerateService.generate(
-                session=session,
-                app_model=app_model,
-                user=current_user,
-                args=args,
-                invoke_from=InvokeFrom.DEBUGGER,
-                streaming=streaming,
+                app_model=app_model, user=current_user, args=args, invoke_from=InvokeFrom.DEBUGGER, streaming=streaming
             )
 
-            # response-contract:ignore compact_generate_response
             return helper.compact_generate_response(response)
         except services.errors.conversation.ConversationNotExistsError:
             raise NotFound("Conversation Not Exists.")
@@ -221,7 +177,7 @@ class CompletionMessageStopApi(Resource):
             app_mode=AppMode.value_of(app_model.mode),
         )
 
-        return SimpleResultResponse(result="success").model_dump(mode="json"), 200
+        return {"result": "success"}, 200
 
 
 @console_ns.route("/apps/<uuid:app_id>/chat-messages")
@@ -230,7 +186,7 @@ class ChatMessageApi(Resource):
     @console_ns.doc(description="Generate chat message for debugging")
     @console_ns.doc(params={"app_id": "Application ID"})
     @console_ns.expect(console_ns.models[ChatMessagePayload.__name__])
-    @console_ns.response(200, "Chat message generated successfully")
+    @console_ns.response(200, "Chat message generated successfully", console_ns.models[GeneratedAppResponse.__name__])
     @console_ns.response(400, "Invalid request parameters")
     @console_ns.response(404, "App or conversation not found")
     @setup_required
@@ -241,11 +197,8 @@ class ChatMessageApi(Resource):
     @with_current_tenant_id
     @rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_TEST_AND_RUN)
     @get_app_model(mode=[AppMode.CHAT, AppMode.AGENT_CHAT, AppMode.AGENT])
-    @with_session
-    def post(self, session: Session, current_tenant_id: str, current_user: Account, app_model: App):
-        return _create_chat_message(
-            session=session, current_tenant_id=current_tenant_id, current_user=current_user, app_model=app_model
-        )
+    def post(self, current_tenant_id: str, current_user: Account, app_model: App):
+        return _create_chat_message(current_tenant_id=current_tenant_id, current_user=current_user, app_model=app_model)
 
 
 @console_ns.route("/agent/<uuid:agent_id>/chat-messages")
@@ -254,7 +207,7 @@ class AgentChatMessageApi(Resource):
     @console_ns.doc(description="Generate an Agent App chat message for debugging")
     @console_ns.doc(params={"agent_id": "Agent ID"})
     @console_ns.expect(console_ns.models[ChatMessagePayload.__name__])
-    @console_ns.response(200, "Chat message generated successfully")
+    @console_ns.response(200, "Chat message generated successfully", console_ns.models[GeneratedAppResponse.__name__])
     @console_ns.response(400, "Invalid request parameters")
     @console_ns.response(404, "Agent or conversation not found")
     @setup_required
@@ -264,38 +217,9 @@ class AgentChatMessageApi(Resource):
     @rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_TEST_AND_RUN)
     @with_current_user
     @with_current_tenant_id
-    @with_session
-    def post(self, session: Session, current_tenant_id: str, current_user: Account, agent_id: UUID):
-        app_model = resolve_agent_runtime_app_model(tenant_id=current_tenant_id, agent_id=agent_id)
+    def post(self, current_tenant_id: str, current_user: Account, agent_id: UUID):
+        app_model = resolve_agent_app_model(tenant_id=current_tenant_id, agent_id=agent_id)
         return _create_chat_message(
-            session=session,
-            current_tenant_id=current_tenant_id,
-            current_user=current_user,
-            app_model=app_model,
-            agent_id=str(agent_id),
-        )
-
-
-@console_ns.route("/agent/<uuid:agent_id>/build-chat/finalize")
-class AgentBuildChatFinalizeApi(Resource):
-    @console_ns.doc("finalize_agent_build_chat")
-    @console_ns.doc(description="Run a build-draft Agent App turn that asks the agent to push config updates")
-    @console_ns.doc(params={"agent_id": "Agent ID"})
-    @console_ns.response(200, "Success", console_ns.models[SimpleResultResponse.__name__])
-    @console_ns.response(400, "Invalid request parameters")
-    @console_ns.response(404, "Agent, build draft, or conversation not found")
-    @setup_required
-    @login_required
-    @account_initialization_required
-    @edit_permission_required
-    @rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_TEST_AND_RUN)
-    @with_current_user
-    @with_current_tenant_id
-    @with_session
-    def post(self, session: Session, current_tenant_id: str, current_user: Account, agent_id: UUID):
-        app_model = resolve_agent_runtime_app_model(tenant_id=current_tenant_id, agent_id=agent_id)
-        return _create_build_chat_finalization_message(
-            session=session,
             current_tenant_id=current_tenant_id,
             current_user=current_user,
             app_model=app_model,
@@ -330,7 +254,7 @@ class AgentChatMessageStopApi(Resource):
     @with_current_user_id
     @with_current_tenant_id
     def post(self, current_tenant_id: str, current_user_id: str, agent_id: UUID, task_id: str):
-        app_model = resolve_agent_runtime_app_model(tenant_id=current_tenant_id, agent_id=agent_id)
+        app_model = resolve_agent_app_model(tenant_id=current_tenant_id, agent_id=agent_id)
         return _stop_chat_message(current_user_id=current_user_id, app_model=app_model, task_id=task_id)
 
 
@@ -356,12 +280,7 @@ def _resolve_current_user_agent_debug_conversation_id(
 
 
 def _create_chat_message(
-    *,
-    session: Session,
-    current_user: Account,
-    app_model: App,
-    current_tenant_id: str | None = None,
-    agent_id: str | None = None,
+    *, current_user: Account, app_model: App, current_tenant_id: str | None = None, agent_id: str | None = None
 ):
     raw_payload = console_ns.payload or {}
     args_model = ChatMessagePayload.model_validate(raw_payload)
@@ -391,107 +310,12 @@ def _create_chat_message(
     if external_trace_id:
         args["external_trace_id"] = external_trace_id
 
-    return _generate_chat_message_response(
-        session=session,
-        current_user=current_user,
-        app_model=app_model,
-        args=args,
-        streaming=streaming,
-    )
-
-
-def _create_build_chat_finalization_message(
-    *, session: Session, current_user: Account, app_model: App, current_tenant_id: str, agent_id: str
-):
-    debug_conversation_id = _resolve_current_user_agent_debug_conversation_id(
-        current_tenant_id=current_tenant_id,
-        current_user=current_user,
-        app_model=app_model,
-        agent_id=agent_id,
-    )
-    args: dict[str, Any] = {
-        "query": _BUILD_CHAT_FINALIZATION_QUERY,
-        "inputs": {},
-        "response_mode": "streaming",
-        "draft_type": "debug_build",
-        "conversation_id": debug_conversation_id,
-        "auto_generate_name": False,
-    }
-    external_trace_id = get_external_trace_id(request)
-    if external_trace_id:
-        args["external_trace_id"] = external_trace_id
-
-    response = _generate_chat_message(
-        session=session,
-        current_user=current_user,
-        app_model=app_model,
-        args=args,
-        streaming=True,
-    )
-    _drain_streaming_generate_response(response)
-    return {"result": "success"}, 200
-
-
-def _drain_streaming_generate_response(response: RateLimitGenerator | Generator[str, None, None]) -> None:
-    """Consume a streamed app-generate response until a terminal message event arrives.
-
-    Finalize keeps the normal Agent App streaming path so the existing queue,
-    persistence, and runtime-session behavior stay intact. The console API only
-    changes the HTTP boundary: it drains the SSE stream server-side and returns
-    success after the generated build-chat message reaches ``message_end``.
-    """
     try:
-        for chunk in response:
-            for raw_event in chunk.split("\n\n"):
-                if not raw_event.strip():
-                    continue
-
-                event_name: str | None = None
-                data_lines: list[str] = []
-                for line in raw_event.splitlines():
-                    if line.startswith("event: "):
-                        event_name = line.removeprefix("event: ").strip()
-                    elif line.startswith("data: "):
-                        data_lines.append(line.removeprefix("data: "))
-
-                if not data_lines:
-                    if event_name == "ping":
-                        continue
-                    continue
-
-                payload = json.loads("\n".join(data_lines))
-                if not isinstance(payload, dict):
-                    continue
-
-                payload_event = payload.get("event")
-                if payload_event == "message_end":
-                    return
-                if payload_event == "error":
-                    raise CompletionRequestError(str(payload.get("message") or "Build chat finalization failed."))
-    finally:
-        if isinstance(response, _ClosableStream):
-            response.close()
-
-    raise CompletionRequestError("Build chat finalization did not complete.")
-
-
-def _generate_chat_message(
-    *,
-    session: Session,
-    current_user: Account,
-    app_model: App,
-    args: dict[str, Any],
-    streaming: bool,
-):
-    try:
-        return AppGenerateService.generate(
-            session=session,
-            app_model=app_model,
-            user=current_user,
-            args=args,
-            invoke_from=InvokeFrom.DEBUGGER,
-            streaming=streaming,
+        response = AppGenerateService.generate(
+            app_model=app_model, user=current_user, args=args, invoke_from=InvokeFrom.DEBUGGER, streaming=streaming
         )
+
+        return helper.compact_generate_response(response)
     except services.errors.conversation.ConversationNotExistsError:
         raise NotFound("Conversation Not Exists.")
     except services.errors.conversation.ConversationCompletedError:
@@ -507,8 +331,6 @@ def _generate_chat_message(
         raise ProviderModelCurrentlyNotSupportError()
     except InvokeRateLimitError as ex:
         raise InvokeRateLimitHttpError(ex.description)
-    except CompletionRequestError:
-        raise
     except InvokeError as e:
         raise CompletionRequestError(e.description)
     except ValueError as e:
@@ -518,26 +340,6 @@ def _generate_chat_message(
         raise InternalServerError()
 
 
-def _generate_chat_message_response(
-    *,
-    session: Session,
-    current_user: Account,
-    app_model: App,
-    args: dict[str, Any],
-    streaming: bool,
-):
-    response = _generate_chat_message(
-        session=session,
-        current_user=current_user,
-        app_model=app_model,
-        args=args,
-        streaming=streaming,
-    )
-    if AppMode.value_of(app_model.mode) == AppMode.AGENT and streaming:
-        response = _raise_agent_stream_error_before_response(response)
-    return helper.compact_generate_response(response)
-
-
 def _stop_chat_message(*, current_user_id: str, app_model: App, task_id: str):
     AppTaskService.stop_task(
         task_id=task_id,
@@ -546,68 +348,4 @@ def _stop_chat_message(*, current_user_id: str, app_model: App, task_id: str):
         app_mode=AppMode.value_of(app_model.mode),
     )
 
-    return SimpleResultResponse(result="success").model_dump(mode="json"), 200
-
-
-def _raise_agent_stream_error_before_response(response):
-    """Surface immediate Agent App stream errors as HTTP errors before SSE starts.
-
-    The shared streaming helper always returns HTTP 200 once the SSE response is
-    created. Agent v2 configuration errors, such as an invalid model API key,
-    can be the first real stream event after the initial ping; pre-reading that
-    first non-ping event lets the console API return the existing 400 error
-    contract instead of a successful HTTP response carrying only an SSE error.
-    """
-    if isinstance(response, Mapping):
-        return response
-
-    buffered: list[str] = []
-    iterator = iter(response)
-    while True:
-        try:
-            chunk = next(iterator)
-        except StopIteration:
-            return iter(buffered)
-
-        if not isinstance(chunk, str):
-            return _prepend_stream_chunks(buffered, chunk, iterator)
-
-        if _is_sse_ping(chunk):
-            buffered.append(chunk)
-            continue
-
-        error_payload = _extract_sse_error_payload(chunk)
-        if error_payload is not None:
-            if isinstance(response, _ClosableStream):
-                response.close()
-            message = error_payload.get("message")
-            raise CompletionRequestError(str(message or "Agent App chat failed."))
-
-        return _prepend_stream_chunks(buffered, chunk, iterator)
-
-
-def _prepend_stream_chunks(buffered: list[Any], first: Any, iterator: Iterator[Any]) -> Generator[Any, None, None]:
-    yield from buffered
-    yield first
-    yield from iterator
-
-
-def _is_sse_ping(chunk: str) -> bool:
-    return chunk.strip() == "event: ping"
-
-
-def _extract_sse_error_payload(chunk: str) -> dict[str, Any] | None:
-    for raw_event in chunk.split("\n\n"):
-        data_lines: list[str] = []
-        for line in raw_event.splitlines():
-            if line.startswith("data: "):
-                data_lines.append(line.removeprefix("data: "))
-        if not data_lines:
-            continue
-        try:
-            payload = json.loads("\n".join(data_lines))
-        except json.JSONDecodeError:
-            continue
-        if isinstance(payload, dict) and payload.get("event") == "error":
-            return payload
-    return None
+    return {"result": "success"}, 200

api/controllers/console/app/conversation.py

@@ -9,7 +9,7 @@ from sqlalchemy import func, or_
 from sqlalchemy.orm import selectinload
 from werkzeug.exceptions import NotFound
 
-from controllers.common.schema import query_params_from_model, register_response_schema_models, register_schema_models
+from controllers.common.schema import query_params_from_model, register_schema_models
 from controllers.console import console_ns
 from controllers.console.app.wraps import get_app_model
 from controllers.console.wraps import (
@@ -39,9 +39,7 @@ from fields.conversation_fields import (
     ConversationWithSummaryPagination as ConversationWithSummaryPaginationResponse,
 )
 from libs.datetime_utils import naive_utc_now, parse_time_range
-from libs.helper import dump_response
 from libs.login import login_required
-from libs.pagination import paginate_query
 from models import Conversation, EndUser, Message, MessageAnnotation
 from models.account import Account
 from models.model import App, AppMode
@@ -81,14 +79,13 @@ register_schema_models(
     console_ns,
     CompletionConversationQuery,
     ChatConversationQuery,
-)
-register_response_schema_models(
-    console_ns,
     ConversationResponse,
     ConversationPaginationResponse,
     ConversationMessageDetailResponse,
     ConversationWithSummaryPaginationResponse,
     ConversationDetailResponse,
+    CompletionConversationQuery,
+    ChatConversationQuery,
 )
 
 
@@ -96,7 +93,8 @@ register_response_schema_models(
 class CompletionConversationApi(Resource):
     @console_ns.doc("list_completion_conversations")
     @console_ns.doc(description="Get completion conversations with pagination and filtering")
-    @console_ns.doc(params={"app_id": "Application ID", **query_params_from_model(CompletionConversationQuery)})
+    @console_ns.doc(params={"app_id": "Application ID"})
+    @console_ns.doc(params=query_params_from_model(CompletionConversationQuery))
     @console_ns.response(200, "Success", console_ns.models[ConversationPaginationResponse.__name__])
     @console_ns.response(403, "Insufficient permissions")
     @setup_required
@@ -157,9 +155,11 @@ class CompletionConversationApi(Resource):
 
         query = query.order_by(Conversation.created_at.desc())
 
-        conversations = paginate_query(query, page=args.page, per_page=args.limit)
+        conversations = db.paginate(query, page=args.page, per_page=args.limit, error_out=False)
 
-        return dump_response(ConversationPaginationResponse, conversations)
+        return ConversationPaginationResponse.model_validate(conversations, from_attributes=True).model_dump(
+            mode="json"
+        )
 
 
 @console_ns.route("/apps/<uuid:app_id>/completion-conversations/<uuid:conversation_id>")
@@ -179,9 +179,9 @@ class CompletionConversationDetailApi(Resource):
     @get_app_model(mode=AppMode.COMPLETION)
     def get(self, current_user: Account, app_model: App, conversation_id: UUID):
         conversation_id_str = str(conversation_id)
-        return dump_response(
-            ConversationMessageDetailResponse, _get_conversation(current_user, app_model, conversation_id_str)
-        )
+        return ConversationMessageDetailResponse.model_validate(
+            _get_conversation(current_user, app_model, conversation_id_str), from_attributes=True
+        ).model_dump(mode="json")
 
     @console_ns.doc("delete_completion_conversation")
     @console_ns.doc(description="Delete a completion conversation")
@@ -200,7 +200,7 @@ class CompletionConversationDetailApi(Resource):
         conversation_id_str = str(conversation_id)
 
         try:
-            ConversationService.delete(app_model, conversation_id_str, current_user, session=db.session())
+            ConversationService.delete(app_model, conversation_id_str, current_user)
         except ConversationNotExistsError:
             raise NotFound("Conversation Not Exists.")
 
@@ -211,7 +211,8 @@ class CompletionConversationDetailApi(Resource):
 class ChatConversationApi(Resource):
     @console_ns.doc("list_chat_conversations")
     @console_ns.doc(description="Get chat conversations with pagination, filtering and summary")
-    @console_ns.doc(params={"app_id": "Application ID", **query_params_from_model(ChatConversationQuery)})
+    @console_ns.doc(params={"app_id": "Application ID"})
+    @console_ns.doc(params=query_params_from_model(ChatConversationQuery))
     @console_ns.response(200, "Success", console_ns.models[ConversationWithSummaryPaginationResponse.__name__])
     @console_ns.response(403, "Insufficient permissions")
     @setup_required
@@ -311,9 +312,11 @@ class ChatConversationApi(Resource):
             case _:
                 query = query.order_by(Conversation.created_at.desc())
 
-        conversations = paginate_query(query, page=args.page, per_page=args.limit)
+        conversations = db.paginate(query, page=args.page, per_page=args.limit, error_out=False)
 
-        return dump_response(ConversationWithSummaryPaginationResponse, conversations)
+        return ConversationWithSummaryPaginationResponse.model_validate(conversations, from_attributes=True).model_dump(
+            mode="json"
+        )
 
 
 @console_ns.route("/apps/<uuid:app_id>/chat-conversations/<uuid:conversation_id>")
@@ -333,9 +336,9 @@ class ChatConversationDetailApi(Resource):
     @get_app_model(mode=[AppMode.CHAT, AppMode.AGENT_CHAT, AppMode.ADVANCED_CHAT, AppMode.AGENT])
     def get(self, current_user: Account, app_model: App, conversation_id: UUID):
         conversation_id_str = str(conversation_id)
-        return dump_response(
-            ConversationDetailResponse, _get_conversation(current_user, app_model, conversation_id_str)
-        )
+        return ConversationDetailResponse.model_validate(
+            _get_conversation(current_user, app_model, conversation_id_str), from_attributes=True
+        ).model_dump(mode="json")
 
     @console_ns.doc("delete_chat_conversation")
     @console_ns.doc(description="Delete a chat conversation")
@@ -354,7 +357,7 @@ class ChatConversationDetailApi(Resource):
         conversation_id_str = str(conversation_id)
 
         try:
-            ConversationService.delete(app_model, conversation_id_str, current_user, session=db.session())
+            ConversationService.delete(app_model, conversation_id_str, current_user)
         except ConversationNotExistsError:
             raise NotFound("Conversation Not Exists.")
 

api/controllers/console/app/conversation_variables.py

@@ -22,7 +22,7 @@ from controllers.console.wraps import (
 from extensions.ext_database import db
 from fields._value_type_serializer import serialize_value_type
 from fields.base import ResponseModel
-from libs.helper import dump_response, to_timestamp
+from libs.helper import to_timestamp
 from libs.login import login_required
 from models import ConversationVariable
 from models.model import App, AppMode
@@ -119,8 +119,7 @@ class ConversationVariablesApi(Resource):
         with sessionmaker(db.engine, expire_on_commit=False).begin() as session:
             rows = session.scalars(stmt).all()
 
-        return dump_response(
-            PaginatedConversationVariableResponse,
+        response = PaginatedConversationVariableResponse.model_validate(
             {
                 "page": page,
                 "limit": page_size,
@@ -136,5 +135,6 @@ class ConversationVariablesApi(Resource):
                     )
                     for row in rows
                 ],
-            },
+            }
         )
+        return response.model_dump(mode="json")

api/controllers/console/app/generator.py

@@ -1,10 +1,8 @@
-import json
-from collections.abc import Generator, Sequence
+from collections.abc import Sequence
 from typing import Any, Literal
 
 from flask_restx import Resource
 from pydantic import BaseModel, Field, RootModel
-from sqlalchemy import select
 from sqlalchemy.orm import Session
 
 from controllers.common.fields import SimpleDataResponse
@@ -25,10 +23,8 @@ from core.helper.code_executor.javascript.javascript_code_provider import Javasc
 from core.helper.code_executor.python3.python3_code_provider import Python3CodeProvider
 from core.llm_generator.entities import RuleCodeGeneratePayload, RuleGeneratePayload, RuleStructuredOutputPayload
 from core.llm_generator.llm_generator import LLMGenerator
-from core.workflow.generator.types import WorkflowGenerateErrorCode
 from graphon.model_runtime.entities.llm_entities import LLMMode
 from graphon.model_runtime.errors.invoke import InvokeError
-from libs.helper import compact_generate_response
 from libs.login import login_required
 from models import App
 from services.workflow_generator_service import WorkflowGeneratorService
@@ -68,10 +64,7 @@ class WorkflowGeneratePayload(BaseModel):
     can reuse its existing handler.
     """
 
-    mode: Literal["workflow", "advanced-chat", "auto"] = Field(
-        ...,
-        description="Target app mode for the generated graph; 'auto' lets the backend classify the instruction",
-    )
+    mode: Literal["workflow", "advanced-chat"] = Field(..., description="Target app mode for the generated graph")
     instruction: str = Field(..., description="Natural-language workflow description")
     ideal_output: str = Field(default="", description="Optional sample output for grounding")
     model_config_data: ModelConfig = Field(
@@ -85,19 +78,6 @@ class WorkflowGeneratePayload(BaseModel):
     )
 
 
-class WorkflowInstructionSuggestionsPayload(BaseModel):
-    """Payload for the workflow-generator instruction-suggestions endpoint.
-
-    Runs before the user picks a model, so the suggestions come from the
-    tenant's default model. The underlying generator never raises — an empty
-    ``suggestions`` list is a valid 200 (soft-fail).
-    """
-
-    mode: Literal["workflow", "advanced-chat"] = Field(..., description="Target app mode for the suggestions")
-    language: str | None = Field(default=None, description="Optional language to write the suggestions in")
-    count: int = Field(default=4, ge=1, le=6, description="Number of suggestions to return (1-6)")
-
-
 class GeneratorResponse(RootModel[Any]):
     root: Any
 
@@ -111,7 +91,6 @@ register_schema_models(
     InstructionGeneratePayload,
     InstructionTemplatePayload,
     WorkflowGeneratePayload,
-    WorkflowInstructionSuggestionsPayload,
     ModelConfig,
 )
 register_response_schema_models(console_ns, GeneratorResponse, SimpleDataResponse)
@@ -237,9 +216,7 @@ class InstructionGenerateApi(Resource):
         try:
             # Generate from nothing for a workflow node
             if (args.current in (code_template, "")) and args.node_id != "":
-                app = session.scalar(
-                    select(App).where(App.id == args.flow_id, App.tenant_id == current_tenant_id).limit(1)
-                )
+                app = session.get(App, args.flow_id)
                 if not app:
                     return {"error": f"app {args.flow_id} not found"}, 400
                 workflow = WorkflowService().get_draft_workflow(app_model=app, session=session)
@@ -336,34 +313,6 @@ class InstructionGenerationTemplateApi(Resource):
                 raise ValueError(f"Invalid type: {args.type}")
 
 
-def _workflow_instruction_guard(args: WorkflowGeneratePayload) -> tuple[dict, int] | None:
-    """Shared boundary guard for the workflow-generate endpoints.
-
-    Returns a ``(body, 400)`` tuple when the instruction is empty / whitespace
-    or either free-text field exceeds the cap, else ``None``. Pydantic only
-    validates the field is a str; a whitespace-only or pasted-document input
-    would otherwise waste a slow planner+builder roundtrip on a response the
-    validator rejects anyway. Both the blocking and streaming endpoints call
-    this so they reject identical inputs.
-    """
-    if not args.instruction.strip():
-        return {
-            "error": "Instruction is required",
-            "errors": [{"code": WorkflowGenerateErrorCode.EMPTY_INSTRUCTION, "detail": "Instruction is required"}],
-        }, 400
-    if len(args.instruction) > _MAX_INSTRUCTION_LENGTH or len(args.ideal_output) > _MAX_INSTRUCTION_LENGTH:
-        return {
-            "error": "Instruction is too long",
-            "errors": [
-                {
-                    "code": WorkflowGenerateErrorCode.INSTRUCTION_TOO_LONG,
-                    "detail": f"Instruction and ideal output must each be at most {_MAX_INSTRUCTION_LENGTH} characters",
-                }
-            ],
-        }, 400
-    return None
-
-
 @console_ns.route("/workflow-generate")
 class WorkflowGenerateApi(Resource):
     """Generate a Workflow / Chatflow draft graph from a natural-language description.
@@ -386,11 +335,31 @@ class WorkflowGenerateApi(Resource):
     def post(self, current_tenant_id: str):
         args = WorkflowGeneratePayload.model_validate(console_ns.payload)
 
-        # Reject empty / over-length instructions at the boundary (shared with
-        # the streaming endpoint) before spending a planner+builder roundtrip.
-        guard = _workflow_instruction_guard(args)
-        if guard is not None:
-            return guard
+        # Reject obviously-empty instructions at the boundary — Pydantic only
+        # validates ``instruction`` is a str, but a whitespace-only string
+        # would still hit the LLM and waste a planner+builder roundtrip on a
+        # response that the postprocess validator would reject anyway.
+        if not args.instruction.strip():
+            return {
+                "error": "Instruction is required",
+                "errors": [{"code": "EMPTY_INSTRUCTION", "detail": "Instruction is required"}],
+            }, 400
+
+        # Bound the prompt at the boundary too: an arbitrarily long
+        # instruction (or pasted document) blows the planner/builder context
+        # window and fails with an opaque provider error after two slow LLM
+        # calls. The cap matches the frontend textarea's maxLength.
+        if len(args.instruction) > _MAX_INSTRUCTION_LENGTH or len(args.ideal_output) > _MAX_INSTRUCTION_LENGTH:
+            return {
+                "error": "Instruction is too long",
+                "errors": [
+                    {
+                        "code": "INSTRUCTION_TOO_LONG",
+                        "detail": f"Instruction and ideal output must each be at most "
+                        f"{_MAX_INSTRUCTION_LENGTH} characters",
+                    }
+                ],
+            }, 400
 
         try:
             result = WorkflowGeneratorService.generate_workflow_graph(
@@ -411,93 +380,3 @@ class WorkflowGenerateApi(Resource):
             raise CompletionRequestError(e.description)
 
         return result
-
-
-@console_ns.route("/workflow-generate/suggestions")
-class WorkflowInstructionSuggestionsApi(Resource):
-    """Suggest short, buildable example instructions for the cmd+k generator.
-
-    Runs before a model is selected (uses the tenant's default model). The
-    underlying generator never raises, so an empty list is a valid 200 — the
-    frontend renders "no suggestions" rather than an error, so no provider-error
-    mapping is needed here.
-    """
-
-    @console_ns.doc("generate_workflow_instruction_suggestions")
-    @console_ns.doc(description="Suggest example workflow-generator instructions for the tenant")
-    @console_ns.expect(console_ns.models[WorkflowInstructionSuggestionsPayload.__name__])
-    @console_ns.response(200, "Suggestions generated successfully", console_ns.models[GeneratorResponse.__name__])
-    @console_ns.response(400, "Invalid request parameters")
-    @setup_required
-    @login_required
-    @account_initialization_required
-    @with_current_tenant_id
-    def post(self, current_tenant_id: str):
-        args = WorkflowInstructionSuggestionsPayload.model_validate(console_ns.payload)
-        suggestions = LLMGenerator.generate_workflow_instruction_suggestions(
-            tenant_id=current_tenant_id,
-            mode=args.mode,
-            language=args.language,
-            count=args.count,
-        )
-        return {"suggestions": suggestions}
-
-
-@console_ns.route("/workflow-generate/stream")
-class WorkflowGenerateStreamApi(Resource):
-    """Plan-first streaming variant of ``/workflow-generate`` (Server-Sent Events).
-
-    Emits a ``plan`` event (high-level node list + app metadata) as soon as the
-    planner returns, then a final ``result`` event with the full graph — the
-    SAME envelope ``/workflow-generate`` returns. Provider-init / invoke errors
-    are surfaced as a single ``result`` event (code ``MODEL_ERROR``) so the
-    frontend's stream parser always receives a result rather than a non-SSE HTTP
-    error.
-    """
-
-    @console_ns.doc("generate_workflow_graph_stream")
-    @console_ns.doc(description="Stream a Dify workflow graph (plan then result) via SSE")
-    @console_ns.expect(console_ns.models[WorkflowGeneratePayload.__name__])
-    @console_ns.response(200, "Server-Sent Events stream of plan/result events")
-    @console_ns.response(400, "Invalid request parameters")
-    @setup_required
-    @login_required
-    @account_initialization_required
-    @with_current_tenant_id
-    def post(self, current_tenant_id: str):
-        args = WorkflowGeneratePayload.model_validate(console_ns.payload)
-
-        # Same boundary guards as the blocking endpoint — return a normal 400
-        # JSON for these BEFORE opening the stream.
-        guard = _workflow_instruction_guard(args)
-        if guard is not None:
-            return guard
-
-        def generate() -> Generator[str, None, None]:
-            try:
-                for event_name, payload in WorkflowGeneratorService.generate_workflow_graph_stream(
-                    tenant_id=current_tenant_id,
-                    mode=args.mode,
-                    instruction=args.instruction,
-                    model_config=args.model_config_data,
-                    ideal_output=args.ideal_output,
-                    current_graph=args.current_graph,
-                ):
-                    body = {"event": event_name, **payload}
-                    yield f"data: {json.dumps(body)}\n\n"
-            except (ProviderTokenNotInitError, QuotaExceededError, ModelCurrentlyNotSupportError, InvokeError) as e:
-                # The model instance is resolved inside the service (lazily, on
-                # first iteration), so a provider / init error surfaces here.
-                # Emit it as a single SSE result event rather than a non-SSE
-                # error response so the frontend's stream parser always gets a
-                # result it can render.
-                detail = getattr(e, "description", None) or str(e) or "Model invocation failed"
-                error_body = {
-                    "event": "result",
-                    "graph": {"nodes": [], "edges": [], "viewport": {"x": 0.0, "y": 0.0, "zoom": 0.7}},
-                    "error": detail,
-                    "errors": [{"code": WorkflowGenerateErrorCode.MODEL_ERROR, "detail": detail}],
-                }
-                yield f"data: {json.dumps(error_body)}\n\n"
-
-        return compact_generate_response(generate())

api/controllers/console/app/mcp_server.py

@@ -22,11 +22,10 @@ from controllers.console.wraps import (
 )
 from extensions.ext_database import db
 from fields.base import ResponseModel
-from libs.helper import dump_response, to_timestamp
+from libs.helper import to_timestamp
 from libs.login import login_required
 from models.enums import AppMCPServerStatus
 from models.model import App, AppMCPServer
-from services.app_ref_service import AppRefService
 
 
 class MCPServerCreatePayload(BaseModel):
@@ -93,7 +92,7 @@ class AppMCPServerController(Resource):
         server = db.session.scalar(select(AppMCPServer).where(AppMCPServer.app_id == app_model.id).limit(1))
         if server is None:
             return {}
-        return dump_response(AppMCPServerResponse, server)
+        return AppMCPServerResponse.model_validate(server, from_attributes=True).model_dump(mode="json")
 
     @console_ns.doc("create_app_mcp_server")
     @console_ns.doc(description="Create MCP server configuration for an application")
@@ -128,7 +127,7 @@ class AppMCPServerController(Resource):
         )
         db.session.add(server)
         db.session.commit()
-        return dump_response(AppMCPServerResponse, server), 201
+        return AppMCPServerResponse.model_validate(server, from_attributes=True).model_dump(mode="json"), 201
 
     @console_ns.doc("update_app_mcp_server")
     @console_ns.doc(description="Update MCP server configuration for an application")
@@ -147,17 +146,7 @@ class AppMCPServerController(Resource):
     @get_app_model
     def put(self, app_model: App):
         payload = MCPServerUpdatePayload.model_validate(console_ns.payload or {})
-        app_ref = AppRefService.create_app_ref(app_model)
-        server_ref = AppRefService.create_mcp_server_ref(app_ref, payload.id)
-        server = db.session.scalar(
-            select(AppMCPServer)
-            .where(
-                AppMCPServer.id == server_ref.server_id,
-                AppMCPServer.tenant_id == server_ref.tenant_id,
-                AppMCPServer.app_id == server_ref.app_id,
-            )
-            .limit(1)
-        )
+        server = db.session.get(AppMCPServer, payload.id)
         if not server:
             raise NotFound()
 
@@ -176,7 +165,7 @@ class AppMCPServerController(Resource):
             except ValueError:
                 raise ValueError("Invalid status")
         db.session.commit()
-        return dump_response(AppMCPServerResponse, server)
+        return AppMCPServerResponse.model_validate(server, from_attributes=True).model_dump(mode="json")
 
 
 @console_ns.route("/apps/<uuid:server_id>/server/refresh")
@@ -203,4 +192,4 @@ class AppMCPServerRefreshController(Resource):
             raise NotFound()
         server.server_code = AppMCPServer.generate_server_code(16)
         db.session.commit()
-        return dump_response(AppMCPServerResponse, server)
+        return AppMCPServerResponse.model_validate(server, from_attributes=True).model_dump(mode="json")

api/controllers/console/app/message.py

@@ -1,4 +1,5 @@
 import logging
+from datetime import datetime
 from typing import Literal
 from uuid import UUID
 
@@ -12,7 +13,7 @@ from controllers.common.controller_schemas import MessageFeedbackPayload as _Mes
 from controllers.common.fields import SimpleResultResponse, TextFileResponse
 from controllers.common.schema import query_params_from_model, register_response_schema_models, register_schema_models
 from controllers.console import console_ns
-from controllers.console.agent.app_helpers import resolve_agent_runtime_app_model
+from controllers.console.agent.app_helpers import resolve_agent_app_model
 from controllers.console.app.error import (
     CompletionRequestError,
     ProviderModelCurrentlyNotSupportError,
@@ -37,10 +38,16 @@ from core.errors.error import ModelCurrentlyNotSupportError, ProviderTokenNotIni
 from extensions.ext_database import db
 from fields.base import ResponseModel
 from fields.conversation_fields import (
-    MessageDetail as BaseMessageDetailResponse,
+    AgentThought,
+    ConversationAnnotation,
+    ConversationAnnotationHitHistory,
+    Feedback,
+    JSONValue,
+    MessageFile,
+    format_files_contained,
 )
 from graphon.model_runtime.errors.invoke import InvokeError
-from libs.helper import dump_response, uuid_value
+from libs.helper import to_timestamp, uuid_value
 from libs.infinite_scroll_pagination import InfiniteScrollPagination
 from libs.login import login_required
 from models.account import Account
@@ -104,16 +111,49 @@ class FeedbackExportQuery(BaseModel):
         raise ValueError("has_comment must be a boolean value")
 
 
-class AnnotationCountResponse(ResponseModel):
+class AnnotationCountResponse(BaseModel):
     count: int = Field(description="Number of annotations")
 
 
-class SuggestedQuestionsResponse(ResponseModel):
+class SuggestedQuestionsResponse(BaseModel):
     data: list[str] = Field(description="Suggested question")
 
 
-class MessageDetailResponse(BaseMessageDetailResponse):
+class MessageDetailResponse(ResponseModel):
+    id: str
+    conversation_id: str
+    inputs: dict[str, JSONValue]
+    query: str
+    message: JSONValue | None = None
+    message_tokens: int | None = None
+    answer: str = Field(validation_alias="re_sign_file_url_answer")
+    answer_tokens: int | None = None
+    provider_response_latency: float | None = None
+    from_source: str
+    from_end_user_id: str | None = None
+    from_account_id: str | None = None
+    feedbacks: list[Feedback] = Field(default_factory=list)
+    workflow_run_id: str | None = None
+    annotation: ConversationAnnotation | None = None
+    annotation_hit_history: ConversationAnnotationHitHistory | None = None
+    created_at: int | None = None
+    agent_thoughts: list[AgentThought] = Field(default_factory=list)
+    message_files: list[MessageFile] = Field(default_factory=list)
     extra_contents: list[ExecutionExtraContentDomainModel] = Field(default_factory=list)
+    metadata: JSONValue | None = Field(default=None, validation_alias="message_metadata_dict")
+    status: str
+    error: str | None = None
+    parent_message_id: str | None = None
+
+    @field_validator("inputs", mode="before")
+    @classmethod
+    def _normalize_inputs(cls, value: JSONValue) -> JSONValue:
+        return format_files_contained(value)
+
+    @field_validator("created_at", mode="before")
+    @classmethod
+    def _normalize_created_at(cls, value: datetime | int | None) -> int | None:
+        return to_timestamp(value)
 
 
 class MessageInfiniteScrollPaginationResponse(ResponseModel):
@@ -143,7 +183,8 @@ register_response_schema_models(
 class ChatMessageListApi(Resource):
     @console_ns.doc("list_chat_messages")
     @console_ns.doc(description="Get chat messages for a conversation with pagination")
-    @console_ns.doc(params={"app_id": "Application ID", **query_params_from_model(ChatMessagesQuery)})
+    @console_ns.doc(params={"app_id": "Application ID"})
+    @console_ns.doc(params=query_params_from_model(ChatMessagesQuery))
     @console_ns.response(200, "Success", console_ns.models[MessageInfiniteScrollPaginationResponse.__name__])
     @console_ns.response(404, "Conversation not found")
     @login_required
@@ -173,7 +214,7 @@ class AgentChatMessageListApi(Resource):
     @with_current_user
     @with_current_tenant_id
     def get(self, current_tenant_id: str, current_user: Account, agent_id: UUID):
-        app_model = resolve_agent_runtime_app_model(tenant_id=current_tenant_id, agent_id=agent_id)
+        app_model = resolve_agent_app_model(tenant_id=current_tenant_id, agent_id=agent_id)
         return _list_chat_messages(app_model=app_model, current_user=current_user)
 
 
@@ -209,7 +250,7 @@ class AgentMessageFeedbackApi(Resource):
     @with_current_user
     @with_current_tenant_id
     def post(self, current_tenant_id: str, current_user: Account, agent_id: UUID):
-        app_model = resolve_agent_runtime_app_model(tenant_id=current_tenant_id, agent_id=agent_id)
+        app_model = resolve_agent_app_model(tenant_id=current_tenant_id, agent_id=agent_id)
         return _update_message_feedback(current_user=current_user, app_model=app_model)
 
 
@@ -233,7 +274,7 @@ class MessageAnnotationCountApi(Resource):
             select(func.count(MessageAnnotation.id)).where(MessageAnnotation.app_id == app_model.id)
         )
 
-        return AnnotationCountResponse(count=count or 0).model_dump(mode="json")
+        return {"count": count}
 
 
 @console_ns.route("/apps/<uuid:app_id>/chat-messages/<uuid:message_id>/suggested-questions")
@@ -274,7 +315,7 @@ class AgentMessageSuggestedQuestionApi(Resource):
     @with_current_user
     @with_current_tenant_id
     def get(self, current_tenant_id: str, current_user: Account, agent_id: UUID, message_id: UUID):
-        app_model = resolve_agent_runtime_app_model(tenant_id=current_tenant_id, agent_id=agent_id)
+        app_model = resolve_agent_app_model(tenant_id=current_tenant_id, agent_id=agent_id)
         return _get_message_suggested_questions(current_user=current_user, app_model=app_model, message_id=message_id)
 
 
@@ -282,12 +323,13 @@ class AgentMessageSuggestedQuestionApi(Resource):
 class MessageFeedbackExportApi(Resource):
     @console_ns.doc("export_feedbacks")
     @console_ns.doc(description="Export user feedback data for Google Sheets")
+    @console_ns.doc(params={"app_id": "Application ID"})
+    @console_ns.doc(params=query_params_from_model(FeedbackExportQuery))
     @console_ns.response(
         200,
         "Feedback data exported successfully",
         console_ns.models[TextFileResponse.__name__],
     )
-    @console_ns.doc(params={"app_id": "Application ID", **query_params_from_model(FeedbackExportQuery)})
     @console_ns.response(400, "Invalid parameters")
     @console_ns.response(500, "Internal server error")
     @setup_required
@@ -312,6 +354,7 @@ class MessageFeedbackExportApi(Resource):
                 end_date=args.end_date,
                 format_type=args.format,
             )
+
             return export_data
 
         except ValueError as e:
@@ -350,7 +393,7 @@ class AgentMessageApi(Resource):
     @account_initialization_required
     @with_current_tenant_id
     def get(self, current_tenant_id: str, agent_id: UUID, message_id: UUID):
-        app_model = resolve_agent_runtime_app_model(tenant_id=current_tenant_id, agent_id=agent_id)
+        app_model = resolve_agent_app_model(tenant_id=current_tenant_id, agent_id=agent_id)
         return _get_message_detail(app_model=app_model, message_id=message_id)
 
 
@@ -363,7 +406,6 @@ def _list_chat_messages(*, app_model: App, current_user: Account | None = None):
                 app_model=app_model,
                 conversation_id=args.conversation_id,
                 user=current_user,
-                session=db.session(),
             )
         except ConversationNotExistsError:
             raise NotFound("Conversation Not Exists.")
@@ -423,16 +465,16 @@ def _list_chat_messages(*, app_model: App, current_user: Account | None = None):
     history_messages = list(reversed(history_messages))
     attach_message_extra_contents(history_messages)
 
-    return dump_response(
-        MessageInfiniteScrollPaginationResponse,
+    return MessageInfiniteScrollPaginationResponse.model_validate(
         InfiniteScrollPagination(data=history_messages, limit=args.limit, has_more=has_more),
-    )
+        from_attributes=True,
+    ).model_dump(mode="json")
 
 
 def _update_message_feedback(*, current_user: Account, app_model: App):
     args = MessageFeedbackPayload.model_validate(console_ns.payload)
 
-    message_id = args.message_id
+    message_id = str(args.message_id)
 
     message = db.session.scalar(
         select(Message).where(Message.id == message_id, Message.app_id == app_model.id).limit(1)
@@ -467,7 +509,7 @@ def _update_message_feedback(*, current_user: Account, app_model: App):
 
     db.session.commit()
 
-    return SimpleResultResponse(result="success").model_dump(mode="json")
+    return {"result": "success"}
 
 
 def _get_message_suggested_questions(*, current_user: Account, app_model: App, message_id: UUID):
@@ -475,11 +517,7 @@ def _get_message_suggested_questions(*, current_user: Account, app_model: App, m
 
     try:
         questions = MessageService.get_suggested_questions_after_answer(
-            app_model=app_model,
-            message_id=message_id_str,
-            user=current_user,
-            invoke_from=InvokeFrom.DEBUGGER,
-            session=db.session(),
+            app_model=app_model, message_id=message_id_str, user=current_user, invoke_from=InvokeFrom.DEBUGGER
         )
     except MessageNotExistsError:
         raise NotFound("Message not found")
@@ -499,7 +537,7 @@ def _get_message_suggested_questions(*, current_user: Account, app_model: App, m
         logger.exception("internal server error.")
         raise InternalServerError()
 
-    return dump_response(SuggestedQuestionsResponse, {"data": questions})
+    return {"data": questions}
 
 
 def _get_message_detail(*, app_model: App, message_id: UUID):
@@ -513,4 +551,4 @@ def _get_message_detail(*, app_model: App, message_id: UUID):
         raise NotFound("Message Not Exists.")
 
     attach_message_extra_contents([message])
-    return dump_response(MessageDetailResponse, message)
+    return MessageDetailResponse.model_validate(message, from_attributes=True).model_dump(mode="json")

api/controllers/console/app/ops_trace.py

@@ -17,7 +17,6 @@ from controllers.console.wraps import (
     rbac_permission_required,
     setup_required,
 )
-from extensions.ext_database import db
 from fields.base import ResponseModel
 from libs.login import login_required
 from models import App
@@ -79,7 +78,7 @@ class TraceAppConfigApi(Resource):
 
         try:
             trace_config = OpsService.get_tracing_app_config(
-                app_id=app_model.id, tracing_provider=args.tracing_provider, session=db.session()
+                app_id=app_model.id, tracing_provider=args.tracing_provider
             )
             if not trace_config:
                 return {"has_not_configured": True}
@@ -110,10 +109,7 @@ class TraceAppConfigApi(Resource):
 
         try:
             result = OpsService.create_tracing_app_config(
-                app_id=app_model.id,
-                tracing_provider=args.tracing_provider,
-                tracing_config=args.tracing_config,
-                session=db.session(),
+                app_id=app_model.id, tracing_provider=args.tracing_provider, tracing_config=args.tracing_config
             )
             if not result:
                 raise TracingConfigIsExist()
@@ -146,10 +142,7 @@ class TraceAppConfigApi(Resource):
 
         try:
             result = OpsService.update_tracing_app_config(
-                app_id=app_model.id,
-                tracing_provider=args.tracing_provider,
-                tracing_config=args.tracing_config,
-                session=db.session(),
+                app_id=app_model.id, tracing_provider=args.tracing_provider, tracing_config=args.tracing_config
             )
             if not result:
                 raise TracingConfigNotExist()
@@ -175,9 +168,7 @@ class TraceAppConfigApi(Resource):
         args = TraceProviderQuery.model_validate(request.args.to_dict(flat=True))
 
         try:
-            result = OpsService.delete_tracing_app_config(
-                app_id=app_model.id, tracing_provider=args.tracing_provider, session=db.session()
-            )
+            result = OpsService.delete_tracing_app_config(app_id=app_model.id, tracing_provider=args.tracing_provider)
             if not result:
                 raise TracingConfigNotExist()
             return "", 204

api/controllers/console/app/permission_keys.py

@@ -1,9 +1,6 @@
-from extensions.ext_database import db
 from services.enterprise import rbac_service as enterprise_rbac_service
 
 
 def get_app_permission_keys(tenant_id: str, account_id: str | None, app_id: str) -> list[str]:
-    permission_keys_map = enterprise_rbac_service.RBACService.AppPermissions.batch_get(
-        tenant_id, account_id, [app_id], session=db.session()
-    )
+    permission_keys_map = enterprise_rbac_service.RBACService.AppPermissions.batch_get(tenant_id, account_id, [app_id])
     return permission_keys_map.get(app_id, [])

api/controllers/console/app/site.py

@@ -22,7 +22,6 @@ from controllers.console.wraps import (
 from extensions.ext_database import db
 from fields.base import ResponseModel
 from libs.datetime_utils import naive_utc_now
-from libs.helper import dump_response
 from libs.login import login_required
 from models import Site
 from models.account import Account
@@ -41,7 +40,6 @@ class AppSiteUpdatePayload(BaseModel):
     customize_domain: str | None = Field(default=None)
     copyright: str | None = Field(default=None)
     privacy_policy: str | None = Field(default=None)
-    input_placeholder: str | None = Field(default=None)
     custom_disclaimer: str | None = Field(default=None)
     customize_token_strategy: Literal["must", "allow", "not_allow"] | None = Field(default=None)
     prompt_public: bool | None = Field(default=None)
@@ -68,7 +66,6 @@ class AppSiteResponse(ResponseModel):
     customize_domain: str | None = None
     copyright: str | None = None
     privacy_policy: str | None = None
-    input_placeholder: str | None = None
     custom_disclaimer: str | None = None
     customize_token_strategy: str
     prompt_public: bool
@@ -113,7 +110,6 @@ class AppSite(Resource):
             "customize_domain",
             "copyright",
             "privacy_policy",
-            "input_placeholder",
             "custom_disclaimer",
             "customize_token_strategy",
             "prompt_public",
@@ -128,7 +124,7 @@ class AppSite(Resource):
         site.updated_at = naive_utc_now()
         db.session.commit()
 
-        return dump_response(AppSiteResponse, site)
+        return AppSiteResponse.model_validate(site, from_attributes=True).model_dump(mode="json")
 
 
 @console_ns.route("/apps/<uuid:app_id>/site/access-token-reset")
@@ -157,4 +153,4 @@ class AppSiteAccessTokenReset(Resource):
         site.updated_at = naive_utc_now()
         db.session.commit()
 
-        return dump_response(AppSiteResponse, site)
+        return AppSiteResponse.model_validate(site, from_attributes=True).model_dump(mode="json")

api/controllers/console/app/statistic.py

@@ -1,7 +1,7 @@
 from decimal import Decimal
 
 import sqlalchemy as sa
-from flask import abort, request
+from flask import abort, jsonify, request
 from flask_restx import Resource
 from pydantic import BaseModel, Field, field_validator
 
@@ -20,7 +20,7 @@ from core.app.entities.app_invoke_entities import InvokeFrom
 from extensions.ext_database import db
 from fields.base import ResponseModel
 from libs.datetime_utils import parse_time_range
-from libs.helper import convert_datetime_to_date, dump_response
+from libs.helper import convert_datetime_to_date
 from libs.login import login_required
 from models import AppMode
 from models.account import Account
@@ -44,15 +44,8 @@ class DailyMessageStatisticItem(ResponseModel):
     message_count: int
 
 
-register_schema_models(console_ns, StatisticTimeRangeQuery)
-
-
-class StatisticDataResponse[T](ResponseModel):
-    data: list[T]
-
-
-class DailyMessageStatisticResponse(StatisticDataResponse[DailyMessageStatisticItem]):
-    pass
+class DailyMessageStatisticResponse(ResponseModel):
+    data: list[DailyMessageStatisticItem]
 
 
 class DailyConversationStatisticItem(ResponseModel):
@@ -60,8 +53,8 @@ class DailyConversationStatisticItem(ResponseModel):
     conversation_count: int
 
 
-class DailyConversationStatisticResponse(StatisticDataResponse[DailyConversationStatisticItem]):
-    pass
+class DailyConversationStatisticResponse(ResponseModel):
+    data: list[DailyConversationStatisticItem]
 
 
 class DailyTerminalStatisticItem(ResponseModel):
@@ -69,19 +62,19 @@ class DailyTerminalStatisticItem(ResponseModel):
     terminal_count: int
 
 
-class DailyTerminalStatisticResponse(StatisticDataResponse[DailyTerminalStatisticItem]):
-    pass
+class DailyTerminalStatisticResponse(ResponseModel):
+    data: list[DailyTerminalStatisticItem]
 
 
 class DailyTokenCostStatisticItem(ResponseModel):
     date: str
-    token_count: int | None = None
-    total_price: Decimal | None = None
-    currency: str | None = None
+    token_count: int
+    total_price: str | float
+    currency: str
 
 
-class DailyTokenCostStatisticResponse(StatisticDataResponse[DailyTokenCostStatisticItem]):
-    pass
+class DailyTokenCostStatisticResponse(ResponseModel):
+    data: list[DailyTokenCostStatisticItem]
 
 
 class AverageSessionInteractionStatisticItem(ResponseModel):
@@ -89,8 +82,8 @@ class AverageSessionInteractionStatisticItem(ResponseModel):
     interactions: float
 
 
-class AverageSessionInteractionStatisticResponse(StatisticDataResponse[AverageSessionInteractionStatisticItem]):
-    pass
+class AverageSessionInteractionStatisticResponse(ResponseModel):
+    data: list[AverageSessionInteractionStatisticItem]
 
 
 class UserSatisfactionRateStatisticItem(ResponseModel):
@@ -98,8 +91,8 @@ class UserSatisfactionRateStatisticItem(ResponseModel):
     rate: float
 
 
-class UserSatisfactionRateStatisticResponse(StatisticDataResponse[UserSatisfactionRateStatisticItem]):
-    pass
+class UserSatisfactionRateStatisticResponse(ResponseModel):
+    data: list[UserSatisfactionRateStatisticItem]
 
 
 class AverageResponseTimeStatisticItem(ResponseModel):
@@ -107,8 +100,8 @@ class AverageResponseTimeStatisticItem(ResponseModel):
     latency: float
 
 
-class AverageResponseTimeStatisticResponse(StatisticDataResponse[AverageResponseTimeStatisticItem]):
-    pass
+class AverageResponseTimeStatisticResponse(ResponseModel):
+    data: list[AverageResponseTimeStatisticItem]
 
 
 class TokensPerSecondStatisticItem(ResponseModel):
@@ -116,27 +109,20 @@ class TokensPerSecondStatisticItem(ResponseModel):
     tps: float
 
 
-class TokensPerSecondStatisticResponse(StatisticDataResponse[TokensPerSecondStatisticItem]):
-    pass
+class TokensPerSecondStatisticResponse(ResponseModel):
+    data: list[TokensPerSecondStatisticItem]
 
 
+register_schema_models(console_ns, StatisticTimeRangeQuery)
 register_response_schema_models(
     console_ns,
-    DailyMessageStatisticItem,
     DailyMessageStatisticResponse,
-    DailyConversationStatisticItem,
     DailyConversationStatisticResponse,
-    DailyTerminalStatisticItem,
     DailyTerminalStatisticResponse,
-    DailyTokenCostStatisticItem,
     DailyTokenCostStatisticResponse,
-    AverageSessionInteractionStatisticItem,
     AverageSessionInteractionStatisticResponse,
-    UserSatisfactionRateStatisticItem,
     UserSatisfactionRateStatisticResponse,
-    AverageResponseTimeStatisticItem,
     AverageResponseTimeStatisticResponse,
-    TokensPerSecondStatisticItem,
     TokensPerSecondStatisticResponse,
 )
 
@@ -145,7 +131,8 @@ register_response_schema_models(
 class DailyMessageStatistic(Resource):
     @console_ns.doc("get_daily_message_statistics")
     @console_ns.doc(description="Get daily message statistics for an application")
-    @console_ns.doc(params={"app_id": "Application ID", **query_params_from_model(StatisticTimeRangeQuery)})
+    @console_ns.doc(params={"app_id": "Application ID"})
+    @console_ns.doc(params=query_params_from_model(StatisticTimeRangeQuery))
     @console_ns.response(
         200,
         "Daily message statistics retrieved successfully",
@@ -198,14 +185,15 @@ WHERE
             for i in rs:
                 response_data.append({"date": str(i.date), "message_count": i.message_count})
 
-        return dump_response(DailyMessageStatisticResponse, {"data": response_data})
+        return jsonify({"data": response_data})
 
 
 @console_ns.route("/apps/<uuid:app_id>/statistics/daily-conversations")
 class DailyConversationStatistic(Resource):
     @console_ns.doc("get_daily_conversation_statistics")
     @console_ns.doc(description="Get daily conversation statistics for an application")
-    @console_ns.doc(params={"app_id": "Application ID", **query_params_from_model(StatisticTimeRangeQuery)})
+    @console_ns.doc(params={"app_id": "Application ID"})
+    @console_ns.doc(params=query_params_from_model(StatisticTimeRangeQuery))
     @console_ns.response(
         200,
         "Daily conversation statistics retrieved successfully",
@@ -257,14 +245,15 @@ WHERE
             for i in rs:
                 response_data.append({"date": str(i.date), "conversation_count": i.conversation_count})
 
-        return dump_response(DailyConversationStatisticResponse, {"data": response_data})
+        return jsonify({"data": response_data})
 
 
 @console_ns.route("/apps/<uuid:app_id>/statistics/daily-end-users")
 class DailyTerminalsStatistic(Resource):
     @console_ns.doc("get_daily_terminals_statistics")
     @console_ns.doc(description="Get daily terminal/end-user statistics for an application")
-    @console_ns.doc(params={"app_id": "Application ID", **query_params_from_model(StatisticTimeRangeQuery)})
+    @console_ns.doc(params={"app_id": "Application ID"})
+    @console_ns.doc(params=query_params_from_model(StatisticTimeRangeQuery))
     @console_ns.response(
         200,
         "Daily terminal statistics retrieved successfully",
@@ -317,14 +306,15 @@ WHERE
             for i in rs:
                 response_data.append({"date": str(i.date), "terminal_count": i.terminal_count})
 
-        return dump_response(DailyTerminalStatisticResponse, {"data": response_data})
+        return jsonify({"data": response_data})
 
 
 @console_ns.route("/apps/<uuid:app_id>/statistics/token-costs")
 class DailyTokenCostStatistic(Resource):
     @console_ns.doc("get_daily_token_cost_statistics")
     @console_ns.doc(description="Get daily token cost statistics for an application")
-    @console_ns.doc(params={"app_id": "Application ID", **query_params_from_model(StatisticTimeRangeQuery)})
+    @console_ns.doc(params={"app_id": "Application ID"})
+    @console_ns.doc(params=query_params_from_model(StatisticTimeRangeQuery))
     @console_ns.response(
         200,
         "Daily token cost statistics retrieved successfully",
@@ -380,14 +370,15 @@ WHERE
                     {"date": str(i.date), "token_count": i.token_count, "total_price": i.total_price, "currency": "USD"}
                 )
 
-        return dump_response(DailyTokenCostStatisticResponse, {"data": response_data})
+        return jsonify({"data": response_data})
 
 
 @console_ns.route("/apps/<uuid:app_id>/statistics/average-session-interactions")
 class AverageSessionInteractionStatistic(Resource):
     @console_ns.doc("get_average_session_interaction_statistics")
     @console_ns.doc(description="Get average session interaction statistics for an application")
-    @console_ns.doc(params={"app_id": "Application ID", **query_params_from_model(StatisticTimeRangeQuery)})
+    @console_ns.doc(params={"app_id": "Application ID"})
+    @console_ns.doc(params=query_params_from_model(StatisticTimeRangeQuery))
     @console_ns.response(
         200,
         "Average session interaction statistics retrieved successfully",
@@ -459,14 +450,15 @@ ORDER BY
                     {"date": str(i.date), "interactions": float(i.interactions.quantize(Decimal("0.01")))}
                 )
 
-        return dump_response(AverageSessionInteractionStatisticResponse, {"data": response_data})
+        return jsonify({"data": response_data})
 
 
 @console_ns.route("/apps/<uuid:app_id>/statistics/user-satisfaction-rate")
 class UserSatisfactionRateStatistic(Resource):
     @console_ns.doc("get_user_satisfaction_rate_statistics")
     @console_ns.doc(description="Get user satisfaction rate statistics for an application")
-    @console_ns.doc(params={"app_id": "Application ID", **query_params_from_model(StatisticTimeRangeQuery)})
+    @console_ns.doc(params={"app_id": "Application ID"})
+    @console_ns.doc(params=query_params_from_model(StatisticTimeRangeQuery))
     @console_ns.response(
         200,
         "User satisfaction rate statistics retrieved successfully",
@@ -528,14 +520,15 @@ WHERE
                     }
                 )
 
-        return dump_response(UserSatisfactionRateStatisticResponse, {"data": response_data})
+        return jsonify({"data": response_data})
 
 
 @console_ns.route("/apps/<uuid:app_id>/statistics/average-response-time")
 class AverageResponseTimeStatistic(Resource):
     @console_ns.doc("get_average_response_time_statistics")
     @console_ns.doc(description="Get average response time statistics for an application")
-    @console_ns.doc(params={"app_id": "Application ID", **query_params_from_model(StatisticTimeRangeQuery)})
+    @console_ns.doc(params={"app_id": "Application ID"})
+    @console_ns.doc(params=query_params_from_model(StatisticTimeRangeQuery))
     @console_ns.response(
         200,
         "Average response time statistics retrieved successfully",
@@ -588,14 +581,15 @@ WHERE
             for i in rs:
                 response_data.append({"date": str(i.date), "latency": round(i.latency * 1000, 4)})
 
-        return dump_response(AverageResponseTimeStatisticResponse, {"data": response_data})
+        return jsonify({"data": response_data})
 
 
 @console_ns.route("/apps/<uuid:app_id>/statistics/tokens-per-second")
 class TokensPerSecondStatistic(Resource):
     @console_ns.doc("get_tokens_per_second_statistics")
     @console_ns.doc(description="Get tokens per second statistics for an application")
-    @console_ns.doc(params={"app_id": "Application ID", **query_params_from_model(StatisticTimeRangeQuery)})
+    @console_ns.doc(params={"app_id": "Application ID"})
+    @console_ns.doc(params=query_params_from_model(StatisticTimeRangeQuery))
     @console_ns.response(
         200,
         "Tokens per second statistics retrieved successfully",
@@ -651,4 +645,4 @@ WHERE
             for i in rs:
                 response_data.append({"date": str(i.date), "tps": round(i.tokens_per_second, 4)})
 
-        return dump_response(TokensPerSecondStatisticResponse, {"data": response_data})
+        return jsonify({"data": response_data})

api/controllers/console/app/workflow.py

@@ -2,11 +2,11 @@ import json
 import logging
 from collections.abc import Sequence
 from datetime import datetime
-from typing import Any, NotRequired, TypedDict
+from typing import Any, NotRequired, TypedDict, cast
 
 from flask import abort, request
 from flask_restx import Resource, fields
-from pydantic import AliasChoices, BaseModel, ConfigDict, Field, RootModel, ValidationError, field_validator
+from pydantic import AliasChoices, BaseModel, Field, RootModel, ValidationError, field_validator
 from sqlalchemy.orm import Session, sessionmaker
 from werkzeug.exceptions import BadRequest, Forbidden, InternalServerError, NotFound
 
@@ -27,7 +27,7 @@ from controllers.console.app.error import (
     DraftWorkflowNotSync,
 )
 from controllers.console.app.permission_keys import get_app_permission_keys
-from controllers.console.app.wraps import get_app_model, with_session
+from controllers.console.app.wraps import get_app_model
 from controllers.console.wraps import (
     RBACPermission,
     RBACResourceScope,
@@ -78,7 +78,6 @@ from repositories.workflow_collaboration_repository import WORKFLOW_ONLINE_USERS
 from services.app_generate_service import AppGenerateService
 from services.errors.app import IsDraftWorkflowError, WorkflowHashNotEqualError, WorkflowNotFoundError
 from services.errors.llm import InvokeRateLimitError
-from services.workflow_ref_service import WorkflowRefService
 from services.workflow_service import DraftWorkflowDeletionError, WorkflowInUseError, WorkflowService
 
 logger = logging.getLogger(__name__)
@@ -159,71 +158,8 @@ class ConvertToWorkflowPayload(BaseModel):
     icon_background: str | None = None
 
 
-class WorkflowFeatureTogglePayload(BaseModel):
-    model_config = ConfigDict(extra="allow")
-
-    enabled: bool | None = None
-
-
-class WorkflowSuggestedQuestionsAfterAnswerPayload(WorkflowFeatureTogglePayload):
-    model: dict[str, Any] | None = None
-    prompt: str | None = None
-
-
-class WorkflowTextToSpeechPayload(WorkflowFeatureTogglePayload):
-    language: str | None = None
-    voice: str | None = None
-    autoPlay: str | None = None
-
-
-class WorkflowSensitiveWordAvoidancePayload(WorkflowFeatureTogglePayload):
-    type: str | None = None
-    config: dict[str, Any] | None = None
-
-
-class WorkflowFileUploadTransferPayload(WorkflowFeatureTogglePayload):
-    number_limits: int | None = None
-    transfer_methods: list[str] | None = None
-
-
-class WorkflowFileUploadImagePayload(WorkflowFileUploadTransferPayload):
-    detail: str | None = None
-
-
-class WorkflowFileUploadPreviewConfigPayload(BaseModel):
-    mode: str | None = None
-    file_type_list: list[str] | None = None
-
-
-class WorkflowFileUploadPayload(WorkflowFeatureTogglePayload):
-    allowed_file_types: list[str] | None = None
-    allowed_file_extensions: list[str] | None = None
-    allowed_file_upload_methods: list[str] | None = None
-    number_limits: int | None = None
-    image: WorkflowFileUploadImagePayload | None = None
-    document: WorkflowFileUploadTransferPayload | None = None
-    audio: WorkflowFileUploadTransferPayload | None = None
-    video: WorkflowFileUploadTransferPayload | None = None
-    custom: WorkflowFileUploadTransferPayload | None = None
-    preview_config: WorkflowFileUploadPreviewConfigPayload | None = None
-    fileUploadConfig: dict[str, Any] | None = None
-
-
-class WorkflowFeaturesConfigPayload(BaseModel):
-    model_config = ConfigDict(extra="allow")
-
-    opening_statement: str | None = None
-    suggested_questions: list[str] | None = None
-    suggested_questions_after_answer: WorkflowSuggestedQuestionsAfterAnswerPayload | None = None
-    text_to_speech: WorkflowTextToSpeechPayload | None = None
-    speech_to_text: WorkflowFeatureTogglePayload | None = None
-    retriever_resource: WorkflowFeatureTogglePayload | None = None
-    sensitive_word_avoidance: WorkflowSensitiveWordAvoidancePayload | None = None
-    file_upload: WorkflowFileUploadPayload | None = None
-
-
 class WorkflowFeaturesPayload(BaseModel):
-    features: WorkflowFeaturesConfigPayload = Field(
+    features: dict[str, Any] = Field(
         ...,
         description="Workflow feature configuration",
     )
@@ -407,15 +343,6 @@ register_schema_models(
     ConvertToWorkflowPayload,
     WorkflowListQuery,
     WorkflowUpdatePayload,
-    WorkflowFeatureTogglePayload,
-    WorkflowSuggestedQuestionsAfterAnswerPayload,
-    WorkflowTextToSpeechPayload,
-    WorkflowSensitiveWordAvoidancePayload,
-    WorkflowFileUploadTransferPayload,
-    WorkflowFileUploadImagePayload,
-    WorkflowFileUploadPreviewConfigPayload,
-    WorkflowFileUploadPayload,
-    WorkflowFeaturesConfigPayload,
     WorkflowFeaturesPayload,
     WorkflowOnlineUsersPayload,
     DraftWorkflowTriggerRunPayload,
@@ -522,7 +449,7 @@ class DraftWorkflowApi(Resource):
         """
         # fetch draft workflow by app_model
         workflow_service = WorkflowService()
-        workflow = workflow_service.get_draft_workflow(app_model=app_model, session=db.session())
+        workflow = workflow_service.get_draft_workflow(app_model=app_model)
 
         if not workflow:
             raise DraftWorkflowNotExist()
@@ -533,7 +460,7 @@ class DraftWorkflowApi(Resource):
         # front-end can treat draft graph node data as the editing source.
         response = WorkflowResponse.model_validate(workflow, from_attributes=True).model_dump(mode="json")
         response["graph"] = WorkflowAgentPublishService.project_draft_bindings_to_graph(
-            session=db.session(),
+            session=cast(Session, db.session),
             draft_workflow=workflow,
         )
         return response
@@ -602,7 +529,6 @@ class DraftWorkflowApi(Resource):
                 account=current_user,
                 environment_variables=environment_variables,
                 conversation_variables=conversation_variables,
-                session=db.session(),
             )
         except WorkflowHashNotEqualError:
             raise DraftWorkflowNotSync()
@@ -632,8 +558,7 @@ class AdvancedChatDraftWorkflowRunApi(Resource):
     @get_app_model(mode=[AppMode.ADVANCED_CHAT])
     @with_current_user
     @edit_permission_required
-    @with_session
-    def post(self, session: Session, current_user: Account, app_model: App):
+    def post(self, current_user: Account, app_model: App):
         """
         Run draft workflow
         """
@@ -646,12 +571,7 @@ class AdvancedChatDraftWorkflowRunApi(Resource):
 
         try:
             response = AppGenerateService.generate(
-                session=session,
-                app_model=app_model,
-                user=current_user,
-                args=args,
-                invoke_from=InvokeFrom.DEBUGGER,
-                streaming=True,
+                app_model=app_model, user=current_user, args=args, invoke_from=InvokeFrom.DEBUGGER, streaming=True
             )
 
             return helper.compact_generate_response(response)
@@ -696,12 +616,7 @@ class AdvancedChatDraftRunIterationNodeApi(Resource):
 
         try:
             response = AppGenerateService.generate_single_iteration(
-                app_model=app_model,
-                user=current_user,
-                node_id=node_id,
-                args=args,
-                session=db.session(),
-                streaming=True,
+                app_model=app_model, user=current_user, node_id=node_id, args=args, streaming=True
             )
 
             return helper.compact_generate_response(response)
@@ -744,12 +659,7 @@ class WorkflowDraftRunIterationNodeApi(Resource):
 
         try:
             response = AppGenerateService.generate_single_iteration(
-                app_model=app_model,
-                user=current_user,
-                node_id=node_id,
-                args=args,
-                session=db.session(),
-                streaming=True,
+                app_model=app_model, user=current_user, node_id=node_id, args=args, streaming=True
             )
 
             return helper.compact_generate_response(response)
@@ -788,12 +698,7 @@ class AdvancedChatDraftRunLoopNodeApi(Resource):
 
         try:
             response = AppGenerateService.generate_single_loop(
-                app_model=app_model,
-                user=current_user,
-                node_id=node_id,
-                args=args,
-                session=db.session(),
-                streaming=True,
+                app_model=app_model, user=current_user, node_id=node_id, args=args, streaming=True
             )
 
             return helper.compact_generate_response(response)
@@ -836,12 +741,7 @@ class WorkflowDraftRunLoopNodeApi(Resource):
 
         try:
             response = AppGenerateService.generate_single_loop(
-                app_model=app_model,
-                user=current_user,
-                node_id=node_id,
-                args=args,
-                session=db.session(),
-                streaming=True,
+                app_model=app_model, user=current_user, node_id=node_id, args=args, streaming=True
             )
 
             return helper.compact_generate_response(response)
@@ -918,7 +818,6 @@ class AdvancedChatDraftHumanInputFormPreviewApi(Resource):
             account=current_user,
             node_id=node_id,
             inputs=inputs,
-            session=db.session(),
         )
         return jsonable_encoder(preview)
 
@@ -954,7 +853,6 @@ class AdvancedChatDraftHumanInputFormRunApi(Resource):
             form_inputs=args.form_inputs,
             inputs=args.inputs,
             action=args.action,
-            session=db.session(),
         )
         return jsonable_encoder(result)
 
@@ -986,7 +884,6 @@ class WorkflowDraftHumanInputFormPreviewApi(Resource):
             account=current_user,
             node_id=node_id,
             inputs=inputs,
-            session=db.session(),
         )
         return jsonable_encoder(preview)
 
@@ -1022,7 +919,6 @@ class WorkflowDraftHumanInputFormRunApi(Resource):
             form_inputs=args.form_inputs,
             inputs=args.inputs,
             action=args.action,
-            session=db.session(),
         )
         return jsonable_encoder(result)
 
@@ -1053,7 +949,6 @@ class WorkflowDraftHumanInputDeliveryTestApi(Resource):
             node_id=node_id,
             delivery_method_id=args.delivery_method_id,
             inputs=args.inputs,
-            session=db.session(),
         )
         return jsonable_encoder({})
 
@@ -1077,8 +972,7 @@ class DraftWorkflowRunApi(Resource):
     @get_app_model(mode=[AppMode.WORKFLOW])
     @with_current_user
     @edit_permission_required
-    @with_session
-    def post(self, session: Session, current_user: Account, app_model: App):
+    def post(self, current_user: Account, app_model: App):
         """
         Run draft workflow
         """
@@ -1090,7 +984,6 @@ class DraftWorkflowRunApi(Resource):
 
         try:
             response = AppGenerateService.generate(
-                session=session,
                 app_model=app_model,
                 user=current_user,
                 args=args,
@@ -1164,7 +1057,7 @@ class DraftWorkflowNodeRunApi(Resource):
 
         workflow_srv = WorkflowService()
         # fetch draft workflow by app_model
-        draft_workflow = workflow_srv.get_draft_workflow(app_model=app_model, session=db.session())
+        draft_workflow = workflow_srv.get_draft_workflow(app_model=app_model)
         if not draft_workflow:
             raise ValueError("Workflow not initialized")
         files = _parse_file(draft_workflow, args.get("files"))
@@ -1207,7 +1100,7 @@ class PublishedWorkflowApi(Resource):
         """
         # fetch published workflow by app_model
         workflow_service = WorkflowService()
-        workflow = workflow_service.get_published_workflow(app_model=app_model, session=db.session())
+        workflow = workflow_service.get_published_workflow(app_model=app_model)
 
         # return workflow, if not found, return None
         if workflow is None:
@@ -1349,9 +1242,7 @@ class ConvertToWorkflowApi(Resource):
 
         # convert to workflow mode
         workflow_service = WorkflowService()
-        new_app_model = workflow_service.convert_to_workflow(
-            app_model=app_model, account=current_user, args=args, session=db.session()
-        )
+        new_app_model = workflow_service.convert_to_workflow(app_model=app_model, account=current_user, args=args)
 
         # return app id
         return {
@@ -1383,12 +1274,10 @@ class WorkflowFeaturesApi(Resource):
     def post(self, current_user: Account, app_model: App):
 
         args = WorkflowFeaturesPayload.model_validate(console_ns.payload or {})
-        features = args.features.model_dump(mode="json", exclude_unset=True)
+        features = args.features
 
         workflow_service = WorkflowService()
-        workflow_service.update_draft_workflow_features(
-            app_model=app_model, features=features, account=current_user, session=db.session()
-        )
+        workflow_service.update_draft_workflow_features(app_model=app_model, features=features, account=current_user)
 
         return {"result": "success"}
 
@@ -1469,7 +1358,6 @@ class DraftWorkflowRestoreApi(Resource):
                 app_model=app_model,
                 workflow_id=workflow_id,
                 account=current_user,
-                session=db.session(),
             )
         except IsDraftWorkflowError as exc:
             raise BadRequest(RESTORE_SOURCE_WORKFLOW_MUST_BE_PUBLISHED_MESSAGE) from exc
@@ -1518,15 +1406,15 @@ class WorkflowByIdApi(Resource):
             return {"message": "No valid fields to update"}, 400
 
         workflow_service = WorkflowService()
-        workflow_ref = WorkflowRefService.create_app_workflow_ref(app_model, workflow_id)
 
         # Create a session and manage the transaction
         with sessionmaker(db.engine, expire_on_commit=False).begin() as session:
             workflow = workflow_service.update_workflow(
                 session=session,
+                workflow_id=workflow_id,
+                tenant_id=app_model.tenant_id,
                 account_id=current_user.id,
                 data=update_data,
-                workflow_ref=workflow_ref,
             )
 
             if not workflow:
@@ -1546,14 +1434,12 @@ class WorkflowByIdApi(Resource):
         Delete workflow
         """
         workflow_service = WorkflowService()
-        workflow_ref = WorkflowRefService.create_app_workflow_ref(app_model, workflow_id)
 
         # Create a session and manage the transaction
         with sessionmaker(db.engine).begin() as session:
             try:
                 workflow_service.delete_workflow(
-                    session=session,
-                    workflow_ref=workflow_ref,
+                    session=session, workflow_id=workflow_id, tenant_id=app_model.tenant_id
                 )
             except WorkflowInUseError as e:
                 abort(400, description=str(e))
@@ -1584,7 +1470,7 @@ class DraftWorkflowNodeLastRunApi(Resource):
     @get_app_model(mode=[AppMode.ADVANCED_CHAT, AppMode.WORKFLOW])
     def get(self, app_model: App, node_id: str):
         srv = WorkflowService()
-        workflow = srv.get_draft_workflow(app_model, session=db.session())
+        workflow = srv.get_draft_workflow(app_model)
         if not workflow:
             raise NotFound("Workflow not found")
         node_exec = srv.get_node_last_run(
@@ -1629,15 +1515,14 @@ class DraftWorkflowTriggerRunApi(Resource):
     @get_app_model(mode=[AppMode.WORKFLOW])
     @with_current_user
     @edit_permission_required
-    @with_session
-    def post(self, session: Session, current_user: Account, app_model: App):
+    def post(self, current_user: Account, app_model: App):
         """
         Poll for trigger events and execute full workflow when event arrives
         """
         args = DraftWorkflowTriggerRunPayload.model_validate(console_ns.payload or {})
         node_id = args.node_id
         workflow_service = WorkflowService()
-        draft_workflow = workflow_service.get_draft_workflow(app_model, session=db.session())
+        draft_workflow = workflow_service.get_draft_workflow(app_model)
         if not draft_workflow:
             raise ValueError("Workflow not found")
 
@@ -1658,7 +1543,6 @@ class DraftWorkflowTriggerRunApi(Resource):
             workflow_args[SKIP_PREPARE_USER_INPUTS_KEY] = True
             return helper.compact_generate_response(
                 AppGenerateService.generate(
-                    session=session,
                     app_model=app_model,
                     user=current_user,
                     args=workflow_args,
@@ -1706,7 +1590,7 @@ class DraftWorkflowTriggerNodeApi(Resource):
         """
 
         workflow_service = WorkflowService()
-        draft_workflow = workflow_service.get_draft_workflow(app_model, session=db.session())
+        draft_workflow = workflow_service.get_draft_workflow(app_model)
         if not draft_workflow:
             raise ValueError("Workflow not found")
 
@@ -1781,8 +1665,7 @@ class DraftWorkflowTriggerRunAllApi(Resource):
     @with_current_user
     @edit_permission_required
     @rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_TEST_AND_RUN)
-    @with_session
-    def post(self, session: Session, current_user: Account, app_model: App):
+    def post(self, current_user: Account, app_model: App):
         """
         Full workflow debug when the start node is a trigger
         """
@@ -1790,7 +1673,7 @@ class DraftWorkflowTriggerRunAllApi(Resource):
         args = DraftWorkflowTriggerRunAllPayload.model_validate(console_ns.payload or {})
         node_ids = args.node_ids
         workflow_service = WorkflowService()
-        draft_workflow = workflow_service.get_draft_workflow(app_model, session=db.session())
+        draft_workflow = workflow_service.get_draft_workflow(app_model)
         if not draft_workflow:
             raise ValueError("Workflow not found")
 
@@ -1814,7 +1697,6 @@ class DraftWorkflowTriggerRunAllApi(Resource):
 
             workflow_args[SKIP_PREPARE_USER_INPUTS_KEY] = True
             response = AppGenerateService.generate(
-                session=session,
                 app_model=app_model,
                 user=current_user,
                 args=workflow_args,
@@ -1859,7 +1741,7 @@ class WorkflowOnlineUsersApi(Resource):
             return {"data": []}
 
         workflow_service = WorkflowService()
-        accessible_app_ids = workflow_service.get_accessible_app_ids(app_ids, current_tenant_id, session=db.session())
+        accessible_app_ids = workflow_service.get_accessible_app_ids(app_ids, current_tenant_id)
         ordered_accessible_app_ids = [app_id for app_id in app_ids if app_id in accessible_app_ids]
 
         users_json_by_app_id: dict[str, Any] = {}

api/controllers/console/app/workflow_comment.py

@@ -189,14 +189,14 @@ class WorkflowCommentReplyUpdate(ResponseModel):
 
 register_schema_models(
     console_ns,
+    AccountWithRole,
+    WorkflowCommentMentionUsersPayload,
     WorkflowCommentCreatePayload,
     WorkflowCommentUpdatePayload,
     WorkflowCommentReplyPayload,
 )
 register_response_schema_models(
     console_ns,
-    AccountWithRole,
-    WorkflowCommentMentionUsersPayload,
     WorkflowCommentAccount,
     WorkflowCommentReply,
     WorkflowCommentMention,
@@ -490,7 +490,7 @@ class WorkflowCommentMentionUsersApi(Resource):
         current_tenant = current_user.current_tenant  # need the tenant object here
         if current_tenant is None:
             raise ValueError("current tenant is required")
-        members = TenantService.get_tenant_members(current_tenant, session=db.session())
+        members = TenantService.get_tenant_members(current_tenant, session=db.session)
         users = TypeAdapter(list[AccountWithRole]).validate_python(members, from_attributes=True)
         response = WorkflowCommentMentionUsersPayload(users=users)
         return response.model_dump(mode="json"), 200

api/controllers/console/app/workflow_draft_variable.py

@@ -6,7 +6,7 @@ from uuid import UUID
 
 from flask import Response, request
 from flask_restx import Resource, fields, marshal, marshal_with
-from pydantic import BaseModel, ConfigDict, Field
+from pydantic import BaseModel, Field
 from sqlalchemy.orm import sessionmaker
 
 from controllers.common.errors import InvalidArgumentError, NotFoundError
@@ -79,33 +79,15 @@ class WorkflowDraftVariableUpdatePayload(BaseModel):
     value: Any | None = Field(default=None, description="Variable value")
 
 
-class WorkflowVariableItemPayload(BaseModel):
-    model_config = ConfigDict(extra="allow")
-
-    id: str | None = None
-    name: str | None = None
-    value_type: str | None = None
-    value: Any | None = None
-    description: str | None = None
-
-
-class ConversationVariableItemPayload(WorkflowVariableItemPayload):
-    pass
-
-
-class EnvironmentVariableItemPayload(WorkflowVariableItemPayload):
-    pass
-
-
 class ConversationVariableUpdatePayload(BaseModel):
-    conversation_variables: list[ConversationVariableItemPayload] = Field(
+    conversation_variables: list[dict[str, Any]] = Field(
         ...,
         description="Conversation variables for the draft workflow",
     )
 
 
 class EnvironmentVariableUpdatePayload(BaseModel):
-    environment_variables: list[EnvironmentVariableItemPayload] = Field(
+    environment_variables: list[dict[str, Any]] = Field(
         ...,
         description="Environment variables for the draft workflow",
     )
@@ -132,9 +114,7 @@ register_schema_models(
     console_ns,
     WorkflowDraftVariableListQuery,
     WorkflowDraftVariableUpdatePayload,
-    ConversationVariableItemPayload,
     ConversationVariableUpdatePayload,
-    EnvironmentVariableItemPayload,
     EnvironmentVariableUpdatePayload,
 )
 register_response_schema_models(console_ns, SimpleResultResponse, EnvironmentVariableListResponse)
@@ -337,7 +317,7 @@ class WorkflowVariableCollectionApi(Resource):
 
         # fetch draft workflow by app_model
         workflow_service = WorkflowService()
-        workflow_exist = workflow_service.is_workflow_exist(app_model=app_model, session=db.session())
+        workflow_exist = workflow_service.is_workflow_exist(app_model=app_model)
         if not workflow_exist:
             raise DraftWorkflowNotExist()
 
@@ -553,7 +533,7 @@ class VariableResetApi(Resource):
         )
 
         workflow_srv = WorkflowService()
-        draft_workflow = workflow_srv.get_draft_workflow(app_model, session=db.session())
+        draft_workflow = workflow_srv.get_draft_workflow(app_model)
         if draft_workflow is None:
             raise NotFoundError(
                 f"Draft workflow not found, app_id={app_model.id}",
@@ -606,7 +586,7 @@ class ConversationVariableCollectionApi(Resource):
         # NOTE(QuantumGhost): Prefill conversation variables into the draft variables table
         # so their IDs can be returned to the caller.
         workflow_srv = WorkflowService()
-        draft_workflow = workflow_srv.get_draft_workflow(app_model, session=db.session())
+        draft_workflow = workflow_srv.get_draft_workflow(app_model)
         if draft_workflow is None:
             raise NotFoundError(description=f"draft workflow not found, id={app_model.id}")
         draft_var_srv = WorkflowDraftVariableService(db.session())
@@ -635,9 +615,7 @@ class ConversationVariableCollectionApi(Resource):
 
         workflow_service = WorkflowService()
 
-        conversation_variables_list = [
-            variable.model_dump(mode="json", exclude_unset=True) for variable in payload.conversation_variables
-        ]
+        conversation_variables_list = payload.conversation_variables
         conversation_variables = [
             variable_factory.build_conversation_variable_from_mapping(obj) for obj in conversation_variables_list
         ]
@@ -646,7 +624,6 @@ class ConversationVariableCollectionApi(Resource):
             app_model=app_model,
             account=current_user,
             conversation_variables=conversation_variables,
-            session=db.session(),
         )
 
         return {"result": "success"}
@@ -684,7 +661,7 @@ class EnvironmentVariableCollectionApi(Resource):
         """
         # fetch draft workflow by app_model
         workflow_service = WorkflowService()
-        workflow = workflow_service.get_draft_workflow(app_model=app_model, session=db.session())
+        workflow = workflow_service.get_draft_workflow(app_model=app_model)
         if workflow is None:
             raise DraftWorkflowNotExist()
 
@@ -730,9 +707,7 @@ class EnvironmentVariableCollectionApi(Resource):
 
         workflow_service = WorkflowService()
 
-        environment_variables_list = [
-            variable.model_dump(mode="json", exclude_unset=True) for variable in payload.environment_variables
-        ]
+        environment_variables_list = payload.environment_variables
         environment_variables = [
             variable_factory.build_environment_variable_from_mapping(obj) for obj in environment_variables_list
         ]
@@ -741,7 +716,6 @@ class EnvironmentVariableCollectionApi(Resource):
             app_model=app_model,
             account=current_user,
             environment_variables=environment_variables,
-            session=db.session(),
         )
 
         return {"result": "success"}

api/controllers/console/app/workflow_node_output_inspector.py

@@ -41,7 +41,6 @@ from controllers.console.wraps import (
     rbac_permission_required,
     setup_required,
 )
-from extensions.ext_database import db
 from libs.exception import BaseHTTPException
 from libs.login import login_required
 from models import App, AppMode
@@ -93,9 +92,7 @@ def _serve_snapshot(app_model: App, run_id: UUID) -> dict:
     Flask request context.
     """
     try:
-        snapshot = _service().snapshot_workflow_run(
-            app_model=app_model, workflow_run_id=str(run_id), session=db.session()
-        )
+        snapshot = _service().snapshot_workflow_run(app_model=app_model, workflow_run_id=str(run_id))
     except NodeOutputInspectorError as error:
         raise _InspectorNotFound(error) from error
     return snapshot.model_dump(mode="json")
@@ -108,7 +105,6 @@ def _serve_node_detail(app_model: App, run_id: UUID, node_id: str) -> dict:
             app_model=app_model,
             workflow_run_id=str(run_id),
             node_id=node_id,
-            session=db.session(),
         )
     except NodeOutputInspectorError as error:
         raise _InspectorNotFound(error) from error
@@ -123,7 +119,6 @@ def _serve_output_preview(app_model: App, run_id: UUID, node_id: str, output_nam
             workflow_run_id=str(run_id),
             node_id=node_id,
             output_name=output_name,
-            session=db.session(),
         )
     except NodeOutputInspectorError as error:
         raise _InspectorNotFound(error) from error
@@ -250,7 +245,7 @@ def _stream_inspector_events(app_model: App, run_id: UUID) -> Iterator[str]:
     # if the run is gone (raised before yielding any bytes, so Flask turns it
     # into the normal HTTP 404 path).
     try:
-        snapshot = service.snapshot_workflow_run(app_model=app_model, workflow_run_id=run_id_str, session=db.session())
+        snapshot = service.snapshot_workflow_run(app_model=app_model, workflow_run_id=run_id_str)
     except NodeOutputInspectorError as error:
         raise _InspectorNotFound(error) from error
 
@@ -313,7 +308,6 @@ def _stream_inspector_events(app_model: App, run_id: UUID) -> Iterator[str]:
                 app_model=app_model,
                 workflow_run_id=run_id_str,
                 node_id=message.node_id,
-                session=db.session(),
             )
         except NodeOutputInspectorError:
             # Node may not appear in the graph yet (race with persistence); skip.

api/controllers/console/app/workflow_run.py

@@ -23,7 +23,6 @@ from controllers.console.wraps import (
     with_current_user,
 )
 from core.workflow.human_input_forms import load_form_tokens_by_form_id as _load_form_tokens_by_form_id
-from core.workflow.nodes.human_input.pause_reason import HumanInputRequired
 from extensions.ext_database import db
 from fields.base import ResponseModel
 from fields.workflow_run_fields import (
@@ -34,6 +33,7 @@ from fields.workflow_run_fields import (
     WorkflowRunNodeExecutionResponse,
     WorkflowRunPaginationResponse,
 )
+from graphon.entities.pause_reason import HumanInputRequired
 from graphon.enums import WorkflowExecutionStatus
 from libs.archive_storage import ArchiveStorageNotConfiguredError, get_archive_storage
 from libs.custom_inputs import time_duration

api/controllers/console/app/workflow_trigger.py

@@ -12,7 +12,6 @@ from configs import dify_config
 from controllers.common.schema import query_params_from_model, register_schema_models
 from extensions.ext_database import db
 from fields.base import ResponseModel
-from libs.helper import dump_response
 from libs.login import login_required
 from models.enums import AppTriggerStatus
 from models.model import App, AppMode
@@ -122,7 +121,7 @@ class WebhookTriggerApi(Resource):
             if not webhook_trigger:
                 raise NotFound("Webhook trigger not found for this node")
 
-            return dump_response(WebhookTriggerResponse, webhook_trigger)
+            return WebhookTriggerResponse.model_validate(webhook_trigger, from_attributes=True).model_dump(mode="json")
 
 
 @console_ns.route("/apps/<uuid:app_id>/triggers")
@@ -161,7 +160,9 @@ class AppTriggersApi(Resource):
             else:
                 trigger.icon = ""  # type: ignore
 
-        return dump_response(WorkflowTriggerListResponse, {"data": triggers})
+        return WorkflowTriggerListResponse.model_validate({"data": triggers}, from_attributes=True).model_dump(
+            mode="json"
+        )
 
 
 @console_ns.route("/apps/<uuid:app_id>/trigger-enable")
@@ -203,4 +204,4 @@ class AppTriggerEnableApi(Resource):
         else:
             trigger.icon = ""  # type: ignore
 
-        return dump_response(WorkflowTriggerResponse, trigger)
+        return WorkflowTriggerResponse.model_validate(trigger, from_attributes=True).model_dump(mode="json")

api/controllers/console/app/wraps.py

@@ -1,26 +1,26 @@
 """Controller decorators for console app resources.
 
-App-loading decorators prefer a session injected by
-`controllers.common.session.with_session` when present, while still supporting
-existing handlers that have not been migrated yet and still rely on
-Flask-SQLAlchemy's scoped `db.session`.
+`with_session` opens one SQLAlchemy session for a request handler and injects it
+as the first argument after `self`. Handlers use a transaction by default so
+migrated write paths keep commit/rollback handling; pure read handlers may opt
+out with `write=False`. App-loading decorators prefer that injected session when
+present, while still supporting existing handlers that have not been migrated
+yet and still rely on Flask-SQLAlchemy's scoped `db.session`.
 """
 
 from collections.abc import Callable
 from functools import wraps
-from typing import cast, overload
+from typing import Concatenate, cast, overload
 
 from sqlalchemy import select
 from sqlalchemy.orm import Session
 
-from controllers.common.session import with_session
 from controllers.console.app.error import AppNotFoundError
+from core.db.session_factory import session_factory
 from extensions.ext_database import db
 from libs.login import current_account_with_tenant
 from models import App, AppMode
 
-__all__ = ["get_app_model", "get_app_model_with_trial", "with_session"]
-
 
 def _load_app_model(session: Session, app_id: str) -> App | None:
     """Load the tenant-scoped app row with the request session owned by `with_session`."""
@@ -45,6 +45,48 @@ def _load_app_model_with_trial(app_id: str) -> App | None:
     return app_model
 
 
+@overload
+def with_session[T, **P, R](
+    view: Callable[Concatenate[T, Session, P], R],
+    *,
+    write: bool = True,
+) -> Callable[Concatenate[T, P], R]: ...
+
+
+@overload
+def with_session[T, **P, R](
+    view: None = None,
+    *,
+    write: bool = True,
+) -> Callable[[Callable[Concatenate[T, Session, P], R]], Callable[Concatenate[T, P], R]]: ...
+
+
+def with_session[T, **P, R](
+    view: Callable[Concatenate[T, Session, P], R] | None = None,
+    *,
+    write: bool = True,
+) -> (
+    Callable[Concatenate[T, P], R] | Callable[[Callable[Concatenate[T, Session, P], R]], Callable[Concatenate[T, P], R]]
+):
+    """Inject a request-scoped session, using a transaction only for write handlers."""
+
+    def decorator(view: Callable[Concatenate[T, Session, P], R]) -> Callable[Concatenate[T, P], R]:
+        @wraps(view)
+        def wrapper(self: T, *args: P.args, **kwargs: P.kwargs) -> R:
+            if write:
+                with session_factory.get_session_maker().begin() as session:
+                    return view(self, session, *args, **kwargs)
+
+            with session_factory.create_session() as session:
+                return view(self, session, *args, **kwargs)
+
+        return wrapper
+
+    if view is None:
+        return decorator
+    return decorator(view)
+
+
 def _get_injected_session(args: tuple[object, ...]) -> Session | None:
     """Return the request session inserted by `with_session`, if this handler has been migrated."""
     if len(args) < 2:

api/controllers/console/auth/activate.py

@@ -90,7 +90,7 @@ class ActivateCheckApi(Resource):
         token = args.token
 
         invitation = RegisterService.get_invitation_with_case_fallback(
-            workspaceId, args.email, token, session=db.session()
+            workspaceId, args.email, token, session=db.session
         )
         if invitation:
             data = invitation.get("data", {})
@@ -140,7 +140,7 @@ class ActivateApi(Resource):
 
         normalized_request_email = args.email.lower() if args.email else None
         invitation = RegisterService.get_invitation_with_case_fallback(
-            args.workspace_id, args.email, args.token, session=db.session()
+            args.workspace_id, args.email, args.token, session=db.session
         )
         if invitation is None:
             raise AlreadyActivateError()
@@ -178,7 +178,7 @@ class ActivateApi(Resource):
         RegisterService.revoke_token(args.workspace_id, normalized_request_email, args.token)
 
         if membership_id is None:
-            TenantService.create_tenant_member(tenant, account, db.session(), role=role)
+            TenantService.create_tenant_member(tenant, account, db.session, role=role)
 
         if setup_fields:
             account.name = setup_fields[0]
@@ -188,6 +188,6 @@ class ActivateApi(Resource):
             account.status = AccountStatus.ACTIVE
             account.initialized_at = naive_utc_now()
 
-        TenantService.switch_tenant(account, tenant.id, session=db.session())
+        TenantService.switch_tenant(account, tenant.id, session=db.session)
 
         return {"result": "success"}

api/controllers/console/auth/email_register.py

@@ -101,7 +101,7 @@ class EmailRegisterSendEmailApi(Resource):
         if dify_config.BILLING_ENABLED and BillingService.is_email_in_freeze(normalized_email):
             raise AccountInFreezeError()
 
-        account = AccountService.get_account_by_email_with_case_fallback(db.session(), args.email)
+        account = AccountService.get_account_by_email_with_case_fallback(db.session, args.email)
         token = AccountService.send_email_register_email(email=normalized_email, account=account, language=language)
         return {"result": "success", "data": token}
 
@@ -176,7 +176,7 @@ class EmailRegisterResetApi(Resource):
         email = register_data.get("email", "")
         normalized_email = email.lower()
 
-        account = AccountService.get_account_by_email_with_case_fallback(db.session(), email)
+        account = AccountService.get_account_by_email_with_case_fallback(db.session, email)
 
         if account:
             raise EmailAlreadyInUseError()
@@ -187,7 +187,7 @@ class EmailRegisterResetApi(Resource):
             timezone=args.timezone,
             language=args.language,
         )
-        token_pair = AccountService.login(account=account, session=db.session(), ip_address=extract_remote_ip(request))
+        token_pair = AccountService.login(account=account, session=db.session, ip_address=extract_remote_ip(request))
         AccountService.reset_login_error_rate_limit(normalized_email)
 
         return {"result": "success", "data": token_pair.model_dump()}
@@ -206,7 +206,7 @@ class EmailRegisterResetApi(Resource):
                 password=password,
                 interface_language=get_valid_language(language),
                 timezone=timezone,
-                session=db.session(),
+                session=db.session,
             )
         except AccountRegisterError:
             raise AccountInFreezeError()

api/controllers/console/auth/forgot_password.py

@@ -82,7 +82,7 @@ class ForgotPasswordSendEmailApi(Resource):
         else:
             language = "en-US"
 
-        account = AccountService.get_account_by_email_with_case_fallback(db.session(), args.email)
+        account = AccountService.get_account_by_email_with_case_fallback(db.session, args.email)
 
         token = AccountService.send_reset_password_email(
             account=account,
@@ -180,7 +180,7 @@ class ForgotPasswordResetApi(Resource):
         password_hashed = hash_password(args.new_password, salt)
 
         email = reset_data.get("email", "")
-        account = AccountService.get_account_by_email_with_case_fallback(db.session(), email)
+        account = AccountService.get_account_by_email_with_case_fallback(db.session, email)
 
         if account:
             account = db.session.merge(account)
@@ -198,10 +198,10 @@ class ForgotPasswordResetApi(Resource):
 
         # Create workspace if needed
         if (
-            not TenantService.get_join_tenants(account, session=db.session())
+            not TenantService.get_join_tenants(account, session=db.session)
             and FeatureService.get_system_features().is_allow_create_workspace
         ):
-            tenant = TenantService.create_tenant(f"{account.name}'s Workspace", session=db.session())
-            TenantService.create_tenant_member(tenant, account, db.session(), role="owner")
+            tenant = TenantService.create_tenant(f"{account.name}'s Workspace", session=db.session)
+            TenantService.create_tenant_member(tenant, account, db.session, role="owner")
             account.current_tenant = tenant
             tenant_was_created.send(tenant)

api/controllers/console/auth/login.py

@@ -9,12 +9,7 @@ from werkzeug.exceptions import Unauthorized
 import services
 from configs import dify_config
 from constants.languages import get_valid_language
-from controllers.common.fields import (
-    SimpleResultDataResponse,
-    SimpleResultMessageResponse,
-    SimpleResultOptionalDataResponse,
-    SimpleResultResponse,
-)
+from controllers.common.fields import SimpleResultDataResponse, SimpleResultOptionalDataResponse, SimpleResultResponse
 from controllers.common.schema import register_response_schema_models, register_schema_models
 from controllers.console import console_ns
 from controllers.console.auth.error import (
@@ -56,7 +51,7 @@ from models.account import Account
 from services.account_service import AccountService, InvitationDetailDict, RegisterService, TenantService
 from services.billing_service import BillingService
 from services.entities.auth_entities import LoginFailureReason, LoginPayloadBase
-from services.errors.account import AccountRegisterError, RefreshTokenAccountNotFoundError, RefreshTokenNotFoundError
+from services.errors.account import AccountRegisterError
 from services.errors.workspace import WorkSpaceNotAllowedCreateError, WorkspacesLimitExceededError
 from services.feature_service import FeatureService
 
@@ -92,7 +87,6 @@ register_schema_models(console_ns, LoginPayload, EmailPayload, EmailCodeLoginPay
 register_response_schema_models(
     console_ns,
     SimpleResultDataResponse,
-    SimpleResultMessageResponse,
     SimpleResultOptionalDataResponse,
     SimpleResultResponse,
 )
@@ -126,7 +120,7 @@ class LoginApi(Resource):
         invitation_data: InvitationDetailDict | None = None
         if invite_token:
             invitation_data = RegisterService.get_invitation_with_case_fallback(
-                None, request_email, invite_token, session=db.session()
+                None, request_email, invite_token, session=db.session
             )
             if invitation_data is None:
                 invite_token = None
@@ -153,26 +147,23 @@ class LoginApi(Resource):
             _log_console_login_failure(email=normalized_email, reason=LoginFailureReason.INVALID_CREDENTIALS)
             raise AuthenticationFailedError() from exc
         # SELF_HOSTED only have one workspace
-        tenants = TenantService.get_join_tenants(account, session=db.session())
+        tenants = TenantService.get_join_tenants(account, session=db.session)
         if len(tenants) == 0:
             system_features = FeatureService.get_system_features()
 
             if system_features.is_allow_create_workspace and not system_features.license.workspaces.is_available():
                 raise WorkspacesLimitExceeded()
             else:
-                return SimpleResultOptionalDataResponse(
-                    result="fail",
-                    data="workspace not found, please contact system admin to invite you to join in a workspace",
-                ).model_dump(mode="json")
+                return {
+                    "result": "fail",
+                    "data": "workspace not found, please contact system admin to invite you to join in a workspace",
+                }
 
-        token_pair = AccountService.login(account=account, session=db.session(), ip_address=extract_remote_ip(request))
+        token_pair = AccountService.login(account=account, session=db.session, ip_address=extract_remote_ip(request))
         AccountService.reset_login_error_rate_limit(normalized_email)
 
         # Create response with cookies instead of returning tokens in body
-        # response-contract:ignore cookie-bearing Flask response
-        response = make_response(
-            SimpleResultOptionalDataResponse(result="success").model_dump(mode="json", exclude_none=True)
-        )
+        response = make_response({"result": "success"})
 
         set_access_token_to_cookie(request, response, token_pair.access_token)
         set_refresh_token_to_cookie(request, response, token_pair.refresh_token)
@@ -187,11 +178,12 @@ class LogoutApi(Resource):
     @console_ns.response(200, "Success", console_ns.models[SimpleResultResponse.__name__])
     @with_current_user
     def post(self, account: Account):
-        # response-contract:ignore cookie-bearing Flask response
-        response = make_response(SimpleResultResponse(result="success").model_dump(mode="json"))
-        if not isinstance(account, flask_login.AnonymousUserMixin):
+        if isinstance(account, flask_login.AnonymousUserMixin):
+            response = make_response({"result": "success"})
+        else:
             AccountService.logout(account=account)
             flask_login.logout_user()
+            response = make_response({"result": "success"})
 
         # Clear cookies on logout
         clear_access_token_from_cookie(response)
@@ -227,7 +219,7 @@ class ResetPasswordSendEmailApi(Resource):
             is_allow_register=FeatureService.get_system_features().is_allow_register,
         )
 
-        return SimpleResultDataResponse(result="success", data=token).model_dump(mode="json")
+        return {"result": "success", "data": token}
 
 
 @console_ns.route("/email-code-login")
@@ -260,7 +252,7 @@ class EmailCodeLoginSendEmailApi(Resource):
         else:
             token = AccountService.send_email_code_login_email(account=account, language=language)
 
-        return SimpleResultDataResponse(result="success", data=token).model_dump(mode="json")
+        return {"result": "success", "data": token}
 
 
 @console_ns.route("/email-code-login/validity")
@@ -301,7 +293,7 @@ class EmailCodeLoginApi(Resource):
             _log_console_login_failure(email=user_email, reason=LoginFailureReason.ACCOUNT_IN_FREEZE)
             raise AccountInFreezeError()
         if account:
-            tenants = TenantService.get_join_tenants(account, session=db.session())
+            tenants = TenantService.get_join_tenants(account, session=db.session)
             if not tenants:
                 workspaces = FeatureService.get_system_features().license.workspaces
                 if not workspaces.is_available():
@@ -309,8 +301,8 @@ class EmailCodeLoginApi(Resource):
                 if not FeatureService.get_system_features().is_allow_create_workspace:
                     raise NotAllowedCreateWorkspace()
                 else:
-                    new_tenant = TenantService.create_tenant(f"{account.name}'s Workspace", session=db.session())
-                    TenantService.create_tenant_member(new_tenant, account, db.session(), role="owner")
+                    new_tenant = TenantService.create_tenant(f"{account.name}'s Workspace", session=db.session)
+                    TenantService.create_tenant_member(new_tenant, account, db.session, role="owner")
                     account.current_tenant = new_tenant
                     tenant_was_created.send(new_tenant)
 
@@ -321,7 +313,7 @@ class EmailCodeLoginApi(Resource):
                     name=user_email,
                     interface_language=get_valid_language(language),
                     timezone=args.timezone,
-                    session=db.session(),
+                    session=db.session,
                 )
             except WorkSpaceNotAllowedCreateError:
                 raise NotAllowedCreateWorkspace()
@@ -330,12 +322,11 @@ class EmailCodeLoginApi(Resource):
                 raise AccountInFreezeError()
             except WorkspacesLimitExceededError:
                 raise WorkspacesLimitExceeded()
-        token_pair = AccountService.login(account, session=db.session(), ip_address=extract_remote_ip(request))
+        token_pair = AccountService.login(account, session=db.session, ip_address=extract_remote_ip(request))
         AccountService.reset_login_error_rate_limit(user_email)
 
         # Create response with cookies instead of returning tokens in body
-        # response-contract:ignore cookie-bearing Flask response
-        response = make_response(SimpleResultResponse(result="success").model_dump(mode="json"))
+        response = make_response({"result": "success"})
 
         set_csrf_token_to_cookie(request, response, token_pair.csrf_token)
         # Set HTTP-only secure cookies for tokens
@@ -347,53 +338,45 @@ class EmailCodeLoginApi(Resource):
 @console_ns.route("/refresh-token")
 class RefreshTokenApi(Resource):
     @console_ns.response(200, "Success", console_ns.models[SimpleResultResponse.__name__])
-    @console_ns.response(401, "Unauthorized", console_ns.models[SimpleResultMessageResponse.__name__])
     def post(self):
         # Get refresh token from cookie instead of request body
         refresh_token = extract_refresh_token(request)
 
         if not refresh_token:
-            return SimpleResultMessageResponse(result="fail", message="No refresh token provided").model_dump(
-                mode="json"
-            ), 401
+            return {"result": "fail", "message": "No refresh token provided"}, 401
 
         try:
-            new_token_pair = AccountService.refresh_token(refresh_token, session=db.session())
-        except Unauthorized as exc:
-            return SimpleResultMessageResponse(result="fail", message=exc.description or "Unauthorized.").model_dump(
-                mode="json"
-            ), 401
-        except (RefreshTokenNotFoundError, RefreshTokenAccountNotFoundError) as exc:
-            return SimpleResultMessageResponse(result="fail", message=str(exc)).model_dump(mode="json"), 401
+            new_token_pair = AccountService.refresh_token(refresh_token, session=db.session)
 
-        # Create response with new cookies
-        # response-contract:ignore cookie-bearing Flask response
-        response = make_response(SimpleResultResponse(result="success").model_dump(mode="json"))
+            # Create response with new cookies
+            response = make_response({"result": "success"})
 
-        # Update cookies with new tokens
-        set_csrf_token_to_cookie(request, response, new_token_pair.csrf_token)
-        set_access_token_to_cookie(request, response, new_token_pair.access_token)
-        set_refresh_token_to_cookie(request, response, new_token_pair.refresh_token)
-        return response
+            # Update cookies with new tokens
+            set_csrf_token_to_cookie(request, response, new_token_pair.csrf_token)
+            set_access_token_to_cookie(request, response, new_token_pair.access_token)
+            set_refresh_token_to_cookie(request, response, new_token_pair.refresh_token)
+            return response
+        except Exception as e:
+            return {"result": "fail", "message": str(e)}, 401
 
 
 def _get_account_with_case_fallback(email: str):
-    account = AccountService.get_user_through_email(email, session=db.session())
+    account = AccountService.get_user_through_email(email, session=db.session)
     if account or email == email.lower():
         return account
 
-    return AccountService.get_user_through_email(email.lower(), session=db.session())
+    return AccountService.get_user_through_email(email.lower(), session=db.session)
 
 
 def _authenticate_account_with_case_fallback(
     original_email: str, normalized_email: str, password: str, invite_token: str | None
 ):
     try:
-        return AccountService.authenticate(original_email, password, invite_token, session=db.session())
+        return AccountService.authenticate(original_email, password, invite_token, session=db.session)
     except services.errors.account.AccountPasswordError:
         if original_email == normalized_email:
             raise
-        return AccountService.authenticate(normalized_email, password, invite_token, session=db.session())
+        return AccountService.authenticate(normalized_email, password, invite_token, session=db.session)
 
 
 def _log_console_login_failure(*, email: str, reason: LoginFailureReason) -> None:

api/controllers/console/auth/oauth.py

@@ -195,7 +195,7 @@ class OAuthCallback(Resource):
             db.session.commit()
 
         try:
-            TenantService.create_owner_tenant_if_not_exist(account, session=db.session())
+            TenantService.create_owner_tenant_if_not_exist(account, session=db.session)
         except Unauthorized:
             return redirect(f"{dify_config.CONSOLE_WEB_URL}/signin?message=Workspace not found.")
         except WorkSpaceNotAllowedCreateError:
@@ -206,7 +206,7 @@ class OAuthCallback(Resource):
 
         token_pair = AccountService.login(
             account=account,
-            session=db.session(),
+            session=db.session,
             ip_address=extract_remote_ip(request),
         )
 
@@ -225,7 +225,7 @@ def _get_account_by_openid_or_email(provider: str, user_info: OAuthUserInfo) ->
     account: Account | None = Account.get_by_openid(provider, user_info.id)
 
     if not account:
-        account = AccountService.get_account_by_email_with_case_fallback(db.session(), user_info.email)
+        account = AccountService.get_account_by_email_with_case_fallback(db.session, user_info.email)
 
     return account
 
@@ -241,13 +241,13 @@ def _generate_account(
     oauth_new_user = False
 
     if account:
-        tenants = TenantService.get_join_tenants(account, session=db.session())
+        tenants = TenantService.get_join_tenants(account, session=db.session)
         if not tenants:
             if not FeatureService.get_system_features().is_allow_create_workspace:
                 raise WorkSpaceNotAllowedCreateError()
             else:
-                new_tenant = TenantService.create_tenant(f"{account.name}'s Workspace", session=db.session())
-                TenantService.create_tenant_member(new_tenant, account, db.session(), role="owner")
+                new_tenant = TenantService.create_tenant(f"{account.name}'s Workspace", session=db.session)
+                TenantService.create_tenant_member(new_tenant, account, db.session, role="owner")
                 account.current_tenant = new_tenant
                 tenant_was_created.send(new_tenant)
 
@@ -273,10 +273,10 @@ def _generate_account(
             provider=provider,
             language=interface_language,
             timezone=timezone,
-            session=db.session(),
+            session=db.session,
         )
 
     # Link account
-    AccountService.link_account_integrate(provider, user_info.id, account, session=db.session())
+    AccountService.link_account_integrate(provider, user_info.id, account, session=db.session)
 
     return account, oauth_new_user

api/controllers/console/auth/oauth_server.py

@@ -10,7 +10,6 @@ from werkzeug.exceptions import BadRequest, NotFound
 
 from controllers.common.schema import register_response_schema_models, register_schema_models
 from controllers.console.wraps import account_initialization_required, setup_required, with_current_user
-from extensions.ext_database import db
 from graphon.model_runtime.utils.encoders import jsonable_encoder
 from libs.login import login_required
 from models import Account
@@ -132,9 +131,7 @@ def oauth_server_access_token_required[T, **P, R](
             response.headers["WWW-Authenticate"] = "Bearer"
             return response
 
-        account = OAuthServerService.validate_oauth_access_token(
-            oauth_provider_app.client_id, access_token, db.session()
-        )
+        account = OAuthServerService.validate_oauth_access_token(oauth_provider_app.client_id, access_token)
         if not account:
             response = jsonify({"error": "access_token or client_id is invalid"})
             response.status_code = 401

api/controllers/console/billing/billing.py

@@ -16,8 +16,6 @@ from controllers.console.wraps import (
     with_current_user,
 )
 from enums.cloud_plan import CloudPlan
-from extensions.ext_database import db
-from fields.base import ResponseModel
 from libs.login import login_required
 from models import Account
 from services.billing_service import BillingService
@@ -36,12 +34,8 @@ class BillingResponse(RootModel[dict[str, Any]]):
     root: dict[str, Any]
 
 
-class BillingInvoiceResponse(ResponseModel):
-    url: str
-
-
 register_schema_models(console_ns, SubscriptionQuery, PartnerTenantsPayload)
-register_response_schema_models(console_ns, BillingResponse, BillingInvoiceResponse)
+register_response_schema_models(console_ns, BillingResponse)
 
 
 @console_ns.route("/billing/subscription")
@@ -56,13 +50,13 @@ class Subscription(Resource):
     @with_current_tenant_id
     def get(self, current_tenant_id: str, current_user: Account):
         args = SubscriptionQuery.model_validate(request.args.to_dict(flat=True))
-        BillingService.is_tenant_owner_or_admin(db.session(), current_user)
+        BillingService.is_tenant_owner_or_admin(current_user)
         return BillingService.get_subscription(args.plan, args.interval, current_user.email, current_tenant_id)
 
 
 @console_ns.route("/billing/invoices")
 class Invoices(Resource):
-    @console_ns.response(200, "Success", console_ns.models[BillingInvoiceResponse.__name__])
+    @console_ns.response(200, "Success", console_ns.models[BillingResponse.__name__])
     @setup_required
     @login_required
     @account_initialization_required
@@ -70,7 +64,7 @@ class Invoices(Resource):
     @with_current_user
     @with_current_tenant_id
     def get(self, current_tenant_id: str, current_user: Account):
-        BillingService.is_tenant_owner_or_admin(db.session(), current_user)
+        BillingService.is_tenant_owner_or_admin(current_user)
         return BillingService.get_invoices(current_user.email, current_tenant_id)
 
 

api/controllers/console/datasets/data_source.py

@@ -243,71 +243,72 @@ class DataSourceNotionListApi(Resource):
         if not credential:
             raise NotFound("Credential not found.")
         exist_page_ids = []
-        # import notion in the exist dataset
-        if query.dataset_id:
-            dataset = DatasetService.get_dataset(query.dataset_id, db.session())
-            if not dataset:
-                raise NotFound("Dataset not found.")
-            if dataset.data_source_type != "notion_import":
-                raise ValueError("Dataset is not notion type.")
+        with sessionmaker(db.engine).begin() as session:
+            # import notion in the exist dataset
+            if query.dataset_id:
+                dataset = DatasetService.get_dataset(query.dataset_id)
+                if not dataset:
+                    raise NotFound("Dataset not found.")
+                if dataset.data_source_type != "notion_import":
+                    raise ValueError("Dataset is not notion type.")
 
-            documents = db.session.scalars(
-                select(Document).where(
-                    Document.dataset_id == query.dataset_id,
-                    Document.tenant_id == current_tenant_id,
-                    Document.data_source_type == "notion_import",
-                    Document.enabled.is_(True),
-                )
-            ).all()
-            if documents:
-                for document in documents:
-                    data_source_info = json.loads(document.data_source_info)
-                    exist_page_ids.append(data_source_info["notion_page_id"])
-        # get all authorized pages
-        from core.datasource.datasource_manager import DatasourceManager
+                documents = session.scalars(
+                    select(Document).where(
+                        Document.dataset_id == query.dataset_id,
+                        Document.tenant_id == current_tenant_id,
+                        Document.data_source_type == "notion_import",
+                        Document.enabled.is_(True),
+                    )
+                ).all()
+                if documents:
+                    for document in documents:
+                        data_source_info = json.loads(document.data_source_info)
+                        exist_page_ids.append(data_source_info["notion_page_id"])
+            # get all authorized pages
+            from core.datasource.datasource_manager import DatasourceManager
 
-        datasource_runtime = DatasourceManager.get_datasource_runtime(
-            provider_id="langgenius/notion_datasource/notion_datasource",
-            datasource_name="notion_datasource",
-            tenant_id=current_tenant_id,
-            datasource_type=DatasourceProviderType.ONLINE_DOCUMENT,
-        )
-        datasource_provider_service = DatasourceProviderService()
-        if credential:
-            datasource_runtime.runtime.credentials = credential
-        datasource_runtime = cast(OnlineDocumentDatasourcePlugin, datasource_runtime)
-        online_document_result: Generator[OnlineDocumentPagesMessage, None, None] = (
-            datasource_runtime.get_online_document_pages(
-                user_id=current_user.id,
-                datasource_parameters={},
-                provider_type=datasource_runtime.datasource_provider_type(),
+            datasource_runtime = DatasourceManager.get_datasource_runtime(
+                provider_id="langgenius/notion_datasource/notion_datasource",
+                datasource_name="notion_datasource",
+                tenant_id=current_tenant_id,
+                datasource_type=DatasourceProviderType.ONLINE_DOCUMENT,
             )
-        )
-        try:
-            pages = []
-            workspace_info = {}
-            for message in online_document_result:
-                result = message.result
-                for info in result:
-                    workspace_info = {
-                        "workspace_id": info.workspace_id,
-                        "workspace_name": info.workspace_name,
-                        "workspace_icon": info.workspace_icon,
-                    }
-                    for page in info.pages:
-                        page_info = {
-                            "page_id": page.page_id,
-                            "page_name": page.page_name,
-                            "type": page.type,
-                            "parent_id": page.parent_id,
-                            "is_bound": page.page_id in exist_page_ids,
-                            "page_icon": page.page_icon,
+            datasource_provider_service = DatasourceProviderService()
+            if credential:
+                datasource_runtime.runtime.credentials = credential
+            datasource_runtime = cast(OnlineDocumentDatasourcePlugin, datasource_runtime)
+            online_document_result: Generator[OnlineDocumentPagesMessage, None, None] = (
+                datasource_runtime.get_online_document_pages(
+                    user_id=current_user.id,
+                    datasource_parameters={},
+                    provider_type=datasource_runtime.datasource_provider_type(),
+                )
+            )
+            try:
+                pages = []
+                workspace_info = {}
+                for message in online_document_result:
+                    result = message.result
+                    for info in result:
+                        workspace_info = {
+                            "workspace_id": info.workspace_id,
+                            "workspace_name": info.workspace_name,
+                            "workspace_icon": info.workspace_icon,
                         }
-                        pages.append(page_info)
-        except Exception as e:
-            raise e
-        notion_info = [{**workspace_info, "pages": pages}] if workspace_info else []
-        return dump_response(NotionIntegrateInfoListResponse, {"notion_info": notion_info}), 200
+                        for page in info.pages:
+                            page_info = {
+                                "page_id": page.page_id,
+                                "page_name": page.page_name,
+                                "type": page.type,
+                                "parent_id": page.parent_id,
+                                "is_bound": page.page_id in exist_page_ids,
+                                "page_icon": page.page_icon,
+                            }
+                            pages.append(page_info)
+            except Exception as e:
+                raise e
+            notion_info = [{**workspace_info, "pages": pages}] if workspace_info else []
+            return dump_response(NotionIntegrateInfoListResponse, {"notion_info": notion_info}), 200
 
 
 @console_ns.route("/notion/pages/<uuid:page_id>/<string:page_type>/preview")
@@ -400,11 +401,11 @@ class DataSourceNotionDatasetSyncApi(Resource):
     @rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_CREATE_AND_MANAGEMENT)
     def get(self, dataset_id: UUID) -> tuple[dict[str, str], int]:
         dataset_id_str = str(dataset_id)
-        dataset = DatasetService.get_dataset(dataset_id_str, db.session())
+        dataset = DatasetService.get_dataset(dataset_id_str)
         if dataset is None:
             raise NotFound("Dataset not found.")
 
-        documents = DocumentService.get_document_by_dataset_id(dataset_id_str, db.session())
+        documents = DocumentService.get_document_by_dataset_id(dataset_id_str)
         for document in documents:
             document_indexing_sync_task.delay(dataset_id_str, document.id)
         return {"result": "success"}, 200
@@ -420,11 +421,11 @@ class DataSourceNotionDocumentSyncApi(Resource):
     def get(self, dataset_id: UUID, document_id: UUID) -> tuple[dict[str, str], int]:
         dataset_id_str = str(dataset_id)
         document_id_str = str(document_id)
-        dataset = DatasetService.get_dataset(dataset_id_str, db.session())
+        dataset = DatasetService.get_dataset(dataset_id_str)
         if dataset is None:
             raise NotFound("Dataset not found.")
 
-        document = DocumentService.get_document(dataset_id_str, document_id_str, session=db.session())
+        document = DocumentService.get_document(dataset_id_str, document_id_str)
         if document is None:
             raise NotFound("Document not found.")
         document_indexing_sync_task.delay(dataset_id_str, document_id_str)

api/controllers/console/datasets/datasets.py

@@ -6,7 +6,6 @@ from flask import request
 from flask_restx import Resource
 from pydantic import BaseModel, Field, field_validator, model_validator
 from sqlalchemy import func, select
-from sqlalchemy.orm import Session
 from werkzeug.exceptions import Forbidden, NotFound
 
 import services
@@ -16,7 +15,6 @@ from controllers.common.schema import query_params_from_model, register_response
 from controllers.console import console_ns
 from controllers.console.apikey import ApiKeyItem, ApiKeyList
 from controllers.console.app.error import ProviderNotInitializeError
-from controllers.console.app.wraps import with_session
 from controllers.console.datasets.error import DatasetInUseError, DatasetNameDuplicateError, IndexingEstimateError
 from controllers.console.wraps import (
     RBACPermission,
@@ -418,7 +416,6 @@ class DatasetListApi(Resource):
         permissions = enterprise_rbac_service.RBACService.MyPermissions.get(
             str(current_tenant_id),
             current_user.id,
-            session=db.session(),
         )
 
         accessible_dataset_ids: list[str] | None = None
@@ -462,7 +459,7 @@ class DatasetListApi(Resource):
             datasets, total = DatasetService.get_datasets(
                 query.page,
                 query.limit,
-                db.session(),
+                db.session,
                 current_tenant_id,
                 current_user,
                 query.keyword,
@@ -541,8 +538,7 @@ class DatasetListApi(Resource):
     @cloud_edition_billing_rate_limit_check("knowledge")
     @with_current_user
     @with_current_tenant_id
-    @with_session
-    def post(self, session: Session, current_tenant_id: str, current_user: Account):
+    def post(self, current_tenant_id: str, current_user: Account):
         payload = DatasetCreatePayload.model_validate(console_ns.payload or {})
 
         # The role of the current user in the ta table must be admin, owner, or editor, or dataset_operator
@@ -556,7 +552,6 @@ class DatasetListApi(Resource):
 
         try:
             dataset = DatasetService.create_empty_dataset(
-                session=session,
                 tenant_id=current_tenant_id,
                 name=payload.name,
                 description=payload.description,
@@ -571,16 +566,15 @@ class DatasetListApi(Resource):
             raise DatasetNameDuplicateError()
 
         permission_keys_map = enterprise_rbac_service.RBACService.DatasetPermissions.batch_get(
-            current_tenant_id,
+            str(current_tenant_id),
             current_user.id,
-            [dataset.id],
-            session=session,
+            [str(dataset.id)],
         )
 
         item = DatasetDetailWithPartialMembersResponse.model_validate(dataset, from_attributes=True).model_dump(
             mode="json"
         )
-        item["permission_keys"] = permission_keys_map.get(dataset.id, [])
+        item["permission_keys"] = permission_keys_map.get(str(dataset.id), [])
         return item, 201
 
 
@@ -604,18 +598,17 @@ class DatasetApi(Resource):
     @with_current_tenant_id
     def get(self, current_tenant_id: str, current_user: Account, dataset_id: UUID):
         dataset_id_str = str(dataset_id)
-        dataset = DatasetService.get_dataset(dataset_id_str, db.session())
+        dataset = DatasetService.get_dataset(dataset_id_str)
         if dataset is None:
             raise NotFound("Dataset not found.")
         try:
-            DatasetService.check_dataset_permission(dataset, current_user, db.session())
+            DatasetService.check_dataset_permission(dataset, current_user)
         except services.errors.account.NoPermissionError as e:
             raise Forbidden(str(e))
         permissions = enterprise_rbac_service.RBACService.MyPermissions.get(
-            current_tenant_id,
+            str(current_tenant_id),
             current_user.id,
             dataset_id=dataset_id_str,
-            session=db.session(),
         )
         permission_keys_map = permissions.dataset.permission_keys_by_resource_ids([dataset_id_str])
         data = dump_response(DatasetDetailResponse, dataset)
@@ -625,7 +618,7 @@ class DatasetApi(Resource):
                 provider_id = ModelProviderID(dataset.embedding_model_provider)
                 data["embedding_model_provider"] = str(provider_id)
         if data.get("permission") == "partial_members":
-            part_users_list = DatasetPermissionService.get_dataset_partial_member_list(dataset_id_str, db.session())
+            part_users_list = DatasetPermissionService.get_dataset_partial_member_list(dataset_id_str)
             data.update({"partial_member_list": part_users_list})
 
         # check embedding setting
@@ -666,10 +659,9 @@ class DatasetApi(Resource):
     @with_current_user
     @with_current_tenant_id
     @rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_EDIT)
-    @with_session
-    def patch(self, session: Session, current_tenant_id: str, current_user: Account, dataset_id: UUID):
+    def patch(self, current_tenant_id: str, current_user: Account, dataset_id: UUID):
         dataset_id_str = str(dataset_id)
-        dataset = DatasetService.get_dataset(dataset_id_str, db.session())
+        dataset = DatasetService.get_dataset(dataset_id_str)
         if dataset is None:
             raise NotFound("Dataset not found.")
 
@@ -688,33 +680,30 @@ class DatasetApi(Resource):
         # The role of the current user in the ta table must be admin, owner, editor, or dataset_operator
         if not dify_config.RBAC_ENABLED:
             DatasetPermissionService.check_permission(
-                session, current_user, dataset, payload.permission, payload.partial_member_list
+                current_user, dataset, payload.permission, payload.partial_member_list
             )
 
-        dataset = DatasetService.update_dataset(session, dataset_id_str, payload_data, current_user)
+        dataset = DatasetService.update_dataset(dataset_id_str, payload_data, current_user)
 
         if dataset is None:
             raise NotFound("Dataset not found.")
 
         permission_keys_map = enterprise_rbac_service.RBACService.DatasetPermissions.batch_get(
-            current_tenant_id,
+            str(current_tenant_id),
             current_user.id,
             [dataset_id_str],
-            session=session,
         )
         result_data = dump_response(DatasetDetailResponse, dataset)
         result_data["permission_keys"] = permission_keys_map.get(dataset_id_str, [])
         tenant_id = current_tenant_id
 
         if payload.partial_member_list is not None and payload.permission == DatasetPermissionEnum.PARTIAL_TEAM:
-            DatasetPermissionService.update_partial_member_list(
-                tenant_id, dataset_id_str, payload.partial_member_list, db.session()
-            )
+            DatasetPermissionService.update_partial_member_list(tenant_id, dataset_id_str, payload.partial_member_list)
         # clear partial member list when permission is only_me or all_team_members
         elif payload.permission in {DatasetPermissionEnum.ONLY_ME, DatasetPermissionEnum.ALL_TEAM}:
-            DatasetPermissionService.clear_partial_member_list(dataset_id_str, db.session())
+            DatasetPermissionService.clear_partial_member_list(dataset_id_str)
 
-        partial_member_list = DatasetPermissionService.get_dataset_partial_member_list(dataset_id_str, db.session())
+        partial_member_list = DatasetPermissionService.get_dataset_partial_member_list(dataset_id_str)
         result_data.update({"partial_member_list": partial_member_list})
 
         return result_data, 200
@@ -733,8 +722,8 @@ class DatasetApi(Resource):
             raise Forbidden()
 
         try:
-            if DatasetService.delete_dataset(dataset_id_str, current_user, db.session()):
-                DatasetPermissionService.clear_partial_member_list(dataset_id_str, db.session())
+            if DatasetService.delete_dataset(dataset_id_str, current_user):
+                DatasetPermissionService.clear_partial_member_list(dataset_id_str)
                 return "", 204
             else:
                 raise NotFound("Dataset not found.")
@@ -759,7 +748,7 @@ class DatasetUseCheckApi(Resource):
     def get(self, dataset_id: UUID):
         dataset_id_str = str(dataset_id)
 
-        dataset_is_using = DatasetService.dataset_use_check(dataset_id_str, db.session())
+        dataset_is_using = DatasetService.dataset_use_check(dataset_id_str)
         return {"is_using": dataset_is_using}, 200
 
 
@@ -780,12 +769,12 @@ class DatasetQueryApi(Resource):
     @rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_READONLY)
     def get(self, current_user: Account, dataset_id: UUID):
         dataset_id_str = str(dataset_id)
-        dataset = DatasetService.get_dataset(dataset_id_str, db.session())
+        dataset = DatasetService.get_dataset(dataset_id_str)
         if dataset is None:
             raise NotFound("Dataset not found.")
 
         try:
-            DatasetService.check_dataset_permission(dataset, current_user, db.session())
+            DatasetService.check_dataset_permission(dataset, current_user)
         except services.errors.account.NoPermissionError as e:
             raise Forbidden(str(e))
 
@@ -921,16 +910,16 @@ class DatasetRelatedAppListApi(Resource):
     @rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_READONLY)
     def get(self, current_user: Account, dataset_id: UUID):
         dataset_id_str = str(dataset_id)
-        dataset = DatasetService.get_dataset(dataset_id_str, db.session())
+        dataset = DatasetService.get_dataset(dataset_id_str)
         if dataset is None:
             raise NotFound("Dataset not found.")
 
         try:
-            DatasetService.check_dataset_permission(dataset, current_user, db.session())
+            DatasetService.check_dataset_permission(dataset, current_user)
         except services.errors.account.NoPermissionError as e:
             raise Forbidden(str(e))
 
-        app_dataset_joins = DatasetService.get_related_apps(dataset.id, db.session())
+        app_dataset_joins = DatasetService.get_related_apps(dataset.id)
 
         related_apps = []
         for app_dataset_join in app_dataset_joins:
@@ -1105,7 +1094,7 @@ class DatasetEnableApiApi(Resource):
     def post(self, dataset_id: UUID, status: str):
         dataset_id_str = str(dataset_id)
 
-        DatasetService.update_dataset_api_status(dataset_id_str, status == "enable", db.session())
+        DatasetService.update_dataset_api_status(dataset_id_str, status == "enable")
 
         return {"result": "success"}, 200
 
@@ -1174,10 +1163,10 @@ class DatasetErrorDocs(Resource):
     @rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_READONLY)
     def get(self, dataset_id: UUID):
         dataset_id_str = str(dataset_id)
-        dataset = DatasetService.get_dataset(dataset_id_str, db.session())
+        dataset = DatasetService.get_dataset(dataset_id_str)
         if dataset is None:
             raise NotFound("Dataset not found.")
-        results = DocumentService.get_error_documents_by_dataset_id(dataset_id_str, db.session())
+        results = DocumentService.get_error_documents_by_dataset_id(dataset_id_str)
 
         return dump_response(ErrorDocsResponse, {"data": results, "total": len(results)}), 200
 
@@ -1201,15 +1190,15 @@ class DatasetPermissionUserListApi(Resource):
     @rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_READONLY)
     def get(self, current_user: Account, dataset_id: UUID):
         dataset_id_str = str(dataset_id)
-        dataset = DatasetService.get_dataset(dataset_id_str, db.session())
+        dataset = DatasetService.get_dataset(dataset_id_str)
         if dataset is None:
             raise NotFound("Dataset not found.")
         try:
-            DatasetService.check_dataset_permission(dataset, current_user, db.session())
+            DatasetService.check_dataset_permission(dataset, current_user)
         except services.errors.account.NoPermissionError as e:
             raise Forbidden(str(e))
 
-        partial_members_list = DatasetPermissionService.get_dataset_partial_member_list(dataset_id_str, db.session())
+        partial_members_list = DatasetPermissionService.get_dataset_partial_member_list(dataset_id_str)
 
         return dump_response(PartialMemberListResponse, {"data": partial_members_list}), 200
 
@@ -1231,8 +1220,7 @@ class DatasetAutoDisableLogApi(Resource):
     @rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_READONLY)
     def get(self, dataset_id: UUID):
         dataset_id_str = str(dataset_id)
-        dataset = DatasetService.get_dataset(dataset_id_str, db.session())
+        dataset = DatasetService.get_dataset(dataset_id_str)
         if dataset is None:
             raise NotFound("Dataset not found.")
-        auto_disable_logs = DatasetService.get_dataset_auto_disable_logs(dataset_id_str, db.session())
-        return dump_response(AutoDisableLogsResponse, auto_disable_logs), 200
+        return dump_response(AutoDisableLogsResponse, DatasetService.get_dataset_auto_disable_logs(dataset_id_str)), 200

api/controllers/console/datasets/datasets_document.py

@@ -46,11 +46,9 @@ from graphon.model_runtime.errors.invoke import InvokeAuthorizationError
 from libs.datetime_utils import naive_utc_now
 from libs.helper import dump_response, to_timestamp
 from libs.login import login_required
-from libs.pagination import paginate_query
 from models import Account, DatasetProcessRule, Document, DocumentSegment, UploadFile
 from models.dataset import DocumentPipelineExecutionLog
 from models.enums import IndexingStatus, SegmentStatus
-from services.dataset_ref_service import DatasetRefService
 from services.dataset_service import DatasetService, DocumentService
 from services.entities.knowledge_entities.knowledge_entities import KnowledgeConfig, ProcessRule, RetrievalModel
 from services.file_service import FileService
@@ -183,16 +181,16 @@ class DocumentResource(Resource):
     def get_document(
         self, dataset_id: str, document_id: str, current_user: Account, current_tenant_id: str
     ) -> Document:
-        dataset = DatasetService.get_dataset(dataset_id, db.session())
+        dataset = DatasetService.get_dataset(dataset_id)
         if not dataset:
             raise NotFound("Dataset not found.")
 
         try:
-            DatasetService.check_dataset_permission(dataset, current_user, db.session())
+            DatasetService.check_dataset_permission(dataset, current_user)
         except services.errors.account.NoPermissionError as e:
             raise Forbidden(str(e))
 
-        document = DocumentService.get_document(dataset_id, document_id, session=db.session())
+        document = DocumentService.get_document(dataset_id, document_id)
 
         if not document:
             raise NotFound("Document not found.")
@@ -203,16 +201,16 @@ class DocumentResource(Resource):
         return document
 
     def get_batch_documents(self, dataset_id: str, batch: str, current_user: Account) -> Sequence[Document]:
-        dataset = DatasetService.get_dataset(dataset_id, db.session())
+        dataset = DatasetService.get_dataset(dataset_id)
         if not dataset:
             raise NotFound("Dataset not found.")
 
         try:
-            DatasetService.check_dataset_permission(dataset, current_user, db.session())
+            DatasetService.check_dataset_permission(dataset, current_user)
         except services.errors.account.NoPermissionError as e:
             raise Forbidden(str(e))
 
-        documents = DocumentService.get_batch_documents(dataset_id, batch, db.session())
+        documents = DocumentService.get_batch_documents(dataset_id, batch)
 
         if not documents:
             raise NotFound("Documents not found.")
@@ -243,13 +241,13 @@ class GetProcessRuleApi(Resource):
             # get the latest process rule
             document = db.get_or_404(Document, document_id)
 
-            dataset = DatasetService.get_dataset(document.dataset_id, db.session())
+            dataset = DatasetService.get_dataset(document.dataset_id)
 
             if not dataset:
                 raise NotFound("Dataset not found.")
 
             try:
-                DatasetService.check_dataset_permission(dataset, current_user, db.session())
+                DatasetService.check_dataset_permission(dataset, current_user)
             except services.errors.account.NoPermissionError as e:
                 raise Forbidden(str(e))
 
@@ -319,12 +317,12 @@ class DatasetDocumentListApi(Resource):
                     )
         except (ArgumentTypeError, ValueError, Exception):
             fetch = False
-        dataset = DatasetService.get_dataset(dataset_id_str, db.session())
+        dataset = DatasetService.get_dataset(dataset_id_str)
         if not dataset:
             raise NotFound("Dataset not found.")
 
         try:
-            DatasetService.check_dataset_permission(dataset, current_user, db.session())
+            DatasetService.check_dataset_permission(dataset, current_user)
         except services.errors.account.NoPermissionError as e:
             raise Forbidden(str(e))
 
@@ -369,14 +367,13 @@ class DatasetDocumentListApi(Resource):
                     desc(Document.position),
                 )
 
-        paginated_documents = paginate_query(query, page=page, per_page=limit, max_per_page=100)
+        paginated_documents = db.paginate(select=query, page=page, per_page=limit, max_per_page=100, error_out=False)
         documents = paginated_documents.items
 
         DocumentService.enrich_documents_with_summary_index_status(
             documents=documents,
             dataset=dataset,
             tenant_id=current_tenant_id,
-            session=db.session(),
         )
 
         if fetch:
@@ -424,7 +421,7 @@ class DatasetDocumentListApi(Resource):
     def post(self, current_user: Account, dataset_id: UUID):
         dataset_id_str = str(dataset_id)
 
-        dataset = DatasetService.get_dataset(dataset_id_str, db.session())
+        dataset = DatasetService.get_dataset(dataset_id_str)
 
         if not dataset:
             raise NotFound("Dataset not found.")
@@ -434,7 +431,7 @@ class DatasetDocumentListApi(Resource):
             raise Forbidden()
 
         try:
-            DatasetService.check_dataset_permission(dataset, current_user, db.session())
+            DatasetService.check_dataset_permission(dataset, current_user)
         except services.errors.account.NoPermissionError as e:
             raise Forbidden(str(e))
 
@@ -447,10 +444,8 @@ class DatasetDocumentListApi(Resource):
         DocumentService.document_create_args_validate(knowledge_config)
 
         try:
-            documents, batch = DocumentService.save_document_with_dataset_id(
-                dataset, knowledge_config, current_user, session=db.session()
-            )
-            dataset = DatasetService.get_dataset(dataset_id_str, db.session())
+            documents, batch = DocumentService.save_document_with_dataset_id(dataset, knowledge_config, current_user)
+            dataset = DatasetService.get_dataset(dataset_id_str)
 
         except ProviderTokenNotInitError as ex:
             raise ProviderNotInitializeError(ex.description)
@@ -469,7 +464,7 @@ class DatasetDocumentListApi(Resource):
     @rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_EDIT)
     def delete(self, dataset_id: UUID):
         dataset_id_str = str(dataset_id)
-        dataset = DatasetService.get_dataset(dataset_id_str, db.session())
+        dataset = DatasetService.get_dataset(dataset_id_str)
         if dataset is None:
             raise NotFound("Dataset not found.")
         # check user's model setting
@@ -477,8 +472,7 @@ class DatasetDocumentListApi(Resource):
 
         try:
             document_ids = request.args.getlist("document_id")
-            dataset_ref = DatasetRefService.create_dataset_ref(dataset)
-            DocumentService.delete_documents(dataset_ref, document_ids, dataset.doc_form, db.session())
+            DocumentService.delete_documents(dataset, document_ids)
         except services.errors.document.DocumentIndexingError:
             raise DocumentIndexingError("Cannot delete document during indexing.")
 
@@ -537,7 +531,6 @@ class DatasetInitApi(Resource):
                 tenant_id=current_tenant_id,
                 knowledge_config=knowledge_config,
                 account=current_user,
-                session=db.session(),
             )
         except ProviderTokenNotInitError as ex:
             raise ProviderNotInitializeError(ex.description)
@@ -874,7 +867,7 @@ class DocumentApi(DocumentResource):
         if metadata == "only":
             response = {"id": document.id, "doc_type": document.doc_type, "doc_metadata": document.doc_metadata_details}
         elif metadata == "without":
-            dataset_process_rules = DatasetService.get_process_rules(dataset_id_str, db.session())
+            dataset_process_rules = DatasetService.get_process_rules(dataset_id_str)
             document_process_rules = document.dataset_process_rule.to_dict() if document.dataset_process_rule else {}
             response = {
                 "id": document.id,
@@ -908,7 +901,7 @@ class DocumentApi(DocumentResource):
                 "need_summary": document.need_summary if document.need_summary is not None else False,
             }
         else:
-            dataset_process_rules = DatasetService.get_process_rules(dataset_id_str, db.session())
+            dataset_process_rules = DatasetService.get_process_rules(dataset_id_str)
             document_process_rules = document.dataset_process_rule.to_dict() if document.dataset_process_rule else {}
             response = {
                 "id": document.id,
@@ -957,7 +950,7 @@ class DocumentApi(DocumentResource):
     def delete(self, current_tenant_id: str, current_user: Account, dataset_id: UUID, document_id: UUID):
         dataset_id_str = str(dataset_id)
         document_id_str = str(document_id)
-        dataset = DatasetService.get_dataset(dataset_id_str, db.session())
+        dataset = DatasetService.get_dataset(dataset_id_str)
         if dataset is None:
             raise NotFound("Dataset not found.")
         # check user's model setting
@@ -966,7 +959,7 @@ class DocumentApi(DocumentResource):
         document = self.get_document(dataset_id_str, document_id_str, current_user, current_tenant_id)
 
         try:
-            DocumentService.delete_document(document, db.session())
+            DocumentService.delete_document(document)
         except services.errors.document.DocumentIndexingError:
             raise DocumentIndexingError("Cannot delete document during indexing.")
 
@@ -990,7 +983,7 @@ class DocumentDownloadApi(DocumentResource):
     def get(self, current_tenant_id: str, current_user: Account, dataset_id: UUID, document_id: UUID) -> dict[str, Any]:
         # Reuse the shared permission/tenant checks implemented in DocumentResource.
         document = self.get_document(str(dataset_id), str(document_id), current_user, current_tenant_id)
-        return {"url": DocumentService.get_document_download_url(document, db.session())}
+        return {"url": DocumentService.get_document_download_url(document)}
 
 
 @console_ns.route("/datasets/<uuid:dataset_id>/documents/download-zip")
@@ -1020,7 +1013,6 @@ class DocumentBatchDownloadZipApi(DocumentResource):
             document_ids=document_ids,
             tenant_id=current_tenant_id,
             current_user=current_user,
-            session=db.session(),
         )
 
         # Delegate ZIP packing to FileService, but keep Flask response+cleanup in the route.
@@ -1169,7 +1161,7 @@ class DocumentStatusApi(DocumentResource):
         self, current_user: Account, dataset_id: UUID, action: Literal["enable", "disable", "archive", "un_archive"]
     ):
         dataset_id_str = str(dataset_id)
-        dataset = DatasetService.get_dataset(dataset_id_str, db.session())
+        dataset = DatasetService.get_dataset(dataset_id_str)
         if dataset is None:
             raise NotFound("Dataset not found.")
 
@@ -1181,12 +1173,12 @@ class DocumentStatusApi(DocumentResource):
         DatasetService.check_dataset_model_setting(dataset)
 
         # check user's permission
-        DatasetService.check_dataset_permission(dataset, current_user, db.session())
+        DatasetService.check_dataset_permission(dataset, current_user)
 
         document_ids = request.args.getlist("document_id")
 
         try:
-            DocumentService.batch_update_document_status(dataset, document_ids, action, current_user, db.session())
+            DocumentService.batch_update_document_status(dataset, document_ids, action, current_user)
         except services.errors.document.DocumentIndexingError as e:
             raise InvalidActionError(str(e))
         except ValueError as e:
@@ -1210,11 +1202,11 @@ class DocumentPauseApi(DocumentResource):
         dataset_id_str = str(dataset_id)
         document_id_str = str(document_id)
 
-        dataset = DatasetService.get_dataset(dataset_id_str, db.session())
+        dataset = DatasetService.get_dataset(dataset_id_str)
         if not dataset:
             raise NotFound("Dataset not found.")
 
-        document = DocumentService.get_document(dataset.id, document_id_str, session=db.session())
+        document = DocumentService.get_document(dataset.id, document_id_str)
 
         # 404 if document not found
         if document is None:
@@ -1226,7 +1218,7 @@ class DocumentPauseApi(DocumentResource):
 
         try:
             # pause document
-            DocumentService.pause_document(document, db.session())
+            DocumentService.pause_document(document)
         except services.errors.document.DocumentIndexingError:
             raise DocumentIndexingError("Cannot pause completed document.")
 
@@ -1245,10 +1237,10 @@ class DocumentRecoverApi(DocumentResource):
         """recover document."""
         dataset_id_str = str(dataset_id)
         document_id_str = str(document_id)
-        dataset = DatasetService.get_dataset(dataset_id_str, db.session())
+        dataset = DatasetService.get_dataset(dataset_id_str)
         if not dataset:
             raise NotFound("Dataset not found.")
-        document = DocumentService.get_document(dataset.id, document_id_str, session=db.session())
+        document = DocumentService.get_document(dataset.id, document_id_str)
 
         # 404 if document not found
         if document is None:
@@ -1259,7 +1251,7 @@ class DocumentRecoverApi(DocumentResource):
             raise ArchivedDocumentImmutableError()
         try:
             # pause document
-            DocumentService.recover_document(document, db.session())
+            DocumentService.recover_document(document)
         except services.errors.document.DocumentIndexingError:
             raise DocumentIndexingError("Document is not in paused status.")
 
@@ -1279,13 +1271,13 @@ class DocumentRetryApi(DocumentResource):
         """retry document."""
         payload = DocumentRetryPayload.model_validate(console_ns.payload or {})
         dataset_id_str = str(dataset_id)
-        dataset = DatasetService.get_dataset(dataset_id_str, db.session())
+        dataset = DatasetService.get_dataset(dataset_id_str)
         retry_documents = []
         if not dataset:
             raise NotFound("Dataset not found.")
         for document_id in payload.document_ids:
             try:
-                document = DocumentService.get_document(dataset.id, document_id, session=db.session())
+                document = DocumentService.get_document(dataset.id, document_id)
 
                 # 404 if document not found
                 if document is None:
@@ -1303,7 +1295,7 @@ class DocumentRetryApi(DocumentResource):
                 logger.exception("Failed to retry document, document id: %s", document_id)
                 continue
         # retry document
-        DocumentService.retry_document(dataset_id_str, retry_documents, db.session())
+        DocumentService.retry_document(dataset_id_str, retry_documents)
 
         return "", 204
 
@@ -1321,14 +1313,14 @@ class DocumentRenameApi(DocumentResource):
         # The role of the current user in the ta table must be admin, owner, editor, or dataset_operator
         if not current_user.is_dataset_editor:
             raise Forbidden()
-        dataset = DatasetService.get_dataset(dataset_id, db.session())
+        dataset = DatasetService.get_dataset(dataset_id)
         if not dataset:
             raise NotFound("Dataset not found.")
-        DatasetService.check_dataset_operator_permission(current_user, dataset, session=db.session())
+        DatasetService.check_dataset_operator_permission(current_user, dataset)
         payload = DocumentRenamePayload.model_validate(console_ns.payload or {})
 
         try:
-            document = DocumentService.rename_document(str(dataset_id), str(document_id), payload.name, db.session())
+            document = DocumentService.rename_document(str(dataset_id), str(document_id), payload.name)
         except services.errors.document.DocumentIndexingError:
             raise DocumentIndexingError("Cannot delete document during indexing.")
 
@@ -1346,11 +1338,11 @@ class WebsiteDocumentSyncApi(DocumentResource):
     def get(self, current_tenant_id: str, dataset_id: UUID, document_id: UUID):
         """sync website document."""
         dataset_id_str = str(dataset_id)
-        dataset = DatasetService.get_dataset(dataset_id_str, db.session())
+        dataset = DatasetService.get_dataset(dataset_id_str)
         if not dataset:
             raise NotFound("Dataset not found.")
         document_id_str = str(document_id)
-        document = DocumentService.get_document(dataset.id, document_id_str, session=db.session())
+        document = DocumentService.get_document(dataset.id, document_id_str)
         if not document:
             raise NotFound("Document not found.")
         if document.tenant_id != current_tenant_id:
@@ -1361,7 +1353,7 @@ class WebsiteDocumentSyncApi(DocumentResource):
         if DocumentService.check_archived(document):
             raise ArchivedDocumentImmutableError()
         # sync document
-        DocumentService.sync_website_document(dataset_id_str, document, db.session())
+        DocumentService.sync_website_document(dataset_id_str, document)
 
         return {"result": "success"}, 200
 
@@ -1381,10 +1373,10 @@ class DocumentPipelineExecutionLogApi(DocumentResource):
         dataset_id_str = str(dataset_id)
         document_id_str = str(document_id)
 
-        dataset = DatasetService.get_dataset(dataset_id_str, db.session())
+        dataset = DatasetService.get_dataset(dataset_id_str)
         if not dataset:
             raise NotFound("Dataset not found.")
-        document = DocumentService.get_document(dataset.id, document_id_str, session=db.session())
+        document = DocumentService.get_document(dataset.id, document_id_str)
         if not document:
             raise NotFound("Document not found.")
         log = db.session.scalar(
@@ -1439,7 +1431,7 @@ class DocumentGenerateSummaryApi(Resource):
         dataset_id_str = str(dataset_id)
 
         # Get dataset
-        dataset = DatasetService.get_dataset(dataset_id_str, db.session())
+        dataset = DatasetService.get_dataset(dataset_id_str)
         if not dataset:
             raise NotFound("Dataset not found.")
 
@@ -1448,7 +1440,7 @@ class DocumentGenerateSummaryApi(Resource):
             raise Forbidden()
 
         try:
-            DatasetService.check_dataset_permission(dataset, current_user, db.session())
+            DatasetService.check_dataset_permission(dataset, current_user)
         except services.errors.account.NoPermissionError as e:
             raise Forbidden(str(e))
 
@@ -1473,7 +1465,7 @@ class DocumentGenerateSummaryApi(Resource):
             raise ValueError("Summary index is not enabled for this dataset. Please enable it in the dataset settings.")
 
         # Verify all documents exist and belong to the dataset
-        documents = DocumentService.get_documents_by_ids(dataset_id_str, document_list, db.session())
+        documents = DocumentService.get_documents_by_ids(dataset_id_str, document_list)
 
         if len(documents) != len(document_list):
             found_ids = {doc.id for doc in documents}
@@ -1489,7 +1481,6 @@ class DocumentGenerateSummaryApi(Resource):
             DocumentService.update_documents_need_summary(
                 dataset_id=dataset_id_str,
                 document_ids=document_ids_to_update,
-                session=db.session(),
                 need_summary=True,
             )
 
@@ -1540,13 +1531,13 @@ class DocumentSummaryStatusApi(DocumentResource):
         document_id_str = str(document_id)
 
         # Get dataset
-        dataset = DatasetService.get_dataset(dataset_id_str, db.session())
+        dataset = DatasetService.get_dataset(dataset_id_str)
         if not dataset:
             raise NotFound("Dataset not found.")
 
         # Check permissions
         try:
-            DatasetService.check_dataset_permission(dataset, current_user, db.session())
+            DatasetService.check_dataset_permission(dataset, current_user)
         except services.errors.account.NoPermissionError as e:
             raise Forbidden(str(e))
 
@@ -1556,7 +1547,6 @@ class DocumentSummaryStatusApi(DocumentResource):
         result = SummaryIndexService.get_document_summary_status_detail(
             document_id=document_id_str,
             dataset_id=dataset_id_str,
-            session=db.session(),
         )
 
         return result, 200

api/controllers/console/datasets/datasets_segments.py

@@ -57,11 +57,9 @@ from fields.segment_fields import (
 from graphon.model_runtime.entities.model_entities import ModelType
 from libs.helper import dump_response, escape_like_pattern
 from libs.login import login_required
-from libs.pagination import paginate_query
 from models import Account
-from models.dataset import Dataset, Document, DocumentSegment
+from models.dataset import ChildChunk, DocumentSegment
 from models.model import UploadFile
-from services.dataset_ref_service import DatasetRefService, SegmentRef
 from services.dataset_service import DatasetService, DocumentService, SegmentService
 from services.entities.knowledge_entities.knowledge_entities import ChildChunkUpdateArgs, SegmentUpdateArgs
 from services.errors.chunk import ChildChunkDeleteIndexError as ChildChunkDeleteIndexServiceError
@@ -164,21 +162,6 @@ register_response_schema_models(
 )
 
 
-def _get_segment_for_document(
-    dataset: Dataset, document: Document, segment_id: str
-) -> tuple[SegmentRef, DocumentSegment]:
-    dataset_ref = DatasetRefService.create_dataset_ref(dataset)
-    document_ref = DatasetRefService.create_document_ref(dataset_ref, document)
-    if document_ref is None:
-        raise NotFound("Document not found.")
-
-    segment_ref = DatasetRefService.create_segment_ref(document_ref, segment_id)
-    segment = SegmentService.get_segment_by_ref(segment_ref, db.session())
-    if not segment:
-        raise NotFound("Segment not found.")
-    return segment_ref, segment
-
-
 @console_ns.route("/datasets/<uuid:dataset_id>/documents/<uuid:document_id>/segments")
 class DatasetDocumentSegmentListApi(Resource):
     @console_ns.doc(params=SegmentDocParams.DATASET_DOCUMENT)
@@ -193,16 +176,16 @@ class DatasetDocumentSegmentListApi(Resource):
     def get(self, current_tenant_id: str, current_user: Account, dataset_id: UUID, document_id: UUID):
         dataset_id_str = str(dataset_id)
         document_id_str = str(document_id)
-        dataset = DatasetService.get_dataset(dataset_id_str, db.session())
+        dataset = DatasetService.get_dataset(dataset_id_str)
         if not dataset:
             raise NotFound("Dataset not found.")
 
         try:
-            DatasetService.check_dataset_permission(dataset, current_user, db.session())
+            DatasetService.check_dataset_permission(dataset, current_user)
         except services.errors.account.NoPermissionError as e:
             raise Forbidden(str(e))
 
-        document = DocumentService.get_document(dataset_id_str, document_id_str, session=db.session())
+        document = DocumentService.get_document(dataset_id_str, document_id_str)
 
         if not document:
             raise NotFound("Document not found.")
@@ -271,14 +254,14 @@ class DatasetDocumentSegmentListApi(Resource):
             elif args.enabled.lower() == "false":
                 query = query.where(DocumentSegment.enabled == False)
 
-        segments = paginate_query(query, page=page, per_page=limit, max_per_page=100)
+        segments = db.paginate(select=query, page=page, per_page=limit, max_per_page=100, error_out=False)
 
         segment_list = list(segments.items)
         segment_ids = [segment.id for segment in segment_list]
         summaries: dict[str, str | None] = {}
         if segment_ids:
             summary_records = SummaryIndexService.get_segments_summaries(
-                segment_ids=segment_ids, dataset_id=dataset_id_str, session=db.session()
+                segment_ids=segment_ids, dataset_id=dataset_id_str
             )
             summaries = {chunk_id: summary.summary_content for chunk_id, summary in summary_records.items()}
 
@@ -303,14 +286,14 @@ class DatasetDocumentSegmentListApi(Resource):
     def delete(self, current_user: Account, dataset_id: UUID, document_id: UUID):
         # check dataset
         dataset_id_str = str(dataset_id)
-        dataset = DatasetService.get_dataset(dataset_id_str, db.session())
+        dataset = DatasetService.get_dataset(dataset_id_str)
         if not dataset:
             raise NotFound("Dataset not found.")
         # check user's model setting
         DatasetService.check_dataset_model_setting(dataset)
         # check document
         document_id_str = str(document_id)
-        document = DocumentService.get_document(dataset_id_str, document_id_str, session=db.session())
+        document = DocumentService.get_document(dataset_id_str, document_id_str)
         if not document:
             raise NotFound("Document not found.")
         segment_ids = request.args.getlist("segment_id")
@@ -319,10 +302,10 @@ class DatasetDocumentSegmentListApi(Resource):
         if not current_user.is_dataset_editor:
             raise Forbidden()
         try:
-            DatasetService.check_dataset_permission(dataset, current_user, db.session())
+            DatasetService.check_dataset_permission(dataset, current_user)
         except services.errors.account.NoPermissionError as e:
             raise Forbidden(str(e))
-        SegmentService.delete_segments(segment_ids, document, dataset, db.session())
+        SegmentService.delete_segments(segment_ids, document, dataset)
         return "", 204
 
 
@@ -348,11 +331,11 @@ class DatasetDocumentSegmentApi(Resource):
         action: Literal["enable", "disable"],
     ):
         dataset_id_str = str(dataset_id)
-        dataset = DatasetService.get_dataset(dataset_id_str, db.session())
+        dataset = DatasetService.get_dataset(dataset_id_str)
         if not dataset:
             raise NotFound("Dataset not found.")
         document_id_str = str(document_id)
-        document = DocumentService.get_document(dataset_id_str, document_id_str, session=db.session())
+        document = DocumentService.get_document(dataset_id_str, document_id_str)
         if not document:
             raise NotFound("Document not found.")
         # check user's model setting
@@ -362,7 +345,7 @@ class DatasetDocumentSegmentApi(Resource):
             raise Forbidden()
 
         try:
-            DatasetService.check_dataset_permission(dataset, current_user, db.session())
+            DatasetService.check_dataset_permission(dataset, current_user)
         except services.errors.account.NoPermissionError as e:
             raise Forbidden(str(e))
         if dataset.indexing_technique == IndexTechniqueType.HIGH_QUALITY:
@@ -388,7 +371,7 @@ class DatasetDocumentSegmentApi(Resource):
         if cache_result is not None:
             raise InvalidActionError("Document is being indexed, please try again later")
         try:
-            SegmentService.update_segments_status(segment_ids, action, dataset, document, db.session())
+            SegmentService.update_segments_status(segment_ids, action, dataset, document)
         except Exception as e:
             raise InvalidActionError(str(e))
         return dump_response(SimpleResultResponse, {"result": "success"}), 200
@@ -411,12 +394,12 @@ class DatasetDocumentSegmentAddApi(Resource):
     def post(self, current_tenant_id: str, current_user: Account, dataset_id: UUID, document_id: UUID):
         # check dataset
         dataset_id_str = str(dataset_id)
-        dataset = DatasetService.get_dataset(dataset_id_str, db.session())
+        dataset = DatasetService.get_dataset(dataset_id_str)
         if not dataset:
             raise NotFound("Dataset not found.")
         # check document
         document_id_str = str(document_id)
-        document = DocumentService.get_document(dataset_id_str, document_id_str, session=db.session())
+        document = DocumentService.get_document(dataset_id_str, document_id_str)
         if not document:
             raise NotFound("Document not found.")
         if not current_user.is_dataset_editor:
@@ -438,20 +421,15 @@ class DatasetDocumentSegmentAddApi(Resource):
             except ProviderTokenNotInitError as ex:
                 raise ProviderNotInitializeError(ex.description)
         try:
-            DatasetService.check_dataset_permission(dataset, current_user, db.session())
+            DatasetService.check_dataset_permission(dataset, current_user)
         except services.errors.account.NoPermissionError as e:
             raise Forbidden(str(e))
         # validate args
         payload = SegmentCreatePayload.model_validate(console_ns.payload or {})
         payload_dict = payload.model_dump(exclude_none=True)
         SegmentService.segment_create_args_validate(payload_dict, document)
-        segment = type_cast(
-            DocumentSegment,
-            SegmentService.create_segment(payload_dict, document, dataset, db.session()),
-        )
-        summary = SummaryIndexService.get_segment_summary(
-            segment_id=segment.id, dataset_id=dataset_id_str, session=db.session()
-        )
+        segment = type_cast(DocumentSegment, SegmentService.create_segment(payload_dict, document, dataset))
+        summary = SummaryIndexService.get_segment_summary(segment_id=segment.id, dataset_id=dataset_id_str)
         response = {
             "data": segment_response_with_summary(segment, summary.summary_content if summary else None),
             "doc_form": document.doc_form,
@@ -477,23 +455,16 @@ class DatasetDocumentSegmentUpdateApi(Resource):
     ):
         # check dataset
         dataset_id_str = str(dataset_id)
-        dataset = DatasetService.get_dataset(dataset_id_str, db.session())
+        dataset = DatasetService.get_dataset(dataset_id_str)
         if not dataset:
             raise NotFound("Dataset not found.")
         # check user's model setting
         DatasetService.check_dataset_model_setting(dataset)
         # check document
         document_id_str = str(document_id)
-        document = DocumentService.get_document(dataset_id_str, document_id_str, session=db.session())
+        document = DocumentService.get_document(dataset_id_str, document_id_str)
         if not document:
             raise NotFound("Document not found.")
-        # The role of the current user in the ta table must be admin, owner, dataset_operator, or editor
-        if not current_user.is_dataset_editor:
-            raise Forbidden()
-        try:
-            DatasetService.check_dataset_permission(dataset, current_user, db.session())
-        except services.errors.account.NoPermissionError as e:
-            raise Forbidden(str(e))
         if dataset.indexing_technique == IndexTechniqueType.HIGH_QUALITY:
             # check embedding model setting
             try:
@@ -510,8 +481,22 @@ class DatasetDocumentSegmentUpdateApi(Resource):
                 )
             except ProviderTokenNotInitError as ex:
                 raise ProviderNotInitializeError(ex.description)
+            # check segment
         segment_id_str = str(segment_id)
-        _, segment = _get_segment_for_document(dataset, document, segment_id_str)
+        segment = db.session.scalar(
+            select(DocumentSegment)
+            .where(DocumentSegment.id == segment_id_str, DocumentSegment.tenant_id == current_tenant_id)
+            .limit(1)
+        )
+        if not segment:
+            raise NotFound("Segment not found.")
+        # The role of the current user in the ta table must be admin, owner, dataset_operator, or editor
+        if not current_user.is_dataset_editor:
+            raise Forbidden()
+        try:
+            DatasetService.check_dataset_permission(dataset, current_user)
+        except services.errors.account.NoPermissionError as e:
+            raise Forbidden(str(e))
         # validate args
         payload = SegmentUpdatePayload.model_validate(console_ns.payload or {})
         payload_dict = payload.model_dump(exclude_none=True)
@@ -519,15 +504,9 @@ class DatasetDocumentSegmentUpdateApi(Resource):
 
         # Update segment (summary update with change detection is handled in SegmentService.update_segment)
         segment = SegmentService.update_segment(
-            SegmentUpdateArgs.model_validate(payload.model_dump(exclude_none=True)),
-            segment,
-            document,
-            dataset,
-            db.session(),
-        )
-        summary = SummaryIndexService.get_segment_summary(
-            segment_id=segment.id, dataset_id=dataset_id_str, session=db.session()
+            SegmentUpdateArgs.model_validate(payload.model_dump(exclude_none=True)), segment, document, dataset
         )
+        summary = SummaryIndexService.get_segment_summary(segment_id=segment.id, dataset_id=dataset_id_str)
         response = {
             "data": segment_response_with_summary(segment, summary.summary_content if summary else None),
             "doc_form": document.doc_form,
@@ -548,26 +527,33 @@ class DatasetDocumentSegmentUpdateApi(Resource):
     ):
         # check dataset
         dataset_id_str = str(dataset_id)
-        dataset = DatasetService.get_dataset(dataset_id_str, db.session())
+        dataset = DatasetService.get_dataset(dataset_id_str)
         if not dataset:
             raise NotFound("Dataset not found.")
         # check user's model setting
         DatasetService.check_dataset_model_setting(dataset)
         # check document
         document_id_str = str(document_id)
-        document = DocumentService.get_document(dataset_id_str, document_id_str, session=db.session())
+        document = DocumentService.get_document(dataset_id_str, document_id_str)
         if not document:
             raise NotFound("Document not found.")
+        # check segment
+        segment_id_str = str(segment_id)
+        segment = db.session.scalar(
+            select(DocumentSegment)
+            .where(DocumentSegment.id == segment_id_str, DocumentSegment.tenant_id == current_tenant_id)
+            .limit(1)
+        )
+        if not segment:
+            raise NotFound("Segment not found.")
         # The role of the current user in the ta table must be admin, owner, dataset_operator, or editor
         if not current_user.is_dataset_editor:
             raise Forbidden()
         try:
-            DatasetService.check_dataset_permission(dataset, current_user, db.session())
+            DatasetService.check_dataset_permission(dataset, current_user)
         except services.errors.account.NoPermissionError as e:
             raise Forbidden(str(e))
-        segment_id_str = str(segment_id)
-        _, segment = _get_segment_for_document(dataset, document, segment_id_str)
-        SegmentService.delete_segment(segment, document, dataset, db.session())
+        SegmentService.delete_segment(segment, document, dataset)
         return "", 204
 
 
@@ -590,12 +576,12 @@ class DatasetDocumentSegmentBatchImportApi(Resource):
     def post(self, current_tenant_id: str, current_user: Account, dataset_id: UUID, document_id: UUID):
         # check dataset
         dataset_id_str = str(dataset_id)
-        dataset = DatasetService.get_dataset(dataset_id_str, db.session())
+        dataset = DatasetService.get_dataset(dataset_id_str)
         if not dataset:
             raise NotFound("Dataset not found.")
         # check document
         document_id_str = str(document_id)
-        document = DocumentService.get_document(dataset_id_str, document_id_str, session=db.session())
+        document = DocumentService.get_document(dataset_id_str, document_id_str)
         if not document:
             raise NotFound("Document not found.")
 
@@ -665,20 +651,25 @@ class ChildChunkAddApi(Resource):
     ):
         # check dataset
         dataset_id_str = str(dataset_id)
-        dataset = DatasetService.get_dataset(dataset_id_str, db.session())
+        dataset = DatasetService.get_dataset(dataset_id_str)
         if not dataset:
             raise NotFound("Dataset not found.")
         # check document
         document_id_str = str(document_id)
-        document = DocumentService.get_document(dataset_id_str, document_id_str, session=db.session())
+        document = DocumentService.get_document(dataset_id_str, document_id_str)
         if not document:
             raise NotFound("Document not found.")
+        # check segment
+        segment_id_str = str(segment_id)
+        segment = db.session.scalar(
+            select(DocumentSegment)
+            .where(DocumentSegment.id == segment_id_str, DocumentSegment.tenant_id == current_tenant_id)
+            .limit(1)
+        )
+        if not segment:
+            raise NotFound("Segment not found.")
         if not current_user.is_dataset_editor:
             raise Forbidden()
-        try:
-            DatasetService.check_dataset_permission(dataset, current_user, db.session())
-        except services.errors.account.NoPermissionError as e:
-            raise Forbidden(str(e))
         # check embedding model setting
         if dataset.indexing_technique == IndexTechniqueType.HIGH_QUALITY:
             try:
@@ -695,12 +686,14 @@ class ChildChunkAddApi(Resource):
                 )
             except ProviderTokenNotInitError as ex:
                 raise ProviderNotInitializeError(ex.description)
-        segment_id_str = str(segment_id)
-        _, segment = _get_segment_for_document(dataset, document, segment_id_str)
+        try:
+            DatasetService.check_dataset_permission(dataset, current_user)
+        except services.errors.account.NoPermissionError as e:
+            raise Forbidden(str(e))
         # validate args
         try:
             payload = ChildChunkCreatePayload.model_validate(console_ns.payload or {})
-            child_chunk = SegmentService.create_child_chunk(payload.content, segment, document, dataset, db.session())
+            child_chunk = SegmentService.create_child_chunk(payload.content, segment, document, dataset)
         except ChildChunkIndexingServiceError as e:
             raise ChildChunkIndexingError(str(e))
         return dump_response(ChildChunkDetailResponse, {"data": child_chunk}), 200
@@ -716,18 +709,25 @@ class ChildChunkAddApi(Resource):
     def get(self, current_tenant_id: str, dataset_id: UUID, document_id: UUID, segment_id: UUID):
         # check dataset
         dataset_id_str = str(dataset_id)
-        dataset = DatasetService.get_dataset(dataset_id_str, db.session())
+        dataset = DatasetService.get_dataset(dataset_id_str)
         if not dataset:
             raise NotFound("Dataset not found.")
         # check user's model setting
         DatasetService.check_dataset_model_setting(dataset)
         # check document
         document_id_str = str(document_id)
-        document = DocumentService.get_document(dataset_id_str, document_id_str, session=db.session())
+        document = DocumentService.get_document(dataset_id_str, document_id_str)
         if not document:
             raise NotFound("Document not found.")
+        # check segment
         segment_id_str = str(segment_id)
-        _get_segment_for_document(dataset, document, segment_id_str)
+        segment = db.session.scalar(
+            select(DocumentSegment)
+            .where(DocumentSegment.id == segment_id_str, DocumentSegment.tenant_id == current_tenant_id)
+            .limit(1)
+        )
+        if not segment:
+            raise NotFound("Segment not found.")
         args = query_params_from_request(ChildChunkListQuery, use_defaults_for_malformed_ints=True)
 
         page = args.page
@@ -766,29 +766,36 @@ class ChildChunkAddApi(Resource):
     ):
         # check dataset
         dataset_id_str = str(dataset_id)
-        dataset = DatasetService.get_dataset(dataset_id_str, db.session())
+        dataset = DatasetService.get_dataset(dataset_id_str)
         if not dataset:
             raise NotFound("Dataset not found.")
         # check user's model setting
         DatasetService.check_dataset_model_setting(dataset)
         # check document
         document_id_str = str(document_id)
-        document = DocumentService.get_document(dataset_id_str, document_id_str, session=db.session())
+        document = DocumentService.get_document(dataset_id_str, document_id_str)
         if not document:
             raise NotFound("Document not found.")
+            # check segment
+        segment_id_str = str(segment_id)
+        segment = db.session.scalar(
+            select(DocumentSegment)
+            .where(DocumentSegment.id == segment_id_str, DocumentSegment.tenant_id == current_tenant_id)
+            .limit(1)
+        )
+        if not segment:
+            raise NotFound("Segment not found.")
         # The role of the current user in the ta table must be admin, owner, dataset_operator, or editor
         if not current_user.is_dataset_editor:
             raise Forbidden()
         try:
-            DatasetService.check_dataset_permission(dataset, current_user, db.session())
+            DatasetService.check_dataset_permission(dataset, current_user)
         except services.errors.account.NoPermissionError as e:
             raise Forbidden(str(e))
-        segment_id_str = str(segment_id)
-        _, segment = _get_segment_for_document(dataset, document, segment_id_str)
         # validate args
         payload = ChildChunkBatchUpdatePayload.model_validate(console_ns.payload or {})
         try:
-            child_chunks = SegmentService.update_child_chunks(payload.chunks, segment, document, dataset, db.session())
+            child_chunks = SegmentService.update_child_chunks(payload.chunks, segment, document, dataset)
         except ChildChunkIndexingServiceError as e:
             raise ChildChunkIndexingError(str(e))
         return dump_response(ChildChunkBatchUpdateResponse, {"data": child_chunks}), 200
@@ -818,31 +825,48 @@ class ChildChunkUpdateApi(Resource):
     ):
         # check dataset
         dataset_id_str = str(dataset_id)
-        dataset = DatasetService.get_dataset(dataset_id_str, db.session())
+        dataset = DatasetService.get_dataset(dataset_id_str)
         if not dataset:
             raise NotFound("Dataset not found.")
         # check user's model setting
         DatasetService.check_dataset_model_setting(dataset)
         # check document
         document_id_str = str(document_id)
-        document = DocumentService.get_document(dataset_id_str, document_id_str, session=db.session())
+        document = DocumentService.get_document(dataset_id_str, document_id_str)
         if not document:
             raise NotFound("Document not found.")
+        # check segment
+        segment_id_str = str(segment_id)
+        segment = db.session.scalar(
+            select(DocumentSegment)
+            .where(DocumentSegment.id == segment_id_str, DocumentSegment.tenant_id == current_tenant_id)
+            .limit(1)
+        )
+        if not segment:
+            raise NotFound("Segment not found.")
+        # check child chunk
+        child_chunk_id_str = str(child_chunk_id)
+        child_chunk = db.session.scalar(
+            select(ChildChunk)
+            .where(
+                ChildChunk.id == child_chunk_id_str,
+                ChildChunk.tenant_id == current_tenant_id,
+                ChildChunk.segment_id == segment.id,
+                ChildChunk.document_id == document_id_str,
+            )
+            .limit(1)
+        )
+        if not child_chunk:
+            raise NotFound("Child chunk not found.")
         # The role of the current user in the ta table must be admin, owner, dataset_operator, or editor
         if not current_user.is_dataset_editor:
             raise Forbidden()
         try:
-            DatasetService.check_dataset_permission(dataset, current_user, db.session())
+            DatasetService.check_dataset_permission(dataset, current_user)
         except services.errors.account.NoPermissionError as e:
             raise Forbidden(str(e))
-        segment_id_str = str(segment_id)
-        segment_ref, _ = _get_segment_for_document(dataset, document, segment_id_str)
-        child_chunk_id_str = str(child_chunk_id)
-        child_chunk = SegmentService.get_child_chunk_by_segment_ref(child_chunk_id_str, segment_ref, db.session())
-        if not child_chunk:
-            raise NotFound("Child chunk not found.")
         try:
-            SegmentService.delete_child_chunk(child_chunk, dataset, db.session())
+            SegmentService.delete_child_chunk(child_chunk, dataset)
         except ChildChunkDeleteIndexServiceError as e:
             raise ChildChunkDeleteIndexError(str(e))
         return "", 204
@@ -869,35 +893,50 @@ class ChildChunkUpdateApi(Resource):
     ):
         # check dataset
         dataset_id_str = str(dataset_id)
-        dataset = DatasetService.get_dataset(dataset_id_str, db.session())
+        dataset = DatasetService.get_dataset(dataset_id_str)
         if not dataset:
             raise NotFound("Dataset not found.")
         # check user's model setting
         DatasetService.check_dataset_model_setting(dataset)
         # check document
         document_id_str = str(document_id)
-        document = DocumentService.get_document(dataset_id_str, document_id_str, session=db.session())
+        document = DocumentService.get_document(dataset_id_str, document_id_str)
         if not document:
             raise NotFound("Document not found.")
+            # check segment
+        segment_id_str = str(segment_id)
+        segment = db.session.scalar(
+            select(DocumentSegment)
+            .where(DocumentSegment.id == segment_id_str, DocumentSegment.tenant_id == current_tenant_id)
+            .limit(1)
+        )
+        if not segment:
+            raise NotFound("Segment not found.")
+        # check child chunk
+        child_chunk_id_str = str(child_chunk_id)
+        child_chunk = db.session.scalar(
+            select(ChildChunk)
+            .where(
+                ChildChunk.id == child_chunk_id_str,
+                ChildChunk.tenant_id == current_tenant_id,
+                ChildChunk.segment_id == segment.id,
+                ChildChunk.document_id == document_id_str,
+            )
+            .limit(1)
+        )
+        if not child_chunk:
+            raise NotFound("Child chunk not found.")
         # The role of the current user in the ta table must be admin, owner, dataset_operator, or editor
         if not current_user.is_dataset_editor:
             raise Forbidden()
         try:
-            DatasetService.check_dataset_permission(dataset, current_user, db.session())
+            DatasetService.check_dataset_permission(dataset, current_user)
         except services.errors.account.NoPermissionError as e:
             raise Forbidden(str(e))
-        segment_id_str = str(segment_id)
-        segment_ref, segment = _get_segment_for_document(dataset, document, segment_id_str)
-        child_chunk_id_str = str(child_chunk_id)
-        child_chunk = SegmentService.get_child_chunk_by_segment_ref(child_chunk_id_str, segment_ref, db.session())
-        if not child_chunk:
-            raise NotFound("Child chunk not found.")
         # validate args
         try:
             payload = ChildChunkUpdatePayload.model_validate(console_ns.payload or {})
-            child_chunk = SegmentService.update_child_chunk(
-                payload.content, child_chunk, segment, document, dataset, db.session()
-            )
+            child_chunk = SegmentService.update_child_chunk(payload.content, child_chunk, segment, document, dataset)
         except ChildChunkIndexingServiceError as e:
             raise ChildChunkIndexingError(str(e))
         return dump_response(ChildChunkDetailResponse, {"data": child_chunk}), 200

api/controllers/console/datasets/external.py

@@ -4,7 +4,6 @@ from uuid import UUID
 from flask import request
 from flask_restx import Resource, fields, marshal
 from pydantic import BaseModel, Field, RootModel
-from sqlalchemy.orm import Session
 from werkzeug.exceptions import Forbidden, InternalServerError, NotFound
 
 import services
@@ -16,7 +15,6 @@ from controllers.common.schema import (
     register_schema_models,
 )
 from controllers.console import console_ns
-from controllers.console.app.wraps import with_session
 from controllers.console.datasets.error import DatasetNameDuplicateError
 from controllers.console.wraps import (
     RBACPermission,
@@ -209,8 +207,7 @@ class ExternalApiTemplateListApi(Resource):
     )
     @with_current_user
     @with_current_tenant_id
-    @with_session
-    def post(self, session: Session, current_tenant_id: str, current_user: Account):
+    def post(self, current_tenant_id: str, current_user: Account):
         payload = ExternalKnowledgeApiPayload.model_validate(console_ns.payload or {})
 
         ExternalDatasetService.validate_api_list(payload.settings)
@@ -221,10 +218,7 @@ class ExternalApiTemplateListApi(Resource):
 
         try:
             external_knowledge_api = ExternalDatasetService.create_external_knowledge_api(
-                tenant_id=current_tenant_id,
-                user_id=current_user.id,
-                args=payload.model_dump(),
-                session=session,
+                tenant_id=current_tenant_id, user_id=current_user.id, args=payload.model_dump()
             )
         except services.errors.dataset.DatasetNameDuplicateError:
             raise DatasetNameDuplicateError()
@@ -247,11 +241,10 @@ class ExternalApiTemplateApi(Resource):
     @login_required
     @account_initialization_required
     @with_current_tenant_id
-    @with_session
-    def get(self, session: Session, current_tenant_id: str, external_knowledge_api_id: UUID):
+    def get(self, current_tenant_id: str, external_knowledge_api_id: UUID):
         external_knowledge_api_id_str = str(external_knowledge_api_id)
         external_knowledge_api = ExternalDatasetService.get_external_knowledge_api(
-            external_knowledge_api_id=external_knowledge_api_id_str, tenant_id=current_tenant_id, session=session
+            external_knowledge_api_id_str, current_tenant_id
         )
         if external_knowledge_api is None:
             raise NotFound("API template not found.")
@@ -269,8 +262,7 @@ class ExternalApiTemplateApi(Resource):
     @console_ns.expect(console_ns.models[ExternalKnowledgeApiPayload.__name__])
     @with_current_user
     @with_current_tenant_id
-    @with_session
-    def patch(self, session: Session, current_tenant_id: str, current_user: Account, external_knowledge_api_id: UUID):
+    def patch(self, current_tenant_id: str, current_user: Account, external_knowledge_api_id: UUID):
         external_knowledge_api_id_str = str(external_knowledge_api_id)
 
         payload = ExternalKnowledgeApiPayload.model_validate(console_ns.payload or {})
@@ -281,7 +273,6 @@ class ExternalApiTemplateApi(Resource):
             user_id=current_user.id,
             external_knowledge_api_id=external_knowledge_api_id_str,
             args=payload.model_dump(),
-            session=session,
         )
 
         return external_knowledge_api.to_dict(), 200
@@ -292,16 +283,13 @@ class ExternalApiTemplateApi(Resource):
     @console_ns.response(204, "External knowledge API deleted successfully")
     @with_current_user
     @with_current_tenant_id
-    @with_session
-    def delete(self, session: Session, current_tenant_id: str, current_user: Account, external_knowledge_api_id: UUID):
+    def delete(self, current_tenant_id: str, current_user: Account, external_knowledge_api_id: UUID):
         external_knowledge_api_id_str = str(external_knowledge_api_id)
 
         if not (current_user.has_edit_permission or current_user.is_dataset_operator):
             raise Forbidden()
 
-        ExternalDatasetService.delete_external_knowledge_api(
-            current_tenant_id, external_knowledge_api_id_str, session=session
-        )
+        ExternalDatasetService.delete_external_knowledge_api(current_tenant_id, external_knowledge_api_id_str)
         return "", 204
 
 
@@ -315,12 +303,11 @@ class ExternalApiUseCheckApi(Resource):
     @login_required
     @account_initialization_required
     @with_current_tenant_id
-    @with_session
-    def get(self, session: Session, current_tenant_id: str, external_knowledge_api_id: UUID):
+    def get(self, current_tenant_id: str, external_knowledge_api_id: UUID):
         external_knowledge_api_id_str = str(external_knowledge_api_id)
 
         external_knowledge_api_is_using, count = ExternalDatasetService.external_knowledge_api_use_check(
-            external_knowledge_api_id_str, current_tenant_id, session=session
+            external_knowledge_api_id_str, current_tenant_id
         )
         return {"is_using": external_knowledge_api_is_using, "count": count}, 200
 
@@ -340,8 +327,7 @@ class ExternalDatasetCreateApi(Resource):
     @rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_EXTERNAL_CONNECT)
     @with_current_user
     @with_current_tenant_id
-    @with_session
-    def post(self, session: Session, current_tenant_id: str, current_user: Account):
+    def post(self, current_tenant_id: str, current_user: Account):
         # The role of the current user in the ta table must be admin, owner, or editor
         payload = ExternalDatasetCreatePayload.model_validate(console_ns.payload or {})
         args = payload.model_dump(exclude_none=True)
@@ -355,7 +341,6 @@ class ExternalDatasetCreateApi(Resource):
                 tenant_id=current_tenant_id,
                 user_id=current_user.id,
                 args=args,
-                session=session,
             )
         except services.errors.dataset.DatasetNameDuplicateError:
             raise DatasetNameDuplicateError()
@@ -366,7 +351,6 @@ class ExternalDatasetCreateApi(Resource):
             str(current_tenant_id),
             current_user.id,
             [dataset_id_str],
-            session=session,
         )
         item["permission_keys"] = permission_keys_map.get(dataset_id_str, [])
 
@@ -391,15 +375,14 @@ class ExternalKnowledgeHitTestingApi(Resource):
     @account_initialization_required
     @with_current_user
     @rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_PIPELINE_TEST)
-    @with_session
-    def post(self, session: Session, current_user: Account, dataset_id: UUID):
+    def post(self, current_user: Account, dataset_id: UUID):
         dataset_id_str = str(dataset_id)
-        dataset = DatasetService.get_dataset(dataset_id_str, db.session())
+        dataset = DatasetService.get_dataset(dataset_id_str)
         if dataset is None:
             raise NotFound("Dataset not found.")
 
         try:
-            DatasetService.check_dataset_permission(dataset, current_user, db.session())
+            DatasetService.check_dataset_permission(dataset, current_user)
         except services.errors.account.NoPermissionError as e:
             raise Forbidden(str(e))
 
@@ -408,7 +391,7 @@ class ExternalKnowledgeHitTestingApi(Resource):
 
         try:
             response = HitTestingService.external_retrieve(
-                session=session,
+                session=db.session,
                 dataset=dataset,
                 query=payload.query,
                 account=current_user,

api/controllers/console/datasets/hit_testing.py

@@ -3,10 +3,8 @@ from __future__ import annotations
 from uuid import UUID
 
 from flask_restx import Resource
-from sqlalchemy.orm import Session
 
 from controllers.common.schema import register_response_schema_models, register_schema_models
-from controllers.console.app.wraps import with_session
 from controllers.console.wraps import RBACPermission, RBACResourceScope, rbac_permission_required
 from fields.hit_testing_fields import HitTestingResponse
 from libs.helper import dump_response
@@ -47,10 +45,7 @@ class HitTestingApi(Resource, DatasetsHitTestingBase):
     @with_current_tenant_id
     @with_current_user
     @rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_PIPELINE_TEST)
-    @with_session
-    def post(
-        self, session: Session, current_user: Account, current_tenant_id: str, dataset_id: UUID
-    ) -> dict[str, object]:
+    def post(self, current_user: Account, current_tenant_id: str, dataset_id: UUID) -> dict[str, object]:
         dataset_id_str = str(dataset_id)
 
         dataset = self.get_and_validate_dataset(dataset_id_str, current_user, current_tenant_id)
@@ -59,5 +54,5 @@ class HitTestingApi(Resource, DatasetsHitTestingBase):
 
         return dump_response(
             HitTestingResponse,
-            self.perform_hit_testing(session, dataset, args, current_user, current_tenant_id),
+            self.perform_hit_testing(dataset, args, current_user, current_tenant_id),
         )

api/controllers/console/datasets/hit_testing_base.py

@@ -2,7 +2,6 @@ import logging
 from typing import Any, cast
 
 from pydantic import BaseModel, Field
-from sqlalchemy.orm import Session
 from werkzeug.exceptions import Forbidden, InternalServerError, NotFound
 
 import services
@@ -86,12 +85,12 @@ class DatasetsHitTestingBase:
         dataset_id: str, current_user: Account | None = None, current_tenant_id: str | None = None
     ) -> Dataset:
         current_user, _ = resolve_account_fallback(current_user, current_tenant_id)
-        dataset = DatasetService.get_dataset(dataset_id, db.session())
+        dataset = DatasetService.get_dataset(dataset_id)
         if dataset is None:
             raise NotFound("Dataset not found.")
 
         try:
-            DatasetService.check_dataset_permission(dataset, current_user, db.session())
+            DatasetService.check_dataset_permission(dataset, current_user)
         except services.errors.account.NoPermissionError as e:
             raise Forbidden(str(e))
 
@@ -109,7 +108,6 @@ class DatasetsHitTestingBase:
 
     @staticmethod
     def perform_hit_testing(
-        session: Session,
         dataset: Dataset,
         args: dict[str, Any],
         current_user: Account | None = None,
@@ -118,7 +116,7 @@ class DatasetsHitTestingBase:
         try:
             current_user, _ = resolve_account_fallback(current_user, current_tenant_id)
             response = HitTestingService.retrieve(
-                session=session,
+                session=db.session,
                 dataset=dataset,
                 query=cast(str, args.get("query")),
                 account=current_user,

api/controllers/console/datasets/metadata.py

@@ -61,10 +61,10 @@ class DatasetMetadataCreateApi(Resource):
         metadata_args = MetadataArgs.model_validate(console_ns.payload or {})
 
         dataset_id_str = str(dataset_id)
-        dataset = DatasetService.get_dataset(dataset_id_str, db.session())
+        dataset = DatasetService.get_dataset(dataset_id_str)
         if dataset is None:
             raise NotFound("Dataset not found.")
-        DatasetService.check_dataset_permission(dataset, current_user, db.session())
+        DatasetService.check_dataset_permission(dataset, current_user)
 
         metadata = MetadataService.create_metadata(
             db.session(), dataset_id_str, metadata_args, current_user, current_tenant_id
@@ -81,7 +81,7 @@ class DatasetMetadataCreateApi(Resource):
     @rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_CREATE_AND_MANAGEMENT)
     def get(self, dataset_id: UUID):
         dataset_id_str = str(dataset_id)
-        dataset = DatasetService.get_dataset(dataset_id_str, db.session())
+        dataset = DatasetService.get_dataset(dataset_id_str)
         if dataset is None:
             raise NotFound("Dataset not found.")
         metadata = MetadataService.get_dataset_metadatas(db.session(), dataset)
@@ -105,10 +105,10 @@ class DatasetMetadataApi(Resource):
 
         dataset_id_str = str(dataset_id)
         metadata_id_str = str(metadata_id)
-        dataset = DatasetService.get_dataset(dataset_id_str, db.session())
+        dataset = DatasetService.get_dataset(dataset_id_str)
         if dataset is None:
             raise NotFound("Dataset not found.")
-        DatasetService.check_dataset_permission(dataset, current_user, db.session())
+        DatasetService.check_dataset_permission(dataset, current_user)
 
         metadata = MetadataService.update_metadata_name(
             db.session(), dataset_id_str, metadata_id_str, name, current_user, current_tenant_id
@@ -125,10 +125,10 @@ class DatasetMetadataApi(Resource):
     def delete(self, current_user: Account, dataset_id: UUID, metadata_id: UUID):
         dataset_id_str = str(dataset_id)
         metadata_id_str = str(metadata_id)
-        dataset = DatasetService.get_dataset(dataset_id_str, db.session())
+        dataset = DatasetService.get_dataset(dataset_id_str)
         if dataset is None:
             raise NotFound("Dataset not found.")
-        DatasetService.check_dataset_permission(dataset, current_user, db.session())
+        DatasetService.check_dataset_permission(dataset, current_user)
 
         MetadataService.delete_metadata(db.session(), dataset_id_str, metadata_id_str)
         # Frontend callers only await success and invalidate metadata caches; no response body is consumed.
@@ -162,10 +162,10 @@ class DatasetMetadataBuiltInFieldActionApi(Resource):
     @rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_EDIT)
     def post(self, current_user: Account, dataset_id: UUID, action: Literal["enable", "disable"]):
         dataset_id_str = str(dataset_id)
-        dataset = DatasetService.get_dataset(dataset_id_str, db.session())
+        dataset = DatasetService.get_dataset(dataset_id_str)
         if dataset is None:
             raise NotFound("Dataset not found.")
-        DatasetService.check_dataset_permission(dataset, current_user, db.session())
+        DatasetService.check_dataset_permission(dataset, current_user)
 
         match action:
             case "enable":
@@ -191,10 +191,10 @@ class DocumentMetadataEditApi(Resource):
     @rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_EDIT)
     def post(self, current_user: Account, dataset_id: UUID):
         dataset_id_str = str(dataset_id)
-        dataset = DatasetService.get_dataset(dataset_id_str, db.session())
+        dataset = DatasetService.get_dataset(dataset_id_str)
         if dataset is None:
             raise NotFound("Dataset not found.")
-        DatasetService.check_dataset_permission(dataset, current_user, db.session())
+        DatasetService.check_dataset_permission(dataset, current_user)
 
         metadata_args = MetadataOperationData.model_validate(console_ns.payload or {})
 

api/controllers/console/datasets/rag_pipeline/datasource_auth.py

@@ -6,7 +6,7 @@ from pydantic import BaseModel, Field
 from werkzeug.exceptions import Forbidden, NotFound
 
 from configs import dify_config
-from controllers.common.fields import SimpleResultResponse
+from controllers.common.fields import RedirectResponse, SimpleResultResponse
 from controllers.common.schema import query_params_from_model, register_response_schema_models, register_schema_models
 from controllers.console import console_ns
 from controllers.console.wraps import (
@@ -19,14 +19,11 @@ from controllers.console.wraps import (
     with_current_tenant_id,
     with_current_user,
 )
-from core.entities.provider_entities import ProviderConfig
 from core.plugin.entities.plugin_daemon import PluginOAuthAuthorizationUrlResponse
 from core.plugin.impl.oauth import OAuthHandler
-from core.tools.entities.common_entities import I18nObject
-from extensions.ext_database import db
 from fields.base import ResponseModel
 from graphon.model_runtime.errors.validate import CredentialsValidateFailedError
-from libs.helper import dump_response
+from graphon.model_runtime.utils.encoders import jsonable_encoder
 from libs.login import login_required
 from models import Account
 from models.provider_ids import DatasourceProviderID
@@ -36,9 +33,7 @@ from services.plugin.oauth_service import OAuthProxyService
 
 class DatasourceCredentialPayload(BaseModel):
     name: str | None = Field(default=None, max_length=100)
-    credentials: dict[str, Any] = Field(
-        description="Plugin-defined credential parameters. The schema is declared by the datasource provider."
-    )
+    credentials: dict[str, Any]
 
 
 class DatasourceCredentialDeletePayload(BaseModel):
@@ -48,17 +43,11 @@ class DatasourceCredentialDeletePayload(BaseModel):
 class DatasourceCredentialUpdatePayload(BaseModel):
     credential_id: str
     name: str | None = Field(default=None, max_length=100)
-    credentials: dict[str, Any] | None = Field(
-        default=None,
-        description="Plugin-defined credential parameters. The schema is declared by the datasource provider.",
-    )
+    credentials: dict[str, Any] | None = Field(default=None)
 
 
 class DatasourceCustomClientPayload(BaseModel):
-    client_params: dict[str, Any] | None = Field(
-        default=None,
-        description="Plugin-defined OAuth client parameters. The schema is declared by the datasource provider.",
-    )
+    client_params: dict[str, Any] | None = Field(default=None)
     enable_oauth_custom_client: bool | None = None
 
 
@@ -82,48 +71,8 @@ class DatasourceOAuthCallbackQuery(BaseModel):
     context_id: str | None = Field(default=None, description="OAuth proxy context ID")
 
 
-class DatasourceCredentialResponse(ResponseModel):
-    credential: dict[str, Any] = Field(
-        description="Obfuscated plugin-defined credential parameters from the datasource provider."
-    )
-    type: str
-    name: str
-    avatar_url: str | None
-    id: str
-    is_default: bool
-
-
-class DatasourceCredentialListResponse(ResponseModel):
-    result: list[DatasourceCredentialResponse]
-
-
-class DatasourceOAuthSchemaResponse(ResponseModel):
-    client_schema: list[ProviderConfig]
-    credentials_schema: list[ProviderConfig]
-    oauth_custom_client_params: dict[str, Any] | None = Field(
-        description="Masked plugin-defined OAuth client parameters, when configured for the tenant."
-    )
-    is_oauth_custom_client_enabled: bool
-    is_system_oauth_params_exists: bool
-    redirect_uri: str
-
-
-class DatasourceProviderAuthResponse(ResponseModel):
-    author: str
-    provider: str
-    plugin_id: str
-    plugin_unique_identifier: str
-    icon: str
-    name: str
-    label: I18nObject
-    description: I18nObject
-    credential_schema: list[ProviderConfig]
-    oauth_schema: DatasourceOAuthSchemaResponse | None
-    credentials_list: list[DatasourceCredentialResponse]
-
-
-class DatasourceProviderAuthListResponse(ResponseModel):
-    result: list[DatasourceProviderAuthResponse]
+class DatasourceCredentialsResponse(ResponseModel):
+    result: Any
 
 
 register_schema_models(
@@ -139,9 +88,9 @@ register_schema_models(
 )
 register_response_schema_models(
     console_ns,
-    DatasourceCredentialListResponse,
-    DatasourceProviderAuthListResponse,
+    DatasourceCredentialsResponse,
     PluginOAuthAuthorizationUrlResponse,
+    RedirectResponse,
     SimpleResultResponse,
 )
 
@@ -151,7 +100,7 @@ class DatasourcePluginOAuthAuthorizationUrl(Resource):
     @console_ns.doc(params=query_params_from_model(DatasourceOAuthAuthorizationQuery))
     @console_ns.response(
         200,
-        "Datasource OAuth authorization URL generated successfully",
+        "Authorization URL retrieved successfully",
         console_ns.models[PluginOAuthAuthorizationUrlResponse.__name__],
     )
     @setup_required
@@ -191,8 +140,7 @@ class DatasourcePluginOAuthAuthorizationUrl(Resource):
             redirect_uri=redirect_uri,
             system_credentials=oauth_config,
         )
-        # response-contract:ignore cookie-bearing Flask response
-        response = make_response(dump_response(PluginOAuthAuthorizationUrlResponse, authorization_url_response))
+        response = make_response(jsonable_encoder(authorization_url_response))
         response.set_cookie(
             "context_id",
             context_id,
@@ -206,8 +154,11 @@ class DatasourcePluginOAuthAuthorizationUrl(Resource):
 @console_ns.route("/oauth/plugin/<path:provider_id>/datasource/callback")
 class DatasourceOAuthCallback(Resource):
     @console_ns.doc(params=query_params_from_model(DatasourceOAuthCallbackQuery))
-    # response-contract:ignore redirect response
-    @console_ns.response(302, "Redirect to OAuth callback page")
+    @console_ns.response(
+        302,
+        "Redirect to console OAuth callback page",
+        console_ns.models[RedirectResponse.__name__],
+    )
     @setup_required
     def get(self, provider_id: str):
         context_id = request.cookies.get("context_id") or request.args.get("context_id")
@@ -266,9 +217,7 @@ class DatasourceOAuthCallback(Resource):
 @console_ns.route("/auth/plugin/datasource/<path:provider_id>")
 class DatasourceAuth(Resource):
     @console_ns.expect(console_ns.models[DatasourceCredentialPayload.__name__])
-    @console_ns.response(
-        200, "Datasource credential created successfully", console_ns.models[SimpleResultResponse.__name__]
-    )
+    @console_ns.response(200, "Success", console_ns.models[SimpleResultResponse.__name__])
     @setup_required
     @login_required
     @account_initialization_required
@@ -289,16 +238,12 @@ class DatasourceAuth(Resource):
             )
         except CredentialsValidateFailedError as ex:
             raise ValueError(str(ex))
-        return SimpleResultResponse(result="success").model_dump(mode="json"), 200
+        return {"result": "success"}, 200
 
-    @console_ns.response(
-        200,
-        "Datasource credentials retrieved successfully",
-        console_ns.models[DatasourceCredentialListResponse.__name__],
-    )
     @setup_required
     @login_required
     @account_initialization_required
+    @console_ns.response(200, "Success", console_ns.models[DatasourceCredentialsResponse.__name__])
     @with_current_user
     @with_current_tenant_id
     def get(self, current_tenant_id: str, user: Account, provider_id: str):
@@ -310,9 +255,8 @@ class DatasourceAuth(Resource):
             provider=datasource_provider_id.provider_name,
             plugin_id=datasource_provider_id.plugin_id,
             user=user,
-            session=db.session(),
         )
-        return dump_response(DatasourceCredentialListResponse, {"result": datasources}), 200
+        return {"result": datasources}, 200
 
 
 @console_ns.route("/auth/plugin/datasource/<path:provider_id>/delete")
@@ -337,17 +281,14 @@ class DatasourceAuthDeleteApi(Resource):
             auth_id=payload.credential_id,
             provider=provider_name,
             plugin_id=plugin_id,
-            session=db.session(),
         )
-        return SimpleResultResponse(result="success").model_dump(mode="json"), 200
+        return {"result": "success"}, 200
 
 
 @console_ns.route("/auth/plugin/datasource/<path:provider_id>/update")
 class DatasourceAuthUpdateApi(Resource):
     @console_ns.expect(console_ns.models[DatasourceCredentialUpdatePayload.__name__])
-    @console_ns.response(
-        201, "Datasource credential updated successfully", console_ns.models[SimpleResultResponse.__name__]
-    )
+    @console_ns.response(201, "Success", console_ns.models[SimpleResultResponse.__name__])
     @setup_required
     @login_required
     @account_initialization_required
@@ -367,53 +308,39 @@ class DatasourceAuthUpdateApi(Resource):
             credentials=payload.credentials or {},
             name=payload.name,
         )
-        return SimpleResultResponse(result="success").model_dump(mode="json"), 201
+        return {"result": "success"}, 201
 
 
 @console_ns.route("/auth/plugin/datasource/list")
 class DatasourceAuthListApi(Resource):
-    @console_ns.response(
-        200,
-        "Datasource credentials retrieved successfully",
-        console_ns.models[DatasourceProviderAuthListResponse.__name__],
-    )
+    @console_ns.response(200, "Success", console_ns.models[DatasourceCredentialsResponse.__name__])
     @setup_required
     @login_required
     @account_initialization_required
     @with_current_tenant_id
     def get(self, current_tenant_id: str):
         datasource_provider_service = DatasourceProviderService()
-        datasources = datasource_provider_service.get_all_datasource_credentials(
-            tenant_id=current_tenant_id, session=db.session()
-        )
-        return dump_response(DatasourceProviderAuthListResponse, {"result": datasources}), 200
+        datasources = datasource_provider_service.get_all_datasource_credentials(tenant_id=current_tenant_id)
+        return {"result": jsonable_encoder(datasources)}, 200
 
 
 @console_ns.route("/auth/plugin/datasource/default-list")
 class DatasourceHardCodeAuthListApi(Resource):
-    @console_ns.response(
-        200,
-        "Default datasource credentials retrieved successfully",
-        console_ns.models[DatasourceProviderAuthListResponse.__name__],
-    )
+    @console_ns.response(200, "Success", console_ns.models[DatasourceCredentialsResponse.__name__])
     @setup_required
     @login_required
     @account_initialization_required
     @with_current_tenant_id
     def get(self, current_tenant_id: str):
         datasource_provider_service = DatasourceProviderService()
-        datasources = datasource_provider_service.get_hard_code_datasource_credentials(
-            tenant_id=current_tenant_id, session=db.session()
-        )
-        return dump_response(DatasourceProviderAuthListResponse, {"result": datasources}), 200
+        datasources = datasource_provider_service.get_hard_code_datasource_credentials(tenant_id=current_tenant_id)
+        return {"result": jsonable_encoder(datasources)}, 200
 
 
 @console_ns.route("/auth/plugin/datasource/<path:provider_id>/custom-client")
 class DatasourceAuthOauthCustomClient(Resource):
     @console_ns.expect(console_ns.models[DatasourceCustomClientPayload.__name__])
-    @console_ns.response(
-        200, "Datasource OAuth custom client saved successfully", console_ns.models[SimpleResultResponse.__name__]
-    )
+    @console_ns.response(200, "Success", console_ns.models[SimpleResultResponse.__name__])
     @setup_required
     @login_required
     @account_initialization_required
@@ -430,7 +357,7 @@ class DatasourceAuthOauthCustomClient(Resource):
             client_params=payload.client_params or {},
             enabled=payload.enable_oauth_custom_client or False,
         )
-        return SimpleResultResponse(result="success").model_dump(mode="json"), 200
+        return {"result": "success"}, 200
 
     @setup_required
     @login_required
@@ -444,7 +371,7 @@ class DatasourceAuthOauthCustomClient(Resource):
             tenant_id=current_tenant_id,
             datasource_provider_id=datasource_provider_id,
         )
-        return SimpleResultResponse(result="success").model_dump(mode="json"), 200
+        return {"result": "success"}, 200
 
 
 @console_ns.route("/auth/plugin/datasource/<path:provider_id>/default")
@@ -466,7 +393,7 @@ class DatasourceAuthDefaultApi(Resource):
             datasource_provider_id=datasource_provider_id,
             credential_id=payload.id,
         )
-        return SimpleResultResponse(result="success").model_dump(mode="json"), 200
+        return {"result": "success"}, 200
 
 
 @console_ns.route("/auth/plugin/datasource/<path:provider_id>/update-name")
@@ -489,4 +416,4 @@ class DatasourceUpdateProviderNameApi(Resource):
             name=payload.name,
             credential_id=payload.credential_id,
         )
-        return SimpleResultResponse(result="success").model_dump(mode="json"), 200
+        return {"result": "success"}, 200

api/controllers/console/datasets/rag_pipeline/datasource_content_preview.py

@@ -3,13 +3,12 @@ from typing import Any
 from flask_restx import (  # type: ignore
     Resource,  # type: ignore
 )
-from pydantic import BaseModel
+from pydantic import BaseModel, RootModel
 
-from controllers.common.schema import register_schema_models
+from controllers.common.schema import register_response_schema_models, register_schema_models
 from controllers.console import console_ns
 from controllers.console.datasets.wraps import get_rag_pipeline
 from controllers.console.wraps import account_initialization_required, setup_required, with_current_user
-from extensions.ext_database import db
 from libs.login import login_required
 from models import Account
 from models.dataset import Pipeline
@@ -22,13 +21,18 @@ class Parser(BaseModel):
     credential_id: str | None = None
 
 
+class DataSourceContentPreviewResponse(RootModel[Any]):
+    root: Any
+
+
 register_schema_models(console_ns, Parser)
+register_response_schema_models(console_ns, DataSourceContentPreviewResponse)
 
 
 @console_ns.route("/rag/pipelines/<uuid:pipeline_id>/workflows/published/datasource/nodes/<string:node_id>/preview")
 class DataSourceContentPreviewApi(Resource):
     @console_ns.expect(console_ns.models[Parser.__name__])
-    @console_ns.response(200, "Success")
+    @console_ns.response(200, "Success", console_ns.models[DataSourceContentPreviewResponse.__name__])
     @setup_required
     @login_required
     @account_initialization_required
@@ -42,7 +46,7 @@ class DataSourceContentPreviewApi(Resource):
 
         inputs = args.inputs
         datasource_type = args.datasource_type
-        rag_pipeline_service = RagPipelineService(db.session())
+        rag_pipeline_service = RagPipelineService()
         preview_content = rag_pipeline_service.run_datasource_node_preview(
             pipeline=pipeline,
             node_id=node_id,

api/controllers/console/datasets/rag_pipeline/rag_pipeline.py

@@ -5,7 +5,7 @@ from flask import request
 from flask_restx import Resource
 from pydantic import BaseModel, Field
 from sqlalchemy import select
-from sqlalchemy.orm import Session, sessionmaker
+from sqlalchemy.orm import sessionmaker
 from werkzeug.exceptions import NotFound
 
 from controllers.common.fields import SimpleDataResponse
@@ -16,7 +16,6 @@ from controllers.common.schema import (
     register_schema_models,
 )
 from controllers.console import console_ns
-from controllers.console.app.wraps import with_session
 from controllers.console.wraps import (
     account_initialization_required,
     enterprise_license_required,
@@ -103,16 +102,10 @@ class PipelineTemplateListApi(Resource):
     @account_initialization_required
     @enterprise_license_required
     @with_current_tenant_id
-    @with_session
-    def get(self, session: Session, current_tenant_id: str) -> JsonResponseWithStatus:
+    def get(self, current_tenant_id: str) -> JsonResponseWithStatus:
         query = PipelineTemplateListQuery.model_validate(request.args.to_dict(flat=True))
         # get pipeline templates
-        pipeline_templates = RagPipelineService.get_pipeline_templates(
-            type=query.type,
-            language=query.language,
-            current_tenant_id=current_tenant_id,
-            session=session,
-        )
+        pipeline_templates = RagPipelineService.get_pipeline_templates(query.type, query.language, current_tenant_id)
         return dump_response(PipelineTemplateListResponse, pipeline_templates), 200
 
 
@@ -124,14 +117,10 @@ class PipelineTemplateDetailApi(Resource):
     @login_required
     @account_initialization_required
     @enterprise_license_required
-    @with_session
-    def get(self, session: Session, template_id: str) -> JsonResponseWithStatus:
+    def get(self, template_id: str) -> JsonResponseWithStatus:
         query = PipelineTemplateDetailQuery.model_validate(request.args.to_dict(flat=True))
-        pipeline_template = RagPipelineService.get_pipeline_template_detail(
-            template_id,
-            type=query.type,
-            session=session,
-        )
+        rag_pipeline_service = RagPipelineService()
+        pipeline_template = rag_pipeline_service.get_pipeline_template_detail(template_id, query.type)
         if pipeline_template is None:
             raise NotFound("Pipeline template not found from upstream service.")
         return dump_response(PipelineTemplateDetailResponse, pipeline_template), 200
@@ -151,7 +140,7 @@ class CustomizedPipelineTemplateApi(Resource):
         payload = CustomizedPipelineTemplatePayload.model_validate(console_ns.payload or {})
         pipeline_template_info = PipelineTemplateInfoEntity.model_validate(payload.model_dump())
         RagPipelineService.update_customized_pipeline_template(
-            template_id, pipeline_template_info, current_user, current_tenant_id, session=db.session()
+            template_id, pipeline_template_info, current_user, current_tenant_id
         )
         return "", 204
 
@@ -162,7 +151,7 @@ class CustomizedPipelineTemplateApi(Resource):
     @enterprise_license_required
     @with_current_tenant_id
     def delete(self, current_tenant_id: str, template_id: str) -> tuple[str, int]:
-        RagPipelineService.delete_customized_pipeline_template(template_id, current_tenant_id, session=db.session())
+        RagPipelineService.delete_customized_pipeline_template(template_id, current_tenant_id)
         return "", 204
 
     @setup_required
@@ -194,8 +183,8 @@ class PublishCustomizedPipelineTemplateApi(Resource):
     @with_current_tenant_id
     def post(self, current_tenant_id: str, current_user: Account, pipeline_id: str) -> tuple[str, int]:
         payload = CustomizedPipelineTemplatePayload.model_validate(console_ns.payload or {})
-        rag_pipeline_service = RagPipelineService(db.session())
+        rag_pipeline_service = RagPipelineService()
         rag_pipeline_service.publish_customized_pipeline_template(
-            pipeline_id, payload.model_dump(), current_user, current_tenant_id, session=db.session()
+            pipeline_id, payload.model_dump(), current_user, current_tenant_id
         )
         return "", 204

api/controllers/console/datasets/rag_pipeline/rag_pipeline_datasets.py

@@ -1,5 +1,6 @@
 from flask_restx import Resource
 from pydantic import BaseModel
+from sqlalchemy.orm import Session
 from werkzeug.exceptions import Forbidden
 
 import services
@@ -65,19 +66,19 @@ class CreateRagPipelineDatasetApi(Resource):
             yaml_content=payload.yaml_content,
         )
         try:
-            rag_pipeline_dsl_service = RagPipelineDslService(db.session())
-            import_info = rag_pipeline_dsl_service.create_rag_pipeline_dataset(
-                tenant_id=current_tenant_id,
-                rag_pipeline_dataset_create_entity=rag_pipeline_dataset_create_entity,
-            )
+            with Session(db.engine, expire_on_commit=False) as session:
+                rag_pipeline_dsl_service = RagPipelineDslService(session)
+                import_info = rag_pipeline_dsl_service.create_rag_pipeline_dataset(
+                    tenant_id=current_tenant_id,
+                    rag_pipeline_dataset_create_entity=rag_pipeline_dataset_create_entity,
+                )
+                session.commit()
             if rag_pipeline_dataset_create_entity.permission == "partial_members":
                 DatasetPermissionService.update_partial_member_list(
                     current_tenant_id,
                     import_info["dataset_id"],
                     rag_pipeline_dataset_create_entity.partial_member_list,
-                    db.session(),
                 )
-            db.session.commit()
         except services.errors.dataset.DatasetNameDuplicateError:
             raise DatasetNameDuplicateError()
 
@@ -110,6 +111,5 @@ class CreateEmptyRagPipelineDatasetApi(Resource):
                 permission=DatasetPermissionEnum.ONLY_ME,
                 partial_member_list=None,
             ),
-            session=db.session(),
         )
         return dump_response(DatasetDetailResponse, dataset), 201