fix(api): scope segment batch deletes

refactor(api): migrate console explore endpoints to BaseModel (#37953 )
fix(api/tasks): use f-string for raise messages in resume_workflow_ex… (#37607 )
2026-07-06 12:47:02 +08:00 · 2026-07-06 02:51:10 +00:00 · 2026-07-06 02:36:14 +00:00 · 2026-07-06 01:03:01 +00:00 · 2026-07-05 14:01:36 +00:00 · 2026-07-05 14:01:21 +00:00
2316 changed files with 112221 additions and 42882 deletions
@@ -5,7 +5,7 @@ description: Write, update, or review Dify end-to-end tests under `e2e/` that us

 # Dify E2E Cucumber + Playwright

-Use this skill for Dify's repository-level E2E suite in `e2e/`. Use [`e2e/AGENTS.md`](../../../e2e/AGENTS.md) as the canonical guide for local architecture and conventions, then apply Playwright/Cucumber best practices only where they fit the current suite.
+Use this skill for Dify's repository-level E2E suite in `e2e/`. Use [`e2e/AGENTS.md`](../../../e2e/AGENTS.md) as the canonical package guide for local architecture and conventions, then read any feature-scoped `AGENTS.md` that owns the target area. Apply Playwright/Cucumber best practices only where they fit the current suite.

 ## Scope

@@ -25,6 +25,8 @@ Use this skill for Dify's repository-level E2E suite in `e2e/`. Use [`e2e/AGENTS
 4. Read [`references/cucumber-best-practices.md`](references/cucumber-best-practices.md) only when scenario wording, step granularity, tags, or expression design are involved.
 5. Re-check official Playwright or Cucumber docs with the available documentation tools before introducing a new framework pattern.

+Keep this skill focused on Cucumber, Playwright, and package-level E2E guidance. Put feature-specific conventions in the owning feature's `AGENTS.md` instead of adding them here.
+
 ## Local Rules

 - `e2e/` uses Cucumber for scenarios and Playwright as the browser layer.
@@ -43,6 +45,7 @@ Use this skill for Dify's repository-level E2E suite in `e2e/`. Use [`e2e/AGENTS
   - Inspect the target feature area.
   - Reuse an existing step when wording and behavior already match.
   - Add a new step only for a genuinely new user action or assertion.
+   - Before adding several similar steps, scan the target capability for an existing domain noun that can be parameterized without hiding behavior.
   - Keep edits close to the current capability folder unless the step is broadly reusable.
 2. Write behavior-first scenarios.
   - Describe user-observable behavior, not DOM mechanics.
@@ -51,15 +54,19 @@ Use this skill for Dify's repository-level E2E suite in `e2e/`. Use [`e2e/AGENTS
 3. Write step definitions in the local style.
   - Keep one step to one user-visible action or one assertion.
   - Prefer Cucumber Expressions such as `{string}` and `{int}`.
+   - Use a bounded regex only when the accepted values are a small explicit domain set and Cucumber Expressions would make the Gherkin less natural.
+   - Do not create one-off steps for each case variant when the same domain action or outcome applies to named surfaces, modes, or resources.
   - Scope locators to stable containers when the page has repeated elements.
   - Avoid page-object layers or extra helper abstractions unless repeated complexity clearly justifies them.
 4. Use Playwright in the local style.
   - Prefer user-facing locators: `getByRole`, `getByLabel`, `getByPlaceholder`, `getByText`, then `getByTestId` for explicit contracts.
   - Use web-first `expect(...)` assertions.
   - Do not use `waitForTimeout`, manual polling, or raw visibility checks when a locator action or retrying assertion already expresses the behavior.
+   - Use `expect.poll` for API persistence, backend eventual consistency, captured browser events, or other non-DOM state; prefer locator assertions for DOM readiness and visible UI state.
+   - If a product element has real user-facing semantics but no accessible name, prefer fixing that accessible contract over adding a test id.
 5. Validate narrowly.
   - Run the narrowest tagged scenario or flow that exercises the change.
-   - Run `pnpm -C e2e check`.
+   - Run `vpr lint --fix --quiet` from the repository root and `pnpm -C e2e type-check`.
   - Broaden verification only when the change affects hooks, tags, setup, or shared step semantics.

 ## Review Checklist
@@ -39,6 +39,7 @@ Prefer reuse when:
 - the user action is genuinely the same
 - the expected outcome is genuinely the same
 - the wording stays natural across features
+- the parameter is a real product domain value such as a named surface, mode, resource, or status

 Write a new step when:

@@ -46,6 +47,8 @@ Write a new step when:
 - reusing the old wording would make the scenario misleading
 - a supposedly generic step would become an implementation-detail wrapper

+Do not optimize for a low step count by making vague steps. Optimize for a small set of truthful, domain-owned steps.
+
 ### 4. Prefer Cucumber Expressions

 Use Cucumber Expressions for parameters unless regex is clearly necessary.
@@ -59,6 +62,8 @@ Common examples:

 Keep expressions readable. If a step needs complicated parsing logic, first ask whether the scenario wording should be simpler.

+Use regex for a bounded natural-language alternative only when it keeps Gherkin readable, for example `/(Web app|Backend service API)/`. Avoid broad regexes that accept unowned language.
+
 ### 5. Keep step definitions thin and meaningful

 Step definitions are glue between Gherkin and automation, not a second abstraction language.
@@ -41,6 +41,7 @@ Also remember:
 - repeated content usually needs scoping to a stable container
 - exact text matching is often too brittle when role/name or label already exists
 - `getByTestId` is acceptable when semantics are weak but the contract is intentional
+- when a real UI region, card, status, or icon lacks an accessible name, prefer adding that semantic contract in product code before falling back to `getByTestId`

 ### 3. Use web-first assertions

@@ -62,6 +63,8 @@ Avoid:

 If a condition genuinely needs custom retry logic, use Playwright's polling/assertion tools deliberately and keep that choice local and explicit.

+Use `expect.poll` for non-DOM truth such as API state, backend eventual consistency, generated resources, or captured browser events. For DOM state, use locator assertions so Playwright can apply actionability and web-first retry semantics.
+
 ### 4. Let actions wait for actionability

 Locator actions already wait for the element to be actionable. Do not preface every click/fill with extra timing logic unless the action needs a specific visible/ready assertion for clarity.
@@ -40,9 +40,13 @@ Flag:
 - JavaScript conditional class logic for visual states that the Dify UI/Base UI primitive already exposes through `data-*` attributes or CSS variables.
 - Controlled props added when uncontrolled DOM state or CSS variables would be enough.
 - Thin wrappers that rename Base UI parts without adding semantics.
+- Generic Base UI selection primitives wrapped without preserving their value generics, such as `Select.Root<Value, Multiple>`, `RadioGroup<Value>`, or `Radio.Root<Value>`.
+- Shared select/radio option components that type selected values as `string` while callers pass enums, unions, booleans, numbers, objects, or nullable placeholder values.

 Prefer Base UI/Dify UI data attributes and CSS variables for visual state: `data-open`, `data-checked`, `data-disabled`, `data-highlighted`, `data-popup-open`, `group-data-*`, `peer-data-*`, `has-[:focus-visible]`, and primitive CSS variables such as anchor width or transform origin. Use JS conditional classes for product/business state that the primitive does not expose.

+For non-string `Select` and `RadioGroup` values, prefer explicit domain generics at the root and at child value carriers. JSX children do not inherit the parent generic, so `RadioGroup<PromptMode>` should compose with `Radio<PromptMode>`, `RadioRoot<PromptMode>`, or option values from a typed collection. For `Select`, prefer the Base UI `items` collection pattern for typed value-to-label rendering, and flag string coercion helpers used only to recover display labels.
+
 ## Forms

 Flag:
@@ -12,6 +12,7 @@ Use this as the component decision guide for Dify web. Existing code is referenc
 | Question | Default | Promote or extract only when |
 | --- | --- | --- |
 | Where should code live? | Keep it local to the feature workflow, route, or owner. | Multiple verticals need the same stable primitive. |
+| How should route/tab folders be named? | Match the current route segment, tab name, or user-visible surface. | Keep a historical or broader parent only when it still owns multiple surfaces. |
 | Who owns state, data, and handlers? | The lowest component that uses them. | A parent coordinates shared loading, errors, empty UI, selection, submission, navigation, or one consistent snapshot. |
 | Should this become Jotai state? | Keep synchronous UI/form state in component or DOM state. | Siblings need one source of truth, the value drives atoms, or scoped workflow state must survive hidden/unmounted steps. |
 | Should URL state enter Jotai? | Let Next.js route params and `nuqs` own URL state and updates. | Query atoms or shared derived atoms need a read-only bridge hydrated at the route/surface boundary. |
@@ -23,7 +24,9 @@ Use this as the component decision guide for Dify web. Existing code is referenc

 - Search before adding UI, hooks, helpers, query utilities, or styling patterns. Reuse existing base components, feature components, hooks, utilities, and design styles when they fit.
 - Follow Dify's CSS-first Tailwind v4 contract from `packages/dify-ui/README.md` and `packages/dify-ui/AGENTS.md`. Prefer design-system tokens, utilities, and radius mappings over generic Tailwind choices.
- Group feature code by workflow, route, or ownership area: components, hooks, local types, query helpers, atoms, constants, and small utilities should live near the code that changes with them.
+- Group feature code by workflow, route, or ownership area with route-aligned names: components, hooks, local types, query helpers, atoms, constants, tests, and small utilities should live near the code that changes with them.
+- For each feature module, keep a module-local `README.md` as a boundary note. Start with the module name, a brief one-sentence description, then split dependencies into `Internal Modules` and `External Modules` sections; keep both sections and write `None.` when one category is empty. `Internal Modules` lists modules inside the same overall feature using paths from that feature root, such as `shared/domain/runtime-status`; `External Modules` lists project modules outside the feature using paths from the web root without a `web/` prefix, such as `app/components/base/skeleton`. Omit npm packages, workspace package dependencies, and whitelisted plumbing modules. Do not copy caller-relative import paths into the README.
+- Module README whitelist: `@/service/client`, `@/next/*`.
 - Keep source/default selection, validation, dirty checks, and payload shaping close to the workflow that owns submit behavior. Do not hide flow-specific priority order, fallback behavior, or submit semantics in generic utilities.
 - Prefer direct conditionals for small branch-specific decisions, especially form source selection and request payload assembly.
 - Loading states for page sections, cards, lists, tables, forms, and drawers should be skeletons scoped to the content being loaded. Use spinners only for small inline busy indicators.
@@ -32,6 +35,8 @@ Use this as the component decision guide for Dify web. Existing code is referenc

 - State-heavy wizards, drawers, modals, and secondary workflows can be a small feature surface: an entry file, one feature-local state file when Jotai is actually needed, and shallow `ui/` owners that match real visual regions.
 - The entry file handles route integration, provider wiring, close behavior, and surface mounting. The composition owner handles high-level workflow branching. The closest visual owner handles section branching.
+- When a page or tab maps to a route segment, name its feature folder after that route/tab surface instead of a stale parent grouping. Remove misleading intermediate folders when only one surface remains.
+- When a tab folder grows into several independent sections or action areas, split the first level by product/visual owners. Keep the root for the entry component and cross-owner state, colocate tests with the owner folder, and put truly shared local UI under a specifically named `components/` file.
 - Repeated TanStack query calls in sibling components are acceptable when each component independently consumes the data; TanStack Query deduplicates and shares cache.
 - Pass stable domain identity across boundaries. Do not forward derived presentation state when the receiver can derive it from its own data source.
 - A component that owns a visual surface should also own data access, loading, empty, and error states for content rendered inside it unless a parent truly coordinates that state.
@@ -46,6 +51,7 @@ Use this as the component decision guide for Dify web. Existing code is referenc
 - Use uncontrolled `@langgenius/dify-ui/form` and `@langgenius/dify-ui/field` controls for edit/create forms whose fields are read only at submit time. Initialize query-backed defaults with `defaultValue` and keyed remounts.
 - Promote form state to atoms only when another component must react to in-progress values, a draft must survive unmount/remount in the scoped workflow, or multiple steps share the same editable draft before submit.
 - Treat `useParams`, route args, and `nuqs` query state as framework-owned state. When atom logic needs those values, hydrate primitive atoms at the route or surface boundary, such as with `useHydrateAtoms(..., { dangerouslyForceHydrate: true })`; keep URL updates in the route/query-state APIs instead of write atoms.
+- Within a route-owned feature, choose one source for route identity. If route params are bridged into feature atoms, use that bridge consistently for route-derived queries and actions instead of also threading the same route id through page, tab, and section props.
 - For async work tied to atom state, use `atomWithQuery` or `atomWithMutation`; write atoms should update only the inputs that drive those atoms. This applies to pure frontend async work as well as network requests, so do not hand-roll loading/error/in-flight state with `useState` or `useRef` for atom-orchestrated async behavior. For component-owned remote work, use `useQuery` or `useMutation` directly.
 - Row-local async state belongs to the row owner unless it participates in a shared Jotai workflow or needs atom-scoped reset semantics.
 - Leave query and mutation atoms unscoped so they keep shared QueryClient cache and invalidation behavior. Scope resettable primitives and explicit hydration tuples; scope a derived atom only when every dependency should be private to that surface.
@@ -60,8 +66,11 @@ Use this as the component decision guide for Dify web. Existing code is referenc
 - Type component signatures directly; do not use `FC` or `React.FC`.
 - Prefer `function` for top-level components and module helpers. Use arrow functions for local callbacks, handlers, and lambda-style APIs.
 - Prefer named exports. Use default exports only where the framework requires them, such as Next.js route files.
+- Avoid barrel files that only re-export secondary owners. `index.tsx` is acceptable for a route/tab entry component; import header controls, switches, sections, and row owners from their concrete owner files.
 - Type simple one-off props inline. Use a named `Props` type only when reused, exported, complex, or clearer.
 - Use API-generated or API-returned types at component boundaries. Keep small UI conversion helpers and one-off UI extensions beside the component that needs them.
+- Preserve domain value types for selection components. Do not widen enum, union, boolean, numeric, object, or nullable select/radio values to `string`; keep wrappers and option value carriers typed from their feature option collection.
+- Avoid `common.tsx` buckets for shared UI. Use a feature-local `components/` folder with concrete filenames that describe the shared role.
 - Do not create type aliases that only rename another type. Use aliases only for real UI concepts, refinements, or reusable local contracts.
 - Name values by their domain role and backend API contract, especially persistent IDs and route params. Normalize framework or route params at the boundary.
 - Put fallback and invariant checks in the lowest component that already handles that state. Do not extract helpers whose only behavior is hiding missing display data.
@@ -83,11 +92,13 @@ Use this as the component decision guide for Dify web. Existing code is referenc

 - Keep `web/contract/*` as the API shape source of truth and follow the `{ params, query?, body? }` input shape.
 - Consume generated queries with `useQuery(consoleQuery.xxx.queryOptions(...))` or `useQuery(marketplaceQuery.xxx.queryOptions(...))`.
+- If a generated query input comes from an atom, including a route-identity bridge atom, keep the query in `atomWithQuery`; do not unwrap the atom in a component just to call `useQuery`.
 - Consume owner-local mutations with `useMutation(consoleQuery.xxx.mutationOptions(...))` or `useMutation(marketplaceQuery.xxx.mutationOptions(...))` when pending/error state is not consumed by feature atoms.
 - In `atomWithQuery`, `atomWithInfiniteQuery`, and `atomWithMutation`, return generated `queryOptions()`, `infiniteOptions()`, or `mutationOptions()` directly. Pass `enabled`, `retry`, `placeholderData`, `select`, and pagination options into the generated call instead of spreading options into a hand-built object.
 - For generated oRPC options with missing required input, branch the whole input with `input: condition ? validInput : skipToken` and `enabled: Boolean(condition)`. Never place `skipToken` inside a nested placeholder payload or coerce required IDs to `''`.
 - When prefetch and render use the same request, extract local query options or a query-options atom so `prefetchQuery` and `useQuery`/`atomWithQuery` share the exact options.
 - For custom query or mutation functions, wrap options with TanStack `queryOptions(...)` or `mutationOptions(...)`.
+- Do not extract generated `queryOptions(...)` into a helper solely to share input construction; extract only when prefetch/render must share exact options or the helper owns real domain behavior.
 - Avoid pass-through hooks and thin `web/service/use-*` wrappers that only rename generated options. Keep feature hooks for real orchestration, workflow state, or shared domain behavior.
 - Put shared cache behavior in `createTanstackQueryUtils(...experimental_defaults...)`. Component or atom callbacks may handle local toasts, closing dialogs, and navigation, but should not replace shared invalidation or patch shared server state locally.
 - For overlays that may open heavier secondary content, prefetch from the trigger/menu open event with `queryClient.prefetchQuery(queryOptions)` when `onOpenChange` is available. Do not mount hidden subscribers just to warm cache.
@@ -96,7 +107,7 @@ Use this as the component decision guide for Dify web. Existing code is referenc

 ## Boundaries And Overlays

- Use the first level below a page or tab to organize independent page sections when it adds structure. This layer is layout/semantic first, not automatically the data owner.
+- Use the first level below a page or tab to organize independent page sections when it adds structure or the root folder becomes noisy. This layer is layout/semantic first, not automatically the data owner.
 - Treat component names, semantic roles, and user- or design-marked visual regions as boundary constraints. Keep adjacent UI as a sibling owner or introduce a correctly named broader owner.
 - Keep cohesive forms, menu bodies, and one-off helpers local unless they need their own state, reuse, or semantic boundary.
 - Separate hidden secondary surfaces from the trigger's main flow. For dialogs, dropdowns, popovers, and similar branches, extract a small local component when hidden content would obscure the parent.
@@ -26,6 +26,9 @@
 /cli/ @GareArc
 /.github/workflows/cli-tests.yml @GareArc

+# E2E
+/e2e/ @lyzno1
+
 # Backend (default owner, more specific rules below will override)
 /api/ @QuantumGhost

@@ -7,3 +7,9 @@ web:
          - 'pnpm-lock.yaml'
          - 'pnpm-workspace.yaml'
          - '.nvmrc'
+
+e2e:
+  - changed-files:
+      - any-glob-to-any-file:
+          - 'e2e/**'
+          - '.github/workflows/web-e2e.yml'
@@ -16,7 +16,7 @@ concurrency:
 jobs:
  api-unit:
    name: API Unit Tests
-    runs-on: depot-ubuntu-24.04-4
+    runs-on: depot-ubuntu-24.04
    env:
      COVERAGE_FILE: coverage-unit
    defaults:
@@ -75,7 +75,7 @@ jobs:

  api-integration:
    name: API Integration Tests
-    runs-on: depot-ubuntu-24.04-4
+    runs-on: depot-ubuntu-24.04
    env:
      COVERAGE_FILE: coverage-integration
      STORAGE_TYPE: opendal
@@ -129,7 +129,7 @@ jobs:

  api-coverage:
    name: API Coverage
-    runs-on: depot-ubuntu-24.04-4
+    runs-on: depot-ubuntu-24.04
    needs:
      - api-unit
      - api-integration
@@ -51,6 +51,18 @@ jobs:
        with:
          files: |
            api/**
+      - name: Check frontend contract inputs
+        if: github.event_name != 'merge_group'
+        id: frontend-contract-changes
+        uses: tj-actions/changed-files@9426d40962ed5378910ee2e21d5f8c6fcbf2dd96 # v47.0.6
+        with:
+          files: |
+            api/**
+            packages/contracts/openapi-ts.api.config.ts
+            packages/contracts/package.json
+            packages/contracts/openapi/**
+            pnpm-lock.yaml
+            pnpm-workspace.yaml
      - name: Check dify-agent inputs
        if: github.event_name != 'merge_group'
        id: dify-agent-changes
@@ -143,8 +155,8 @@ jobs:
          uv run dev/generate_swagger_markdown_docs.py --swagger-dir ../packages/contracts/openapi --markdown-dir openapi/markdown --keep-swagger-json

      - name: Generate frontend contracts
-        if: github.event_name != 'merge_group' && steps.api-changes.outputs.any_changed == 'true'
-        run: pnpm --dir packages/contracts gen-api-contract-from-openapi
+        if: github.event_name != 'merge_group' && steps.frontend-contract-changes.outputs.any_changed == 'true'
+        run: pnpm --dir packages/contracts gen-api-contract

      - name: ESLint autofix
        if: github.event_name != 'merge_group' && steps.web-changes.outputs.any_changed == 'true'
@@ -171,7 +171,7 @@ jobs:

  create-manifest:
    needs: build
-    runs-on: depot-ubuntu-24.04-4
+    runs-on: depot-ubuntu-24.04
    if: github.repository == 'langgenius/dify'
    strategy:
      matrix:
@@ -23,7 +23,7 @@ concurrency:
 jobs:
  validate:
    name: validate manifest + resolve target Dify release
-    runs-on: depot-ubuntu-24.04-4
+    runs-on: depot-ubuntu-24.04
    if: github.repository == 'langgenius/dify'
    permissions:
      contents: read
@@ -87,7 +87,7 @@ jobs:
  release:
    name: build + attach standalone binaries (all targets)
    needs: validate
-    runs-on: depot-ubuntu-24.04-4
+    runs-on: depot-ubuntu-24.04
    permissions:
      contents: write
    defaults:
@@ -9,7 +9,7 @@ concurrency:

 jobs:
  db-migration-test-postgres:
-    runs-on: depot-ubuntu-24.04-4
+    runs-on: depot-ubuntu-24.04

    steps:
      - name: Checkout code
@@ -59,7 +59,7 @@ jobs:
        run: uv run --directory api flask upgrade-db

  db-migration-test-mysql:
-    runs-on: depot-ubuntu-24.04-4
+    runs-on: depot-ubuntu-24.04

    steps:
      - name: Checkout code
@@ -13,7 +13,7 @@ on:

 jobs:
  deploy:
-    runs-on: depot-ubuntu-24.04-4
+    runs-on: depot-ubuntu-24.04
    if: |
      github.event.workflow_run.conclusion == 'success' &&
      github.event.workflow_run.head_branch == 'deploy/agent'
@@ -10,7 +10,7 @@ on:

 jobs:
  deploy:
-    runs-on: depot-ubuntu-24.04-4
+    runs-on: depot-ubuntu-24.04
    if: |
      github.event.workflow_run.conclusion == 'success' &&
      github.event.workflow_run.head_branch == 'deploy/dev'
@@ -13,7 +13,7 @@ on:

 jobs:
  deploy:
-    runs-on: depot-ubuntu-24.04-4
+    runs-on: depot-ubuntu-24.04
    if: |
      github.event.workflow_run.conclusion == 'success' &&
      github.event.workflow_run.head_branch == 'deploy/enterprise'
@@ -13,7 +13,7 @@ on:

 jobs:
  deploy:
-    runs-on: depot-ubuntu-24.04-4
+    runs-on: depot-ubuntu-24.04
    if: |
      github.event.workflow_run.conclusion == 'success' &&
      github.event.workflow_run.head_branch == 'deploy/saas'
@@ -22,7 +22,7 @@ concurrency:
 jobs:
  check-cherry-pick-provenance:
    name: Require cherry-pick provenance
-    runs-on: depot-ubuntu-24.04-4
+    runs-on: depot-ubuntu-24.04
    steps:
      - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
        with:
@@ -7,7 +7,7 @@ jobs:
    permissions:
      contents: read
      pull-requests: write
-    runs-on: depot-ubuntu-24.04-4
+    runs-on: depot-ubuntu-24.04
    steps:
      - uses: actions/labeler@f27b608878404679385c85cfa523b85ccb86e213 # v6.1.0
        with:
@@ -6,8 +6,6 @@ on:
  merge_group:
    branches: ["main"]
    types: [checks_requested]
-  push:
-    branches: ["main"]

 permissions:
  actions: write
@@ -23,7 +21,7 @@ concurrency:
 jobs:
  pre_job:
    name: Skip Duplicate Checks
-    runs-on: depot-ubuntu-24.04-4
+    runs-on: depot-ubuntu-24.04
    outputs:
      should_skip: ${{ steps.skip_check.outputs.should_skip || 'false' }}
    steps:
@@ -39,7 +37,7 @@ jobs:
    name: Check Changed Files
    needs: pre_job
    if: needs.pre_job.outputs.should_skip != 'true'
-    runs-on: depot-ubuntu-24.04-4
+    runs-on: depot-ubuntu-24.04
    outputs:
      api-changed: ${{ steps.changes.outputs.api }}
      cli-changed: ${{ steps.changes.outputs.cli }}
@@ -152,7 +150,7 @@ jobs:
      - pre_job
      - check-changes
    if: needs.pre_job.outputs.should_skip != 'true' && needs.check-changes.outputs.api-changed != 'true'
-    runs-on: depot-ubuntu-24.04-4
+    runs-on: depot-ubuntu-24.04
    steps:
      - name: Report skipped API tests
        run: echo "No API-related changes detected; skipping API tests."
@@ -165,7 +163,7 @@ jobs:
      - check-changes
      - api-tests-run
      - api-tests-skip
-    runs-on: depot-ubuntu-24.04-4
+    runs-on: depot-ubuntu-24.04
    steps:
      - name: Finalize API Tests status
        env:
@@ -212,7 +210,7 @@ jobs:
      - pre_job
      - check-changes
    if: needs.pre_job.outputs.should_skip != 'true' && needs.check-changes.outputs.cli-changed != 'true'
-    runs-on: depot-ubuntu-24.04-4
+    runs-on: depot-ubuntu-24.04
    steps:
      - name: Report skipped CLI tests
        run: echo "No CLI-related changes detected; skipping CLI tests."
@@ -225,7 +223,7 @@ jobs:
      - check-changes
      - cli-tests-run
      - cli-tests-skip
-    runs-on: depot-ubuntu-24.04-4
+    runs-on: depot-ubuntu-24.04
    steps:
      - name: Finalize CLI Tests status
        env:
@@ -272,7 +270,7 @@ jobs:
      - pre_job
      - check-changes
    if: needs.pre_job.outputs.should_skip != 'true' && needs.check-changes.outputs.web-changed != 'true'
-    runs-on: depot-ubuntu-24.04-4
+    runs-on: depot-ubuntu-24.04
    steps:
      - name: Report skipped web tests
        run: echo "No web-related changes detected; skipping web tests."
@@ -285,7 +283,7 @@ jobs:
      - check-changes
      - web-tests-run
      - web-tests-skip
-    runs-on: depot-ubuntu-24.04-4
+    runs-on: depot-ubuntu-24.04
    steps:
      - name: Finalize Web Tests status
        env:
@@ -324,6 +322,7 @@ jobs:
      - check-changes
    if: needs.pre_job.outputs.should_skip != 'true' && needs.check-changes.outputs.e2e-changed == 'true'
    uses: ./.github/workflows/web-e2e.yml
+    secrets: inherit

  web-e2e-skip:
    name: Skip Web Full-Stack E2E
@@ -331,7 +330,7 @@ jobs:
      - pre_job
      - check-changes
    if: needs.pre_job.outputs.should_skip != 'true' && needs.check-changes.outputs.e2e-changed != 'true'
-    runs-on: depot-ubuntu-24.04-4
+    runs-on: depot-ubuntu-24.04
    steps:
      - name: Report skipped web full-stack e2e
        run: echo "No E2E-related changes detected; skipping web full-stack E2E."
@@ -344,7 +343,7 @@ jobs:
      - check-changes
      - web-e2e-run
      - web-e2e-skip
-    runs-on: depot-ubuntu-24.04-4
+    runs-on: depot-ubuntu-24.04
    steps:
      - name: Finalize Web Full-Stack E2E status
        env:
@@ -396,7 +395,7 @@ jobs:
      - pre_job
      - check-changes
    if: needs.pre_job.outputs.should_skip != 'true' && needs.check-changes.outputs.vdb-changed != 'true'
-    runs-on: depot-ubuntu-24.04-4
+    runs-on: depot-ubuntu-24.04
    steps:
      - name: Report skipped VDB tests
        run: echo "No VDB-related changes detected; skipping VDB tests."
@@ -409,7 +408,7 @@ jobs:
      - check-changes
      - vdb-tests-run
      - vdb-tests-skip
-    runs-on: depot-ubuntu-24.04-4
+    runs-on: depot-ubuntu-24.04
    steps:
      - name: Finalize VDB Tests status
        env:
@@ -455,7 +454,7 @@ jobs:
      - pre_job
      - check-changes
    if: needs.pre_job.outputs.should_skip != 'true' && needs.check-changes.outputs.migration-changed != 'true'
-    runs-on: depot-ubuntu-24.04-4
+    runs-on: depot-ubuntu-24.04
    steps:
      - name: Report skipped DB migration tests
        run: echo "No migration-related changes detected; skipping DB migration tests."
@@ -468,7 +467,7 @@ jobs:
      - check-changes
      - db-migration-test-run
      - db-migration-test-skip
-    runs-on: depot-ubuntu-24.04-4
+    runs-on: depot-ubuntu-24.04
    steps:
      - name: Finalize DB Migration Test status
        env:
@@ -0,0 +1,54 @@
+name: Post-Merge Checks
+
+on:
+  push:
+    branches: ["main"]
+
+permissions:
+  contents: read
+
+concurrency:
+  group: post-merge-${{ github.sha }}
+  cancel-in-progress: false
+
+jobs:
+  check-changes:
+    name: Check Changed Files
+    runs-on: depot-ubuntu-24.04
+    outputs:
+      external-e2e-changed: ${{ steps.changes.outputs.external_e2e }}
+    steps:
+      - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
+      - uses: dorny/paths-filter@fbd0ab8f3e69293af611ebaee6363fc25e6d187d # v4.0.1
+        id: changes
+        with:
+          filters: |
+            external_e2e:
+              - 'e2e/features/agent-v2/**'
+              - 'e2e/scripts/**'
+              - 'e2e/support/**'
+              - '.github/workflows/post-merge.yml'
+              - '.github/workflows/web-e2e.yml'
+              - '.github/actions/setup-web/**'
+              - 'dify-agent/**'
+              - 'api/clients/agent_backend/**'
+              - 'api/core/app/apps/agent_app/**'
+              - 'api/core/workflow/nodes/agent_v2/**'
+              - 'api/controllers/console/agent/**'
+              - 'api/services/agent/**'
+              - 'api/core/plugin/**'
+              - 'api/services/plugin/**'
+              - 'api/core/tools/**'
+              - 'api/services/tools/**'
+              - 'web/features/agent-v2/**'
+              - 'web/app/(commonLayout)/roster/**'
+              - 'web/app/components/workflow/nodes/agent-v2/**'
+
+  external-e2e:
+    name: External Runtime E2E
+    needs: check-changes
+    if: needs.check-changes.outputs.external-e2e-changed == 'true'
+    uses: ./.github/workflows/web-e2e.yml
+    with:
+      run-external-runtime: true
+    secrets: inherit
@@ -12,7 +12,7 @@ permissions: {}
 jobs:
  comment:
    name: Comment PR with pyrefly diff
-    runs-on: depot-ubuntu-24.04-4
+    runs-on: depot-ubuntu-24.04
    permissions:
      actions: read
      contents: read
@@ -10,7 +10,7 @@ permissions:

 jobs:
  pyrefly-diff:
-    runs-on: depot-ubuntu-24.04-4
+    runs-on: depot-ubuntu-24.04
    permissions:
      contents: read
      issues: write
@@ -12,7 +12,7 @@ permissions: {}
 jobs:
  comment:
    name: Comment PR with type coverage
-    runs-on: depot-ubuntu-24.04-4
+    runs-on: depot-ubuntu-24.04
    permissions:
      actions: read
      contents: read
@@ -10,7 +10,7 @@ permissions:

 jobs:
  pyrefly-type-coverage:
-    runs-on: depot-ubuntu-24.04-4
+    runs-on: depot-ubuntu-24.04
    permissions:
      contents: read
      issues: write
@@ -16,7 +16,7 @@ jobs:
    name: Validate PR title
    permissions:
      pull-requests: read
-    runs-on: depot-ubuntu-24.04-4
+    runs-on: depot-ubuntu-24.04
    steps:
      - name: Complete merge group check
        if: github.event_name == 'merge_group'
@@ -12,7 +12,7 @@ on:
 jobs:
  stale:

-    runs-on: depot-ubuntu-24.04-4
+    runs-on: depot-ubuntu-24.04
    permissions:
      issues: write
      pull-requests: write
@@ -15,13 +15,14 @@ permissions:
 jobs:
  python-style:
    name: Python Style
-    runs-on: depot-ubuntu-24.04-4
+    runs-on: depot-ubuntu-24.04

    steps:
      - name: Checkout code
        uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
        with:
          persist-credentials: false
+          fetch-depth: 0

      - name: Check changed files
        id: changed-files
@@ -29,6 +30,8 @@ jobs:
        with:
          files: |
            api/**
+            scripts/check_no_new_getattr.py
+            scripts/ast_grep_rules/no_new_getattr.yml
            .github/workflows/style.yml

      - name: Setup UV and Python
@@ -51,8 +54,22 @@ jobs:
        if: steps.changed-files.outputs.any_changed == 'true'
        run: uv run --project api --dev python api/dev/lint_response_contracts.py --fail-on-mismatch

+      - name: Fetch merge target ref for getattr guard
+        if: steps.changed-files.outputs.any_changed == 'true'
+        run: git fetch --no-tags --depth=1 origin +refs/heads/main:refs/remotes/origin/main
+
+      - name: Bind merge target branch for getattr guard
+        if: steps.changed-files.outputs.any_changed == 'true'
+        run: git show-ref --verify --quiet refs/heads/main || git branch main origin/main
+
+      - name: Run No New Getattr Guard
+        if: steps.changed-files.outputs.any_changed == 'true'
+        run: uv run --project api python scripts/check_no_new_getattr.py --mode ci --merge-target main
+
      - name: Run Type Checks
        if: steps.changed-files.outputs.any_changed == 'true'
+        env:
+          PYREFLY_OUTPUT_FORMAT: github
        run: make type-check-core

      - name: Dotenv check
@@ -61,7 +78,7 @@ jobs:

  web-style:
    name: Web Style
-    runs-on: depot-ubuntu-24.04-4
+    runs-on: depot-ubuntu-24.04
    defaults:
      run:
        working-directory: ./web
@@ -175,7 +192,7 @@ jobs:

  superlinter:
    name: SuperLinter
-    runs-on: depot-ubuntu-24.04-4
+    runs-on: depot-ubuntu-24.04

    steps:
      - name: Checkout code
@@ -17,7 +17,7 @@ concurrency:
 jobs:
  build:
    name: unit test for Node.js SDK
-    runs-on: depot-ubuntu-24.04-4
+    runs-on: depot-ubuntu-24.04

    defaults:
      run:
@@ -35,7 +35,7 @@ concurrency:
 jobs:
  translate:
    if: github.repository == 'langgenius/dify'
-    runs-on: depot-ubuntu-24.04-4
+    runs-on: depot-ubuntu-24.04
    timeout-minutes: 120

    steps:
@@ -16,7 +16,7 @@ concurrency:
 jobs:
  trigger:
    if: github.repository == 'langgenius/dify'
-    runs-on: depot-ubuntu-24.04-4
+    runs-on: depot-ubuntu-24.04
    timeout-minutes: 5

    steps:
@@ -16,7 +16,7 @@ jobs:
  test:
    name: Full VDB Tests
    if: github.repository == 'langgenius/dify'
-    runs-on: depot-ubuntu-24.04-4
+    runs-on: depot-ubuntu-24.04
    strategy:
      matrix:
        python-version:
@@ -13,7 +13,7 @@ concurrency:
 jobs:
  test:
    name: VDB Smoke Tests
-    runs-on: depot-ubuntu-24.04-4
+    runs-on: depot-ubuntu-24.04
    strategy:
      matrix:
        python-version:
@@ -2,6 +2,11 @@ name: Web Full-Stack E2E

 on:
  workflow_call:
+    inputs:
+      run-external-runtime:
+        required: false
+        type: boolean
+        default: false

 permissions:
  contents: read
@@ -32,7 +37,9 @@ jobs:
        with:
          enable-cache: true
          python-version: "3.12"
-          cache-dependency-glob: api/uv.lock
+          cache-dependency-glob: |
+            api/uv.lock
+            dify-agent/uv.lock

      - name: Install API dependencies
        run: uv sync --project api --dev
@@ -51,12 +58,52 @@ jobs:
          E2E_INIT_PASSWORD: E2eInit12345
        run: vp run e2e:full

+      - name: Run external runtime E2E tests
+        if: ${{ inputs.run-external-runtime }}
+        working-directory: ./e2e
+        env:
+          E2E_ADMIN_EMAIL: e2e-admin@example.com
+          E2E_ADMIN_NAME: E2E Admin
+          E2E_ADMIN_PASSWORD: E2eAdmin12345
+          E2E_AGENT_DECISION_MODEL_NAME: ${{ vars.E2E_AGENT_DECISION_MODEL_NAME || 'gpt-5.5' }}
+          E2E_AGENT_DECISION_MODEL_PROVIDER: ${{ vars.E2E_AGENT_DECISION_MODEL_PROVIDER || 'openai' }}
+          E2E_AGENT_DECISION_MODEL_TYPE: ${{ vars.E2E_AGENT_DECISION_MODEL_TYPE || 'llm' }}
+          E2E_EXTERNAL_RUNTIME_SEED_SPECS: ${{ vars.E2E_EXTERNAL_RUNTIME_SEED_SPECS }}
+          E2E_EXTERNAL_RUNTIME_TAGS: ${{ vars.E2E_EXTERNAL_RUNTIME_TAGS }}
+          E2E_FORCE_WEB_BUILD: "1"
+          E2E_INIT_PASSWORD: E2eInit12345
+          E2E_MARKETPLACE_API_URL: ${{ vars.E2E_MARKETPLACE_API_URL }}
+          E2E_MARKETPLACE_PLUGIN_IDS: ${{ vars.E2E_MARKETPLACE_PLUGIN_IDS }}
+          E2E_MARKETPLACE_PLUGIN_UNIQUE_IDENTIFIERS: ${{ vars.E2E_MARKETPLACE_PLUGIN_UNIQUE_IDENTIFIERS }}
+          E2E_MODEL_PROVIDER_CREDENTIALS_JSON: ${{ secrets.E2E_MODEL_PROVIDER_CREDENTIALS_JSON }}
+          E2E_START_AGENT_BACKEND: ${{ vars.E2E_START_AGENT_BACKEND || '1' }}
+          E2E_STABLE_MODEL_NAME: ${{ vars.E2E_STABLE_MODEL_NAME || 'gpt-5-nano' }}
+          E2E_STABLE_MODEL_PROVIDER: ${{ vars.E2E_STABLE_MODEL_PROVIDER || 'openai' }}
+          E2E_STABLE_MODEL_TYPE: ${{ vars.E2E_STABLE_MODEL_TYPE || 'llm' }}
+        run: |
+          if [[ -z "${E2E_MODEL_PROVIDER_CREDENTIALS_JSON}" ]]; then
+            echo "E2E_MODEL_PROVIDER_CREDENTIALS_JSON is required for external runtime E2E." >&2
+            exit 1
+          fi
+
+          if [[ -d cucumber-report ]]; then
+            rm -rf cucumber-report-non-external
+            mv cucumber-report cucumber-report-non-external
+          fi
+
+          trap 'vp run e2e:middleware:down' EXIT
+          vp run e2e:middleware:up
+          vp run e2e:external:prepare
+          vp run e2e:external
+
      - name: Upload Cucumber report
        if: ${{ !cancelled() }}
        uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
        with:
          name: cucumber-report
-          path: e2e/cucumber-report
+          path: |
+            e2e/cucumber-report
+            e2e/cucumber-report-non-external
          retention-days: 7

      - name: Upload E2E logs
@@ -0,0 +1,30 @@
+# HITL timeout semantics implementation report
+
+## What changed
+
+- Updated `api/core/workflow/nodes/human_input/callback.py` so `DifyHITLCallback` now preserves Dify's timeout split at the boundary:
+  - `HumanInputFormStatus.TIMEOUT` returns the graphon timeout branch via `Expired(selected_handle="__timeout__", ...)`.
+  - `HumanInputFormStatus.EXPIRED` is treated as an invalid resume state and raises `AssertionError`.
+  - `HumanInputFormStatus.WAITING` with a past global deadline is treated as an invalid resume state and raises `AssertionError`.
+  - `HumanInputFormStatus.WAITING` with only the node-level deadline expired still returns the timeout branch.
+- Added `created_at` to `HumanInputFormEntity` and `_HumanInputFormEntityImpl` so the callback can compute the global deadline using Dify's shared `HUMAN_INPUT_GLOBAL_TIMEOUT_SECONDS` invariant.
+- Kept the submitted and pause flows unchanged.
+- Added focused unit coverage in `api/tests/unit_tests/core/workflow/test_human_input_callback.py` for:
+  - node timeout branch
+  - global expiration rejection
+  - waiting-form past node deadline timeout
+  - waiting-form past global deadline rejection
+
+## Verification
+
+- `uv run --project api pytest -o addopts='' api/tests/unit_tests/core/workflow/test_human_input_callback.py api/tests/unit_tests/core/workflow/nodes/human_input/test_human_input_form_filled_event.py -q`
+- `git diff --check`
+
+## Result
+
+- The focused test set is expected to pass with the new `created_at` boundary in place.
+- No unrelated files were modified.
+
+## Concerns
+
+- The callback now fails fast on invalid resume states by design. That is intentional, but any caller that previously relied on `EXPIRED` being mapped to the timeout branch will now see an assertion failure instead.
@@ -2,14 +2,14 @@ import logging
 import time

 import socketio
-from flask import g, request
+from flask import request
 from opentelemetry.trace import get_current_span
 from opentelemetry.trace.span import INVALID_SPAN_ID, INVALID_TRACE_ID

 from configs import dify_config
 from contexts.wrapper import RecyclableContextVar
 from controllers.console.error import UnauthorizedAndForceLogout
-from core.logging.context import clear_request_context, init_request_context
+from core.logging.context import init_request_context
 from dify_app import DifyApp
 from extensions.ext_socketio import sio
 from services.enterprise.enterprise_service import EnterpriseService
@@ -60,7 +60,6 @@ def create_flask_app_with_configs() -> DifyApp:
    def before_request():
        # Initialize logging context for this request
        init_request_context()
-        g._logging_context_cleanup_scheduled = False
        RecyclableContextVar.increment_thread_recycles()

        # Enterprise license validation for API endpoints (both console and webapp)
@@ -118,26 +117,9 @@ def create_flask_app_with_configs() -> DifyApp:
            logger.warning("Failed to add trace headers to response", exc_info=True)
        return response

-    @dify_app.after_request
-    def schedule_logging_context_cleanup(response):
-        """Keep logging context through streaming, then clear it when WSGI closes the response."""
-        if response.direct_passthrough:
-            return response
-        g._logging_context_cleanup_scheduled = True
-        response.call_on_close(clear_request_context)
-        return response
-
-    @dify_app.teardown_request
-    def clear_unscheduled_logging_context(_error: BaseException | None) -> None:
-        """Clear when no response-close callback can own cleanup."""
-        if not g.get("_logging_context_cleanup_scheduled", False):
-            clear_request_context()
-
    # Capture the decorator return values so static checkers do not treat the hooks as unused.
    _ = before_request
    _ = add_trace_headers
-    _ = schedule_logging_context_cleanup
-    _ = clear_unscheduled_logging_context

    return dify_app

@@ -32,6 +32,8 @@ from clients.agent_backend.factory import create_agent_backend_run_client
 from clients.agent_backend.fake_client import FakeAgentBackendRunClient, FakeAgentBackendScenario
 from clients.agent_backend.request_builder import (
    AGENT_SOUL_PROMPT_LAYER_ID,
+    DIFY_CONFIG_LAYER_ID,
+    DIFY_CORE_TOOLS_LAYER_ID,
    DIFY_EXECUTION_CONTEXT_LAYER_ID,
    DIFY_KNOWLEDGE_BASE_LAYER_ID,
    DIFY_PLUGIN_TOOLS_LAYER_ID,
@@ -47,6 +49,8 @@ from clients.agent_backend.request_builder import (

 __all__ = [
    "AGENT_SOUL_PROMPT_LAYER_ID",
+    "DIFY_CONFIG_LAYER_ID",
+    "DIFY_CORE_TOOLS_LAYER_ID",
    "DIFY_EXECUTION_CONTEXT_LAYER_ID",
    "DIFY_KNOWLEDGE_BASE_LAYER_ID",
    "DIFY_PLUGIN_TOOLS_LAYER_ID",
@@ -67,6 +67,7 @@ class AgentBackendRunSucceededInternalEvent(AgentBackendInternalEventBase):
    type: Literal[AgentBackendInternalEventType.RUN_SUCCEEDED] = AgentBackendInternalEventType.RUN_SUCCEEDED
    output: JsonValue
    session_snapshot: CompositorSessionSnapshot
+    usage: dict[str, JsonValue] | None = None


 class AgentBackendDeferredToolCallInternalEvent(AgentBackendInternalEventBase):
@@ -76,6 +77,7 @@ class AgentBackendDeferredToolCallInternalEvent(AgentBackendInternalEventBase):
    deferred_tool_call: DeferredToolCallPayload
    message: str | None = None
    session_snapshot: CompositorSessionSnapshot
+    usage: dict[str, JsonValue] | None = None


 class AgentBackendRunFailedInternalEvent(AgentBackendInternalEventBase):
@@ -140,6 +142,7 @@ class AgentBackendRunEventAdapter:
                            deferred_tool_call=event.data.deferred_tool_call,
                            message=_deferred_tool_call_message(event.data.deferred_tool_call),
                            session_snapshot=event.data.session_snapshot,
+                            usage=_agent_run_usage(event.data.usage),
                        )
                    ]
                return [
@@ -148,6 +151,7 @@ class AgentBackendRunEventAdapter:
                        source_event_id=event.id,
                        output=event.data.output,
                        session_snapshot=event.data.session_snapshot,
+                        usage=_agent_run_usage(event.data.usage),
                    )
                ]
            case RunFailedEvent():
@@ -184,3 +188,13 @@ def _deferred_tool_call_message(payload: DeferredToolCallPayload) -> str:
            return title

    return f"Agent backend requested external input via deferred tool '{payload.tool_name}'."
+
+
+def _agent_run_usage(usage: object | None) -> dict[str, JsonValue] | None:
+    """Return JSON-safe usage metadata from optional Agent backend usage."""
+    if usage is None:
+        return None
+    dumped = _EVENT_DATA_ADAPTER.dump_python(usage, mode="json")
+    if not isinstance(dumped, dict):
+        return None
+    return cast(dict[str, JsonValue], dumped)
@@ -20,6 +20,8 @@ from agenton.layers import ExitIntent
 from agenton_collections.layers.plain import PLAIN_PROMPT_LAYER_TYPE_ID, PromptLayerConfig
 from agenton_collections.layers.pydantic_ai import PYDANTIC_AI_HISTORY_LAYER_TYPE_ID
 from dify_agent.layers.ask_human import DIFY_ASK_HUMAN_LAYER_TYPE_ID, DifyAskHumanLayerConfig
+from dify_agent.layers.config import DIFY_CONFIG_LAYER_TYPE_ID, DifyConfigLayerConfig
+from dify_agent.layers.dify_core_tools import DIFY_CORE_TOOLS_LAYER_TYPE_ID, DifyCoreToolsLayerConfig
 from dify_agent.layers.dify_plugin import (
    DIFY_PLUGIN_LLM_LAYER_TYPE_ID,
    DIFY_PLUGIN_TOOLS_LAYER_TYPE_ID,
@@ -54,8 +56,10 @@ WORKFLOW_NODE_JOB_PROMPT_LAYER_ID = "workflow_node_job_prompt"
 WORKFLOW_USER_PROMPT_LAYER_ID = "workflow_user_prompt"
 AGENT_APP_USER_PROMPT_LAYER_ID = "agent_app_user_prompt"
 DIFY_EXECUTION_CONTEXT_LAYER_ID = "execution_context"
+DIFY_CONFIG_LAYER_ID = "config"
 DIFY_DRIVE_LAYER_ID = "drive"
 DIFY_PLUGIN_TOOLS_LAYER_ID = "tools"
+DIFY_CORE_TOOLS_LAYER_ID = "core_tools"
 DIFY_KNOWLEDGE_BASE_LAYER_ID = "knowledge"
 DIFY_ASK_HUMAN_LAYER_ID = "ask_human"
 DIFY_SHELL_LAYER_ID = "shell"
@@ -78,11 +82,26 @@ def _filter_snapshot_to_specs(
    return CompositorSessionSnapshot(schema_version=snapshot.schema_version, layers=filtered_layers)


-def _shell_layer_deps(*, include_drive: bool) -> dict[str, str]:
-    deps = {"execution_context": DIFY_EXECUTION_CONTEXT_LAYER_ID}
-    if include_drive:
-        deps["drive"] = DIFY_DRIVE_LAYER_ID
-    return deps
+def _shell_layer_deps() -> dict[str, str]:
+    return {"execution_context": DIFY_EXECUTION_CONTEXT_LAYER_ID}
+
+
+def _drive_layer_deps() -> dict[str, str]:
+    return {"shell": DIFY_SHELL_LAYER_ID}
+
+
+def _config_layer_deps() -> dict[str, str]:
+    return {"shell": DIFY_SHELL_LAYER_ID}
+
+
+def _shell_config_with_drive_ref(
+    shell_config: DifyShellLayerConfig | None,
+    drive_config: DifyDriveLayerConfig | None,
+) -> DifyShellLayerConfig:
+    config = shell_config or DifyShellLayerConfig()
+    if drive_config is None:
+        return config
+    return config.model_copy(update={"agent_stub_drive_ref": drive_config.drive_ref})


 class AgentBackendModelConfig(BaseModel):
@@ -148,9 +167,11 @@ class AgentBackendWorkflowNodeRunInput(BaseModel):
    idempotency_key: str | None = None
    output: AgentBackendOutputConfig | None = None
    tools: DifyPluginToolsLayerConfig | None = None
+    core_tools: DifyCoreToolsLayerConfig | None = None
    knowledge: DifyKnowledgeBaseLayerConfig | None = None
+    config_layer_config: DifyConfigLayerConfig | None = None
    # Drive Skills & Files declaration (dify.drive) — an index the agent pulls
-    # through the back proxy, never inline content; see AGENT_DRIVE_MANIFEST_ENABLED.
+    # through the back proxy, never inline content.
    drive_config: DifyDriveLayerConfig | None = None
    # Human-in-the-loop ask_human deferred tool (dify.ask_human). Present only when
    # the Agent Soul configures human involvement; a deferred call ends the run and
@@ -195,9 +216,11 @@ class AgentBackendAgentAppRunInput(BaseModel):
    idempotency_key: str | None = None
    output: AgentBackendOutputConfig | None = None
    tools: DifyPluginToolsLayerConfig | None = None
+    core_tools: DifyCoreToolsLayerConfig | None = None
    knowledge: DifyKnowledgeBaseLayerConfig | None = None
+    config_layer_config: DifyConfigLayerConfig | None = None
    # Drive Skills & Files declaration (dify.drive) — an index the agent pulls
-    # through the back proxy, never inline content; see AGENT_DRIVE_MANIFEST_ENABLED.
+    # through the back proxy, never inline content.
    drive_config: DifyDriveLayerConfig | None = None
    # Human-in-the-loop ask_human deferred tool (dify.ask_human). Present only when
    # the Agent Soul configures human involvement (ENG-635).
@@ -231,9 +254,11 @@ class AgentBackendRunRequestBuilder:
        """Build an Agent App conversation-turn run request.

        Layer graph: optional Agent Soul system prompt → user prompt →
-        execution context → optional history (multi-turn) → LLM → optional
-        plugin tools / knowledge search → optional structured output. Mirrors the workflow-node
-        layer ordering minus the workflow-job / previous-node prompt.
+        execution context → optional shell / config / drive / history
+        (multi-turn) → LLM → optional plugin-direct tools / core-routed tools /
+        knowledge search / ask_human / structured output. Mirrors the
+        workflow-node layer ordering minus the workflow-job / previous-node
+        prompt.
        """
        layers: list[RunLayerSpec] = []
        if run_input.agent_soul_prompt:
@@ -263,14 +288,42 @@ class AgentBackendRunRequestBuilder:
            ]
        )

+        include_shell = (
+            run_input.include_shell or run_input.config_layer_config is not None or run_input.drive_config is not None
+        )
+        if include_shell:
+            # Sandboxed bash workspace (dify.shell). It enters before config/drive
+            # so eager pulls materialize content in the same filesystem used by
+            # model commands.
+            layers.append(
+                RunLayerSpec(
+                    name=DIFY_SHELL_LAYER_ID,
+                    type=DIFY_SHELL_LAYER_TYPE_ID,
+                    deps=_shell_layer_deps(),
+                    metadata=run_input.metadata,
+                    config=_shell_config_with_drive_ref(run_input.shell_config, run_input.drive_config),
+                )
+            )
+
+        if run_input.config_layer_config is not None:
+            layers.append(
+                RunLayerSpec(
+                    name=DIFY_CONFIG_LAYER_ID,
+                    type=DIFY_CONFIG_LAYER_TYPE_ID,
+                    deps=_config_layer_deps(),
+                    metadata=run_input.metadata,
+                    config=run_input.config_layer_config,
+                )
+            )
+
        if run_input.drive_config is not None:
-            # Drive Skills & Files declaration (dify.drive): a config-only index;
-            # the agent pulls listed entries through the back proxy by drive_ref.
+            # Drive Skills & Files declaration (dify.drive): the catalog plus
+            # prompt-mentioned entries eagerly pulled through the shell layer.
            layers.append(
                RunLayerSpec(
                    name=DIFY_DRIVE_LAYER_ID,
                    type=DIFY_DRIVE_LAYER_TYPE_ID,
-                    deps={"execution_context": DIFY_EXECUTION_CONTEXT_LAYER_ID},
+                    deps=_drive_layer_deps(),
                    metadata=run_input.metadata,
                    config=run_input.drive_config,
                )
@@ -302,17 +355,31 @@ class AgentBackendRunRequestBuilder:
        )

        if run_input.tools is not None and run_input.tools.tools:
+            plugin_tool_deps = {"execution_context": DIFY_EXECUTION_CONTEXT_LAYER_ID}
+            if include_shell:
+                plugin_tool_deps["shell"] = DIFY_SHELL_LAYER_ID
            layers.append(
                RunLayerSpec(
                    name=DIFY_PLUGIN_TOOLS_LAYER_ID,
                    type=DIFY_PLUGIN_TOOLS_LAYER_TYPE_ID,
-                    deps={"execution_context": DIFY_EXECUTION_CONTEXT_LAYER_ID},
+                    deps=plugin_tool_deps,
                    metadata=run_input.metadata,
                    config=run_input.tools,
                )
            )

-        if run_input.knowledge is not None and run_input.knowledge.dataset_ids:
+        if run_input.core_tools is not None and run_input.core_tools.tools:
+            layers.append(
+                RunLayerSpec(
+                    name=DIFY_CORE_TOOLS_LAYER_ID,
+                    type=DIFY_CORE_TOOLS_LAYER_TYPE_ID,
+                    deps={"execution_context": DIFY_EXECUTION_CONTEXT_LAYER_ID},
+                    metadata=run_input.metadata,
+                    config=run_input.core_tools,
+                )
+            )
+
+        if run_input.knowledge is not None and run_input.knowledge.sets:
            layers.append(
                RunLayerSpec(
                    name=DIFY_KNOWLEDGE_BASE_LAYER_ID,
@@ -336,21 +403,6 @@ class AgentBackendRunRequestBuilder:
                )
            )

-        if run_input.include_shell:
-            # Sandboxed bash workspace (dify.shell). Depends on execution_context
-            # so the agent server can mint per-command Agent Stub env, and on
-            # drive when present so that env points at /mnt/drive/<drive_ref>.
-            # shellctl connection itself is server-injected.
-            layers.append(
-                RunLayerSpec(
-                    name=DIFY_SHELL_LAYER_ID,
-                    type=DIFY_SHELL_LAYER_TYPE_ID,
-                    deps=_shell_layer_deps(include_drive=run_input.drive_config is not None),
-                    metadata=run_input.metadata,
-                    config=run_input.shell_config or DifyShellLayerConfig(),
-                )
-            )
-
        if run_input.output is not None:
            layers.append(
                RunLayerSpec(
@@ -392,7 +444,8 @@ class AgentBackendRunRequestBuilder:
        non-plugin layer graph that produced the snapshot. Plugin layers
        (``dify.plugin.llm``, ``dify.plugin.tools``) are excluded from both the
        composition and the snapshot before submission because their configs
-        require credentials that are not persisted between runs.
+        may carry credentials or runtime-only declarations that are not
+        persisted between runs.
        """
        if not runtime_layer_specs:
            raise ValueError(
@@ -425,8 +478,9 @@ class AgentBackendRunRequestBuilder:
        """Build a workflow Agent Node run request without defining another wire schema.

        Layer graph mirrors the workflow surface: prompts → execution context →
-        optional drive/history → LLM → optional plugin tools / knowledge search
-        → optional auxiliary layers such as ask_human, shell, and structured output.
+        optional shell / config / drive / history → LLM → optional
+        plugin-direct tools / core-routed tools / knowledge search /
+        ask_human / structured output.
        """
        layers: list[RunLayerSpec] = []
        if run_input.agent_soul_prompt:
@@ -445,7 +499,7 @@ class AgentBackendRunRequestBuilder:
                    name=WORKFLOW_NODE_JOB_PROMPT_LAYER_ID,
                    type=PLAIN_PROMPT_LAYER_TYPE_ID,
                    metadata={**run_input.metadata, "origin": "workflow_node_job"},
-                    config=PromptLayerConfig(prefix=run_input.workflow_node_job_prompt),
+                    config=PromptLayerConfig(user=run_input.workflow_node_job_prompt),
                ),
                RunLayerSpec(
                    name=WORKFLOW_USER_PROMPT_LAYER_ID,
@@ -462,14 +516,42 @@ class AgentBackendRunRequestBuilder:
            ]
        )

+        include_shell = (
+            run_input.include_shell or run_input.config_layer_config is not None or run_input.drive_config is not None
+        )
+        if include_shell:
+            # Sandboxed bash workspace (dify.shell). It enters before drive so
+            # drive can materialize mentioned targets with `dify-agent drive pull`
+            # in the same shell-visible filesystem used by model commands.
+            layers.append(
+                RunLayerSpec(
+                    name=DIFY_SHELL_LAYER_ID,
+                    type=DIFY_SHELL_LAYER_TYPE_ID,
+                    deps=_shell_layer_deps(),
+                    metadata=run_input.metadata,
+                    config=_shell_config_with_drive_ref(run_input.shell_config, run_input.drive_config),
+                )
+            )
+
+        if run_input.config_layer_config is not None:
+            layers.append(
+                RunLayerSpec(
+                    name=DIFY_CONFIG_LAYER_ID,
+                    type=DIFY_CONFIG_LAYER_TYPE_ID,
+                    deps=_config_layer_deps(),
+                    metadata=run_input.metadata,
+                    config=run_input.config_layer_config,
+                )
+            )
+
        if run_input.drive_config is not None:
-            # Drive Skills & Files declaration (dify.drive): a config-only index;
-            # the agent pulls listed entries through the back proxy by drive_ref.
+            # Drive Skills & Files declaration (dify.drive): the catalog plus
+            # prompt-mentioned entries eagerly pulled through the shell layer.
            layers.append(
                RunLayerSpec(
                    name=DIFY_DRIVE_LAYER_ID,
                    type=DIFY_DRIVE_LAYER_TYPE_ID,
-                    deps={"execution_context": DIFY_EXECUTION_CONTEXT_LAYER_ID},
+                    deps=_drive_layer_deps(),
                    metadata=run_input.metadata,
                    config=run_input.drive_config,
                )
@@ -503,17 +585,31 @@ class AgentBackendRunRequestBuilder:
        )

        if run_input.tools is not None and run_input.tools.tools:
+            plugin_tool_deps = {"execution_context": DIFY_EXECUTION_CONTEXT_LAYER_ID}
+            if include_shell:
+                plugin_tool_deps["shell"] = DIFY_SHELL_LAYER_ID
            layers.append(
                RunLayerSpec(
                    name=DIFY_PLUGIN_TOOLS_LAYER_ID,
                    type=DIFY_PLUGIN_TOOLS_LAYER_TYPE_ID,
-                    deps={"execution_context": DIFY_EXECUTION_CONTEXT_LAYER_ID},
+                    deps=plugin_tool_deps,
                    metadata=run_input.metadata,
                    config=run_input.tools,
                )
            )

-        if run_input.knowledge is not None and run_input.knowledge.dataset_ids:
+        if run_input.core_tools is not None and run_input.core_tools.tools:
+            layers.append(
+                RunLayerSpec(
+                    name=DIFY_CORE_TOOLS_LAYER_ID,
+                    type=DIFY_CORE_TOOLS_LAYER_TYPE_ID,
+                    deps={"execution_context": DIFY_EXECUTION_CONTEXT_LAYER_ID},
+                    metadata=run_input.metadata,
+                    config=run_input.core_tools,
+                )
+            )
+
+        if run_input.knowledge is not None and run_input.knowledge.sets:
            layers.append(
                RunLayerSpec(
                    name=DIFY_KNOWLEDGE_BASE_LAYER_ID,
@@ -537,21 +633,6 @@ class AgentBackendRunRequestBuilder:
                )
            )

-        if run_input.include_shell:
-            # Sandboxed bash workspace (dify.shell). Depends on execution_context
-            # so the agent server can mint per-command Agent Stub env, and on
-            # drive when present so that env points at /mnt/drive/<drive_ref>.
-            # shellctl connection itself is server-injected.
-            layers.append(
-                RunLayerSpec(
-                    name=DIFY_SHELL_LAYER_ID,
-                    type=DIFY_SHELL_LAYER_TYPE_ID,
-                    deps=_shell_layer_deps(include_drive=run_input.drive_config is not None),
-                    metadata=run_input.metadata,
-                    config=run_input.shell_config or DifyShellLayerConfig(),
-                )
-            )
-
        if run_input.output is not None:
            layers.append(
                RunLayerSpec(
@@ -10,6 +10,7 @@ import click
 import sqlalchemy as sa
 import yaml

+from core.db.session_factory import session_factory
 from extensions.ext_database import db
 from models import Tenant
 from models.model import App
@@ -106,7 +107,8 @@ def export_migration_data(input_file: str | None, output_file: str | None, overw
        assert output_file is not None
        raw_config = _load_json_object(input_file, "Export config")
        selection = ExportConfigParser().parse(raw_config)
-        result = MigrationExportService().export(selection)
+        with session_factory.create_session() as session:
+            result = MigrationExportService().export(session, selection)
        MigrationPackageService().save_package(result.package, output_file, overwrite=overwrite)
        click.echo(click.style(f"Output written to {output_file}", fg="green"))
        _render_report(result.report_items, context=_with_output_path(result.report_context, output_file))
@@ -153,19 +155,21 @@ def import_migration_data(
        _require_options(("--input", input_file))
        assert input_file is not None
        package = MigrationPackageService().load_package(input_file)
-        result = MigrationImportService().import_package(
-            ImportRequest(
-                package=package,
-                cli_target_tenant=target_tenant,
-                operator_email=operator_email,
-                options_override=_build_options_override(
-                    package.metadata.import_options,
-                    id_strategy=id_strategy,
-                    conflict_strategy=conflict_strategy,
-                    create_app_api_token_on_import=create_app_api_token_on_import,
+        with session_factory.create_session() as session:
+            result = MigrationImportService().import_package(
+                session,
+                ImportRequest(
+                    package=package,
+                    cli_target_tenant=target_tenant,
+                    operator_email=operator_email,
+                    options_override=_build_options_override(
+                        package.metadata.import_options,
+                        id_strategy=id_strategy,
+                        conflict_strategy=conflict_strategy,
+                        create_app_api_token_on_import=create_app_api_token_on_import,
+                    ),
                ),
            )
-        )
        _render_report(result.report_items, context=result.report_context)
    except MigrationDataError as exc:
        raise click.ClickException(str(exc)) from exc
@@ -248,7 +252,8 @@ def migration_data_wizard() -> None:
            conflict_strategy=conflict_strategy,
            output_file=output_file,
        )
-        result = MigrationExportService().export(selection)
+        with session_factory.create_session() as session:
+            result = MigrationExportService().export(session, selection)
        MigrationPackageService().save_package(result.package, output_file, overwrite=overwrite)
        click.echo(click.style(f"Output written to {output_file}", fg="green"))
        _print_wizard_step("Report")
@@ -192,7 +192,9 @@ def migrate_member_roles_to_rbac(
                account_id=owner_account_id,
                member_account_ids=[account_id for account_id, _ in batch],
            )
-            current_roles_by_account_id = {item.account_id: {role.id for role in item.roles} for item in current_roles}
+            current_roles_by_account_id = {
+                item.account_id: {str(role.id) for role in item.roles} for item in current_roles
+            }

        replace_jobs: list[tuple[str, str]] = []
        for member_account_id, legacy_role in batch:
@@ -439,9 +441,7 @@ def migrate_dataset_permissions_to_rbac(
                                        "account_id": operator_account_id,
                                        "dataset_id": current_dataset_id,
                                        "target_account_id": member_account_id,
-                                        "payload": replace_user_access_policies_payload.model_dump(
-                                            mode="json", exclude_unset=True
-                                        ),
+                                        "payload": replace_user_access_policies_payload.model_dump(mode="json"),
                                    },
                                },
                            }
@@ -35,6 +35,12 @@ class WorkflowRunArchiveTenantPlan(TypedDict):
    unpaid_tenant_ids: list[str]


+def _normalize_utc_datetime(value: datetime.datetime) -> datetime.datetime:
+    if value.tzinfo is None:
+        return value.replace(tzinfo=datetime.UTC)
+    return value.astimezone(datetime.UTC)
+
+
 def _parse_tenant_prefixes(prefixes: str | None) -> list[str]:
    if not prefixes:
        return []
@@ -156,11 +162,16 @@ def _resolve_archive_time_range(
            raise click.UsageError("Choose either day offsets or explicit dates, not both.")
        if from_days_ago <= to_days_ago:
            raise click.UsageError("--from-days-ago must be greater than --to-days-ago.")
-        now = datetime.datetime.now()
+        now = datetime.datetime.now(datetime.UTC)
        start_from = now - datetime.timedelta(days=from_days_ago)
        end_before = now - datetime.timedelta(days=to_days_ago)
        before_days = 0

+    if start_from is not None:
+        start_from = _normalize_utc_datetime(start_from)
+    if end_before is not None:
+        end_before = _normalize_utc_datetime(end_before)
+
    if start_from and end_before and start_from >= end_before:
        raise click.UsageError("--start-from must be earlier than --end-before.")

@@ -402,6 +413,13 @@ def archive_workflow_runs_plan(
            fg="white",
        )
    )
+    click.echo(
+        click.style(
+            "fixed_archive_window="
+            f"{start_from.isoformat() if start_from else 'unbounded'},{plan_end_before.isoformat()}",
+            fg="white",
+        )
+    )
    click.echo("tenant_prefix,total_tenants,workflow_runs,workflow_node_executions,paid_tenants,unpaid_tenants")
    for row in rows:
        click.echo(
@@ -451,7 +469,7 @@ def archive_workflow_runs_plan(
    default=None,
    help="Archive runs created before this timestamp (UTC if no timezone).",
 )
-@click.option("--batch-size", default=100, show_default=True, help="Maximum workflow runs per archive bundle.")
+@click.option("--batch-size", default=10000, show_default=True, help="Maximum workflow runs per archive bundle.")
@click.option(
    "--workers",
    default=1,
@@ -521,6 +539,7 @@ def archive_workflow_runs(
        )
    )

+    uses_relative_window = start_from is None and end_before is None
    try:
        before_days, start_from, end_before = _resolve_archive_time_range(
            before_days=before_days,
@@ -546,6 +565,14 @@ def archive_workflow_runs(
    if delete_after_archive:
        click.echo(click.style("delete-after-archive is not supported by bundle archive.", fg="red"))
        return
+    if uses_relative_window:
+        click.echo(
+            click.style(
+                "Relative archive windows are evaluated at command start. For multi-day prefix/shard rollout, "
+                "reuse absolute --start-from/--end-before values from archive-workflow-runs-plan.",
+                fg="yellow",
+            )
+        )

    try:
        tenant_plan = _resolve_archive_tenant_ids_from_plan(
@@ -13,6 +13,7 @@ from core.rag.index_processor.constant.built_in_field import BuiltInField
 from core.rag.index_processor.constant.index_type import IndexStructureType, IndexTechniqueType
 from core.rag.models.document import ChildDocument, Document
 from extensions.ext_database import db
+from libs.pagination import paginate_query
 from models.dataset import Dataset, DatasetCollectionBinding, DatasetMetadata, DatasetMetadataBinding, DocumentSegment
 from models.dataset import Document as DatasetDocument
 from models.enums import DatasetMetadataType, IndexingStatus, SegmentStatus
@@ -183,7 +184,7 @@ def migrate_knowledge_vector_database():
                .order_by(Dataset.created_at.desc())
            )

-            datasets = db.paginate(select=stmt, page=page, per_page=50, max_per_page=50, error_out=False)
+            datasets = paginate_query(stmt, page=page, per_page=50, max_per_page=50)
            if not datasets.items:
                break
        except SQLAlchemyError:
@@ -409,7 +410,7 @@ def old_metadata_migration():
                .where(DatasetDocument.doc_metadata.is_not(None))
                .order_by(DatasetDocument.created_at.desc())
            )
-            documents = db.paginate(select=stmt, page=page, per_page=50, max_per_page=50, error_out=False)
+            documents = paginate_query(stmt, page=page, per_page=50, max_per_page=50)
        except SQLAlchemyError:
            raise
        if not documents:
@@ -16,7 +16,7 @@ class EnterpriseFeatureConfig(BaseSettings):

    CAN_REPLACE_LOGO: bool = Field(
        description="Allow customization of the enterprise logo.",
-        default=True,
+        default=False,
    )

    ENTERPRISE_REQUEST_TIMEOUT: int = Field(
@@ -1,4 +1,4 @@
-from pydantic import Field
+from pydantic import Field, NonNegativeFloat
 from pydantic_settings import BaseSettings


@@ -32,12 +32,10 @@ class AgentBackendConfig(BaseSettings):
        default=False,
    )

-    AGENT_DRIVE_MANIFEST_ENABLED: bool = Field(
+    AGENT_APP_TEXT_DELTA_DEBOUNCE_SECONDS: NonNegativeFloat = Field(
        description=(
-            "Inject the dify.drive layer (Skills & Files drive manifest declaration) "
-            "into Agent runs. The declaration is an index only — the agent backend "
-            "pulls the actual SKILL.md / files through the back proxy. Keep it off "
-            "until the agent backend registers the dify.drive layer type."
+            "Buffer Agent App assistant text deltas for up to this many seconds before "
+            "publishing SSE chunks. Set to 0 to publish each delta immediately."
        ),
-        default=False,
+        default=0.5,
    )
@@ -183,6 +183,7 @@ class Site(BaseModel):
    description: str | None = None
    copyright: str | None = None
    privacy_policy: str | None = None
+    input_placeholder: str | None = None
    custom_disclaimer: str | None = None
    default_language: str
    show_workflow_steps: bool
@@ -0,0 +1,57 @@
+"""Controller session decorators.
+
+`with_session` is an HTTP controller helper: it opens one SQLAlchemy session
+for a Resource handler and injects it as the first argument after `self`.
+Handlers use a transaction by default so migrated write paths keep
+commit/rollback handling; pure read handlers may opt out with `write=False`.
+"""
+
+from collections.abc import Callable
+from functools import wraps
+from typing import Concatenate, overload
+
+from sqlalchemy.orm import Session
+
+from core.db.session_factory import session_factory
+
+
+@overload
+def with_session[T, **P, R](
+    view: Callable[Concatenate[T, Session, P], R],
+    *,
+    write: bool = True,
+) -> Callable[Concatenate[T, P], R]: ...
+
+
+@overload
+def with_session[T, **P, R](
+    view: None = None,
+    *,
+    write: bool = True,
+) -> Callable[[Callable[Concatenate[T, Session, P], R]], Callable[Concatenate[T, P], R]]: ...
+
+
+def with_session[T, **P, R](
+    view: Callable[Concatenate[T, Session, P], R] | None = None,
+    *,
+    write: bool = True,
+) -> (
+    Callable[Concatenate[T, P], R] | Callable[[Callable[Concatenate[T, Session, P], R]], Callable[Concatenate[T, P], R]]
+):
+    """Inject a request-scoped session, using a transaction only for write handlers."""
+
+    def decorator(view: Callable[Concatenate[T, Session, P], R]) -> Callable[Concatenate[T, P], R]:
+        @wraps(view)
+        def wrapper(self: T, *args: P.args, **kwargs: P.kwargs) -> R:
+            if write:
+                with session_factory.get_session_maker().begin() as session:
+                    return view(self, session, *args, **kwargs)
+
+            with session_factory.create_session() as session:
+                return view(self, session, *args, **kwargs)
+
+        return wrapper
+
+    if view is None:
+        return decorator
+    return decorator(view)
@@ -54,6 +54,7 @@ from .app import (
    agent_app_access,
    agent_app_feature,
    agent_app_sandbox,
+    agent_config_inspector,
    agent_drive_inspector,
    annotation,
    app,
@@ -157,6 +158,7 @@ __all__ = [
    "agent_app_feature",
    "agent_app_sandbox",
    "agent_composer",
+    "agent_config_inspector",
    "agent_drive_inspector",
    "agent_providers",
    "agent_roster",
@@ -6,5 +6,15 @@ from services.agent.roster_service import AgentRosterService


 def resolve_agent_app_model(*, tenant_id: str, agent_id: UUID) -> App:
-    """Resolve the hidden Agent App backing an Agent Console resource."""
+    """Resolve a roster Agent's public Agent App."""
    return AgentRosterService(db.session).get_agent_app_model(tenant_id=tenant_id, agent_id=str(agent_id))
+
+
+def resolve_agent_runtime_app_model(*, tenant_id: str, agent_id: UUID) -> App:
+    """Resolve the App that backs an Agent runtime surface.
+
+    This accepts both roster Agent Apps and workflow-only inline Agents with a
+    hidden backing App.
+    """
+
+    return AgentRosterService(db.session).get_agent_runtime_app_model(tenant_id=tenant_id, agent_id=str(agent_id))
@@ -1,10 +1,10 @@
 from uuid import UUID

+from flask import request
 from flask_restx import Resource

-from controllers.common.schema import register_response_schema_models, register_schema_models
+from controllers.common.schema import query_params_from_model, register_response_schema_models, register_schema_models
 from controllers.console import console_ns
-from controllers.console.agent.app_helpers import resolve_agent_app_model
 from controllers.console.app.wraps import get_app_model
 from controllers.console.wraps import (
    RBACPermission,
@@ -28,9 +28,15 @@ from libs.login import login_required
 from models.model import App, AppMode
 from services.agent.composer_service import AgentComposerService
 from services.agent.composer_validator import ComposerConfigValidator
-from services.entities.agent_entities import ComposerSavePayload, WorkflowComposerCopyFromRosterPayload
+from services.entities.agent_entities import (
+    ComposerSavePayload,
+    WorkflowAgentComposerQuery,
+    WorkflowComposerCopyFromRosterPayload,
+)

-register_schema_models(console_ns, ComposerSavePayload, WorkflowComposerCopyFromRosterPayload)
+register_schema_models(
+    console_ns, ComposerSavePayload, WorkflowAgentComposerQuery, WorkflowComposerCopyFromRosterPayload
+)
 register_response_schema_models(
    console_ns,
    AgentAppComposerResponse,
@@ -41,27 +47,28 @@ register_response_schema_models(
 )


-def _resolve_agent_app_id(*, tenant_id: str, agent_id: UUID) -> str:
-    return resolve_agent_app_model(tenant_id=tenant_id, agent_id=agent_id).id
-
-
@console_ns.route("/apps/<uuid:app_id>/workflows/draft/nodes/<string:node_id>/agent-composer")
 class WorkflowAgentComposerApi(Resource):
    @console_ns.response(
        200, "Workflow agent composer state", console_ns.models[WorkflowAgentComposerResponse.__name__]
    )
+    @console_ns.doc(params=query_params_from_model(WorkflowAgentComposerQuery))
    @setup_required
    @login_required
    @account_initialization_required
    @get_app_model(mode=[AppMode.WORKFLOW, AppMode.ADVANCED_CHAT])
+    @with_current_user_id
    @with_current_tenant_id
-    def get(self, tenant_id: str, app_model: App, node_id: str):
+    def get(self, tenant_id: str, account_id: str, app_model: App, node_id: str):
+        query = WorkflowAgentComposerQuery.model_validate(request.args.to_dict(flat=True))
        return dump_response(
            WorkflowAgentComposerResponse,
            AgentComposerService.load_workflow_composer(
                tenant_id=tenant_id,
                app_id=app_model.id,
                node_id=node_id,
+                account_id=account_id,
+                snapshot_id=query.snapshot_id,
            ),
        )

@@ -137,6 +144,7 @@ class WorkflowAgentComposerValidateApi(Resource):
    def post(self, tenant_id: str, app_model: App, node_id: str):
        payload = ComposerSavePayload.model_validate(console_ns.payload or {})
        ComposerConfigValidator.validate_publish_payload(payload)
+        AgentComposerService.validate_knowledge_datasets(tenant_id=tenant_id, agent_soul=payload.agent_soul)
        findings = AgentComposerService.collect_validation_findings(
            tenant_id=tenant_id,
            payload=payload,
@@ -228,10 +236,9 @@ class AgentComposerApi(Resource):
    @account_initialization_required
    @with_current_tenant_id
    def get(self, tenant_id: str, agent_id: UUID):
-        app_id = _resolve_agent_app_id(tenant_id=tenant_id, agent_id=agent_id)
        return dump_response(
            AgentAppComposerResponse,
-            AgentComposerService.load_agent_app_composer(tenant_id=tenant_id, app_id=app_id),
+            AgentComposerService.load_agent_composer(tenant_id=tenant_id, agent_id=str(agent_id)),
        )

    @console_ns.expect(console_ns.models[ComposerSavePayload.__name__])
@@ -244,13 +251,12 @@ class AgentComposerApi(Resource):
    @with_current_user_id
    @with_current_tenant_id
    def put(self, tenant_id: str, account_id: str, agent_id: UUID):
-        app_id = _resolve_agent_app_id(tenant_id=tenant_id, agent_id=agent_id)
        payload = ComposerSavePayload.model_validate(console_ns.payload or {})
        return dump_response(
            AgentAppComposerResponse,
-            AgentComposerService.save_agent_app_composer(
+            AgentComposerService.save_agent_composer(
                tenant_id=tenant_id,
-                app_id=app_id,
+                agent_id=str(agent_id),
                account_id=account_id,
                payload=payload,
            ),
@@ -268,9 +274,10 @@ class AgentComposerValidateApi(Resource):
    @account_initialization_required
    @with_current_tenant_id
    def post(self, tenant_id: str, agent_id: UUID):
-        _resolve_agent_app_id(tenant_id=tenant_id, agent_id=agent_id)
+        AgentComposerService.load_agent_composer(tenant_id=tenant_id, agent_id=str(agent_id))
        payload = ComposerSavePayload.model_validate(console_ns.payload or {})
        ComposerConfigValidator.validate_publish_payload(payload)
+        AgentComposerService.validate_knowledge_datasets(tenant_id=tenant_id, agent_soul=payload.agent_soul)
        findings = AgentComposerService.collect_validation_findings(
            tenant_id=tenant_id,
            payload=payload,
@@ -290,12 +297,11 @@ class AgentComposerCandidatesApi(Resource):
    @with_current_user_id
    @with_current_tenant_id
    def get(self, tenant_id: str, current_user_id: str, agent_id: UUID):
-        app_id = _resolve_agent_app_id(tenant_id=tenant_id, agent_id=agent_id)
        return dump_response(
            AgentComposerCandidatesResponse,
            AgentComposerService.get_agent_app_candidates(
                tenant_id=tenant_id,
-                app_id=app_id,
+                agent_id=str(agent_id),
                user_id=current_user_id,
            ),
        )
@@ -5,16 +5,21 @@ from flask_restx import Resource
 from pydantic import AliasChoices, BaseModel, Field, field_validator
 from sqlalchemy import func, select

-from controllers.common.schema import query_params_from_model, register_response_schema_models, register_schema_models
+from controllers.common.schema import (
+    query_params_from_model,
+    query_params_from_request,
+    register_response_schema_models,
+    register_schema_models,
+)
 from controllers.console import console_ns
-from controllers.console.agent.app_helpers import resolve_agent_app_model
+from controllers.console.agent.app_helpers import resolve_agent_app_model, resolve_agent_runtime_app_model
 from controllers.console.apikey import ApiKeyItem, ApiKeyList, BaseApiKeyListResource, BaseApiKeyResource
 from controllers.console.app.app import (
-    AppDetailWithSite as GenericAppDetailWithSite,
+    APP_LIST_QUERY_ARRAY_FIELDS,
+    AppListQuery,
 )
 from controllers.console.app.app import (
-    AppListQuery,
-    _normalize_app_list_query_args,
+    AppDetailWithSite as GenericAppDetailWithSite,
 )
 from controllers.console.app.app import (
    AppPagination as GenericAppPagination,
@@ -39,9 +44,11 @@ from controllers.console.wraps import (
 )
 from extensions.ext_database import db
 from fields.agent_fields import (
+    AgentConfigDraftSummaryResponse,
    AgentConfigSnapshotDetailResponse,
    AgentConfigSnapshotListResponse,
    AgentConfigSnapshotRestoreResponse,
+    AgentConfigSnapshotSummaryResponse,
    AgentInviteOptionsResponse,
    AgentLogListResponse,
    AgentLogMessageListResponse,
@@ -50,12 +57,16 @@ from fields.agent_fields import (
    AgentRosterListResponse,
    AgentStatisticSummaryEnvelopeResponse,
 )
+from fields.base import ResponseModel
 from libs.datetime_utils import parse_time_range
 from libs.helper import dump_response
 from libs.login import login_required
 from models import Account
+from models.agent import Agent, AgentStatus
+from models.agent_config_entities import AgentSoulConfig
 from models.enums import ApiTokenType
 from models.model import ApiToken, App, IconType
+from services.agent.composer_service import AgentComposerService
 from services.agent.errors import AgentNotFoundError
 from services.agent.observability_service import (
    AgentLogQueryParams,
@@ -65,7 +76,7 @@ from services.agent.observability_service import (
 from services.agent.roster_service import AgentRosterService
 from services.app_service import AppListParams, AppService, CreateAppParams
 from services.enterprise.enterprise_service import EnterpriseService
-from services.entities.agent_entities import RosterListQuery
+from services.entities.agent_entities import ComposerSavePayload, RosterListQuery
 from services.feature_service import FeatureService


@@ -80,33 +91,31 @@ class AgentIdPath(BaseModel):
 class AgentAppCreatePayload(BaseModel):
    name: str = Field(..., min_length=1, description="Agent name")
    description: str | None = Field(default=None, description="Agent description (max 400 chars)", max_length=400)
-    role: str = Field(..., min_length=1, description="Agent role", max_length=255)
+    role: str | None = Field(default=None, description="Agent role", max_length=255)
    icon_type: IconType | None = Field(default=None, description="Icon type")
    icon: str | None = Field(default=None, description="Icon")
    icon_background: str | None = Field(default=None, description="Icon background color")

    @field_validator("role")
    @classmethod
-    def validate_role(cls, value: str) -> str:
-        role = value.strip()
-        if not role:
-            raise ValueError("Agent role is required.")
-        return role
+    def validate_role(cls, value: str | None) -> str | None:
+        if value is None:
+            return None
+        return value.strip()


 # Keep agent-app roster DTOs agent-specific instead of reusing the shared
 # /apps response/request models. The roster surface needs Agent-only fields such
 # as `role`, while the generic console/apps contracts must stay unchanged.
 class AgentAppUpdatePayload(GenericUpdateAppPayload):
-    role: str = Field(..., min_length=1, description="Agent role", max_length=255)
+    role: str | None = Field(default=None, description="Agent role", max_length=255)

    @field_validator("role")
    @classmethod
-    def validate_role(cls, value: str) -> str:
-        role = value.strip()
-        if not role:
-            raise ValueError("Agent role is required.")
-        return role
+    def validate_role(cls, value: str | None) -> str | None:
+        if value is None:
+            return None
+        return value.strip()


 class AgentAppCopyPayload(BaseModel):
@@ -122,10 +131,7 @@ class AgentAppCopyPayload(BaseModel):
    def validate_role(cls, value: str | None) -> str | None:
        if value is None:
            return None
-        role = value.strip()
-        if not role:
-            raise ValueError("Agent role is required when provided.")
-        return role
+        return value.strip()


 class AgentApiStatusPayload(BaseModel):
@@ -191,11 +197,9 @@ class AgentLogsQuery(BaseModel):
    def empty_list_values_to_list(cls, value: object) -> list[str]:
        if value in (None, ""):
            return []
-        if isinstance(value, str):
-            return [value]
        if isinstance(value, list):
-            return [item for item in value if item]
-        return []
+            return [str(item).strip() for item in value if str(item).strip()]
+        raise ValueError("Unsupported query list type.")

    @field_validator("sort_by")
    @classmethod
@@ -232,6 +236,8 @@ class AgentStatisticsQuery(BaseModel):

 class AgentAppPartial(GenericAppPartial):
    app_id: str | None = None
+    backing_app_id: str | None = None
+    hidden_app_backed: bool = False
    debug_conversation_id: str | None = None
    role: str | None = None
    active_config_is_published: bool = False
@@ -241,13 +247,49 @@ class AgentAppPartial(GenericAppPartial):

 class AgentAppDetailWithSite(GenericAppDetailWithSite):
    app_id: str | None = None
+    backing_app_id: str | None = None
+    hidden_app_backed: bool = False
    debug_conversation_id: str | None = None
+    debug_conversation_has_messages: bool = False
+    debug_conversation_message_count: int = 0
    role: str | None = None
    active_config_is_published: bool = False


 class AgentDebugConversationRefreshResponse(BaseModel):
    debug_conversation_id: str
+    debug_conversation_has_messages: bool = False
+    debug_conversation_message_count: int = 0
+
+
+class AgentPublishPayload(BaseModel):
+    version_note: str | None = Field(default=None, description="Optional note for this published Agent version")
+
+
+class AgentPublishResponse(ResponseModel):
+    result: str
+    active_config_snapshot_id: str
+    active_config_snapshot: AgentConfigSnapshotSummaryResponse | None = None
+    draft: AgentConfigDraftSummaryResponse | None = None
+
+
+class AgentBuildDraftCheckoutPayload(BaseModel):
+    force: bool = Field(default=False, description="Overwrite the existing current-user build draft")
+
+
+class AgentBuildDraftResponse(ResponseModel):
+    variant: str
+    draft: AgentConfigDraftSummaryResponse
+    agent_soul: AgentSoulConfig
+
+
+class AgentBuildDraftApplyResponse(BaseModel):
+    result: str
+    draft: dict[str, object]
+
+
+class AgentSimpleResultResponse(BaseModel):
+    result: str


 class AgentAppPagination(GenericAppPagination):
@@ -261,6 +303,9 @@ register_schema_models(
    AgentAppCreatePayload,
    AgentAppUpdatePayload,
    AgentAppCopyPayload,
+    AgentPublishPayload,
+    AgentBuildDraftCheckoutPayload,
+    ComposerSavePayload,
    AgentApiStatusPayload,
    AgentInviteOptionsQuery,
    AgentLogsQuery,
@@ -277,6 +322,10 @@ register_response_schema_models(
    AgentAppDetailWithSite,
    AgentAppPartial,
    AgentDebugConversationRefreshResponse,
+    AgentPublishResponse,
+    AgentBuildDraftResponse,
+    AgentBuildDraftApplyResponse,
+    AgentSimpleResultResponse,
    AgentConfigSnapshotDetailResponse,
    AgentConfigSnapshotListResponse,
    AgentConfigSnapshotRestoreResponse,
@@ -294,7 +343,7 @@ def _agent_roster_service() -> AgentRosterService:
    return AgentRosterService(db.session)


-def _serialize_agent_app_detail(app_model, *, current_user: Account) -> dict:
+def _serialize_agent_app_detail(app_model, *, current_user: Account, agent_id: str | None = None) -> dict:
    """Serialize an Agent App detail using roster-only DTOs.

    `/agent` responses are roster-shaped rather than raw app-shaped: `id`
@@ -311,17 +360,35 @@ def _serialize_agent_app_detail(app_model, *, current_user: Account) -> dict:

    roster_service = _agent_roster_service()
    payload = AgentAppDetailWithSite.model_validate(app_model, from_attributes=True).model_dump(mode="json")
-    agent = roster_service.get_app_backing_agent(tenant_id=app_model.tenant_id, app_id=str(app_model.id))
+    agent = (
+        db.session.scalar(
+            select(Agent).where(
+                Agent.tenant_id == app_model.tenant_id,
+                Agent.id == agent_id,
+                Agent.status == AgentStatus.ACTIVE,
+            )
+        )
+        if agent_id
+        else roster_service.get_app_backing_agent(tenant_id=app_model.tenant_id, app_id=str(app_model.id))
+    )
    if not agent:
        raise AgentNotFoundError()
    payload.pop("bound_agent_id", None)
-    payload["app_id"] = str(app_model.id)
+    payload["app_id"] = agent.app_id
+    payload["backing_app_id"] = roster_service.runtime_backing_app_id(agent)
+    payload["hidden_app_backed"] = bool(agent.backing_app_id and agent.backing_app_id != agent.app_id)
    payload["id"] = agent.id
-    payload["debug_conversation_id"] = roster_service.get_or_create_agent_app_debug_conversation_id(
+    debug_conversation_id = roster_service.get_or_create_agent_app_debug_conversation_id(
        tenant_id=app_model.tenant_id,
        agent_id=agent.id,
        account_id=current_user.id,
    )
+    message_count = roster_service.count_agent_app_debug_conversation_messages(
+        conversation_id=debug_conversation_id,
+    )
+    payload["debug_conversation_id"] = debug_conversation_id
+    payload["debug_conversation_has_messages"] = message_count > 0
+    payload["debug_conversation_message_count"] = message_count
    payload["role"] = agent.role or ""
    payload["active_config_is_published"] = roster_service.active_config_is_published(
        tenant_id=app_model.tenant_id,
@@ -365,6 +432,8 @@ def _serialize_agent_app_pagination(app_pagination, *, tenant_id: str, current_u
        agent = agents_by_app_id.get(app_id)
        if agent:
            item["app_id"] = app_id
+            item["backing_app_id"] = agent.backing_app_id or app_id
+            item["hidden_app_backed"] = False
            item["id"] = agent.id
            item["debug_conversation_id"] = debug_conversation_ids_by_agent_id.get(agent.id)
            item["role"] = agent.role or ""
@@ -435,16 +504,11 @@ def _parse_observability_time_range(start: str | None, end: str | None, account:
        abort(400, description=str(exc))


-def _multi_query_values(name: str, legacy_name: str | None = None) -> list[str]:
-    values: list[str] = []
-    for query_name in (name, f"{name}[]"):
-        values.extend(request.args.getlist(query_name))
-    if legacy_name:
-        values.extend(request.args.getlist(legacy_name))
-    parsed: list[str] = []
-    for value in values:
-        parsed.extend(item.strip() for item in value.split(",") if item.strip())
-    return parsed
+def _query_values(name: str, alias_name: str | None = None) -> list[str]:
+    values = request.args.getlist(name)
+    if alias_name:
+        values.extend(request.args.getlist(alias_name))
+    return [value.strip() for value in values if value.strip()]


@console_ns.route("/agent")
@@ -457,11 +521,12 @@ class AgentAppListApi(Resource):
    @with_current_user
    @with_current_tenant_id
    def get(self, current_tenant_id: str, current_user: Account):
-        args = AppListQuery.model_validate(_normalize_app_list_query_args(request.args))
+        args = query_params_from_request(AppListQuery, list_fields=APP_LIST_QUERY_ARRAY_FIELDS)
        params = AppListParams(
            page=args.page,
            limit=args.limit,
            mode="agent",
+            sort_by=args.sort_by,
            name=args.name,
            tag_ids=args.tag_ids,
            creator_ids=args.creator_ids,
@@ -496,7 +561,7 @@ class AgentAppListApi(Resource):
            name=args.name,
            description=args.description,
            mode="agent",
-            agent_role=args.role,
+            agent_role=args.role or "",
            icon_type=args.icon_type,
            icon=args.icon,
            icon_background=args.icon_background,
@@ -516,8 +581,8 @@ class AgentAppApi(Resource):
    @with_current_user
    @with_current_tenant_id
    def get(self, tenant_id: str, current_user: Account, agent_id: UUID):
-        app_model = _resolve_agent_app_model(tenant_id=tenant_id, agent_id=agent_id)
-        return _serialize_agent_app_detail(app_model, current_user=current_user)
+        app_model = resolve_agent_runtime_app_model(tenant_id=tenant_id, agent_id=agent_id)
+        return _serialize_agent_app_detail(app_model, current_user=current_user, agent_id=str(agent_id))

    @console_ns.expect(console_ns.models[AgentAppUpdatePayload.__name__])
    @console_ns.response(200, "Agent app updated successfully", console_ns.models[AgentAppDetailWithSite.__name__])
@@ -578,8 +643,116 @@ class AgentDebugConversationRefreshApi(Resource):
            agent_id=str(agent_id),
            account_id=current_user.id,
        )
-        return AgentDebugConversationRefreshResponse(debug_conversation_id=debug_conversation_id).model_dump(
-            mode="json"
+        return AgentDebugConversationRefreshResponse(
+            debug_conversation_id=debug_conversation_id,
+            debug_conversation_has_messages=False,
+            debug_conversation_message_count=0,
+        ).model_dump(mode="json")
+
+
+@console_ns.route("/agent/<uuid:agent_id>/publish")
+class AgentPublishApi(Resource):
+    @console_ns.expect(console_ns.models[AgentPublishPayload.__name__])
+    @console_ns.response(200, "Agent draft published", console_ns.models[AgentPublishResponse.__name__])
+    @console_ns.response(403, "Insufficient permissions")
+    @setup_required
+    @login_required
+    @account_initialization_required
+    @edit_permission_required
+    @with_current_user
+    @with_current_tenant_id
+    def post(self, tenant_id: str, current_user: Account, agent_id: UUID):
+        args = AgentPublishPayload.model_validate(console_ns.payload or {})
+        return AgentComposerService.publish_agent_app_draft(
+            tenant_id=tenant_id,
+            agent_id=str(agent_id),
+            account_id=current_user.id,
+            version_note=args.version_note,
+        )
+
+
+@console_ns.route("/agent/<uuid:agent_id>/build-draft/checkout")
+class AgentBuildDraftCheckoutApi(Resource):
+    @console_ns.expect(console_ns.models[AgentBuildDraftCheckoutPayload.__name__])
+    @console_ns.response(200, "Agent build draft checked out", console_ns.models[AgentBuildDraftResponse.__name__])
+    @setup_required
+    @login_required
+    @account_initialization_required
+    @edit_permission_required
+    @with_current_user
+    @with_current_tenant_id
+    def post(self, tenant_id: str, current_user: Account, agent_id: UUID):
+        args = AgentBuildDraftCheckoutPayload.model_validate(console_ns.payload or {})
+        return AgentComposerService.checkout_agent_app_build_draft(
+            tenant_id=tenant_id,
+            agent_id=str(agent_id),
+            account_id=current_user.id,
+            force=args.force,
+        )
+
+
+@console_ns.route("/agent/<uuid:agent_id>/build-draft")
+class AgentBuildDraftApi(Resource):
+    @console_ns.response(200, "Agent build draft", console_ns.models[AgentBuildDraftResponse.__name__])
+    @setup_required
+    @login_required
+    @account_initialization_required
+    @edit_permission_required
+    @with_current_user
+    @with_current_tenant_id
+    def get(self, tenant_id: str, current_user: Account, agent_id: UUID):
+        return AgentComposerService.load_agent_app_build_draft(
+            tenant_id=tenant_id,
+            agent_id=str(agent_id),
+            account_id=current_user.id,
+        )
+
+    @console_ns.expect(console_ns.models[ComposerSavePayload.__name__])
+    @console_ns.response(200, "Agent build draft saved", console_ns.models[AgentBuildDraftResponse.__name__])
+    @setup_required
+    @login_required
+    @account_initialization_required
+    @edit_permission_required
+    @with_current_user
+    @with_current_tenant_id
+    def put(self, tenant_id: str, current_user: Account, agent_id: UUID):
+        payload = ComposerSavePayload.model_validate(console_ns.payload or {})
+        return AgentComposerService.save_agent_app_build_draft(
+            tenant_id=tenant_id,
+            agent_id=str(agent_id),
+            account_id=current_user.id,
+            payload=payload,
+        )
+
+    @console_ns.response(200, "Agent build draft discarded", console_ns.models[AgentSimpleResultResponse.__name__])
+    @setup_required
+    @login_required
+    @account_initialization_required
+    @edit_permission_required
+    @with_current_user
+    @with_current_tenant_id
+    def delete(self, tenant_id: str, current_user: Account, agent_id: UUID):
+        return AgentComposerService.discard_agent_app_build_draft(
+            tenant_id=tenant_id,
+            agent_id=str(agent_id),
+            account_id=current_user.id,
+        )
+
+
+@console_ns.route("/agent/<uuid:agent_id>/build-draft/apply")
+class AgentBuildDraftApplyApi(Resource):
+    @console_ns.response(200, "Agent build draft applied", console_ns.models[AgentBuildDraftApplyResponse.__name__])
+    @setup_required
+    @login_required
+    @account_initialization_required
+    @edit_permission_required
+    @with_current_user
+    @with_current_tenant_id
+    def post(self, tenant_id: str, current_user: Account, agent_id: UUID):
+        return AgentComposerService.apply_agent_app_build_draft(
+            tenant_id=tenant_id,
+            agent_id=str(agent_id),
+            account_id=current_user.id,
        )


@@ -712,10 +885,10 @@ class AgentLogsApi(Resource):
    @with_current_user
    @with_current_tenant_id
    def get(self, tenant_id: str, current_user: Account, agent_id: UUID):
-        app_model = _resolve_agent_app_model(tenant_id=tenant_id, agent_id=agent_id)
+        app_model = resolve_agent_runtime_app_model(tenant_id=tenant_id, agent_id=agent_id)
        query_data: dict[str, object] = dict(request.args.to_dict(flat=True))
-        query_data["sources"] = _multi_query_values("sources", "source")
-        query_data["statuses"] = _multi_query_values("statuses", "status")
+        query_data["sources"] = _query_values("sources", "source")
+        query_data["statuses"] = _query_values("statuses", "status")
        query = AgentLogsQuery.model_validate(query_data)
        start, end = _parse_observability_time_range(query.start, query.end, current_user)
        try:
@@ -749,10 +922,10 @@ class AgentLogMessagesApi(Resource):
    @with_current_user
    @with_current_tenant_id
    def get(self, tenant_id: str, current_user: Account, agent_id: UUID, conversation_id: UUID):
-        app_model = _resolve_agent_app_model(tenant_id=tenant_id, agent_id=agent_id)
+        app_model = resolve_agent_runtime_app_model(tenant_id=tenant_id, agent_id=agent_id)
        query_data: dict[str, object] = dict(request.args.to_dict(flat=True))
-        query_data["sources"] = _multi_query_values("sources", "source")
-        query_data["statuses"] = _multi_query_values("statuses", "status")
+        query_data["sources"] = _query_values("sources", "source")
+        query_data["statuses"] = _query_values("statuses", "status")
        query = AgentLogsQuery.model_validate(query_data)
        start, end = _parse_observability_time_range(query.start, query.end, current_user)
        try:
@@ -786,7 +959,7 @@ class AgentLogSourcesApi(Resource):
    @with_current_user
    @with_current_tenant_id
    def get(self, tenant_id: str, current_user: Account, agent_id: UUID):
-        app_model = _resolve_agent_app_model(tenant_id=tenant_id, agent_id=agent_id)
+        app_model = resolve_agent_runtime_app_model(tenant_id=tenant_id, agent_id=agent_id)
        payload = _agent_observability_service().list_log_sources(app=app_model, agent_id=str(agent_id))
        return dump_response(AgentLogSourceListResponse, payload)

@@ -805,7 +978,7 @@ class AgentStatisticsSummaryApi(Resource):
    @with_current_user
    @with_current_tenant_id
    def get(self, tenant_id: str, current_user: Account, agent_id: UUID):
-        app_model = _resolve_agent_app_model(tenant_id=tenant_id, agent_id=agent_id)
+        app_model = resolve_agent_runtime_app_model(tenant_id=tenant_id, agent_id=agent_id)
        query = AgentStatisticsQuery.model_validate(request.args.to_dict(flat=True))
        timezone = current_user.timezone or "UTC"
        start, end = _parse_observability_time_range(query.start, query.end, current_user)
@@ -13,7 +13,7 @@ from controllers.common.schema import (
    register_schema_models,
 )
 from controllers.console import console_ns
-from controllers.console.agent.app_helpers import resolve_agent_app_model
+from controllers.console.agent.app_helpers import resolve_agent_runtime_app_model
 from controllers.console.app.wraps import get_app_model
 from controllers.console.wraps import (
    RBACPermission,
@@ -351,7 +351,7 @@ class AgentSkillUploadByAgentApi(Resource):
    @with_current_user
    @with_current_tenant_id
    def post(self, tenant_id: str, current_user: Account, agent_id: UUID):
-        app_model = resolve_agent_app_model(tenant_id=tenant_id, agent_id=agent_id)
+        app_model = resolve_agent_runtime_app_model(tenant_id=tenant_id, agent_id=agent_id)
        return _upload_skill_for_app(current_user=current_user, app_model=app_model)


@@ -394,7 +394,7 @@ class AgentDriveFilesByAgentApi(Resource):
    @with_current_user
    @with_current_tenant_id
    def post(self, tenant_id: str, current_user: Account, agent_id: UUID):
-        app_model = resolve_agent_app_model(tenant_id=tenant_id, agent_id=agent_id)
+        app_model = resolve_agent_runtime_app_model(tenant_id=tenant_id, agent_id=agent_id)
        return _commit_drive_file_for_app(current_user=current_user, app_model=app_model, allow_node_id=False)

    @console_ns.doc("delete_agent_drive_file_by_agent")
@@ -407,7 +407,7 @@ class AgentDriveFilesByAgentApi(Resource):
    @with_current_user
    @with_current_tenant_id
    def delete(self, tenant_id: str, current_user: Account, agent_id: UUID):
-        app_model = resolve_agent_app_model(tenant_id=tenant_id, agent_id=agent_id)
+        app_model = resolve_agent_runtime_app_model(tenant_id=tenant_id, agent_id=agent_id)
        return _delete_drive_file_for_app(current_user=current_user, app_model=app_model, allow_node_id=False)


@@ -454,7 +454,7 @@ class AgentSkillByAgentApi(Resource):
    @with_current_user
    @with_current_tenant_id
    def delete(self, tenant_id: str, current_user: Account, agent_id: UUID, slug: str):
-        app_model = resolve_agent_app_model(tenant_id=tenant_id, agent_id=agent_id)
+        app_model = resolve_agent_runtime_app_model(tenant_id=tenant_id, agent_id=agent_id)
        return _delete_skill_for_app(current_user=current_user, app_model=app_model, slug=slug, allow_node_id=False)


@@ -494,7 +494,7 @@ class AgentSkillInferToolsByAgentApi(Resource):
    @account_initialization_required
    @with_current_tenant_id
    def post(self, tenant_id: str, agent_id: UUID, slug: str):
-        app_model = resolve_agent_app_model(tenant_id=tenant_id, agent_id=agent_id)
+        app_model = resolve_agent_runtime_app_model(tenant_id=tenant_id, agent_id=agent_id)
        return _infer_skill_tools_for_app(app_model=app_model, slug=slug)


@@ -17,7 +17,7 @@ from pydantic import BaseModel, Field
 from controllers.common.fields import SimpleResultResponse
 from controllers.common.schema import register_response_schema_models, register_schema_models
 from controllers.console import console_ns
-from controllers.console.agent.app_helpers import resolve_agent_app_model
+from controllers.console.agent.app_helpers import resolve_agent_runtime_app_model
 from controllers.console.wraps import (
    RBACPermission,
    RBACResourceScope,
@@ -30,7 +30,6 @@ from controllers.console.wraps import (
 )
 from events.app_event import app_model_config_was_updated
 from extensions.ext_database import db
-from libs.helper import dump_response
 from libs.login import login_required
 from models import Account
 from models.agent_config_entities import (
@@ -87,7 +86,7 @@ class AgentAppFeatureConfigResource(Resource):
    @with_current_user
    @with_current_tenant_id
    def post(self, tenant_id: str, current_user: Account, agent_id: UUID):
-        app_model = resolve_agent_app_model(tenant_id=tenant_id, agent_id=agent_id)
+        app_model = resolve_agent_runtime_app_model(tenant_id=tenant_id, agent_id=agent_id)
        args = AgentAppFeaturesPayload.model_validate(console_ns.payload or {})

        new_app_model_config = AgentAppFeatureConfigService.update_features(
@@ -99,4 +98,4 @@ class AgentAppFeatureConfigResource(Resource):

        app_model_config_was_updated.send(app_model, app_model_config=new_app_model_config)

-        return dump_response(SimpleResultResponse, {"result": "success"})
+        return SimpleResultResponse(result="success").model_dump(mode="json")
@@ -22,7 +22,7 @@ from controllers.common.schema import (
    register_schema_models,
 )
 from controllers.console import console_ns
-from controllers.console.agent.app_helpers import resolve_agent_app_model
+from controllers.console.agent.app_helpers import resolve_agent_runtime_app_model
 from controllers.console.app.wraps import get_app_model
 from controllers.console.wraps import account_initialization_required, setup_required, with_current_tenant_id
 from fields.base import ResponseModel
@@ -44,6 +44,10 @@ class AgentSandboxListQuery(BaseModel):
    path: str = Field(default=".", description="Directory path relative to the sandbox workspace")


+class AgentSandboxInfoQuery(BaseModel):
+    conversation_id: str = Field(min_length=1, description="Agent App conversation ID")
+
+
 class AgentSandboxFileQuery(BaseModel):
    conversation_id: str = Field(min_length=1, description="Agent App conversation ID")
    path: str = Field(min_length=1, description="File path relative to the sandbox workspace")
@@ -91,6 +95,11 @@ class SandboxListResponse(ResponseModel):
    truncated: bool = False


+class SandboxInfoResponse(ResponseModel):
+    session_id: str
+    workspace_cwd: str
+
+
 class SandboxReadResponse(ResponseModel):
    path: str
    size: int | None = None
@@ -114,7 +123,13 @@ register_schema_models(
    AgentSandboxUploadPayload,
    WorkflowAgentSandboxUploadPayload,
 )
-register_response_schema_models(console_ns, SandboxListResponse, SandboxReadResponse, SandboxUploadResponse)
+register_response_schema_models(
+    console_ns,
+    SandboxInfoResponse,
+    SandboxListResponse,
+    SandboxReadResponse,
+    SandboxUploadResponse,
+)


 def _handle(exc: Exception) -> tuple[dict[str, object], int]:
@@ -133,6 +148,30 @@ def _handle(exc: Exception) -> tuple[dict[str, object], int]:
    raise exc


+@console_ns.route("/agent/<uuid:agent_id>/sandbox")
+class AgentAppSandboxInfoResource(Resource):
+    @console_ns.doc("get_agent_app_sandbox_info")
+    @console_ns.doc(description="Get basic information for an Agent App conversation sandbox")
+    @console_ns.doc(params={"agent_id": "Agent ID", **query_params_from_model(AgentSandboxInfoQuery)})
+    @console_ns.response(200, "Sandbox information returned", console_ns.models[SandboxInfoResponse.__name__])
+    @setup_required
+    @login_required
+    @account_initialization_required
+    @with_current_tenant_id
+    def get(self, tenant_id: str, agent_id: UUID):
+        app_model = resolve_agent_runtime_app_model(tenant_id=tenant_id, agent_id=agent_id)
+        query = query_params_from_request(AgentSandboxInfoQuery)
+        try:
+            result = AgentAppSandboxService().get_info(
+                tenant_id=tenant_id,
+                app_id=app_model.id,
+                conversation_id=query.conversation_id,
+            )
+        except Exception as exc:
+            return _handle(exc)
+        return result.model_dump()
+
+
@console_ns.route("/agent/<uuid:agent_id>/sandbox/files")
 class AgentAppSandboxListResource(Resource):
    @console_ns.doc("list_agent_app_sandbox_files")
@@ -144,7 +183,7 @@ class AgentAppSandboxListResource(Resource):
    @account_initialization_required
    @with_current_tenant_id
    def get(self, tenant_id: str, agent_id: UUID):
-        app_model = resolve_agent_app_model(tenant_id=tenant_id, agent_id=agent_id)
+        app_model = resolve_agent_runtime_app_model(tenant_id=tenant_id, agent_id=agent_id)
        query = query_params_from_request(AgentSandboxListQuery)
        try:
            result = AgentAppSandboxService().list_files(
@@ -169,7 +208,7 @@ class AgentAppSandboxReadResource(Resource):
    @account_initialization_required
    @with_current_tenant_id
    def get(self, tenant_id: str, agent_id: UUID):
-        app_model = resolve_agent_app_model(tenant_id=tenant_id, agent_id=agent_id)
+        app_model = resolve_agent_runtime_app_model(tenant_id=tenant_id, agent_id=agent_id)
        query = query_params_from_request(AgentSandboxFileQuery)
        try:
            result = AgentAppSandboxService().read_file(
@@ -194,7 +233,7 @@ class AgentAppSandboxUploadResource(Resource):
    @account_initialization_required
    @with_current_tenant_id
    def post(self, tenant_id: str, agent_id: UUID):
-        app_model = resolve_agent_app_model(tenant_id=tenant_id, agent_id=agent_id)
+        app_model = resolve_agent_runtime_app_model(tenant_id=tenant_id, agent_id=agent_id)
        payload = AgentSandboxUploadPayload.model_validate(request.get_json(silent=True) or {})
        try:
            result = AgentAppSandboxService().upload_file(
@@ -25,7 +25,7 @@ from controllers.common.schema import (
    register_response_schema_models,
 )
 from controllers.console import console_ns
-from controllers.console.agent.app_helpers import resolve_agent_app_model
+from controllers.console.agent.app_helpers import resolve_agent_runtime_app_model
 from controllers.console.app.wraps import get_app_model
 from controllers.console.wraps import account_initialization_required, setup_required, with_current_tenant_id
 from fields.base import ResponseModel
@@ -182,7 +182,7 @@ class AgentDriveListByAgentApi(Resource):
    @with_current_tenant_id
    def get(self, tenant_id: str, agent_id: UUID):
        query = query_params_from_request(AgentDriveListByAgentQuery)
-        resolve_agent_app_model(tenant_id=tenant_id, agent_id=agent_id)
+        resolve_agent_runtime_app_model(tenant_id=tenant_id, agent_id=agent_id)
        try:
            items = AgentDriveService().manifest(tenant_id=tenant_id, agent_id=str(agent_id), prefix=query.prefix)
        except AgentDriveError as exc:
@@ -201,7 +201,7 @@ class AgentDriveSkillListByAgentApi(Resource):
    @account_initialization_required
    @with_current_tenant_id
    def get(self, tenant_id: str, agent_id: UUID):
-        resolve_agent_app_model(tenant_id=tenant_id, agent_id=agent_id)
+        resolve_agent_runtime_app_model(tenant_id=tenant_id, agent_id=agent_id)
        try:
            items = AgentDriveService().list_skills(tenant_id=tenant_id, agent_id=str(agent_id))
        except AgentDriveError as exc:
@@ -220,7 +220,7 @@ class AgentDriveSkillInspectByAgentApi(Resource):
    @account_initialization_required
    @with_current_tenant_id
    def get(self, tenant_id: str, agent_id: UUID, skill_path: str):
-        resolve_agent_app_model(tenant_id=tenant_id, agent_id=agent_id)
+        resolve_agent_runtime_app_model(tenant_id=tenant_id, agent_id=agent_id)
        try:
            return _json_response(
                AgentDriveService().inspect_skill(
@@ -245,7 +245,7 @@ class AgentDrivePreviewByAgentApi(Resource):
    @with_current_tenant_id
    def get(self, tenant_id: str, agent_id: UUID):
        query = query_params_from_request(AgentDriveFileByAgentQuery)
-        resolve_agent_app_model(tenant_id=tenant_id, agent_id=agent_id)
+        resolve_agent_runtime_app_model(tenant_id=tenant_id, agent_id=agent_id)
        try:
            return AgentDriveService().preview(tenant_id=tenant_id, agent_id=str(agent_id), key=query.key)
        except AgentDriveError as exc:
@@ -264,7 +264,7 @@ class AgentDriveDownloadByAgentApi(Resource):
    @with_current_tenant_id
    def get(self, tenant_id: str, agent_id: UUID):
        query = query_params_from_request(AgentDriveFileByAgentQuery)
-        resolve_agent_app_model(tenant_id=tenant_id, agent_id=agent_id)
+        resolve_agent_runtime_app_model(tenant_id=tenant_id, agent_id=agent_id)
        try:
            url = AgentDriveService().download_url(tenant_id=tenant_id, agent_id=str(agent_id), key=query.key)
        except AgentDriveError as exc:
@@ -1,9 +1,11 @@
 from typing import Any, Literal
 from uuid import UUID

-from flask import abort, make_response, request
+from flask import abort, request
 from flask_restx import Resource
 from pydantic import BaseModel, Field, TypeAdapter, field_validator
+from sqlalchemy import select
+from werkzeug.exceptions import NotFound

 from controllers.common.errors import NoFileUploadedError, TooManyFilesError
 from controllers.common.schema import query_params_from_model, register_response_schema_models, register_schema_models
@@ -26,11 +28,14 @@ from fields.annotation_fields import (
    AnnotationExportList,
    AnnotationHitHistory,
    AnnotationHitHistoryList,
+    AnnotationJobStatusDetailResponse,
+    AnnotationJobStatusResponse,
    AnnotationList,
 )
 from fields.base import ResponseModel
-from libs.helper import uuid_value
-from libs.login import login_required
+from libs.helper import dump_response, uuid_value
+from libs.login import current_account_with_tenant, login_required
+from models.model import App
 from services.annotation_service import (
    AppAnnotationService,
    EnableAnnotationArgs,
@@ -38,6 +43,17 @@ from services.annotation_service import (
    UpdateAnnotationSettingArgs,
    UpsertAnnotationArgs,
 )
+from services.app_ref_service import AppRef, AppRefService
+
+
+def _get_app_ref(app_id: str) -> AppRef:
+    _, current_tenant_id = current_account_with_tenant()
+    app = db.session.scalar(
+        select(App).where(App.id == app_id, App.tenant_id == current_tenant_id, App.status == "normal").limit(1)
+    )
+    if app is None:
+        raise NotFound("App not found")
+    return AppRefService.create_app_ref(app)


 class AnnotationReplyPayload(BaseModel):
@@ -99,23 +115,23 @@ class AnnotationFilePayload(BaseModel):
        return uuid_value(value)


-class AnnotationJobStatusResponse(ResponseModel):
-    job_id: str | None = None
-    job_status: str | None = None
-    error_msg: str | None = None
-    record_count: int | None = None
-
-
-class AnnotationEmbeddingModelResponse(ResponseModel):
+class AnnotationSettingEmbeddingModelResponse(ResponseModel):
    embedding_provider_name: str | None = None
    embedding_model_name: str | None = None


 class AnnotationSettingResponse(ResponseModel):
-    id: str | None = None
    enabled: bool
+    id: str | None = None
    score_threshold: float | None = None
-    embedding_model: AnnotationEmbeddingModelResponse | None = None
+    embedding_model: AnnotationSettingEmbeddingModelResponse | None = None
+
+
+class AnnotationBatchImportResponse(ResponseModel):
+    job_id: str | None = None
+    job_status: str | None = None
+    record_count: int | None = None
+    error_msg: str | None = None


 register_schema_models(
@@ -142,7 +158,10 @@ register_response_schema_models(
    AnnotationHitHistory,
    AnnotationHitHistoryList,
    AnnotationJobStatusResponse,
+    AnnotationJobStatusDetailResponse,
+    AnnotationSettingEmbeddingModelResponse,
    AnnotationSettingResponse,
+    AnnotationBatchImportResponse,
 )


@@ -172,7 +191,7 @@ class AnnotationReplyActionApi(Resource):
                result = AppAnnotationService.enable_app_annotation(enable_args, str(app_id))
            case "disable":
                result = AppAnnotationService.disable_app_annotation(str(app_id))
-        return result, 200
+        return dump_response(AnnotationJobStatusResponse, result), 200


@console_ns.route("/apps/<uuid:app_id>/annotation-setting")
@@ -193,7 +212,7 @@ class AppAnnotationSettingDetailApi(Resource):
    @rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_VIEW_LAYOUT)
    def get(self, app_id: UUID):
        result = AppAnnotationService.get_app_annotation_setting_by_app_id(str(app_id))
-        return result, 200
+        return dump_response(AnnotationSettingResponse, result), 200


@console_ns.route("/apps/<uuid:app_id>/annotation-settings/<uuid:annotation_setting_id>")
@@ -218,7 +237,7 @@ class AppAnnotationSettingUpdateApi(Resource):
        result = AppAnnotationService.update_app_annotation_setting(
            str(app_id), annotation_setting_id_str, setting_args
        )
-        return result, 200
+        return dump_response(AnnotationSettingResponse, result), 200


@console_ns.route("/apps/<uuid:app_id>/annotation-reply/<string:action>/status/<uuid:job_id>")
@@ -227,9 +246,7 @@ class AnnotationReplyActionStatusApi(Resource):
    @console_ns.doc(description="Get status of annotation reply action job")
    @console_ns.doc(params={"app_id": "Application ID", "job_id": "Job ID", "action": "Action type"})
    @console_ns.response(
-        200,
-        "Job status retrieved successfully",
-        console_ns.models[AnnotationJobStatusResponse.__name__],
+        200, "Job status retrieved successfully", console_ns.models[AnnotationJobStatusDetailResponse.__name__]
    )
    @console_ns.response(403, "Insufficient permissions")
    @setup_required
@@ -251,7 +268,9 @@ class AnnotationReplyActionStatusApi(Resource):
            app_annotation_error_key = f"{action}_app_annotation_error_{job_id_str}"
            error_msg = redis_client.get(app_annotation_error_key).decode()

-        return {"job_id": job_id_str, "job_status": job_status, "error_msg": error_msg}, 200
+        return AnnotationJobStatusDetailResponse(
+            job_id=job_id_str, job_status=job_status, error_msg=error_msg
+        ).model_dump(mode="json"), 200


@console_ns.route("/apps/<uuid:app_id>/annotations")
@@ -275,14 +294,9 @@ class AnnotationApi(Resource):

        annotation_list, total = AppAnnotationService.get_annotation_list_by_app_id(str(app_id), page, limit, keyword)
        annotation_models = TypeAdapter(list[Annotation]).validate_python(annotation_list, from_attributes=True)
-        response = AnnotationList(
-            data=annotation_models,
-            has_more=len(annotation_list) == limit,
-            limit=limit,
-            total=total,
-            page=page,
-        )
-        return response.model_dump(mode="json"), 200
+        return AnnotationList(
+            data=annotation_models, has_more=len(annotation_list) == limit, limit=limit, total=total, page=page
+        ).model_dump(mode="json"), 200

    @console_ns.doc("create_annotation")
    @console_ns.doc(description="Create a new annotation for an app")
@@ -308,7 +322,7 @@ class AnnotationApi(Resource):
        if args.question is not None:
            upsert_args["question"] = args.question
        annotation = AppAnnotationService.up_insert_app_annotation_from_message(upsert_args, str(app_id))
-        return Annotation.model_validate(annotation, from_attributes=True).model_dump(mode="json")
+        return dump_response(Annotation, annotation), 201

    @setup_required
    @login_required
@@ -330,7 +344,8 @@ class AnnotationApi(Resource):
                    "message": "annotation_ids are required if the parameter is provided.",
                }, 400

-            AppAnnotationService.delete_app_annotations_in_batch(str(app_id), annotation_ids)
+            app_ref = _get_app_ref(str(app_id))
+            AppAnnotationService.delete_app_annotations_in_batch(app_ref, annotation_ids)
            return "", 204
        # If no annotation_ids are provided, handle clearing all annotations
        else:
@@ -357,14 +372,14 @@ class AnnotationExportApi(Resource):
    def get(self, app_id: UUID):
        annotation_list = AppAnnotationService.export_annotation_list_by_app_id(str(app_id))
        annotation_models = TypeAdapter(list[Annotation]).validate_python(annotation_list, from_attributes=True)
-        response_data = AnnotationExportList(data=annotation_models).model_dump(mode="json")
-
-        # Create response with secure headers for CSV export
-        response = make_response(response_data, 200)
-        response.headers["Content-Type"] = "application/json; charset=utf-8"
-        response.headers["X-Content-Type-Options"] = "nosniff"
-
-        return response
+        return (
+            AnnotationExportList(data=annotation_models).model_dump(mode="json"),
+            200,
+            {
+                "Content-Type": "application/json; charset=utf-8",
+                "X-Content-Type-Options": "nosniff",
+            },
+        )


@console_ns.route("/apps/<uuid:app_id>/annotations/<uuid:annotation_id>")
@@ -389,9 +404,9 @@ class AnnotationUpdateDeleteApi(Resource):
            update_args["answer"] = args.answer
        if args.question is not None:
            update_args["question"] = args.question
-        annotation = AppAnnotationService.update_app_annotation_directly(
-            update_args, str(app_id), str(annotation_id), db.session
-        )
+        app_ref = _get_app_ref(str(app_id))
+        annotation_ref = AppRefService.create_annotation_ref(app_ref, str(annotation_id))
+        annotation = AppAnnotationService.update_app_annotation_directly(update_args, annotation_ref, db.session)
        return Annotation.model_validate(annotation, from_attributes=True).model_dump(mode="json")

    @setup_required
@@ -401,7 +416,9 @@ class AnnotationUpdateDeleteApi(Resource):
    @rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_EDIT)
    @console_ns.response(204, "Annotation deleted successfully")
    def delete(self, app_id: UUID, annotation_id: UUID):
-        AppAnnotationService.delete_app_annotation(str(app_id), str(annotation_id), db.session)
+        app_ref = _get_app_ref(str(app_id))
+        annotation_ref = AppRefService.create_annotation_ref(app_ref, str(annotation_id))
+        AppAnnotationService.delete_app_annotation(annotation_ref, db.session)
        return "", 204


@@ -411,9 +428,7 @@ class AnnotationBatchImportApi(Resource):
    @console_ns.doc(description="Batch import annotations from CSV file with rate limiting and security checks")
    @console_ns.doc(params={"app_id": "Application ID"})
    @console_ns.response(
-        200,
-        "Batch import started successfully",
-        console_ns.models[AnnotationJobStatusResponse.__name__],
+        200, "Batch import started successfully", console_ns.models[AnnotationBatchImportResponse.__name__]
    )
    @console_ns.response(403, "Insufficient permissions")
    @console_ns.response(400, "No file uploaded or too many files")
@@ -460,7 +475,10 @@ class AnnotationBatchImportApi(Resource):
        if file_size == 0:
            raise ValueError("The uploaded file is empty")

-        return AppAnnotationService.batch_import_app_annotations(str(app_id), file)
+        return dump_response(
+            AnnotationBatchImportResponse,
+            AppAnnotationService.batch_import_app_annotations(str(app_id), file),
+        )


@console_ns.route("/apps/<uuid:app_id>/annotations/batch-import-status/<uuid:job_id>")
@@ -469,9 +487,7 @@ class AnnotationBatchImportStatusApi(Resource):
    @console_ns.doc(description="Get status of batch import job")
    @console_ns.doc(params={"app_id": "Application ID", "job_id": "Job ID"})
    @console_ns.response(
-        200,
-        "Job status retrieved successfully",
-        console_ns.models[AnnotationJobStatusResponse.__name__],
+        200, "Job status retrieved successfully", console_ns.models[AnnotationJobStatusDetailResponse.__name__]
    )
    @console_ns.response(403, "Insufficient permissions")
    @setup_required
@@ -491,7 +507,9 @@ class AnnotationBatchImportStatusApi(Resource):
            indexing_error_msg_key = f"app_annotation_batch_import_error_msg_{str(job_id)}"
            error_msg = redis_client.get(indexing_error_msg_key).decode()

-        return {"job_id": job_id, "job_status": job_status, "error_msg": error_msg}, 200
+        return AnnotationJobStatusDetailResponse(
+            job_id=str(job_id), job_status=job_status, error_msg=error_msg
+        ).model_dump(mode="json"), 200


@console_ns.route("/apps/<uuid:app_id>/annotations/<uuid:annotation_id>/hit-histories")
@@ -514,17 +532,16 @@ class AnnotationHitHistoryListApi(Resource):
    def get(self, app_id: UUID, annotation_id: UUID):
        page = request.args.get("page", default=1, type=int)
        limit = request.args.get("limit", default=20, type=int)
+        app_ref = _get_app_ref(str(app_id))
+        annotation_ref = AppRefService.create_annotation_ref(app_ref, str(annotation_id))
        annotation_hit_history_list, total = AppAnnotationService.get_annotation_hit_histories(
-            str(app_id), str(annotation_id), page, limit
+            annotation_ref,
+            page,
+            limit,
        )
        history_models = TypeAdapter(list[AnnotationHitHistory]).validate_python(
            annotation_hit_history_list, from_attributes=True
        )
-        response = AnnotationHitHistoryList(
-            data=history_models,
-            has_more=len(annotation_hit_history_list) == limit,
-            limit=limit,
-            total=total,
-            page=page,
-        )
-        return response.model_dump(mode="json")
+        return AnnotationHitHistoryList(
+            data=history_models, has_more=len(annotation_hit_history_list) == limit, limit=limit, total=total, page=page
+        ).model_dump(mode="json")
@@ -1,5 +1,4 @@
 import logging
-import re
 import uuid
 from collections.abc import Sequence
 from datetime import datetime
@@ -10,7 +9,6 @@ from flask_restx import Resource
 from pydantic import AliasChoices, BaseModel, Field, computed_field, field_validator
 from sqlalchemy import select
 from sqlalchemy.orm import Session
-from werkzeug.datastructures import MultiDict
 from werkzeug.exceptions import BadRequest, NotFound

 from configs import dify_config
@@ -19,6 +17,7 @@ from controllers.common.fields import RedirectUrlResponse, SimpleResultResponse
 from controllers.common.helpers import FileInfo
 from controllers.common.schema import (
    query_params_from_model,
+    query_params_from_request,
    register_enum_models,
    register_response_schema_models,
    register_schema_models,
@@ -43,7 +42,6 @@ from controllers.console.wraps import (
 from core.ops.ops_trace_manager import OpsTraceManager
 from core.rag.entities import PreProcessingRule, Rule, Segmentation
 from core.rag.retrieval.retrieval_methods import RetrievalMethod
-from core.rbac import RBACResourceWhitelistScope
 from core.trigger.constants import TRIGGER_NODE_TYPES
 from extensions.ext_database import db
 from fields.base import ResponseModel
@@ -70,17 +68,15 @@ from services.entities.knowledge_entities.knowledge_entities import (
    WeightVectorSetting,
 )
 from services.feature_service import FeatureService
-from tasks.initialize_created_app_rbac_access_task import initialize_created_app_rbac_access_task

 ALLOW_CREATE_APP_MODES = ["chat", "agent-chat", "advanced-chat", "workflow", "completion"]

 register_enum_models(console_ns, IconType)

 _logger = logging.getLogger(__name__)
-_TAG_IDS_BRACKET_PATTERN = re.compile(r"^tag_ids\[(\d+)\]$")
-_CREATOR_IDS_BRACKET_PATTERN = re.compile(r"^creator_ids\[(\d+)\]$")
 AppListMode = Literal["completion", "chat", "advanced-chat", "workflow", "agent-chat", "agent", "channel", "all"]
 DEFAULT_APP_LIST_MODE: AppListMode = "all"
+APP_LIST_QUERY_ARRAY_FIELDS = ("tag_ids", "creator_ids")


 class AppListBaseQuery(BaseModel):
@@ -141,34 +137,6 @@ class StarredAppListQuery(AppListBaseQuery):
    pass


-def _normalize_app_list_query_args(query_args: MultiDict[str, str]) -> dict[str, str | list[str]]:
-    normalized: dict[str, str | list[str]] = {}
-    indexed_tag_ids: list[tuple[int, str]] = []
-    indexed_creator_ids: list[tuple[int, str]] = []
-
-    for key in query_args:
-        match = _TAG_IDS_BRACKET_PATTERN.fullmatch(key)
-        if match:
-            indexed_tag_ids.extend((int(match.group(1)), value) for value in query_args.getlist(key))
-            continue
-
-        match = _CREATOR_IDS_BRACKET_PATTERN.fullmatch(key)
-        if match:
-            indexed_creator_ids.extend((int(match.group(1)), value) for value in query_args.getlist(key))
-            continue
-
-        value = query_args.get(key)
-        if value is not None:
-            normalized[key] = value
-
-    if indexed_tag_ids:
-        normalized["tag_ids"] = [value for _, value in sorted(indexed_tag_ids)]
-    if indexed_creator_ids:
-        normalized["creator_ids"] = [value for _, value in sorted(indexed_creator_ids)]
-
-    return normalized
-
-
 class CreateAppPayload(BaseModel):
    name: str = Field(..., min_length=1, description="App name")
    description: str | None = Field(default=None, description="App description (max 400 chars)", max_length=400)
@@ -232,13 +200,10 @@ class AppTracePayload(BaseModel):


 class AppTraceResponse(ResponseModel):
-    enabled: bool
+    enabled: bool = False
    tracing_provider: str | None = None


-type JSONValue = Any
-
-
 class Tag(ResponseModel):
    id: str
    name: str
@@ -259,7 +224,7 @@ class WorkflowPartial(ResponseModel):


 class ModelConfigPartial(ResponseModel):
-    model: JSONValue | None = Field(default=None, validation_alias=AliasChoices("model_dict", "model"))
+    model: Any | None = Field(default=None, validation_alias=AliasChoices("model_dict", "model"))
    pre_prompt: str | None = None
    created_by: str | None = None
    created_at: int | None = None
@@ -274,54 +239,52 @@ class ModelConfigPartial(ResponseModel):

 class ModelConfig(ResponseModel):
    opening_statement: str | None = None
-    suggested_questions: JSONValue | None = Field(
+    suggested_questions: Any | None = Field(
        default=None, validation_alias=AliasChoices("suggested_questions_list", "suggested_questions")
    )
-    suggested_questions_after_answer: JSONValue | None = Field(
+    suggested_questions_after_answer: Any | None = Field(
        default=None,
        validation_alias=AliasChoices("suggested_questions_after_answer_dict", "suggested_questions_after_answer"),
    )
-    speech_to_text: JSONValue | None = Field(
+    speech_to_text: Any | None = Field(
        default=None, validation_alias=AliasChoices("speech_to_text_dict", "speech_to_text")
    )
-    text_to_speech: JSONValue | None = Field(
+    text_to_speech: Any | None = Field(
        default=None, validation_alias=AliasChoices("text_to_speech_dict", "text_to_speech")
    )
-    retriever_resource: JSONValue | None = Field(
+    retriever_resource: Any | None = Field(
        default=None, validation_alias=AliasChoices("retriever_resource_dict", "retriever_resource")
    )
-    annotation_reply: JSONValue | None = Field(
+    annotation_reply: Any | None = Field(
        default=None, validation_alias=AliasChoices("annotation_reply_dict", "annotation_reply")
    )
-    more_like_this: JSONValue | None = Field(
+    more_like_this: Any | None = Field(
        default=None, validation_alias=AliasChoices("more_like_this_dict", "more_like_this")
    )
-    sensitive_word_avoidance: JSONValue | None = Field(
+    sensitive_word_avoidance: Any | None = Field(
        default=None, validation_alias=AliasChoices("sensitive_word_avoidance_dict", "sensitive_word_avoidance")
    )
-    external_data_tools: JSONValue | None = Field(
+    external_data_tools: Any | None = Field(
        default=None, validation_alias=AliasChoices("external_data_tools_list", "external_data_tools")
    )
-    model: JSONValue | None = Field(default=None, validation_alias=AliasChoices("model_dict", "model"))
-    user_input_form: JSONValue | None = Field(
+    model: Any | None = Field(default=None, validation_alias=AliasChoices("model_dict", "model"))
+    user_input_form: Any | None = Field(
        default=None, validation_alias=AliasChoices("user_input_form_list", "user_input_form")
    )
    dataset_query_variable: str | None = None
    pre_prompt: str | None = None
-    agent_mode: JSONValue | None = Field(default=None, validation_alias=AliasChoices("agent_mode_dict", "agent_mode"))
+    agent_mode: Any | None = Field(default=None, validation_alias=AliasChoices("agent_mode_dict", "agent_mode"))
    prompt_type: str | None = None
-    chat_prompt_config: JSONValue | None = Field(
+    chat_prompt_config: Any | None = Field(
        default=None, validation_alias=AliasChoices("chat_prompt_config_dict", "chat_prompt_config")
    )
-    completion_prompt_config: JSONValue | None = Field(
+    completion_prompt_config: Any | None = Field(
        default=None, validation_alias=AliasChoices("completion_prompt_config_dict", "completion_prompt_config")
    )
-    dataset_configs: JSONValue | None = Field(
+    dataset_configs: Any | None = Field(
        default=None, validation_alias=AliasChoices("dataset_configs_dict", "dataset_configs")
    )
-    file_upload: JSONValue | None = Field(
-        default=None, validation_alias=AliasChoices("file_upload_dict", "file_upload")
-    )
+    file_upload: Any | None = Field(default=None, validation_alias=AliasChoices("file_upload_dict", "file_upload"))
    created_by: str | None = None
    created_at: int | None = None
    updated_by: str | None = None
@@ -333,7 +296,7 @@ class ModelConfig(ResponseModel):
        return to_timestamp(value)


-class Site(ResponseModel):
+class AppDetailSiteResponse(ResponseModel):
    access_token: str | None = Field(default=None, validation_alias="code")
    code: str | None = None
    title: str | None = None
@@ -347,6 +310,7 @@ class Site(ResponseModel):
    customize_domain: str | None = None
    copyright: str | None = None
    privacy_policy: str | None = None
+    input_placeholder: str | None = None
    custom_disclaimer: str | None = None
    customize_token_strategy: str | None = None
    prompt_public: bool | None = None
@@ -441,7 +405,7 @@ class AppDetail(ResponseModel):
        alias="model_config",
    )
    workflow: WorkflowPartial | None = None
-    tracing: JSONValue | None = None
+    tracing: Any | None = None
    use_icon_as_answer_icon: bool | None = None
    created_by: str | None = None
    created_at: int | None = None
@@ -463,7 +427,7 @@ class AppDetailWithSite(AppDetail):
    api_base_url: str | None = None
    max_active_requests: int | None = None
    deleted_tools: list[DeletedTool] = Field(default_factory=list)
-    site: Site | None = None
+    site: AppDetailSiteResponse | None = None
    # For Agent App type: the roster Agent backing this app (None otherwise).
    bound_agent_id: str | None = None
    # For Agent App responses exposed through /agent.
@@ -487,6 +451,16 @@ class AppExportResponse(ResponseModel):
    data: str


+class AppImportResponse(ResponseModel):
+    id: str
+    status: ImportStatus
+    app_id: str | None = None
+    app_mode: str | None = None
+    current_dsl_version: str
+    imported_dsl_version: str = ""
+    error: str = ""
+
+
 def _enrich_app_list_items(session: Session, *, apps: Sequence[App], tenant_id: str) -> None:
    if FeatureService.get_system_features().webapp_auth.enabled:
        app_ids = [str(app.id) for app in apps]
@@ -529,7 +503,9 @@ def _enrich_app_list_items(session: Session, *, apps: Sequence[App], tenant_id:


 register_enum_models(console_ns, RetrievalMethod, WorkflowExecutionStatus, DatasetPermissionEnum)
-register_response_schema_models(console_ns, AppTraceResponse, RedirectUrlResponse, SimpleResultResponse)
+register_response_schema_models(
+    console_ns, RedirectUrlResponse, SimpleResultResponse, AppImportResponse, AppTraceResponse
+)

 register_schema_models(
    console_ns,
@@ -548,7 +524,7 @@ register_schema_models(
    WorkflowPartial,
    ModelConfigPartial,
    ModelConfig,
-    Site,
+    AppDetailSiteResponse,
    DeletedTool,
    AppDetail,
    AppExportResponse,
@@ -593,7 +569,7 @@ class AppListApi(Resource):
    @with_current_tenant_id
    def get(self, current_tenant_id: str, current_user_id: str, session: Session):
        """Get app list"""
-        args = AppListQuery.model_validate(_normalize_app_list_query_args(request.args))
+        args = query_params_from_request(AppListQuery, list_fields=APP_LIST_QUERY_ARRAY_FIELDS)
        params = AppListParams(
            page=args.page,
            limit=args.limit,
@@ -621,8 +597,8 @@ class AppListApi(Resource):
        app_service = AppService()
        app_pagination = app_service.get_paginate_apps(current_user_id, current_tenant_id, params, db.session)
        if not app_pagination:
-            empty = AppPagination(page=args.page, limit=args.limit, total=0, has_more=False, data=[])
-            return empty.model_dump(mode="json"), 200
+            response = AppPagination(page=args.page, limit=args.limit, total=0, has_more=False, data=[])
+            return response.model_dump(mode="json"), 200

        app_ids = [str(app.id) for app in app_pagination.items]
        permission_keys_map = permissions.app.permission_keys_by_resource_ids(app_ids)
@@ -668,14 +644,6 @@ class AppListApi(Resource):

        app_service = AppService()
        app = app_service.create_app(current_tenant_id, params, current_user)
-        if dify_config.RBAC_ENABLED:
-            enterprise_rbac_service.RBACService.AppAccess.replace_whitelist(
-                tenant_id=str(current_tenant_id),
-                account_id=current_user.id,
-                app_id=str(app.id),
-                payload=enterprise_rbac_service.ReplaceMemberBindings(scope=RBACResourceWhitelistScope.ALL),
-            )
-            initialize_created_app_rbac_access_task.delay(current_tenant_id, current_user.id, app_id=app.id)
        permission_keys_map = enterprise_rbac_service.RBACService.AppPermissions.batch_get(
            str(current_tenant_id),
            current_user.id,
@@ -701,7 +669,7 @@ class StarredAppListApi(Resource):
    @with_current_user_id
    @with_current_tenant_id
    def get(self, current_tenant_id: str, current_user_id: str, session: Session):
-        args = StarredAppListQuery.model_validate(_normalize_app_list_query_args(request.args))
+        args = query_params_from_request(StarredAppListQuery, list_fields=APP_LIST_QUERY_ARRAY_FIELDS)
        params = StarredAppListParams(
            page=args.page,
            limit=args.limit,
@@ -719,9 +687,7 @@ class StarredAppListApi(Resource):
            return empty.model_dump(mode="json"), 200

        _enrich_app_list_items(session, apps=app_pagination.items, tenant_id=current_tenant_id)
-
-        pagination_model = AppPagination.model_validate(app_pagination, from_attributes=True)
-        return pagination_model.model_dump(mode="json"), 200
+        return AppPagination.model_validate(app_pagination, from_attributes=True).model_dump(mode="json"), 200


@console_ns.route("/apps/<uuid:app_id>/star")
@@ -740,7 +706,7 @@ class AppStarApi(Resource):
    @get_app_model(mode=None)
    def post(self, session: Session, current_user_id: str, app_model: App):
        AppService.star_app(session, app=app_model, account_id=current_user_id)
-        return dump_response(SimpleResultResponse, {"result": "success"})
+        return SimpleResultResponse(result="success").model_dump(mode="json")

    @console_ns.doc("unstar_app")
    @console_ns.doc(description="Remove the current account's star from an application")
@@ -756,7 +722,7 @@ class AppStarApi(Resource):
    @get_app_model(mode=None)
    def delete(self, session: Session, current_user_id: str, app_model: App):
        AppService.unstar_app(session, app=app_model, account_id=current_user_id)
-        return dump_response(SimpleResultResponse, {"result": "success"})
+        return SimpleResultResponse(result="success").model_dump(mode="json")


@console_ns.route("/apps/<uuid:app_id>")
@@ -824,8 +790,7 @@ class AppApi(Resource):
            "max_active_requests": args.max_active_requests or 0,
        }
        app_model = app_service.update_app(app_model, args_dict)
-        response_model = AppDetailWithSite.model_validate(app_model, from_attributes=True)
-        return response_model.model_dump(mode="json")
+        return dump_response(AppDetailWithSite, app_model)

    @console_ns.doc("delete_app")
    @console_ns.doc(description="Delete application")
@@ -853,6 +818,7 @@ class AppCopyApi(Resource):
    @console_ns.doc(params={"app_id": "Application ID to copy"})
    @console_ns.expect(console_ns.models[CopyAppPayload.__name__])
    @console_ns.response(201, "App copied successfully", console_ns.models[AppDetailWithSite.__name__])
+    @console_ns.response(202, "App copy requires confirmation", console_ns.models[AppImportResponse.__name__])
    @console_ns.response(403, "Insufficient permissions")
    @setup_required
    @login_required
@@ -882,10 +848,10 @@ class AppCopyApi(Resource):
            )
            if result.status == ImportStatus.FAILED:
                session.rollback()
-                return result.model_dump(mode="json"), 400
+                return dump_response(AppImportResponse, result), 400
            if result.status == ImportStatus.PENDING:
                session.rollback()
-                return result.model_dump(mode="json"), 202
+                return dump_response(AppImportResponse, result), 202
            session.commit()

            # Inherit web app permission from original app
@@ -936,14 +902,14 @@ class AppExportApi(Resource):
        """Export app"""
        args = AppExportQuery.model_validate(request.args.to_dict(flat=True))

-        payload = AppExportResponse(
+        response = AppExportResponse(
            data=AppDslService.export_dsl(
                app_model=app_model,
                include_secret=args.include_secret,
                workflow_id=args.workflow_id,
            )
        )
-        return payload.model_dump(mode="json")
+        return response.model_dump(mode="json")


@console_ns.route("/apps/<uuid:app_id>/publish-to-creators-platform")
@@ -969,7 +935,7 @@ class AppPublishToCreatorsPlatformApi(Resource):
        claim_code = upload_dsl(dsl_bytes)
        redirect_url = get_redirect_url(current_user_id, claim_code)

-        return {"redirect_url": redirect_url}
+        return RedirectUrlResponse(redirect_url=redirect_url).model_dump(mode="json")


@console_ns.route("/apps/<uuid:app_id>/name")
@@ -990,8 +956,7 @@ class AppNameApi(Resource):

        app_service = AppService()
        app_model = app_service.update_app_name(app_model, args.name)
-        response_model = AppDetail.model_validate(app_model, from_attributes=True)
-        return response_model.model_dump(mode="json")
+        return dump_response(AppDetail, app_model)


@console_ns.route("/apps/<uuid:app_id>/icon")
@@ -1018,8 +983,7 @@ class AppIconApi(Resource):
            args.icon_background or "",
            args.icon_type,
        )
-        response_model = AppDetail.model_validate(app_model, from_attributes=True)
-        return response_model.model_dump(mode="json")
+        return dump_response(AppDetail, app_model)


@console_ns.route("/apps/<uuid:app_id>/site-enable")
@@ -1041,8 +1005,7 @@ class AppSiteStatus(Resource):

        app_service = AppService()
        app_model = app_service.update_app_site_status(app_model, args.enable_site)
-        response_model = AppDetail.model_validate(app_model, from_attributes=True)
-        return response_model.model_dump(mode="json")
+        return dump_response(AppDetail, app_model)


@console_ns.route("/apps/<uuid:app_id>/api-enable")
@@ -1064,8 +1027,7 @@ class AppApiStatus(Resource):

        app_service = AppService()
        app_model = app_service.update_app_api_status(app_model, args.enable_api)
-        response_model = AppDetail.model_validate(app_model, from_attributes=True)
-        return response_model.model_dump(mode="json")
+        return dump_response(AppDetail, app_model)


@console_ns.route("/apps/<uuid:app_id>/trace")
@@ -1088,7 +1050,7 @@ class AppTraceApi(Resource):
        """Get app trace"""
        app_trace_config = OpsTraceManager.get_app_tracing_config(app_model.id, session)

-        return app_trace_config
+        return dump_response(AppTraceResponse, app_trace_config)

    @console_ns.doc("update_app_trace")
    @console_ns.doc(description="Update app tracing configuration")
@@ -1116,4 +1078,4 @@ class AppTraceApi(Resource):
            tracing_provider=args.tracing_provider,
        )

-        return {"result": "success"}
+        return SimpleResultResponse(result="success").model_dump(mode="json")
@@ -1,5 +1,4 @@
 import logging
-from typing import Any

 from flask import request
 from flask_restx import Resource
@@ -7,7 +6,6 @@ from pydantic import BaseModel, Field, RootModel
 from werkzeug.exceptions import InternalServerError

 import services
-from controllers.common.fields import AudioBinaryResponse
 from controllers.common.schema import query_params_from_model, register_response_schema_models, register_schema_models
 from controllers.console import console_ns
 from controllers.console.app.error import (
@@ -31,9 +29,12 @@ from controllers.console.wraps import (
 )
 from core.errors.error import ModelCurrentlyNotSupportError, ProviderTokenNotInitError, QuotaExceededError
 from extensions.ext_database import db
+from fields.base import ResponseModel
 from graphon.model_runtime.errors.invoke import InvokeError
-from libs.login import login_required
+from libs.helper import dump_response
+from libs.login import current_user, login_required
 from models import App, AppMode
+from services.app_ref_service import AppRefService
 from services.audio_service import AudioService
 from services.errors.audio import (
    AudioTooLargeServiceError,
@@ -56,16 +57,27 @@ class TextToSpeechVoiceQuery(BaseModel):
    language: str = Field(..., description="Language code")


-class AudioTranscriptResponse(BaseModel):
+class AudioTranscriptResponse(ResponseModel):
    text: str = Field(description="Transcribed text from audio")


-class TextToSpeechVoiceListResponse(RootModel[list[dict[str, Any]]]):
-    root: list[dict[str, Any]]
+class TextToSpeechVoiceResponse(ResponseModel):
+    # see api/core/plugin/impl/model.py
+    name: str = Field(description="Voice display name")
+    value: str = Field(description="Voice identifier")


-register_schema_models(console_ns, AudioTranscriptResponse, TextToSpeechPayload, TextToSpeechVoiceQuery)
-register_response_schema_models(console_ns, AudioBinaryResponse, TextToSpeechVoiceListResponse)
+class TextToSpeechVoiceListResponse(RootModel[list[TextToSpeechVoiceResponse]]):
+    root: list[TextToSpeechVoiceResponse] = Field(description="Available voices")
+
+
+register_schema_models(console_ns, TextToSpeechPayload, TextToSpeechVoiceQuery)
+register_response_schema_models(
+    console_ns,
+    AudioTranscriptResponse,
+    TextToSpeechVoiceResponse,
+    TextToSpeechVoiceListResponse,
+)


@console_ns.route("/apps/<uuid:app_id>/audio-to-text")
@@ -94,7 +106,7 @@ class ChatMessageAudioApi(Resource):
                end_user=None,
            )

-            return response
+            return dump_response(AudioTranscriptResponse, response)
        except services.errors.app_model_config.AppModelConfigBrokenError:
            logger.exception("App model config broken.")
            raise AppUnavailableError()
@@ -127,11 +139,8 @@ class ChatMessageTextApi(Resource):
    @console_ns.doc(description="Convert text to speech for chat messages")
    @console_ns.doc(params={"app_id": "App ID"})
    @console_ns.expect(console_ns.models[TextToSpeechPayload.__name__])
-    @console_ns.response(
-        200,
-        "Text to speech conversion successful",
-        console_ns.models[AudioBinaryResponse.__name__],
-    )
+    # TTS returns provider audio bytes, so the success response is intentionally schema-less.
+    @console_ns.response(200, "Text to speech conversion successful")
    @console_ns.response(400, "Bad request - Invalid parameters")
    @setup_required
    @login_required
@@ -140,16 +149,24 @@ class ChatMessageTextApi(Resource):
    def post(self, app_model: App):
        try:
            payload = TextToSpeechPayload.model_validate(console_ns.payload)
+            message_ref = None
+            if payload.message_id:
+                app_ref = AppRefService.create_app_ref(app_model)
+                message_ref = AppRefService.create_message_ref(
+                    app_ref,
+                    payload.message_id,
+                    account_id=current_user.id,
+                )

-            response = AudioService.transcript_tts(
+            # response-contract:ignore
+            return AudioService.transcript_tts(
                app_model=app_model,
                session=db.session,
                text=payload.text,
                voice=payload.voice,
-                message_id=payload.message_id,
+                message_ref=message_ref,
                is_draft=True,
            )
-            return response
        except services.errors.app_model_config.AppModelConfigBrokenError:
            logger.exception("App model config broken.")
            raise AppUnavailableError()
@@ -180,8 +197,7 @@ class ChatMessageTextApi(Resource):
 class TextModesApi(Resource):
    @console_ns.doc("get_text_to_speech_voices")
    @console_ns.doc(description="Get available TTS voices for a specific language")
-    @console_ns.doc(params={"app_id": "App ID"})
-    @console_ns.doc(params=query_params_from_model(TextToSpeechVoiceQuery))
+    @console_ns.doc(params={"app_id": "App ID", **query_params_from_model(TextToSpeechVoiceQuery)})
    @console_ns.response(
        200,
        "TTS voices retrieved successfully",
@@ -202,7 +218,7 @@ class TextModesApi(Resource):
                language=args.language,
            )

-            return response
+            return dump_response(TextToSpeechVoiceListResponse, response)
        except services.errors.audio.ProviderNotSupportTextToSpeechLanageServiceError:
            raise AppUnavailableError("Text to audio voices language parameter loss.")
        except NoAudioUploadedServiceError:
@@ -1,17 +1,20 @@
+import json
 import logging
-from typing import Any, Literal
+from collections.abc import Generator, Iterator, Mapping
+from typing import Any, Literal, Protocol, runtime_checkable
 from uuid import UUID

 from flask import request
 from flask_restx import Resource
 from pydantic import BaseModel, Field, field_validator
+from sqlalchemy.orm import Session
 from werkzeug.exceptions import BadRequest, InternalServerError, NotFound

 import services
-from controllers.common.fields import GeneratedAppResponse, SimpleResultResponse
+from controllers.common.fields import SimpleResultResponse
 from controllers.common.schema import register_response_schema_models, register_schema_models
 from controllers.console import console_ns
-from controllers.console.agent.app_helpers import resolve_agent_app_model
+from controllers.console.agent.app_helpers import resolve_agent_runtime_app_model
 from controllers.console.app.error import (
    AppUnavailableError,
    CompletionRequestError,
@@ -20,7 +23,7 @@ from controllers.console.app.error import (
    ProviderNotInitializeError,
    ProviderQuotaExceededError,
 )
-from controllers.console.app.wraps import get_app_model
+from controllers.console.app.wraps import get_app_model, with_session
 from controllers.console.wraps import (
    RBACPermission,
    RBACResourceScope,
@@ -34,6 +37,7 @@ from controllers.console.wraps import (
 )
 from controllers.web.error import InvokeRateLimitError as InvokeRateLimitHttpError
 from core.app.entities.app_invoke_entities import InvokeFrom
+from core.app.features.rate_limiting.rate_limit import RateLimitGenerator
 from core.errors.error import (
    ModelCurrentlyNotSupportError,
    ProviderTokenNotInitError,
@@ -56,6 +60,11 @@ from services.errors.llm import InvokeRateLimitError
 logger = logging.getLogger(__name__)


+@runtime_checkable
+class _ClosableStream(Protocol):
+    def close(self) -> None: ...
+
+
 def _resolve_debugger_chat_streaming(
    *, app_mode: AppMode, response_mode: str, response_mode_provided: bool = True
 ) -> bool:
@@ -93,6 +102,10 @@ class ChatMessagePayload(BaseMessagePayload):
    query: str = Field(..., description="User query")
    conversation_id: str | None = Field(default=None, description="Conversation ID")
    parent_message_id: str | None = Field(default=None, description="Parent message ID")
+    draft_type: Literal["draft", "debug_build"] = Field(
+        default="draft",
+        description="Agent App debug config source. Use debug_build while the Agent is in build mode.",
+    )

    @field_validator("conversation_id", "parent_message_id")
    @classmethod
@@ -102,8 +115,31 @@ class ChatMessagePayload(BaseMessagePayload):
        return uuid_value(value)


+_BUILD_CHAT_FINALIZATION_QUERY = """Finalize this Build chat configuration for the agent.
+
+This step is only for persisting Agent config changes discovered in the current Build chat. Do not install packages,
+edit workspace files, run validation or debugging commands, make exploratory checks, or perform other work.
+
+Use only the current Build chat message history to identify changes that need to be persisted. Do not inspect, test, or
+validate old config unless the message history already shows that the old config is invalid.
+
+Only update the build-draft config note when the current Build chat contains durable context that later runs need.
+Do not create, update, delete, inspect, or fill gaps in other Agent config resources, including config files, config
+skills, config env, tools, models, knowledge, or prompt settings.
+
+When updating the config note with the Agent config CLI usage provided in the runtime prompt, record only durable
+context needed by later runs, such as:
+
+- what you installed or configured outside the workspace for this agent,
+- where those external updates live, including CLI tools, packages, and persistent $HOME paths,
+- how the agent should use it in later runs,
+- any setup, authentication, or user action still required.
+
+After config persistence completes, respond FINISHED."""
+
+
 register_schema_models(console_ns, CompletionMessagePayload, ChatMessagePayload)
-register_response_schema_models(console_ns, GeneratedAppResponse, SimpleResultResponse)
+register_response_schema_models(console_ns, SimpleResultResponse)


 # define completion message api for user
@@ -113,7 +149,7 @@ class CompletionMessageApi(Resource):
    @console_ns.doc(description="Generate completion message for debugging")
    @console_ns.doc(params={"app_id": "Application ID"})
    @console_ns.expect(console_ns.models[CompletionMessagePayload.__name__])
-    @console_ns.response(200, "Completion generated successfully", console_ns.models[GeneratedAppResponse.__name__])
+    @console_ns.response(200, "Completion generated successfully")
    @console_ns.response(400, "Invalid request parameters")
    @console_ns.response(404, "App not found")
    @setup_required
@@ -122,7 +158,8 @@ class CompletionMessageApi(Resource):
    @with_current_user
    @rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_TEST_AND_RUN)
    @get_app_model(mode=AppMode.COMPLETION)
-    def post(self, current_user: Account, app_model: App):
+    @with_session
+    def post(self, session: Session, current_user: Account, app_model: App):
        args_model = CompletionMessagePayload.model_validate(console_ns.payload)
        args = args_model.model_dump(exclude_none=True, by_alias=True)

@@ -131,9 +168,15 @@ class CompletionMessageApi(Resource):

        try:
            response = AppGenerateService.generate(
-                app_model=app_model, user=current_user, args=args, invoke_from=InvokeFrom.DEBUGGER, streaming=streaming
+                session=session,
+                app_model=app_model,
+                user=current_user,
+                args=args,
+                invoke_from=InvokeFrom.DEBUGGER,
+                streaming=streaming,
            )

+            # response-contract:ignore compact_generate_response
            return helper.compact_generate_response(response)
        except services.errors.conversation.ConversationNotExistsError:
            raise NotFound("Conversation Not Exists.")
@@ -177,7 +220,7 @@ class CompletionMessageStopApi(Resource):
            app_mode=AppMode.value_of(app_model.mode),
        )

-        return {"result": "success"}, 200
+        return SimpleResultResponse(result="success").model_dump(mode="json"), 200


@console_ns.route("/apps/<uuid:app_id>/chat-messages")
@@ -186,7 +229,7 @@ class ChatMessageApi(Resource):
    @console_ns.doc(description="Generate chat message for debugging")
    @console_ns.doc(params={"app_id": "Application ID"})
    @console_ns.expect(console_ns.models[ChatMessagePayload.__name__])
-    @console_ns.response(200, "Chat message generated successfully", console_ns.models[GeneratedAppResponse.__name__])
+    @console_ns.response(200, "Chat message generated successfully")
    @console_ns.response(400, "Invalid request parameters")
    @console_ns.response(404, "App or conversation not found")
    @setup_required
@@ -197,8 +240,11 @@ class ChatMessageApi(Resource):
    @with_current_tenant_id
    @rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_TEST_AND_RUN)
    @get_app_model(mode=[AppMode.CHAT, AppMode.AGENT_CHAT, AppMode.AGENT])
-    def post(self, current_tenant_id: str, current_user: Account, app_model: App):
-        return _create_chat_message(current_tenant_id=current_tenant_id, current_user=current_user, app_model=app_model)
+    @with_session
+    def post(self, session: Session, current_tenant_id: str, current_user: Account, app_model: App):
+        return _create_chat_message(
+            session=session, current_tenant_id=current_tenant_id, current_user=current_user, app_model=app_model
+        )


@console_ns.route("/agent/<uuid:agent_id>/chat-messages")
@@ -207,7 +253,7 @@ class AgentChatMessageApi(Resource):
    @console_ns.doc(description="Generate an Agent App chat message for debugging")
    @console_ns.doc(params={"agent_id": "Agent ID"})
    @console_ns.expect(console_ns.models[ChatMessagePayload.__name__])
-    @console_ns.response(200, "Chat message generated successfully", console_ns.models[GeneratedAppResponse.__name__])
+    @console_ns.response(200, "Chat message generated successfully")
    @console_ns.response(400, "Invalid request parameters")
    @console_ns.response(404, "Agent or conversation not found")
    @setup_required
@@ -217,9 +263,38 @@ class AgentChatMessageApi(Resource):
    @rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_TEST_AND_RUN)
    @with_current_user
    @with_current_tenant_id
-    def post(self, current_tenant_id: str, current_user: Account, agent_id: UUID):
-        app_model = resolve_agent_app_model(tenant_id=current_tenant_id, agent_id=agent_id)
+    @with_session
+    def post(self, session: Session, current_tenant_id: str, current_user: Account, agent_id: UUID):
+        app_model = resolve_agent_runtime_app_model(tenant_id=current_tenant_id, agent_id=agent_id)
        return _create_chat_message(
+            session=session,
+            current_tenant_id=current_tenant_id,
+            current_user=current_user,
+            app_model=app_model,
+            agent_id=str(agent_id),
+        )
+
+
+@console_ns.route("/agent/<uuid:agent_id>/build-chat/finalize")
+class AgentBuildChatFinalizeApi(Resource):
+    @console_ns.doc("finalize_agent_build_chat")
+    @console_ns.doc(description="Run a build-draft Agent App turn that asks the agent to push config updates")
+    @console_ns.doc(params={"agent_id": "Agent ID"})
+    @console_ns.response(200, "Success", console_ns.models[SimpleResultResponse.__name__])
+    @console_ns.response(400, "Invalid request parameters")
+    @console_ns.response(404, "Agent, build draft, or conversation not found")
+    @setup_required
+    @login_required
+    @account_initialization_required
+    @edit_permission_required
+    @rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_TEST_AND_RUN)
+    @with_current_user
+    @with_current_tenant_id
+    @with_session
+    def post(self, session: Session, current_tenant_id: str, current_user: Account, agent_id: UUID):
+        app_model = resolve_agent_runtime_app_model(tenant_id=current_tenant_id, agent_id=agent_id)
+        return _create_build_chat_finalization_message(
+            session=session,
            current_tenant_id=current_tenant_id,
            current_user=current_user,
            app_model=app_model,
@@ -254,7 +329,7 @@ class AgentChatMessageStopApi(Resource):
    @with_current_user_id
    @with_current_tenant_id
    def post(self, current_tenant_id: str, current_user_id: str, agent_id: UUID, task_id: str):
-        app_model = resolve_agent_app_model(tenant_id=current_tenant_id, agent_id=agent_id)
+        app_model = resolve_agent_runtime_app_model(tenant_id=current_tenant_id, agent_id=agent_id)
        return _stop_chat_message(current_user_id=current_user_id, app_model=app_model, task_id=task_id)


@@ -280,7 +355,12 @@ def _resolve_current_user_agent_debug_conversation_id(


 def _create_chat_message(
-    *, current_user: Account, app_model: App, current_tenant_id: str | None = None, agent_id: str | None = None
+    *,
+    session: Session,
+    current_user: Account,
+    app_model: App,
+    current_tenant_id: str | None = None,
+    agent_id: str | None = None,
 ):
    raw_payload = console_ns.payload or {}
    args_model = ChatMessagePayload.model_validate(raw_payload)
@@ -310,12 +390,107 @@ def _create_chat_message(
    if external_trace_id:
        args["external_trace_id"] = external_trace_id

-    try:
-        response = AppGenerateService.generate(
-            app_model=app_model, user=current_user, args=args, invoke_from=InvokeFrom.DEBUGGER, streaming=streaming
-        )
+    return _generate_chat_message_response(
+        session=session,
+        current_user=current_user,
+        app_model=app_model,
+        args=args,
+        streaming=streaming,
+    )

-        return helper.compact_generate_response(response)
+
+def _create_build_chat_finalization_message(
+    *, session: Session, current_user: Account, app_model: App, current_tenant_id: str, agent_id: str
+):
+    debug_conversation_id = _resolve_current_user_agent_debug_conversation_id(
+        current_tenant_id=current_tenant_id,
+        current_user=current_user,
+        app_model=app_model,
+        agent_id=agent_id,
+    )
+    args: dict[str, Any] = {
+        "query": _BUILD_CHAT_FINALIZATION_QUERY,
+        "inputs": {},
+        "response_mode": "streaming",
+        "draft_type": "debug_build",
+        "conversation_id": debug_conversation_id,
+        "auto_generate_name": False,
+    }
+    external_trace_id = get_external_trace_id(request)
+    if external_trace_id:
+        args["external_trace_id"] = external_trace_id
+
+    response = _generate_chat_message(
+        session=session,
+        current_user=current_user,
+        app_model=app_model,
+        args=args,
+        streaming=True,
+    )
+    _drain_streaming_generate_response(response)
+    return {"result": "success"}, 200
+
+
+def _drain_streaming_generate_response(response: RateLimitGenerator | Generator[str, None, None]) -> None:
+    """Consume a streamed app-generate response until a terminal message event arrives.
+
+    Finalize keeps the normal Agent App streaming path so the existing queue,
+    persistence, and runtime-session behavior stay intact. The console API only
+    changes the HTTP boundary: it drains the SSE stream server-side and returns
+    success after the generated build-chat message reaches ``message_end``.
+    """
+    try:
+        for chunk in response:
+            for raw_event in chunk.split("\n\n"):
+                if not raw_event.strip():
+                    continue
+
+                event_name: str | None = None
+                data_lines: list[str] = []
+                for line in raw_event.splitlines():
+                    if line.startswith("event: "):
+                        event_name = line.removeprefix("event: ").strip()
+                    elif line.startswith("data: "):
+                        data_lines.append(line.removeprefix("data: "))
+
+                if not data_lines:
+                    if event_name == "ping":
+                        continue
+                    continue
+
+                payload = json.loads("\n".join(data_lines))
+                if not isinstance(payload, dict):
+                    continue
+
+                payload_event = payload.get("event")
+                if payload_event == "message_end":
+                    return
+                if payload_event == "error":
+                    raise CompletionRequestError(str(payload.get("message") or "Build chat finalization failed."))
+    finally:
+        if isinstance(response, _ClosableStream):
+            response.close()
+
+    raise CompletionRequestError("Build chat finalization did not complete.")
+
+
+def _generate_chat_message(
+    *,
+    session: Session,
+    current_user: Account,
+    app_model: App,
+    args: dict[str, Any],
+    streaming: bool,
+):
+    try:
+        return AppGenerateService.generate(
+            session=session,
+            app_model=app_model,
+            user=current_user,
+            args=args,
+            invoke_from=InvokeFrom.DEBUGGER,
+            streaming=streaming,
+        )
    except services.errors.conversation.ConversationNotExistsError:
        raise NotFound("Conversation Not Exists.")
    except services.errors.conversation.ConversationCompletedError:
@@ -331,6 +506,8 @@ def _create_chat_message(
        raise ProviderModelCurrentlyNotSupportError()
    except InvokeRateLimitError as ex:
        raise InvokeRateLimitHttpError(ex.description)
+    except CompletionRequestError:
+        raise
    except InvokeError as e:
        raise CompletionRequestError(e.description)
    except ValueError as e:
@@ -340,6 +517,26 @@ def _create_chat_message(
        raise InternalServerError()


+def _generate_chat_message_response(
+    *,
+    session: Session,
+    current_user: Account,
+    app_model: App,
+    args: dict[str, Any],
+    streaming: bool,
+):
+    response = _generate_chat_message(
+        session=session,
+        current_user=current_user,
+        app_model=app_model,
+        args=args,
+        streaming=streaming,
+    )
+    if AppMode.value_of(app_model.mode) == AppMode.AGENT and streaming:
+        response = _raise_agent_stream_error_before_response(response)
+    return helper.compact_generate_response(response)
+
+
 def _stop_chat_message(*, current_user_id: str, app_model: App, task_id: str):
    AppTaskService.stop_task(
        task_id=task_id,
@@ -348,4 +545,68 @@ def _stop_chat_message(*, current_user_id: str, app_model: App, task_id: str):
        app_mode=AppMode.value_of(app_model.mode),
    )

-    return {"result": "success"}, 200
+    return SimpleResultResponse(result="success").model_dump(mode="json"), 200
+
+
+def _raise_agent_stream_error_before_response(response):
+    """Surface immediate Agent App stream errors as HTTP errors before SSE starts.
+
+    The shared streaming helper always returns HTTP 200 once the SSE response is
+    created. Agent v2 configuration errors, such as an invalid model API key,
+    can be the first real stream event after the initial ping; pre-reading that
+    first non-ping event lets the console API return the existing 400 error
+    contract instead of a successful HTTP response carrying only an SSE error.
+    """
+    if isinstance(response, Mapping):
+        return response
+
+    buffered: list[str] = []
+    iterator = iter(response)
+    while True:
+        try:
+            chunk = next(iterator)
+        except StopIteration:
+            return iter(buffered)
+
+        if not isinstance(chunk, str):
+            return _prepend_stream_chunks(buffered, chunk, iterator)
+
+        if _is_sse_ping(chunk):
+            buffered.append(chunk)
+            continue
+
+        error_payload = _extract_sse_error_payload(chunk)
+        if error_payload is not None:
+            if isinstance(response, _ClosableStream):
+                response.close()
+            message = error_payload.get("message")
+            raise CompletionRequestError(str(message or "Agent App chat failed."))
+
+        return _prepend_stream_chunks(buffered, chunk, iterator)
+
+
+def _prepend_stream_chunks(buffered: list[Any], first: Any, iterator: Iterator[Any]) -> Generator[Any, None, None]:
+    yield from buffered
+    yield first
+    yield from iterator
+
+
+def _is_sse_ping(chunk: str) -> bool:
+    return chunk.strip() == "event: ping"
+
+
+def _extract_sse_error_payload(chunk: str) -> dict[str, Any] | None:
+    for raw_event in chunk.split("\n\n"):
+        data_lines: list[str] = []
+        for line in raw_event.splitlines():
+            if line.startswith("data: "):
+                data_lines.append(line.removeprefix("data: "))
+        if not data_lines:
+            continue
+        try:
+            payload = json.loads("\n".join(data_lines))
+        except json.JSONDecodeError:
+            continue
+        if isinstance(payload, dict) and payload.get("event") == "error":
+            return payload
+    return None
@@ -9,7 +9,7 @@ from sqlalchemy import func, or_
 from sqlalchemy.orm import selectinload
 from werkzeug.exceptions import NotFound

-from controllers.common.schema import query_params_from_model, register_schema_models
+from controllers.common.schema import query_params_from_model, register_response_schema_models, register_schema_models
 from controllers.console import console_ns
 from controllers.console.app.wraps import get_app_model
 from controllers.console.wraps import (
@@ -39,7 +39,9 @@ from fields.conversation_fields import (
    ConversationWithSummaryPagination as ConversationWithSummaryPaginationResponse,
 )
 from libs.datetime_utils import naive_utc_now, parse_time_range
+from libs.helper import dump_response
 from libs.login import login_required
+from libs.pagination import paginate_query
 from models import Conversation, EndUser, Message, MessageAnnotation
 from models.account import Account
 from models.model import App, AppMode
@@ -79,13 +81,14 @@ register_schema_models(
    console_ns,
    CompletionConversationQuery,
    ChatConversationQuery,
+)
+register_response_schema_models(
+    console_ns,
    ConversationResponse,
    ConversationPaginationResponse,
    ConversationMessageDetailResponse,
    ConversationWithSummaryPaginationResponse,
    ConversationDetailResponse,
-    CompletionConversationQuery,
-    ChatConversationQuery,
 )


@@ -93,8 +96,7 @@ register_schema_models(
 class CompletionConversationApi(Resource):
    @console_ns.doc("list_completion_conversations")
    @console_ns.doc(description="Get completion conversations with pagination and filtering")
-    @console_ns.doc(params={"app_id": "Application ID"})
-    @console_ns.doc(params=query_params_from_model(CompletionConversationQuery))
+    @console_ns.doc(params={"app_id": "Application ID", **query_params_from_model(CompletionConversationQuery)})
    @console_ns.response(200, "Success", console_ns.models[ConversationPaginationResponse.__name__])
    @console_ns.response(403, "Insufficient permissions")
    @setup_required
@@ -155,11 +157,9 @@ class CompletionConversationApi(Resource):

        query = query.order_by(Conversation.created_at.desc())

-        conversations = db.paginate(query, page=args.page, per_page=args.limit, error_out=False)
+        conversations = paginate_query(query, page=args.page, per_page=args.limit)

-        return ConversationPaginationResponse.model_validate(conversations, from_attributes=True).model_dump(
-            mode="json"
-        )
+        return dump_response(ConversationPaginationResponse, conversations)


@console_ns.route("/apps/<uuid:app_id>/completion-conversations/<uuid:conversation_id>")
@@ -179,9 +179,9 @@ class CompletionConversationDetailApi(Resource):
    @get_app_model(mode=AppMode.COMPLETION)
    def get(self, current_user: Account, app_model: App, conversation_id: UUID):
        conversation_id_str = str(conversation_id)
-        return ConversationMessageDetailResponse.model_validate(
-            _get_conversation(current_user, app_model, conversation_id_str), from_attributes=True
-        ).model_dump(mode="json")
+        return dump_response(
+            ConversationMessageDetailResponse, _get_conversation(current_user, app_model, conversation_id_str)
+        )

    @console_ns.doc("delete_completion_conversation")
    @console_ns.doc(description="Delete a completion conversation")
@@ -211,8 +211,7 @@ class CompletionConversationDetailApi(Resource):
 class ChatConversationApi(Resource):
    @console_ns.doc("list_chat_conversations")
    @console_ns.doc(description="Get chat conversations with pagination, filtering and summary")
-    @console_ns.doc(params={"app_id": "Application ID"})
-    @console_ns.doc(params=query_params_from_model(ChatConversationQuery))
+    @console_ns.doc(params={"app_id": "Application ID", **query_params_from_model(ChatConversationQuery)})
    @console_ns.response(200, "Success", console_ns.models[ConversationWithSummaryPaginationResponse.__name__])
    @console_ns.response(403, "Insufficient permissions")
    @setup_required
@@ -312,11 +311,9 @@ class ChatConversationApi(Resource):
            case _:
                query = query.order_by(Conversation.created_at.desc())

-        conversations = db.paginate(query, page=args.page, per_page=args.limit, error_out=False)
+        conversations = paginate_query(query, page=args.page, per_page=args.limit)

-        return ConversationWithSummaryPaginationResponse.model_validate(conversations, from_attributes=True).model_dump(
-            mode="json"
-        )
+        return dump_response(ConversationWithSummaryPaginationResponse, conversations)


@console_ns.route("/apps/<uuid:app_id>/chat-conversations/<uuid:conversation_id>")
@@ -336,9 +333,9 @@ class ChatConversationDetailApi(Resource):
    @get_app_model(mode=[AppMode.CHAT, AppMode.AGENT_CHAT, AppMode.ADVANCED_CHAT, AppMode.AGENT])
    def get(self, current_user: Account, app_model: App, conversation_id: UUID):
        conversation_id_str = str(conversation_id)
-        return ConversationDetailResponse.model_validate(
-            _get_conversation(current_user, app_model, conversation_id_str), from_attributes=True
-        ).model_dump(mode="json")
+        return dump_response(
+            ConversationDetailResponse, _get_conversation(current_user, app_model, conversation_id_str)
+        )

    @console_ns.doc("delete_chat_conversation")
    @console_ns.doc(description="Delete a chat conversation")
@@ -22,7 +22,7 @@ from controllers.console.wraps import (
 from extensions.ext_database import db
 from fields._value_type_serializer import serialize_value_type
 from fields.base import ResponseModel
-from libs.helper import to_timestamp
+from libs.helper import dump_response, to_timestamp
 from libs.login import login_required
 from models import ConversationVariable
 from models.model import App, AppMode
@@ -119,7 +119,8 @@ class ConversationVariablesApi(Resource):
        with sessionmaker(db.engine, expire_on_commit=False).begin() as session:
            rows = session.scalars(stmt).all()

-        response = PaginatedConversationVariableResponse.model_validate(
+        return dump_response(
+            PaginatedConversationVariableResponse,
            {
                "page": page,
                "limit": page_size,
@@ -135,6 +136,5 @@ class ConversationVariablesApi(Resource):
                    )
                    for row in rows
                ],
-            }
+            },
        )
-        return response.model_dump(mode="json")
@@ -1,8 +1,10 @@
-from collections.abc import Sequence
+import json
+from collections.abc import Generator, Sequence
 from typing import Any, Literal

 from flask_restx import Resource
 from pydantic import BaseModel, Field, RootModel
+from sqlalchemy import select
 from sqlalchemy.orm import Session

 from controllers.common.fields import SimpleDataResponse
@@ -23,8 +25,10 @@ from core.helper.code_executor.javascript.javascript_code_provider import Javasc
 from core.helper.code_executor.python3.python3_code_provider import Python3CodeProvider
 from core.llm_generator.entities import RuleCodeGeneratePayload, RuleGeneratePayload, RuleStructuredOutputPayload
 from core.llm_generator.llm_generator import LLMGenerator
+from core.workflow.generator.types import WorkflowGenerateErrorCode
 from graphon.model_runtime.entities.llm_entities import LLMMode
 from graphon.model_runtime.errors.invoke import InvokeError
+from libs.helper import compact_generate_response
 from libs.login import login_required
 from models import App
 from services.workflow_generator_service import WorkflowGeneratorService
@@ -64,7 +68,10 @@ class WorkflowGeneratePayload(BaseModel):
    can reuse its existing handler.
    """

-    mode: Literal["workflow", "advanced-chat"] = Field(..., description="Target app mode for the generated graph")
+    mode: Literal["workflow", "advanced-chat", "auto"] = Field(
+        ...,
+        description="Target app mode for the generated graph; 'auto' lets the backend classify the instruction",
+    )
    instruction: str = Field(..., description="Natural-language workflow description")
    ideal_output: str = Field(default="", description="Optional sample output for grounding")
    model_config_data: ModelConfig = Field(
@@ -78,6 +85,19 @@ class WorkflowGeneratePayload(BaseModel):
    )


+class WorkflowInstructionSuggestionsPayload(BaseModel):
+    """Payload for the workflow-generator instruction-suggestions endpoint.
+
+    Runs before the user picks a model, so the suggestions come from the
+    tenant's default model. The underlying generator never raises — an empty
+    ``suggestions`` list is a valid 200 (soft-fail).
+    """
+
+    mode: Literal["workflow", "advanced-chat"] = Field(..., description="Target app mode for the suggestions")
+    language: str | None = Field(default=None, description="Optional language to write the suggestions in")
+    count: int = Field(default=4, ge=1, le=6, description="Number of suggestions to return (1-6)")
+
+
 class GeneratorResponse(RootModel[Any]):
    root: Any

@@ -91,6 +111,7 @@ register_schema_models(
    InstructionGeneratePayload,
    InstructionTemplatePayload,
    WorkflowGeneratePayload,
+    WorkflowInstructionSuggestionsPayload,
    ModelConfig,
 )
 register_response_schema_models(console_ns, GeneratorResponse, SimpleDataResponse)
@@ -216,7 +237,9 @@ class InstructionGenerateApi(Resource):
        try:
            # Generate from nothing for a workflow node
            if (args.current in (code_template, "")) and args.node_id != "":
-                app = session.get(App, args.flow_id)
+                app = session.scalar(
+                    select(App).where(App.id == args.flow_id, App.tenant_id == current_tenant_id).limit(1)
+                )
                if not app:
                    return {"error": f"app {args.flow_id} not found"}, 400
                workflow = WorkflowService().get_draft_workflow(app_model=app, session=session)
@@ -313,6 +336,34 @@ class InstructionGenerationTemplateApi(Resource):
                raise ValueError(f"Invalid type: {args.type}")


+def _workflow_instruction_guard(args: WorkflowGeneratePayload) -> tuple[dict, int] | None:
+    """Shared boundary guard for the workflow-generate endpoints.
+
+    Returns a ``(body, 400)`` tuple when the instruction is empty / whitespace
+    or either free-text field exceeds the cap, else ``None``. Pydantic only
+    validates the field is a str; a whitespace-only or pasted-document input
+    would otherwise waste a slow planner+builder roundtrip on a response the
+    validator rejects anyway. Both the blocking and streaming endpoints call
+    this so they reject identical inputs.
+    """
+    if not args.instruction.strip():
+        return {
+            "error": "Instruction is required",
+            "errors": [{"code": WorkflowGenerateErrorCode.EMPTY_INSTRUCTION, "detail": "Instruction is required"}],
+        }, 400
+    if len(args.instruction) > _MAX_INSTRUCTION_LENGTH or len(args.ideal_output) > _MAX_INSTRUCTION_LENGTH:
+        return {
+            "error": "Instruction is too long",
+            "errors": [
+                {
+                    "code": WorkflowGenerateErrorCode.INSTRUCTION_TOO_LONG,
+                    "detail": f"Instruction and ideal output must each be at most {_MAX_INSTRUCTION_LENGTH} characters",
+                }
+            ],
+        }, 400
+    return None
+
+
@console_ns.route("/workflow-generate")
 class WorkflowGenerateApi(Resource):
    """Generate a Workflow / Chatflow draft graph from a natural-language description.
@@ -335,31 +386,11 @@ class WorkflowGenerateApi(Resource):
    def post(self, current_tenant_id: str):
        args = WorkflowGeneratePayload.model_validate(console_ns.payload)

-        # Reject obviously-empty instructions at the boundary — Pydantic only
-        # validates ``instruction`` is a str, but a whitespace-only string
-        # would still hit the LLM and waste a planner+builder roundtrip on a
-        # response that the postprocess validator would reject anyway.
-        if not args.instruction.strip():
-            return {
-                "error": "Instruction is required",
-                "errors": [{"code": "EMPTY_INSTRUCTION", "detail": "Instruction is required"}],
-            }, 400
-
-        # Bound the prompt at the boundary too: an arbitrarily long
-        # instruction (or pasted document) blows the planner/builder context
-        # window and fails with an opaque provider error after two slow LLM
-        # calls. The cap matches the frontend textarea's maxLength.
-        if len(args.instruction) > _MAX_INSTRUCTION_LENGTH or len(args.ideal_output) > _MAX_INSTRUCTION_LENGTH:
-            return {
-                "error": "Instruction is too long",
-                "errors": [
-                    {
-                        "code": "INSTRUCTION_TOO_LONG",
-                        "detail": f"Instruction and ideal output must each be at most "
-                        f"{_MAX_INSTRUCTION_LENGTH} characters",
-                    }
-                ],
-            }, 400
+        # Reject empty / over-length instructions at the boundary (shared with
+        # the streaming endpoint) before spending a planner+builder roundtrip.
+        guard = _workflow_instruction_guard(args)
+        if guard is not None:
+            return guard

        try:
            result = WorkflowGeneratorService.generate_workflow_graph(
@@ -380,3 +411,93 @@ class WorkflowGenerateApi(Resource):
            raise CompletionRequestError(e.description)

        return result
+
+
+@console_ns.route("/workflow-generate/suggestions")
+class WorkflowInstructionSuggestionsApi(Resource):
+    """Suggest short, buildable example instructions for the cmd+k generator.
+
+    Runs before a model is selected (uses the tenant's default model). The
+    underlying generator never raises, so an empty list is a valid 200 — the
+    frontend renders "no suggestions" rather than an error, so no provider-error
+    mapping is needed here.
+    """
+
+    @console_ns.doc("generate_workflow_instruction_suggestions")
+    @console_ns.doc(description="Suggest example workflow-generator instructions for the tenant")
+    @console_ns.expect(console_ns.models[WorkflowInstructionSuggestionsPayload.__name__])
+    @console_ns.response(200, "Suggestions generated successfully", console_ns.models[GeneratorResponse.__name__])
+    @console_ns.response(400, "Invalid request parameters")
+    @setup_required
+    @login_required
+    @account_initialization_required
+    @with_current_tenant_id
+    def post(self, current_tenant_id: str):
+        args = WorkflowInstructionSuggestionsPayload.model_validate(console_ns.payload)
+        suggestions = LLMGenerator.generate_workflow_instruction_suggestions(
+            tenant_id=current_tenant_id,
+            mode=args.mode,
+            language=args.language,
+            count=args.count,
+        )
+        return {"suggestions": suggestions}
+
+
+@console_ns.route("/workflow-generate/stream")
+class WorkflowGenerateStreamApi(Resource):
+    """Plan-first streaming variant of ``/workflow-generate`` (Server-Sent Events).
+
+    Emits a ``plan`` event (high-level node list + app metadata) as soon as the
+    planner returns, then a final ``result`` event with the full graph — the
+    SAME envelope ``/workflow-generate`` returns. Provider-init / invoke errors
+    are surfaced as a single ``result`` event (code ``MODEL_ERROR``) so the
+    frontend's stream parser always receives a result rather than a non-SSE HTTP
+    error.
+    """
+
+    @console_ns.doc("generate_workflow_graph_stream")
+    @console_ns.doc(description="Stream a Dify workflow graph (plan then result) via SSE")
+    @console_ns.expect(console_ns.models[WorkflowGeneratePayload.__name__])
+    @console_ns.response(200, "Server-Sent Events stream of plan/result events")
+    @console_ns.response(400, "Invalid request parameters")
+    @setup_required
+    @login_required
+    @account_initialization_required
+    @with_current_tenant_id
+    def post(self, current_tenant_id: str):
+        args = WorkflowGeneratePayload.model_validate(console_ns.payload)
+
+        # Same boundary guards as the blocking endpoint — return a normal 400
+        # JSON for these BEFORE opening the stream.
+        guard = _workflow_instruction_guard(args)
+        if guard is not None:
+            return guard
+
+        def generate() -> Generator[str, None, None]:
+            try:
+                for event_name, payload in WorkflowGeneratorService.generate_workflow_graph_stream(
+                    tenant_id=current_tenant_id,
+                    mode=args.mode,
+                    instruction=args.instruction,
+                    model_config=args.model_config_data,
+                    ideal_output=args.ideal_output,
+                    current_graph=args.current_graph,
+                ):
+                    body = {"event": event_name, **payload}
+                    yield f"data: {json.dumps(body)}\n\n"
+            except (ProviderTokenNotInitError, QuotaExceededError, ModelCurrentlyNotSupportError, InvokeError) as e:
+                # The model instance is resolved inside the service (lazily, on
+                # first iteration), so a provider / init error surfaces here.
+                # Emit it as a single SSE result event rather than a non-SSE
+                # error response so the frontend's stream parser always gets a
+                # result it can render.
+                detail = getattr(e, "description", None) or str(e) or "Model invocation failed"
+                error_body = {
+                    "event": "result",
+                    "graph": {"nodes": [], "edges": [], "viewport": {"x": 0.0, "y": 0.0, "zoom": 0.7}},
+                    "error": detail,
+                    "errors": [{"code": WorkflowGenerateErrorCode.MODEL_ERROR, "detail": detail}],
+                }
+                yield f"data: {json.dumps(error_body)}\n\n"
+
+        return compact_generate_response(generate())
@@ -22,10 +22,11 @@ from controllers.console.wraps import (
 )
 from extensions.ext_database import db
 from fields.base import ResponseModel
-from libs.helper import to_timestamp
+from libs.helper import dump_response, to_timestamp
 from libs.login import login_required
 from models.enums import AppMCPServerStatus
 from models.model import App, AppMCPServer
+from services.app_ref_service import AppRefService


 class MCPServerCreatePayload(BaseModel):
@@ -92,7 +93,7 @@ class AppMCPServerController(Resource):
        server = db.session.scalar(select(AppMCPServer).where(AppMCPServer.app_id == app_model.id).limit(1))
        if server is None:
            return {}
-        return AppMCPServerResponse.model_validate(server, from_attributes=True).model_dump(mode="json")
+        return dump_response(AppMCPServerResponse, server)

    @console_ns.doc("create_app_mcp_server")
    @console_ns.doc(description="Create MCP server configuration for an application")
@@ -127,7 +128,7 @@ class AppMCPServerController(Resource):
        )
        db.session.add(server)
        db.session.commit()
-        return AppMCPServerResponse.model_validate(server, from_attributes=True).model_dump(mode="json"), 201
+        return dump_response(AppMCPServerResponse, server), 201

    @console_ns.doc("update_app_mcp_server")
    @console_ns.doc(description="Update MCP server configuration for an application")
@@ -146,7 +147,17 @@ class AppMCPServerController(Resource):
    @get_app_model
    def put(self, app_model: App):
        payload = MCPServerUpdatePayload.model_validate(console_ns.payload or {})
-        server = db.session.get(AppMCPServer, payload.id)
+        app_ref = AppRefService.create_app_ref(app_model)
+        server_ref = AppRefService.create_mcp_server_ref(app_ref, payload.id)
+        server = db.session.scalar(
+            select(AppMCPServer)
+            .where(
+                AppMCPServer.id == server_ref.server_id,
+                AppMCPServer.tenant_id == server_ref.tenant_id,
+                AppMCPServer.app_id == server_ref.app_id,
+            )
+            .limit(1)
+        )
        if not server:
            raise NotFound()

@@ -165,7 +176,7 @@ class AppMCPServerController(Resource):
            except ValueError:
                raise ValueError("Invalid status")
        db.session.commit()
-        return AppMCPServerResponse.model_validate(server, from_attributes=True).model_dump(mode="json")
+        return dump_response(AppMCPServerResponse, server)


@console_ns.route("/apps/<uuid:server_id>/server/refresh")
@@ -192,4 +203,4 @@ class AppMCPServerRefreshController(Resource):
            raise NotFound()
        server.server_code = AppMCPServer.generate_server_code(16)
        db.session.commit()
-        return AppMCPServerResponse.model_validate(server, from_attributes=True).model_dump(mode="json")
+        return dump_response(AppMCPServerResponse, server)
@@ -1,5 +1,4 @@
 import logging
-from datetime import datetime
 from typing import Literal
 from uuid import UUID

@@ -13,7 +12,7 @@ from controllers.common.controller_schemas import MessageFeedbackPayload as _Mes
 from controllers.common.fields import SimpleResultResponse, TextFileResponse
 from controllers.common.schema import query_params_from_model, register_response_schema_models, register_schema_models
 from controllers.console import console_ns
-from controllers.console.agent.app_helpers import resolve_agent_app_model
+from controllers.console.agent.app_helpers import resolve_agent_runtime_app_model
 from controllers.console.app.error import (
    CompletionRequestError,
    ProviderModelCurrentlyNotSupportError,
@@ -38,16 +37,10 @@ from core.errors.error import ModelCurrentlyNotSupportError, ProviderTokenNotIni
 from extensions.ext_database import db
 from fields.base import ResponseModel
 from fields.conversation_fields import (
-    AgentThought,
-    ConversationAnnotation,
-    ConversationAnnotationHitHistory,
-    Feedback,
-    JSONValue,
-    MessageFile,
-    format_files_contained,
+    MessageDetail as BaseMessageDetailResponse,
 )
 from graphon.model_runtime.errors.invoke import InvokeError
-from libs.helper import to_timestamp, uuid_value
+from libs.helper import dump_response, uuid_value
 from libs.infinite_scroll_pagination import InfiniteScrollPagination
 from libs.login import login_required
 from models.account import Account
@@ -111,49 +104,16 @@ class FeedbackExportQuery(BaseModel):
        raise ValueError("has_comment must be a boolean value")


-class AnnotationCountResponse(BaseModel):
+class AnnotationCountResponse(ResponseModel):
    count: int = Field(description="Number of annotations")


-class SuggestedQuestionsResponse(BaseModel):
+class SuggestedQuestionsResponse(ResponseModel):
    data: list[str] = Field(description="Suggested question")


-class MessageDetailResponse(ResponseModel):
-    id: str
-    conversation_id: str
-    inputs: dict[str, JSONValue]
-    query: str
-    message: JSONValue | None = None
-    message_tokens: int | None = None
-    answer: str = Field(validation_alias="re_sign_file_url_answer")
-    answer_tokens: int | None = None
-    provider_response_latency: float | None = None
-    from_source: str
-    from_end_user_id: str | None = None
-    from_account_id: str | None = None
-    feedbacks: list[Feedback] = Field(default_factory=list)
-    workflow_run_id: str | None = None
-    annotation: ConversationAnnotation | None = None
-    annotation_hit_history: ConversationAnnotationHitHistory | None = None
-    created_at: int | None = None
-    agent_thoughts: list[AgentThought] = Field(default_factory=list)
-    message_files: list[MessageFile] = Field(default_factory=list)
+class MessageDetailResponse(BaseMessageDetailResponse):
    extra_contents: list[ExecutionExtraContentDomainModel] = Field(default_factory=list)
-    metadata: JSONValue | None = Field(default=None, validation_alias="message_metadata_dict")
-    status: str
-    error: str | None = None
-    parent_message_id: str | None = None
-
-    @field_validator("inputs", mode="before")
-    @classmethod
-    def _normalize_inputs(cls, value: JSONValue) -> JSONValue:
-        return format_files_contained(value)
-
-    @field_validator("created_at", mode="before")
-    @classmethod
-    def _normalize_created_at(cls, value: datetime | int | None) -> int | None:
-        return to_timestamp(value)


 class MessageInfiniteScrollPaginationResponse(ResponseModel):
@@ -167,20 +127,23 @@ register_schema_models(
    ChatMessagesQuery,
    MessageFeedbackPayload,
    FeedbackExportQuery,
+)
+register_response_schema_models(
+    console_ns,
    AnnotationCountResponse,
    SuggestedQuestionsResponse,
    MessageDetailResponse,
    MessageInfiniteScrollPaginationResponse,
+    SimpleResultResponse,
+    TextFileResponse,
 )
-register_response_schema_models(console_ns, SimpleResultResponse, TextFileResponse)


@console_ns.route("/apps/<uuid:app_id>/chat-messages")
 class ChatMessageListApi(Resource):
    @console_ns.doc("list_chat_messages")
    @console_ns.doc(description="Get chat messages for a conversation with pagination")
-    @console_ns.doc(params={"app_id": "Application ID"})
-    @console_ns.doc(params=query_params_from_model(ChatMessagesQuery))
+    @console_ns.doc(params={"app_id": "Application ID", **query_params_from_model(ChatMessagesQuery)})
    @console_ns.response(200, "Success", console_ns.models[MessageInfiniteScrollPaginationResponse.__name__])
    @console_ns.response(404, "Conversation not found")
    @login_required
@@ -210,7 +173,7 @@ class AgentChatMessageListApi(Resource):
    @with_current_user
    @with_current_tenant_id
    def get(self, current_tenant_id: str, current_user: Account, agent_id: UUID):
-        app_model = resolve_agent_app_model(tenant_id=current_tenant_id, agent_id=agent_id)
+        app_model = resolve_agent_runtime_app_model(tenant_id=current_tenant_id, agent_id=agent_id)
        return _list_chat_messages(app_model=app_model, current_user=current_user)


@@ -246,7 +209,7 @@ class AgentMessageFeedbackApi(Resource):
    @with_current_user
    @with_current_tenant_id
    def post(self, current_tenant_id: str, current_user: Account, agent_id: UUID):
-        app_model = resolve_agent_app_model(tenant_id=current_tenant_id, agent_id=agent_id)
+        app_model = resolve_agent_runtime_app_model(tenant_id=current_tenant_id, agent_id=agent_id)
        return _update_message_feedback(current_user=current_user, app_model=app_model)


@@ -270,7 +233,7 @@ class MessageAnnotationCountApi(Resource):
            select(func.count(MessageAnnotation.id)).where(MessageAnnotation.app_id == app_model.id)
        )

-        return {"count": count}
+        return AnnotationCountResponse(count=count or 0).model_dump(mode="json")


@console_ns.route("/apps/<uuid:app_id>/chat-messages/<uuid:message_id>/suggested-questions")
@@ -311,7 +274,7 @@ class AgentMessageSuggestedQuestionApi(Resource):
    @with_current_user
    @with_current_tenant_id
    def get(self, current_tenant_id: str, current_user: Account, agent_id: UUID, message_id: UUID):
-        app_model = resolve_agent_app_model(tenant_id=current_tenant_id, agent_id=agent_id)
+        app_model = resolve_agent_runtime_app_model(tenant_id=current_tenant_id, agent_id=agent_id)
        return _get_message_suggested_questions(current_user=current_user, app_model=app_model, message_id=message_id)


@@ -319,13 +282,12 @@ class AgentMessageSuggestedQuestionApi(Resource):
 class MessageFeedbackExportApi(Resource):
    @console_ns.doc("export_feedbacks")
    @console_ns.doc(description="Export user feedback data for Google Sheets")
-    @console_ns.doc(params={"app_id": "Application ID"})
-    @console_ns.doc(params=query_params_from_model(FeedbackExportQuery))
    @console_ns.response(
        200,
        "Feedback data exported successfully",
        console_ns.models[TextFileResponse.__name__],
    )
+    @console_ns.doc(params={"app_id": "Application ID", **query_params_from_model(FeedbackExportQuery)})
    @console_ns.response(400, "Invalid parameters")
    @console_ns.response(500, "Internal server error")
    @setup_required
@@ -350,7 +312,6 @@ class MessageFeedbackExportApi(Resource):
                end_date=args.end_date,
                format_type=args.format,
            )
-
            return export_data

        except ValueError as e:
@@ -389,7 +350,7 @@ class AgentMessageApi(Resource):
    @account_initialization_required
    @with_current_tenant_id
    def get(self, current_tenant_id: str, agent_id: UUID, message_id: UUID):
-        app_model = resolve_agent_app_model(tenant_id=current_tenant_id, agent_id=agent_id)
+        app_model = resolve_agent_runtime_app_model(tenant_id=current_tenant_id, agent_id=agent_id)
        return _get_message_detail(app_model=app_model, message_id=message_id)


@@ -461,16 +422,16 @@ def _list_chat_messages(*, app_model: App, current_user: Account | None = None):
    history_messages = list(reversed(history_messages))
    attach_message_extra_contents(history_messages)

-    return MessageInfiniteScrollPaginationResponse.model_validate(
+    return dump_response(
+        MessageInfiniteScrollPaginationResponse,
        InfiniteScrollPagination(data=history_messages, limit=args.limit, has_more=has_more),
-        from_attributes=True,
-    ).model_dump(mode="json")
+    )


 def _update_message_feedback(*, current_user: Account, app_model: App):
    args = MessageFeedbackPayload.model_validate(console_ns.payload)

-    message_id = str(args.message_id)
+    message_id = args.message_id

    message = db.session.scalar(
        select(Message).where(Message.id == message_id, Message.app_id == app_model.id).limit(1)
@@ -505,7 +466,7 @@ def _update_message_feedback(*, current_user: Account, app_model: App):

    db.session.commit()

-    return {"result": "success"}
+    return SimpleResultResponse(result="success").model_dump(mode="json")


 def _get_message_suggested_questions(*, current_user: Account, app_model: App, message_id: UUID):
@@ -533,7 +494,7 @@ def _get_message_suggested_questions(*, current_user: Account, app_model: App, m
        logger.exception("internal server error.")
        raise InternalServerError()

-    return {"data": questions}
+    return dump_response(SuggestedQuestionsResponse, {"data": questions})


 def _get_message_detail(*, app_model: App, message_id: UUID):
@@ -547,4 +508,4 @@ def _get_message_detail(*, app_model: App, message_id: UUID):
        raise NotFound("Message Not Exists.")

    attach_message_extra_contents([message])
-    return MessageDetailResponse.model_validate(message, from_attributes=True).model_dump(mode="json")
+    return dump_response(MessageDetailResponse, message)
@@ -22,6 +22,7 @@ from controllers.console.wraps import (
 from extensions.ext_database import db
 from fields.base import ResponseModel
 from libs.datetime_utils import naive_utc_now
+from libs.helper import dump_response
 from libs.login import login_required
 from models import Site
 from models.account import Account
@@ -40,6 +41,7 @@ class AppSiteUpdatePayload(BaseModel):
    customize_domain: str | None = Field(default=None)
    copyright: str | None = Field(default=None)
    privacy_policy: str | None = Field(default=None)
+    input_placeholder: str | None = Field(default=None)
    custom_disclaimer: str | None = Field(default=None)
    customize_token_strategy: Literal["must", "allow", "not_allow"] | None = Field(default=None)
    prompt_public: bool | None = Field(default=None)
@@ -66,6 +68,7 @@ class AppSiteResponse(ResponseModel):
    customize_domain: str | None = None
    copyright: str | None = None
    privacy_policy: str | None = None
+    input_placeholder: str | None = None
    custom_disclaimer: str | None = None
    customize_token_strategy: str
    prompt_public: bool
@@ -110,6 +113,7 @@ class AppSite(Resource):
            "customize_domain",
            "copyright",
            "privacy_policy",
+            "input_placeholder",
            "custom_disclaimer",
            "customize_token_strategy",
            "prompt_public",
@@ -124,7 +128,7 @@ class AppSite(Resource):
        site.updated_at = naive_utc_now()
        db.session.commit()

-        return AppSiteResponse.model_validate(site, from_attributes=True).model_dump(mode="json")
+        return dump_response(AppSiteResponse, site)


@console_ns.route("/apps/<uuid:app_id>/site/access-token-reset")
@@ -153,4 +157,4 @@ class AppSiteAccessTokenReset(Resource):
        site.updated_at = naive_utc_now()
        db.session.commit()

-        return AppSiteResponse.model_validate(site, from_attributes=True).model_dump(mode="json")
+        return dump_response(AppSiteResponse, site)
@@ -1,7 +1,7 @@
 from decimal import Decimal

 import sqlalchemy as sa
-from flask import abort, jsonify, request
+from flask import abort, request
 from flask_restx import Resource
 from pydantic import BaseModel, Field, field_validator

@@ -20,7 +20,7 @@ from core.app.entities.app_invoke_entities import InvokeFrom
 from extensions.ext_database import db
 from fields.base import ResponseModel
 from libs.datetime_utils import parse_time_range
-from libs.helper import convert_datetime_to_date
+from libs.helper import convert_datetime_to_date, dump_response
 from libs.login import login_required
 from models import AppMode
 from models.account import Account
@@ -44,8 +44,15 @@ class DailyMessageStatisticItem(ResponseModel):
    message_count: int


-class DailyMessageStatisticResponse(ResponseModel):
-    data: list[DailyMessageStatisticItem]
+register_schema_models(console_ns, StatisticTimeRangeQuery)
+
+
+class StatisticDataResponse[T](ResponseModel):
+    data: list[T]
+
+
+class DailyMessageStatisticResponse(StatisticDataResponse[DailyMessageStatisticItem]):
+    pass


 class DailyConversationStatisticItem(ResponseModel):
@@ -53,8 +60,8 @@ class DailyConversationStatisticItem(ResponseModel):
    conversation_count: int


-class DailyConversationStatisticResponse(ResponseModel):
-    data: list[DailyConversationStatisticItem]
+class DailyConversationStatisticResponse(StatisticDataResponse[DailyConversationStatisticItem]):
+    pass


 class DailyTerminalStatisticItem(ResponseModel):
@@ -62,19 +69,19 @@ class DailyTerminalStatisticItem(ResponseModel):
    terminal_count: int


-class DailyTerminalStatisticResponse(ResponseModel):
-    data: list[DailyTerminalStatisticItem]
+class DailyTerminalStatisticResponse(StatisticDataResponse[DailyTerminalStatisticItem]):
+    pass


 class DailyTokenCostStatisticItem(ResponseModel):
    date: str
-    token_count: int
-    total_price: str | float
-    currency: str
+    token_count: int | None = None
+    total_price: Decimal | None = None
+    currency: str | None = None


-class DailyTokenCostStatisticResponse(ResponseModel):
-    data: list[DailyTokenCostStatisticItem]
+class DailyTokenCostStatisticResponse(StatisticDataResponse[DailyTokenCostStatisticItem]):
+    pass


 class AverageSessionInteractionStatisticItem(ResponseModel):
@@ -82,8 +89,8 @@ class AverageSessionInteractionStatisticItem(ResponseModel):
    interactions: float


-class AverageSessionInteractionStatisticResponse(ResponseModel):
-    data: list[AverageSessionInteractionStatisticItem]
+class AverageSessionInteractionStatisticResponse(StatisticDataResponse[AverageSessionInteractionStatisticItem]):
+    pass


 class UserSatisfactionRateStatisticItem(ResponseModel):
@@ -91,8 +98,8 @@ class UserSatisfactionRateStatisticItem(ResponseModel):
    rate: float


-class UserSatisfactionRateStatisticResponse(ResponseModel):
-    data: list[UserSatisfactionRateStatisticItem]
+class UserSatisfactionRateStatisticResponse(StatisticDataResponse[UserSatisfactionRateStatisticItem]):
+    pass


 class AverageResponseTimeStatisticItem(ResponseModel):
@@ -100,8 +107,8 @@ class AverageResponseTimeStatisticItem(ResponseModel):
    latency: float


-class AverageResponseTimeStatisticResponse(ResponseModel):
-    data: list[AverageResponseTimeStatisticItem]
+class AverageResponseTimeStatisticResponse(StatisticDataResponse[AverageResponseTimeStatisticItem]):
+    pass


 class TokensPerSecondStatisticItem(ResponseModel):
@@ -109,20 +116,27 @@ class TokensPerSecondStatisticItem(ResponseModel):
    tps: float


-class TokensPerSecondStatisticResponse(ResponseModel):
-    data: list[TokensPerSecondStatisticItem]
+class TokensPerSecondStatisticResponse(StatisticDataResponse[TokensPerSecondStatisticItem]):
+    pass


-register_schema_models(console_ns, StatisticTimeRangeQuery)
 register_response_schema_models(
    console_ns,
+    DailyMessageStatisticItem,
    DailyMessageStatisticResponse,
+    DailyConversationStatisticItem,
    DailyConversationStatisticResponse,
+    DailyTerminalStatisticItem,
    DailyTerminalStatisticResponse,
+    DailyTokenCostStatisticItem,
    DailyTokenCostStatisticResponse,
+    AverageSessionInteractionStatisticItem,
    AverageSessionInteractionStatisticResponse,
+    UserSatisfactionRateStatisticItem,
    UserSatisfactionRateStatisticResponse,
+    AverageResponseTimeStatisticItem,
    AverageResponseTimeStatisticResponse,
+    TokensPerSecondStatisticItem,
    TokensPerSecondStatisticResponse,
 )

@@ -131,8 +145,7 @@ register_response_schema_models(
 class DailyMessageStatistic(Resource):
    @console_ns.doc("get_daily_message_statistics")
    @console_ns.doc(description="Get daily message statistics for an application")
-    @console_ns.doc(params={"app_id": "Application ID"})
-    @console_ns.doc(params=query_params_from_model(StatisticTimeRangeQuery))
+    @console_ns.doc(params={"app_id": "Application ID", **query_params_from_model(StatisticTimeRangeQuery)})
    @console_ns.response(
        200,
        "Daily message statistics retrieved successfully",
@@ -185,15 +198,14 @@ WHERE
            for i in rs:
                response_data.append({"date": str(i.date), "message_count": i.message_count})

-        return jsonify({"data": response_data})
+        return dump_response(DailyMessageStatisticResponse, {"data": response_data})


@console_ns.route("/apps/<uuid:app_id>/statistics/daily-conversations")
 class DailyConversationStatistic(Resource):
    @console_ns.doc("get_daily_conversation_statistics")
    @console_ns.doc(description="Get daily conversation statistics for an application")
-    @console_ns.doc(params={"app_id": "Application ID"})
-    @console_ns.doc(params=query_params_from_model(StatisticTimeRangeQuery))
+    @console_ns.doc(params={"app_id": "Application ID", **query_params_from_model(StatisticTimeRangeQuery)})
    @console_ns.response(
        200,
        "Daily conversation statistics retrieved successfully",
@@ -245,15 +257,14 @@ WHERE
            for i in rs:
                response_data.append({"date": str(i.date), "conversation_count": i.conversation_count})

-        return jsonify({"data": response_data})
+        return dump_response(DailyConversationStatisticResponse, {"data": response_data})


@console_ns.route("/apps/<uuid:app_id>/statistics/daily-end-users")
 class DailyTerminalsStatistic(Resource):
    @console_ns.doc("get_daily_terminals_statistics")
    @console_ns.doc(description="Get daily terminal/end-user statistics for an application")
-    @console_ns.doc(params={"app_id": "Application ID"})
-    @console_ns.doc(params=query_params_from_model(StatisticTimeRangeQuery))
+    @console_ns.doc(params={"app_id": "Application ID", **query_params_from_model(StatisticTimeRangeQuery)})
    @console_ns.response(
        200,
        "Daily terminal statistics retrieved successfully",
@@ -306,15 +317,14 @@ WHERE
            for i in rs:
                response_data.append({"date": str(i.date), "terminal_count": i.terminal_count})

-        return jsonify({"data": response_data})
+        return dump_response(DailyTerminalStatisticResponse, {"data": response_data})


@console_ns.route("/apps/<uuid:app_id>/statistics/token-costs")
 class DailyTokenCostStatistic(Resource):
    @console_ns.doc("get_daily_token_cost_statistics")
    @console_ns.doc(description="Get daily token cost statistics for an application")
-    @console_ns.doc(params={"app_id": "Application ID"})
-    @console_ns.doc(params=query_params_from_model(StatisticTimeRangeQuery))
+    @console_ns.doc(params={"app_id": "Application ID", **query_params_from_model(StatisticTimeRangeQuery)})
    @console_ns.response(
        200,
        "Daily token cost statistics retrieved successfully",
@@ -370,15 +380,14 @@ WHERE
                    {"date": str(i.date), "token_count": i.token_count, "total_price": i.total_price, "currency": "USD"}
                )

-        return jsonify({"data": response_data})
+        return dump_response(DailyTokenCostStatisticResponse, {"data": response_data})


@console_ns.route("/apps/<uuid:app_id>/statistics/average-session-interactions")
 class AverageSessionInteractionStatistic(Resource):
    @console_ns.doc("get_average_session_interaction_statistics")
    @console_ns.doc(description="Get average session interaction statistics for an application")
-    @console_ns.doc(params={"app_id": "Application ID"})
-    @console_ns.doc(params=query_params_from_model(StatisticTimeRangeQuery))
+    @console_ns.doc(params={"app_id": "Application ID", **query_params_from_model(StatisticTimeRangeQuery)})
    @console_ns.response(
        200,
        "Average session interaction statistics retrieved successfully",
@@ -450,15 +459,14 @@ ORDER BY
                    {"date": str(i.date), "interactions": float(i.interactions.quantize(Decimal("0.01")))}
                )

-        return jsonify({"data": response_data})
+        return dump_response(AverageSessionInteractionStatisticResponse, {"data": response_data})


@console_ns.route("/apps/<uuid:app_id>/statistics/user-satisfaction-rate")
 class UserSatisfactionRateStatistic(Resource):
    @console_ns.doc("get_user_satisfaction_rate_statistics")
    @console_ns.doc(description="Get user satisfaction rate statistics for an application")
-    @console_ns.doc(params={"app_id": "Application ID"})
-    @console_ns.doc(params=query_params_from_model(StatisticTimeRangeQuery))
+    @console_ns.doc(params={"app_id": "Application ID", **query_params_from_model(StatisticTimeRangeQuery)})
    @console_ns.response(
        200,
        "User satisfaction rate statistics retrieved successfully",
@@ -520,15 +528,14 @@ WHERE
                    }
                )

-        return jsonify({"data": response_data})
+        return dump_response(UserSatisfactionRateStatisticResponse, {"data": response_data})


@console_ns.route("/apps/<uuid:app_id>/statistics/average-response-time")
 class AverageResponseTimeStatistic(Resource):
    @console_ns.doc("get_average_response_time_statistics")
    @console_ns.doc(description="Get average response time statistics for an application")
-    @console_ns.doc(params={"app_id": "Application ID"})
-    @console_ns.doc(params=query_params_from_model(StatisticTimeRangeQuery))
+    @console_ns.doc(params={"app_id": "Application ID", **query_params_from_model(StatisticTimeRangeQuery)})
    @console_ns.response(
        200,
        "Average response time statistics retrieved successfully",
@@ -581,15 +588,14 @@ WHERE
            for i in rs:
                response_data.append({"date": str(i.date), "latency": round(i.latency * 1000, 4)})

-        return jsonify({"data": response_data})
+        return dump_response(AverageResponseTimeStatisticResponse, {"data": response_data})


@console_ns.route("/apps/<uuid:app_id>/statistics/tokens-per-second")
 class TokensPerSecondStatistic(Resource):
    @console_ns.doc("get_tokens_per_second_statistics")
    @console_ns.doc(description="Get tokens per second statistics for an application")
-    @console_ns.doc(params={"app_id": "Application ID"})
-    @console_ns.doc(params=query_params_from_model(StatisticTimeRangeQuery))
+    @console_ns.doc(params={"app_id": "Application ID", **query_params_from_model(StatisticTimeRangeQuery)})
    @console_ns.response(
        200,
        "Tokens per second statistics retrieved successfully",
@@ -645,4 +651,4 @@ WHERE
            for i in rs:
                response_data.append({"date": str(i.date), "tps": round(i.tokens_per_second, 4)})

-        return jsonify({"data": response_data})
+        return dump_response(TokensPerSecondStatisticResponse, {"data": response_data})
@@ -6,7 +6,7 @@ from typing import Any, NotRequired, TypedDict, cast

 from flask import abort, request
 from flask_restx import Resource, fields
-from pydantic import AliasChoices, BaseModel, Field, RootModel, ValidationError, field_validator
+from pydantic import AliasChoices, BaseModel, ConfigDict, Field, RootModel, ValidationError, field_validator
 from sqlalchemy.orm import Session, sessionmaker
 from werkzeug.exceptions import BadRequest, Forbidden, InternalServerError, NotFound

@@ -27,7 +27,7 @@ from controllers.console.app.error import (
    DraftWorkflowNotSync,
 )
 from controllers.console.app.permission_keys import get_app_permission_keys
-from controllers.console.app.wraps import get_app_model
+from controllers.console.app.wraps import get_app_model, with_session
 from controllers.console.wraps import (
    RBACPermission,
    RBACResourceScope,
@@ -78,6 +78,7 @@ from repositories.workflow_collaboration_repository import WORKFLOW_ONLINE_USERS
 from services.app_generate_service import AppGenerateService
 from services.errors.app import IsDraftWorkflowError, WorkflowHashNotEqualError, WorkflowNotFoundError
 from services.errors.llm import InvokeRateLimitError
+from services.workflow_ref_service import WorkflowRefService
 from services.workflow_service import DraftWorkflowDeletionError, WorkflowInUseError, WorkflowService

 logger = logging.getLogger(__name__)
@@ -158,8 +159,71 @@ class ConvertToWorkflowPayload(BaseModel):
    icon_background: str | None = None


+class WorkflowFeatureTogglePayload(BaseModel):
+    model_config = ConfigDict(extra="allow")
+
+    enabled: bool | None = None
+
+
+class WorkflowSuggestedQuestionsAfterAnswerPayload(WorkflowFeatureTogglePayload):
+    model: dict[str, Any] | None = None
+    prompt: str | None = None
+
+
+class WorkflowTextToSpeechPayload(WorkflowFeatureTogglePayload):
+    language: str | None = None
+    voice: str | None = None
+    autoPlay: str | None = None
+
+
+class WorkflowSensitiveWordAvoidancePayload(WorkflowFeatureTogglePayload):
+    type: str | None = None
+    config: dict[str, Any] | None = None
+
+
+class WorkflowFileUploadTransferPayload(WorkflowFeatureTogglePayload):
+    number_limits: int | None = None
+    transfer_methods: list[str] | None = None
+
+
+class WorkflowFileUploadImagePayload(WorkflowFileUploadTransferPayload):
+    detail: str | None = None
+
+
+class WorkflowFileUploadPreviewConfigPayload(BaseModel):
+    mode: str | None = None
+    file_type_list: list[str] | None = None
+
+
+class WorkflowFileUploadPayload(WorkflowFeatureTogglePayload):
+    allowed_file_types: list[str] | None = None
+    allowed_file_extensions: list[str] | None = None
+    allowed_file_upload_methods: list[str] | None = None
+    number_limits: int | None = None
+    image: WorkflowFileUploadImagePayload | None = None
+    document: WorkflowFileUploadTransferPayload | None = None
+    audio: WorkflowFileUploadTransferPayload | None = None
+    video: WorkflowFileUploadTransferPayload | None = None
+    custom: WorkflowFileUploadTransferPayload | None = None
+    preview_config: WorkflowFileUploadPreviewConfigPayload | None = None
+    fileUploadConfig: dict[str, Any] | None = None
+
+
+class WorkflowFeaturesConfigPayload(BaseModel):
+    model_config = ConfigDict(extra="allow")
+
+    opening_statement: str | None = None
+    suggested_questions: list[str] | None = None
+    suggested_questions_after_answer: WorkflowSuggestedQuestionsAfterAnswerPayload | None = None
+    text_to_speech: WorkflowTextToSpeechPayload | None = None
+    speech_to_text: WorkflowFeatureTogglePayload | None = None
+    retriever_resource: WorkflowFeatureTogglePayload | None = None
+    sensitive_word_avoidance: WorkflowSensitiveWordAvoidancePayload | None = None
+    file_upload: WorkflowFileUploadPayload | None = None
+
+
 class WorkflowFeaturesPayload(BaseModel):
-    features: dict[str, Any] = Field(
+    features: WorkflowFeaturesConfigPayload = Field(
        ...,
        description="Workflow feature configuration",
    )
@@ -343,6 +407,15 @@ register_schema_models(
    ConvertToWorkflowPayload,
    WorkflowListQuery,
    WorkflowUpdatePayload,
+    WorkflowFeatureTogglePayload,
+    WorkflowSuggestedQuestionsAfterAnswerPayload,
+    WorkflowTextToSpeechPayload,
+    WorkflowSensitiveWordAvoidancePayload,
+    WorkflowFileUploadTransferPayload,
+    WorkflowFileUploadImagePayload,
+    WorkflowFileUploadPreviewConfigPayload,
+    WorkflowFileUploadPayload,
+    WorkflowFeaturesConfigPayload,
    WorkflowFeaturesPayload,
    WorkflowOnlineUsersPayload,
    DraftWorkflowTriggerRunPayload,
@@ -558,7 +631,8 @@ class AdvancedChatDraftWorkflowRunApi(Resource):
    @get_app_model(mode=[AppMode.ADVANCED_CHAT])
    @with_current_user
    @edit_permission_required
-    def post(self, current_user: Account, app_model: App):
+    @with_session
+    def post(self, session: Session, current_user: Account, app_model: App):
        """
        Run draft workflow
        """
@@ -571,7 +645,12 @@ class AdvancedChatDraftWorkflowRunApi(Resource):

        try:
            response = AppGenerateService.generate(
-                app_model=app_model, user=current_user, args=args, invoke_from=InvokeFrom.DEBUGGER, streaming=True
+                session=session,
+                app_model=app_model,
+                user=current_user,
+                args=args,
+                invoke_from=InvokeFrom.DEBUGGER,
+                streaming=True,
            )

            return helper.compact_generate_response(response)
@@ -972,7 +1051,8 @@ class DraftWorkflowRunApi(Resource):
    @get_app_model(mode=[AppMode.WORKFLOW])
    @with_current_user
    @edit_permission_required
-    def post(self, current_user: Account, app_model: App):
+    @with_session
+    def post(self, session: Session, current_user: Account, app_model: App):
        """
        Run draft workflow
        """
@@ -984,6 +1064,7 @@ class DraftWorkflowRunApi(Resource):

        try:
            response = AppGenerateService.generate(
+                session=session,
                app_model=app_model,
                user=current_user,
                args=args,
@@ -1274,7 +1355,7 @@ class WorkflowFeaturesApi(Resource):
    def post(self, current_user: Account, app_model: App):

        args = WorkflowFeaturesPayload.model_validate(console_ns.payload or {})
-        features = args.features
+        features = args.features.model_dump(mode="json", exclude_unset=True)

        workflow_service = WorkflowService()
        workflow_service.update_draft_workflow_features(app_model=app_model, features=features, account=current_user)
@@ -1406,15 +1487,15 @@ class WorkflowByIdApi(Resource):
            return {"message": "No valid fields to update"}, 400

        workflow_service = WorkflowService()
+        workflow_ref = WorkflowRefService.create_app_workflow_ref(app_model, workflow_id)

        # Create a session and manage the transaction
        with sessionmaker(db.engine, expire_on_commit=False).begin() as session:
            workflow = workflow_service.update_workflow(
                session=session,
-                workflow_id=workflow_id,
-                tenant_id=app_model.tenant_id,
                account_id=current_user.id,
                data=update_data,
+                workflow_ref=workflow_ref,
            )

            if not workflow:
@@ -1434,12 +1515,14 @@ class WorkflowByIdApi(Resource):
        Delete workflow
        """
        workflow_service = WorkflowService()
+        workflow_ref = WorkflowRefService.create_app_workflow_ref(app_model, workflow_id)

        # Create a session and manage the transaction
        with sessionmaker(db.engine).begin() as session:
            try:
                workflow_service.delete_workflow(
-                    session=session, workflow_id=workflow_id, tenant_id=app_model.tenant_id
+                    session=session,
+                    workflow_ref=workflow_ref,
                )
            except WorkflowInUseError as e:
                abort(400, description=str(e))
@@ -1515,7 +1598,8 @@ class DraftWorkflowTriggerRunApi(Resource):
    @get_app_model(mode=[AppMode.WORKFLOW])
    @with_current_user
    @edit_permission_required
-    def post(self, current_user: Account, app_model: App):
+    @with_session
+    def post(self, session: Session, current_user: Account, app_model: App):
        """
        Poll for trigger events and execute full workflow when event arrives
        """
@@ -1543,6 +1627,7 @@ class DraftWorkflowTriggerRunApi(Resource):
            workflow_args[SKIP_PREPARE_USER_INPUTS_KEY] = True
            return helper.compact_generate_response(
                AppGenerateService.generate(
+                    session=session,
                    app_model=app_model,
                    user=current_user,
                    args=workflow_args,
@@ -1665,7 +1750,8 @@ class DraftWorkflowTriggerRunAllApi(Resource):
    @with_current_user
    @edit_permission_required
    @rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_TEST_AND_RUN)
-    def post(self, current_user: Account, app_model: App):
+    @with_session
+    def post(self, session: Session, current_user: Account, app_model: App):
        """
        Full workflow debug when the start node is a trigger
        """
@@ -1697,6 +1783,7 @@ class DraftWorkflowTriggerRunAllApi(Resource):

            workflow_args[SKIP_PREPARE_USER_INPUTS_KEY] = True
            response = AppGenerateService.generate(
+                session=session,
                app_model=app_model,
                user=current_user,
                args=workflow_args,
@@ -189,14 +189,14 @@ class WorkflowCommentReplyUpdate(ResponseModel):

 register_schema_models(
    console_ns,
-    AccountWithRole,
-    WorkflowCommentMentionUsersPayload,
    WorkflowCommentCreatePayload,
    WorkflowCommentUpdatePayload,
    WorkflowCommentReplyPayload,
 )
 register_response_schema_models(
    console_ns,
+    AccountWithRole,
+    WorkflowCommentMentionUsersPayload,
    WorkflowCommentAccount,
    WorkflowCommentReply,
    WorkflowCommentMention,
@@ -6,7 +6,7 @@ from uuid import UUID

 from flask import Response, request
 from flask_restx import Resource, fields, marshal, marshal_with
-from pydantic import BaseModel, Field
+from pydantic import BaseModel, ConfigDict, Field
 from sqlalchemy.orm import sessionmaker

 from controllers.common.errors import InvalidArgumentError, NotFoundError
@@ -79,15 +79,33 @@ class WorkflowDraftVariableUpdatePayload(BaseModel):
    value: Any | None = Field(default=None, description="Variable value")


+class WorkflowVariableItemPayload(BaseModel):
+    model_config = ConfigDict(extra="allow")
+
+    id: str | None = None
+    name: str | None = None
+    value_type: str | None = None
+    value: Any | None = None
+    description: str | None = None
+
+
+class ConversationVariableItemPayload(WorkflowVariableItemPayload):
+    pass
+
+
+class EnvironmentVariableItemPayload(WorkflowVariableItemPayload):
+    pass
+
+
 class ConversationVariableUpdatePayload(BaseModel):
-    conversation_variables: list[dict[str, Any]] = Field(
+    conversation_variables: list[ConversationVariableItemPayload] = Field(
        ...,
        description="Conversation variables for the draft workflow",
    )


 class EnvironmentVariableUpdatePayload(BaseModel):
-    environment_variables: list[dict[str, Any]] = Field(
+    environment_variables: list[EnvironmentVariableItemPayload] = Field(
        ...,
        description="Environment variables for the draft workflow",
    )
@@ -114,7 +132,9 @@ register_schema_models(
    console_ns,
    WorkflowDraftVariableListQuery,
    WorkflowDraftVariableUpdatePayload,
+    ConversationVariableItemPayload,
    ConversationVariableUpdatePayload,
+    EnvironmentVariableItemPayload,
    EnvironmentVariableUpdatePayload,
 )
 register_response_schema_models(console_ns, SimpleResultResponse, EnvironmentVariableListResponse)
@@ -615,7 +635,9 @@ class ConversationVariableCollectionApi(Resource):

        workflow_service = WorkflowService()

-        conversation_variables_list = payload.conversation_variables
+        conversation_variables_list = [
+            variable.model_dump(mode="json", exclude_unset=True) for variable in payload.conversation_variables
+        ]
        conversation_variables = [
            variable_factory.build_conversation_variable_from_mapping(obj) for obj in conversation_variables_list
        ]
@@ -707,7 +729,9 @@ class EnvironmentVariableCollectionApi(Resource):

        workflow_service = WorkflowService()

-        environment_variables_list = payload.environment_variables
+        environment_variables_list = [
+            variable.model_dump(mode="json", exclude_unset=True) for variable in payload.environment_variables
+        ]
        environment_variables = [
            variable_factory.build_environment_variable_from_mapping(obj) for obj in environment_variables_list
        ]
@@ -23,6 +23,7 @@ from controllers.console.wraps import (
    with_current_user,
 )
 from core.workflow.human_input_forms import load_form_tokens_by_form_id as _load_form_tokens_by_form_id
+from core.workflow.nodes.human_input.pause_reason import HumanInputRequired
 from extensions.ext_database import db
 from fields.base import ResponseModel
 from fields.workflow_run_fields import (
@@ -33,7 +34,6 @@ from fields.workflow_run_fields import (
    WorkflowRunNodeExecutionResponse,
    WorkflowRunPaginationResponse,
 )
-from graphon.entities.pause_reason import HumanInputRequired
 from graphon.enums import WorkflowExecutionStatus
 from libs.archive_storage import ArchiveStorageNotConfiguredError, get_archive_storage
 from libs.custom_inputs import time_duration
@@ -12,6 +12,7 @@ from configs import dify_config
 from controllers.common.schema import query_params_from_model, register_schema_models
 from extensions.ext_database import db
 from fields.base import ResponseModel
+from libs.helper import dump_response
 from libs.login import login_required
 from models.enums import AppTriggerStatus
 from models.model import App, AppMode
@@ -121,7 +122,7 @@ class WebhookTriggerApi(Resource):
            if not webhook_trigger:
                raise NotFound("Webhook trigger not found for this node")

-            return WebhookTriggerResponse.model_validate(webhook_trigger, from_attributes=True).model_dump(mode="json")
+            return dump_response(WebhookTriggerResponse, webhook_trigger)


@console_ns.route("/apps/<uuid:app_id>/triggers")
@@ -160,9 +161,7 @@ class AppTriggersApi(Resource):
            else:
                trigger.icon = ""  # type: ignore

-        return WorkflowTriggerListResponse.model_validate({"data": triggers}, from_attributes=True).model_dump(
-            mode="json"
-        )
+        return dump_response(WorkflowTriggerListResponse, {"data": triggers})


@console_ns.route("/apps/<uuid:app_id>/trigger-enable")
@@ -204,4 +203,4 @@ class AppTriggerEnableApi(Resource):
        else:
            trigger.icon = ""  # type: ignore

-        return WorkflowTriggerResponse.model_validate(trigger, from_attributes=True).model_dump(mode="json")
+        return dump_response(WorkflowTriggerResponse, trigger)
@@ -1,26 +1,25 @@
 """Controller decorators for console app resources.

-`with_session` opens one SQLAlchemy session for a request handler and injects it
-as the first argument after `self`. Handlers use a transaction by default so
-migrated write paths keep commit/rollback handling; pure read handlers may opt
-out with `write=False`. App-loading decorators prefer that injected session when
-present, while still supporting existing handlers that have not been migrated
-yet and still rely on Flask-SQLAlchemy's scoped `db.session`.
+App-loading decorators prefer a session injected by
+`controllers.common.session.with_session` when present, while still supporting
+existing handlers that have not been migrated yet and still rely on
+Flask-SQLAlchemy's scoped `db.session`.
 """

 from collections.abc import Callable
 from functools import wraps
-from typing import Concatenate, cast, overload
+from typing import cast, overload

 from sqlalchemy import select
 from sqlalchemy.orm import Session

+from controllers.common.session import with_session
 from controllers.console.app.error import AppNotFoundError
-from core.db.session_factory import session_factory
 from extensions.ext_database import db
 from libs.login import current_account_with_tenant
-from models import App, AppMode, TrialApp
-from services.recommended_app_service import RecommendedAppService
+from models import App, AppMode
+
+__all__ = ["get_app_model", "get_app_model_with_trial", "with_session"]


 def _load_app_model(session: Session, app_id: str) -> App | None:
@@ -42,55 +41,10 @@ def _load_app_model_from_scoped_session(app_id: str) -> App | None:


 def _load_app_model_with_trial(app_id: str) -> App | None:
-    """Load a normal app through its trial registration without applying current-tenant scope."""
-    app_model = db.session.scalar(
-        select(App).join(TrialApp, TrialApp.app_id == App.id).where(App.id == app_id, App.status == "normal").limit(1)
-    )
+    app_model = db.session.scalar(select(App).where(App.id == app_id, App.status == "normal").limit(1))
    return app_model


-@overload
-def with_session[T, **P, R](
-    view: Callable[Concatenate[T, Session, P], R],
-    *,
-    write: bool = True,
-) -> Callable[Concatenate[T, P], R]: ...
-
-
-@overload
-def with_session[T, **P, R](
-    view: None = None,
-    *,
-    write: bool = True,
-) -> Callable[[Callable[Concatenate[T, Session, P], R]], Callable[Concatenate[T, P], R]]: ...
-
-
-def with_session[T, **P, R](
-    view: Callable[Concatenate[T, Session, P], R] | None = None,
-    *,
-    write: bool = True,
-) -> (
-    Callable[Concatenate[T, P], R] | Callable[[Callable[Concatenate[T, Session, P], R]], Callable[Concatenate[T, P], R]]
-):
-    """Inject a request-scoped session, using a transaction only for write handlers."""
-
-    def decorator(view: Callable[Concatenate[T, Session, P], R]) -> Callable[Concatenate[T, P], R]:
-        @wraps(view)
-        def wrapper(self: T, *args: P.args, **kwargs: P.kwargs) -> R:
-            if write:
-                with session_factory.get_session_maker().begin() as session:
-                    return view(self, session, *args, **kwargs)
-
-            with session_factory.create_session() as session:
-                return view(self, session, *args, **kwargs)
-
-        return wrapper
-
-    if view is None:
-        return decorator
-    return decorator(view)
-
-
 def _get_injected_session(args: tuple[object, ...]) -> Session | None:
    """Return the request session inserted by `with_session`, if this handler has been migrated."""
    if len(args) < 2:
@@ -199,8 +153,6 @@ def get_app_model_with_trial[**P, R](
    *,
    mode: AppMode | list[AppMode] | None = None,
 ) -> Callable[P, R] | Callable[[Callable[P, R]], Callable[P, R]]:
-    """Inject an app registered for trial or available from the recommended catalog."""
-
    def decorator(view_func: Callable[P, R]) -> Callable[P, R]:
        @wraps(view_func)
        def decorated_view(*args: P.args, **kwargs: P.kwargs) -> R:
@@ -213,8 +165,6 @@ def get_app_model_with_trial[**P, R](
            del kwargs["app_id"]

            app_model = _load_app_model_with_trial(app_id)
-            if app_model is None:
-                app_model = RecommendedAppService.get_app(app_id, session=db.session())

            if not app_model:
                raise AppNotFoundError()
@@ -23,9 +23,9 @@ from libs.password import valid_password
 from models import Account
 from services.account_service import AccountService
 from services.billing_service import BillingService
-from services.errors.account import AccountRegisterError, SeatsLimitExceededError
+from services.errors.account import AccountRegisterError

-from ..error import AccountInFreezeError, EmailSendIpLimitError, SeatsLimitExceeded
+from ..error import AccountInFreezeError, EmailSendIpLimitError
 from ..wraps import email_password_login_enabled, email_register_enabled, setup_required


@@ -208,7 +208,5 @@ class EmailRegisterResetApi(Resource):
                timezone=timezone,
                session=db.session,
            )
-        except SeatsLimitExceededError:
-            raise SeatsLimitExceeded()
        except AccountRegisterError:
            raise AccountInFreezeError()
@@ -25,7 +25,6 @@ from controllers.console.error import (
    AccountNotFound,
    EmailSendIpLimitError,
    NotAllowedCreateWorkspace,
-    SeatsLimitExceeded,
    WorkspacesLimitExceeded,
 )
 from controllers.console.wraps import (
@@ -52,7 +51,7 @@ from models.account import Account
 from services.account_service import AccountService, InvitationDetailDict, RegisterService, TenantService
 from services.billing_service import BillingService
 from services.entities.auth_entities import LoginFailureReason, LoginPayloadBase
-from services.errors.account import AccountRegisterError, SeatsLimitExceededError
+from services.errors.account import AccountRegisterError
 from services.errors.workspace import WorkSpaceNotAllowedCreateError, WorkspacesLimitExceededError
 from services.feature_service import FeatureService

@@ -318,8 +317,6 @@ class EmailCodeLoginApi(Resource):
                )
            except WorkSpaceNotAllowedCreateError:
                raise NotAllowedCreateWorkspace()
-            except SeatsLimitExceededError:
-                raise SeatsLimitExceeded()
            except AccountRegisterError:
                _log_console_login_failure(email=user_email, reason=LoginFailureReason.ACCOUNT_IN_FREEZE)
                raise AccountInFreezeError()
@@ -25,7 +25,7 @@ from libs.token import (
 from models import Account, AccountStatus
 from services.account_service import AccountService, RegisterService, TenantService
 from services.billing_service import BillingService
-from services.errors.account import AccountNotFoundError, AccountRegisterError, SeatsLimitExceededError
+from services.errors.account import AccountNotFoundError, AccountRegisterError
 from services.errors.workspace import WorkSpaceNotAllowedCreateError, WorkSpaceNotFoundError
 from services.feature_service import FeatureService

@@ -38,7 +38,6 @@ class OAuthLoginQuery(BaseModel):
    invite_token: str | None = Field(default=None, description="Optional invitation token")
    timezone: str | None = Field(default=None, description="Preferred timezone")
    language: str | None = Field(default=None, description="Preferred interface language")
-    redirect_url: str | None = Field(default=None, description="Relative page to resume after login")


 class OAuthCallbackQuery(BaseModel):
@@ -88,36 +87,6 @@ def _validated_language(value: str | None) -> str | None:
    return None


-def _url_origin(url: str) -> tuple[str, str, int] | None:
-    parsed_url = urllib.parse.urlsplit(url)
-    if parsed_url.scheme not in {"http", "https"} or parsed_url.hostname is None:
-        return None
-
-    try:
-        port = parsed_url.port
-    except ValueError:
-        return None
-
-    if port is None:
-        port = 443 if parsed_url.scheme == "https" else 80
-    return parsed_url.scheme, parsed_url.hostname, port
-
-
-def _get_redirect_target(redirect_url: str | None) -> str:
-    if not redirect_url:
-        return dify_config.CONSOLE_WEB_URL
-
-    parsed_url = urllib.parse.urlsplit(redirect_url)
-    normalized_path = redirect_url.lstrip().replace("\\", "/")
-    if not parsed_url.scheme and not parsed_url.netloc and not normalized_path.startswith("//"):
-        return redirect_url
-
-    redirect_origin = _url_origin(redirect_url)
-    if redirect_origin is not None and redirect_origin == _url_origin(dify_config.CONSOLE_WEB_URL):
-        return redirect_url
-    return dify_config.CONSOLE_WEB_URL
-
-
 def _preferred_interface_language(language: str | None = None) -> str:
    if language:
        return language
@@ -140,7 +109,6 @@ class OAuthLogin(Resource):
        invite_token = request.args.get("invite_token") or None
        timezone = _validated_timezone(request.args.get("timezone") or None)
        language = _validated_language(request.args.get("language") or None)
-        redirect_url = request.args.get("redirect_url") or None
        OAUTH_PROVIDERS = get_oauth_providers()
        with current_app.app_context():
            oauth_provider = OAUTH_PROVIDERS.get(provider)
@@ -151,7 +119,6 @@ class OAuthLogin(Resource):
            invite_token=invite_token,
            timezone=timezone,
            language=language,
-            redirect_url=redirect_url,
        )
        return redirect(auth_url)

@@ -177,7 +144,6 @@ class OAuthCallback(Resource):
        invite_token = oauth_state.get("invite_token")
        timezone = _validated_timezone(oauth_state.get("timezone"))
        language = _validated_language(oauth_state.get("language"))
-        redirect_url = oauth_state.get("redirect_url")

        if not code:
            return {"error": "Authorization code is required"}, 400
@@ -216,8 +182,6 @@ class OAuthCallback(Resource):
                f"{dify_config.CONSOLE_WEB_URL}/signin"
                "?message=Workspace not found, please contact system admin to invite you to join in a workspace."
            )
-        except SeatsLimitExceededError:
-            return redirect(f"{dify_config.CONSOLE_WEB_URL}/signin?message=Licensed seats limit exceeded.")
        except AccountRegisterError as e:
            return redirect(f"{dify_config.CONSOLE_WEB_URL}/signin?message={e.description}")

@@ -246,9 +210,9 @@ class OAuthCallback(Resource):
            ip_address=extract_remote_ip(request),
        )

-        target_url = _get_redirect_target(redirect_url)
-        query_char = "&" if "?" in target_url else "?"
-        target_url = f"{target_url}{query_char}oauth_new_user={str(oauth_new_user).lower()}"
+        base_url = dify_config.CONSOLE_WEB_URL
+        query_char = "&" if "?" in base_url else "?"
+        target_url = f"{base_url}{query_char}oauth_new_user={str(oauth_new_user).lower()}"
        response = redirect(target_url)

        set_access_token_to_cookie(request, response, token_pair.access_token)
@@ -16,6 +16,8 @@ from controllers.console.wraps import (
    with_current_user,
 )
 from enums.cloud_plan import CloudPlan
+from extensions.ext_database import db
+from fields.base import ResponseModel
 from libs.login import login_required
 from models import Account
 from services.billing_service import BillingService
@@ -34,8 +36,12 @@ class BillingResponse(RootModel[dict[str, Any]]):
    root: dict[str, Any]


+class BillingInvoiceResponse(ResponseModel):
+    url: str
+
+
 register_schema_models(console_ns, SubscriptionQuery, PartnerTenantsPayload)
-register_response_schema_models(console_ns, BillingResponse)
+register_response_schema_models(console_ns, BillingResponse, BillingInvoiceResponse)


@console_ns.route("/billing/subscription")
@@ -50,13 +56,13 @@ class Subscription(Resource):
    @with_current_tenant_id
    def get(self, current_tenant_id: str, current_user: Account):
        args = SubscriptionQuery.model_validate(request.args.to_dict(flat=True))
-        BillingService.is_tenant_owner_or_admin(current_user)
+        BillingService.is_tenant_owner_or_admin(db.session, current_user)
        return BillingService.get_subscription(args.plan, args.interval, current_user.email, current_tenant_id)


@console_ns.route("/billing/invoices")
 class Invoices(Resource):
-    @console_ns.response(200, "Success", console_ns.models[BillingResponse.__name__])
+    @console_ns.response(200, "Success", console_ns.models[BillingInvoiceResponse.__name__])
    @setup_required
    @login_required
    @account_initialization_required
@@ -64,7 +70,7 @@ class Invoices(Resource):
    @with_current_user
    @with_current_tenant_id
    def get(self, current_tenant_id: str, current_user: Account):
-        BillingService.is_tenant_owner_or_admin(current_user)
+        BillingService.is_tenant_owner_or_admin(db.session, current_user)
        return BillingService.get_invoices(current_user.email, current_tenant_id)


@@ -243,72 +243,71 @@ class DataSourceNotionListApi(Resource):
        if not credential:
            raise NotFound("Credential not found.")
        exist_page_ids = []
-        with sessionmaker(db.engine).begin() as session:
-            # import notion in the exist dataset
-            if query.dataset_id:
-                dataset = DatasetService.get_dataset(query.dataset_id)
-                if not dataset:
-                    raise NotFound("Dataset not found.")
-                if dataset.data_source_type != "notion_import":
-                    raise ValueError("Dataset is not notion type.")
+        # import notion in the exist dataset
+        if query.dataset_id:
+            dataset = DatasetService.get_dataset(query.dataset_id, db.session)
+            if not dataset:
+                raise NotFound("Dataset not found.")
+            if dataset.data_source_type != "notion_import":
+                raise ValueError("Dataset is not notion type.")

-                documents = session.scalars(
-                    select(Document).where(
-                        Document.dataset_id == query.dataset_id,
-                        Document.tenant_id == current_tenant_id,
-                        Document.data_source_type == "notion_import",
-                        Document.enabled.is_(True),
-                    )
-                ).all()
-                if documents:
-                    for document in documents:
-                        data_source_info = json.loads(document.data_source_info)
-                        exist_page_ids.append(data_source_info["notion_page_id"])
-            # get all authorized pages
-            from core.datasource.datasource_manager import DatasourceManager
-
-            datasource_runtime = DatasourceManager.get_datasource_runtime(
-                provider_id="langgenius/notion_datasource/notion_datasource",
-                datasource_name="notion_datasource",
-                tenant_id=current_tenant_id,
-                datasource_type=DatasourceProviderType.ONLINE_DOCUMENT,
-            )
-            datasource_provider_service = DatasourceProviderService()
-            if credential:
-                datasource_runtime.runtime.credentials = credential
-            datasource_runtime = cast(OnlineDocumentDatasourcePlugin, datasource_runtime)
-            online_document_result: Generator[OnlineDocumentPagesMessage, None, None] = (
-                datasource_runtime.get_online_document_pages(
-                    user_id=current_user.id,
-                    datasource_parameters={},
-                    provider_type=datasource_runtime.datasource_provider_type(),
+            documents = db.session.scalars(
+                select(Document).where(
+                    Document.dataset_id == query.dataset_id,
+                    Document.tenant_id == current_tenant_id,
+                    Document.data_source_type == "notion_import",
+                    Document.enabled.is_(True),
                )
+            ).all()
+            if documents:
+                for document in documents:
+                    data_source_info = json.loads(document.data_source_info)
+                    exist_page_ids.append(data_source_info["notion_page_id"])
+        # get all authorized pages
+        from core.datasource.datasource_manager import DatasourceManager
+
+        datasource_runtime = DatasourceManager.get_datasource_runtime(
+            provider_id="langgenius/notion_datasource/notion_datasource",
+            datasource_name="notion_datasource",
+            tenant_id=current_tenant_id,
+            datasource_type=DatasourceProviderType.ONLINE_DOCUMENT,
+        )
+        datasource_provider_service = DatasourceProviderService()
+        if credential:
+            datasource_runtime.runtime.credentials = credential
+        datasource_runtime = cast(OnlineDocumentDatasourcePlugin, datasource_runtime)
+        online_document_result: Generator[OnlineDocumentPagesMessage, None, None] = (
+            datasource_runtime.get_online_document_pages(
+                user_id=current_user.id,
+                datasource_parameters={},
+                provider_type=datasource_runtime.datasource_provider_type(),
            )
-            try:
-                pages = []
-                workspace_info = {}
-                for message in online_document_result:
-                    result = message.result
-                    for info in result:
-                        workspace_info = {
-                            "workspace_id": info.workspace_id,
-                            "workspace_name": info.workspace_name,
-                            "workspace_icon": info.workspace_icon,
+        )
+        try:
+            pages = []
+            workspace_info = {}
+            for message in online_document_result:
+                result = message.result
+                for info in result:
+                    workspace_info = {
+                        "workspace_id": info.workspace_id,
+                        "workspace_name": info.workspace_name,
+                        "workspace_icon": info.workspace_icon,
+                    }
+                    for page in info.pages:
+                        page_info = {
+                            "page_id": page.page_id,
+                            "page_name": page.page_name,
+                            "type": page.type,
+                            "parent_id": page.parent_id,
+                            "is_bound": page.page_id in exist_page_ids,
+                            "page_icon": page.page_icon,
                        }
-                        for page in info.pages:
-                            page_info = {
-                                "page_id": page.page_id,
-                                "page_name": page.page_name,
-                                "type": page.type,
-                                "parent_id": page.parent_id,
-                                "is_bound": page.page_id in exist_page_ids,
-                                "page_icon": page.page_icon,
-                            }
-                            pages.append(page_info)
-            except Exception as e:
-                raise e
-            notion_info = [{**workspace_info, "pages": pages}] if workspace_info else []
-            return dump_response(NotionIntegrateInfoListResponse, {"notion_info": notion_info}), 200
+                        pages.append(page_info)
+        except Exception as e:
+            raise e
+        notion_info = [{**workspace_info, "pages": pages}] if workspace_info else []
+        return dump_response(NotionIntegrateInfoListResponse, {"notion_info": notion_info}), 200


@console_ns.route("/notion/pages/<uuid:page_id>/<string:page_type>/preview")
@@ -401,11 +400,11 @@ class DataSourceNotionDatasetSyncApi(Resource):
    @rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_CREATE_AND_MANAGEMENT)
    def get(self, dataset_id: UUID) -> tuple[dict[str, str], int]:
        dataset_id_str = str(dataset_id)
-        dataset = DatasetService.get_dataset(dataset_id_str)
+        dataset = DatasetService.get_dataset(dataset_id_str, db.session)
        if dataset is None:
            raise NotFound("Dataset not found.")

-        documents = DocumentService.get_document_by_dataset_id(dataset_id_str)
+        documents = DocumentService.get_document_by_dataset_id(dataset_id_str, db.session)
        for document in documents:
            document_indexing_sync_task.delay(dataset_id_str, document.id)
        return {"result": "success"}, 200
@@ -421,11 +420,11 @@ class DataSourceNotionDocumentSyncApi(Resource):
    def get(self, dataset_id: UUID, document_id: UUID) -> tuple[dict[str, str], int]:
        dataset_id_str = str(dataset_id)
        document_id_str = str(document_id)
-        dataset = DatasetService.get_dataset(dataset_id_str)
+        dataset = DatasetService.get_dataset(dataset_id_str, db.session)
        if dataset is None:
            raise NotFound("Dataset not found.")

-        document = DocumentService.get_document(dataset_id_str, document_id_str)
+        document = DocumentService.get_document(dataset_id_str, document_id_str, session=db.session)
        if document is None:
            raise NotFound("Document not found.")
        document_indexing_sync_task.delay(dataset_id_str, document_id_str)
@@ -6,6 +6,7 @@ from flask import request
 from flask_restx import Resource
 from pydantic import BaseModel, Field, field_validator, model_validator
 from sqlalchemy import func, select
+from sqlalchemy.orm import Session
 from werkzeug.exceptions import Forbidden, NotFound

 import services
@@ -15,6 +16,7 @@ from controllers.common.schema import query_params_from_model, register_response
 from controllers.console import console_ns
 from controllers.console.apikey import ApiKeyItem, ApiKeyList
 from controllers.console.app.error import ProviderNotInitializeError
+from controllers.console.app.wraps import with_session
 from controllers.console.datasets.error import DatasetInUseError, DatasetNameDuplicateError, IndexingEstimateError
 from controllers.console.wraps import (
    RBACPermission,
@@ -50,8 +52,6 @@ from models.provider_ids import ModelProviderID
 from services.api_token_service import ApiTokenCache
 from services.dataset_service import DatasetPermissionService, DatasetService, DocumentService
 from services.enterprise import rbac_service as enterprise_rbac_service
-from services.enterprise.rbac_service import RBACResourceWhitelistScope, ReplaceMemberBindings
-from tasks.initialize_created_app_rbac_access_task import initialize_created_app_rbac_access_task

 register_response_schema_models(console_ns, ApiBaseUrlResponse, SimpleResultResponse, UsageCheckResponse)

@@ -540,7 +540,8 @@ class DatasetListApi(Resource):
    @cloud_edition_billing_rate_limit_check("knowledge")
    @with_current_user
    @with_current_tenant_id
-    def post(self, current_tenant_id: str, current_user: Account):
+    @with_session
+    def post(self, session: Session, current_tenant_id: str, current_user: Account):
        payload = DatasetCreatePayload.model_validate(console_ns.payload or {})

        # The role of the current user in the ta table must be admin, owner, or editor, or dataset_operator
@@ -554,6 +555,7 @@ class DatasetListApi(Resource):

        try:
            dataset = DatasetService.create_empty_dataset(
+                session=session,
                tenant_id=current_tenant_id,
                name=payload.name,
                description=payload.description,
@@ -567,26 +569,16 @@ class DatasetListApi(Resource):
        except services.errors.dataset.DatasetNameDuplicateError:
            raise DatasetNameDuplicateError()

-        if dify_config.RBAC_ENABLED:
-            if permission == DatasetPermissionEnum.ALL_TEAM:
-                enterprise_rbac_service.RBACService.DatasetAccess.replace_whitelist(
-                    current_tenant_id,
-                    current_user.id,
-                    dataset.id,
-                    ReplaceMemberBindings(scope=RBACResourceWhitelistScope.ALL),
-                )
-                initialize_created_app_rbac_access_task.delay(current_tenant_id, current_user.id, dataset_id=dataset.id)
-
        permission_keys_map = enterprise_rbac_service.RBACService.DatasetPermissions.batch_get(
-            str(current_tenant_id),
+            current_tenant_id,
            current_user.id,
-            [str(dataset.id)],
+            [dataset.id],
        )

        item = DatasetDetailWithPartialMembersResponse.model_validate(dataset, from_attributes=True).model_dump(
            mode="json"
        )
-        item["permission_keys"] = permission_keys_map.get(str(dataset.id), [])
+        item["permission_keys"] = permission_keys_map.get(dataset.id, [])
        return item, 201


@@ -610,15 +602,15 @@ class DatasetApi(Resource):
    @with_current_tenant_id
    def get(self, current_tenant_id: str, current_user: Account, dataset_id: UUID):
        dataset_id_str = str(dataset_id)
-        dataset = DatasetService.get_dataset(dataset_id_str)
+        dataset = DatasetService.get_dataset(dataset_id_str, db.session)
        if dataset is None:
            raise NotFound("Dataset not found.")
        try:
-            DatasetService.check_dataset_permission(dataset, current_user)
+            DatasetService.check_dataset_permission(dataset, current_user, db.session)
        except services.errors.account.NoPermissionError as e:
            raise Forbidden(str(e))
        permissions = enterprise_rbac_service.RBACService.MyPermissions.get(
-            str(current_tenant_id),
+            current_tenant_id,
            current_user.id,
            dataset_id=dataset_id_str,
        )
@@ -630,7 +622,7 @@ class DatasetApi(Resource):
                provider_id = ModelProviderID(dataset.embedding_model_provider)
                data["embedding_model_provider"] = str(provider_id)
        if data.get("permission") == "partial_members":
-            part_users_list = DatasetPermissionService.get_dataset_partial_member_list(dataset_id_str)
+            part_users_list = DatasetPermissionService.get_dataset_partial_member_list(dataset_id_str, db.session)
            data.update({"partial_member_list": part_users_list})

        # check embedding setting
@@ -671,9 +663,10 @@ class DatasetApi(Resource):
    @with_current_user
    @with_current_tenant_id
    @rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_EDIT)
-    def patch(self, current_tenant_id: str, current_user: Account, dataset_id: UUID):
+    @with_session
+    def patch(self, session: Session, current_tenant_id: str, current_user: Account, dataset_id: UUID):
        dataset_id_str = str(dataset_id)
-        dataset = DatasetService.get_dataset(dataset_id_str)
+        dataset = DatasetService.get_dataset(dataset_id_str, db.session)
        if dataset is None:
            raise NotFound("Dataset not found.")

@@ -692,16 +685,16 @@ class DatasetApi(Resource):
        # The role of the current user in the ta table must be admin, owner, editor, or dataset_operator
        if not dify_config.RBAC_ENABLED:
            DatasetPermissionService.check_permission(
-                current_user, dataset, payload.permission, payload.partial_member_list
+                session, current_user, dataset, payload.permission, payload.partial_member_list
            )

-        dataset = DatasetService.update_dataset(dataset_id_str, payload_data, current_user)
+        dataset = DatasetService.update_dataset(session, dataset_id_str, payload_data, current_user)

        if dataset is None:
            raise NotFound("Dataset not found.")

        permission_keys_map = enterprise_rbac_service.RBACService.DatasetPermissions.batch_get(
-            str(current_tenant_id),
+            current_tenant_id,
            current_user.id,
            [dataset_id_str],
        )
@@ -710,12 +703,14 @@ class DatasetApi(Resource):
        tenant_id = current_tenant_id

        if payload.partial_member_list is not None and payload.permission == DatasetPermissionEnum.PARTIAL_TEAM:
-            DatasetPermissionService.update_partial_member_list(tenant_id, dataset_id_str, payload.partial_member_list)
+            DatasetPermissionService.update_partial_member_list(
+                tenant_id, dataset_id_str, payload.partial_member_list, db.session
+            )
        # clear partial member list when permission is only_me or all_team_members
        elif payload.permission in {DatasetPermissionEnum.ONLY_ME, DatasetPermissionEnum.ALL_TEAM}:
-            DatasetPermissionService.clear_partial_member_list(dataset_id_str)
+            DatasetPermissionService.clear_partial_member_list(dataset_id_str, db.session)

-        partial_member_list = DatasetPermissionService.get_dataset_partial_member_list(dataset_id_str)
+        partial_member_list = DatasetPermissionService.get_dataset_partial_member_list(dataset_id_str, db.session)
        result_data.update({"partial_member_list": partial_member_list})

        return result_data, 200
@@ -734,8 +729,8 @@ class DatasetApi(Resource):
            raise Forbidden()

        try:
-            if DatasetService.delete_dataset(dataset_id_str, current_user):
-                DatasetPermissionService.clear_partial_member_list(dataset_id_str)
+            if DatasetService.delete_dataset(dataset_id_str, current_user, db.session):
+                DatasetPermissionService.clear_partial_member_list(dataset_id_str, db.session)
                return "", 204
            else:
                raise NotFound("Dataset not found.")
@@ -760,7 +755,7 @@ class DatasetUseCheckApi(Resource):
    def get(self, dataset_id: UUID):
        dataset_id_str = str(dataset_id)

-        dataset_is_using = DatasetService.dataset_use_check(dataset_id_str)
+        dataset_is_using = DatasetService.dataset_use_check(dataset_id_str, db.session)
        return {"is_using": dataset_is_using}, 200


@@ -781,12 +776,12 @@ class DatasetQueryApi(Resource):
    @rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_READONLY)
    def get(self, current_user: Account, dataset_id: UUID):
        dataset_id_str = str(dataset_id)
-        dataset = DatasetService.get_dataset(dataset_id_str)
+        dataset = DatasetService.get_dataset(dataset_id_str, db.session)
        if dataset is None:
            raise NotFound("Dataset not found.")

        try:
-            DatasetService.check_dataset_permission(dataset, current_user)
+            DatasetService.check_dataset_permission(dataset, current_user, db.session)
        except services.errors.account.NoPermissionError as e:
            raise Forbidden(str(e))

@@ -922,16 +917,16 @@ class DatasetRelatedAppListApi(Resource):
    @rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_READONLY)
    def get(self, current_user: Account, dataset_id: UUID):
        dataset_id_str = str(dataset_id)
-        dataset = DatasetService.get_dataset(dataset_id_str)
+        dataset = DatasetService.get_dataset(dataset_id_str, db.session)
        if dataset is None:
            raise NotFound("Dataset not found.")

        try:
-            DatasetService.check_dataset_permission(dataset, current_user)
+            DatasetService.check_dataset_permission(dataset, current_user, db.session)
        except services.errors.account.NoPermissionError as e:
            raise Forbidden(str(e))

-        app_dataset_joins = DatasetService.get_related_apps(dataset.id)
+        app_dataset_joins = DatasetService.get_related_apps(dataset.id, db.session)

        related_apps = []
        for app_dataset_join in app_dataset_joins:
@@ -1106,7 +1101,7 @@ class DatasetEnableApiApi(Resource):
    def post(self, dataset_id: UUID, status: str):
        dataset_id_str = str(dataset_id)

-        DatasetService.update_dataset_api_status(dataset_id_str, status == "enable")
+        DatasetService.update_dataset_api_status(dataset_id_str, status == "enable", db.session)

        return {"result": "success"}, 200

@@ -1175,10 +1170,10 @@ class DatasetErrorDocs(Resource):
    @rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_READONLY)
    def get(self, dataset_id: UUID):
        dataset_id_str = str(dataset_id)
-        dataset = DatasetService.get_dataset(dataset_id_str)
+        dataset = DatasetService.get_dataset(dataset_id_str, db.session)
        if dataset is None:
            raise NotFound("Dataset not found.")
-        results = DocumentService.get_error_documents_by_dataset_id(dataset_id_str)
+        results = DocumentService.get_error_documents_by_dataset_id(dataset_id_str, db.session)

        return dump_response(ErrorDocsResponse, {"data": results, "total": len(results)}), 200

@@ -1202,15 +1197,15 @@ class DatasetPermissionUserListApi(Resource):
    @rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_READONLY)
    def get(self, current_user: Account, dataset_id: UUID):
        dataset_id_str = str(dataset_id)
-        dataset = DatasetService.get_dataset(dataset_id_str)
+        dataset = DatasetService.get_dataset(dataset_id_str, db.session)
        if dataset is None:
            raise NotFound("Dataset not found.")
        try:
-            DatasetService.check_dataset_permission(dataset, current_user)
+            DatasetService.check_dataset_permission(dataset, current_user, db.session)
        except services.errors.account.NoPermissionError as e:
            raise Forbidden(str(e))

-        partial_members_list = DatasetPermissionService.get_dataset_partial_member_list(dataset_id_str)
+        partial_members_list = DatasetPermissionService.get_dataset_partial_member_list(dataset_id_str, db.session)

        return dump_response(PartialMemberListResponse, {"data": partial_members_list}), 200

@@ -1232,7 +1227,8 @@ class DatasetAutoDisableLogApi(Resource):
    @rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_READONLY)
    def get(self, dataset_id: UUID):
        dataset_id_str = str(dataset_id)
-        dataset = DatasetService.get_dataset(dataset_id_str)
+        dataset = DatasetService.get_dataset(dataset_id_str, db.session)
        if dataset is None:
            raise NotFound("Dataset not found.")
-        return dump_response(AutoDisableLogsResponse, DatasetService.get_dataset_auto_disable_logs(dataset_id_str)), 200
+        auto_disable_logs = DatasetService.get_dataset_auto_disable_logs(dataset_id_str, db.session)
+        return dump_response(AutoDisableLogsResponse, auto_disable_logs), 200
@@ -46,9 +46,11 @@ from graphon.model_runtime.errors.invoke import InvokeAuthorizationError
 from libs.datetime_utils import naive_utc_now
 from libs.helper import dump_response, to_timestamp
 from libs.login import login_required
+from libs.pagination import paginate_query
 from models import Account, DatasetProcessRule, Document, DocumentSegment, UploadFile
 from models.dataset import DocumentPipelineExecutionLog
 from models.enums import IndexingStatus, SegmentStatus
+from services.dataset_ref_service import DatasetRefService
 from services.dataset_service import DatasetService, DocumentService
 from services.entities.knowledge_entities.knowledge_entities import KnowledgeConfig, ProcessRule, RetrievalModel
 from services.file_service import FileService
@@ -181,16 +183,16 @@ class DocumentResource(Resource):
    def get_document(
        self, dataset_id: str, document_id: str, current_user: Account, current_tenant_id: str
    ) -> Document:
-        dataset = DatasetService.get_dataset(dataset_id)
+        dataset = DatasetService.get_dataset(dataset_id, db.session)
        if not dataset:
            raise NotFound("Dataset not found.")

        try:
-            DatasetService.check_dataset_permission(dataset, current_user)
+            DatasetService.check_dataset_permission(dataset, current_user, db.session)
        except services.errors.account.NoPermissionError as e:
            raise Forbidden(str(e))

-        document = DocumentService.get_document(dataset_id, document_id)
+        document = DocumentService.get_document(dataset_id, document_id, session=db.session)

        if not document:
            raise NotFound("Document not found.")
@@ -201,16 +203,16 @@ class DocumentResource(Resource):
        return document

    def get_batch_documents(self, dataset_id: str, batch: str, current_user: Account) -> Sequence[Document]:
-        dataset = DatasetService.get_dataset(dataset_id)
+        dataset = DatasetService.get_dataset(dataset_id, db.session)
        if not dataset:
            raise NotFound("Dataset not found.")

        try:
-            DatasetService.check_dataset_permission(dataset, current_user)
+            DatasetService.check_dataset_permission(dataset, current_user, db.session)
        except services.errors.account.NoPermissionError as e:
            raise Forbidden(str(e))

-        documents = DocumentService.get_batch_documents(dataset_id, batch)
+        documents = DocumentService.get_batch_documents(dataset_id, batch, db.session)

        if not documents:
            raise NotFound("Documents not found.")
@@ -241,13 +243,13 @@ class GetProcessRuleApi(Resource):
            # get the latest process rule
            document = db.get_or_404(Document, document_id)

-            dataset = DatasetService.get_dataset(document.dataset_id)
+            dataset = DatasetService.get_dataset(document.dataset_id, db.session)

            if not dataset:
                raise NotFound("Dataset not found.")

            try:
-                DatasetService.check_dataset_permission(dataset, current_user)
+                DatasetService.check_dataset_permission(dataset, current_user, db.session)
            except services.errors.account.NoPermissionError as e:
                raise Forbidden(str(e))

@@ -317,12 +319,12 @@ class DatasetDocumentListApi(Resource):
                    )
        except (ArgumentTypeError, ValueError, Exception):
            fetch = False
-        dataset = DatasetService.get_dataset(dataset_id_str)
+        dataset = DatasetService.get_dataset(dataset_id_str, db.session)
        if not dataset:
            raise NotFound("Dataset not found.")

        try:
-            DatasetService.check_dataset_permission(dataset, current_user)
+            DatasetService.check_dataset_permission(dataset, current_user, db.session)
        except services.errors.account.NoPermissionError as e:
            raise Forbidden(str(e))

@@ -367,7 +369,7 @@ class DatasetDocumentListApi(Resource):
                    desc(Document.position),
                )

-        paginated_documents = db.paginate(select=query, page=page, per_page=limit, max_per_page=100, error_out=False)
+        paginated_documents = paginate_query(query, page=page, per_page=limit, max_per_page=100)
        documents = paginated_documents.items

        DocumentService.enrich_documents_with_summary_index_status(
@@ -421,7 +423,7 @@ class DatasetDocumentListApi(Resource):
    def post(self, current_user: Account, dataset_id: UUID):
        dataset_id_str = str(dataset_id)

-        dataset = DatasetService.get_dataset(dataset_id_str)
+        dataset = DatasetService.get_dataset(dataset_id_str, db.session)

        if not dataset:
            raise NotFound("Dataset not found.")
@@ -431,7 +433,7 @@ class DatasetDocumentListApi(Resource):
            raise Forbidden()

        try:
-            DatasetService.check_dataset_permission(dataset, current_user)
+            DatasetService.check_dataset_permission(dataset, current_user, db.session)
        except services.errors.account.NoPermissionError as e:
            raise Forbidden(str(e))

@@ -444,8 +446,10 @@ class DatasetDocumentListApi(Resource):
        DocumentService.document_create_args_validate(knowledge_config)

        try:
-            documents, batch = DocumentService.save_document_with_dataset_id(dataset, knowledge_config, current_user)
-            dataset = DatasetService.get_dataset(dataset_id_str)
+            documents, batch = DocumentService.save_document_with_dataset_id(
+                dataset, knowledge_config, current_user, session=db.session
+            )
+            dataset = DatasetService.get_dataset(dataset_id_str, db.session)

        except ProviderTokenNotInitError as ex:
            raise ProviderNotInitializeError(ex.description)
@@ -464,7 +468,7 @@ class DatasetDocumentListApi(Resource):
    @rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_EDIT)
    def delete(self, dataset_id: UUID):
        dataset_id_str = str(dataset_id)
-        dataset = DatasetService.get_dataset(dataset_id_str)
+        dataset = DatasetService.get_dataset(dataset_id_str, db.session)
        if dataset is None:
            raise NotFound("Dataset not found.")
        # check user's model setting
@@ -472,7 +476,8 @@ class DatasetDocumentListApi(Resource):

        try:
            document_ids = request.args.getlist("document_id")
-            DocumentService.delete_documents(dataset, document_ids)
+            dataset_ref = DatasetRefService.create_dataset_ref(dataset)
+            DocumentService.delete_documents(dataset_ref, document_ids, dataset.doc_form, db.session)
        except services.errors.document.DocumentIndexingError:
            raise DocumentIndexingError("Cannot delete document during indexing.")

@@ -531,6 +536,7 @@ class DatasetInitApi(Resource):
                tenant_id=current_tenant_id,
                knowledge_config=knowledge_config,
                account=current_user,
+                session=db.session,
            )
        except ProviderTokenNotInitError as ex:
            raise ProviderNotInitializeError(ex.description)
@@ -867,7 +873,7 @@ class DocumentApi(DocumentResource):
        if metadata == "only":
            response = {"id": document.id, "doc_type": document.doc_type, "doc_metadata": document.doc_metadata_details}
        elif metadata == "without":
-            dataset_process_rules = DatasetService.get_process_rules(dataset_id_str)
+            dataset_process_rules = DatasetService.get_process_rules(dataset_id_str, db.session)
            document_process_rules = document.dataset_process_rule.to_dict() if document.dataset_process_rule else {}
            response = {
                "id": document.id,
@@ -901,7 +907,7 @@ class DocumentApi(DocumentResource):
                "need_summary": document.need_summary if document.need_summary is not None else False,
            }
        else:
-            dataset_process_rules = DatasetService.get_process_rules(dataset_id_str)
+            dataset_process_rules = DatasetService.get_process_rules(dataset_id_str, db.session)
            document_process_rules = document.dataset_process_rule.to_dict() if document.dataset_process_rule else {}
            response = {
                "id": document.id,
@@ -950,7 +956,7 @@ class DocumentApi(DocumentResource):
    def delete(self, current_tenant_id: str, current_user: Account, dataset_id: UUID, document_id: UUID):
        dataset_id_str = str(dataset_id)
        document_id_str = str(document_id)
-        dataset = DatasetService.get_dataset(dataset_id_str)
+        dataset = DatasetService.get_dataset(dataset_id_str, db.session)
        if dataset is None:
            raise NotFound("Dataset not found.")
        # check user's model setting
@@ -959,7 +965,7 @@ class DocumentApi(DocumentResource):
        document = self.get_document(dataset_id_str, document_id_str, current_user, current_tenant_id)

        try:
-            DocumentService.delete_document(document)
+            DocumentService.delete_document(document, db.session)
        except services.errors.document.DocumentIndexingError:
            raise DocumentIndexingError("Cannot delete document during indexing.")

@@ -983,7 +989,7 @@ class DocumentDownloadApi(DocumentResource):
    def get(self, current_tenant_id: str, current_user: Account, dataset_id: UUID, document_id: UUID) -> dict[str, Any]:
        # Reuse the shared permission/tenant checks implemented in DocumentResource.
        document = self.get_document(str(dataset_id), str(document_id), current_user, current_tenant_id)
-        return {"url": DocumentService.get_document_download_url(document)}
+        return {"url": DocumentService.get_document_download_url(document, db.session)}


@console_ns.route("/datasets/<uuid:dataset_id>/documents/download-zip")
@@ -1013,6 +1019,7 @@ class DocumentBatchDownloadZipApi(DocumentResource):
            document_ids=document_ids,
            tenant_id=current_tenant_id,
            current_user=current_user,
+            session=db.session,
        )

        # Delegate ZIP packing to FileService, but keep Flask response+cleanup in the route.
@@ -1161,7 +1168,7 @@ class DocumentStatusApi(DocumentResource):
        self, current_user: Account, dataset_id: UUID, action: Literal["enable", "disable", "archive", "un_archive"]
    ):
        dataset_id_str = str(dataset_id)
-        dataset = DatasetService.get_dataset(dataset_id_str)
+        dataset = DatasetService.get_dataset(dataset_id_str, db.session)
        if dataset is None:
            raise NotFound("Dataset not found.")

@@ -1173,12 +1180,12 @@ class DocumentStatusApi(DocumentResource):
        DatasetService.check_dataset_model_setting(dataset)

        # check user's permission
-        DatasetService.check_dataset_permission(dataset, current_user)
+        DatasetService.check_dataset_permission(dataset, current_user, db.session)

        document_ids = request.args.getlist("document_id")

        try:
-            DocumentService.batch_update_document_status(dataset, document_ids, action, current_user)
+            DocumentService.batch_update_document_status(dataset, document_ids, action, current_user, db.session)
        except services.errors.document.DocumentIndexingError as e:
            raise InvalidActionError(str(e))
        except ValueError as e:
@@ -1202,11 +1209,11 @@ class DocumentPauseApi(DocumentResource):
        dataset_id_str = str(dataset_id)
        document_id_str = str(document_id)

-        dataset = DatasetService.get_dataset(dataset_id_str)
+        dataset = DatasetService.get_dataset(dataset_id_str, db.session)
        if not dataset:
            raise NotFound("Dataset not found.")

-        document = DocumentService.get_document(dataset.id, document_id_str)
+        document = DocumentService.get_document(dataset.id, document_id_str, session=db.session)

        # 404 if document not found
        if document is None:
@@ -1218,7 +1225,7 @@ class DocumentPauseApi(DocumentResource):

        try:
            # pause document
-            DocumentService.pause_document(document)
+            DocumentService.pause_document(document, db.session)
        except services.errors.document.DocumentIndexingError:
            raise DocumentIndexingError("Cannot pause completed document.")

@@ -1237,10 +1244,10 @@ class DocumentRecoverApi(DocumentResource):
        """recover document."""
        dataset_id_str = str(dataset_id)
        document_id_str = str(document_id)
-        dataset = DatasetService.get_dataset(dataset_id_str)
+        dataset = DatasetService.get_dataset(dataset_id_str, db.session)
        if not dataset:
            raise NotFound("Dataset not found.")
-        document = DocumentService.get_document(dataset.id, document_id_str)
+        document = DocumentService.get_document(dataset.id, document_id_str, session=db.session)

        # 404 if document not found
        if document is None:
@@ -1251,7 +1258,7 @@ class DocumentRecoverApi(DocumentResource):
            raise ArchivedDocumentImmutableError()
        try:
            # pause document
-            DocumentService.recover_document(document)
+            DocumentService.recover_document(document, db.session)
        except services.errors.document.DocumentIndexingError:
            raise DocumentIndexingError("Document is not in paused status.")

@@ -1271,13 +1278,13 @@ class DocumentRetryApi(DocumentResource):
        """retry document."""
        payload = DocumentRetryPayload.model_validate(console_ns.payload or {})
        dataset_id_str = str(dataset_id)
-        dataset = DatasetService.get_dataset(dataset_id_str)
+        dataset = DatasetService.get_dataset(dataset_id_str, db.session)
        retry_documents = []
        if not dataset:
            raise NotFound("Dataset not found.")
        for document_id in payload.document_ids:
            try:
-                document = DocumentService.get_document(dataset.id, document_id)
+                document = DocumentService.get_document(dataset.id, document_id, session=db.session)

                # 404 if document not found
                if document is None:
@@ -1295,7 +1302,7 @@ class DocumentRetryApi(DocumentResource):
                logger.exception("Failed to retry document, document id: %s", document_id)
                continue
        # retry document
-        DocumentService.retry_document(dataset_id_str, retry_documents)
+        DocumentService.retry_document(dataset_id_str, retry_documents, db.session)

        return "", 204

@@ -1313,14 +1320,14 @@ class DocumentRenameApi(DocumentResource):
        # The role of the current user in the ta table must be admin, owner, editor, or dataset_operator
        if not current_user.is_dataset_editor:
            raise Forbidden()
-        dataset = DatasetService.get_dataset(dataset_id)
+        dataset = DatasetService.get_dataset(dataset_id, db.session)
        if not dataset:
            raise NotFound("Dataset not found.")
-        DatasetService.check_dataset_operator_permission(current_user, dataset)
+        DatasetService.check_dataset_operator_permission(current_user, dataset, session=db.session)
        payload = DocumentRenamePayload.model_validate(console_ns.payload or {})

        try:
-            document = DocumentService.rename_document(str(dataset_id), str(document_id), payload.name)
+            document = DocumentService.rename_document(str(dataset_id), str(document_id), payload.name, db.session)
        except services.errors.document.DocumentIndexingError:
            raise DocumentIndexingError("Cannot delete document during indexing.")

@@ -1338,11 +1345,11 @@ class WebsiteDocumentSyncApi(DocumentResource):
    def get(self, current_tenant_id: str, dataset_id: UUID, document_id: UUID):
        """sync website document."""
        dataset_id_str = str(dataset_id)
-        dataset = DatasetService.get_dataset(dataset_id_str)
+        dataset = DatasetService.get_dataset(dataset_id_str, db.session)
        if not dataset:
            raise NotFound("Dataset not found.")
        document_id_str = str(document_id)
-        document = DocumentService.get_document(dataset.id, document_id_str)
+        document = DocumentService.get_document(dataset.id, document_id_str, session=db.session)
        if not document:
            raise NotFound("Document not found.")
        if document.tenant_id != current_tenant_id:
@@ -1353,7 +1360,7 @@ class WebsiteDocumentSyncApi(DocumentResource):
        if DocumentService.check_archived(document):
            raise ArchivedDocumentImmutableError()
        # sync document
-        DocumentService.sync_website_document(dataset_id_str, document)
+        DocumentService.sync_website_document(dataset_id_str, document, db.session)

        return {"result": "success"}, 200

@@ -1373,10 +1380,10 @@ class DocumentPipelineExecutionLogApi(DocumentResource):
        dataset_id_str = str(dataset_id)
        document_id_str = str(document_id)

-        dataset = DatasetService.get_dataset(dataset_id_str)
+        dataset = DatasetService.get_dataset(dataset_id_str, db.session)
        if not dataset:
            raise NotFound("Dataset not found.")
-        document = DocumentService.get_document(dataset.id, document_id_str)
+        document = DocumentService.get_document(dataset.id, document_id_str, session=db.session)
        if not document:
            raise NotFound("Document not found.")
        log = db.session.scalar(
@@ -1431,7 +1438,7 @@ class DocumentGenerateSummaryApi(Resource):
        dataset_id_str = str(dataset_id)

        # Get dataset
-        dataset = DatasetService.get_dataset(dataset_id_str)
+        dataset = DatasetService.get_dataset(dataset_id_str, db.session)
        if not dataset:
            raise NotFound("Dataset not found.")

@@ -1440,7 +1447,7 @@ class DocumentGenerateSummaryApi(Resource):
            raise Forbidden()

        try:
-            DatasetService.check_dataset_permission(dataset, current_user)
+            DatasetService.check_dataset_permission(dataset, current_user, db.session)
        except services.errors.account.NoPermissionError as e:
            raise Forbidden(str(e))

@@ -1465,7 +1472,7 @@ class DocumentGenerateSummaryApi(Resource):
            raise ValueError("Summary index is not enabled for this dataset. Please enable it in the dataset settings.")

        # Verify all documents exist and belong to the dataset
-        documents = DocumentService.get_documents_by_ids(dataset_id_str, document_list)
+        documents = DocumentService.get_documents_by_ids(dataset_id_str, document_list, db.session)

        if len(documents) != len(document_list):
            found_ids = {doc.id for doc in documents}
@@ -1481,6 +1488,7 @@ class DocumentGenerateSummaryApi(Resource):
            DocumentService.update_documents_need_summary(
                dataset_id=dataset_id_str,
                document_ids=document_ids_to_update,
+                session=db.session,
                need_summary=True,
            )

@@ -1531,13 +1539,13 @@ class DocumentSummaryStatusApi(DocumentResource):
        document_id_str = str(document_id)

        # Get dataset
-        dataset = DatasetService.get_dataset(dataset_id_str)
+        dataset = DatasetService.get_dataset(dataset_id_str, db.session)
        if not dataset:
            raise NotFound("Dataset not found.")

        # Check permissions
        try:
-            DatasetService.check_dataset_permission(dataset, current_user)
+            DatasetService.check_dataset_permission(dataset, current_user, db.session)
        except services.errors.account.NoPermissionError as e:
            raise Forbidden(str(e))

@@ -1547,6 +1555,7 @@ class DocumentSummaryStatusApi(DocumentResource):
        result = SummaryIndexService.get_document_summary_status_detail(
            document_id=document_id_str,
            dataset_id=dataset_id_str,
+            session=db.session,
        )

        return result, 200
@@ -57,9 +57,11 @@ from fields.segment_fields import (
 from graphon.model_runtime.entities.model_entities import ModelType
 from libs.helper import dump_response, escape_like_pattern
 from libs.login import login_required
+from libs.pagination import paginate_query
 from models import Account
-from models.dataset import ChildChunk, DocumentSegment
+from models.dataset import Dataset, Document, DocumentSegment
 from models.model import UploadFile
+from services.dataset_ref_service import DatasetRefService, SegmentRef
 from services.dataset_service import DatasetService, DocumentService, SegmentService
 from services.entities.knowledge_entities.knowledge_entities import ChildChunkUpdateArgs, SegmentUpdateArgs
 from services.errors.chunk import ChildChunkDeleteIndexError as ChildChunkDeleteIndexServiceError
@@ -162,6 +164,21 @@ register_response_schema_models(
 )


+def _get_segment_for_document(
+    dataset: Dataset, document: Document, segment_id: str
+) -> tuple[SegmentRef, DocumentSegment]:
+    dataset_ref = DatasetRefService.create_dataset_ref(dataset)
+    document_ref = DatasetRefService.create_document_ref(dataset_ref, document)
+    if document_ref is None:
+        raise NotFound("Document not found.")
+
+    segment_ref = DatasetRefService.create_segment_ref(document_ref, segment_id)
+    segment = SegmentService.get_segment_by_ref(segment_ref)
+    if not segment:
+        raise NotFound("Segment not found.")
+    return segment_ref, segment
+
+
@console_ns.route("/datasets/<uuid:dataset_id>/documents/<uuid:document_id>/segments")
 class DatasetDocumentSegmentListApi(Resource):
    @console_ns.doc(params=SegmentDocParams.DATASET_DOCUMENT)
@@ -176,16 +193,16 @@ class DatasetDocumentSegmentListApi(Resource):
    def get(self, current_tenant_id: str, current_user: Account, dataset_id: UUID, document_id: UUID):
        dataset_id_str = str(dataset_id)
        document_id_str = str(document_id)
-        dataset = DatasetService.get_dataset(dataset_id_str)
+        dataset = DatasetService.get_dataset(dataset_id_str, db.session)
        if not dataset:
            raise NotFound("Dataset not found.")

        try:
-            DatasetService.check_dataset_permission(dataset, current_user)
+            DatasetService.check_dataset_permission(dataset, current_user, db.session)
        except services.errors.account.NoPermissionError as e:
            raise Forbidden(str(e))

-        document = DocumentService.get_document(dataset_id_str, document_id_str)
+        document = DocumentService.get_document(dataset_id_str, document_id_str, session=db.session)

        if not document:
            raise NotFound("Document not found.")
@@ -254,7 +271,7 @@ class DatasetDocumentSegmentListApi(Resource):
            elif args.enabled.lower() == "false":
                query = query.where(DocumentSegment.enabled == False)

-        segments = db.paginate(select=query, page=page, per_page=limit, max_per_page=100, error_out=False)
+        segments = paginate_query(query, page=page, per_page=limit, max_per_page=100)

        segment_list = list(segments.items)
        segment_ids = [segment.id for segment in segment_list]
@@ -286,14 +303,14 @@ class DatasetDocumentSegmentListApi(Resource):
    def delete(self, current_user: Account, dataset_id: UUID, document_id: UUID):
        # check dataset
        dataset_id_str = str(dataset_id)
-        dataset = DatasetService.get_dataset(dataset_id_str)
+        dataset = DatasetService.get_dataset(dataset_id_str, db.session)
        if not dataset:
            raise NotFound("Dataset not found.")
        # check user's model setting
        DatasetService.check_dataset_model_setting(dataset)
        # check document
        document_id_str = str(document_id)
-        document = DocumentService.get_document(dataset_id_str, document_id_str)
+        document = DocumentService.get_document(dataset_id_str, document_id_str, session=db.session)
        if not document:
            raise NotFound("Document not found.")
        segment_ids = request.args.getlist("segment_id")
@@ -302,10 +319,10 @@ class DatasetDocumentSegmentListApi(Resource):
        if not current_user.is_dataset_editor:
            raise Forbidden()
        try:
-            DatasetService.check_dataset_permission(dataset, current_user)
+            DatasetService.check_dataset_permission(dataset, current_user, db.session)
        except services.errors.account.NoPermissionError as e:
            raise Forbidden(str(e))
-        SegmentService.delete_segments(segment_ids, document, dataset)
+        SegmentService.delete_segments(segment_ids, document, dataset, db.session)
        return "", 204


@@ -331,11 +348,11 @@ class DatasetDocumentSegmentApi(Resource):
        action: Literal["enable", "disable"],
    ):
        dataset_id_str = str(dataset_id)
-        dataset = DatasetService.get_dataset(dataset_id_str)
+        dataset = DatasetService.get_dataset(dataset_id_str, db.session)
        if not dataset:
            raise NotFound("Dataset not found.")
        document_id_str = str(document_id)
-        document = DocumentService.get_document(dataset_id_str, document_id_str)
+        document = DocumentService.get_document(dataset_id_str, document_id_str, session=db.session)
        if not document:
            raise NotFound("Document not found.")
        # check user's model setting
@@ -345,7 +362,7 @@ class DatasetDocumentSegmentApi(Resource):
            raise Forbidden()

        try:
-            DatasetService.check_dataset_permission(dataset, current_user)
+            DatasetService.check_dataset_permission(dataset, current_user, db.session)
        except services.errors.account.NoPermissionError as e:
            raise Forbidden(str(e))
        if dataset.indexing_technique == IndexTechniqueType.HIGH_QUALITY:
@@ -371,7 +388,7 @@ class DatasetDocumentSegmentApi(Resource):
        if cache_result is not None:
            raise InvalidActionError("Document is being indexed, please try again later")
        try:
-            SegmentService.update_segments_status(segment_ids, action, dataset, document)
+            SegmentService.update_segments_status(segment_ids, action, dataset, document, db.session)
        except Exception as e:
            raise InvalidActionError(str(e))
        return dump_response(SimpleResultResponse, {"result": "success"}), 200
@@ -394,12 +411,12 @@ class DatasetDocumentSegmentAddApi(Resource):
    def post(self, current_tenant_id: str, current_user: Account, dataset_id: UUID, document_id: UUID):
        # check dataset
        dataset_id_str = str(dataset_id)
-        dataset = DatasetService.get_dataset(dataset_id_str)
+        dataset = DatasetService.get_dataset(dataset_id_str, db.session)
        if not dataset:
            raise NotFound("Dataset not found.")
        # check document
        document_id_str = str(document_id)
-        document = DocumentService.get_document(dataset_id_str, document_id_str)
+        document = DocumentService.get_document(dataset_id_str, document_id_str, session=db.session)
        if not document:
            raise NotFound("Document not found.")
        if not current_user.is_dataset_editor:
@@ -421,14 +438,14 @@ class DatasetDocumentSegmentAddApi(Resource):
            except ProviderTokenNotInitError as ex:
                raise ProviderNotInitializeError(ex.description)
        try:
-            DatasetService.check_dataset_permission(dataset, current_user)
+            DatasetService.check_dataset_permission(dataset, current_user, db.session)
        except services.errors.account.NoPermissionError as e:
            raise Forbidden(str(e))
        # validate args
        payload = SegmentCreatePayload.model_validate(console_ns.payload or {})
        payload_dict = payload.model_dump(exclude_none=True)
        SegmentService.segment_create_args_validate(payload_dict, document)
-        segment = type_cast(DocumentSegment, SegmentService.create_segment(payload_dict, document, dataset))
+        segment = type_cast(DocumentSegment, SegmentService.create_segment(payload_dict, document, dataset, db.session))
        summary = SummaryIndexService.get_segment_summary(segment_id=segment.id, dataset_id=dataset_id_str)
        response = {
            "data": segment_response_with_summary(segment, summary.summary_content if summary else None),
@@ -455,16 +472,23 @@ class DatasetDocumentSegmentUpdateApi(Resource):
    ):
        # check dataset
        dataset_id_str = str(dataset_id)
-        dataset = DatasetService.get_dataset(dataset_id_str)
+        dataset = DatasetService.get_dataset(dataset_id_str, db.session)
        if not dataset:
            raise NotFound("Dataset not found.")
        # check user's model setting
        DatasetService.check_dataset_model_setting(dataset)
        # check document
        document_id_str = str(document_id)
-        document = DocumentService.get_document(dataset_id_str, document_id_str)
+        document = DocumentService.get_document(dataset_id_str, document_id_str, session=db.session)
        if not document:
            raise NotFound("Document not found.")
+        # The role of the current user in the ta table must be admin, owner, dataset_operator, or editor
+        if not current_user.is_dataset_editor:
+            raise Forbidden()
+        try:
+            DatasetService.check_dataset_permission(dataset, current_user, db.session)
+        except services.errors.account.NoPermissionError as e:
+            raise Forbidden(str(e))
        if dataset.indexing_technique == IndexTechniqueType.HIGH_QUALITY:
            # check embedding model setting
            try:
@@ -481,22 +505,8 @@ class DatasetDocumentSegmentUpdateApi(Resource):
                )
            except ProviderTokenNotInitError as ex:
                raise ProviderNotInitializeError(ex.description)
-            # check segment
        segment_id_str = str(segment_id)
-        segment = db.session.scalar(
-            select(DocumentSegment)
-            .where(DocumentSegment.id == segment_id_str, DocumentSegment.tenant_id == current_tenant_id)
-            .limit(1)
-        )
-        if not segment:
-            raise NotFound("Segment not found.")
-        # The role of the current user in the ta table must be admin, owner, dataset_operator, or editor
-        if not current_user.is_dataset_editor:
-            raise Forbidden()
-        try:
-            DatasetService.check_dataset_permission(dataset, current_user)
-        except services.errors.account.NoPermissionError as e:
-            raise Forbidden(str(e))
+        _, segment = _get_segment_for_document(dataset, document, segment_id_str)
        # validate args
        payload = SegmentUpdatePayload.model_validate(console_ns.payload or {})
        payload_dict = payload.model_dump(exclude_none=True)
@@ -504,7 +514,11 @@ class DatasetDocumentSegmentUpdateApi(Resource):

        # Update segment (summary update with change detection is handled in SegmentService.update_segment)
        segment = SegmentService.update_segment(
-            SegmentUpdateArgs.model_validate(payload.model_dump(exclude_none=True)), segment, document, dataset
+            SegmentUpdateArgs.model_validate(payload.model_dump(exclude_none=True)),
+            segment,
+            document,
+            dataset,
+            db.session,
        )
        summary = SummaryIndexService.get_segment_summary(segment_id=segment.id, dataset_id=dataset_id_str)
        response = {
@@ -527,33 +541,26 @@ class DatasetDocumentSegmentUpdateApi(Resource):
    ):
        # check dataset
        dataset_id_str = str(dataset_id)
-        dataset = DatasetService.get_dataset(dataset_id_str)
+        dataset = DatasetService.get_dataset(dataset_id_str, db.session)
        if not dataset:
            raise NotFound("Dataset not found.")
        # check user's model setting
        DatasetService.check_dataset_model_setting(dataset)
        # check document
        document_id_str = str(document_id)
-        document = DocumentService.get_document(dataset_id_str, document_id_str)
+        document = DocumentService.get_document(dataset_id_str, document_id_str, session=db.session)
        if not document:
            raise NotFound("Document not found.")
-        # check segment
-        segment_id_str = str(segment_id)
-        segment = db.session.scalar(
-            select(DocumentSegment)
-            .where(DocumentSegment.id == segment_id_str, DocumentSegment.tenant_id == current_tenant_id)
-            .limit(1)
-        )
-        if not segment:
-            raise NotFound("Segment not found.")
        # The role of the current user in the ta table must be admin, owner, dataset_operator, or editor
        if not current_user.is_dataset_editor:
            raise Forbidden()
        try:
-            DatasetService.check_dataset_permission(dataset, current_user)
+            DatasetService.check_dataset_permission(dataset, current_user, db.session)
        except services.errors.account.NoPermissionError as e:
            raise Forbidden(str(e))
-        SegmentService.delete_segment(segment, document, dataset)
+        segment_id_str = str(segment_id)
+        _, segment = _get_segment_for_document(dataset, document, segment_id_str)
+        SegmentService.delete_segment(segment, document, dataset, db.session)
        return "", 204


@@ -576,12 +583,12 @@ class DatasetDocumentSegmentBatchImportApi(Resource):
    def post(self, current_tenant_id: str, current_user: Account, dataset_id: UUID, document_id: UUID):
        # check dataset
        dataset_id_str = str(dataset_id)
-        dataset = DatasetService.get_dataset(dataset_id_str)
+        dataset = DatasetService.get_dataset(dataset_id_str, db.session)
        if not dataset:
            raise NotFound("Dataset not found.")
        # check document
        document_id_str = str(document_id)
-        document = DocumentService.get_document(dataset_id_str, document_id_str)
+        document = DocumentService.get_document(dataset_id_str, document_id_str, session=db.session)
        if not document:
            raise NotFound("Document not found.")

@@ -651,25 +658,20 @@ class ChildChunkAddApi(Resource):
    ):
        # check dataset
        dataset_id_str = str(dataset_id)
-        dataset = DatasetService.get_dataset(dataset_id_str)
+        dataset = DatasetService.get_dataset(dataset_id_str, db.session)
        if not dataset:
            raise NotFound("Dataset not found.")
        # check document
        document_id_str = str(document_id)
-        document = DocumentService.get_document(dataset_id_str, document_id_str)
+        document = DocumentService.get_document(dataset_id_str, document_id_str, session=db.session)
        if not document:
            raise NotFound("Document not found.")
-        # check segment
-        segment_id_str = str(segment_id)
-        segment = db.session.scalar(
-            select(DocumentSegment)
-            .where(DocumentSegment.id == segment_id_str, DocumentSegment.tenant_id == current_tenant_id)
-            .limit(1)
-        )
-        if not segment:
-            raise NotFound("Segment not found.")
        if not current_user.is_dataset_editor:
            raise Forbidden()
+        try:
+            DatasetService.check_dataset_permission(dataset, current_user, db.session)
+        except services.errors.account.NoPermissionError as e:
+            raise Forbidden(str(e))
        # check embedding model setting
        if dataset.indexing_technique == IndexTechniqueType.HIGH_QUALITY:
            try:
@@ -686,14 +688,12 @@ class ChildChunkAddApi(Resource):
                )
            except ProviderTokenNotInitError as ex:
                raise ProviderNotInitializeError(ex.description)
-        try:
-            DatasetService.check_dataset_permission(dataset, current_user)
-        except services.errors.account.NoPermissionError as e:
-            raise Forbidden(str(e))
+        segment_id_str = str(segment_id)
+        _, segment = _get_segment_for_document(dataset, document, segment_id_str)
        # validate args
        try:
            payload = ChildChunkCreatePayload.model_validate(console_ns.payload or {})
-            child_chunk = SegmentService.create_child_chunk(payload.content, segment, document, dataset)
+            child_chunk = SegmentService.create_child_chunk(payload.content, segment, document, dataset, db.session)
        except ChildChunkIndexingServiceError as e:
            raise ChildChunkIndexingError(str(e))
        return dump_response(ChildChunkDetailResponse, {"data": child_chunk}), 200
@@ -709,25 +709,18 @@ class ChildChunkAddApi(Resource):
    def get(self, current_tenant_id: str, dataset_id: UUID, document_id: UUID, segment_id: UUID):
        # check dataset
        dataset_id_str = str(dataset_id)
-        dataset = DatasetService.get_dataset(dataset_id_str)
+        dataset = DatasetService.get_dataset(dataset_id_str, db.session)
        if not dataset:
            raise NotFound("Dataset not found.")
        # check user's model setting
        DatasetService.check_dataset_model_setting(dataset)
        # check document
        document_id_str = str(document_id)
-        document = DocumentService.get_document(dataset_id_str, document_id_str)
+        document = DocumentService.get_document(dataset_id_str, document_id_str, session=db.session)
        if not document:
            raise NotFound("Document not found.")
-        # check segment
        segment_id_str = str(segment_id)
-        segment = db.session.scalar(
-            select(DocumentSegment)
-            .where(DocumentSegment.id == segment_id_str, DocumentSegment.tenant_id == current_tenant_id)
-            .limit(1)
-        )
-        if not segment:
-            raise NotFound("Segment not found.")
+        _get_segment_for_document(dataset, document, segment_id_str)
        args = query_params_from_request(ChildChunkListQuery, use_defaults_for_malformed_ints=True)

        page = args.page
@@ -766,36 +759,29 @@ class ChildChunkAddApi(Resource):
    ):
        # check dataset
        dataset_id_str = str(dataset_id)
-        dataset = DatasetService.get_dataset(dataset_id_str)
+        dataset = DatasetService.get_dataset(dataset_id_str, db.session)
        if not dataset:
            raise NotFound("Dataset not found.")
        # check user's model setting
        DatasetService.check_dataset_model_setting(dataset)
        # check document
        document_id_str = str(document_id)
-        document = DocumentService.get_document(dataset_id_str, document_id_str)
+        document = DocumentService.get_document(dataset_id_str, document_id_str, session=db.session)
        if not document:
            raise NotFound("Document not found.")
-            # check segment
-        segment_id_str = str(segment_id)
-        segment = db.session.scalar(
-            select(DocumentSegment)
-            .where(DocumentSegment.id == segment_id_str, DocumentSegment.tenant_id == current_tenant_id)
-            .limit(1)
-        )
-        if not segment:
-            raise NotFound("Segment not found.")
        # The role of the current user in the ta table must be admin, owner, dataset_operator, or editor
        if not current_user.is_dataset_editor:
            raise Forbidden()
        try:
-            DatasetService.check_dataset_permission(dataset, current_user)
+            DatasetService.check_dataset_permission(dataset, current_user, db.session)
        except services.errors.account.NoPermissionError as e:
            raise Forbidden(str(e))
+        segment_id_str = str(segment_id)
+        _, segment = _get_segment_for_document(dataset, document, segment_id_str)
        # validate args
        payload = ChildChunkBatchUpdatePayload.model_validate(console_ns.payload or {})
        try:
-            child_chunks = SegmentService.update_child_chunks(payload.chunks, segment, document, dataset)
+            child_chunks = SegmentService.update_child_chunks(payload.chunks, segment, document, dataset, db.session)
        except ChildChunkIndexingServiceError as e:
            raise ChildChunkIndexingError(str(e))
        return dump_response(ChildChunkBatchUpdateResponse, {"data": child_chunks}), 200
@@ -825,48 +811,31 @@ class ChildChunkUpdateApi(Resource):
    ):
        # check dataset
        dataset_id_str = str(dataset_id)
-        dataset = DatasetService.get_dataset(dataset_id_str)
+        dataset = DatasetService.get_dataset(dataset_id_str, db.session)
        if not dataset:
            raise NotFound("Dataset not found.")
        # check user's model setting
        DatasetService.check_dataset_model_setting(dataset)
        # check document
        document_id_str = str(document_id)
-        document = DocumentService.get_document(dataset_id_str, document_id_str)
+        document = DocumentService.get_document(dataset_id_str, document_id_str, session=db.session)
        if not document:
            raise NotFound("Document not found.")
-        # check segment
-        segment_id_str = str(segment_id)
-        segment = db.session.scalar(
-            select(DocumentSegment)
-            .where(DocumentSegment.id == segment_id_str, DocumentSegment.tenant_id == current_tenant_id)
-            .limit(1)
-        )
-        if not segment:
-            raise NotFound("Segment not found.")
-        # check child chunk
-        child_chunk_id_str = str(child_chunk_id)
-        child_chunk = db.session.scalar(
-            select(ChildChunk)
-            .where(
-                ChildChunk.id == child_chunk_id_str,
-                ChildChunk.tenant_id == current_tenant_id,
-                ChildChunk.segment_id == segment.id,
-                ChildChunk.document_id == document_id_str,
-            )
-            .limit(1)
-        )
-        if not child_chunk:
-            raise NotFound("Child chunk not found.")
        # The role of the current user in the ta table must be admin, owner, dataset_operator, or editor
        if not current_user.is_dataset_editor:
            raise Forbidden()
        try:
-            DatasetService.check_dataset_permission(dataset, current_user)
+            DatasetService.check_dataset_permission(dataset, current_user, db.session)
        except services.errors.account.NoPermissionError as e:
            raise Forbidden(str(e))
+        segment_id_str = str(segment_id)
+        segment_ref, _ = _get_segment_for_document(dataset, document, segment_id_str)
+        child_chunk_id_str = str(child_chunk_id)
+        child_chunk = SegmentService.get_child_chunk_by_segment_ref(child_chunk_id_str, segment_ref)
+        if not child_chunk:
+            raise NotFound("Child chunk not found.")
        try:
-            SegmentService.delete_child_chunk(child_chunk, dataset)
+            SegmentService.delete_child_chunk(child_chunk, dataset, db.session)
        except ChildChunkDeleteIndexServiceError as e:
            raise ChildChunkDeleteIndexError(str(e))
        return "", 204
@@ -893,50 +862,35 @@ class ChildChunkUpdateApi(Resource):
    ):
        # check dataset
        dataset_id_str = str(dataset_id)
-        dataset = DatasetService.get_dataset(dataset_id_str)
+        dataset = DatasetService.get_dataset(dataset_id_str, db.session)
        if not dataset:
            raise NotFound("Dataset not found.")
        # check user's model setting
        DatasetService.check_dataset_model_setting(dataset)
        # check document
        document_id_str = str(document_id)
-        document = DocumentService.get_document(dataset_id_str, document_id_str)
+        document = DocumentService.get_document(dataset_id_str, document_id_str, session=db.session)
        if not document:
            raise NotFound("Document not found.")
-            # check segment
-        segment_id_str = str(segment_id)
-        segment = db.session.scalar(
-            select(DocumentSegment)
-            .where(DocumentSegment.id == segment_id_str, DocumentSegment.tenant_id == current_tenant_id)
-            .limit(1)
-        )
-        if not segment:
-            raise NotFound("Segment not found.")
-        # check child chunk
-        child_chunk_id_str = str(child_chunk_id)
-        child_chunk = db.session.scalar(
-            select(ChildChunk)
-            .where(
-                ChildChunk.id == child_chunk_id_str,
-                ChildChunk.tenant_id == current_tenant_id,
-                ChildChunk.segment_id == segment.id,
-                ChildChunk.document_id == document_id_str,
-            )
-            .limit(1)
-        )
-        if not child_chunk:
-            raise NotFound("Child chunk not found.")
        # The role of the current user in the ta table must be admin, owner, dataset_operator, or editor
        if not current_user.is_dataset_editor:
            raise Forbidden()
        try:
-            DatasetService.check_dataset_permission(dataset, current_user)
+            DatasetService.check_dataset_permission(dataset, current_user, db.session)
        except services.errors.account.NoPermissionError as e:
            raise Forbidden(str(e))
+        segment_id_str = str(segment_id)
+        segment_ref, segment = _get_segment_for_document(dataset, document, segment_id_str)
+        child_chunk_id_str = str(child_chunk_id)
+        child_chunk = SegmentService.get_child_chunk_by_segment_ref(child_chunk_id_str, segment_ref)
+        if not child_chunk:
+            raise NotFound("Child chunk not found.")
        # validate args
        try:
            payload = ChildChunkUpdatePayload.model_validate(console_ns.payload or {})
-            child_chunk = SegmentService.update_child_chunk(payload.content, child_chunk, segment, document, dataset)
+            child_chunk = SegmentService.update_child_chunk(
+                payload.content, child_chunk, segment, document, dataset, db.session
+            )
        except ChildChunkIndexingServiceError as e:
            raise ChildChunkIndexingError(str(e))
        return dump_response(ChildChunkDetailResponse, {"data": child_chunk}), 200
@@ -4,6 +4,7 @@ from uuid import UUID
 from flask import request
 from flask_restx import Resource, fields, marshal
 from pydantic import BaseModel, Field, RootModel
+from sqlalchemy.orm import Session
 from werkzeug.exceptions import Forbidden, InternalServerError, NotFound

 import services
@@ -15,6 +16,7 @@ from controllers.common.schema import (
    register_schema_models,
 )
 from controllers.console import console_ns
+from controllers.console.app.wraps import with_session
 from controllers.console.datasets.error import DatasetNameDuplicateError
 from controllers.console.wraps import (
    RBACPermission,
@@ -207,7 +209,8 @@ class ExternalApiTemplateListApi(Resource):
    )
    @with_current_user
    @with_current_tenant_id
-    def post(self, current_tenant_id: str, current_user: Account):
+    @with_session
+    def post(self, session: Session, current_tenant_id: str, current_user: Account):
        payload = ExternalKnowledgeApiPayload.model_validate(console_ns.payload or {})

        ExternalDatasetService.validate_api_list(payload.settings)
@@ -218,7 +221,10 @@ class ExternalApiTemplateListApi(Resource):

        try:
            external_knowledge_api = ExternalDatasetService.create_external_knowledge_api(
-                tenant_id=current_tenant_id, user_id=current_user.id, args=payload.model_dump()
+                tenant_id=current_tenant_id,
+                user_id=current_user.id,
+                args=payload.model_dump(),
+                session=session,
            )
        except services.errors.dataset.DatasetNameDuplicateError:
            raise DatasetNameDuplicateError()
@@ -241,10 +247,11 @@ class ExternalApiTemplateApi(Resource):
    @login_required
    @account_initialization_required
    @with_current_tenant_id
-    def get(self, current_tenant_id: str, external_knowledge_api_id: UUID):
+    @with_session
+    def get(self, session: Session, current_tenant_id: str, external_knowledge_api_id: UUID):
        external_knowledge_api_id_str = str(external_knowledge_api_id)
        external_knowledge_api = ExternalDatasetService.get_external_knowledge_api(
-            external_knowledge_api_id_str, current_tenant_id
+            external_knowledge_api_id=external_knowledge_api_id_str, tenant_id=current_tenant_id, session=session
        )
        if external_knowledge_api is None:
            raise NotFound("API template not found.")
@@ -262,7 +269,8 @@ class ExternalApiTemplateApi(Resource):
    @console_ns.expect(console_ns.models[ExternalKnowledgeApiPayload.__name__])
    @with_current_user
    @with_current_tenant_id
-    def patch(self, current_tenant_id: str, current_user: Account, external_knowledge_api_id: UUID):
+    @with_session
+    def patch(self, session: Session, current_tenant_id: str, current_user: Account, external_knowledge_api_id: UUID):
        external_knowledge_api_id_str = str(external_knowledge_api_id)

        payload = ExternalKnowledgeApiPayload.model_validate(console_ns.payload or {})
@@ -273,6 +281,7 @@ class ExternalApiTemplateApi(Resource):
            user_id=current_user.id,
            external_knowledge_api_id=external_knowledge_api_id_str,
            args=payload.model_dump(),
+            session=session,
        )

        return external_knowledge_api.to_dict(), 200
@@ -283,13 +292,14 @@ class ExternalApiTemplateApi(Resource):
    @console_ns.response(204, "External knowledge API deleted successfully")
    @with_current_user
    @with_current_tenant_id
-    def delete(self, current_tenant_id: str, current_user: Account, external_knowledge_api_id: UUID):
+    @with_session
+    def delete(self, session: Session, current_tenant_id: str, current_user: Account, external_knowledge_api_id: UUID):
        external_knowledge_api_id_str = str(external_knowledge_api_id)

        if not (current_user.has_edit_permission or current_user.is_dataset_operator):
            raise Forbidden()

-        ExternalDatasetService.delete_external_knowledge_api(current_tenant_id, external_knowledge_api_id_str)
+        ExternalDatasetService.delete_external_knowledge_api(session, current_tenant_id, external_knowledge_api_id_str)
        return "", 204


@@ -303,11 +313,14 @@ class ExternalApiUseCheckApi(Resource):
    @login_required
    @account_initialization_required
    @with_current_tenant_id
-    def get(self, current_tenant_id: str, external_knowledge_api_id: UUID):
+    @with_session
+    def get(self, session: Session, current_tenant_id: str, external_knowledge_api_id: UUID):
        external_knowledge_api_id_str = str(external_knowledge_api_id)

        external_knowledge_api_is_using, count = ExternalDatasetService.external_knowledge_api_use_check(
-            external_knowledge_api_id_str, current_tenant_id
+            session,
+            external_knowledge_api_id_str,
+            current_tenant_id,
        )
        return {"is_using": external_knowledge_api_is_using, "count": count}, 200

@@ -327,7 +340,8 @@ class ExternalDatasetCreateApi(Resource):
    @rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_EXTERNAL_CONNECT)
    @with_current_user
    @with_current_tenant_id
-    def post(self, current_tenant_id: str, current_user: Account):
+    @with_session
+    def post(self, session: Session, current_tenant_id: str, current_user: Account):
        # The role of the current user in the ta table must be admin, owner, or editor
        payload = ExternalDatasetCreatePayload.model_validate(console_ns.payload or {})
        args = payload.model_dump(exclude_none=True)
@@ -341,6 +355,7 @@ class ExternalDatasetCreateApi(Resource):
                tenant_id=current_tenant_id,
                user_id=current_user.id,
                args=args,
+                session=session,
            )
        except services.errors.dataset.DatasetNameDuplicateError:
            raise DatasetNameDuplicateError()
@@ -375,14 +390,15 @@ class ExternalKnowledgeHitTestingApi(Resource):
    @account_initialization_required
    @with_current_user
    @rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_PIPELINE_TEST)
-    def post(self, current_user: Account, dataset_id: UUID):
+    @with_session
+    def post(self, session: Session, current_user: Account, dataset_id: UUID):
        dataset_id_str = str(dataset_id)
-        dataset = DatasetService.get_dataset(dataset_id_str)
+        dataset = DatasetService.get_dataset(dataset_id_str, db.session)
        if dataset is None:
            raise NotFound("Dataset not found.")

        try:
-            DatasetService.check_dataset_permission(dataset, current_user)
+            DatasetService.check_dataset_permission(dataset, current_user, db.session)
        except services.errors.account.NoPermissionError as e:
            raise Forbidden(str(e))

@@ -391,7 +407,7 @@ class ExternalKnowledgeHitTestingApi(Resource):

        try:
            response = HitTestingService.external_retrieve(
-                session=db.session,
+                session=session,
                dataset=dataset,
                query=payload.query,
                account=current_user,
@@ -3,8 +3,10 @@ from __future__ import annotations
 from uuid import UUID

 from flask_restx import Resource
+from sqlalchemy.orm import Session

 from controllers.common.schema import register_response_schema_models, register_schema_models
+from controllers.console.app.wraps import with_session
 from controllers.console.wraps import RBACPermission, RBACResourceScope, rbac_permission_required
 from fields.hit_testing_fields import HitTestingResponse
 from libs.helper import dump_response
@@ -45,7 +47,10 @@ class HitTestingApi(Resource, DatasetsHitTestingBase):
    @with_current_tenant_id
    @with_current_user
    @rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_PIPELINE_TEST)
-    def post(self, current_user: Account, current_tenant_id: str, dataset_id: UUID) -> dict[str, object]:
+    @with_session
+    def post(
+        self, session: Session, current_user: Account, current_tenant_id: str, dataset_id: UUID
+    ) -> dict[str, object]:
        dataset_id_str = str(dataset_id)

        dataset = self.get_and_validate_dataset(dataset_id_str, current_user, current_tenant_id)
@@ -54,5 +59,5 @@ class HitTestingApi(Resource, DatasetsHitTestingBase):

        return dump_response(
            HitTestingResponse,
-            self.perform_hit_testing(dataset, args, current_user, current_tenant_id),
+            self.perform_hit_testing(session, dataset, args, current_user, current_tenant_id),
        )
@@ -2,6 +2,7 @@ import logging
 from typing import Any, cast

 from pydantic import BaseModel, Field
+from sqlalchemy.orm import Session
 from werkzeug.exceptions import Forbidden, InternalServerError, NotFound

 import services
@@ -85,12 +86,12 @@ class DatasetsHitTestingBase:
        dataset_id: str, current_user: Account | None = None, current_tenant_id: str | None = None
    ) -> Dataset:
        current_user, _ = resolve_account_fallback(current_user, current_tenant_id)
-        dataset = DatasetService.get_dataset(dataset_id)
+        dataset = DatasetService.get_dataset(dataset_id, db.session)
        if dataset is None:
            raise NotFound("Dataset not found.")

        try:
-            DatasetService.check_dataset_permission(dataset, current_user)
+            DatasetService.check_dataset_permission(dataset, current_user, db.session)
        except services.errors.account.NoPermissionError as e:
            raise Forbidden(str(e))

@@ -108,6 +109,7 @@ class DatasetsHitTestingBase:

    @staticmethod
    def perform_hit_testing(
+        session: Session,
        dataset: Dataset,
        args: dict[str, Any],
        current_user: Account | None = None,
@@ -116,7 +118,7 @@ class DatasetsHitTestingBase:
        try:
            current_user, _ = resolve_account_fallback(current_user, current_tenant_id)
            response = HitTestingService.retrieve(
-                session=db.session,
+                session=session,
                dataset=dataset,
                query=cast(str, args.get("query")),
                account=current_user,
@@ -61,10 +61,10 @@ class DatasetMetadataCreateApi(Resource):
        metadata_args = MetadataArgs.model_validate(console_ns.payload or {})

        dataset_id_str = str(dataset_id)
-        dataset = DatasetService.get_dataset(dataset_id_str)
+        dataset = DatasetService.get_dataset(dataset_id_str, db.session)
        if dataset is None:
            raise NotFound("Dataset not found.")
-        DatasetService.check_dataset_permission(dataset, current_user)
+        DatasetService.check_dataset_permission(dataset, current_user, db.session)

        metadata = MetadataService.create_metadata(
            db.session(), dataset_id_str, metadata_args, current_user, current_tenant_id
@@ -81,7 +81,7 @@ class DatasetMetadataCreateApi(Resource):
    @rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_CREATE_AND_MANAGEMENT)
    def get(self, dataset_id: UUID):
        dataset_id_str = str(dataset_id)
-        dataset = DatasetService.get_dataset(dataset_id_str)
+        dataset = DatasetService.get_dataset(dataset_id_str, db.session)
        if dataset is None:
            raise NotFound("Dataset not found.")
        metadata = MetadataService.get_dataset_metadatas(db.session(), dataset)
@@ -105,10 +105,10 @@ class DatasetMetadataApi(Resource):

        dataset_id_str = str(dataset_id)
        metadata_id_str = str(metadata_id)
-        dataset = DatasetService.get_dataset(dataset_id_str)
+        dataset = DatasetService.get_dataset(dataset_id_str, db.session)
        if dataset is None:
            raise NotFound("Dataset not found.")
-        DatasetService.check_dataset_permission(dataset, current_user)
+        DatasetService.check_dataset_permission(dataset, current_user, db.session)

        metadata = MetadataService.update_metadata_name(
            db.session(), dataset_id_str, metadata_id_str, name, current_user, current_tenant_id
@@ -125,10 +125,10 @@ class DatasetMetadataApi(Resource):
    def delete(self, current_user: Account, dataset_id: UUID, metadata_id: UUID):
        dataset_id_str = str(dataset_id)
        metadata_id_str = str(metadata_id)
-        dataset = DatasetService.get_dataset(dataset_id_str)
+        dataset = DatasetService.get_dataset(dataset_id_str, db.session)
        if dataset is None:
            raise NotFound("Dataset not found.")
-        DatasetService.check_dataset_permission(dataset, current_user)
+        DatasetService.check_dataset_permission(dataset, current_user, db.session)

        MetadataService.delete_metadata(db.session(), dataset_id_str, metadata_id_str)
        # Frontend callers only await success and invalidate metadata caches; no response body is consumed.
@@ -162,10 +162,10 @@ class DatasetMetadataBuiltInFieldActionApi(Resource):
    @rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_EDIT)
    def post(self, current_user: Account, dataset_id: UUID, action: Literal["enable", "disable"]):
        dataset_id_str = str(dataset_id)
-        dataset = DatasetService.get_dataset(dataset_id_str)
+        dataset = DatasetService.get_dataset(dataset_id_str, db.session)
        if dataset is None:
            raise NotFound("Dataset not found.")
-        DatasetService.check_dataset_permission(dataset, current_user)
+        DatasetService.check_dataset_permission(dataset, current_user, db.session)

        match action:
            case "enable":
@@ -191,10 +191,10 @@ class DocumentMetadataEditApi(Resource):
    @rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_EDIT)
    def post(self, current_user: Account, dataset_id: UUID):
        dataset_id_str = str(dataset_id)
-        dataset = DatasetService.get_dataset(dataset_id_str)
+        dataset = DatasetService.get_dataset(dataset_id_str, db.session)
        if dataset is None:
            raise NotFound("Dataset not found.")
-        DatasetService.check_dataset_permission(dataset, current_user)
+        DatasetService.check_dataset_permission(dataset, current_user, db.session)

        metadata_args = MetadataOperationData.model_validate(console_ns.payload or {})

@@ -5,7 +5,7 @@ from flask import request
 from flask_restx import Resource
 from pydantic import BaseModel, Field
 from sqlalchemy import select
-from sqlalchemy.orm import sessionmaker
+from sqlalchemy.orm import Session, sessionmaker
 from werkzeug.exceptions import NotFound

 from controllers.common.fields import SimpleDataResponse
@@ -16,6 +16,7 @@ from controllers.common.schema import (
    register_schema_models,
 )
 from controllers.console import console_ns
+from controllers.console.app.wraps import with_session
 from controllers.console.wraps import (
    account_initialization_required,
    enterprise_license_required,
@@ -102,10 +103,13 @@ class PipelineTemplateListApi(Resource):
    @account_initialization_required
    @enterprise_license_required
    @with_current_tenant_id
-    def get(self, current_tenant_id: str) -> JsonResponseWithStatus:
+    @with_session
+    def get(self, session: Session, current_tenant_id: str) -> JsonResponseWithStatus:
        query = PipelineTemplateListQuery.model_validate(request.args.to_dict(flat=True))
        # get pipeline templates
-        pipeline_templates = RagPipelineService.get_pipeline_templates(query.type, query.language, current_tenant_id)
+        pipeline_templates = RagPipelineService.get_pipeline_templates(
+            session, query.type, query.language, current_tenant_id
+        )
        return dump_response(PipelineTemplateListResponse, pipeline_templates), 200


@@ -117,10 +121,11 @@ class PipelineTemplateDetailApi(Resource):
    @login_required
    @account_initialization_required
    @enterprise_license_required
-    def get(self, template_id: str) -> JsonResponseWithStatus:
+    @with_session
+    def get(self, session: Session, template_id: str) -> JsonResponseWithStatus:
        query = PipelineTemplateDetailQuery.model_validate(request.args.to_dict(flat=True))
        rag_pipeline_service = RagPipelineService()
-        pipeline_template = rag_pipeline_service.get_pipeline_template_detail(template_id, query.type)
+        pipeline_template = rag_pipeline_service.get_pipeline_template_detail(session, template_id, query.type)
        if pipeline_template is None:
            raise NotFound("Pipeline template not found from upstream service.")
        return dump_response(PipelineTemplateDetailResponse, pipeline_template), 200
@@ -1,6 +1,5 @@
 from flask_restx import Resource
 from pydantic import BaseModel
-from sqlalchemy.orm import Session
 from werkzeug.exceptions import Forbidden

 import services
@@ -66,19 +65,19 @@ class CreateRagPipelineDatasetApi(Resource):
            yaml_content=payload.yaml_content,
        )
        try:
-            with Session(db.engine, expire_on_commit=False) as session:
-                rag_pipeline_dsl_service = RagPipelineDslService(session)
-                import_info = rag_pipeline_dsl_service.create_rag_pipeline_dataset(
-                    tenant_id=current_tenant_id,
-                    rag_pipeline_dataset_create_entity=rag_pipeline_dataset_create_entity,
-                )
-                session.commit()
+            rag_pipeline_dsl_service = RagPipelineDslService(db.session)
+            import_info = rag_pipeline_dsl_service.create_rag_pipeline_dataset(
+                tenant_id=current_tenant_id,
+                rag_pipeline_dataset_create_entity=rag_pipeline_dataset_create_entity,
+            )
            if rag_pipeline_dataset_create_entity.permission == "partial_members":
                DatasetPermissionService.update_partial_member_list(
                    current_tenant_id,
                    import_info["dataset_id"],
                    rag_pipeline_dataset_create_entity.partial_member_list,
+                    db.session,
                )
+            db.session.commit()
        except services.errors.dataset.DatasetNameDuplicateError:
            raise DatasetNameDuplicateError()

@@ -111,5 +110,6 @@ class CreateEmptyRagPipelineDatasetApi(Resource):
                permission=DatasetPermissionEnum.ONLY_ME,
                partial_member_list=None,
            ),
+            session=db.session,
        )
        return dump_response(DatasetDetailResponse, dataset), 201
@@ -6,7 +6,7 @@ from uuid import UUID
 from flask import abort, request
 from flask_restx import Resource
 from pydantic import BaseModel, Field, RootModel, ValidationError
-from sqlalchemy.orm import sessionmaker
+from sqlalchemy.orm import Session, sessionmaker
 from werkzeug.exceptions import BadRequest, Forbidden, InternalServerError, NotFound

 import services
@@ -26,6 +26,7 @@ from controllers.console.app.workflow import (
    WorkflowPaginationResponse,
    WorkflowResponse,
 )
+from controllers.console.app.wraps import with_session
 from controllers.console.datasets.wraps import get_rag_pipeline
 from controllers.console.wraps import (
    RBACPermission,
@@ -64,6 +65,7 @@ from services.rag_pipeline.pipeline_generate_service import PipelineGenerateServ
 from services.rag_pipeline.rag_pipeline import RagPipelineService
 from services.rag_pipeline.rag_pipeline_manage_service import RagPipelineManageService
 from services.rag_pipeline.rag_pipeline_transform_service import RagPipelineTransformService
+from services.workflow_ref_service import WorkflowRefService
 from services.workflow_service import DraftWorkflowDeletionError, WorkflowInUseError, WorkflowService

 logger = logging.getLogger(__name__)
@@ -342,7 +344,8 @@ class DraftRagPipelineRunApi(Resource):
    @rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_EDIT)
    @with_current_user
    @get_rag_pipeline
-    def post(self, current_user: Account, pipeline: Pipeline):
+    @with_session
+    def post(self, session: Session, current_user: Account, pipeline: Pipeline):
        """
        Run draft workflow
        """
@@ -351,6 +354,7 @@ class DraftRagPipelineRunApi(Resource):

        try:
            response = PipelineGenerateService.generate(
+                session=session,
                pipeline=pipeline,
                user=current_user,
                args=args,
@@ -374,7 +378,8 @@ class PublishedRagPipelineRunApi(Resource):
    @rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_EDIT)
    @with_current_user
    @get_rag_pipeline
-    def post(self, current_user: Account, pipeline: Pipeline):
+    @with_session
+    def post(self, session: Session, current_user: Account, pipeline: Pipeline):
        """
        Run published workflow
        """
@@ -384,6 +389,7 @@ class PublishedRagPipelineRunApi(Resource):

        try:
            response = PipelineGenerateService.generate(
+                session=session,
                pipeline=pipeline,
                user=current_user,
                args=args,
@@ -738,15 +744,15 @@ class RagPipelineByIdApi(Resource):
            return {"message": "No valid fields to update"}, 400

        rag_pipeline_service = RagPipelineService()
+        workflow_ref = WorkflowRefService.create_pipeline_workflow_ref(pipeline, workflow_id)

        # Create a session and manage the transaction
        with sessionmaker(db.engine, expire_on_commit=False).begin() as session:
            workflow = rag_pipeline_service.update_workflow(
                session=session,
-                workflow_id=workflow_id,
-                tenant_id=pipeline.tenant_id,
                account_id=current_user.id,
                data=update_data,
+                workflow_ref=workflow_ref,
            )

            if not workflow:
@@ -769,13 +775,13 @@ class RagPipelineByIdApi(Resource):
            abort(400, description=f"Cannot delete workflow that is currently in use by pipeline '{pipeline.id}'")

        workflow_service = WorkflowService()
+        workflow_ref = WorkflowRefService.create_pipeline_workflow_ref(pipeline, workflow_id)

        with sessionmaker(db.engine).begin() as session:
            try:
                workflow_service.delete_workflow(
                    session=session,
-                    workflow_id=workflow_id,
-                    tenant_id=pipeline.tenant_id,
+                    workflow_ref=workflow_ref,
                )
            except WorkflowInUseError as e:
                abort(400, description=str(e))
@@ -1013,13 +1019,14 @@ class RagPipelineTransformApi(Resource):
    @login_required
    @account_initialization_required
    @with_current_user
-    def post(self, current_user: Account, dataset_id: UUID):
+    @with_session
+    def post(self, session: Session, current_user: Account, dataset_id: UUID):
        if not (current_user.has_edit_permission or current_user.is_dataset_operator):
            raise Forbidden()

        dataset_id_str = str(dataset_id)
        rag_pipeline_transform_service = RagPipelineTransformService()
-        result = rag_pipeline_transform_service.transform_dataset(dataset_id_str, db.session)
+        result = rag_pipeline_transform_service.transform_dataset(dataset_id_str, session)
        return result


@@ -58,12 +58,6 @@ class WorkspacesLimitExceeded(BaseHTTPException):
    code = 400


-class SeatsLimitExceeded(BaseHTTPException):
-    error_code = "limit_exceeded"
-    description = "Unable to create account because the licensed seats limit was exceeded"
-    code = 400
-
-
 class AccountBannedError(BaseHTTPException):
    error_code = "account_banned"
    description = "Account is banned."
@@ -22,7 +22,9 @@ from controllers.console.explore.wraps import InstalledAppResource
 from core.errors.error import ModelCurrentlyNotSupportError, ProviderTokenNotInitError, QuotaExceededError
 from extensions.ext_database import db
 from graphon.model_runtime.errors.invoke import InvokeError
+from libs.login import current_account_with_tenant
 from models.model import InstalledApp
+from services.app_ref_service import AppRefService
 from services.audio_service import AudioService
 from services.errors.audio import (
    AudioTooLargeServiceError,
@@ -99,13 +101,22 @@ class ChatTextApi(InstalledAppResource):
            message_id = payload.message_id
            text = payload.text
            voice = payload.voice
+            message_ref = None
+            if message_id:
+                current_user, _ = current_account_with_tenant()
+                app_ref = AppRefService.create_app_ref(app_model)
+                message_ref = AppRefService.create_message_ref(
+                    app_ref,
+                    message_id,
+                    account_id=current_user.id,
+                )

            response = AudioService.transcript_tts(
                app_model=app_model,
                session=db.session,
                text=text,
                voice=voice,
-                message_id=message_id,
+                message_ref=message_ref,
            )
            return response
        except services.errors.app_model_config.AppModelConfigBrokenError:
@@ -3,10 +3,11 @@ from typing import Any, Literal
 from uuid import UUID

 from pydantic import BaseModel, Field, field_validator
+from sqlalchemy.orm import Session
 from werkzeug.exceptions import InternalServerError, NotFound

 import services
-from controllers.common.fields import GeneratedAppResponse, SimpleResultResponse
+from controllers.common.fields import SimpleResultResponse
 from controllers.common.schema import register_response_schema_models, register_schema_models
 from controllers.console.app.error import (
    AppUnavailableError,
@@ -16,6 +17,7 @@ from controllers.console.app.error import (
    ProviderNotInitializeError,
    ProviderQuotaExceededError,
 )
+from controllers.console.app.wraps import with_session
 from controllers.console.explore.error import NotChatAppError, NotCompletionAppError
 from controllers.console.explore.wraps import InstalledAppResource
 from controllers.console.wraps import with_current_user, with_current_user_id
@@ -73,7 +75,7 @@ class ChatMessagePayload(BaseModel):


 register_schema_models(console_ns, CompletionMessageExplorePayload, ChatMessagePayload)
-register_response_schema_models(console_ns, GeneratedAppResponse, SimpleResultResponse)
+register_response_schema_models(console_ns, SimpleResultResponse)


 # define completion api for user
@@ -83,9 +85,10 @@ register_response_schema_models(console_ns, GeneratedAppResponse, SimpleResultRe
 )
 class CompletionApi(InstalledAppResource):
    @console_ns.expect(console_ns.models[CompletionMessageExplorePayload.__name__])
-    @console_ns.response(200, "Success", console_ns.models[GeneratedAppResponse.__name__])
+    @console_ns.response(200, "Success")
    @with_current_user
-    def post(self, current_user: Account, installed_app: InstalledApp):
+    @with_session
+    def post(self, session: Session, current_user: Account, installed_app: InstalledApp):
        app_model = installed_app.app
        if app_model is None:
            raise AppUnavailableError()
@@ -103,9 +106,15 @@ class CompletionApi(InstalledAppResource):

        try:
            response = AppGenerateService.generate(
-                app_model=app_model, user=current_user, args=args, invoke_from=InvokeFrom.EXPLORE, streaming=streaming
+                session=session,
+                app_model=app_model,
+                user=current_user,
+                args=args,
+                invoke_from=InvokeFrom.EXPLORE,
+                streaming=streaming,
            )

+            # response-contract:ignore compact_generate_response
            return helper.compact_generate_response(response)
        except services.errors.conversation.ConversationNotExistsError:
            raise NotFound("Conversation Not Exists.")
@@ -150,7 +159,7 @@ class CompletionStopApi(InstalledAppResource):
            app_mode=AppMode.value_of(app_model.mode),
        )

-        return {"result": "success"}, 200
+        return SimpleResultResponse(result="success").model_dump(mode="json"), 200


@console_ns.route(
@@ -159,9 +168,10 @@ class CompletionStopApi(InstalledAppResource):
 )
 class ChatApi(InstalledAppResource):
    @console_ns.expect(console_ns.models[ChatMessagePayload.__name__])
-    @console_ns.response(200, "Success", console_ns.models[GeneratedAppResponse.__name__])
+    @console_ns.response(200, "Success")
    @with_current_user
-    def post(self, current_user: Account, installed_app: InstalledApp):
+    @with_session
+    def post(self, session: Session, current_user: Account, installed_app: InstalledApp):
        app_model = installed_app.app
        if app_model is None:
            raise AppUnavailableError()
@@ -179,9 +189,15 @@ class ChatApi(InstalledAppResource):

        try:
            response = AppGenerateService.generate(
-                app_model=app_model, user=current_user, args=args, invoke_from=InvokeFrom.EXPLORE, streaming=True
+                session=session,
+                app_model=app_model,
+                user=current_user,
+                args=args,
+                invoke_from=InvokeFrom.EXPLORE,
+                streaming=True,
            )

+            # response-contract:ignore compact_generate_response
            return helper.compact_generate_response(response)
        except services.errors.conversation.ConversationNotExistsError:
            raise NotFound("Conversation Not Exists.")
@@ -229,4 +245,4 @@ class ChatStopApi(InstalledAppResource):
            app_mode=app_mode,
        )

-        return {"result": "success"}, 200
+        return SimpleResultResponse(result="success").model_dump(mode="json"), 200
@@ -85,6 +85,7 @@ def _published_app_filter():
 class InstalledAppInfoResponse(ResponseModel):
    id: str
    name: str | None = None
+    description: str | None = None
    mode: str | None = None
    icon_type: str | None = None
    icon: str | None = None
@@ -123,6 +124,7 @@ class InstalledAppResponse(ResponseModel):
        return {
            "id": _safe_primitive(getattr(value, "id", "")) or "",
            "name": _safe_primitive(getattr(value, "name", None)),
+            "description": _safe_primitive(getattr(value, "description", None)),
            "mode": _safe_primitive(getattr(value, "mode", None)),
            "icon_type": _safe_primitive(getattr(value, "icon_type", None)),
            "icon": _safe_primitive(getattr(value, "icon", None)),
@@ -145,11 +147,15 @@ register_schema_models(
    InstalledAppCreatePayload,
    InstalledAppUpdatePayload,
    InstalledAppsListQuery,
+)
+register_response_schema_models(
+    console_ns,
    InstalledAppInfoResponse,
    InstalledAppResponse,
    InstalledAppListResponse,
+    SimpleMessageResponse,
+    SimpleResultMessageResponse,
 )
-register_response_schema_models(console_ns, SimpleMessageResponse, SimpleResultMessageResponse)


@console_ns.route("/installed-apps")
@@ -4,10 +4,10 @@ from uuid import UUID

 from flask import request
 from pydantic import BaseModel, TypeAdapter
+from sqlalchemy.orm import Session
 from werkzeug.exceptions import InternalServerError, NotFound

 from controllers.common.controller_schemas import MessageFeedbackPayload, MessageListQuery
-from controllers.common.fields import GeneratedAppResponse
 from controllers.common.schema import query_params_from_model, register_response_schema_models, register_schema_models
 from controllers.console.app.error import (
    AppMoreLikeThisDisabledError,
@@ -17,6 +17,7 @@ from controllers.console.app.error import (
    ProviderNotInitializeError,
    ProviderQuotaExceededError,
 )
+from controllers.console.app.wraps import with_session
 from controllers.console.explore.error import (
    AppSuggestedQuestionsAfterAnswerDisabledError,
    NotChatAppError,
@@ -59,7 +60,6 @@ class MoreLikeThisQuery(BaseModel):
 register_schema_models(console_ns, MessageListQuery, MessageFeedbackPayload, MoreLikeThisQuery)
 register_response_schema_models(
    console_ns,
-    GeneratedAppResponse,
    ExploreMessageInfiniteScrollPagination,
    ResultResponse,
    SuggestedQuestionsResponse,
@@ -88,8 +88,8 @@ class MessageListApi(InstalledAppResource):
            pagination = MessageService.pagination_by_first_id(
                app_model,
                current_user,
-                str(args.conversation_id),
-                str(args.first_id) if args.first_id else None,
+                args.conversation_id,
+                args.first_id or None,
                args.limit,
            )
            adapter = TypeAdapter(ExploreMessageListItem)
@@ -142,9 +142,10 @@ class MessageFeedbackApi(InstalledAppResource):
 )
 class MessageMoreLikeThisApi(InstalledAppResource):
    @console_ns.doc(params=query_params_from_model(MoreLikeThisQuery))
-    @console_ns.response(200, "Success", console_ns.models[GeneratedAppResponse.__name__])
+    @console_ns.response(200, "Success")
    @with_current_user
-    def get(self, current_user: Account, installed_app: InstalledApp, message_id: UUID):
+    @with_session
+    def get(self, session: Session, current_user: Account, installed_app: InstalledApp, message_id: UUID):
        app_model = installed_app.app
        if app_model is None:
            raise AppUnavailableError()
@@ -159,12 +160,14 @@ class MessageMoreLikeThisApi(InstalledAppResource):

        try:
            response = AppGenerateService.generate_more_like_this(
+                session=session,
                app_model=app_model,
                user=current_user,
                message_id=message_id_str,
                invoke_from=InvokeFrom.EXPLORE,
                streaming=streaming,
            )
+            # response-contract:ignore compact_generate_response
            return helper.compact_generate_response(response)
        except MessageNotExistsError:
            raise NotFound("Message Not Exists.")
@@ -13,7 +13,12 @@ from services.app_service import AppService


 class ExploreAppMetaResponse(BaseModel):
-    tool_icons: dict[str, Any] = Field(default_factory=dict)
+    """Metadata consumed by the installed-app chat UI.
+
+    Built-in tool icons are URL strings; API-based tool icons are provider-defined payload objects.
+    """
+
+    tool_icons: dict[str, str | dict[str, Any]] = Field(default_factory=dict)


 register_response_schema_models(console_ns, fields.Parameters, ExploreAppMetaResponse)
@@ -70,19 +70,33 @@ class LearnDifyAppListResponse(ResponseModel):
    recommended_apps: list[RecommendedAppResponse]


-class RecommendedAppDetailResponse(RootModel[dict[str, Any]]):
-    root: dict[str, Any]
+class RecommendedAppDetailResponse(ResponseModel):
+    id: str
+    name: str
+    icon: str | None = None
+    icon_background: str | None = None
+    mode: str
+    export_data: str
+    can_trial: bool | None = None
+
+
+class RecommendedAppDetailNullableResponse(RootModel[RecommendedAppDetailResponse | None]):
+    pass


 register_schema_models(
    console_ns,
    RecommendedAppsQuery,
+)
+register_response_schema_models(
+    console_ns,
    RecommendedAppInfoResponse,
    RecommendedAppResponse,
    RecommendedAppListResponse,
    LearnDifyAppListResponse,
+    RecommendedAppDetailResponse,
+    RecommendedAppDetailNullableResponse,
 )
-register_response_schema_models(console_ns, RecommendedAppDetailResponse)


 def _resolve_language(language: str | None, user: Account) -> str:
@@ -106,7 +120,7 @@ class RecommendedAppListApi(Resource):
        language_prefix = _resolve_language(args.language, current_user)

        return RecommendedAppListResponse.model_validate(
-            RecommendedAppService.get_recommended_apps_and_categories(language_prefix, session=db.session()),
+            RecommendedAppService.get_recommended_apps_and_categories(db.session, language_prefix),
            from_attributes=True,
        ).model_dump(mode="json")

@@ -130,7 +144,7 @@ class LearnDifyAppListApi(Resource):

@console_ns.route("/explore/apps/<uuid:app_id>")
 class RecommendedAppApi(Resource):
-    @console_ns.response(200, "Success", console_ns.models[RecommendedAppDetailResponse.__name__])
+    @console_ns.response(200, "Success", console_ns.models[RecommendedAppDetailNullableResponse.__name__])
    @login_required
    @account_initialization_required
    def get(self, app_id: UUID):
@@ -1,10 +1,12 @@
 import logging
-from typing import Any, Literal, cast
+from datetime import datetime
+from typing import Any, Literal

 from flask import request
-from flask_restx import Resource, fields, marshal, marshal_with
-from pydantic import BaseModel, Field
+from flask_restx import Resource
+from pydantic import AliasChoices, BaseModel, Field, field_validator
 from sqlalchemy import select
+from sqlalchemy.orm import Session
 from werkzeug.exceptions import Forbidden, InternalServerError, NotFound

 import services
@@ -17,7 +19,6 @@ from controllers.common.fields import (
 from controllers.common.fields import Parameters as ParametersResponse
 from controllers.common.fields import Site as SiteResponse
 from controllers.common.schema import (
-    get_or_create_model,
    query_params_from_model,
    register_response_schema_models,
    register_schema_models,
@@ -36,7 +37,7 @@ from controllers.console.app.error import (
    ProviderQuotaExceededError,
    UnsupportedAudioTypeError,
 )
-from controllers.console.app.wraps import get_app_model_with_trial
+from controllers.console.app.wraps import get_app_model_with_trial, with_session
 from controllers.console.explore.error import (
    AppSuggestedQuestionsAfterAnswerDisabledError,
    NotChatAppError,
@@ -44,9 +45,7 @@ from controllers.console.explore.error import (
    NotWorkflowAppError,
 )
 from controllers.console.explore.wraps import TrialAppResource, trial_feature_enable
-from controllers.console.files import FILE_UPLOAD_PARAMS, upload_file_from_request
-from controllers.console.remote_files import RemoteFileUploadPayload, upload_remote_file_from_request
-from controllers.console.wraps import cloud_edition_billing_resource_check, with_current_user
+from controllers.console.wraps import with_current_user
 from controllers.web.error import InvokeRateLimitError as InvokeRateLimitHttpError
 from core.app.app_config.common.parameters_mapping import get_parameters_from_feature_dict
 from core.app.apps.base_app_queue_manager import AppQueueManager
@@ -58,32 +57,18 @@ from core.errors.error import (
 )
 from extensions.ext_database import db
 from extensions.ext_redis import redis_client
-from fields.app_fields import (
-    app_detail_fields_with_site,
-    deleted_tool_fields,
-    model_config_fields,
-    site_fields,
-    tag_fields,
-)
-from fields.dataset_fields import dataset_fields
-from fields.file_fields import FileResponse, FileWithSignedUrl
-from fields.member_fields import simple_account_fields
+from fields.base import ResponseModel
 from fields.message_fields import SuggestedQuestionsResponse
-from fields.workflow_fields import (
-    conversation_variable_fields,
-    pipeline_variable_fields,
-    workflow_fields,
-    workflow_partial_fields,
-)
 from graphon.graph_engine.manager import GraphEngineManager
 from graphon.model_runtime.errors.invoke import InvokeError
 from libs import helper
-from libs.helper import dump_response, uuid_value
-from models import Account, App
+from libs.helper import dump_response, to_timestamp, uuid_value
+from models import Account
 from models.account import TenantStatus
 from models.model import AppMode, Site
 from models.workflow import Workflow
 from services.app_generate_service import AppGenerateService
+from services.app_ref_service import AppRefService
 from services.app_service import AppService
 from services.audio_service import AudioService
 from services.dataset_service import DatasetService
@@ -105,48 +90,6 @@ from services.recommended_app_service import RecommendedAppService
 logger = logging.getLogger(__name__)


-model_config_model = get_or_create_model("TrialAppModelConfig", model_config_fields)
-workflow_partial_model = get_or_create_model("TrialWorkflowPartial", workflow_partial_fields)
-deleted_tool_model = get_or_create_model("TrialDeletedTool", deleted_tool_fields)
-tag_model = get_or_create_model("TrialTag", tag_fields)
-site_model = get_or_create_model("TrialSite", site_fields)
-
-app_detail_fields_with_site_copy = app_detail_fields_with_site.copy()
-app_detail_fields_with_site_copy["model_config"] = fields.Nested(
-    model_config_model, attribute="app_model_config", allow_null=True
-)
-app_detail_fields_with_site_copy["workflow"] = fields.Nested(workflow_partial_model, allow_null=True)
-app_detail_fields_with_site_copy["deleted_tools"] = fields.List(fields.Nested(deleted_tool_model))
-app_detail_fields_with_site_copy["tags"] = fields.List(fields.Nested(tag_model))
-app_detail_fields_with_site_copy["site"] = fields.Nested(site_model)
-app_detail_with_site_model = get_or_create_model("TrialAppDetailWithSite", app_detail_fields_with_site_copy)
-
-simple_account_model = get_or_create_model("TrialSimpleAccount", simple_account_fields)
-conversation_variable_model = get_or_create_model("TrialConversationVariable", conversation_variable_fields)
-pipeline_variable_model = get_or_create_model("TrialPipelineVariable", pipeline_variable_fields)
-
-workflow_fields_copy = workflow_fields.copy()
-workflow_fields_copy["created_by"] = fields.Nested(simple_account_model, attribute="created_by_account")
-workflow_fields_copy["updated_by"] = fields.Nested(
-    simple_account_model, attribute="updated_by_account", allow_null=True
-)
-workflow_fields_copy["conversation_variables"] = fields.List(fields.Nested(conversation_variable_model))
-workflow_fields_copy["rag_pipeline_variables"] = fields.List(fields.Nested(pipeline_variable_model))
-workflow_model = get_or_create_model("TrialWorkflow", workflow_fields_copy)
-
-dataset_model = get_or_create_model("TrialDataset", dataset_fields)
-dataset_list_model = get_or_create_model(
-    "TrialDatasetList",
-    {
-        "data": fields.List(fields.Nested(dataset_model)),
-        "has_more": fields.Boolean,
-        "limit": fields.Integer,
-        "total": fields.Integer,
-        "page": fields.Integer,
-    },
-)
-
-
 class WorkflowRunRequest(BaseModel):
    inputs: dict
    files: list | None = Field(default=None)
@@ -182,6 +125,259 @@ class TrialDatasetListQuery(BaseModel):
    ids: list[str] = Field(default_factory=list, description="Dataset IDs")


+type TrialAppMode = Literal["chat", "agent-chat", "advanced-chat", "workflow", "completion"]
+type TrialIconType = Literal["emoji", "image", "link"]
+type JsonObject = dict[str, Any]
+
+
+class TrialAppModel(ResponseModel):
+    provider: str
+    name: str
+    mode: str | None = None
+    completion_params: JsonObject = Field(default_factory=dict)
+
+
+class TrialAppAgentMode(ResponseModel):
+    enabled: bool | None = None
+    strategy: str | None = None
+    tools: list[JsonObject] = Field(default_factory=list)
+
+
+class TrialAppModelConfigResponse(ResponseModel):
+    opening_statement: str | None = None
+    suggested_questions: list[str] = Field(
+        default_factory=list,
+        validation_alias=AliasChoices("suggested_questions_list", "suggested_questions"),
+    )
+    suggested_questions_after_answer: JsonObject | None = Field(
+        default=None,
+        validation_alias=AliasChoices("suggested_questions_after_answer_dict", "suggested_questions_after_answer"),
+    )
+    speech_to_text: JsonObject | None = Field(
+        default=None, validation_alias=AliasChoices("speech_to_text_dict", "speech_to_text")
+    )
+    text_to_speech: JsonObject | None = Field(
+        default=None, validation_alias=AliasChoices("text_to_speech_dict", "text_to_speech")
+    )
+    retriever_resource: JsonObject | None = Field(
+        default=None, validation_alias=AliasChoices("retriever_resource_dict", "retriever_resource")
+    )
+    annotation_reply: JsonObject | None = Field(
+        default=None, validation_alias=AliasChoices("annotation_reply_dict", "annotation_reply")
+    )
+    more_like_this: JsonObject | None = Field(
+        default=None, validation_alias=AliasChoices("more_like_this_dict", "more_like_this")
+    )
+    sensitive_word_avoidance: JsonObject | None = Field(
+        default=None, validation_alias=AliasChoices("sensitive_word_avoidance_dict", "sensitive_word_avoidance")
+    )
+    external_data_tools: list[JsonObject] = Field(
+        default_factory=list, validation_alias=AliasChoices("external_data_tools_list", "external_data_tools")
+    )
+    model: TrialAppModel | None = Field(default=None, validation_alias=AliasChoices("model_dict", "model"))
+    user_input_form: list[JsonObject] = Field(
+        default_factory=list, validation_alias=AliasChoices("user_input_form_list", "user_input_form")
+    )
+    dataset_query_variable: str | None = None
+    pre_prompt: str | None = None
+    agent_mode: TrialAppAgentMode | None = Field(
+        default=None,
+        validation_alias=AliasChoices("agent_mode_dict", "agent_mode"),
+    )
+    prompt_type: str | None = None
+    chat_prompt_config: JsonObject | None = Field(
+        default=None, validation_alias=AliasChoices("chat_prompt_config_dict", "chat_prompt_config")
+    )
+    completion_prompt_config: JsonObject | None = Field(
+        default=None, validation_alias=AliasChoices("completion_prompt_config_dict", "completion_prompt_config")
+    )
+    dataset_configs: JsonObject | None = Field(
+        default=None,
+        validation_alias=AliasChoices("dataset_configs_dict", "dataset_configs"),
+    )
+    file_upload: JsonObject | None = Field(
+        default=None,
+        validation_alias=AliasChoices("file_upload_dict", "file_upload"),
+    )
+    created_by: str | None = None
+    created_at: int | None = None
+    updated_by: str | None = None
+    updated_at: int | None = None
+
+    @field_validator("created_at", "updated_at", mode="before")
+    @classmethod
+    def _normalize_timestamp(cls, value: datetime | int | None) -> int | None:
+        return to_timestamp(value)
+
+
+class TrialDeletedToolResponse(ResponseModel):
+    type: str
+    tool_name: str
+    provider_id: str
+
+
+class TrialTagResponse(ResponseModel):
+    id: str
+    name: str
+    type: str
+
+
+class TrialSiteResponse(ResponseModel):
+    access_token: str | None = Field(default=None, validation_alias="code")
+    code: str | None = None
+    title: str
+    icon_type: TrialIconType | None = None
+    icon: str | None = None
+    icon_background: str | None = None
+    description: str | None = None
+    default_language: str
+    chat_color_theme: str | None = None
+    chat_color_theme_inverted: bool | None = None
+    customize_domain: str | None = None
+    copyright: str | None = None
+    privacy_policy: str | None = None
+    input_placeholder: str | None = None
+    custom_disclaimer: str | None = None
+    customize_token_strategy: str | None = None
+    prompt_public: bool | None = None
+    app_base_url: str | None = None
+    show_workflow_steps: bool | None = None
+    use_icon_as_answer_icon: bool | None = None
+    created_by: str | None = None
+    created_at: int | None = None
+    updated_by: str | None = None
+    updated_at: int | None = None
+    icon_url: str | None = None
+
+    @field_validator("icon_type", mode="before")
+    @classmethod
+    def _normalize_icon_type(cls, value: Any) -> str | None:
+        if hasattr(value, "value"):
+            return value.value
+        return value
+
+    @field_validator("created_at", "updated_at", mode="before")
+    @classmethod
+    def _normalize_timestamp(cls, value: datetime | int | None) -> int | None:
+        return to_timestamp(value)
+
+
+class TrialWorkflowPartialResponse(ResponseModel):
+    id: str
+    created_by: str | None = None
+    created_at: int | None = None
+    updated_by: str | None = None
+    updated_at: int | None = None
+
+    @field_validator("created_at", "updated_at", mode="before")
+    @classmethod
+    def _normalize_timestamp(cls, value: datetime | int | None) -> int | None:
+        return to_timestamp(value)
+
+
+class TrialAppDetailResponse(ResponseModel):
+    id: str
+    name: str
+    description: str | None = None
+    mode: TrialAppMode = Field(validation_alias="mode_compatible_with_agent")
+    icon_type: TrialIconType | None = None
+    icon: str | None = None
+    icon_background: str | None = None
+    icon_url: str | None = None
+    enable_site: bool
+    enable_api: bool
+    model_config_: TrialAppModelConfigResponse | None = Field(
+        default=None,
+        validation_alias=AliasChoices("app_model_config", "model_config"),
+        alias="model_config",
+    )
+    workflow: TrialWorkflowPartialResponse | None = None
+    api_base_url: str | None = None
+    use_icon_as_answer_icon: bool | None = None
+    max_active_requests: int | None = None
+    created_by: str | None = None
+    created_at: int | None = None
+    updated_by: str | None = None
+    updated_at: int | None = None
+    deleted_tools: list[TrialDeletedToolResponse] = Field(default_factory=list)
+    access_mode: str | None = None
+    tags: list[TrialTagResponse] = Field(default_factory=list)
+    permission_keys: list[str] = Field(default_factory=list)
+    site: TrialSiteResponse
+
+    @field_validator("icon_type", mode="before")
+    @classmethod
+    def _normalize_icon_type(cls, value: Any) -> str | None:
+        if hasattr(value, "value"):
+            return value.value
+        return value
+
+    @field_validator("created_at", "updated_at", mode="before")
+    @classmethod
+    def _normalize_timestamp(cls, value: datetime | int | None) -> int | None:
+        return to_timestamp(value)
+
+
+class TrialDatasetResponse(ResponseModel):
+    id: str
+    name: str
+    description: str | None = None
+    permission: str | None = None
+    data_source_type: str | None = None
+    indexing_technique: str | None = None
+    created_by: str | None = None
+    created_at: int | None = None
+    permission_keys: list[str] = Field(default_factory=list)
+
+    @field_validator("created_at", mode="before")
+    @classmethod
+    def _normalize_timestamp(cls, value: datetime | int | None) -> int | None:
+        return to_timestamp(value)
+
+
+class TrialDatasetListResponse(ResponseModel):
+    data: list[TrialDatasetResponse]
+    has_more: bool
+    limit: int
+    total: int
+    page: int
+
+
+class TrialSimpleAccount(ResponseModel):
+    id: str
+    name: str | None = None
+    email: str | None = None
+
+
+class TrialWorkflowResponse(ResponseModel):
+    id: str
+    graph: JsonObject = Field(validation_alias=AliasChoices("graph_dict", "graph"))
+    features: JsonObject = Field(default_factory=dict, validation_alias=AliasChoices("features_dict", "features"))
+    hash: str | None = Field(default=None, validation_alias=AliasChoices("unique_hash", "hash"))
+    version: str | None = None
+    marked_name: str | None = None
+    marked_comment: str | None = None
+    created_by: TrialSimpleAccount | None = Field(
+        default=None,
+        validation_alias=AliasChoices("created_by_account", "created_by"),
+    )
+    created_at: int | None = None
+    updated_by: TrialSimpleAccount | None = Field(
+        default=None,
+        validation_alias=AliasChoices("updated_by_account", "updated_by"),
+    )
+    updated_at: int | None = None
+    tool_published: bool | None = None
+    environment_variables: list[JsonObject] = Field(default_factory=list)
+    conversation_variables: list[JsonObject] = Field(default_factory=list)
+    rag_pipeline_variables: list[JsonObject] = Field(default_factory=list)
+
+    @field_validator("created_at", "updated_at", mode="before")
+    @classmethod
+    def _normalize_timestamp(cls, value: datetime | int | None) -> int | None:
+        return to_timestamp(value)
+
+
 register_schema_models(
    console_ns,
    WorkflowRunRequest,
@@ -199,37 +395,12 @@ register_response_schema_models(
    SimpleResultResponse,
    SiteResponse,
    SuggestedQuestionsResponse,
+    TrialAppDetailResponse,
+    TrialDatasetListResponse,
+    TrialWorkflowResponse,
 )

-
-class TrialAppFileUploadApi(TrialAppResource):
-    @trial_feature_enable
-    @cloud_edition_billing_resource_check("documents")
-    @console_ns.doc(consumes=["multipart/form-data"], params=FILE_UPLOAD_PARAMS)
-    @console_ns.response(201, "File uploaded successfully", console_ns.models[FileResponse.__name__])
-    @with_current_user
-    def post(self, current_user: Account, app_model: App):
-        """Upload a file into the tenant that owns the trial app."""
-        upload_file = upload_file_from_request(
-            current_user=current_user,
-            resource_tenant_id=app_model.tenant_id,
-        )
-        return dump_response(FileResponse, upload_file), 201
-
-
-class TrialAppRemoteFileUploadApi(TrialAppResource):
-    @trial_feature_enable
-    @cloud_edition_billing_resource_check("documents")
-    @console_ns.expect(console_ns.models[RemoteFileUploadPayload.__name__])
-    @console_ns.response(201, "File uploaded successfully", console_ns.models[FileWithSignedUrl.__name__])
-    @with_current_user
-    def post(self, current_user: Account, app_model: App):
-        """Upload a remote file into the tenant that owns the trial app."""
-        remote_file = upload_remote_file_from_request(
-            current_user=current_user,
-            resource_tenant_id=app_model.tenant_id,
-        )
-        return remote_file.model_dump(mode="json"), 201
+simple_account_model = console_ns.models[TrialSimpleAccount.__name__]


 class TrialAppWorkflowRunApi(TrialAppResource):
@@ -237,7 +408,8 @@ class TrialAppWorkflowRunApi(TrialAppResource):
    @console_ns.expect(console_ns.models[WorkflowRunRequest.__name__])
    @console_ns.response(200, "Success", console_ns.models[GeneratedAppResponse.__name__])
    @with_current_user
-    def post(self, current_user: Account, trial_app):
+    @with_session
+    def post(self, session: Session, current_user: Account, trial_app):
        """
        Run workflow
        """
@@ -254,7 +426,12 @@ class TrialAppWorkflowRunApi(TrialAppResource):
            app_id = app_model.id
            user_id = current_user.id
            response = AppGenerateService.generate(
-                app_model=app_model, user=current_user, args=args, invoke_from=InvokeFrom.EXPLORE, streaming=True
+                session=session,
+                app_model=app_model,
+                user=current_user,
+                args=args,
+                invoke_from=InvokeFrom.EXPLORE,
+                streaming=True,
            )
            RecommendedAppService.add_trial_app_record(db.session, app_id, user_id)
            return helper.compact_generate_response(response)
@@ -304,7 +481,8 @@ class TrialChatApi(TrialAppResource):
    @console_ns.response(200, "Success", console_ns.models[GeneratedAppResponse.__name__])
    @trial_feature_enable
    @with_current_user
-    def post(self, current_user: Account, trial_app):
+    @with_session
+    def post(self, session: Session, current_user: Account, trial_app):
        app_model = trial_app
        app_mode = AppMode.value_of(app_model.mode)
        if app_mode not in {AppMode.CHAT, AppMode.AGENT_CHAT, AppMode.ADVANCED_CHAT}:
@@ -327,7 +505,12 @@ class TrialChatApi(TrialAppResource):
            user_id = current_user.id

            response = AppGenerateService.generate(
-                app_model=app_model, user=current_user, args=args, invoke_from=InvokeFrom.EXPLORE, streaming=True
+                session=session,
+                app_model=app_model,
+                user=current_user,
+                args=args,
+                invoke_from=InvokeFrom.EXPLORE,
+                streaming=True,
            )
            RecommendedAppService.add_trial_app_record(db.session, app_id, user_id)
            return helper.compact_generate_response(response)
@@ -447,6 +630,14 @@ class TrialChatTextApi(TrialAppResource):
            message_id = request_data.message_id
            text = request_data.text
            voice = request_data.voice
+            message_ref = None
+            if message_id:
+                app_ref = AppRefService.create_app_ref(app_model)
+                message_ref = AppRefService.create_message_ref(
+                    app_ref,
+                    message_id,
+                    account_id=current_user.id,
+                )

            # Get IDs before they might be detached from session
            app_id = app_model.id
@@ -457,7 +648,7 @@ class TrialChatTextApi(TrialAppResource):
                session=db.session,
                text=text,
                voice=voice,
-                message_id=message_id,
+                message_ref=message_ref,
            )
            RecommendedAppService.add_trial_app_record(db.session, app_id, user_id)
            return response
@@ -492,7 +683,8 @@ class TrialCompletionApi(TrialAppResource):
    @console_ns.response(200, "Success", console_ns.models[GeneratedAppResponse.__name__])
    @trial_feature_enable
    @with_current_user
-    def post(self, current_user: Account, trial_app):
+    @with_session
+    def post(self, session: Session, current_user: Account, trial_app):
        app_model = trial_app
        if app_model.mode != "completion":
            raise NotCompletionAppError()
@@ -509,7 +701,12 @@ class TrialCompletionApi(TrialAppResource):
            user_id = current_user.id

            response = AppGenerateService.generate(
-                app_model=app_model, user=current_user, args=args, invoke_from=InvokeFrom.EXPLORE, streaming=streaming
+                session=session,
+                app_model=app_model,
+                user=current_user,
+                args=args,
+                invoke_from=InvokeFrom.EXPLORE,
+                streaming=streaming,
            )

            RecommendedAppService.add_trial_app_record(db.session, app_id, user_id)
@@ -590,34 +787,35 @@ class TrialAppParameterApi(Resource):


 class AppApi(Resource):
-    @console_ns.response(200, "Success", app_detail_with_site_model)
+    @console_ns.response(200, "Success", console_ns.models[TrialAppDetailResponse.__name__])
    @get_app_model_with_trial(None)
-    @marshal_with(app_detail_with_site_model)
    def get(self, app_model):
        """Get app detail"""

        app_service = AppService()
        app_model = app_service.get_app(app_model)

-        return app_model
+        return dump_response(TrialAppDetailResponse, app_model)


 class AppWorkflowApi(Resource):
-    @console_ns.response(200, "Success", workflow_model)
+    @console_ns.response(200, "Success", console_ns.models[TrialWorkflowResponse.__name__])
    @get_app_model_with_trial(None)
-    @marshal_with(workflow_model)
    def get(self, app_model):
        """Get workflow detail"""
        if not app_model.workflow_id:
            raise AppUnavailableError()

        workflow = db.session.get(Workflow, app_model.workflow_id)
-        return workflow
+        if workflow is None:
+            raise AppUnavailableError()
+
+        return dump_response(TrialWorkflowResponse, workflow)


 class DatasetListApi(Resource):
    @console_ns.doc(params=query_params_from_model(TrialDatasetListQuery))
-    @console_ns.response(200, "Success", dataset_list_model)
+    @console_ns.response(200, "Success", console_ns.models[TrialDatasetListResponse.__name__])
    @get_app_model_with_trial(None)
    def get(self, app_model):
        page = request.args.get("page", default=1, type=int)
@@ -630,26 +828,12 @@ class DatasetListApi(Resource):
        else:
            raise NeedAddIdsError()

-        data = cast(list[dict[str, Any]], marshal(datasets, dataset_fields))
-
-        response = {"data": data, "has_more": len(datasets) == limit, "limit": limit, "total": total, "page": page}
-        return response
+        response = {"data": datasets, "has_more": len(datasets) == limit, "limit": limit, "total": total, "page": page}
+        return dump_response(TrialDatasetListResponse, response)


 console_ns.add_resource(TrialChatApi, "/trial-apps/<uuid:app_id>/chat-messages", endpoint="trial_app_chat_completion")

-console_ns.add_resource(
-    TrialAppFileUploadApi,
-    "/trial-apps/<uuid:app_id>/files/upload",
-    endpoint="trial_app_file_upload",
-)
-
-console_ns.add_resource(
-    TrialAppRemoteFileUploadApi,
-    "/trial-apps/<uuid:app_id>/remote-files/upload",
-    endpoint="trial_app_remote_file_upload",
-)
-
 console_ns.add_resource(
    TrialMessageSuggestedQuestionApi,
    "/trial-apps/<uuid:app_id>/messages/<uuid:message_id>/suggested-questions",
@@ -1,9 +1,10 @@
 import logging

+from sqlalchemy.orm import Session
 from werkzeug.exceptions import InternalServerError

 from controllers.common.controller_schemas import WorkflowRunPayload
-from controllers.common.fields import GeneratedAppResponse, SimpleResultResponse
+from controllers.common.fields import SimpleResultResponse
 from controllers.common.schema import register_response_schema_models, register_schema_model
 from controllers.console.app.error import (
    CompletionRequestError,
@@ -11,6 +12,7 @@ from controllers.console.app.error import (
    ProviderNotInitializeError,
    ProviderQuotaExceededError,
 )
+from controllers.console.app.wraps import with_session
 from controllers.console.explore.error import NotWorkflowAppError
 from controllers.console.explore.wraps import InstalledAppResource
 from controllers.console.wraps import with_current_user
@@ -36,15 +38,16 @@ from .. import console_ns
 logger = logging.getLogger(__name__)

 register_schema_model(console_ns, WorkflowRunPayload)
-register_response_schema_models(console_ns, GeneratedAppResponse, SimpleResultResponse)
+register_response_schema_models(console_ns, SimpleResultResponse)


@console_ns.route("/installed-apps/<uuid:installed_app_id>/workflows/run")
 class InstalledAppWorkflowRunApi(InstalledAppResource):
    @console_ns.expect(console_ns.models[WorkflowRunPayload.__name__])
-    @console_ns.response(200, "Success", console_ns.models[GeneratedAppResponse.__name__])
+    @console_ns.response(200, "Success")
    @with_current_user
-    def post(self, current_user: Account, installed_app: InstalledApp):
+    @with_session
+    def post(self, session: Session, current_user: Account, installed_app: InstalledApp):
        """
        Run workflow
        """
@@ -59,9 +62,15 @@ class InstalledAppWorkflowRunApi(InstalledAppResource):
        args = payload.model_dump(exclude_none=True)
        try:
            response = AppGenerateService.generate(
-                app_model=app_model, user=current_user, args=args, invoke_from=InvokeFrom.EXPLORE, streaming=True
+                session=session,
+                app_model=app_model,
+                user=current_user,
+                args=args,
+                invoke_from=InvokeFrom.EXPLORE,
+                streaming=True,
            )

+            # response-contract:ignore compact_generate_response
            return helper.compact_generate_response(response)
        except ProviderTokenNotInitError as ex:
            raise ProviderNotInitializeError(ex.description)
@@ -101,4 +110,4 @@ class InstalledAppWorkflowTaskStopApi(InstalledAppResource):
        # New graph engine command channel mechanism
        GraphEngineManager(redis_client).send_stop_command(task_id)

-        return {"result": "success"}
+        return SimpleResultResponse(result="success").model_dump(mode="json")
@@ -4,18 +4,18 @@ from uuid import UUID

 from flask import request
 from flask_restx import Resource
-from pydantic import BaseModel, Field, TypeAdapter, field_validator
+from pydantic import BaseModel, Field, RootModel, field_validator

 from constants import HIDDEN_VALUE
 from extensions.ext_database import db
 from fields.base import ResponseModel
-from libs.helper import to_timestamp
+from libs.helper import dump_response, to_timestamp
 from libs.login import login_required
 from models.api_based_extension import APIBasedExtension
 from services.api_based_extension_service import APIBasedExtensionService
 from services.code_based_extension_service import CodeBasedExtensionService

-from ..common.schema import DEFAULT_REF_TEMPLATE_OPENAPI_3_0, query_params_from_model, register_schema_models
+from ..common.schema import query_params_from_model, register_response_schema_models, register_schema_models
 from . import console_ns
 from .wraps import account_initialization_required, setup_required, with_current_tenant_id

@@ -61,36 +61,21 @@ class APIBasedExtensionResponse(ResponseModel):
        return to_timestamp(value)


+class APIBasedExtensionListResponse(RootModel[list[APIBasedExtensionResponse]]):
+    pass
+
+
 register_schema_models(
    console_ns,
    CodeBasedExtensionQuery,
    APIBasedExtensionPayload,
+)
+register_response_schema_models(
+    console_ns,
    CodeBasedExtensionResponse,
    APIBasedExtensionResponse,
+    APIBasedExtensionListResponse,
 )
-console_ns.schema_model(
-    "APIBasedExtensionListResponse",
-    TypeAdapter(list[APIBasedExtensionResponse]).json_schema(ref_template=DEFAULT_REF_TEMPLATE_OPENAPI_3_0),
-)
-
-
-def _serialize_api_based_extension(extension: APIBasedExtension) -> dict[str, Any]:
-    return APIBasedExtensionResponse.model_validate(extension, from_attributes=True).model_dump(mode="json")
-
-
-def _serialize_saved_api_based_extension(extension: APIBasedExtension, api_key: str) -> dict[str, Any]:
-    """Serialize a saved extension with the plaintext key used for response masking only.
-
-    APIBasedExtensionService.save mutates the ORM object to hold the encrypted token before returning it. The response
-    contract, however, should match list/detail responses, where api_key is masked from the decrypted token.
-    """
-    return APIBasedExtensionResponse(
-        id=extension.id,
-        name=extension.name,
-        api_endpoint=extension.api_endpoint,
-        api_key=api_key,
-        created_at=to_timestamp(extension.created_at),
-    ).model_dump(mode="json")


@console_ns.route("/code-based-extension")
@@ -119,16 +104,16 @@ class CodeBasedExtensionAPI(Resource):
 class APIBasedExtensionAPI(Resource):
    @console_ns.doc("get_api_based_extensions")
    @console_ns.doc(description="Get all API-based extensions for current tenant")
-    @console_ns.response(200, "Success", console_ns.models["APIBasedExtensionListResponse"])
+    @console_ns.response(200, "Success", console_ns.models[APIBasedExtensionListResponse.__name__])
    @setup_required
    @login_required
    @account_initialization_required
    @with_current_tenant_id
    def get(self, current_tenant_id: str):
-        return [
-            _serialize_api_based_extension(extension)
-            for extension in APIBasedExtensionService.get_all_by_tenant_id(db.session(), current_tenant_id)
-        ]
+        return dump_response(
+            APIBasedExtensionListResponse,
+            APIBasedExtensionService.get_all_by_tenant_id(db.session(), current_tenant_id),
+        )

    @console_ns.doc("create_api_based_extension")
    @console_ns.doc(description="Create a new API-based extension")
@@ -148,12 +133,14 @@ class APIBasedExtensionAPI(Resource):
            api_key=payload.api_key,
        )

-        return (
-            _serialize_saved_api_based_extension(
-                APIBasedExtensionService.save(db.session(), extension_data), payload.api_key
-            ),
-            201,
-        )
+        extension = APIBasedExtensionService.save(db.session(), extension_data)
+        return APIBasedExtensionResponse(
+            id=extension.id,
+            name=extension.name,
+            api_endpoint=extension.api_endpoint,
+            api_key=payload.api_key,
+            created_at=to_timestamp(extension.created_at),
+        ).model_dump(mode="json"), 201


@console_ns.route("/api-based-extension/<uuid:id>")
@@ -169,8 +156,9 @@ class APIBasedExtensionDetailAPI(Resource):
    def get(self, current_tenant_id: str, id: UUID):
        api_based_extension_id = str(id)

-        return _serialize_api_based_extension(
-            APIBasedExtensionService.get_with_tenant_id(db.session(), current_tenant_id, api_based_extension_id)
+        return dump_response(
+            APIBasedExtensionResponse,
+            APIBasedExtensionService.get_with_tenant_id(db.session(), current_tenant_id, api_based_extension_id),
        )

    @console_ns.doc("update_api_based_extension")
@@ -199,10 +187,14 @@ class APIBasedExtensionDetailAPI(Resource):
            extension_data_from_db.api_key = payload.api_key
            api_key_for_response = payload.api_key

-        return _serialize_saved_api_based_extension(
-            APIBasedExtensionService.save(db.session(), extension_data_from_db),
-            api_key_for_response,
-        )
+        APIBasedExtensionService.save(db.session(), extension_data_from_db)
+        return APIBasedExtensionResponse(
+            id=extension_data_from_db.id,
+            name=extension_data_from_db.name,
+            api_endpoint=extension_data_from_db.api_endpoint,
+            api_key=api_key_for_response,
+            created_at=to_timestamp(extension_data_from_db.created_at),
+        ).model_dump(mode="json")

    @console_ns.doc("delete_api_based_extension")
    @console_ns.doc(description="Delete API-based extension")
@@ -28,7 +28,7 @@ from controllers.console.wraps import (
 from extensions.ext_database import db
 from fields.file_fields import FileResponse, UploadConfig
 from libs.login import login_required
-from models import Account, UploadFile
+from models import Account
 from services.file_service import FileService

 from . import console_ns
@@ -38,7 +38,7 @@ register_response_schema_models(console_ns, AllowedExtensionsResponse, TextConte

 PREVIEW_WORDS_LIMIT = 3000

-FILE_UPLOAD_PARAMS = {
+_FILE_UPLOAD_PARAMS = {
    "file": {
        "description": "File to upload",
        "in": "formData",
@@ -55,43 +55,6 @@ FILE_UPLOAD_PARAMS = {
 }


-def upload_file_from_request(*, current_user: Account, resource_tenant_id: str | None = None) -> UploadFile:
-    """Validate the multipart request and persist the file under the requested resource tenant."""
-    source_str = request.form.get("source")
-    source: Literal["datasets"] | None = "datasets" if source_str == "datasets" else None
-
-    if "file" not in request.files:
-        raise NoFileUploadedError()
-
-    if len(request.files) > 1:
-        raise TooManyFilesError()
-    file = request.files["file"]
-
-    if not file.filename:
-        raise FilenameNotExistsError
-    if source == "datasets" and not current_user.is_dataset_editor:
-        raise Forbidden()
-
-    if source not in ("datasets", None):
-        source = None
-
-    try:
-        return FileService(db.engine).upload_file(
-            filename=file.filename,
-            content=file.stream.read(),
-            mimetype=file.mimetype,
-            user=current_user,
-            tenant_id=resource_tenant_id,
-            source=source,
-        )
-    except services.errors.file.FileTooLargeError as file_too_large_error:
-        raise FileTooLargeError(file_too_large_error.description)
-    except services.errors.file.UnsupportedFileTypeError:
-        raise UnsupportedFileTypeError()
-    except services.errors.file.BlockedFileExtensionError as blocked_extension_error:
-        raise BlockedFileExtensionError(blocked_extension_error.description)
-
-
@console_ns.route("/files/upload")
 class FileApi(Resource):
    @setup_required
@@ -117,11 +80,42 @@ class FileApi(Resource):
    @login_required
    @account_initialization_required
    @cloud_edition_billing_resource_check("documents")
-    @console_ns.doc(consumes=["multipart/form-data"], params=FILE_UPLOAD_PARAMS)
+    @console_ns.doc(consumes=["multipart/form-data"], params=_FILE_UPLOAD_PARAMS)
    @console_ns.response(201, "File uploaded successfully", console_ns.models[FileResponse.__name__])
    @with_current_user
    def post(self, current_user: Account):
-        upload_file = upload_file_from_request(current_user=current_user)
+        source_str = request.form.get("source")
+        source: Literal["datasets"] | None = "datasets" if source_str == "datasets" else None
+
+        if "file" not in request.files:
+            raise NoFileUploadedError()
+
+        if len(request.files) > 1:
+            raise TooManyFilesError()
+        file = request.files["file"]
+
+        if not file.filename:
+            raise FilenameNotExistsError
+        if source == "datasets" and not current_user.is_dataset_editor:
+            raise Forbidden()
+
+        if source not in ("datasets", None):
+            source = None
+
+        try:
+            upload_file = FileService(db.engine).upload_file(
+                filename=file.filename,
+                content=file.stream.read(),
+                mimetype=file.mimetype,
+                user=current_user,
+                source=source,
+            )
+        except services.errors.file.FileTooLargeError as file_too_large_error:
+            raise FileTooLargeError(file_too_large_error.description)
+        except services.errors.file.UnsupportedFileTypeError:
+            raise UnsupportedFileTypeError()
+        except services.errors.file.BlockedFileExtensionError as blocked_extension_error:
+            raise BlockedFileExtensionError(blocked_extension_error.description)

        response = FileResponse.model_validate(upload_file, from_attributes=True)
        return response.model_dump(mode="json"), 201
@@ -46,61 +46,6 @@ class GetRemoteFileInfo(Resource):
        ).model_dump(mode="json")


-def upload_remote_file_from_request(
-    *,
-    current_user: Account,
-    resource_tenant_id: str | None = None,
-) -> FileWithSignedUrl:
-    """Validate the JSON request, fetch its remote file, and persist it under the requested tenant."""
-    payload = RemoteFileUploadPayload.model_validate(console_ns.payload)
-    url = payload.url
-
-    # Try to fetch remote file metadata/content first
-    try:
-        resp = remote_fetcher.make_request("HEAD", url=url)
-        if resp.status_code != httpx.codes.OK:
-            resp = remote_fetcher.make_request("GET", url=url, timeout=3, follow_redirects=True)
-        if resp.status_code != httpx.codes.OK:
-            # Normalize into a user-friendly error message expected by tests
-            raise RemoteFileUploadError(f"Failed to fetch file from {url}: {resp.text}")
-    except httpx.RequestError as e:
-        raise RemoteFileUploadError(f"Failed to fetch file from {url}: {str(e)}")
-
-    file_info = helpers.guess_file_info_from_response(resp)
-
-    # Enforce file size limit with 400 (Bad Request) per tests' expectation
-    if not FileService.is_file_size_within_limit(extension=file_info.extension, file_size=file_info.size):
-        raise FileTooLargeError()
-
-    # Load content if needed
-    content = resp.content if resp.request.method == "GET" else remote_fetcher.make_request("GET", url).content
-
-    try:
-        upload_file = FileService(db.engine).upload_file(
-            filename=file_info.filename,
-            content=content,
-            mimetype=file_info.mimetype,
-            user=current_user,
-            tenant_id=resource_tenant_id,
-            source_url=url,
-        )
-    except services.errors.file.FileTooLargeError as file_too_large_error:
-        raise FileTooLargeError(file_too_large_error.description)
-    except services.errors.file.UnsupportedFileTypeError:
-        raise UnsupportedFileTypeError()
-
-    return FileWithSignedUrl(
-        id=upload_file.id,
-        name=upload_file.name,
-        size=upload_file.size,
-        extension=upload_file.extension,
-        url=file_helpers.get_signed_file_url(upload_file_id=upload_file.id),
-        mime_type=upload_file.mime_type,
-        created_by=upload_file.created_by,
-        created_at=int(upload_file.created_at.timestamp()),
-    )
-
-
@console_ns.route("/remote-files/upload")
 class RemoteFileUpload(Resource):
    @console_ns.expect(console_ns.models[RemoteFileUploadPayload.__name__])
@@ -108,8 +53,53 @@ class RemoteFileUpload(Resource):
    @login_required
    @with_current_user
    def post(self, current_user: Account):
-        remote_file = upload_remote_file_from_request(current_user=current_user)
+        payload = RemoteFileUploadPayload.model_validate(console_ns.payload)
+        url = payload.url
+
+        # Try to fetch remote file metadata/content first
+        try:
+            resp = remote_fetcher.make_request("HEAD", url=url)
+            if resp.status_code != httpx.codes.OK:
+                resp = remote_fetcher.make_request("GET", url=url, timeout=3, follow_redirects=True)
+            if resp.status_code != httpx.codes.OK:
+                # Normalize into a user-friendly error message expected by tests
+                raise RemoteFileUploadError(f"Failed to fetch file from {url}: {resp.text}")
+        except httpx.RequestError as e:
+            raise RemoteFileUploadError(f"Failed to fetch file from {url}: {str(e)}")
+
+        file_info = helpers.guess_file_info_from_response(resp)
+
+        # Enforce file size limit with 400 (Bad Request) per tests' expectation
+        if not FileService.is_file_size_within_limit(extension=file_info.extension, file_size=file_info.size):
+            raise FileTooLargeError()
+
+        # Load content if needed
+        content = resp.content if resp.request.method == "GET" else remote_fetcher.make_request("GET", url).content
+
+        try:
+            upload_file = FileService(db.engine).upload_file(
+                filename=file_info.filename,
+                content=content,
+                mimetype=file_info.mimetype,
+                user=current_user,
+                source_url=url,
+            )
+        except services.errors.file.FileTooLargeError as file_too_large_error:
+            raise FileTooLargeError(file_too_large_error.description)
+        except services.errors.file.UnsupportedFileTypeError:
+            raise UnsupportedFileTypeError()
+
+        # Success: return created resource with 201 status
        return (
-            remote_file.model_dump(mode="json"),
+            FileWithSignedUrl(
+                id=upload_file.id,
+                name=upload_file.name,
+                size=upload_file.size,
+                extension=upload_file.extension,
+                url=file_helpers.get_signed_file_url(upload_file_id=upload_file.id),
+                mime_type=upload_file.mime_type,
+                created_by=upload_file.created_by,
+                created_at=int(upload_file.created_at.timestamp()),
+            ).model_dump(mode="json"),
            201,
        )
--- a/Show More
+++ b/Show More