Compare commits

..
Author SHA1 Message Date
GareArc bc68e02711 feat(openapi): distinguish expired OAuth bearer from invalid token
Previously an expired OAuth bearer and an unknown/invalid one both
surfaced as an indistinguishable generic 401 (and an invalid token
actually leaked a 500), so a client could not tell "session expired,
re-authenticate" apart from "never authenticated."

The resolver now raises a distinct TokenExpiredError for expired DB
rows and records a separate `expired` negative-cache marker, so a
retry within the negative-cache TTL still reports expiry instead of
collapsing into a generic miss. The auth pipeline maps the two domain
errors to unified OpenApiError responses: SessionExpired (code
`token_expired`) and InvalidBearer (code `unauthorized`), both 401.
This also fixes the latent 500 on invalid bearers.

The `token_expired` code is synced through the contract codegen into
the generated types/zod, and the difyctl error mapper branches the
401 on it. The CLI `expired_token` taxonomy member (RFC 8628
device-flow code expiry) is merged into `token_expired`; the RFC 8628
wire value is unchanged.

Closes WTA-1062
2026-06-28 21:45:13 -07:00
2156 changed files with 35792 additions and 112499 deletions
@@ -5,7 +5,7 @@ description: Write, update, or review Dify end-to-end tests under `e2e/` that us
# Dify E2E Cucumber + Playwright
Use this skill for Dify's repository-level E2E suite in `e2e/`. Use [`e2e/AGENTS.md`](../../../e2e/AGENTS.md) as the canonical package guide for local architecture and conventions, then read any feature-scoped `AGENTS.md` that owns the target area. Apply Playwright/Cucumber best practices only where they fit the current suite.
Use this skill for Dify's repository-level E2E suite in `e2e/`. Use [`e2e/AGENTS.md`](../../../e2e/AGENTS.md) as the canonical guide for local architecture and conventions, then apply Playwright/Cucumber best practices only where they fit the current suite.
## Scope
@@ -25,8 +25,6 @@ Use this skill for Dify's repository-level E2E suite in `e2e/`. Use [`e2e/AGENTS
4. Read [`references/cucumber-best-practices.md`](references/cucumber-best-practices.md) only when scenario wording, step granularity, tags, or expression design are involved.
5. Re-check official Playwright or Cucumber docs with the available documentation tools before introducing a new framework pattern.
Keep this skill focused on Cucumber, Playwright, and package-level E2E guidance. Put feature-specific conventions in the owning feature's `AGENTS.md` instead of adding them here.
## Local Rules
- `e2e/` uses Cucumber for scenarios and Playwright as the browser layer.
@@ -45,7 +43,6 @@ Keep this skill focused on Cucumber, Playwright, and package-level E2E guidance.
- Inspect the target feature area.
- Reuse an existing step when wording and behavior already match.
- Add a new step only for a genuinely new user action or assertion.
- Before adding several similar steps, scan the target capability for an existing domain noun that can be parameterized without hiding behavior.
- Keep edits close to the current capability folder unless the step is broadly reusable.
2. Write behavior-first scenarios.
- Describe user-observable behavior, not DOM mechanics.
@@ -54,19 +51,15 @@ Keep this skill focused on Cucumber, Playwright, and package-level E2E guidance.
3. Write step definitions in the local style.
- Keep one step to one user-visible action or one assertion.
- Prefer Cucumber Expressions such as `{string}` and `{int}`.
- Use a bounded regex only when the accepted values are a small explicit domain set and Cucumber Expressions would make the Gherkin less natural.
- Do not create one-off steps for each case variant when the same domain action or outcome applies to named surfaces, modes, or resources.
- Scope locators to stable containers when the page has repeated elements.
- Avoid page-object layers or extra helper abstractions unless repeated complexity clearly justifies them.
4. Use Playwright in the local style.
- Prefer user-facing locators: `getByRole`, `getByLabel`, `getByPlaceholder`, `getByText`, then `getByTestId` for explicit contracts.
- Use web-first `expect(...)` assertions.
- Do not use `waitForTimeout`, manual polling, or raw visibility checks when a locator action or retrying assertion already expresses the behavior.
- Use `expect.poll` for API persistence, backend eventual consistency, captured browser events, or other non-DOM state; prefer locator assertions for DOM readiness and visible UI state.
- If a product element has real user-facing semantics but no accessible name, prefer fixing that accessible contract over adding a test id.
5. Validate narrowly.
- Run the narrowest tagged scenario or flow that exercises the change.
- Run `vpr lint --fix --quiet` from the repository root and `pnpm -C e2e type-check`.
- Run `pnpm -C e2e check`.
- Broaden verification only when the change affects hooks, tags, setup, or shared step semantics.
## Review Checklist
@@ -39,7 +39,6 @@ Prefer reuse when:
- the user action is genuinely the same
- the expected outcome is genuinely the same
- the wording stays natural across features
- the parameter is a real product domain value such as a named surface, mode, resource, or status
Write a new step when:
@@ -47,8 +46,6 @@ Write a new step when:
- reusing the old wording would make the scenario misleading
- a supposedly generic step would become an implementation-detail wrapper
Do not optimize for a low step count by making vague steps. Optimize for a small set of truthful, domain-owned steps.
### 4. Prefer Cucumber Expressions
Use Cucumber Expressions for parameters unless regex is clearly necessary.
@@ -62,8 +59,6 @@ Common examples:
Keep expressions readable. If a step needs complicated parsing logic, first ask whether the scenario wording should be simpler.
Use regex for a bounded natural-language alternative only when it keeps Gherkin readable, for example `/(Web app|Backend service API)/`. Avoid broad regexes that accept unowned language.
### 5. Keep step definitions thin and meaningful
Step definitions are glue between Gherkin and automation, not a second abstraction language.
@@ -41,7 +41,6 @@ Also remember:
- repeated content usually needs scoping to a stable container
- exact text matching is often too brittle when role/name or label already exists
- `getByTestId` is acceptable when semantics are weak but the contract is intentional
- when a real UI region, card, status, or icon lacks an accessible name, prefer adding that semantic contract in product code before falling back to `getByTestId`
### 3. Use web-first assertions
@@ -63,8 +62,6 @@ Avoid:
If a condition genuinely needs custom retry logic, use Playwright's polling/assertion tools deliberately and keep that choice local and explicit.
Use `expect.poll` for non-DOM truth such as API state, backend eventual consistency, generated resources, or captured browser events. For DOM state, use locator assertions so Playwright can apply actionability and web-first retry semantics.
### 4. Let actions wait for actionability
Locator actions already wait for the element to be actionable. Do not preface every click/fill with extra timing logic unless the action needs a specific visible/ready assertion for clarity.
@@ -40,13 +40,9 @@ Flag:
- JavaScript conditional class logic for visual states that the Dify UI/Base UI primitive already exposes through `data-*` attributes or CSS variables.
- Controlled props added when uncontrolled DOM state or CSS variables would be enough.
- Thin wrappers that rename Base UI parts without adding semantics.
- Generic Base UI selection primitives wrapped without preserving their value generics, such as `Select.Root<Value, Multiple>`, `RadioGroup<Value>`, or `Radio.Root<Value>`.
- Shared select/radio option components that type selected values as `string` while callers pass enums, unions, booleans, numbers, objects, or nullable placeholder values.
Prefer Base UI/Dify UI data attributes and CSS variables for visual state: `data-open`, `data-checked`, `data-disabled`, `data-highlighted`, `data-popup-open`, `group-data-*`, `peer-data-*`, `has-[:focus-visible]`, and primitive CSS variables such as anchor width or transform origin. Use JS conditional classes for product/business state that the primitive does not expose.
For non-string `Select` and `RadioGroup` values, prefer explicit domain generics at the root and at child value carriers. JSX children do not inherit the parent generic, so `RadioGroup<PromptMode>` should compose with `Radio<PromptMode>`, `RadioRoot<PromptMode>`, or option values from a typed collection. For `Select`, prefer the Base UI `items` collection pattern for typed value-to-label rendering, and flag string coercion helpers used only to recover display labels.
## Forms
Flag:
@@ -25,8 +25,6 @@ Use this as the component decision guide for Dify web. Existing code is referenc
- Search before adding UI, hooks, helpers, query utilities, or styling patterns. Reuse existing base components, feature components, hooks, utilities, and design styles when they fit.
- Follow Dify's CSS-first Tailwind v4 contract from `packages/dify-ui/README.md` and `packages/dify-ui/AGENTS.md`. Prefer design-system tokens, utilities, and radius mappings over generic Tailwind choices.
- Group feature code by workflow, route, or ownership area with route-aligned names: components, hooks, local types, query helpers, atoms, constants, tests, and small utilities should live near the code that changes with them.
- For each feature module, keep a module-local `README.md` as a boundary note. Start with the module name, a brief one-sentence description, then split dependencies into `Internal Modules` and `External Modules` sections; keep both sections and write `None.` when one category is empty. `Internal Modules` lists modules inside the same overall feature using paths from that feature root, such as `shared/domain/runtime-status`; `External Modules` lists project modules outside the feature using paths from the web root without a `web/` prefix, such as `app/components/base/skeleton`. Omit npm packages, workspace package dependencies, and whitelisted plumbing modules. Do not copy caller-relative import paths into the README.
- Module README whitelist: `@/service/client`, `@/next/*`.
- Keep source/default selection, validation, dirty checks, and payload shaping close to the workflow that owns submit behavior. Do not hide flow-specific priority order, fallback behavior, or submit semantics in generic utilities.
- Prefer direct conditionals for small branch-specific decisions, especially form source selection and request payload assembly.
- Loading states for page sections, cards, lists, tables, forms, and drawers should be skeletons scoped to the content being loaded. Use spinners only for small inline busy indicators.
@@ -69,7 +67,6 @@ Use this as the component decision guide for Dify web. Existing code is referenc
- Avoid barrel files that only re-export secondary owners. `index.tsx` is acceptable for a route/tab entry component; import header controls, switches, sections, and row owners from their concrete owner files.
- Type simple one-off props inline. Use a named `Props` type only when reused, exported, complex, or clearer.
- Use API-generated or API-returned types at component boundaries. Keep small UI conversion helpers and one-off UI extensions beside the component that needs them.
- Preserve domain value types for selection components. Do not widen enum, union, boolean, numeric, object, or nullable select/radio values to `string`; keep wrappers and option value carriers typed from their feature option collection.
- Avoid `common.tsx` buckets for shared UI. Use a feature-local `components/` folder with concrete filenames that describe the shared role.
- Do not create type aliases that only rename another type. Use aliases only for real UI concepts, refinements, or reusable local contracts.
- Name values by their domain role and backend API contract, especially persistent IDs and route params. Normalize framework or route params at the boundary.
-3
View File
@@ -26,9 +26,6 @@
/cli/ @GareArc
/.github/workflows/cli-tests.yml @GareArc
# E2E
/e2e/ @lyzno1
# Backend (default owner, more specific rules below will override)
/api/ @QuantumGhost
-6
View File
@@ -7,9 +7,3 @@ web:
- 'pnpm-lock.yaml'
- 'pnpm-workspace.yaml'
- '.nvmrc'
e2e:
- changed-files:
- any-glob-to-any-file:
- 'e2e/**'
- '.github/workflows/web-e2e.yml'
+2 -14
View File
@@ -51,18 +51,6 @@ jobs:
with:
files: |
api/**
- name: Check frontend contract inputs
if: github.event_name != 'merge_group'
id: frontend-contract-changes
uses: tj-actions/changed-files@9426d40962ed5378910ee2e21d5f8c6fcbf2dd96 # v47.0.6
with:
files: |
api/**
packages/contracts/openapi-ts.api.config.ts
packages/contracts/package.json
packages/contracts/openapi/**
pnpm-lock.yaml
pnpm-workspace.yaml
- name: Check dify-agent inputs
if: github.event_name != 'merge_group'
id: dify-agent-changes
@@ -155,8 +143,8 @@ jobs:
uv run dev/generate_swagger_markdown_docs.py --swagger-dir ../packages/contracts/openapi --markdown-dir openapi/markdown --keep-swagger-json
- name: Generate frontend contracts
if: github.event_name != 'merge_group' && steps.frontend-contract-changes.outputs.any_changed == 'true'
run: pnpm --dir packages/contracts gen-api-contract
if: github.event_name != 'merge_group' && steps.api-changes.outputs.any_changed == 'true'
run: pnpm --dir packages/contracts gen-api-contract-from-openapi
- name: ESLint autofix
if: github.event_name != 'merge_group' && steps.web-changes.outputs.any_changed == 'true'
+2 -1
View File
@@ -6,6 +6,8 @@ on:
merge_group:
branches: ["main"]
types: [checks_requested]
push:
branches: ["main"]
permissions:
actions: write
@@ -322,7 +324,6 @@ jobs:
- check-changes
if: needs.pre_job.outputs.should_skip != 'true' && needs.check-changes.outputs.e2e-changed == 'true'
uses: ./.github/workflows/web-e2e.yml
secrets: inherit
web-e2e-skip:
name: Skip Web Full-Stack E2E
-54
View File
@@ -1,54 +0,0 @@
name: Post-Merge Checks
on:
push:
branches: ["main"]
permissions:
contents: read
concurrency:
group: post-merge-${{ github.sha }}
cancel-in-progress: false
jobs:
check-changes:
name: Check Changed Files
runs-on: depot-ubuntu-24.04
outputs:
external-e2e-changed: ${{ steps.changes.outputs.external_e2e }}
steps:
- uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
- uses: dorny/paths-filter@fbd0ab8f3e69293af611ebaee6363fc25e6d187d # v4.0.1
id: changes
with:
filters: |
external_e2e:
- 'e2e/features/agent-v2/**'
- 'e2e/scripts/**'
- 'e2e/support/**'
- '.github/workflows/post-merge.yml'
- '.github/workflows/web-e2e.yml'
- '.github/actions/setup-web/**'
- 'dify-agent/**'
- 'api/clients/agent_backend/**'
- 'api/core/app/apps/agent_app/**'
- 'api/core/workflow/nodes/agent_v2/**'
- 'api/controllers/console/agent/**'
- 'api/services/agent/**'
- 'api/core/plugin/**'
- 'api/services/plugin/**'
- 'api/core/tools/**'
- 'api/services/tools/**'
- 'web/features/agent-v2/**'
- 'web/app/(commonLayout)/roster/**'
- 'web/app/components/workflow/nodes/agent-v2/**'
external-e2e:
name: External Runtime E2E
needs: check-changes
if: needs.check-changes.outputs.external-e2e-changed == 'true'
uses: ./.github/workflows/web-e2e.yml
with:
run-external-runtime: true
secrets: inherit
-15
View File
@@ -22,7 +22,6 @@ jobs:
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
with:
persist-credentials: false
fetch-depth: 0
- name: Check changed files
id: changed-files
@@ -30,8 +29,6 @@ jobs:
with:
files: |
api/**
scripts/check_no_new_getattr.py
scripts/ast_grep_rules/no_new_getattr.yml
.github/workflows/style.yml
- name: Setup UV and Python
@@ -54,18 +51,6 @@ jobs:
if: steps.changed-files.outputs.any_changed == 'true'
run: uv run --project api --dev python api/dev/lint_response_contracts.py --fail-on-mismatch
- name: Fetch merge target ref for getattr guard
if: steps.changed-files.outputs.any_changed == 'true'
run: git fetch --no-tags --depth=1 origin +refs/heads/main:refs/remotes/origin/main
- name: Bind merge target branch for getattr guard
if: steps.changed-files.outputs.any_changed == 'true'
run: git show-ref --verify --quiet refs/heads/main || git branch main origin/main
- name: Run No New Getattr Guard
if: steps.changed-files.outputs.any_changed == 'true'
run: uv run --project api python scripts/check_no_new_getattr.py --mode ci --merge-target main
- name: Run Type Checks
if: steps.changed-files.outputs.any_changed == 'true'
env:
+2 -49
View File
@@ -2,11 +2,6 @@ name: Web Full-Stack E2E
on:
workflow_call:
inputs:
run-external-runtime:
required: false
type: boolean
default: false
permissions:
contents: read
@@ -37,9 +32,7 @@ jobs:
with:
enable-cache: true
python-version: "3.12"
cache-dependency-glob: |
api/uv.lock
dify-agent/uv.lock
cache-dependency-glob: api/uv.lock
- name: Install API dependencies
run: uv sync --project api --dev
@@ -58,52 +51,12 @@ jobs:
E2E_INIT_PASSWORD: E2eInit12345
run: vp run e2e:full
- name: Run external runtime E2E tests
if: ${{ inputs.run-external-runtime }}
working-directory: ./e2e
env:
E2E_ADMIN_EMAIL: e2e-admin@example.com
E2E_ADMIN_NAME: E2E Admin
E2E_ADMIN_PASSWORD: E2eAdmin12345
E2E_AGENT_DECISION_MODEL_NAME: ${{ vars.E2E_AGENT_DECISION_MODEL_NAME || 'gpt-5.5' }}
E2E_AGENT_DECISION_MODEL_PROVIDER: ${{ vars.E2E_AGENT_DECISION_MODEL_PROVIDER || 'openai' }}
E2E_AGENT_DECISION_MODEL_TYPE: ${{ vars.E2E_AGENT_DECISION_MODEL_TYPE || 'llm' }}
E2E_EXTERNAL_RUNTIME_SEED_SPECS: ${{ vars.E2E_EXTERNAL_RUNTIME_SEED_SPECS }}
E2E_EXTERNAL_RUNTIME_TAGS: ${{ vars.E2E_EXTERNAL_RUNTIME_TAGS }}
E2E_FORCE_WEB_BUILD: "1"
E2E_INIT_PASSWORD: E2eInit12345
E2E_MARKETPLACE_API_URL: ${{ vars.E2E_MARKETPLACE_API_URL }}
E2E_MARKETPLACE_PLUGIN_IDS: ${{ vars.E2E_MARKETPLACE_PLUGIN_IDS }}
E2E_MARKETPLACE_PLUGIN_UNIQUE_IDENTIFIERS: ${{ vars.E2E_MARKETPLACE_PLUGIN_UNIQUE_IDENTIFIERS }}
E2E_MODEL_PROVIDER_CREDENTIALS_JSON: ${{ secrets.E2E_MODEL_PROVIDER_CREDENTIALS_JSON }}
E2E_START_AGENT_BACKEND: ${{ vars.E2E_START_AGENT_BACKEND || '1' }}
E2E_STABLE_MODEL_NAME: ${{ vars.E2E_STABLE_MODEL_NAME || 'gpt-5-nano' }}
E2E_STABLE_MODEL_PROVIDER: ${{ vars.E2E_STABLE_MODEL_PROVIDER || 'openai' }}
E2E_STABLE_MODEL_TYPE: ${{ vars.E2E_STABLE_MODEL_TYPE || 'llm' }}
run: |
if [[ -z "${E2E_MODEL_PROVIDER_CREDENTIALS_JSON}" ]]; then
echo "E2E_MODEL_PROVIDER_CREDENTIALS_JSON is required for external runtime E2E." >&2
exit 1
fi
if [[ -d cucumber-report ]]; then
rm -rf cucumber-report-non-external
mv cucumber-report cucumber-report-non-external
fi
trap 'vp run e2e:middleware:down' EXIT
vp run e2e:middleware:up
vp run e2e:external:prepare
vp run e2e:external
- name: Upload Cucumber report
if: ${{ !cancelled() }}
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: cucumber-report
path: |
e2e/cucumber-report
e2e/cucumber-report-non-external
path: e2e/cucumber-report
retention-days: 7
- name: Upload E2E logs
@@ -1,30 +0,0 @@
# HITL timeout semantics implementation report
## What changed
- Updated `api/core/workflow/nodes/human_input/callback.py` so `DifyHITLCallback` now preserves Dify's timeout split at the boundary:
- `HumanInputFormStatus.TIMEOUT` returns the graphon timeout branch via `Expired(selected_handle="__timeout__", ...)`.
- `HumanInputFormStatus.EXPIRED` is treated as an invalid resume state and raises `AssertionError`.
- `HumanInputFormStatus.WAITING` with a past global deadline is treated as an invalid resume state and raises `AssertionError`.
- `HumanInputFormStatus.WAITING` with only the node-level deadline expired still returns the timeout branch.
- Added `created_at` to `HumanInputFormEntity` and `_HumanInputFormEntityImpl` so the callback can compute the global deadline using Dify's shared `HUMAN_INPUT_GLOBAL_TIMEOUT_SECONDS` invariant.
- Kept the submitted and pause flows unchanged.
- Added focused unit coverage in `api/tests/unit_tests/core/workflow/test_human_input_callback.py` for:
- node timeout branch
- global expiration rejection
- waiting-form past node deadline timeout
- waiting-form past global deadline rejection
## Verification
- `uv run --project api pytest -o addopts='' api/tests/unit_tests/core/workflow/test_human_input_callback.py api/tests/unit_tests/core/workflow/nodes/human_input/test_human_input_form_filled_event.py -q`
- `git diff --check`
## Result
- The focused test set is expected to pass with the new `created_at` boundary in place.
- No unrelated files were modified.
## Concerns
- The callback now fails fast on invalid resume states by design. That is intentional, but any caller that previously relied on `EXPIRED` being mapped to the timeout branch will now see an assertion failure instead.
-4
View File
@@ -32,8 +32,6 @@ from clients.agent_backend.factory import create_agent_backend_run_client
from clients.agent_backend.fake_client import FakeAgentBackendRunClient, FakeAgentBackendScenario
from clients.agent_backend.request_builder import (
AGENT_SOUL_PROMPT_LAYER_ID,
DIFY_CONFIG_LAYER_ID,
DIFY_CORE_TOOLS_LAYER_ID,
DIFY_EXECUTION_CONTEXT_LAYER_ID,
DIFY_KNOWLEDGE_BASE_LAYER_ID,
DIFY_PLUGIN_TOOLS_LAYER_ID,
@@ -49,8 +47,6 @@ from clients.agent_backend.request_builder import (
__all__ = [
"AGENT_SOUL_PROMPT_LAYER_ID",
"DIFY_CONFIG_LAYER_ID",
"DIFY_CORE_TOOLS_LAYER_ID",
"DIFY_EXECUTION_CONTEXT_LAYER_ID",
"DIFY_KNOWLEDGE_BASE_LAYER_ID",
"DIFY_PLUGIN_TOOLS_LAYER_ID",
@@ -67,7 +67,6 @@ class AgentBackendRunSucceededInternalEvent(AgentBackendInternalEventBase):
type: Literal[AgentBackendInternalEventType.RUN_SUCCEEDED] = AgentBackendInternalEventType.RUN_SUCCEEDED
output: JsonValue
session_snapshot: CompositorSessionSnapshot
usage: dict[str, JsonValue] | None = None
class AgentBackendDeferredToolCallInternalEvent(AgentBackendInternalEventBase):
@@ -77,7 +76,6 @@ class AgentBackendDeferredToolCallInternalEvent(AgentBackendInternalEventBase):
deferred_tool_call: DeferredToolCallPayload
message: str | None = None
session_snapshot: CompositorSessionSnapshot
usage: dict[str, JsonValue] | None = None
class AgentBackendRunFailedInternalEvent(AgentBackendInternalEventBase):
@@ -142,7 +140,6 @@ class AgentBackendRunEventAdapter:
deferred_tool_call=event.data.deferred_tool_call,
message=_deferred_tool_call_message(event.data.deferred_tool_call),
session_snapshot=event.data.session_snapshot,
usage=_agent_run_usage(event.data.usage),
)
]
return [
@@ -151,7 +148,6 @@ class AgentBackendRunEventAdapter:
source_event_id=event.id,
output=event.data.output,
session_snapshot=event.data.session_snapshot,
usage=_agent_run_usage(event.data.usage),
)
]
case RunFailedEvent():
@@ -188,13 +184,3 @@ def _deferred_tool_call_message(payload: DeferredToolCallPayload) -> str:
return title
return f"Agent backend requested external input via deferred tool '{payload.tool_name}'."
def _agent_run_usage(usage: object | None) -> dict[str, JsonValue] | None:
"""Return JSON-safe usage metadata from optional Agent backend usage."""
if usage is None:
return None
dumped = _EVENT_DATA_ADAPTER.dump_python(usage, mode="json")
if not isinstance(dumped, dict):
return None
return cast(dict[str, JsonValue], dumped)
+54 -135
View File
@@ -20,8 +20,6 @@ from agenton.layers import ExitIntent
from agenton_collections.layers.plain import PLAIN_PROMPT_LAYER_TYPE_ID, PromptLayerConfig
from agenton_collections.layers.pydantic_ai import PYDANTIC_AI_HISTORY_LAYER_TYPE_ID
from dify_agent.layers.ask_human import DIFY_ASK_HUMAN_LAYER_TYPE_ID, DifyAskHumanLayerConfig
from dify_agent.layers.config import DIFY_CONFIG_LAYER_TYPE_ID, DifyConfigLayerConfig
from dify_agent.layers.dify_core_tools import DIFY_CORE_TOOLS_LAYER_TYPE_ID, DifyCoreToolsLayerConfig
from dify_agent.layers.dify_plugin import (
DIFY_PLUGIN_LLM_LAYER_TYPE_ID,
DIFY_PLUGIN_TOOLS_LAYER_TYPE_ID,
@@ -56,10 +54,8 @@ WORKFLOW_NODE_JOB_PROMPT_LAYER_ID = "workflow_node_job_prompt"
WORKFLOW_USER_PROMPT_LAYER_ID = "workflow_user_prompt"
AGENT_APP_USER_PROMPT_LAYER_ID = "agent_app_user_prompt"
DIFY_EXECUTION_CONTEXT_LAYER_ID = "execution_context"
DIFY_CONFIG_LAYER_ID = "config"
DIFY_DRIVE_LAYER_ID = "drive"
DIFY_PLUGIN_TOOLS_LAYER_ID = "tools"
DIFY_CORE_TOOLS_LAYER_ID = "core_tools"
DIFY_KNOWLEDGE_BASE_LAYER_ID = "knowledge"
DIFY_ASK_HUMAN_LAYER_ID = "ask_human"
DIFY_SHELL_LAYER_ID = "shell"
@@ -82,26 +78,11 @@ def _filter_snapshot_to_specs(
return CompositorSessionSnapshot(schema_version=snapshot.schema_version, layers=filtered_layers)
def _shell_layer_deps() -> dict[str, str]:
return {"execution_context": DIFY_EXECUTION_CONTEXT_LAYER_ID}
def _drive_layer_deps() -> dict[str, str]:
return {"shell": DIFY_SHELL_LAYER_ID}
def _config_layer_deps() -> dict[str, str]:
return {"shell": DIFY_SHELL_LAYER_ID}
def _shell_config_with_drive_ref(
shell_config: DifyShellLayerConfig | None,
drive_config: DifyDriveLayerConfig | None,
) -> DifyShellLayerConfig:
config = shell_config or DifyShellLayerConfig()
if drive_config is None:
return config
return config.model_copy(update={"agent_stub_drive_ref": drive_config.drive_ref})
def _shell_layer_deps(*, include_drive: bool) -> dict[str, str]:
deps = {"execution_context": DIFY_EXECUTION_CONTEXT_LAYER_ID}
if include_drive:
deps["drive"] = DIFY_DRIVE_LAYER_ID
return deps
class AgentBackendModelConfig(BaseModel):
@@ -167,11 +148,9 @@ class AgentBackendWorkflowNodeRunInput(BaseModel):
idempotency_key: str | None = None
output: AgentBackendOutputConfig | None = None
tools: DifyPluginToolsLayerConfig | None = None
core_tools: DifyCoreToolsLayerConfig | None = None
knowledge: DifyKnowledgeBaseLayerConfig | None = None
config_layer_config: DifyConfigLayerConfig | None = None
# Drive Skills & Files declaration (dify.drive) — an index the agent pulls
# through the back proxy, never inline content.
# through the back proxy, never inline content; see AGENT_DRIVE_MANIFEST_ENABLED.
drive_config: DifyDriveLayerConfig | None = None
# Human-in-the-loop ask_human deferred tool (dify.ask_human). Present only when
# the Agent Soul configures human involvement; a deferred call ends the run and
@@ -216,11 +195,9 @@ class AgentBackendAgentAppRunInput(BaseModel):
idempotency_key: str | None = None
output: AgentBackendOutputConfig | None = None
tools: DifyPluginToolsLayerConfig | None = None
core_tools: DifyCoreToolsLayerConfig | None = None
knowledge: DifyKnowledgeBaseLayerConfig | None = None
config_layer_config: DifyConfigLayerConfig | None = None
# Drive Skills & Files declaration (dify.drive) — an index the agent pulls
# through the back proxy, never inline content.
# through the back proxy, never inline content; see AGENT_DRIVE_MANIFEST_ENABLED.
drive_config: DifyDriveLayerConfig | None = None
# Human-in-the-loop ask_human deferred tool (dify.ask_human). Present only when
# the Agent Soul configures human involvement (ENG-635).
@@ -254,11 +231,9 @@ class AgentBackendRunRequestBuilder:
"""Build an Agent App conversation-turn run request.
Layer graph: optional Agent Soul system prompt → user prompt →
execution context → optional shell / config / drive / history
(multi-turn) → LLM → optional plugin-direct tools / core-routed tools /
knowledge search / ask_human / structured output. Mirrors the
workflow-node layer ordering minus the workflow-job / previous-node
prompt.
execution context → optional history (multi-turn) → LLM → optional
plugin tools / knowledge search → optional structured output. Mirrors the workflow-node
layer ordering minus the workflow-job / previous-node prompt.
"""
layers: list[RunLayerSpec] = []
if run_input.agent_soul_prompt:
@@ -288,42 +263,14 @@ class AgentBackendRunRequestBuilder:
]
)
include_shell = (
run_input.include_shell or run_input.config_layer_config is not None or run_input.drive_config is not None
)
if include_shell:
# Sandboxed bash workspace (dify.shell). It enters before config/drive
# so eager pulls materialize content in the same filesystem used by
# model commands.
layers.append(
RunLayerSpec(
name=DIFY_SHELL_LAYER_ID,
type=DIFY_SHELL_LAYER_TYPE_ID,
deps=_shell_layer_deps(),
metadata=run_input.metadata,
config=_shell_config_with_drive_ref(run_input.shell_config, run_input.drive_config),
)
)
if run_input.config_layer_config is not None:
layers.append(
RunLayerSpec(
name=DIFY_CONFIG_LAYER_ID,
type=DIFY_CONFIG_LAYER_TYPE_ID,
deps=_config_layer_deps(),
metadata=run_input.metadata,
config=run_input.config_layer_config,
)
)
if run_input.drive_config is not None:
# Drive Skills & Files declaration (dify.drive): the catalog plus
# prompt-mentioned entries eagerly pulled through the shell layer.
# Drive Skills & Files declaration (dify.drive): a config-only index;
# the agent pulls listed entries through the back proxy by drive_ref.
layers.append(
RunLayerSpec(
name=DIFY_DRIVE_LAYER_ID,
type=DIFY_DRIVE_LAYER_TYPE_ID,
deps=_drive_layer_deps(),
deps={"execution_context": DIFY_EXECUTION_CONTEXT_LAYER_ID},
metadata=run_input.metadata,
config=run_input.drive_config,
)
@@ -355,31 +302,17 @@ class AgentBackendRunRequestBuilder:
)
if run_input.tools is not None and run_input.tools.tools:
plugin_tool_deps = {"execution_context": DIFY_EXECUTION_CONTEXT_LAYER_ID}
if include_shell:
plugin_tool_deps["shell"] = DIFY_SHELL_LAYER_ID
layers.append(
RunLayerSpec(
name=DIFY_PLUGIN_TOOLS_LAYER_ID,
type=DIFY_PLUGIN_TOOLS_LAYER_TYPE_ID,
deps=plugin_tool_deps,
deps={"execution_context": DIFY_EXECUTION_CONTEXT_LAYER_ID},
metadata=run_input.metadata,
config=run_input.tools,
)
)
if run_input.core_tools is not None and run_input.core_tools.tools:
layers.append(
RunLayerSpec(
name=DIFY_CORE_TOOLS_LAYER_ID,
type=DIFY_CORE_TOOLS_LAYER_TYPE_ID,
deps={"execution_context": DIFY_EXECUTION_CONTEXT_LAYER_ID},
metadata=run_input.metadata,
config=run_input.core_tools,
)
)
if run_input.knowledge is not None and run_input.knowledge.sets:
if run_input.knowledge is not None and run_input.knowledge.dataset_ids:
layers.append(
RunLayerSpec(
name=DIFY_KNOWLEDGE_BASE_LAYER_ID,
@@ -403,6 +336,21 @@ class AgentBackendRunRequestBuilder:
)
)
if run_input.include_shell:
# Sandboxed bash workspace (dify.shell). Depends on execution_context
# so the agent server can mint per-command Agent Stub env, and on
# drive when present so that env points at /mnt/drive/<drive_ref>.
# shellctl connection itself is server-injected.
layers.append(
RunLayerSpec(
name=DIFY_SHELL_LAYER_ID,
type=DIFY_SHELL_LAYER_TYPE_ID,
deps=_shell_layer_deps(include_drive=run_input.drive_config is not None),
metadata=run_input.metadata,
config=run_input.shell_config or DifyShellLayerConfig(),
)
)
if run_input.output is not None:
layers.append(
RunLayerSpec(
@@ -444,8 +392,7 @@ class AgentBackendRunRequestBuilder:
non-plugin layer graph that produced the snapshot. Plugin layers
(``dify.plugin.llm``, ``dify.plugin.tools``) are excluded from both the
composition and the snapshot before submission because their configs
may carry credentials or runtime-only declarations that are not
persisted between runs.
require credentials that are not persisted between runs.
"""
if not runtime_layer_specs:
raise ValueError(
@@ -478,9 +425,8 @@ class AgentBackendRunRequestBuilder:
"""Build a workflow Agent Node run request without defining another wire schema.
Layer graph mirrors the workflow surface: prompts → execution context →
optional shell / config / drive / history → LLM → optional
plugin-direct tools / core-routed tools / knowledge search /
ask_human / structured output.
optional drive/history → LLM → optional plugin tools / knowledge search
→ optional auxiliary layers such as ask_human, shell, and structured output.
"""
layers: list[RunLayerSpec] = []
if run_input.agent_soul_prompt:
@@ -499,7 +445,7 @@ class AgentBackendRunRequestBuilder:
name=WORKFLOW_NODE_JOB_PROMPT_LAYER_ID,
type=PLAIN_PROMPT_LAYER_TYPE_ID,
metadata={**run_input.metadata, "origin": "workflow_node_job"},
config=PromptLayerConfig(user=run_input.workflow_node_job_prompt),
config=PromptLayerConfig(prefix=run_input.workflow_node_job_prompt),
),
RunLayerSpec(
name=WORKFLOW_USER_PROMPT_LAYER_ID,
@@ -516,42 +462,14 @@ class AgentBackendRunRequestBuilder:
]
)
include_shell = (
run_input.include_shell or run_input.config_layer_config is not None or run_input.drive_config is not None
)
if include_shell:
# Sandboxed bash workspace (dify.shell). It enters before drive so
# drive can materialize mentioned targets with `dify-agent drive pull`
# in the same shell-visible filesystem used by model commands.
layers.append(
RunLayerSpec(
name=DIFY_SHELL_LAYER_ID,
type=DIFY_SHELL_LAYER_TYPE_ID,
deps=_shell_layer_deps(),
metadata=run_input.metadata,
config=_shell_config_with_drive_ref(run_input.shell_config, run_input.drive_config),
)
)
if run_input.config_layer_config is not None:
layers.append(
RunLayerSpec(
name=DIFY_CONFIG_LAYER_ID,
type=DIFY_CONFIG_LAYER_TYPE_ID,
deps=_config_layer_deps(),
metadata=run_input.metadata,
config=run_input.config_layer_config,
)
)
if run_input.drive_config is not None:
# Drive Skills & Files declaration (dify.drive): the catalog plus
# prompt-mentioned entries eagerly pulled through the shell layer.
# Drive Skills & Files declaration (dify.drive): a config-only index;
# the agent pulls listed entries through the back proxy by drive_ref.
layers.append(
RunLayerSpec(
name=DIFY_DRIVE_LAYER_ID,
type=DIFY_DRIVE_LAYER_TYPE_ID,
deps=_drive_layer_deps(),
deps={"execution_context": DIFY_EXECUTION_CONTEXT_LAYER_ID},
metadata=run_input.metadata,
config=run_input.drive_config,
)
@@ -585,31 +503,17 @@ class AgentBackendRunRequestBuilder:
)
if run_input.tools is not None and run_input.tools.tools:
plugin_tool_deps = {"execution_context": DIFY_EXECUTION_CONTEXT_LAYER_ID}
if include_shell:
plugin_tool_deps["shell"] = DIFY_SHELL_LAYER_ID
layers.append(
RunLayerSpec(
name=DIFY_PLUGIN_TOOLS_LAYER_ID,
type=DIFY_PLUGIN_TOOLS_LAYER_TYPE_ID,
deps=plugin_tool_deps,
deps={"execution_context": DIFY_EXECUTION_CONTEXT_LAYER_ID},
metadata=run_input.metadata,
config=run_input.tools,
)
)
if run_input.core_tools is not None and run_input.core_tools.tools:
layers.append(
RunLayerSpec(
name=DIFY_CORE_TOOLS_LAYER_ID,
type=DIFY_CORE_TOOLS_LAYER_TYPE_ID,
deps={"execution_context": DIFY_EXECUTION_CONTEXT_LAYER_ID},
metadata=run_input.metadata,
config=run_input.core_tools,
)
)
if run_input.knowledge is not None and run_input.knowledge.sets:
if run_input.knowledge is not None and run_input.knowledge.dataset_ids:
layers.append(
RunLayerSpec(
name=DIFY_KNOWLEDGE_BASE_LAYER_ID,
@@ -633,6 +537,21 @@ class AgentBackendRunRequestBuilder:
)
)
if run_input.include_shell:
# Sandboxed bash workspace (dify.shell). Depends on execution_context
# so the agent server can mint per-command Agent Stub env, and on
# drive when present so that env points at /mnt/drive/<drive_ref>.
# shellctl connection itself is server-injected.
layers.append(
RunLayerSpec(
name=DIFY_SHELL_LAYER_ID,
type=DIFY_SHELL_LAYER_TYPE_ID,
deps=_shell_layer_deps(include_drive=run_input.drive_config is not None),
metadata=run_input.metadata,
config=run_input.shell_config or DifyShellLayerConfig(),
)
)
if run_input.output is not None:
layers.append(
RunLayerSpec(
+1 -2
View File
@@ -22,7 +22,7 @@ from .plugin import (
setup_system_trigger_oauth_client,
transform_datasource_credentials,
)
from .rbac import migrate_dataset_permissions_to_rbac, migrate_member_roles_to_rbac
from .rbac import migrate_member_roles_to_rbac
from .retention import (
archive_workflow_runs,
archive_workflow_runs_plan,
@@ -76,7 +76,6 @@ __all__ = [
"legacy_model_types",
"migrate_annotation_vector_database",
"migrate_data_for_plugin",
"migrate_dataset_permissions_to_rbac",
"migrate_knowledge_vector_database",
"migrate_member_roles_to_rbac",
"migrate_oss",
-2
View File
@@ -7,7 +7,6 @@ from typing import cast
import click
from commands.rbac import migrate_dataset_permissions_to_rbac
from extensions.ext_database import db
from graphon.model_runtime.entities.model_entities import ModelType
from services.legacy_model_type_migration import (
@@ -178,4 +177,3 @@ def legacy_model_types(
data_migrate.add_command(legacy_model_types)
data_migrate.add_command(migrate_dataset_permissions_to_rbac)
+13 -18
View File
@@ -10,7 +10,6 @@ import click
import sqlalchemy as sa
import yaml
from core.db.session_factory import session_factory
from extensions.ext_database import db
from models import Tenant
from models.model import App
@@ -107,8 +106,7 @@ def export_migration_data(input_file: str | None, output_file: str | None, overw
assert output_file is not None
raw_config = _load_json_object(input_file, "Export config")
selection = ExportConfigParser().parse(raw_config)
with session_factory.create_session() as session:
result = MigrationExportService().export(session, selection)
result = MigrationExportService().export(selection)
MigrationPackageService().save_package(result.package, output_file, overwrite=overwrite)
click.echo(click.style(f"Output written to {output_file}", fg="green"))
_render_report(result.report_items, context=_with_output_path(result.report_context, output_file))
@@ -155,21 +153,19 @@ def import_migration_data(
_require_options(("--input", input_file))
assert input_file is not None
package = MigrationPackageService().load_package(input_file)
with session_factory.create_session() as session:
result = MigrationImportService().import_package(
session,
ImportRequest(
package=package,
cli_target_tenant=target_tenant,
operator_email=operator_email,
options_override=_build_options_override(
package.metadata.import_options,
id_strategy=id_strategy,
conflict_strategy=conflict_strategy,
create_app_api_token_on_import=create_app_api_token_on_import,
),
result = MigrationImportService().import_package(
ImportRequest(
package=package,
cli_target_tenant=target_tenant,
operator_email=operator_email,
options_override=_build_options_override(
package.metadata.import_options,
id_strategy=id_strategy,
conflict_strategy=conflict_strategy,
create_app_api_token_on_import=create_app_api_token_on_import,
),
)
)
_render_report(result.report_items, context=result.report_context)
except MigrationDataError as exc:
raise click.ClickException(str(exc)) from exc
@@ -252,8 +248,7 @@ def migration_data_wizard() -> None:
conflict_strategy=conflict_strategy,
output_file=output_file,
)
with session_factory.create_session() as session:
result = MigrationExportService().export(session, selection)
result = MigrationExportService().export(selection)
MigrationPackageService().save_package(result.package, output_file, overwrite=overwrite)
click.echo(click.style(f"Output written to {output_file}", fg="green"))
_print_wizard_step("Report")
+65 -437
View File
@@ -1,55 +1,11 @@
from __future__ import annotations
import json
from collections.abc import Iterator
from concurrent.futures import ThreadPoolExecutor, as_completed
import click
from sqlalchemy import select
from configs import dify_config
from core.db.session_factory import session_factory
from core.rbac import RBACResourceWhitelistScope
from models import Dataset, DatasetPermission, DatasetPermissionEnum, TenantAccountJoin, TenantAccountRole
from services.enterprise.rbac_service import ListOption, RBACService, ReplaceMemberBindings, ReplaceUserAccessPolicies
_RBAC_DEFAULT_ACCESS_POLICY_ID = "default"
_LEGACY_ROLE_TO_BUILTIN_TAG = {
TenantAccountRole.OWNER.value: "owner",
TenantAccountRole.ADMIN.value: "admin",
TenantAccountRole.EDITOR.value: "editor",
TenantAccountRole.NORMAL.value: "normal",
TenantAccountRole.DATASET_OPERATOR.value: "dataset_operator",
}
def _resolve_builtin_role_ids(tenant_id: str, operator_account_id: str) -> dict[str, str]:
"""Resolve every legacy workspace role to the current tenant's builtin RBAC role id.
The migration replays the old `TenantAccountJoin.role` values onto the
RBAC member-role binding API. Builtin RBAC roles are tenant-scoped and
identified by runtime ids, so the command must look them up per tenant.
"""
roles = RBACService.Roles.list(
tenant_id=tenant_id,
account_id=operator_account_id,
options=ListOption(page_number=1, results_per_page=100),
).data
role_id_by_tag = {
role.role_tag: role.id
for role in roles
if role.is_builtin and role.category == "global_system_default" and role.role_tag
}
resolved: dict[str, str] = {}
for legacy_role, expected_builtin_tag in _LEGACY_ROLE_TO_BUILTIN_TAG.items():
role_id = role_id_by_tag.get(expected_builtin_tag)
if expected_builtin_tag == "dataset_operator" and not dify_config.DATASET_OPERATOR_ENABLED:
continue
if not role_id:
raise ValueError(f"Builtin RBAC role not found for tenant={tenant_id}, legacy_role={legacy_role}")
resolved[legacy_role] = role_id
return resolved
from models import TenantAccountJoin, TenantAccountRole
from services.enterprise.rbac_service import ListOption, RBACService
def _resolve_builtin_role_id(tenant_id: str, operator_account_id: str, legacy_role: str) -> str:
@@ -59,86 +15,26 @@ def _resolve_builtin_role_id(tenant_id: str, operator_account_id: str, legacy_ro
RBAC member-role binding API. Builtin RBAC roles are tenant-scoped and
identified by runtime ids, so the command must look them up per tenant.
"""
if legacy_role not in _LEGACY_ROLE_TO_BUILTIN_TAG:
expected_builtin_tag = {
TenantAccountRole.OWNER.value: "owner",
TenantAccountRole.ADMIN.value: "admin",
TenantAccountRole.EDITOR.value: "editor",
TenantAccountRole.NORMAL.value: "normal",
TenantAccountRole.DATASET_OPERATOR.value: "dataset_operator",
}.get(legacy_role)
if not expected_builtin_tag:
raise ValueError(f"Unsupported legacy workspace role: {legacy_role}")
return _resolve_builtin_role_ids(tenant_id, operator_account_id)[legacy_role]
def _iter_tenant_member_batches(
tenant_id: str | None,
*,
db_batch_size: int,
api_batch_size: int,
) -> Iterator[tuple[str, str, list[tuple[str, str]]]]:
"""Yield legacy member roles in tenant-scoped API-sized batches.
Rows are projected to primitive values and streamed from the database, so
the command never materializes every TenantAccountJoin ORM object. The
iterator only keeps one tenant's API-sized batches in memory while it
finds that tenant's owner account.
"""
with session_factory.create_session() as session:
stmt = (
select(TenantAccountJoin.tenant_id, TenantAccountJoin.account_id, TenantAccountJoin.role)
.order_by(TenantAccountJoin.tenant_id.asc(), TenantAccountJoin.id.asc())
.execution_options(yield_per=db_batch_size)
)
if tenant_id:
stmt = stmt.where(TenantAccountJoin.tenant_id == tenant_id)
current_tenant_id: str | None = None
owner_account_id: str | None = None
batches: list[list[tuple[str, str]]] = []
batch: list[tuple[str, str]] = []
def flush_current_tenant() -> Iterator[tuple[str, str, list[tuple[str, str]]]]:
if current_tenant_id is None:
return
if batch:
batches.append(batch.copy())
if not owner_account_id:
raise ValueError(f"Workspace owner not found for tenant={current_tenant_id}")
for item in batches:
yield current_tenant_id, owner_account_id, item
for row in session.execute(stmt):
workspace_id = str(row.tenant_id)
if current_tenant_id is not None and workspace_id != current_tenant_id:
yield from flush_current_tenant()
owner_account_id = None
batches = []
batch = []
current_tenant_id = workspace_id
account_id = str(row.account_id)
role = str(row.role)
if role == TenantAccountRole.OWNER.value:
owner_account_id = account_id
batch.append((account_id, role))
if len(batch) >= api_batch_size:
batches.append(batch)
batch = []
yield from flush_current_tenant()
def _member_already_has_role(current_roles_by_account_id: dict[str, set[str]], account_id: str, role_id: str) -> bool:
return current_roles_by_account_id.get(account_id) == {role_id}
def _replace_member_role(
tenant_id: str,
operator_account_id: str,
member_account_id: str,
role_id: str,
) -> str:
RBACService.MemberRoles.replace(
roles = RBACService.Roles.list(
tenant_id=tenant_id,
account_id=operator_account_id,
member_account_id=member_account_id,
role_ids=[role_id],
)
return member_account_id
options=ListOption(page_number=1, results_per_page=100),
).data
for role in roles:
if role.is_builtin and role.category == "global_system_default" and role.role_tag == expected_builtin_tag:
return str(role.id)
raise ValueError(f"Builtin RBAC role not found for tenant={tenant_id}, legacy_role={legacy_role}")
@click.command(
@@ -146,16 +42,7 @@ def _replace_member_role(
)
@click.option("--tenant-id", help="Only migrate a single workspace.")
@click.option("--dry-run", is_flag=True, default=False, help="Preview the migration without writing RBAC bindings.")
@click.option("--db-batch-size", default=5000, show_default=True, help="Rows fetched per database batch.")
@click.option("--api-batch-size", default=200, show_default=True, help="Members checked per RBAC batch_get call.")
@click.option("--workers", default=1, show_default=True, help="Concurrent member role replace calls per tenant batch.")
def migrate_member_roles_to_rbac(
tenant_id: str | None,
dry_run: bool,
db_batch_size: int,
api_batch_size: int,
workers: int,
) -> None:
def migrate_member_roles_to_rbac(tenant_id: str | None, dry_run: bool) -> None:
"""Backfill RBAC member-role bindings from legacy `TenantAccountJoin.role` data.
This is an offline migration command for workspaces that already have
@@ -163,322 +50,63 @@ def migrate_member_roles_to_rbac(
member-role binding store.
"""
click.echo(click.style("Starting RBAC member-role migration.", fg="green"))
if workers < 1:
raise click.BadParameter("workers must be >= 1", param_hint="--workers")
tenant_count = 0
scanned_count = 0
skipped_count = 0
migrated_count = 0
current_tenant_id: str | None = None
role_ids_by_legacy_role: dict[str, str] = {}
with session_factory.create_session() as session:
stmt = select(TenantAccountJoin).order_by(TenantAccountJoin.tenant_id.asc(), TenantAccountJoin.id.asc())
if tenant_id:
stmt = stmt.where(TenantAccountJoin.tenant_id == tenant_id)
for workspace_id, owner_account_id, batch in _iter_tenant_member_batches(
tenant_id,
db_batch_size=db_batch_size,
api_batch_size=api_batch_size,
):
scanned_count += len(batch)
if workspace_id != current_tenant_id:
tenant_count += 1
current_tenant_id = workspace_id
role_ids_by_legacy_role = _resolve_builtin_role_ids(workspace_id, owner_account_id)
click.echo(f"tenant={workspace_id}")
joins = list(session.scalars(stmt).all())
current_roles_by_account_id: dict[str, set[str]] = {}
if not dry_run:
current_roles = RBACService.MemberRoles.batch_get(
tenant_id=workspace_id,
account_id=owner_account_id,
member_account_ids=[account_id for account_id, _ in batch],
)
current_roles_by_account_id = {
item.account_id: {str(role.id) for role in item.roles} for item in current_roles
}
replace_jobs: list[tuple[str, str]] = []
for member_account_id, legacy_role in batch:
resolved_role_id = role_ids_by_legacy_role.get(legacy_role)
if not resolved_role_id:
raise ValueError(f"Unsupported legacy workspace role: {legacy_role}")
if dry_run:
click.echo(
f"tenant={workspace_id} member={member_account_id} "
f"legacy_role={legacy_role} -> rbac_role_id={resolved_role_id}"
)
continue
if _member_already_has_role(current_roles_by_account_id, member_account_id, resolved_role_id):
skipped_count += 1
continue
replace_jobs.append((member_account_id, resolved_role_id))
if replace_jobs:
if workers == 1:
for member_account_id, resolved_role_id in replace_jobs:
_replace_member_role(workspace_id, owner_account_id, member_account_id, resolved_role_id)
migrated_count += 1
else:
with ThreadPoolExecutor(max_workers=workers) as executor:
futures = [
executor.submit(
_replace_member_role,
workspace_id,
owner_account_id,
member_account_id,
resolved_role_id,
)
for member_account_id, resolved_role_id in replace_jobs
]
for future in as_completed(futures):
future.result()
migrated_count += 1
if scanned_count % 10000 == 0:
click.echo(
f"progress scanned={scanned_count} migrated={migrated_count} skipped={skipped_count}",
err=True,
)
if scanned_count == 0:
if not joins:
click.echo(click.style("No workspace members found for migration.", fg="yellow"))
return
if dry_run:
click.echo(
click.style(
f"Dry run completed. Scanned {scanned_count} members across {tenant_count} tenants. "
"No RBAC bindings were written.",
fg="yellow",
owner_account_by_tenant: dict[str, str] = {}
resolved_role_ids: dict[tuple[str, str], str] = {}
migrated_count = 0
for join in joins:
workspace_id = str(join.tenant_id)
member_account_id = str(join.account_id)
legacy_role = str(join.role)
if workspace_id not in owner_account_by_tenant:
owner_join = next(
(
item
for item in joins
if str(item.tenant_id) == workspace_id and str(item.role) == TenantAccountRole.OWNER.value
),
None,
)
)
else:
if not owner_join:
raise ValueError(f"Workspace owner not found for tenant={workspace_id}")
owner_account_by_tenant[workspace_id] = str(owner_join.account_id)
operator_account_id = owner_account_by_tenant[workspace_id]
cache_key = (workspace_id, legacy_role)
if cache_key not in resolved_role_ids:
resolved_role_ids[cache_key] = _resolve_builtin_role_id(workspace_id, operator_account_id, legacy_role)
resolved_role_id = resolved_role_ids[cache_key]
click.echo(
click.style(
f"RBAC member-role migration completed. Scanned {scanned_count} members across {tenant_count} tenants, "
f"migrated {migrated_count}, skipped {skipped_count} already up-to-date.",
fg="green",
)
f"tenant={workspace_id} member={member_account_id} "
f"legacy_role={legacy_role} -> rbac_role_id={resolved_role_id}"
)
if dry_run:
continue
def _dataset_permission_enum(permission: DatasetPermissionEnum | str | None) -> DatasetPermissionEnum:
if permission is None:
return DatasetPermissionEnum.ONLY_ME
try:
return DatasetPermissionEnum(permission)
except ValueError as exc:
raise ValueError(f"Unsupported legacy dataset permission: {permission}") from exc
def _rbac_dataset_scope_for_legacy_permission(permission: DatasetPermissionEnum) -> RBACResourceWhitelistScope:
if permission is DatasetPermissionEnum.ALL_TEAM:
return RBACResourceWhitelistScope.ALL
if permission in {DatasetPermissionEnum.ONLY_ME, DatasetPermissionEnum.PARTIAL_TEAM}:
return RBACResourceWhitelistScope.SPECIFIC
raise ValueError(f"Unsupported legacy dataset permission: {permission}")
def _emit_dataset_permission_migration_event(payload: dict[str, object]) -> None:
click.echo(json.dumps(payload, sort_keys=True))
@click.command(
"rbac-migrate-dataset-permissions",
help=(
"Migrate legacy dataset permission scopes and partial members into RBAC dataset access bindings. "
"Side effect: replacing each dataset whitelist clears existing per-user policy bindings; "
"the command then recreates legacy partial-member default bindings."
),
)
@click.option("--tenant-id", help="Only migrate datasets in a single workspace.")
@click.option("--dataset-id", help="Only migrate a single dataset.")
@click.option("--batch-size", default=500, show_default=True, type=click.IntRange(min=1))
@click.option(
"--dry-run/--apply",
default=True,
show_default=True,
help="Preview the migration without writing RBAC bindings. Use --apply to write changes.",
)
def migrate_dataset_permissions_to_rbac(
tenant_id: str | None,
dataset_id: str | None,
batch_size: int,
dry_run: bool,
) -> None:
"""Backfill RBAC dataset access config from legacy `Dataset.permission`.
Legacy mapping:
- all_team_members -> RBAC dataset whitelist scope "all"
- partial_members -> RBAC dataset whitelist scope "specific" plus each partial member gets the
virtual default policy
- only_me -> RBAC dataset whitelist scope "specific" with no member policy bindings
The command replaces each dataset's RBAC whitelist scope first. RBAC clears
existing per-user policy bindings during that replace, then this command
recreates the legacy partial-member default bindings. Re-running it is
therefore idempotent for a dataset's current legacy configuration.
"""
click.echo(click.style("Starting RBAC dataset permission migration.", fg="green"))
scanned_count = 0
scope_migrated_count = 0
user_policy_migrated_count = 0
partial_dataset_count = 0
last_dataset_id: str | None = None
while True:
with session_factory.create_session() as session:
stmt = (
select(Dataset.id, Dataset.tenant_id, Dataset.permission, Dataset.created_by)
.order_by(Dataset.id.asc())
.limit(batch_size)
)
if tenant_id:
stmt = stmt.where(Dataset.tenant_id == tenant_id)
if dataset_id:
stmt = stmt.where(Dataset.id == dataset_id)
if last_dataset_id:
stmt = stmt.where(Dataset.id > last_dataset_id)
dataset_rows = list(session.execute(stmt).all())
if not dataset_rows:
break
dataset_ids = [str(row.id) for row in dataset_rows]
partial_members_by_dataset_id: dict[str, list[str]] = {item: [] for item in dataset_ids}
permission_rows = session.execute(
select(DatasetPermission.dataset_id, DatasetPermission.account_id).where(
DatasetPermission.dataset_id.in_(dataset_ids)
)
).all()
for row in permission_rows:
partial_members_by_dataset_id[str(row.dataset_id)].append(str(row.account_id))
for dataset in dataset_rows:
workspace_id = str(dataset.tenant_id)
current_dataset_id = str(dataset.id)
operator_account_id = str(dataset.created_by)
permission_value = _dataset_permission_enum(dataset.permission)
scope = _rbac_dataset_scope_for_legacy_permission(permission_value)
partial_member_ids = sorted(set(partial_members_by_dataset_id[current_dataset_id]))
should_bind_partial_members = permission_value is DatasetPermissionEnum.PARTIAL_TEAM
click.echo(
f"tenant={workspace_id} dataset={current_dataset_id} "
f"operator={operator_account_id} "
f"legacy_permission={permission_value} -> rbac_scope={scope} "
f"partial_members={len(partial_member_ids) if should_bind_partial_members else 0}"
)
scanned_count += 1
replace_whitelist_payload = ReplaceMemberBindings(scope=scope)
if dry_run:
_emit_dataset_permission_migration_event(
{
"event": "dataset_permission_migration_proposed_change",
"action": "replace_whitelist",
"dry_run": True,
"tenant_id": workspace_id,
"dataset_id": current_dataset_id,
"operator_account_id": operator_account_id,
"before": {
"legacy_dataset_permission": permission_value.value,
"legacy_partial_member_ids": partial_member_ids if should_bind_partial_members else [],
},
"after": {
"rbac_whitelist_scope": scope.value,
},
"call": {
"method": "RBACService.DatasetAccess.replace_whitelist",
"kwargs": {
"tenant_id": workspace_id,
"account_id": operator_account_id,
"dataset_id": current_dataset_id,
"payload": replace_whitelist_payload.model_dump(mode="json"),
},
},
}
)
if not dry_run:
RBACService.DatasetAccess.replace_whitelist(
tenant_id=workspace_id,
account_id=operator_account_id,
dataset_id=current_dataset_id,
payload=replace_whitelist_payload,
)
scope_migrated_count += 1
if should_bind_partial_members:
partial_dataset_count += 1
for member_account_id in partial_member_ids:
replace_user_access_policies_payload = ReplaceUserAccessPolicies(
access_policy_ids=[_RBAC_DEFAULT_ACCESS_POLICY_ID],
)
if dry_run:
_emit_dataset_permission_migration_event(
{
"event": "dataset_permission_migration_proposed_change",
"action": "replace_user_access_policies",
"dry_run": True,
"tenant_id": workspace_id,
"dataset_id": current_dataset_id,
"operator_account_id": operator_account_id,
"target_account_id": member_account_id,
"before": {
"legacy_dataset_permission": permission_value.value,
"legacy_partial_member_id": member_account_id,
},
"after": {
"rbac_user_access_policy_ids": [_RBAC_DEFAULT_ACCESS_POLICY_ID],
},
"call": {
"method": "RBACService.DatasetAccess.replace_user_access_policies",
"kwargs": {
"tenant_id": workspace_id,
"account_id": operator_account_id,
"dataset_id": current_dataset_id,
"target_account_id": member_account_id,
"payload": replace_user_access_policies_payload.model_dump(mode="json"),
},
},
}
)
continue
RBACService.DatasetAccess.replace_user_access_policies(
tenant_id=workspace_id,
account_id=operator_account_id,
dataset_id=current_dataset_id,
target_account_id=member_account_id,
payload=replace_user_access_policies_payload,
)
user_policy_migrated_count += 1
last_dataset_id = dataset_ids[-1]
if dataset_id:
break
if scanned_count == 0:
click.echo(click.style("No datasets found for migration.", fg="yellow"))
return
RBACService.MemberRoles.replace(
tenant_id=workspace_id,
account_id=operator_account_id,
member_account_id=member_account_id,
role_ids=[resolved_role_id],
)
migrated_count += 1
if dry_run:
click.echo(
click.style(
f"Dry run completed. Scanned {scanned_count} datasets; "
f"{partial_dataset_count} partial-member datasets would be migrated.",
fg="yellow",
)
)
click.echo(click.style("Dry run completed. No RBAC bindings were written.", fg="yellow"))
else:
click.echo(
click.style(
"RBAC dataset permission migration completed. "
f"Scanned {scanned_count} datasets, migrated {scope_migrated_count} scopes, "
f"wrote {user_policy_migrated_count} user default-policy bindings.",
fg="green",
)
)
click.echo(click.style(f"RBAC member-role migration completed. Migrated {migrated_count} members.", fg="green"))
+2 -29
View File
@@ -35,12 +35,6 @@ class WorkflowRunArchiveTenantPlan(TypedDict):
unpaid_tenant_ids: list[str]
def _normalize_utc_datetime(value: datetime.datetime) -> datetime.datetime:
if value.tzinfo is None:
return value.replace(tzinfo=datetime.UTC)
return value.astimezone(datetime.UTC)
def _parse_tenant_prefixes(prefixes: str | None) -> list[str]:
if not prefixes:
return []
@@ -162,16 +156,11 @@ def _resolve_archive_time_range(
raise click.UsageError("Choose either day offsets or explicit dates, not both.")
if from_days_ago <= to_days_ago:
raise click.UsageError("--from-days-ago must be greater than --to-days-ago.")
now = datetime.datetime.now(datetime.UTC)
now = datetime.datetime.now()
start_from = now - datetime.timedelta(days=from_days_ago)
end_before = now - datetime.timedelta(days=to_days_ago)
before_days = 0
if start_from is not None:
start_from = _normalize_utc_datetime(start_from)
if end_before is not None:
end_before = _normalize_utc_datetime(end_before)
if start_from and end_before and start_from >= end_before:
raise click.UsageError("--start-from must be earlier than --end-before.")
@@ -413,13 +402,6 @@ def archive_workflow_runs_plan(
fg="white",
)
)
click.echo(
click.style(
"fixed_archive_window="
f"{start_from.isoformat() if start_from else 'unbounded'},{plan_end_before.isoformat()}",
fg="white",
)
)
click.echo("tenant_prefix,total_tenants,workflow_runs,workflow_node_executions,paid_tenants,unpaid_tenants")
for row in rows:
click.echo(
@@ -469,7 +451,7 @@ def archive_workflow_runs_plan(
default=None,
help="Archive runs created before this timestamp (UTC if no timezone).",
)
@click.option("--batch-size", default=10000, show_default=True, help="Maximum workflow runs per archive bundle.")
@click.option("--batch-size", default=100, show_default=True, help="Maximum workflow runs per archive bundle.")
@click.option(
"--workers",
default=1,
@@ -539,7 +521,6 @@ def archive_workflow_runs(
)
)
uses_relative_window = start_from is None and end_before is None
try:
before_days, start_from, end_before = _resolve_archive_time_range(
before_days=before_days,
@@ -565,14 +546,6 @@ def archive_workflow_runs(
if delete_after_archive:
click.echo(click.style("delete-after-archive is not supported by bundle archive.", fg="red"))
return
if uses_relative_window:
click.echo(
click.style(
"Relative archive windows are evaluated at command start. For multi-day prefix/shard rollout, "
"reuse absolute --start-from/--end-before values from archive-workflow-runs-plan.",
fg="yellow",
)
)
try:
tenant_plan = _resolve_archive_tenant_ids_from_plan(
+2 -3
View File
@@ -13,7 +13,6 @@ from core.rag.index_processor.constant.built_in_field import BuiltInField
from core.rag.index_processor.constant.index_type import IndexStructureType, IndexTechniqueType
from core.rag.models.document import ChildDocument, Document
from extensions.ext_database import db
from libs.pagination import paginate_query
from models.dataset import Dataset, DatasetCollectionBinding, DatasetMetadata, DatasetMetadataBinding, DocumentSegment
from models.dataset import Document as DatasetDocument
from models.enums import DatasetMetadataType, IndexingStatus, SegmentStatus
@@ -184,7 +183,7 @@ def migrate_knowledge_vector_database():
.order_by(Dataset.created_at.desc())
)
datasets = paginate_query(stmt, page=page, per_page=50, max_per_page=50)
datasets = db.paginate(select=stmt, page=page, per_page=50, max_per_page=50, error_out=False)
if not datasets.items:
break
except SQLAlchemyError:
@@ -410,7 +409,7 @@ def old_metadata_migration():
.where(DatasetDocument.doc_metadata.is_not(None))
.order_by(DatasetDocument.created_at.desc())
)
documents = paginate_query(stmt, page=page, per_page=50, max_per_page=50)
documents = db.paginate(select=stmt, page=page, per_page=50, max_per_page=50, error_out=False)
except SQLAlchemyError:
raise
if not documents:
+1 -7
View File
@@ -16,7 +16,7 @@ class EnterpriseFeatureConfig(BaseSettings):
CAN_REPLACE_LOGO: bool = Field(
description="Allow customization of the enterprise logo.",
default=False,
default=True,
)
ENTERPRISE_REQUEST_TIMEOUT: int = Field(
@@ -34,12 +34,6 @@ class EnterpriseFeatureConfig(BaseSettings):
default=False,
)
ENTERPRISE_RBAC_REQUEST_TIMEOUT: int = Field(
ge=1,
description="Maximum timeout in seconds for inner RBAC requests.",
default=30,
)
class EnterpriseTelemetryConfig(BaseSettings):
"""
+7 -5
View File
@@ -1,4 +1,4 @@
from pydantic import Field, NonNegativeFloat
from pydantic import Field
from pydantic_settings import BaseSettings
@@ -32,10 +32,12 @@ class AgentBackendConfig(BaseSettings):
default=False,
)
AGENT_APP_TEXT_DELTA_DEBOUNCE_SECONDS: NonNegativeFloat = Field(
AGENT_DRIVE_MANIFEST_ENABLED: bool = Field(
description=(
"Buffer Agent App assistant text deltas for up to this many seconds before "
"publishing SSE chunks. Set to 0 to publish each delta immediately."
"Inject the dify.drive layer (Skills & Files drive manifest declaration) "
"into Agent runs. The declaration is an index only — the agent backend "
"pulls the actual SKILL.md / files through the back proxy. Keep it off "
"until the agent backend registers the dify.drive layer type."
),
default=0.5,
default=False,
)
File diff suppressed because one or more lines are too long
-1
View File
@@ -183,7 +183,6 @@ class Site(BaseModel):
description: str | None = None
copyright: str | None = None
privacy_policy: str | None = None
input_placeholder: str | None = None
custom_disclaimer: str | None = None
default_language: str
show_workflow_steps: bool
-57
View File
@@ -1,57 +0,0 @@
"""Controller session decorators.
`with_session` is an HTTP controller helper: it opens one SQLAlchemy session
for a Resource handler and injects it as the first argument after `self`.
Handlers use a transaction by default so migrated write paths keep
commit/rollback handling; pure read handlers may opt out with `write=False`.
"""
from collections.abc import Callable
from functools import wraps
from typing import Concatenate, overload
from sqlalchemy.orm import Session
from core.db.session_factory import session_factory
@overload
def with_session[T, **P, R](
view: Callable[Concatenate[T, Session, P], R],
*,
write: bool = True,
) -> Callable[Concatenate[T, P], R]: ...
@overload
def with_session[T, **P, R](
view: None = None,
*,
write: bool = True,
) -> Callable[[Callable[Concatenate[T, Session, P], R]], Callable[Concatenate[T, P], R]]: ...
def with_session[T, **P, R](
view: Callable[Concatenate[T, Session, P], R] | None = None,
*,
write: bool = True,
) -> (
Callable[Concatenate[T, P], R] | Callable[[Callable[Concatenate[T, Session, P], R]], Callable[Concatenate[T, P], R]]
):
"""Inject a request-scoped session, using a transaction only for write handlers."""
def decorator(view: Callable[Concatenate[T, Session, P], R]) -> Callable[Concatenate[T, P], R]:
@wraps(view)
def wrapper(self: T, *args: P.args, **kwargs: P.kwargs) -> R:
if write:
with session_factory.get_session_maker().begin() as session:
return view(self, session, *args, **kwargs)
with session_factory.create_session() as session:
return view(self, session, *args, **kwargs)
return wrapper
if view is None:
return decorator
return decorator(view)
-2
View File
@@ -54,7 +54,6 @@ from .app import (
agent_app_access,
agent_app_feature,
agent_app_sandbox,
agent_config_inspector,
agent_drive_inspector,
annotation,
app,
@@ -158,7 +157,6 @@ __all__ = [
"agent_app_feature",
"agent_app_sandbox",
"agent_composer",
"agent_config_inspector",
"agent_drive_inspector",
"agent_providers",
"agent_roster",
+1 -11
View File
@@ -6,15 +6,5 @@ from services.agent.roster_service import AgentRosterService
def resolve_agent_app_model(*, tenant_id: str, agent_id: UUID) -> App:
"""Resolve a roster Agent's public Agent App."""
"""Resolve the hidden Agent App backing an Agent Console resource."""
return AgentRosterService(db.session).get_agent_app_model(tenant_id=tenant_id, agent_id=str(agent_id))
def resolve_agent_runtime_app_model(*, tenant_id: str, agent_id: UUID) -> App:
"""Resolve the App that backs an Agent runtime surface.
This accepts both roster Agent Apps and workflow-only inline Agents with a
hidden backing App.
"""
return AgentRosterService(db.session).get_agent_runtime_app_model(tenant_id=tenant_id, agent_id=str(agent_id))
+17 -23
View File
@@ -1,10 +1,10 @@
from uuid import UUID
from flask import request
from flask_restx import Resource
from controllers.common.schema import query_params_from_model, register_response_schema_models, register_schema_models
from controllers.common.schema import register_response_schema_models, register_schema_models
from controllers.console import console_ns
from controllers.console.agent.app_helpers import resolve_agent_app_model
from controllers.console.app.wraps import get_app_model
from controllers.console.wraps import (
RBACPermission,
@@ -28,15 +28,9 @@ from libs.login import login_required
from models.model import App, AppMode
from services.agent.composer_service import AgentComposerService
from services.agent.composer_validator import ComposerConfigValidator
from services.entities.agent_entities import (
ComposerSavePayload,
WorkflowAgentComposerQuery,
WorkflowComposerCopyFromRosterPayload,
)
from services.entities.agent_entities import ComposerSavePayload, WorkflowComposerCopyFromRosterPayload
register_schema_models(
console_ns, ComposerSavePayload, WorkflowAgentComposerQuery, WorkflowComposerCopyFromRosterPayload
)
register_schema_models(console_ns, ComposerSavePayload, WorkflowComposerCopyFromRosterPayload)
register_response_schema_models(
console_ns,
AgentAppComposerResponse,
@@ -47,28 +41,27 @@ register_response_schema_models(
)
def _resolve_agent_app_id(*, tenant_id: str, agent_id: UUID) -> str:
return resolve_agent_app_model(tenant_id=tenant_id, agent_id=agent_id).id
@console_ns.route("/apps/<uuid:app_id>/workflows/draft/nodes/<string:node_id>/agent-composer")
class WorkflowAgentComposerApi(Resource):
@console_ns.response(
200, "Workflow agent composer state", console_ns.models[WorkflowAgentComposerResponse.__name__]
)
@console_ns.doc(params=query_params_from_model(WorkflowAgentComposerQuery))
@setup_required
@login_required
@account_initialization_required
@get_app_model(mode=[AppMode.WORKFLOW, AppMode.ADVANCED_CHAT])
@with_current_user_id
@with_current_tenant_id
def get(self, tenant_id: str, account_id: str, app_model: App, node_id: str):
query = WorkflowAgentComposerQuery.model_validate(request.args.to_dict(flat=True))
def get(self, tenant_id: str, app_model: App, node_id: str):
return dump_response(
WorkflowAgentComposerResponse,
AgentComposerService.load_workflow_composer(
tenant_id=tenant_id,
app_id=app_model.id,
node_id=node_id,
account_id=account_id,
snapshot_id=query.snapshot_id,
),
)
@@ -144,7 +137,6 @@ class WorkflowAgentComposerValidateApi(Resource):
def post(self, tenant_id: str, app_model: App, node_id: str):
payload = ComposerSavePayload.model_validate(console_ns.payload or {})
ComposerConfigValidator.validate_publish_payload(payload)
AgentComposerService.validate_knowledge_datasets(tenant_id=tenant_id, agent_soul=payload.agent_soul)
findings = AgentComposerService.collect_validation_findings(
tenant_id=tenant_id,
payload=payload,
@@ -236,9 +228,10 @@ class AgentComposerApi(Resource):
@account_initialization_required
@with_current_tenant_id
def get(self, tenant_id: str, agent_id: UUID):
app_id = _resolve_agent_app_id(tenant_id=tenant_id, agent_id=agent_id)
return dump_response(
AgentAppComposerResponse,
AgentComposerService.load_agent_composer(tenant_id=tenant_id, agent_id=str(agent_id)),
AgentComposerService.load_agent_app_composer(tenant_id=tenant_id, app_id=app_id),
)
@console_ns.expect(console_ns.models[ComposerSavePayload.__name__])
@@ -251,12 +244,13 @@ class AgentComposerApi(Resource):
@with_current_user_id
@with_current_tenant_id
def put(self, tenant_id: str, account_id: str, agent_id: UUID):
app_id = _resolve_agent_app_id(tenant_id=tenant_id, agent_id=agent_id)
payload = ComposerSavePayload.model_validate(console_ns.payload or {})
return dump_response(
AgentAppComposerResponse,
AgentComposerService.save_agent_composer(
AgentComposerService.save_agent_app_composer(
tenant_id=tenant_id,
agent_id=str(agent_id),
app_id=app_id,
account_id=account_id,
payload=payload,
),
@@ -274,10 +268,9 @@ class AgentComposerValidateApi(Resource):
@account_initialization_required
@with_current_tenant_id
def post(self, tenant_id: str, agent_id: UUID):
AgentComposerService.load_agent_composer(tenant_id=tenant_id, agent_id=str(agent_id))
_resolve_agent_app_id(tenant_id=tenant_id, agent_id=agent_id)
payload = ComposerSavePayload.model_validate(console_ns.payload or {})
ComposerConfigValidator.validate_publish_payload(payload)
AgentComposerService.validate_knowledge_datasets(tenant_id=tenant_id, agent_soul=payload.agent_soul)
findings = AgentComposerService.collect_validation_findings(
tenant_id=tenant_id,
payload=payload,
@@ -297,11 +290,12 @@ class AgentComposerCandidatesApi(Resource):
@with_current_user_id
@with_current_tenant_id
def get(self, tenant_id: str, current_user_id: str, agent_id: UUID):
app_id = _resolve_agent_app_id(tenant_id=tenant_id, agent_id=agent_id)
return dump_response(
AgentComposerCandidatesResponse,
AgentComposerService.get_agent_app_candidates(
tenant_id=tenant_id,
agent_id=str(agent_id),
app_id=app_id,
user_id=current_user_id,
),
)
+54 -227
View File
@@ -5,21 +5,16 @@ from flask_restx import Resource
from pydantic import AliasChoices, BaseModel, Field, field_validator
from sqlalchemy import func, select
from controllers.common.schema import (
query_params_from_model,
query_params_from_request,
register_response_schema_models,
register_schema_models,
)
from controllers.common.schema import query_params_from_model, register_response_schema_models, register_schema_models
from controllers.console import console_ns
from controllers.console.agent.app_helpers import resolve_agent_app_model, resolve_agent_runtime_app_model
from controllers.console.agent.app_helpers import resolve_agent_app_model
from controllers.console.apikey import ApiKeyItem, ApiKeyList, BaseApiKeyListResource, BaseApiKeyResource
from controllers.console.app.app import (
APP_LIST_QUERY_ARRAY_FIELDS,
AppListQuery,
AppDetailWithSite as GenericAppDetailWithSite,
)
from controllers.console.app.app import (
AppDetailWithSite as GenericAppDetailWithSite,
AppListQuery,
_normalize_app_list_query_args,
)
from controllers.console.app.app import (
AppPagination as GenericAppPagination,
@@ -44,11 +39,9 @@ from controllers.console.wraps import (
)
from extensions.ext_database import db
from fields.agent_fields import (
AgentConfigDraftSummaryResponse,
AgentConfigSnapshotDetailResponse,
AgentConfigSnapshotListResponse,
AgentConfigSnapshotRestoreResponse,
AgentConfigSnapshotSummaryResponse,
AgentInviteOptionsResponse,
AgentLogListResponse,
AgentLogMessageListResponse,
@@ -57,16 +50,12 @@ from fields.agent_fields import (
AgentRosterListResponse,
AgentStatisticSummaryEnvelopeResponse,
)
from fields.base import ResponseModel
from libs.datetime_utils import parse_time_range
from libs.helper import dump_response
from libs.login import login_required
from models import Account
from models.agent import Agent, AgentStatus
from models.agent_config_entities import AgentSoulConfig
from models.enums import ApiTokenType
from models.model import ApiToken, App, IconType
from services.agent.composer_service import AgentComposerService
from services.agent.errors import AgentNotFoundError
from services.agent.observability_service import (
AgentLogQueryParams,
@@ -76,7 +65,7 @@ from services.agent.observability_service import (
from services.agent.roster_service import AgentRosterService
from services.app_service import AppListParams, AppService, CreateAppParams
from services.enterprise.enterprise_service import EnterpriseService
from services.entities.agent_entities import ComposerSavePayload, RosterListQuery
from services.entities.agent_entities import RosterListQuery
from services.feature_service import FeatureService
@@ -91,31 +80,33 @@ class AgentIdPath(BaseModel):
class AgentAppCreatePayload(BaseModel):
name: str = Field(..., min_length=1, description="Agent name")
description: str | None = Field(default=None, description="Agent description (max 400 chars)", max_length=400)
role: str | None = Field(default=None, description="Agent role", max_length=255)
role: str = Field(..., min_length=1, description="Agent role", max_length=255)
icon_type: IconType | None = Field(default=None, description="Icon type")
icon: str | None = Field(default=None, description="Icon")
icon_background: str | None = Field(default=None, description="Icon background color")
@field_validator("role")
@classmethod
def validate_role(cls, value: str | None) -> str | None:
if value is None:
return None
return value.strip()
def validate_role(cls, value: str) -> str:
role = value.strip()
if not role:
raise ValueError("Agent role is required.")
return role
# Keep agent-app roster DTOs agent-specific instead of reusing the shared
# /apps response/request models. The roster surface needs Agent-only fields such
# as `role`, while the generic console/apps contracts must stay unchanged.
class AgentAppUpdatePayload(GenericUpdateAppPayload):
role: str | None = Field(default=None, description="Agent role", max_length=255)
role: str = Field(..., min_length=1, description="Agent role", max_length=255)
@field_validator("role")
@classmethod
def validate_role(cls, value: str | None) -> str | None:
if value is None:
return None
return value.strip()
def validate_role(cls, value: str) -> str:
role = value.strip()
if not role:
raise ValueError("Agent role is required.")
return role
class AgentAppCopyPayload(BaseModel):
@@ -131,7 +122,10 @@ class AgentAppCopyPayload(BaseModel):
def validate_role(cls, value: str | None) -> str | None:
if value is None:
return None
return value.strip()
role = value.strip()
if not role:
raise ValueError("Agent role is required when provided.")
return role
class AgentApiStatusPayload(BaseModel):
@@ -197,9 +191,11 @@ class AgentLogsQuery(BaseModel):
def empty_list_values_to_list(cls, value: object) -> list[str]:
if value in (None, ""):
return []
if isinstance(value, str):
return [value]
if isinstance(value, list):
return [str(item).strip() for item in value if str(item).strip()]
raise ValueError("Unsupported query list type.")
return [item for item in value if item]
return []
@field_validator("sort_by")
@classmethod
@@ -236,8 +232,6 @@ class AgentStatisticsQuery(BaseModel):
class AgentAppPartial(GenericAppPartial):
app_id: str | None = None
backing_app_id: str | None = None
hidden_app_backed: bool = False
debug_conversation_id: str | None = None
role: str | None = None
active_config_is_published: bool = False
@@ -247,49 +241,13 @@ class AgentAppPartial(GenericAppPartial):
class AgentAppDetailWithSite(GenericAppDetailWithSite):
app_id: str | None = None
backing_app_id: str | None = None
hidden_app_backed: bool = False
debug_conversation_id: str | None = None
debug_conversation_has_messages: bool = False
debug_conversation_message_count: int = 0
role: str | None = None
active_config_is_published: bool = False
class AgentDebugConversationRefreshResponse(BaseModel):
debug_conversation_id: str
debug_conversation_has_messages: bool = False
debug_conversation_message_count: int = 0
class AgentPublishPayload(BaseModel):
version_note: str | None = Field(default=None, description="Optional note for this published Agent version")
class AgentPublishResponse(ResponseModel):
result: str
active_config_snapshot_id: str
active_config_snapshot: AgentConfigSnapshotSummaryResponse | None = None
draft: AgentConfigDraftSummaryResponse | None = None
class AgentBuildDraftCheckoutPayload(BaseModel):
force: bool = Field(default=False, description="Overwrite the existing current-user build draft")
class AgentBuildDraftResponse(ResponseModel):
variant: str
draft: AgentConfigDraftSummaryResponse
agent_soul: AgentSoulConfig
class AgentBuildDraftApplyResponse(BaseModel):
result: str
draft: dict[str, object]
class AgentSimpleResultResponse(BaseModel):
result: str
class AgentAppPagination(GenericAppPagination):
@@ -303,9 +261,6 @@ register_schema_models(
AgentAppCreatePayload,
AgentAppUpdatePayload,
AgentAppCopyPayload,
AgentPublishPayload,
AgentBuildDraftCheckoutPayload,
ComposerSavePayload,
AgentApiStatusPayload,
AgentInviteOptionsQuery,
AgentLogsQuery,
@@ -322,10 +277,6 @@ register_response_schema_models(
AgentAppDetailWithSite,
AgentAppPartial,
AgentDebugConversationRefreshResponse,
AgentPublishResponse,
AgentBuildDraftResponse,
AgentBuildDraftApplyResponse,
AgentSimpleResultResponse,
AgentConfigSnapshotDetailResponse,
AgentConfigSnapshotListResponse,
AgentConfigSnapshotRestoreResponse,
@@ -343,7 +294,7 @@ def _agent_roster_service() -> AgentRosterService:
return AgentRosterService(db.session)
def _serialize_agent_app_detail(app_model, *, current_user: Account, agent_id: str | None = None) -> dict:
def _serialize_agent_app_detail(app_model, *, current_user: Account) -> dict:
"""Serialize an Agent App detail using roster-only DTOs.
`/agent` responses are roster-shaped rather than raw app-shaped: `id`
@@ -360,35 +311,17 @@ def _serialize_agent_app_detail(app_model, *, current_user: Account, agent_id: s
roster_service = _agent_roster_service()
payload = AgentAppDetailWithSite.model_validate(app_model, from_attributes=True).model_dump(mode="json")
agent = (
db.session.scalar(
select(Agent).where(
Agent.tenant_id == app_model.tenant_id,
Agent.id == agent_id,
Agent.status == AgentStatus.ACTIVE,
)
)
if agent_id
else roster_service.get_app_backing_agent(tenant_id=app_model.tenant_id, app_id=str(app_model.id))
)
agent = roster_service.get_app_backing_agent(tenant_id=app_model.tenant_id, app_id=str(app_model.id))
if not agent:
raise AgentNotFoundError()
payload.pop("bound_agent_id", None)
payload["app_id"] = agent.app_id
payload["backing_app_id"] = roster_service.runtime_backing_app_id(agent)
payload["hidden_app_backed"] = bool(agent.backing_app_id and agent.backing_app_id != agent.app_id)
payload["app_id"] = str(app_model.id)
payload["id"] = agent.id
debug_conversation_id = roster_service.get_or_create_agent_app_debug_conversation_id(
payload["debug_conversation_id"] = roster_service.get_or_create_agent_app_debug_conversation_id(
tenant_id=app_model.tenant_id,
agent_id=agent.id,
account_id=current_user.id,
)
message_count = roster_service.count_agent_app_debug_conversation_messages(
conversation_id=debug_conversation_id,
)
payload["debug_conversation_id"] = debug_conversation_id
payload["debug_conversation_has_messages"] = message_count > 0
payload["debug_conversation_message_count"] = message_count
payload["role"] = agent.role or ""
payload["active_config_is_published"] = roster_service.active_config_is_published(
tenant_id=app_model.tenant_id,
@@ -432,8 +365,6 @@ def _serialize_agent_app_pagination(app_pagination, *, tenant_id: str, current_u
agent = agents_by_app_id.get(app_id)
if agent:
item["app_id"] = app_id
item["backing_app_id"] = agent.backing_app_id or app_id
item["hidden_app_backed"] = False
item["id"] = agent.id
item["debug_conversation_id"] = debug_conversation_ids_by_agent_id.get(agent.id)
item["role"] = agent.role or ""
@@ -504,11 +435,16 @@ def _parse_observability_time_range(start: str | None, end: str | None, account:
abort(400, description=str(exc))
def _query_values(name: str, alias_name: str | None = None) -> list[str]:
values = request.args.getlist(name)
if alias_name:
values.extend(request.args.getlist(alias_name))
return [value.strip() for value in values if value.strip()]
def _multi_query_values(name: str, legacy_name: str | None = None) -> list[str]:
values: list[str] = []
for query_name in (name, f"{name}[]"):
values.extend(request.args.getlist(query_name))
if legacy_name:
values.extend(request.args.getlist(legacy_name))
parsed: list[str] = []
for value in values:
parsed.extend(item.strip() for item in value.split(",") if item.strip())
return parsed
@console_ns.route("/agent")
@@ -521,12 +457,11 @@ class AgentAppListApi(Resource):
@with_current_user
@with_current_tenant_id
def get(self, current_tenant_id: str, current_user: Account):
args = query_params_from_request(AppListQuery, list_fields=APP_LIST_QUERY_ARRAY_FIELDS)
args = AppListQuery.model_validate(_normalize_app_list_query_args(request.args))
params = AppListParams(
page=args.page,
limit=args.limit,
mode="agent",
sort_by=args.sort_by,
name=args.name,
tag_ids=args.tag_ids,
creator_ids=args.creator_ids,
@@ -561,7 +496,7 @@ class AgentAppListApi(Resource):
name=args.name,
description=args.description,
mode="agent",
agent_role=args.role or "",
agent_role=args.role,
icon_type=args.icon_type,
icon=args.icon,
icon_background=args.icon_background,
@@ -581,8 +516,8 @@ class AgentAppApi(Resource):
@with_current_user
@with_current_tenant_id
def get(self, tenant_id: str, current_user: Account, agent_id: UUID):
app_model = resolve_agent_runtime_app_model(tenant_id=tenant_id, agent_id=agent_id)
return _serialize_agent_app_detail(app_model, current_user=current_user, agent_id=str(agent_id))
app_model = _resolve_agent_app_model(tenant_id=tenant_id, agent_id=agent_id)
return _serialize_agent_app_detail(app_model, current_user=current_user)
@console_ns.expect(console_ns.models[AgentAppUpdatePayload.__name__])
@console_ns.response(200, "Agent app updated successfully", console_ns.models[AgentAppDetailWithSite.__name__])
@@ -643,116 +578,8 @@ class AgentDebugConversationRefreshApi(Resource):
agent_id=str(agent_id),
account_id=current_user.id,
)
return AgentDebugConversationRefreshResponse(
debug_conversation_id=debug_conversation_id,
debug_conversation_has_messages=False,
debug_conversation_message_count=0,
).model_dump(mode="json")
@console_ns.route("/agent/<uuid:agent_id>/publish")
class AgentPublishApi(Resource):
@console_ns.expect(console_ns.models[AgentPublishPayload.__name__])
@console_ns.response(200, "Agent draft published", console_ns.models[AgentPublishResponse.__name__])
@console_ns.response(403, "Insufficient permissions")
@setup_required
@login_required
@account_initialization_required
@edit_permission_required
@with_current_user
@with_current_tenant_id
def post(self, tenant_id: str, current_user: Account, agent_id: UUID):
args = AgentPublishPayload.model_validate(console_ns.payload or {})
return AgentComposerService.publish_agent_app_draft(
tenant_id=tenant_id,
agent_id=str(agent_id),
account_id=current_user.id,
version_note=args.version_note,
)
@console_ns.route("/agent/<uuid:agent_id>/build-draft/checkout")
class AgentBuildDraftCheckoutApi(Resource):
@console_ns.expect(console_ns.models[AgentBuildDraftCheckoutPayload.__name__])
@console_ns.response(200, "Agent build draft checked out", console_ns.models[AgentBuildDraftResponse.__name__])
@setup_required
@login_required
@account_initialization_required
@edit_permission_required
@with_current_user
@with_current_tenant_id
def post(self, tenant_id: str, current_user: Account, agent_id: UUID):
args = AgentBuildDraftCheckoutPayload.model_validate(console_ns.payload or {})
return AgentComposerService.checkout_agent_app_build_draft(
tenant_id=tenant_id,
agent_id=str(agent_id),
account_id=current_user.id,
force=args.force,
)
@console_ns.route("/agent/<uuid:agent_id>/build-draft")
class AgentBuildDraftApi(Resource):
@console_ns.response(200, "Agent build draft", console_ns.models[AgentBuildDraftResponse.__name__])
@setup_required
@login_required
@account_initialization_required
@edit_permission_required
@with_current_user
@with_current_tenant_id
def get(self, tenant_id: str, current_user: Account, agent_id: UUID):
return AgentComposerService.load_agent_app_build_draft(
tenant_id=tenant_id,
agent_id=str(agent_id),
account_id=current_user.id,
)
@console_ns.expect(console_ns.models[ComposerSavePayload.__name__])
@console_ns.response(200, "Agent build draft saved", console_ns.models[AgentBuildDraftResponse.__name__])
@setup_required
@login_required
@account_initialization_required
@edit_permission_required
@with_current_user
@with_current_tenant_id
def put(self, tenant_id: str, current_user: Account, agent_id: UUID):
payload = ComposerSavePayload.model_validate(console_ns.payload or {})
return AgentComposerService.save_agent_app_build_draft(
tenant_id=tenant_id,
agent_id=str(agent_id),
account_id=current_user.id,
payload=payload,
)
@console_ns.response(200, "Agent build draft discarded", console_ns.models[AgentSimpleResultResponse.__name__])
@setup_required
@login_required
@account_initialization_required
@edit_permission_required
@with_current_user
@with_current_tenant_id
def delete(self, tenant_id: str, current_user: Account, agent_id: UUID):
return AgentComposerService.discard_agent_app_build_draft(
tenant_id=tenant_id,
agent_id=str(agent_id),
account_id=current_user.id,
)
@console_ns.route("/agent/<uuid:agent_id>/build-draft/apply")
class AgentBuildDraftApplyApi(Resource):
@console_ns.response(200, "Agent build draft applied", console_ns.models[AgentBuildDraftApplyResponse.__name__])
@setup_required
@login_required
@account_initialization_required
@edit_permission_required
@with_current_user
@with_current_tenant_id
def post(self, tenant_id: str, current_user: Account, agent_id: UUID):
return AgentComposerService.apply_agent_app_build_draft(
tenant_id=tenant_id,
agent_id=str(agent_id),
account_id=current_user.id,
return AgentDebugConversationRefreshResponse(debug_conversation_id=debug_conversation_id).model_dump(
mode="json"
)
@@ -885,10 +712,10 @@ class AgentLogsApi(Resource):
@with_current_user
@with_current_tenant_id
def get(self, tenant_id: str, current_user: Account, agent_id: UUID):
app_model = resolve_agent_runtime_app_model(tenant_id=tenant_id, agent_id=agent_id)
app_model = _resolve_agent_app_model(tenant_id=tenant_id, agent_id=agent_id)
query_data: dict[str, object] = dict(request.args.to_dict(flat=True))
query_data["sources"] = _query_values("sources", "source")
query_data["statuses"] = _query_values("statuses", "status")
query_data["sources"] = _multi_query_values("sources", "source")
query_data["statuses"] = _multi_query_values("statuses", "status")
query = AgentLogsQuery.model_validate(query_data)
start, end = _parse_observability_time_range(query.start, query.end, current_user)
try:
@@ -922,10 +749,10 @@ class AgentLogMessagesApi(Resource):
@with_current_user
@with_current_tenant_id
def get(self, tenant_id: str, current_user: Account, agent_id: UUID, conversation_id: UUID):
app_model = resolve_agent_runtime_app_model(tenant_id=tenant_id, agent_id=agent_id)
app_model = _resolve_agent_app_model(tenant_id=tenant_id, agent_id=agent_id)
query_data: dict[str, object] = dict(request.args.to_dict(flat=True))
query_data["sources"] = _query_values("sources", "source")
query_data["statuses"] = _query_values("statuses", "status")
query_data["sources"] = _multi_query_values("sources", "source")
query_data["statuses"] = _multi_query_values("statuses", "status")
query = AgentLogsQuery.model_validate(query_data)
start, end = _parse_observability_time_range(query.start, query.end, current_user)
try:
@@ -959,7 +786,7 @@ class AgentLogSourcesApi(Resource):
@with_current_user
@with_current_tenant_id
def get(self, tenant_id: str, current_user: Account, agent_id: UUID):
app_model = resolve_agent_runtime_app_model(tenant_id=tenant_id, agent_id=agent_id)
app_model = _resolve_agent_app_model(tenant_id=tenant_id, agent_id=agent_id)
payload = _agent_observability_service().list_log_sources(app=app_model, agent_id=str(agent_id))
return dump_response(AgentLogSourceListResponse, payload)
@@ -978,7 +805,7 @@ class AgentStatisticsSummaryApi(Resource):
@with_current_user
@with_current_tenant_id
def get(self, tenant_id: str, current_user: Account, agent_id: UUID):
app_model = resolve_agent_runtime_app_model(tenant_id=tenant_id, agent_id=agent_id)
app_model = _resolve_agent_app_model(tenant_id=tenant_id, agent_id=agent_id)
query = AgentStatisticsQuery.model_validate(request.args.to_dict(flat=True))
timezone = current_user.timezone or "UTC"
start, end = _parse_observability_time_range(query.start, query.end, current_user)
+6 -6
View File
@@ -13,7 +13,7 @@ from controllers.common.schema import (
register_schema_models,
)
from controllers.console import console_ns
from controllers.console.agent.app_helpers import resolve_agent_runtime_app_model
from controllers.console.agent.app_helpers import resolve_agent_app_model
from controllers.console.app.wraps import get_app_model
from controllers.console.wraps import (
RBACPermission,
@@ -351,7 +351,7 @@ class AgentSkillUploadByAgentApi(Resource):
@with_current_user
@with_current_tenant_id
def post(self, tenant_id: str, current_user: Account, agent_id: UUID):
app_model = resolve_agent_runtime_app_model(tenant_id=tenant_id, agent_id=agent_id)
app_model = resolve_agent_app_model(tenant_id=tenant_id, agent_id=agent_id)
return _upload_skill_for_app(current_user=current_user, app_model=app_model)
@@ -394,7 +394,7 @@ class AgentDriveFilesByAgentApi(Resource):
@with_current_user
@with_current_tenant_id
def post(self, tenant_id: str, current_user: Account, agent_id: UUID):
app_model = resolve_agent_runtime_app_model(tenant_id=tenant_id, agent_id=agent_id)
app_model = resolve_agent_app_model(tenant_id=tenant_id, agent_id=agent_id)
return _commit_drive_file_for_app(current_user=current_user, app_model=app_model, allow_node_id=False)
@console_ns.doc("delete_agent_drive_file_by_agent")
@@ -407,7 +407,7 @@ class AgentDriveFilesByAgentApi(Resource):
@with_current_user
@with_current_tenant_id
def delete(self, tenant_id: str, current_user: Account, agent_id: UUID):
app_model = resolve_agent_runtime_app_model(tenant_id=tenant_id, agent_id=agent_id)
app_model = resolve_agent_app_model(tenant_id=tenant_id, agent_id=agent_id)
return _delete_drive_file_for_app(current_user=current_user, app_model=app_model, allow_node_id=False)
@@ -454,7 +454,7 @@ class AgentSkillByAgentApi(Resource):
@with_current_user
@with_current_tenant_id
def delete(self, tenant_id: str, current_user: Account, agent_id: UUID, slug: str):
app_model = resolve_agent_runtime_app_model(tenant_id=tenant_id, agent_id=agent_id)
app_model = resolve_agent_app_model(tenant_id=tenant_id, agent_id=agent_id)
return _delete_skill_for_app(current_user=current_user, app_model=app_model, slug=slug, allow_node_id=False)
@@ -494,7 +494,7 @@ class AgentSkillInferToolsByAgentApi(Resource):
@account_initialization_required
@with_current_tenant_id
def post(self, tenant_id: str, agent_id: UUID, slug: str):
app_model = resolve_agent_runtime_app_model(tenant_id=tenant_id, agent_id=agent_id)
app_model = resolve_agent_app_model(tenant_id=tenant_id, agent_id=agent_id)
return _infer_skill_tools_for_app(app_model=app_model, slug=slug)
@@ -17,7 +17,7 @@ from pydantic import BaseModel, Field
from controllers.common.fields import SimpleResultResponse
from controllers.common.schema import register_response_schema_models, register_schema_models
from controllers.console import console_ns
from controllers.console.agent.app_helpers import resolve_agent_runtime_app_model
from controllers.console.agent.app_helpers import resolve_agent_app_model
from controllers.console.wraps import (
RBACPermission,
RBACResourceScope,
@@ -30,6 +30,7 @@ from controllers.console.wraps import (
)
from events.app_event import app_model_config_was_updated
from extensions.ext_database import db
from libs.helper import dump_response
from libs.login import login_required
from models import Account
from models.agent_config_entities import (
@@ -86,7 +87,7 @@ class AgentAppFeatureConfigResource(Resource):
@with_current_user
@with_current_tenant_id
def post(self, tenant_id: str, current_user: Account, agent_id: UUID):
app_model = resolve_agent_runtime_app_model(tenant_id=tenant_id, agent_id=agent_id)
app_model = resolve_agent_app_model(tenant_id=tenant_id, agent_id=agent_id)
args = AgentAppFeaturesPayload.model_validate(console_ns.payload or {})
new_app_model_config = AgentAppFeatureConfigService.update_features(
@@ -98,4 +99,4 @@ class AgentAppFeatureConfigResource(Resource):
app_model_config_was_updated.send(app_model, app_model_config=new_app_model_config)
return SimpleResultResponse(result="success").model_dump(mode="json")
return dump_response(SimpleResultResponse, {"result": "success"})
@@ -22,7 +22,7 @@ from controllers.common.schema import (
register_schema_models,
)
from controllers.console import console_ns
from controllers.console.agent.app_helpers import resolve_agent_runtime_app_model
from controllers.console.agent.app_helpers import resolve_agent_app_model
from controllers.console.app.wraps import get_app_model
from controllers.console.wraps import account_initialization_required, setup_required, with_current_tenant_id
from fields.base import ResponseModel
@@ -44,10 +44,6 @@ class AgentSandboxListQuery(BaseModel):
path: str = Field(default=".", description="Directory path relative to the sandbox workspace")
class AgentSandboxInfoQuery(BaseModel):
conversation_id: str = Field(min_length=1, description="Agent App conversation ID")
class AgentSandboxFileQuery(BaseModel):
conversation_id: str = Field(min_length=1, description="Agent App conversation ID")
path: str = Field(min_length=1, description="File path relative to the sandbox workspace")
@@ -95,11 +91,6 @@ class SandboxListResponse(ResponseModel):
truncated: bool = False
class SandboxInfoResponse(ResponseModel):
session_id: str
workspace_cwd: str
class SandboxReadResponse(ResponseModel):
path: str
size: int | None = None
@@ -123,13 +114,7 @@ register_schema_models(
AgentSandboxUploadPayload,
WorkflowAgentSandboxUploadPayload,
)
register_response_schema_models(
console_ns,
SandboxInfoResponse,
SandboxListResponse,
SandboxReadResponse,
SandboxUploadResponse,
)
register_response_schema_models(console_ns, SandboxListResponse, SandboxReadResponse, SandboxUploadResponse)
def _handle(exc: Exception) -> tuple[dict[str, object], int]:
@@ -148,30 +133,6 @@ def _handle(exc: Exception) -> tuple[dict[str, object], int]:
raise exc
@console_ns.route("/agent/<uuid:agent_id>/sandbox")
class AgentAppSandboxInfoResource(Resource):
@console_ns.doc("get_agent_app_sandbox_info")
@console_ns.doc(description="Get basic information for an Agent App conversation sandbox")
@console_ns.doc(params={"agent_id": "Agent ID", **query_params_from_model(AgentSandboxInfoQuery)})
@console_ns.response(200, "Sandbox information returned", console_ns.models[SandboxInfoResponse.__name__])
@setup_required
@login_required
@account_initialization_required
@with_current_tenant_id
def get(self, tenant_id: str, agent_id: UUID):
app_model = resolve_agent_runtime_app_model(tenant_id=tenant_id, agent_id=agent_id)
query = query_params_from_request(AgentSandboxInfoQuery)
try:
result = AgentAppSandboxService().get_info(
tenant_id=tenant_id,
app_id=app_model.id,
conversation_id=query.conversation_id,
)
except Exception as exc:
return _handle(exc)
return result.model_dump()
@console_ns.route("/agent/<uuid:agent_id>/sandbox/files")
class AgentAppSandboxListResource(Resource):
@console_ns.doc("list_agent_app_sandbox_files")
@@ -183,7 +144,7 @@ class AgentAppSandboxListResource(Resource):
@account_initialization_required
@with_current_tenant_id
def get(self, tenant_id: str, agent_id: UUID):
app_model = resolve_agent_runtime_app_model(tenant_id=tenant_id, agent_id=agent_id)
app_model = resolve_agent_app_model(tenant_id=tenant_id, agent_id=agent_id)
query = query_params_from_request(AgentSandboxListQuery)
try:
result = AgentAppSandboxService().list_files(
@@ -208,7 +169,7 @@ class AgentAppSandboxReadResource(Resource):
@account_initialization_required
@with_current_tenant_id
def get(self, tenant_id: str, agent_id: UUID):
app_model = resolve_agent_runtime_app_model(tenant_id=tenant_id, agent_id=agent_id)
app_model = resolve_agent_app_model(tenant_id=tenant_id, agent_id=agent_id)
query = query_params_from_request(AgentSandboxFileQuery)
try:
result = AgentAppSandboxService().read_file(
@@ -233,7 +194,7 @@ class AgentAppSandboxUploadResource(Resource):
@account_initialization_required
@with_current_tenant_id
def post(self, tenant_id: str, agent_id: UUID):
app_model = resolve_agent_runtime_app_model(tenant_id=tenant_id, agent_id=agent_id)
app_model = resolve_agent_app_model(tenant_id=tenant_id, agent_id=agent_id)
payload = AgentSandboxUploadPayload.model_validate(request.get_json(silent=True) or {})
try:
result = AgentAppSandboxService().upload_file(
File diff suppressed because it is too large Load Diff
@@ -25,7 +25,7 @@ from controllers.common.schema import (
register_response_schema_models,
)
from controllers.console import console_ns
from controllers.console.agent.app_helpers import resolve_agent_runtime_app_model
from controllers.console.agent.app_helpers import resolve_agent_app_model
from controllers.console.app.wraps import get_app_model
from controllers.console.wraps import account_initialization_required, setup_required, with_current_tenant_id
from fields.base import ResponseModel
@@ -182,7 +182,7 @@ class AgentDriveListByAgentApi(Resource):
@with_current_tenant_id
def get(self, tenant_id: str, agent_id: UUID):
query = query_params_from_request(AgentDriveListByAgentQuery)
resolve_agent_runtime_app_model(tenant_id=tenant_id, agent_id=agent_id)
resolve_agent_app_model(tenant_id=tenant_id, agent_id=agent_id)
try:
items = AgentDriveService().manifest(tenant_id=tenant_id, agent_id=str(agent_id), prefix=query.prefix)
except AgentDriveError as exc:
@@ -201,7 +201,7 @@ class AgentDriveSkillListByAgentApi(Resource):
@account_initialization_required
@with_current_tenant_id
def get(self, tenant_id: str, agent_id: UUID):
resolve_agent_runtime_app_model(tenant_id=tenant_id, agent_id=agent_id)
resolve_agent_app_model(tenant_id=tenant_id, agent_id=agent_id)
try:
items = AgentDriveService().list_skills(tenant_id=tenant_id, agent_id=str(agent_id))
except AgentDriveError as exc:
@@ -220,7 +220,7 @@ class AgentDriveSkillInspectByAgentApi(Resource):
@account_initialization_required
@with_current_tenant_id
def get(self, tenant_id: str, agent_id: UUID, skill_path: str):
resolve_agent_runtime_app_model(tenant_id=tenant_id, agent_id=agent_id)
resolve_agent_app_model(tenant_id=tenant_id, agent_id=agent_id)
try:
return _json_response(
AgentDriveService().inspect_skill(
@@ -245,7 +245,7 @@ class AgentDrivePreviewByAgentApi(Resource):
@with_current_tenant_id
def get(self, tenant_id: str, agent_id: UUID):
query = query_params_from_request(AgentDriveFileByAgentQuery)
resolve_agent_runtime_app_model(tenant_id=tenant_id, agent_id=agent_id)
resolve_agent_app_model(tenant_id=tenant_id, agent_id=agent_id)
try:
return AgentDriveService().preview(tenant_id=tenant_id, agent_id=str(agent_id), key=query.key)
except AgentDriveError as exc:
@@ -264,7 +264,7 @@ class AgentDriveDownloadByAgentApi(Resource):
@with_current_tenant_id
def get(self, tenant_id: str, agent_id: UUID):
query = query_params_from_request(AgentDriveFileByAgentQuery)
resolve_agent_runtime_app_model(tenant_id=tenant_id, agent_id=agent_id)
resolve_agent_app_model(tenant_id=tenant_id, agent_id=agent_id)
try:
url = AgentDriveService().download_url(tenant_id=tenant_id, agent_id=str(agent_id), key=query.key)
except AgentDriveError as exc:
+59 -76
View File
@@ -1,11 +1,9 @@
from typing import Any, Literal
from uuid import UUID
from flask import abort, request
from flask import abort, make_response, request
from flask_restx import Resource
from pydantic import BaseModel, Field, TypeAdapter, field_validator
from sqlalchemy import select
from werkzeug.exceptions import NotFound
from controllers.common.errors import NoFileUploadedError, TooManyFilesError
from controllers.common.schema import query_params_from_model, register_response_schema_models, register_schema_models
@@ -28,14 +26,11 @@ from fields.annotation_fields import (
AnnotationExportList,
AnnotationHitHistory,
AnnotationHitHistoryList,
AnnotationJobStatusDetailResponse,
AnnotationJobStatusResponse,
AnnotationList,
)
from fields.base import ResponseModel
from libs.helper import dump_response, uuid_value
from libs.login import current_account_with_tenant, login_required
from models.model import App
from libs.helper import uuid_value
from libs.login import login_required
from services.annotation_service import (
AppAnnotationService,
EnableAnnotationArgs,
@@ -43,17 +38,6 @@ from services.annotation_service import (
UpdateAnnotationSettingArgs,
UpsertAnnotationArgs,
)
from services.app_ref_service import AppRef, AppRefService
def _get_app_ref(app_id: str) -> AppRef:
_, current_tenant_id = current_account_with_tenant()
app = db.session.scalar(
select(App).where(App.id == app_id, App.tenant_id == current_tenant_id, App.status == "normal").limit(1)
)
if app is None:
raise NotFound("App not found")
return AppRefService.create_app_ref(app)
class AnnotationReplyPayload(BaseModel):
@@ -115,23 +99,23 @@ class AnnotationFilePayload(BaseModel):
return uuid_value(value)
class AnnotationSettingEmbeddingModelResponse(ResponseModel):
class AnnotationJobStatusResponse(ResponseModel):
job_id: str | None = None
job_status: str | None = None
error_msg: str | None = None
record_count: int | None = None
class AnnotationEmbeddingModelResponse(ResponseModel):
embedding_provider_name: str | None = None
embedding_model_name: str | None = None
class AnnotationSettingResponse(ResponseModel):
enabled: bool
id: str | None = None
enabled: bool
score_threshold: float | None = None
embedding_model: AnnotationSettingEmbeddingModelResponse | None = None
class AnnotationBatchImportResponse(ResponseModel):
job_id: str | None = None
job_status: str | None = None
record_count: int | None = None
error_msg: str | None = None
embedding_model: AnnotationEmbeddingModelResponse | None = None
register_schema_models(
@@ -158,10 +142,7 @@ register_response_schema_models(
AnnotationHitHistory,
AnnotationHitHistoryList,
AnnotationJobStatusResponse,
AnnotationJobStatusDetailResponse,
AnnotationSettingEmbeddingModelResponse,
AnnotationSettingResponse,
AnnotationBatchImportResponse,
)
@@ -191,7 +172,7 @@ class AnnotationReplyActionApi(Resource):
result = AppAnnotationService.enable_app_annotation(enable_args, str(app_id))
case "disable":
result = AppAnnotationService.disable_app_annotation(str(app_id))
return dump_response(AnnotationJobStatusResponse, result), 200
return result, 200
@console_ns.route("/apps/<uuid:app_id>/annotation-setting")
@@ -212,7 +193,7 @@ class AppAnnotationSettingDetailApi(Resource):
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_VIEW_LAYOUT)
def get(self, app_id: UUID):
result = AppAnnotationService.get_app_annotation_setting_by_app_id(str(app_id))
return dump_response(AnnotationSettingResponse, result), 200
return result, 200
@console_ns.route("/apps/<uuid:app_id>/annotation-settings/<uuid:annotation_setting_id>")
@@ -237,7 +218,7 @@ class AppAnnotationSettingUpdateApi(Resource):
result = AppAnnotationService.update_app_annotation_setting(
str(app_id), annotation_setting_id_str, setting_args
)
return dump_response(AnnotationSettingResponse, result), 200
return result, 200
@console_ns.route("/apps/<uuid:app_id>/annotation-reply/<string:action>/status/<uuid:job_id>")
@@ -246,7 +227,9 @@ class AnnotationReplyActionStatusApi(Resource):
@console_ns.doc(description="Get status of annotation reply action job")
@console_ns.doc(params={"app_id": "Application ID", "job_id": "Job ID", "action": "Action type"})
@console_ns.response(
200, "Job status retrieved successfully", console_ns.models[AnnotationJobStatusDetailResponse.__name__]
200,
"Job status retrieved successfully",
console_ns.models[AnnotationJobStatusResponse.__name__],
)
@console_ns.response(403, "Insufficient permissions")
@setup_required
@@ -268,9 +251,7 @@ class AnnotationReplyActionStatusApi(Resource):
app_annotation_error_key = f"{action}_app_annotation_error_{job_id_str}"
error_msg = redis_client.get(app_annotation_error_key).decode()
return AnnotationJobStatusDetailResponse(
job_id=job_id_str, job_status=job_status, error_msg=error_msg
).model_dump(mode="json"), 200
return {"job_id": job_id_str, "job_status": job_status, "error_msg": error_msg}, 200
@console_ns.route("/apps/<uuid:app_id>/annotations")
@@ -294,9 +275,14 @@ class AnnotationApi(Resource):
annotation_list, total = AppAnnotationService.get_annotation_list_by_app_id(str(app_id), page, limit, keyword)
annotation_models = TypeAdapter(list[Annotation]).validate_python(annotation_list, from_attributes=True)
return AnnotationList(
data=annotation_models, has_more=len(annotation_list) == limit, limit=limit, total=total, page=page
).model_dump(mode="json"), 200
response = AnnotationList(
data=annotation_models,
has_more=len(annotation_list) == limit,
limit=limit,
total=total,
page=page,
)
return response.model_dump(mode="json"), 200
@console_ns.doc("create_annotation")
@console_ns.doc(description="Create a new annotation for an app")
@@ -322,7 +308,7 @@ class AnnotationApi(Resource):
if args.question is not None:
upsert_args["question"] = args.question
annotation = AppAnnotationService.up_insert_app_annotation_from_message(upsert_args, str(app_id))
return dump_response(Annotation, annotation), 201
return Annotation.model_validate(annotation, from_attributes=True).model_dump(mode="json")
@setup_required
@login_required
@@ -344,8 +330,7 @@ class AnnotationApi(Resource):
"message": "annotation_ids are required if the parameter is provided.",
}, 400
app_ref = _get_app_ref(str(app_id))
AppAnnotationService.delete_app_annotations_in_batch(app_ref, annotation_ids)
AppAnnotationService.delete_app_annotations_in_batch(str(app_id), annotation_ids)
return "", 204
# If no annotation_ids are provided, handle clearing all annotations
else:
@@ -372,14 +357,14 @@ class AnnotationExportApi(Resource):
def get(self, app_id: UUID):
annotation_list = AppAnnotationService.export_annotation_list_by_app_id(str(app_id))
annotation_models = TypeAdapter(list[Annotation]).validate_python(annotation_list, from_attributes=True)
return (
AnnotationExportList(data=annotation_models).model_dump(mode="json"),
200,
{
"Content-Type": "application/json; charset=utf-8",
"X-Content-Type-Options": "nosniff",
},
)
response_data = AnnotationExportList(data=annotation_models).model_dump(mode="json")
# Create response with secure headers for CSV export
response = make_response(response_data, 200)
response.headers["Content-Type"] = "application/json; charset=utf-8"
response.headers["X-Content-Type-Options"] = "nosniff"
return response
@console_ns.route("/apps/<uuid:app_id>/annotations/<uuid:annotation_id>")
@@ -404,9 +389,9 @@ class AnnotationUpdateDeleteApi(Resource):
update_args["answer"] = args.answer
if args.question is not None:
update_args["question"] = args.question
app_ref = _get_app_ref(str(app_id))
annotation_ref = AppRefService.create_annotation_ref(app_ref, str(annotation_id))
annotation = AppAnnotationService.update_app_annotation_directly(update_args, annotation_ref, db.session)
annotation = AppAnnotationService.update_app_annotation_directly(
update_args, str(app_id), str(annotation_id), db.session
)
return Annotation.model_validate(annotation, from_attributes=True).model_dump(mode="json")
@setup_required
@@ -416,9 +401,7 @@ class AnnotationUpdateDeleteApi(Resource):
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_EDIT)
@console_ns.response(204, "Annotation deleted successfully")
def delete(self, app_id: UUID, annotation_id: UUID):
app_ref = _get_app_ref(str(app_id))
annotation_ref = AppRefService.create_annotation_ref(app_ref, str(annotation_id))
AppAnnotationService.delete_app_annotation(annotation_ref, db.session)
AppAnnotationService.delete_app_annotation(str(app_id), str(annotation_id), db.session)
return "", 204
@@ -428,7 +411,9 @@ class AnnotationBatchImportApi(Resource):
@console_ns.doc(description="Batch import annotations from CSV file with rate limiting and security checks")
@console_ns.doc(params={"app_id": "Application ID"})
@console_ns.response(
200, "Batch import started successfully", console_ns.models[AnnotationBatchImportResponse.__name__]
200,
"Batch import started successfully",
console_ns.models[AnnotationJobStatusResponse.__name__],
)
@console_ns.response(403, "Insufficient permissions")
@console_ns.response(400, "No file uploaded or too many files")
@@ -475,10 +460,7 @@ class AnnotationBatchImportApi(Resource):
if file_size == 0:
raise ValueError("The uploaded file is empty")
return dump_response(
AnnotationBatchImportResponse,
AppAnnotationService.batch_import_app_annotations(str(app_id), file),
)
return AppAnnotationService.batch_import_app_annotations(str(app_id), file)
@console_ns.route("/apps/<uuid:app_id>/annotations/batch-import-status/<uuid:job_id>")
@@ -487,7 +469,9 @@ class AnnotationBatchImportStatusApi(Resource):
@console_ns.doc(description="Get status of batch import job")
@console_ns.doc(params={"app_id": "Application ID", "job_id": "Job ID"})
@console_ns.response(
200, "Job status retrieved successfully", console_ns.models[AnnotationJobStatusDetailResponse.__name__]
200,
"Job status retrieved successfully",
console_ns.models[AnnotationJobStatusResponse.__name__],
)
@console_ns.response(403, "Insufficient permissions")
@setup_required
@@ -507,9 +491,7 @@ class AnnotationBatchImportStatusApi(Resource):
indexing_error_msg_key = f"app_annotation_batch_import_error_msg_{str(job_id)}"
error_msg = redis_client.get(indexing_error_msg_key).decode()
return AnnotationJobStatusDetailResponse(
job_id=str(job_id), job_status=job_status, error_msg=error_msg
).model_dump(mode="json"), 200
return {"job_id": job_id, "job_status": job_status, "error_msg": error_msg}, 200
@console_ns.route("/apps/<uuid:app_id>/annotations/<uuid:annotation_id>/hit-histories")
@@ -532,16 +514,17 @@ class AnnotationHitHistoryListApi(Resource):
def get(self, app_id: UUID, annotation_id: UUID):
page = request.args.get("page", default=1, type=int)
limit = request.args.get("limit", default=20, type=int)
app_ref = _get_app_ref(str(app_id))
annotation_ref = AppRefService.create_annotation_ref(app_ref, str(annotation_id))
annotation_hit_history_list, total = AppAnnotationService.get_annotation_hit_histories(
annotation_ref,
page,
limit,
str(app_id), str(annotation_id), page, limit
)
history_models = TypeAdapter(list[AnnotationHitHistory]).validate_python(
annotation_hit_history_list, from_attributes=True
)
return AnnotationHitHistoryList(
data=history_models, has_more=len(annotation_hit_history_list) == limit, limit=limit, total=total, page=page
).model_dump(mode="json")
response = AnnotationHitHistoryList(
data=history_models,
has_more=len(annotation_hit_history_list) == limit,
limit=limit,
total=total,
page=page,
)
return response.model_dump(mode="json")
+86 -58
View File
@@ -1,4 +1,5 @@
import logging
import re
import uuid
from collections.abc import Sequence
from datetime import datetime
@@ -9,6 +10,7 @@ from flask_restx import Resource
from pydantic import AliasChoices, BaseModel, Field, computed_field, field_validator
from sqlalchemy import select
from sqlalchemy.orm import Session
from werkzeug.datastructures import MultiDict
from werkzeug.exceptions import BadRequest, NotFound
from configs import dify_config
@@ -17,7 +19,6 @@ from controllers.common.fields import RedirectUrlResponse, SimpleResultResponse
from controllers.common.helpers import FileInfo
from controllers.common.schema import (
query_params_from_model,
query_params_from_request,
register_enum_models,
register_response_schema_models,
register_schema_models,
@@ -74,9 +75,10 @@ ALLOW_CREATE_APP_MODES = ["chat", "agent-chat", "advanced-chat", "workflow", "co
register_enum_models(console_ns, IconType)
_logger = logging.getLogger(__name__)
_TAG_IDS_BRACKET_PATTERN = re.compile(r"^tag_ids\[(\d+)\]$")
_CREATOR_IDS_BRACKET_PATTERN = re.compile(r"^creator_ids\[(\d+)\]$")
AppListMode = Literal["completion", "chat", "advanced-chat", "workflow", "agent-chat", "agent", "channel", "all"]
DEFAULT_APP_LIST_MODE: AppListMode = "all"
APP_LIST_QUERY_ARRAY_FIELDS = ("tag_ids", "creator_ids")
class AppListBaseQuery(BaseModel):
@@ -137,6 +139,34 @@ class StarredAppListQuery(AppListBaseQuery):
pass
def _normalize_app_list_query_args(query_args: MultiDict[str, str]) -> dict[str, str | list[str]]:
normalized: dict[str, str | list[str]] = {}
indexed_tag_ids: list[tuple[int, str]] = []
indexed_creator_ids: list[tuple[int, str]] = []
for key in query_args:
match = _TAG_IDS_BRACKET_PATTERN.fullmatch(key)
if match:
indexed_tag_ids.extend((int(match.group(1)), value) for value in query_args.getlist(key))
continue
match = _CREATOR_IDS_BRACKET_PATTERN.fullmatch(key)
if match:
indexed_creator_ids.extend((int(match.group(1)), value) for value in query_args.getlist(key))
continue
value = query_args.get(key)
if value is not None:
normalized[key] = value
if indexed_tag_ids:
normalized["tag_ids"] = [value for _, value in sorted(indexed_tag_ids)]
if indexed_creator_ids:
normalized["creator_ids"] = [value for _, value in sorted(indexed_creator_ids)]
return normalized
class CreateAppPayload(BaseModel):
name: str = Field(..., min_length=1, description="App name")
description: str | None = Field(default=None, description="App description (max 400 chars)", max_length=400)
@@ -200,10 +230,13 @@ class AppTracePayload(BaseModel):
class AppTraceResponse(ResponseModel):
enabled: bool = False
enabled: bool
tracing_provider: str | None = None
type JSONValue = Any
class Tag(ResponseModel):
id: str
name: str
@@ -224,7 +257,7 @@ class WorkflowPartial(ResponseModel):
class ModelConfigPartial(ResponseModel):
model: Any | None = Field(default=None, validation_alias=AliasChoices("model_dict", "model"))
model: JSONValue | None = Field(default=None, validation_alias=AliasChoices("model_dict", "model"))
pre_prompt: str | None = None
created_by: str | None = None
created_at: int | None = None
@@ -239,52 +272,54 @@ class ModelConfigPartial(ResponseModel):
class ModelConfig(ResponseModel):
opening_statement: str | None = None
suggested_questions: Any | None = Field(
suggested_questions: JSONValue | None = Field(
default=None, validation_alias=AliasChoices("suggested_questions_list", "suggested_questions")
)
suggested_questions_after_answer: Any | None = Field(
suggested_questions_after_answer: JSONValue | None = Field(
default=None,
validation_alias=AliasChoices("suggested_questions_after_answer_dict", "suggested_questions_after_answer"),
)
speech_to_text: Any | None = Field(
speech_to_text: JSONValue | None = Field(
default=None, validation_alias=AliasChoices("speech_to_text_dict", "speech_to_text")
)
text_to_speech: Any | None = Field(
text_to_speech: JSONValue | None = Field(
default=None, validation_alias=AliasChoices("text_to_speech_dict", "text_to_speech")
)
retriever_resource: Any | None = Field(
retriever_resource: JSONValue | None = Field(
default=None, validation_alias=AliasChoices("retriever_resource_dict", "retriever_resource")
)
annotation_reply: Any | None = Field(
annotation_reply: JSONValue | None = Field(
default=None, validation_alias=AliasChoices("annotation_reply_dict", "annotation_reply")
)
more_like_this: Any | None = Field(
more_like_this: JSONValue | None = Field(
default=None, validation_alias=AliasChoices("more_like_this_dict", "more_like_this")
)
sensitive_word_avoidance: Any | None = Field(
sensitive_word_avoidance: JSONValue | None = Field(
default=None, validation_alias=AliasChoices("sensitive_word_avoidance_dict", "sensitive_word_avoidance")
)
external_data_tools: Any | None = Field(
external_data_tools: JSONValue | None = Field(
default=None, validation_alias=AliasChoices("external_data_tools_list", "external_data_tools")
)
model: Any | None = Field(default=None, validation_alias=AliasChoices("model_dict", "model"))
user_input_form: Any | None = Field(
model: JSONValue | None = Field(default=None, validation_alias=AliasChoices("model_dict", "model"))
user_input_form: JSONValue | None = Field(
default=None, validation_alias=AliasChoices("user_input_form_list", "user_input_form")
)
dataset_query_variable: str | None = None
pre_prompt: str | None = None
agent_mode: Any | None = Field(default=None, validation_alias=AliasChoices("agent_mode_dict", "agent_mode"))
agent_mode: JSONValue | None = Field(default=None, validation_alias=AliasChoices("agent_mode_dict", "agent_mode"))
prompt_type: str | None = None
chat_prompt_config: Any | None = Field(
chat_prompt_config: JSONValue | None = Field(
default=None, validation_alias=AliasChoices("chat_prompt_config_dict", "chat_prompt_config")
)
completion_prompt_config: Any | None = Field(
completion_prompt_config: JSONValue | None = Field(
default=None, validation_alias=AliasChoices("completion_prompt_config_dict", "completion_prompt_config")
)
dataset_configs: Any | None = Field(
dataset_configs: JSONValue | None = Field(
default=None, validation_alias=AliasChoices("dataset_configs_dict", "dataset_configs")
)
file_upload: Any | None = Field(default=None, validation_alias=AliasChoices("file_upload_dict", "file_upload"))
file_upload: JSONValue | None = Field(
default=None, validation_alias=AliasChoices("file_upload_dict", "file_upload")
)
created_by: str | None = None
created_at: int | None = None
updated_by: str | None = None
@@ -296,7 +331,7 @@ class ModelConfig(ResponseModel):
return to_timestamp(value)
class AppDetailSiteResponse(ResponseModel):
class Site(ResponseModel):
access_token: str | None = Field(default=None, validation_alias="code")
code: str | None = None
title: str | None = None
@@ -310,7 +345,6 @@ class AppDetailSiteResponse(ResponseModel):
customize_domain: str | None = None
copyright: str | None = None
privacy_policy: str | None = None
input_placeholder: str | None = None
custom_disclaimer: str | None = None
customize_token_strategy: str | None = None
prompt_public: bool | None = None
@@ -405,7 +439,7 @@ class AppDetail(ResponseModel):
alias="model_config",
)
workflow: WorkflowPartial | None = None
tracing: Any | None = None
tracing: JSONValue | None = None
use_icon_as_answer_icon: bool | None = None
created_by: str | None = None
created_at: int | None = None
@@ -427,7 +461,7 @@ class AppDetailWithSite(AppDetail):
api_base_url: str | None = None
max_active_requests: int | None = None
deleted_tools: list[DeletedTool] = Field(default_factory=list)
site: AppDetailSiteResponse | None = None
site: Site | None = None
# For Agent App type: the roster Agent backing this app (None otherwise).
bound_agent_id: str | None = None
# For Agent App responses exposed through /agent.
@@ -451,16 +485,6 @@ class AppExportResponse(ResponseModel):
data: str
class AppImportResponse(ResponseModel):
id: str
status: ImportStatus
app_id: str | None = None
app_mode: str | None = None
current_dsl_version: str
imported_dsl_version: str = ""
error: str = ""
def _enrich_app_list_items(session: Session, *, apps: Sequence[App], tenant_id: str) -> None:
if FeatureService.get_system_features().webapp_auth.enabled:
app_ids = [str(app.id) for app in apps]
@@ -503,9 +527,7 @@ def _enrich_app_list_items(session: Session, *, apps: Sequence[App], tenant_id:
register_enum_models(console_ns, RetrievalMethod, WorkflowExecutionStatus, DatasetPermissionEnum)
register_response_schema_models(
console_ns, RedirectUrlResponse, SimpleResultResponse, AppImportResponse, AppTraceResponse
)
register_response_schema_models(console_ns, AppTraceResponse, RedirectUrlResponse, SimpleResultResponse)
register_schema_models(
console_ns,
@@ -524,7 +546,7 @@ register_schema_models(
WorkflowPartial,
ModelConfigPartial,
ModelConfig,
AppDetailSiteResponse,
Site,
DeletedTool,
AppDetail,
AppExportResponse,
@@ -569,7 +591,7 @@ class AppListApi(Resource):
@with_current_tenant_id
def get(self, current_tenant_id: str, current_user_id: str, session: Session):
"""Get app list"""
args = query_params_from_request(AppListQuery, list_fields=APP_LIST_QUERY_ARRAY_FIELDS)
args = AppListQuery.model_validate(_normalize_app_list_query_args(request.args))
params = AppListParams(
page=args.page,
limit=args.limit,
@@ -597,8 +619,8 @@ class AppListApi(Resource):
app_service = AppService()
app_pagination = app_service.get_paginate_apps(current_user_id, current_tenant_id, params, db.session)
if not app_pagination:
response = AppPagination(page=args.page, limit=args.limit, total=0, has_more=False, data=[])
return response.model_dump(mode="json"), 200
empty = AppPagination(page=args.page, limit=args.limit, total=0, has_more=False, data=[])
return empty.model_dump(mode="json"), 200
app_ids = [str(app.id) for app in app_pagination.items]
permission_keys_map = permissions.app.permission_keys_by_resource_ids(app_ids)
@@ -669,7 +691,7 @@ class StarredAppListApi(Resource):
@with_current_user_id
@with_current_tenant_id
def get(self, current_tenant_id: str, current_user_id: str, session: Session):
args = query_params_from_request(StarredAppListQuery, list_fields=APP_LIST_QUERY_ARRAY_FIELDS)
args = StarredAppListQuery.model_validate(_normalize_app_list_query_args(request.args))
params = StarredAppListParams(
page=args.page,
limit=args.limit,
@@ -687,7 +709,9 @@ class StarredAppListApi(Resource):
return empty.model_dump(mode="json"), 200
_enrich_app_list_items(session, apps=app_pagination.items, tenant_id=current_tenant_id)
return AppPagination.model_validate(app_pagination, from_attributes=True).model_dump(mode="json"), 200
pagination_model = AppPagination.model_validate(app_pagination, from_attributes=True)
return pagination_model.model_dump(mode="json"), 200
@console_ns.route("/apps/<uuid:app_id>/star")
@@ -706,7 +730,7 @@ class AppStarApi(Resource):
@get_app_model(mode=None)
def post(self, session: Session, current_user_id: str, app_model: App):
AppService.star_app(session, app=app_model, account_id=current_user_id)
return SimpleResultResponse(result="success").model_dump(mode="json")
return dump_response(SimpleResultResponse, {"result": "success"})
@console_ns.doc("unstar_app")
@console_ns.doc(description="Remove the current account's star from an application")
@@ -722,7 +746,7 @@ class AppStarApi(Resource):
@get_app_model(mode=None)
def delete(self, session: Session, current_user_id: str, app_model: App):
AppService.unstar_app(session, app=app_model, account_id=current_user_id)
return SimpleResultResponse(result="success").model_dump(mode="json")
return dump_response(SimpleResultResponse, {"result": "success"})
@console_ns.route("/apps/<uuid:app_id>")
@@ -790,7 +814,8 @@ class AppApi(Resource):
"max_active_requests": args.max_active_requests or 0,
}
app_model = app_service.update_app(app_model, args_dict)
return dump_response(AppDetailWithSite, app_model)
response_model = AppDetailWithSite.model_validate(app_model, from_attributes=True)
return response_model.model_dump(mode="json")
@console_ns.doc("delete_app")
@console_ns.doc(description="Delete application")
@@ -818,7 +843,6 @@ class AppCopyApi(Resource):
@console_ns.doc(params={"app_id": "Application ID to copy"})
@console_ns.expect(console_ns.models[CopyAppPayload.__name__])
@console_ns.response(201, "App copied successfully", console_ns.models[AppDetailWithSite.__name__])
@console_ns.response(202, "App copy requires confirmation", console_ns.models[AppImportResponse.__name__])
@console_ns.response(403, "Insufficient permissions")
@setup_required
@login_required
@@ -848,10 +872,10 @@ class AppCopyApi(Resource):
)
if result.status == ImportStatus.FAILED:
session.rollback()
return dump_response(AppImportResponse, result), 400
return result.model_dump(mode="json"), 400
if result.status == ImportStatus.PENDING:
session.rollback()
return dump_response(AppImportResponse, result), 202
return result.model_dump(mode="json"), 202
session.commit()
# Inherit web app permission from original app
@@ -902,14 +926,14 @@ class AppExportApi(Resource):
"""Export app"""
args = AppExportQuery.model_validate(request.args.to_dict(flat=True))
response = AppExportResponse(
payload = AppExportResponse(
data=AppDslService.export_dsl(
app_model=app_model,
include_secret=args.include_secret,
workflow_id=args.workflow_id,
)
)
return response.model_dump(mode="json")
return payload.model_dump(mode="json")
@console_ns.route("/apps/<uuid:app_id>/publish-to-creators-platform")
@@ -935,7 +959,7 @@ class AppPublishToCreatorsPlatformApi(Resource):
claim_code = upload_dsl(dsl_bytes)
redirect_url = get_redirect_url(current_user_id, claim_code)
return RedirectUrlResponse(redirect_url=redirect_url).model_dump(mode="json")
return {"redirect_url": redirect_url}
@console_ns.route("/apps/<uuid:app_id>/name")
@@ -956,7 +980,8 @@ class AppNameApi(Resource):
app_service = AppService()
app_model = app_service.update_app_name(app_model, args.name)
return dump_response(AppDetail, app_model)
response_model = AppDetail.model_validate(app_model, from_attributes=True)
return response_model.model_dump(mode="json")
@console_ns.route("/apps/<uuid:app_id>/icon")
@@ -983,7 +1008,8 @@ class AppIconApi(Resource):
args.icon_background or "",
args.icon_type,
)
return dump_response(AppDetail, app_model)
response_model = AppDetail.model_validate(app_model, from_attributes=True)
return response_model.model_dump(mode="json")
@console_ns.route("/apps/<uuid:app_id>/site-enable")
@@ -1005,7 +1031,8 @@ class AppSiteStatus(Resource):
app_service = AppService()
app_model = app_service.update_app_site_status(app_model, args.enable_site)
return dump_response(AppDetail, app_model)
response_model = AppDetail.model_validate(app_model, from_attributes=True)
return response_model.model_dump(mode="json")
@console_ns.route("/apps/<uuid:app_id>/api-enable")
@@ -1027,7 +1054,8 @@ class AppApiStatus(Resource):
app_service = AppService()
app_model = app_service.update_app_api_status(app_model, args.enable_api)
return dump_response(AppDetail, app_model)
response_model = AppDetail.model_validate(app_model, from_attributes=True)
return response_model.model_dump(mode="json")
@console_ns.route("/apps/<uuid:app_id>/trace")
@@ -1050,7 +1078,7 @@ class AppTraceApi(Resource):
"""Get app trace"""
app_trace_config = OpsTraceManager.get_app_tracing_config(app_model.id, session)
return dump_response(AppTraceResponse, app_trace_config)
return app_trace_config
@console_ns.doc("update_app_trace")
@console_ns.doc(description="Update app tracing configuration")
@@ -1078,4 +1106,4 @@ class AppTraceApi(Resource):
tracing_provider=args.tracing_provider,
)
return SimpleResultResponse(result="success").model_dump(mode="json")
return {"result": "success"}
+20 -36
View File
@@ -1,4 +1,5 @@
import logging
from typing import Any
from flask import request
from flask_restx import Resource
@@ -6,6 +7,7 @@ from pydantic import BaseModel, Field, RootModel
from werkzeug.exceptions import InternalServerError
import services
from controllers.common.fields import AudioBinaryResponse
from controllers.common.schema import query_params_from_model, register_response_schema_models, register_schema_models
from controllers.console import console_ns
from controllers.console.app.error import (
@@ -29,12 +31,9 @@ from controllers.console.wraps import (
)
from core.errors.error import ModelCurrentlyNotSupportError, ProviderTokenNotInitError, QuotaExceededError
from extensions.ext_database import db
from fields.base import ResponseModel
from graphon.model_runtime.errors.invoke import InvokeError
from libs.helper import dump_response
from libs.login import current_user, login_required
from libs.login import login_required
from models import App, AppMode
from services.app_ref_service import AppRefService
from services.audio_service import AudioService
from services.errors.audio import (
AudioTooLargeServiceError,
@@ -57,27 +56,16 @@ class TextToSpeechVoiceQuery(BaseModel):
language: str = Field(..., description="Language code")
class AudioTranscriptResponse(ResponseModel):
class AudioTranscriptResponse(BaseModel):
text: str = Field(description="Transcribed text from audio")
class TextToSpeechVoiceResponse(ResponseModel):
# see api/core/plugin/impl/model.py
name: str = Field(description="Voice display name")
value: str = Field(description="Voice identifier")
class TextToSpeechVoiceListResponse(RootModel[list[dict[str, Any]]]):
root: list[dict[str, Any]]
class TextToSpeechVoiceListResponse(RootModel[list[TextToSpeechVoiceResponse]]):
root: list[TextToSpeechVoiceResponse] = Field(description="Available voices")
register_schema_models(console_ns, TextToSpeechPayload, TextToSpeechVoiceQuery)
register_response_schema_models(
console_ns,
AudioTranscriptResponse,
TextToSpeechVoiceResponse,
TextToSpeechVoiceListResponse,
)
register_schema_models(console_ns, AudioTranscriptResponse, TextToSpeechPayload, TextToSpeechVoiceQuery)
register_response_schema_models(console_ns, AudioBinaryResponse, TextToSpeechVoiceListResponse)
@console_ns.route("/apps/<uuid:app_id>/audio-to-text")
@@ -106,7 +94,7 @@ class ChatMessageAudioApi(Resource):
end_user=None,
)
return dump_response(AudioTranscriptResponse, response)
return response
except services.errors.app_model_config.AppModelConfigBrokenError:
logger.exception("App model config broken.")
raise AppUnavailableError()
@@ -139,8 +127,11 @@ class ChatMessageTextApi(Resource):
@console_ns.doc(description="Convert text to speech for chat messages")
@console_ns.doc(params={"app_id": "App ID"})
@console_ns.expect(console_ns.models[TextToSpeechPayload.__name__])
# TTS returns provider audio bytes, so the success response is intentionally schema-less.
@console_ns.response(200, "Text to speech conversion successful")
@console_ns.response(
200,
"Text to speech conversion successful",
console_ns.models[AudioBinaryResponse.__name__],
)
@console_ns.response(400, "Bad request - Invalid parameters")
@setup_required
@login_required
@@ -149,24 +140,16 @@ class ChatMessageTextApi(Resource):
def post(self, app_model: App):
try:
payload = TextToSpeechPayload.model_validate(console_ns.payload)
message_ref = None
if payload.message_id:
app_ref = AppRefService.create_app_ref(app_model)
message_ref = AppRefService.create_message_ref(
app_ref,
payload.message_id,
account_id=current_user.id,
)
# response-contract:ignore
return AudioService.transcript_tts(
response = AudioService.transcript_tts(
app_model=app_model,
session=db.session,
text=payload.text,
voice=payload.voice,
message_ref=message_ref,
message_id=payload.message_id,
is_draft=True,
)
return response
except services.errors.app_model_config.AppModelConfigBrokenError:
logger.exception("App model config broken.")
raise AppUnavailableError()
@@ -197,7 +180,8 @@ class ChatMessageTextApi(Resource):
class TextModesApi(Resource):
@console_ns.doc("get_text_to_speech_voices")
@console_ns.doc(description="Get available TTS voices for a specific language")
@console_ns.doc(params={"app_id": "App ID", **query_params_from_model(TextToSpeechVoiceQuery)})
@console_ns.doc(params={"app_id": "App ID"})
@console_ns.doc(params=query_params_from_model(TextToSpeechVoiceQuery))
@console_ns.response(
200,
"TTS voices retrieved successfully",
@@ -218,7 +202,7 @@ class TextModesApi(Resource):
language=args.language,
)
return dump_response(TextToSpeechVoiceListResponse, response)
return response
except services.errors.audio.ProviderNotSupportTextToSpeechLanageServiceError:
raise AppUnavailableError("Text to audio voices language parameter loss.")
except NoAudioUploadedServiceError:
+22 -283
View File
@@ -1,20 +1,17 @@
import json
import logging
from collections.abc import Generator, Iterator, Mapping
from typing import Any, Literal, Protocol, runtime_checkable
from typing import Any, Literal
from uuid import UUID
from flask import request
from flask_restx import Resource
from pydantic import BaseModel, Field, field_validator
from sqlalchemy.orm import Session
from werkzeug.exceptions import BadRequest, InternalServerError, NotFound
import services
from controllers.common.fields import SimpleResultResponse
from controllers.common.fields import GeneratedAppResponse, SimpleResultResponse
from controllers.common.schema import register_response_schema_models, register_schema_models
from controllers.console import console_ns
from controllers.console.agent.app_helpers import resolve_agent_runtime_app_model
from controllers.console.agent.app_helpers import resolve_agent_app_model
from controllers.console.app.error import (
AppUnavailableError,
CompletionRequestError,
@@ -23,7 +20,7 @@ from controllers.console.app.error import (
ProviderNotInitializeError,
ProviderQuotaExceededError,
)
from controllers.console.app.wraps import get_app_model, with_session
from controllers.console.app.wraps import get_app_model
from controllers.console.wraps import (
RBACPermission,
RBACResourceScope,
@@ -37,7 +34,6 @@ from controllers.console.wraps import (
)
from controllers.web.error import InvokeRateLimitError as InvokeRateLimitHttpError
from core.app.entities.app_invoke_entities import InvokeFrom
from core.app.features.rate_limiting.rate_limit import RateLimitGenerator
from core.errors.error import (
ModelCurrentlyNotSupportError,
ProviderTokenNotInitError,
@@ -60,11 +56,6 @@ from services.errors.llm import InvokeRateLimitError
logger = logging.getLogger(__name__)
@runtime_checkable
class _ClosableStream(Protocol):
def close(self) -> None: ...
def _resolve_debugger_chat_streaming(
*, app_mode: AppMode, response_mode: str, response_mode_provided: bool = True
) -> bool:
@@ -102,10 +93,6 @@ class ChatMessagePayload(BaseMessagePayload):
query: str = Field(..., description="User query")
conversation_id: str | None = Field(default=None, description="Conversation ID")
parent_message_id: str | None = Field(default=None, description="Parent message ID")
draft_type: Literal["draft", "debug_build"] = Field(
default="draft",
description="Agent App debug config source. Use debug_build while the Agent is in build mode.",
)
@field_validator("conversation_id", "parent_message_id")
@classmethod
@@ -115,31 +102,8 @@ class ChatMessagePayload(BaseMessagePayload):
return uuid_value(value)
_BUILD_CHAT_FINALIZATION_QUERY = """Finalize this Build chat configuration for the agent.
This step is only for persisting Agent config changes discovered in the current Build chat. Do not install packages,
edit workspace files, run validation or debugging commands, make exploratory checks, or perform other work.
Use only the current Build chat message history to identify changes that need to be persisted. Do not inspect, test, or
validate old config unless the message history already shows that the old config is invalid.
Only update the build-draft config note when the current Build chat contains durable context that later runs need.
Do not create, update, delete, inspect, or fill gaps in other Agent config resources, including config files, config
skills, config env, tools, models, knowledge, or prompt settings.
When updating the config note with the Agent config CLI usage provided in the runtime prompt, record only durable
context needed by later runs, such as:
- what you installed or configured outside the workspace for this agent,
- where those external updates live, including CLI tools, packages, and persistent $HOME paths,
- how the agent should use it in later runs,
- any setup, authentication, or user action still required.
After config persistence completes, respond FINISHED."""
register_schema_models(console_ns, CompletionMessagePayload, ChatMessagePayload)
register_response_schema_models(console_ns, SimpleResultResponse)
register_response_schema_models(console_ns, GeneratedAppResponse, SimpleResultResponse)
# define completion message api for user
@@ -149,7 +113,7 @@ class CompletionMessageApi(Resource):
@console_ns.doc(description="Generate completion message for debugging")
@console_ns.doc(params={"app_id": "Application ID"})
@console_ns.expect(console_ns.models[CompletionMessagePayload.__name__])
@console_ns.response(200, "Completion generated successfully")
@console_ns.response(200, "Completion generated successfully", console_ns.models[GeneratedAppResponse.__name__])
@console_ns.response(400, "Invalid request parameters")
@console_ns.response(404, "App not found")
@setup_required
@@ -158,8 +122,7 @@ class CompletionMessageApi(Resource):
@with_current_user
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_TEST_AND_RUN)
@get_app_model(mode=AppMode.COMPLETION)
@with_session
def post(self, session: Session, current_user: Account, app_model: App):
def post(self, current_user: Account, app_model: App):
args_model = CompletionMessagePayload.model_validate(console_ns.payload)
args = args_model.model_dump(exclude_none=True, by_alias=True)
@@ -168,15 +131,9 @@ class CompletionMessageApi(Resource):
try:
response = AppGenerateService.generate(
session=session,
app_model=app_model,
user=current_user,
args=args,
invoke_from=InvokeFrom.DEBUGGER,
streaming=streaming,
app_model=app_model, user=current_user, args=args, invoke_from=InvokeFrom.DEBUGGER, streaming=streaming
)
# response-contract:ignore compact_generate_response
return helper.compact_generate_response(response)
except services.errors.conversation.ConversationNotExistsError:
raise NotFound("Conversation Not Exists.")
@@ -220,7 +177,7 @@ class CompletionMessageStopApi(Resource):
app_mode=AppMode.value_of(app_model.mode),
)
return SimpleResultResponse(result="success").model_dump(mode="json"), 200
return {"result": "success"}, 200
@console_ns.route("/apps/<uuid:app_id>/chat-messages")
@@ -229,7 +186,7 @@ class ChatMessageApi(Resource):
@console_ns.doc(description="Generate chat message for debugging")
@console_ns.doc(params={"app_id": "Application ID"})
@console_ns.expect(console_ns.models[ChatMessagePayload.__name__])
@console_ns.response(200, "Chat message generated successfully")
@console_ns.response(200, "Chat message generated successfully", console_ns.models[GeneratedAppResponse.__name__])
@console_ns.response(400, "Invalid request parameters")
@console_ns.response(404, "App or conversation not found")
@setup_required
@@ -240,11 +197,8 @@ class ChatMessageApi(Resource):
@with_current_tenant_id
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_TEST_AND_RUN)
@get_app_model(mode=[AppMode.CHAT, AppMode.AGENT_CHAT, AppMode.AGENT])
@with_session
def post(self, session: Session, current_tenant_id: str, current_user: Account, app_model: App):
return _create_chat_message(
session=session, current_tenant_id=current_tenant_id, current_user=current_user, app_model=app_model
)
def post(self, current_tenant_id: str, current_user: Account, app_model: App):
return _create_chat_message(current_tenant_id=current_tenant_id, current_user=current_user, app_model=app_model)
@console_ns.route("/agent/<uuid:agent_id>/chat-messages")
@@ -253,7 +207,7 @@ class AgentChatMessageApi(Resource):
@console_ns.doc(description="Generate an Agent App chat message for debugging")
@console_ns.doc(params={"agent_id": "Agent ID"})
@console_ns.expect(console_ns.models[ChatMessagePayload.__name__])
@console_ns.response(200, "Chat message generated successfully")
@console_ns.response(200, "Chat message generated successfully", console_ns.models[GeneratedAppResponse.__name__])
@console_ns.response(400, "Invalid request parameters")
@console_ns.response(404, "Agent or conversation not found")
@setup_required
@@ -263,38 +217,9 @@ class AgentChatMessageApi(Resource):
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_TEST_AND_RUN)
@with_current_user
@with_current_tenant_id
@with_session
def post(self, session: Session, current_tenant_id: str, current_user: Account, agent_id: UUID):
app_model = resolve_agent_runtime_app_model(tenant_id=current_tenant_id, agent_id=agent_id)
def post(self, current_tenant_id: str, current_user: Account, agent_id: UUID):
app_model = resolve_agent_app_model(tenant_id=current_tenant_id, agent_id=agent_id)
return _create_chat_message(
session=session,
current_tenant_id=current_tenant_id,
current_user=current_user,
app_model=app_model,
agent_id=str(agent_id),
)
@console_ns.route("/agent/<uuid:agent_id>/build-chat/finalize")
class AgentBuildChatFinalizeApi(Resource):
@console_ns.doc("finalize_agent_build_chat")
@console_ns.doc(description="Run a build-draft Agent App turn that asks the agent to push config updates")
@console_ns.doc(params={"agent_id": "Agent ID"})
@console_ns.response(200, "Success", console_ns.models[SimpleResultResponse.__name__])
@console_ns.response(400, "Invalid request parameters")
@console_ns.response(404, "Agent, build draft, or conversation not found")
@setup_required
@login_required
@account_initialization_required
@edit_permission_required
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_TEST_AND_RUN)
@with_current_user
@with_current_tenant_id
@with_session
def post(self, session: Session, current_tenant_id: str, current_user: Account, agent_id: UUID):
app_model = resolve_agent_runtime_app_model(tenant_id=current_tenant_id, agent_id=agent_id)
return _create_build_chat_finalization_message(
session=session,
current_tenant_id=current_tenant_id,
current_user=current_user,
app_model=app_model,
@@ -329,7 +254,7 @@ class AgentChatMessageStopApi(Resource):
@with_current_user_id
@with_current_tenant_id
def post(self, current_tenant_id: str, current_user_id: str, agent_id: UUID, task_id: str):
app_model = resolve_agent_runtime_app_model(tenant_id=current_tenant_id, agent_id=agent_id)
app_model = resolve_agent_app_model(tenant_id=current_tenant_id, agent_id=agent_id)
return _stop_chat_message(current_user_id=current_user_id, app_model=app_model, task_id=task_id)
@@ -355,12 +280,7 @@ def _resolve_current_user_agent_debug_conversation_id(
def _create_chat_message(
*,
session: Session,
current_user: Account,
app_model: App,
current_tenant_id: str | None = None,
agent_id: str | None = None,
*, current_user: Account, app_model: App, current_tenant_id: str | None = None, agent_id: str | None = None
):
raw_payload = console_ns.payload or {}
args_model = ChatMessagePayload.model_validate(raw_payload)
@@ -390,107 +310,12 @@ def _create_chat_message(
if external_trace_id:
args["external_trace_id"] = external_trace_id
return _generate_chat_message_response(
session=session,
current_user=current_user,
app_model=app_model,
args=args,
streaming=streaming,
)
def _create_build_chat_finalization_message(
*, session: Session, current_user: Account, app_model: App, current_tenant_id: str, agent_id: str
):
debug_conversation_id = _resolve_current_user_agent_debug_conversation_id(
current_tenant_id=current_tenant_id,
current_user=current_user,
app_model=app_model,
agent_id=agent_id,
)
args: dict[str, Any] = {
"query": _BUILD_CHAT_FINALIZATION_QUERY,
"inputs": {},
"response_mode": "streaming",
"draft_type": "debug_build",
"conversation_id": debug_conversation_id,
"auto_generate_name": False,
}
external_trace_id = get_external_trace_id(request)
if external_trace_id:
args["external_trace_id"] = external_trace_id
response = _generate_chat_message(
session=session,
current_user=current_user,
app_model=app_model,
args=args,
streaming=True,
)
_drain_streaming_generate_response(response)
return {"result": "success"}, 200
def _drain_streaming_generate_response(response: RateLimitGenerator | Generator[str, None, None]) -> None:
"""Consume a streamed app-generate response until a terminal message event arrives.
Finalize keeps the normal Agent App streaming path so the existing queue,
persistence, and runtime-session behavior stay intact. The console API only
changes the HTTP boundary: it drains the SSE stream server-side and returns
success after the generated build-chat message reaches ``message_end``.
"""
try:
for chunk in response:
for raw_event in chunk.split("\n\n"):
if not raw_event.strip():
continue
event_name: str | None = None
data_lines: list[str] = []
for line in raw_event.splitlines():
if line.startswith("event: "):
event_name = line.removeprefix("event: ").strip()
elif line.startswith("data: "):
data_lines.append(line.removeprefix("data: "))
if not data_lines:
if event_name == "ping":
continue
continue
payload = json.loads("\n".join(data_lines))
if not isinstance(payload, dict):
continue
payload_event = payload.get("event")
if payload_event == "message_end":
return
if payload_event == "error":
raise CompletionRequestError(str(payload.get("message") or "Build chat finalization failed."))
finally:
if isinstance(response, _ClosableStream):
response.close()
raise CompletionRequestError("Build chat finalization did not complete.")
def _generate_chat_message(
*,
session: Session,
current_user: Account,
app_model: App,
args: dict[str, Any],
streaming: bool,
):
try:
return AppGenerateService.generate(
session=session,
app_model=app_model,
user=current_user,
args=args,
invoke_from=InvokeFrom.DEBUGGER,
streaming=streaming,
response = AppGenerateService.generate(
app_model=app_model, user=current_user, args=args, invoke_from=InvokeFrom.DEBUGGER, streaming=streaming
)
return helper.compact_generate_response(response)
except services.errors.conversation.ConversationNotExistsError:
raise NotFound("Conversation Not Exists.")
except services.errors.conversation.ConversationCompletedError:
@@ -506,8 +331,6 @@ def _generate_chat_message(
raise ProviderModelCurrentlyNotSupportError()
except InvokeRateLimitError as ex:
raise InvokeRateLimitHttpError(ex.description)
except CompletionRequestError:
raise
except InvokeError as e:
raise CompletionRequestError(e.description)
except ValueError as e:
@@ -517,26 +340,6 @@ def _generate_chat_message(
raise InternalServerError()
def _generate_chat_message_response(
*,
session: Session,
current_user: Account,
app_model: App,
args: dict[str, Any],
streaming: bool,
):
response = _generate_chat_message(
session=session,
current_user=current_user,
app_model=app_model,
args=args,
streaming=streaming,
)
if AppMode.value_of(app_model.mode) == AppMode.AGENT and streaming:
response = _raise_agent_stream_error_before_response(response)
return helper.compact_generate_response(response)
def _stop_chat_message(*, current_user_id: str, app_model: App, task_id: str):
AppTaskService.stop_task(
task_id=task_id,
@@ -545,68 +348,4 @@ def _stop_chat_message(*, current_user_id: str, app_model: App, task_id: str):
app_mode=AppMode.value_of(app_model.mode),
)
return SimpleResultResponse(result="success").model_dump(mode="json"), 200
def _raise_agent_stream_error_before_response(response):
"""Surface immediate Agent App stream errors as HTTP errors before SSE starts.
The shared streaming helper always returns HTTP 200 once the SSE response is
created. Agent v2 configuration errors, such as an invalid model API key,
can be the first real stream event after the initial ping; pre-reading that
first non-ping event lets the console API return the existing 400 error
contract instead of a successful HTTP response carrying only an SSE error.
"""
if isinstance(response, Mapping):
return response
buffered: list[str] = []
iterator = iter(response)
while True:
try:
chunk = next(iterator)
except StopIteration:
return iter(buffered)
if not isinstance(chunk, str):
return _prepend_stream_chunks(buffered, chunk, iterator)
if _is_sse_ping(chunk):
buffered.append(chunk)
continue
error_payload = _extract_sse_error_payload(chunk)
if error_payload is not None:
if isinstance(response, _ClosableStream):
response.close()
message = error_payload.get("message")
raise CompletionRequestError(str(message or "Agent App chat failed."))
return _prepend_stream_chunks(buffered, chunk, iterator)
def _prepend_stream_chunks(buffered: list[Any], first: Any, iterator: Iterator[Any]) -> Generator[Any, None, None]:
yield from buffered
yield first
yield from iterator
def _is_sse_ping(chunk: str) -> bool:
return chunk.strip() == "event: ping"
def _extract_sse_error_payload(chunk: str) -> dict[str, Any] | None:
for raw_event in chunk.split("\n\n"):
data_lines: list[str] = []
for line in raw_event.splitlines():
if line.startswith("data: "):
data_lines.append(line.removeprefix("data: "))
if not data_lines:
continue
try:
payload = json.loads("\n".join(data_lines))
except json.JSONDecodeError:
continue
if isinstance(payload, dict) and payload.get("event") == "error":
return payload
return None
return {"result": "success"}, 200
+21 -18
View File
@@ -9,7 +9,7 @@ from sqlalchemy import func, or_
from sqlalchemy.orm import selectinload
from werkzeug.exceptions import NotFound
from controllers.common.schema import query_params_from_model, register_response_schema_models, register_schema_models
from controllers.common.schema import query_params_from_model, register_schema_models
from controllers.console import console_ns
from controllers.console.app.wraps import get_app_model
from controllers.console.wraps import (
@@ -39,9 +39,7 @@ from fields.conversation_fields import (
ConversationWithSummaryPagination as ConversationWithSummaryPaginationResponse,
)
from libs.datetime_utils import naive_utc_now, parse_time_range
from libs.helper import dump_response
from libs.login import login_required
from libs.pagination import paginate_query
from models import Conversation, EndUser, Message, MessageAnnotation
from models.account import Account
from models.model import App, AppMode
@@ -81,14 +79,13 @@ register_schema_models(
console_ns,
CompletionConversationQuery,
ChatConversationQuery,
)
register_response_schema_models(
console_ns,
ConversationResponse,
ConversationPaginationResponse,
ConversationMessageDetailResponse,
ConversationWithSummaryPaginationResponse,
ConversationDetailResponse,
CompletionConversationQuery,
ChatConversationQuery,
)
@@ -96,7 +93,8 @@ register_response_schema_models(
class CompletionConversationApi(Resource):
@console_ns.doc("list_completion_conversations")
@console_ns.doc(description="Get completion conversations with pagination and filtering")
@console_ns.doc(params={"app_id": "Application ID", **query_params_from_model(CompletionConversationQuery)})
@console_ns.doc(params={"app_id": "Application ID"})
@console_ns.doc(params=query_params_from_model(CompletionConversationQuery))
@console_ns.response(200, "Success", console_ns.models[ConversationPaginationResponse.__name__])
@console_ns.response(403, "Insufficient permissions")
@setup_required
@@ -157,9 +155,11 @@ class CompletionConversationApi(Resource):
query = query.order_by(Conversation.created_at.desc())
conversations = paginate_query(query, page=args.page, per_page=args.limit)
conversations = db.paginate(query, page=args.page, per_page=args.limit, error_out=False)
return dump_response(ConversationPaginationResponse, conversations)
return ConversationPaginationResponse.model_validate(conversations, from_attributes=True).model_dump(
mode="json"
)
@console_ns.route("/apps/<uuid:app_id>/completion-conversations/<uuid:conversation_id>")
@@ -179,9 +179,9 @@ class CompletionConversationDetailApi(Resource):
@get_app_model(mode=AppMode.COMPLETION)
def get(self, current_user: Account, app_model: App, conversation_id: UUID):
conversation_id_str = str(conversation_id)
return dump_response(
ConversationMessageDetailResponse, _get_conversation(current_user, app_model, conversation_id_str)
)
return ConversationMessageDetailResponse.model_validate(
_get_conversation(current_user, app_model, conversation_id_str), from_attributes=True
).model_dump(mode="json")
@console_ns.doc("delete_completion_conversation")
@console_ns.doc(description="Delete a completion conversation")
@@ -211,7 +211,8 @@ class CompletionConversationDetailApi(Resource):
class ChatConversationApi(Resource):
@console_ns.doc("list_chat_conversations")
@console_ns.doc(description="Get chat conversations with pagination, filtering and summary")
@console_ns.doc(params={"app_id": "Application ID", **query_params_from_model(ChatConversationQuery)})
@console_ns.doc(params={"app_id": "Application ID"})
@console_ns.doc(params=query_params_from_model(ChatConversationQuery))
@console_ns.response(200, "Success", console_ns.models[ConversationWithSummaryPaginationResponse.__name__])
@console_ns.response(403, "Insufficient permissions")
@setup_required
@@ -311,9 +312,11 @@ class ChatConversationApi(Resource):
case _:
query = query.order_by(Conversation.created_at.desc())
conversations = paginate_query(query, page=args.page, per_page=args.limit)
conversations = db.paginate(query, page=args.page, per_page=args.limit, error_out=False)
return dump_response(ConversationWithSummaryPaginationResponse, conversations)
return ConversationWithSummaryPaginationResponse.model_validate(conversations, from_attributes=True).model_dump(
mode="json"
)
@console_ns.route("/apps/<uuid:app_id>/chat-conversations/<uuid:conversation_id>")
@@ -333,9 +336,9 @@ class ChatConversationDetailApi(Resource):
@get_app_model(mode=[AppMode.CHAT, AppMode.AGENT_CHAT, AppMode.ADVANCED_CHAT, AppMode.AGENT])
def get(self, current_user: Account, app_model: App, conversation_id: UUID):
conversation_id_str = str(conversation_id)
return dump_response(
ConversationDetailResponse, _get_conversation(current_user, app_model, conversation_id_str)
)
return ConversationDetailResponse.model_validate(
_get_conversation(current_user, app_model, conversation_id_str), from_attributes=True
).model_dump(mode="json")
@console_ns.doc("delete_chat_conversation")
@console_ns.doc(description="Delete a chat conversation")
@@ -22,7 +22,7 @@ from controllers.console.wraps import (
from extensions.ext_database import db
from fields._value_type_serializer import serialize_value_type
from fields.base import ResponseModel
from libs.helper import dump_response, to_timestamp
from libs.helper import to_timestamp
from libs.login import login_required
from models import ConversationVariable
from models.model import App, AppMode
@@ -119,8 +119,7 @@ class ConversationVariablesApi(Resource):
with sessionmaker(db.engine, expire_on_commit=False).begin() as session:
rows = session.scalars(stmt).all()
return dump_response(
PaginatedConversationVariableResponse,
response = PaginatedConversationVariableResponse.model_validate(
{
"page": page,
"limit": page_size,
@@ -136,5 +135,6 @@ class ConversationVariablesApi(Resource):
)
for row in rows
],
},
}
)
return response.model_dump(mode="json")
+28 -149
View File
@@ -1,10 +1,8 @@
import json
from collections.abc import Generator, Sequence
from collections.abc import Sequence
from typing import Any, Literal
from flask_restx import Resource
from pydantic import BaseModel, Field, RootModel
from sqlalchemy import select
from sqlalchemy.orm import Session
from controllers.common.fields import SimpleDataResponse
@@ -25,10 +23,8 @@ from core.helper.code_executor.javascript.javascript_code_provider import Javasc
from core.helper.code_executor.python3.python3_code_provider import Python3CodeProvider
from core.llm_generator.entities import RuleCodeGeneratePayload, RuleGeneratePayload, RuleStructuredOutputPayload
from core.llm_generator.llm_generator import LLMGenerator
from core.workflow.generator.types import WorkflowGenerateErrorCode
from graphon.model_runtime.entities.llm_entities import LLMMode
from graphon.model_runtime.errors.invoke import InvokeError
from libs.helper import compact_generate_response
from libs.login import login_required
from models import App
from services.workflow_generator_service import WorkflowGeneratorService
@@ -68,10 +64,7 @@ class WorkflowGeneratePayload(BaseModel):
can reuse its existing handler.
"""
mode: Literal["workflow", "advanced-chat", "auto"] = Field(
...,
description="Target app mode for the generated graph; 'auto' lets the backend classify the instruction",
)
mode: Literal["workflow", "advanced-chat"] = Field(..., description="Target app mode for the generated graph")
instruction: str = Field(..., description="Natural-language workflow description")
ideal_output: str = Field(default="", description="Optional sample output for grounding")
model_config_data: ModelConfig = Field(
@@ -85,19 +78,6 @@ class WorkflowGeneratePayload(BaseModel):
)
class WorkflowInstructionSuggestionsPayload(BaseModel):
"""Payload for the workflow-generator instruction-suggestions endpoint.
Runs before the user picks a model, so the suggestions come from the
tenant's default model. The underlying generator never raises — an empty
``suggestions`` list is a valid 200 (soft-fail).
"""
mode: Literal["workflow", "advanced-chat"] = Field(..., description="Target app mode for the suggestions")
language: str | None = Field(default=None, description="Optional language to write the suggestions in")
count: int = Field(default=4, ge=1, le=6, description="Number of suggestions to return (1-6)")
class GeneratorResponse(RootModel[Any]):
root: Any
@@ -111,7 +91,6 @@ register_schema_models(
InstructionGeneratePayload,
InstructionTemplatePayload,
WorkflowGeneratePayload,
WorkflowInstructionSuggestionsPayload,
ModelConfig,
)
register_response_schema_models(console_ns, GeneratorResponse, SimpleDataResponse)
@@ -237,9 +216,7 @@ class InstructionGenerateApi(Resource):
try:
# Generate from nothing for a workflow node
if (args.current in (code_template, "")) and args.node_id != "":
app = session.scalar(
select(App).where(App.id == args.flow_id, App.tenant_id == current_tenant_id).limit(1)
)
app = session.get(App, args.flow_id)
if not app:
return {"error": f"app {args.flow_id} not found"}, 400
workflow = WorkflowService().get_draft_workflow(app_model=app, session=session)
@@ -336,34 +313,6 @@ class InstructionGenerationTemplateApi(Resource):
raise ValueError(f"Invalid type: {args.type}")
def _workflow_instruction_guard(args: WorkflowGeneratePayload) -> tuple[dict, int] | None:
"""Shared boundary guard for the workflow-generate endpoints.
Returns a ``(body, 400)`` tuple when the instruction is empty / whitespace
or either free-text field exceeds the cap, else ``None``. Pydantic only
validates the field is a str; a whitespace-only or pasted-document input
would otherwise waste a slow planner+builder roundtrip on a response the
validator rejects anyway. Both the blocking and streaming endpoints call
this so they reject identical inputs.
"""
if not args.instruction.strip():
return {
"error": "Instruction is required",
"errors": [{"code": WorkflowGenerateErrorCode.EMPTY_INSTRUCTION, "detail": "Instruction is required"}],
}, 400
if len(args.instruction) > _MAX_INSTRUCTION_LENGTH or len(args.ideal_output) > _MAX_INSTRUCTION_LENGTH:
return {
"error": "Instruction is too long",
"errors": [
{
"code": WorkflowGenerateErrorCode.INSTRUCTION_TOO_LONG,
"detail": f"Instruction and ideal output must each be at most {_MAX_INSTRUCTION_LENGTH} characters",
}
],
}, 400
return None
@console_ns.route("/workflow-generate")
class WorkflowGenerateApi(Resource):
"""Generate a Workflow / Chatflow draft graph from a natural-language description.
@@ -386,11 +335,31 @@ class WorkflowGenerateApi(Resource):
def post(self, current_tenant_id: str):
args = WorkflowGeneratePayload.model_validate(console_ns.payload)
# Reject empty / over-length instructions at the boundary (shared with
# the streaming endpoint) before spending a planner+builder roundtrip.
guard = _workflow_instruction_guard(args)
if guard is not None:
return guard
# Reject obviously-empty instructions at the boundary — Pydantic only
# validates ``instruction`` is a str, but a whitespace-only string
# would still hit the LLM and waste a planner+builder roundtrip on a
# response that the postprocess validator would reject anyway.
if not args.instruction.strip():
return {
"error": "Instruction is required",
"errors": [{"code": "EMPTY_INSTRUCTION", "detail": "Instruction is required"}],
}, 400
# Bound the prompt at the boundary too: an arbitrarily long
# instruction (or pasted document) blows the planner/builder context
# window and fails with an opaque provider error after two slow LLM
# calls. The cap matches the frontend textarea's maxLength.
if len(args.instruction) > _MAX_INSTRUCTION_LENGTH or len(args.ideal_output) > _MAX_INSTRUCTION_LENGTH:
return {
"error": "Instruction is too long",
"errors": [
{
"code": "INSTRUCTION_TOO_LONG",
"detail": f"Instruction and ideal output must each be at most "
f"{_MAX_INSTRUCTION_LENGTH} characters",
}
],
}, 400
try:
result = WorkflowGeneratorService.generate_workflow_graph(
@@ -411,93 +380,3 @@ class WorkflowGenerateApi(Resource):
raise CompletionRequestError(e.description)
return result
@console_ns.route("/workflow-generate/suggestions")
class WorkflowInstructionSuggestionsApi(Resource):
"""Suggest short, buildable example instructions for the cmd+k generator.
Runs before a model is selected (uses the tenant's default model). The
underlying generator never raises, so an empty list is a valid 200 — the
frontend renders "no suggestions" rather than an error, so no provider-error
mapping is needed here.
"""
@console_ns.doc("generate_workflow_instruction_suggestions")
@console_ns.doc(description="Suggest example workflow-generator instructions for the tenant")
@console_ns.expect(console_ns.models[WorkflowInstructionSuggestionsPayload.__name__])
@console_ns.response(200, "Suggestions generated successfully", console_ns.models[GeneratorResponse.__name__])
@console_ns.response(400, "Invalid request parameters")
@setup_required
@login_required
@account_initialization_required
@with_current_tenant_id
def post(self, current_tenant_id: str):
args = WorkflowInstructionSuggestionsPayload.model_validate(console_ns.payload)
suggestions = LLMGenerator.generate_workflow_instruction_suggestions(
tenant_id=current_tenant_id,
mode=args.mode,
language=args.language,
count=args.count,
)
return {"suggestions": suggestions}
@console_ns.route("/workflow-generate/stream")
class WorkflowGenerateStreamApi(Resource):
"""Plan-first streaming variant of ``/workflow-generate`` (Server-Sent Events).
Emits a ``plan`` event (high-level node list + app metadata) as soon as the
planner returns, then a final ``result`` event with the full graph — the
SAME envelope ``/workflow-generate`` returns. Provider-init / invoke errors
are surfaced as a single ``result`` event (code ``MODEL_ERROR``) so the
frontend's stream parser always receives a result rather than a non-SSE HTTP
error.
"""
@console_ns.doc("generate_workflow_graph_stream")
@console_ns.doc(description="Stream a Dify workflow graph (plan then result) via SSE")
@console_ns.expect(console_ns.models[WorkflowGeneratePayload.__name__])
@console_ns.response(200, "Server-Sent Events stream of plan/result events")
@console_ns.response(400, "Invalid request parameters")
@setup_required
@login_required
@account_initialization_required
@with_current_tenant_id
def post(self, current_tenant_id: str):
args = WorkflowGeneratePayload.model_validate(console_ns.payload)
# Same boundary guards as the blocking endpoint — return a normal 400
# JSON for these BEFORE opening the stream.
guard = _workflow_instruction_guard(args)
if guard is not None:
return guard
def generate() -> Generator[str, None, None]:
try:
for event_name, payload in WorkflowGeneratorService.generate_workflow_graph_stream(
tenant_id=current_tenant_id,
mode=args.mode,
instruction=args.instruction,
model_config=args.model_config_data,
ideal_output=args.ideal_output,
current_graph=args.current_graph,
):
body = {"event": event_name, **payload}
yield f"data: {json.dumps(body)}\n\n"
except (ProviderTokenNotInitError, QuotaExceededError, ModelCurrentlyNotSupportError, InvokeError) as e:
# The model instance is resolved inside the service (lazily, on
# first iteration), so a provider / init error surfaces here.
# Emit it as a single SSE result event rather than a non-SSE
# error response so the frontend's stream parser always gets a
# result it can render.
detail = getattr(e, "description", None) or str(e) or "Model invocation failed"
error_body = {
"event": "result",
"graph": {"nodes": [], "edges": [], "viewport": {"x": 0.0, "y": 0.0, "zoom": 0.7}},
"error": detail,
"errors": [{"code": WorkflowGenerateErrorCode.MODEL_ERROR, "detail": detail}],
}
yield f"data: {json.dumps(error_body)}\n\n"
return compact_generate_response(generate())
+6 -17
View File
@@ -22,11 +22,10 @@ from controllers.console.wraps import (
)
from extensions.ext_database import db
from fields.base import ResponseModel
from libs.helper import dump_response, to_timestamp
from libs.helper import to_timestamp
from libs.login import login_required
from models.enums import AppMCPServerStatus
from models.model import App, AppMCPServer
from services.app_ref_service import AppRefService
class MCPServerCreatePayload(BaseModel):
@@ -93,7 +92,7 @@ class AppMCPServerController(Resource):
server = db.session.scalar(select(AppMCPServer).where(AppMCPServer.app_id == app_model.id).limit(1))
if server is None:
return {}
return dump_response(AppMCPServerResponse, server)
return AppMCPServerResponse.model_validate(server, from_attributes=True).model_dump(mode="json")
@console_ns.doc("create_app_mcp_server")
@console_ns.doc(description="Create MCP server configuration for an application")
@@ -128,7 +127,7 @@ class AppMCPServerController(Resource):
)
db.session.add(server)
db.session.commit()
return dump_response(AppMCPServerResponse, server), 201
return AppMCPServerResponse.model_validate(server, from_attributes=True).model_dump(mode="json"), 201
@console_ns.doc("update_app_mcp_server")
@console_ns.doc(description="Update MCP server configuration for an application")
@@ -147,17 +146,7 @@ class AppMCPServerController(Resource):
@get_app_model
def put(self, app_model: App):
payload = MCPServerUpdatePayload.model_validate(console_ns.payload or {})
app_ref = AppRefService.create_app_ref(app_model)
server_ref = AppRefService.create_mcp_server_ref(app_ref, payload.id)
server = db.session.scalar(
select(AppMCPServer)
.where(
AppMCPServer.id == server_ref.server_id,
AppMCPServer.tenant_id == server_ref.tenant_id,
AppMCPServer.app_id == server_ref.app_id,
)
.limit(1)
)
server = db.session.get(AppMCPServer, payload.id)
if not server:
raise NotFound()
@@ -176,7 +165,7 @@ class AppMCPServerController(Resource):
except ValueError:
raise ValueError("Invalid status")
db.session.commit()
return dump_response(AppMCPServerResponse, server)
return AppMCPServerResponse.model_validate(server, from_attributes=True).model_dump(mode="json")
@console_ns.route("/apps/<uuid:server_id>/server/refresh")
@@ -203,4 +192,4 @@ class AppMCPServerRefreshController(Resource):
raise NotFound()
server.server_code = AppMCPServer.generate_server_code(16)
db.session.commit()
return dump_response(AppMCPServerResponse, server)
return AppMCPServerResponse.model_validate(server, from_attributes=True).model_dump(mode="json")
+63 -20
View File
@@ -1,4 +1,5 @@
import logging
from datetime import datetime
from typing import Literal
from uuid import UUID
@@ -12,7 +13,7 @@ from controllers.common.controller_schemas import MessageFeedbackPayload as _Mes
from controllers.common.fields import SimpleResultResponse, TextFileResponse
from controllers.common.schema import query_params_from_model, register_response_schema_models, register_schema_models
from controllers.console import console_ns
from controllers.console.agent.app_helpers import resolve_agent_runtime_app_model
from controllers.console.agent.app_helpers import resolve_agent_app_model
from controllers.console.app.error import (
CompletionRequestError,
ProviderModelCurrentlyNotSupportError,
@@ -37,10 +38,16 @@ from core.errors.error import ModelCurrentlyNotSupportError, ProviderTokenNotIni
from extensions.ext_database import db
from fields.base import ResponseModel
from fields.conversation_fields import (
MessageDetail as BaseMessageDetailResponse,
AgentThought,
ConversationAnnotation,
ConversationAnnotationHitHistory,
Feedback,
JSONValue,
MessageFile,
format_files_contained,
)
from graphon.model_runtime.errors.invoke import InvokeError
from libs.helper import dump_response, uuid_value
from libs.helper import to_timestamp, uuid_value
from libs.infinite_scroll_pagination import InfiniteScrollPagination
from libs.login import login_required
from models.account import Account
@@ -104,16 +111,49 @@ class FeedbackExportQuery(BaseModel):
raise ValueError("has_comment must be a boolean value")
class AnnotationCountResponse(ResponseModel):
class AnnotationCountResponse(BaseModel):
count: int = Field(description="Number of annotations")
class SuggestedQuestionsResponse(ResponseModel):
class SuggestedQuestionsResponse(BaseModel):
data: list[str] = Field(description="Suggested question")
class MessageDetailResponse(BaseMessageDetailResponse):
class MessageDetailResponse(ResponseModel):
id: str
conversation_id: str
inputs: dict[str, JSONValue]
query: str
message: JSONValue | None = None
message_tokens: int | None = None
answer: str = Field(validation_alias="re_sign_file_url_answer")
answer_tokens: int | None = None
provider_response_latency: float | None = None
from_source: str
from_end_user_id: str | None = None
from_account_id: str | None = None
feedbacks: list[Feedback] = Field(default_factory=list)
workflow_run_id: str | None = None
annotation: ConversationAnnotation | None = None
annotation_hit_history: ConversationAnnotationHitHistory | None = None
created_at: int | None = None
agent_thoughts: list[AgentThought] = Field(default_factory=list)
message_files: list[MessageFile] = Field(default_factory=list)
extra_contents: list[ExecutionExtraContentDomainModel] = Field(default_factory=list)
metadata: JSONValue | None = Field(default=None, validation_alias="message_metadata_dict")
status: str
error: str | None = None
parent_message_id: str | None = None
@field_validator("inputs", mode="before")
@classmethod
def _normalize_inputs(cls, value: JSONValue) -> JSONValue:
return format_files_contained(value)
@field_validator("created_at", mode="before")
@classmethod
def _normalize_created_at(cls, value: datetime | int | None) -> int | None:
return to_timestamp(value)
class MessageInfiniteScrollPaginationResponse(ResponseModel):
@@ -143,7 +183,8 @@ register_response_schema_models(
class ChatMessageListApi(Resource):
@console_ns.doc("list_chat_messages")
@console_ns.doc(description="Get chat messages for a conversation with pagination")
@console_ns.doc(params={"app_id": "Application ID", **query_params_from_model(ChatMessagesQuery)})
@console_ns.doc(params={"app_id": "Application ID"})
@console_ns.doc(params=query_params_from_model(ChatMessagesQuery))
@console_ns.response(200, "Success", console_ns.models[MessageInfiniteScrollPaginationResponse.__name__])
@console_ns.response(404, "Conversation not found")
@login_required
@@ -173,7 +214,7 @@ class AgentChatMessageListApi(Resource):
@with_current_user
@with_current_tenant_id
def get(self, current_tenant_id: str, current_user: Account, agent_id: UUID):
app_model = resolve_agent_runtime_app_model(tenant_id=current_tenant_id, agent_id=agent_id)
app_model = resolve_agent_app_model(tenant_id=current_tenant_id, agent_id=agent_id)
return _list_chat_messages(app_model=app_model, current_user=current_user)
@@ -209,7 +250,7 @@ class AgentMessageFeedbackApi(Resource):
@with_current_user
@with_current_tenant_id
def post(self, current_tenant_id: str, current_user: Account, agent_id: UUID):
app_model = resolve_agent_runtime_app_model(tenant_id=current_tenant_id, agent_id=agent_id)
app_model = resolve_agent_app_model(tenant_id=current_tenant_id, agent_id=agent_id)
return _update_message_feedback(current_user=current_user, app_model=app_model)
@@ -233,7 +274,7 @@ class MessageAnnotationCountApi(Resource):
select(func.count(MessageAnnotation.id)).where(MessageAnnotation.app_id == app_model.id)
)
return AnnotationCountResponse(count=count or 0).model_dump(mode="json")
return {"count": count}
@console_ns.route("/apps/<uuid:app_id>/chat-messages/<uuid:message_id>/suggested-questions")
@@ -274,7 +315,7 @@ class AgentMessageSuggestedQuestionApi(Resource):
@with_current_user
@with_current_tenant_id
def get(self, current_tenant_id: str, current_user: Account, agent_id: UUID, message_id: UUID):
app_model = resolve_agent_runtime_app_model(tenant_id=current_tenant_id, agent_id=agent_id)
app_model = resolve_agent_app_model(tenant_id=current_tenant_id, agent_id=agent_id)
return _get_message_suggested_questions(current_user=current_user, app_model=app_model, message_id=message_id)
@@ -282,12 +323,13 @@ class AgentMessageSuggestedQuestionApi(Resource):
class MessageFeedbackExportApi(Resource):
@console_ns.doc("export_feedbacks")
@console_ns.doc(description="Export user feedback data for Google Sheets")
@console_ns.doc(params={"app_id": "Application ID"})
@console_ns.doc(params=query_params_from_model(FeedbackExportQuery))
@console_ns.response(
200,
"Feedback data exported successfully",
console_ns.models[TextFileResponse.__name__],
)
@console_ns.doc(params={"app_id": "Application ID", **query_params_from_model(FeedbackExportQuery)})
@console_ns.response(400, "Invalid parameters")
@console_ns.response(500, "Internal server error")
@setup_required
@@ -312,6 +354,7 @@ class MessageFeedbackExportApi(Resource):
end_date=args.end_date,
format_type=args.format,
)
return export_data
except ValueError as e:
@@ -350,7 +393,7 @@ class AgentMessageApi(Resource):
@account_initialization_required
@with_current_tenant_id
def get(self, current_tenant_id: str, agent_id: UUID, message_id: UUID):
app_model = resolve_agent_runtime_app_model(tenant_id=current_tenant_id, agent_id=agent_id)
app_model = resolve_agent_app_model(tenant_id=current_tenant_id, agent_id=agent_id)
return _get_message_detail(app_model=app_model, message_id=message_id)
@@ -422,16 +465,16 @@ def _list_chat_messages(*, app_model: App, current_user: Account | None = None):
history_messages = list(reversed(history_messages))
attach_message_extra_contents(history_messages)
return dump_response(
MessageInfiniteScrollPaginationResponse,
return MessageInfiniteScrollPaginationResponse.model_validate(
InfiniteScrollPagination(data=history_messages, limit=args.limit, has_more=has_more),
)
from_attributes=True,
).model_dump(mode="json")
def _update_message_feedback(*, current_user: Account, app_model: App):
args = MessageFeedbackPayload.model_validate(console_ns.payload)
message_id = args.message_id
message_id = str(args.message_id)
message = db.session.scalar(
select(Message).where(Message.id == message_id, Message.app_id == app_model.id).limit(1)
@@ -466,7 +509,7 @@ def _update_message_feedback(*, current_user: Account, app_model: App):
db.session.commit()
return SimpleResultResponse(result="success").model_dump(mode="json")
return {"result": "success"}
def _get_message_suggested_questions(*, current_user: Account, app_model: App, message_id: UUID):
@@ -494,7 +537,7 @@ def _get_message_suggested_questions(*, current_user: Account, app_model: App, m
logger.exception("internal server error.")
raise InternalServerError()
return dump_response(SuggestedQuestionsResponse, {"data": questions})
return {"data": questions}
def _get_message_detail(*, app_model: App, message_id: UUID):
@@ -508,4 +551,4 @@ def _get_message_detail(*, app_model: App, message_id: UUID):
raise NotFound("Message Not Exists.")
attach_message_extra_contents([message])
return dump_response(MessageDetailResponse, message)
return MessageDetailResponse.model_validate(message, from_attributes=True).model_dump(mode="json")
+2 -6
View File
@@ -22,7 +22,6 @@ from controllers.console.wraps import (
from extensions.ext_database import db
from fields.base import ResponseModel
from libs.datetime_utils import naive_utc_now
from libs.helper import dump_response
from libs.login import login_required
from models import Site
from models.account import Account
@@ -41,7 +40,6 @@ class AppSiteUpdatePayload(BaseModel):
customize_domain: str | None = Field(default=None)
copyright: str | None = Field(default=None)
privacy_policy: str | None = Field(default=None)
input_placeholder: str | None = Field(default=None)
custom_disclaimer: str | None = Field(default=None)
customize_token_strategy: Literal["must", "allow", "not_allow"] | None = Field(default=None)
prompt_public: bool | None = Field(default=None)
@@ -68,7 +66,6 @@ class AppSiteResponse(ResponseModel):
customize_domain: str | None = None
copyright: str | None = None
privacy_policy: str | None = None
input_placeholder: str | None = None
custom_disclaimer: str | None = None
customize_token_strategy: str
prompt_public: bool
@@ -113,7 +110,6 @@ class AppSite(Resource):
"customize_domain",
"copyright",
"privacy_policy",
"input_placeholder",
"custom_disclaimer",
"customize_token_strategy",
"prompt_public",
@@ -128,7 +124,7 @@ class AppSite(Resource):
site.updated_at = naive_utc_now()
db.session.commit()
return dump_response(AppSiteResponse, site)
return AppSiteResponse.model_validate(site, from_attributes=True).model_dump(mode="json")
@console_ns.route("/apps/<uuid:app_id>/site/access-token-reset")
@@ -157,4 +153,4 @@ class AppSiteAccessTokenReset(Resource):
site.updated_at = naive_utc_now()
db.session.commit()
return dump_response(AppSiteResponse, site)
return AppSiteResponse.model_validate(site, from_attributes=True).model_dump(mode="json")
+46 -52
View File
@@ -1,7 +1,7 @@
from decimal import Decimal
import sqlalchemy as sa
from flask import abort, request
from flask import abort, jsonify, request
from flask_restx import Resource
from pydantic import BaseModel, Field, field_validator
@@ -20,7 +20,7 @@ from core.app.entities.app_invoke_entities import InvokeFrom
from extensions.ext_database import db
from fields.base import ResponseModel
from libs.datetime_utils import parse_time_range
from libs.helper import convert_datetime_to_date, dump_response
from libs.helper import convert_datetime_to_date
from libs.login import login_required
from models import AppMode
from models.account import Account
@@ -44,15 +44,8 @@ class DailyMessageStatisticItem(ResponseModel):
message_count: int
register_schema_models(console_ns, StatisticTimeRangeQuery)
class StatisticDataResponse[T](ResponseModel):
data: list[T]
class DailyMessageStatisticResponse(StatisticDataResponse[DailyMessageStatisticItem]):
pass
class DailyMessageStatisticResponse(ResponseModel):
data: list[DailyMessageStatisticItem]
class DailyConversationStatisticItem(ResponseModel):
@@ -60,8 +53,8 @@ class DailyConversationStatisticItem(ResponseModel):
conversation_count: int
class DailyConversationStatisticResponse(StatisticDataResponse[DailyConversationStatisticItem]):
pass
class DailyConversationStatisticResponse(ResponseModel):
data: list[DailyConversationStatisticItem]
class DailyTerminalStatisticItem(ResponseModel):
@@ -69,19 +62,19 @@ class DailyTerminalStatisticItem(ResponseModel):
terminal_count: int
class DailyTerminalStatisticResponse(StatisticDataResponse[DailyTerminalStatisticItem]):
pass
class DailyTerminalStatisticResponse(ResponseModel):
data: list[DailyTerminalStatisticItem]
class DailyTokenCostStatisticItem(ResponseModel):
date: str
token_count: int | None = None
total_price: Decimal | None = None
currency: str | None = None
token_count: int
total_price: str | float
currency: str
class DailyTokenCostStatisticResponse(StatisticDataResponse[DailyTokenCostStatisticItem]):
pass
class DailyTokenCostStatisticResponse(ResponseModel):
data: list[DailyTokenCostStatisticItem]
class AverageSessionInteractionStatisticItem(ResponseModel):
@@ -89,8 +82,8 @@ class AverageSessionInteractionStatisticItem(ResponseModel):
interactions: float
class AverageSessionInteractionStatisticResponse(StatisticDataResponse[AverageSessionInteractionStatisticItem]):
pass
class AverageSessionInteractionStatisticResponse(ResponseModel):
data: list[AverageSessionInteractionStatisticItem]
class UserSatisfactionRateStatisticItem(ResponseModel):
@@ -98,8 +91,8 @@ class UserSatisfactionRateStatisticItem(ResponseModel):
rate: float
class UserSatisfactionRateStatisticResponse(StatisticDataResponse[UserSatisfactionRateStatisticItem]):
pass
class UserSatisfactionRateStatisticResponse(ResponseModel):
data: list[UserSatisfactionRateStatisticItem]
class AverageResponseTimeStatisticItem(ResponseModel):
@@ -107,8 +100,8 @@ class AverageResponseTimeStatisticItem(ResponseModel):
latency: float
class AverageResponseTimeStatisticResponse(StatisticDataResponse[AverageResponseTimeStatisticItem]):
pass
class AverageResponseTimeStatisticResponse(ResponseModel):
data: list[AverageResponseTimeStatisticItem]
class TokensPerSecondStatisticItem(ResponseModel):
@@ -116,27 +109,20 @@ class TokensPerSecondStatisticItem(ResponseModel):
tps: float
class TokensPerSecondStatisticResponse(StatisticDataResponse[TokensPerSecondStatisticItem]):
pass
class TokensPerSecondStatisticResponse(ResponseModel):
data: list[TokensPerSecondStatisticItem]
register_schema_models(console_ns, StatisticTimeRangeQuery)
register_response_schema_models(
console_ns,
DailyMessageStatisticItem,
DailyMessageStatisticResponse,
DailyConversationStatisticItem,
DailyConversationStatisticResponse,
DailyTerminalStatisticItem,
DailyTerminalStatisticResponse,
DailyTokenCostStatisticItem,
DailyTokenCostStatisticResponse,
AverageSessionInteractionStatisticItem,
AverageSessionInteractionStatisticResponse,
UserSatisfactionRateStatisticItem,
UserSatisfactionRateStatisticResponse,
AverageResponseTimeStatisticItem,
AverageResponseTimeStatisticResponse,
TokensPerSecondStatisticItem,
TokensPerSecondStatisticResponse,
)
@@ -145,7 +131,8 @@ register_response_schema_models(
class DailyMessageStatistic(Resource):
@console_ns.doc("get_daily_message_statistics")
@console_ns.doc(description="Get daily message statistics for an application")
@console_ns.doc(params={"app_id": "Application ID", **query_params_from_model(StatisticTimeRangeQuery)})
@console_ns.doc(params={"app_id": "Application ID"})
@console_ns.doc(params=query_params_from_model(StatisticTimeRangeQuery))
@console_ns.response(
200,
"Daily message statistics retrieved successfully",
@@ -198,14 +185,15 @@ WHERE
for i in rs:
response_data.append({"date": str(i.date), "message_count": i.message_count})
return dump_response(DailyMessageStatisticResponse, {"data": response_data})
return jsonify({"data": response_data})
@console_ns.route("/apps/<uuid:app_id>/statistics/daily-conversations")
class DailyConversationStatistic(Resource):
@console_ns.doc("get_daily_conversation_statistics")
@console_ns.doc(description="Get daily conversation statistics for an application")
@console_ns.doc(params={"app_id": "Application ID", **query_params_from_model(StatisticTimeRangeQuery)})
@console_ns.doc(params={"app_id": "Application ID"})
@console_ns.doc(params=query_params_from_model(StatisticTimeRangeQuery))
@console_ns.response(
200,
"Daily conversation statistics retrieved successfully",
@@ -257,14 +245,15 @@ WHERE
for i in rs:
response_data.append({"date": str(i.date), "conversation_count": i.conversation_count})
return dump_response(DailyConversationStatisticResponse, {"data": response_data})
return jsonify({"data": response_data})
@console_ns.route("/apps/<uuid:app_id>/statistics/daily-end-users")
class DailyTerminalsStatistic(Resource):
@console_ns.doc("get_daily_terminals_statistics")
@console_ns.doc(description="Get daily terminal/end-user statistics for an application")
@console_ns.doc(params={"app_id": "Application ID", **query_params_from_model(StatisticTimeRangeQuery)})
@console_ns.doc(params={"app_id": "Application ID"})
@console_ns.doc(params=query_params_from_model(StatisticTimeRangeQuery))
@console_ns.response(
200,
"Daily terminal statistics retrieved successfully",
@@ -317,14 +306,15 @@ WHERE
for i in rs:
response_data.append({"date": str(i.date), "terminal_count": i.terminal_count})
return dump_response(DailyTerminalStatisticResponse, {"data": response_data})
return jsonify({"data": response_data})
@console_ns.route("/apps/<uuid:app_id>/statistics/token-costs")
class DailyTokenCostStatistic(Resource):
@console_ns.doc("get_daily_token_cost_statistics")
@console_ns.doc(description="Get daily token cost statistics for an application")
@console_ns.doc(params={"app_id": "Application ID", **query_params_from_model(StatisticTimeRangeQuery)})
@console_ns.doc(params={"app_id": "Application ID"})
@console_ns.doc(params=query_params_from_model(StatisticTimeRangeQuery))
@console_ns.response(
200,
"Daily token cost statistics retrieved successfully",
@@ -380,14 +370,15 @@ WHERE
{"date": str(i.date), "token_count": i.token_count, "total_price": i.total_price, "currency": "USD"}
)
return dump_response(DailyTokenCostStatisticResponse, {"data": response_data})
return jsonify({"data": response_data})
@console_ns.route("/apps/<uuid:app_id>/statistics/average-session-interactions")
class AverageSessionInteractionStatistic(Resource):
@console_ns.doc("get_average_session_interaction_statistics")
@console_ns.doc(description="Get average session interaction statistics for an application")
@console_ns.doc(params={"app_id": "Application ID", **query_params_from_model(StatisticTimeRangeQuery)})
@console_ns.doc(params={"app_id": "Application ID"})
@console_ns.doc(params=query_params_from_model(StatisticTimeRangeQuery))
@console_ns.response(
200,
"Average session interaction statistics retrieved successfully",
@@ -459,14 +450,15 @@ ORDER BY
{"date": str(i.date), "interactions": float(i.interactions.quantize(Decimal("0.01")))}
)
return dump_response(AverageSessionInteractionStatisticResponse, {"data": response_data})
return jsonify({"data": response_data})
@console_ns.route("/apps/<uuid:app_id>/statistics/user-satisfaction-rate")
class UserSatisfactionRateStatistic(Resource):
@console_ns.doc("get_user_satisfaction_rate_statistics")
@console_ns.doc(description="Get user satisfaction rate statistics for an application")
@console_ns.doc(params={"app_id": "Application ID", **query_params_from_model(StatisticTimeRangeQuery)})
@console_ns.doc(params={"app_id": "Application ID"})
@console_ns.doc(params=query_params_from_model(StatisticTimeRangeQuery))
@console_ns.response(
200,
"User satisfaction rate statistics retrieved successfully",
@@ -528,14 +520,15 @@ WHERE
}
)
return dump_response(UserSatisfactionRateStatisticResponse, {"data": response_data})
return jsonify({"data": response_data})
@console_ns.route("/apps/<uuid:app_id>/statistics/average-response-time")
class AverageResponseTimeStatistic(Resource):
@console_ns.doc("get_average_response_time_statistics")
@console_ns.doc(description="Get average response time statistics for an application")
@console_ns.doc(params={"app_id": "Application ID", **query_params_from_model(StatisticTimeRangeQuery)})
@console_ns.doc(params={"app_id": "Application ID"})
@console_ns.doc(params=query_params_from_model(StatisticTimeRangeQuery))
@console_ns.response(
200,
"Average response time statistics retrieved successfully",
@@ -588,14 +581,15 @@ WHERE
for i in rs:
response_data.append({"date": str(i.date), "latency": round(i.latency * 1000, 4)})
return dump_response(AverageResponseTimeStatisticResponse, {"data": response_data})
return jsonify({"data": response_data})
@console_ns.route("/apps/<uuid:app_id>/statistics/tokens-per-second")
class TokensPerSecondStatistic(Resource):
@console_ns.doc("get_tokens_per_second_statistics")
@console_ns.doc(description="Get tokens per second statistics for an application")
@console_ns.doc(params={"app_id": "Application ID", **query_params_from_model(StatisticTimeRangeQuery)})
@console_ns.doc(params={"app_id": "Application ID"})
@console_ns.doc(params=query_params_from_model(StatisticTimeRangeQuery))
@console_ns.response(
200,
"Tokens per second statistics retrieved successfully",
@@ -651,4 +645,4 @@ WHERE
for i in rs:
response_data.append({"date": str(i.date), "tps": round(i.tokens_per_second, 4)})
return dump_response(TokensPerSecondStatisticResponse, {"data": response_data})
return jsonify({"data": response_data})
+12 -99
View File
@@ -6,7 +6,7 @@ from typing import Any, NotRequired, TypedDict, cast
from flask import abort, request
from flask_restx import Resource, fields
from pydantic import AliasChoices, BaseModel, ConfigDict, Field, RootModel, ValidationError, field_validator
from pydantic import AliasChoices, BaseModel, Field, RootModel, ValidationError, field_validator
from sqlalchemy.orm import Session, sessionmaker
from werkzeug.exceptions import BadRequest, Forbidden, InternalServerError, NotFound
@@ -27,7 +27,7 @@ from controllers.console.app.error import (
DraftWorkflowNotSync,
)
from controllers.console.app.permission_keys import get_app_permission_keys
from controllers.console.app.wraps import get_app_model, with_session
from controllers.console.app.wraps import get_app_model
from controllers.console.wraps import (
RBACPermission,
RBACResourceScope,
@@ -78,7 +78,6 @@ from repositories.workflow_collaboration_repository import WORKFLOW_ONLINE_USERS
from services.app_generate_service import AppGenerateService
from services.errors.app import IsDraftWorkflowError, WorkflowHashNotEqualError, WorkflowNotFoundError
from services.errors.llm import InvokeRateLimitError
from services.workflow_ref_service import WorkflowRefService
from services.workflow_service import DraftWorkflowDeletionError, WorkflowInUseError, WorkflowService
logger = logging.getLogger(__name__)
@@ -159,71 +158,8 @@ class ConvertToWorkflowPayload(BaseModel):
icon_background: str | None = None
class WorkflowFeatureTogglePayload(BaseModel):
model_config = ConfigDict(extra="allow")
enabled: bool | None = None
class WorkflowSuggestedQuestionsAfterAnswerPayload(WorkflowFeatureTogglePayload):
model: dict[str, Any] | None = None
prompt: str | None = None
class WorkflowTextToSpeechPayload(WorkflowFeatureTogglePayload):
language: str | None = None
voice: str | None = None
autoPlay: str | None = None
class WorkflowSensitiveWordAvoidancePayload(WorkflowFeatureTogglePayload):
type: str | None = None
config: dict[str, Any] | None = None
class WorkflowFileUploadTransferPayload(WorkflowFeatureTogglePayload):
number_limits: int | None = None
transfer_methods: list[str] | None = None
class WorkflowFileUploadImagePayload(WorkflowFileUploadTransferPayload):
detail: str | None = None
class WorkflowFileUploadPreviewConfigPayload(BaseModel):
mode: str | None = None
file_type_list: list[str] | None = None
class WorkflowFileUploadPayload(WorkflowFeatureTogglePayload):
allowed_file_types: list[str] | None = None
allowed_file_extensions: list[str] | None = None
allowed_file_upload_methods: list[str] | None = None
number_limits: int | None = None
image: WorkflowFileUploadImagePayload | None = None
document: WorkflowFileUploadTransferPayload | None = None
audio: WorkflowFileUploadTransferPayload | None = None
video: WorkflowFileUploadTransferPayload | None = None
custom: WorkflowFileUploadTransferPayload | None = None
preview_config: WorkflowFileUploadPreviewConfigPayload | None = None
fileUploadConfig: dict[str, Any] | None = None
class WorkflowFeaturesConfigPayload(BaseModel):
model_config = ConfigDict(extra="allow")
opening_statement: str | None = None
suggested_questions: list[str] | None = None
suggested_questions_after_answer: WorkflowSuggestedQuestionsAfterAnswerPayload | None = None
text_to_speech: WorkflowTextToSpeechPayload | None = None
speech_to_text: WorkflowFeatureTogglePayload | None = None
retriever_resource: WorkflowFeatureTogglePayload | None = None
sensitive_word_avoidance: WorkflowSensitiveWordAvoidancePayload | None = None
file_upload: WorkflowFileUploadPayload | None = None
class WorkflowFeaturesPayload(BaseModel):
features: WorkflowFeaturesConfigPayload = Field(
features: dict[str, Any] = Field(
...,
description="Workflow feature configuration",
)
@@ -407,15 +343,6 @@ register_schema_models(
ConvertToWorkflowPayload,
WorkflowListQuery,
WorkflowUpdatePayload,
WorkflowFeatureTogglePayload,
WorkflowSuggestedQuestionsAfterAnswerPayload,
WorkflowTextToSpeechPayload,
WorkflowSensitiveWordAvoidancePayload,
WorkflowFileUploadTransferPayload,
WorkflowFileUploadImagePayload,
WorkflowFileUploadPreviewConfigPayload,
WorkflowFileUploadPayload,
WorkflowFeaturesConfigPayload,
WorkflowFeaturesPayload,
WorkflowOnlineUsersPayload,
DraftWorkflowTriggerRunPayload,
@@ -631,8 +558,7 @@ class AdvancedChatDraftWorkflowRunApi(Resource):
@get_app_model(mode=[AppMode.ADVANCED_CHAT])
@with_current_user
@edit_permission_required
@with_session
def post(self, session: Session, current_user: Account, app_model: App):
def post(self, current_user: Account, app_model: App):
"""
Run draft workflow
"""
@@ -645,12 +571,7 @@ class AdvancedChatDraftWorkflowRunApi(Resource):
try:
response = AppGenerateService.generate(
session=session,
app_model=app_model,
user=current_user,
args=args,
invoke_from=InvokeFrom.DEBUGGER,
streaming=True,
app_model=app_model, user=current_user, args=args, invoke_from=InvokeFrom.DEBUGGER, streaming=True
)
return helper.compact_generate_response(response)
@@ -1051,8 +972,7 @@ class DraftWorkflowRunApi(Resource):
@get_app_model(mode=[AppMode.WORKFLOW])
@with_current_user
@edit_permission_required
@with_session
def post(self, session: Session, current_user: Account, app_model: App):
def post(self, current_user: Account, app_model: App):
"""
Run draft workflow
"""
@@ -1064,7 +984,6 @@ class DraftWorkflowRunApi(Resource):
try:
response = AppGenerateService.generate(
session=session,
app_model=app_model,
user=current_user,
args=args,
@@ -1355,7 +1274,7 @@ class WorkflowFeaturesApi(Resource):
def post(self, current_user: Account, app_model: App):
args = WorkflowFeaturesPayload.model_validate(console_ns.payload or {})
features = args.features.model_dump(mode="json", exclude_unset=True)
features = args.features
workflow_service = WorkflowService()
workflow_service.update_draft_workflow_features(app_model=app_model, features=features, account=current_user)
@@ -1487,15 +1406,15 @@ class WorkflowByIdApi(Resource):
return {"message": "No valid fields to update"}, 400
workflow_service = WorkflowService()
workflow_ref = WorkflowRefService.create_app_workflow_ref(app_model, workflow_id)
# Create a session and manage the transaction
with sessionmaker(db.engine, expire_on_commit=False).begin() as session:
workflow = workflow_service.update_workflow(
session=session,
workflow_id=workflow_id,
tenant_id=app_model.tenant_id,
account_id=current_user.id,
data=update_data,
workflow_ref=workflow_ref,
)
if not workflow:
@@ -1515,14 +1434,12 @@ class WorkflowByIdApi(Resource):
Delete workflow
"""
workflow_service = WorkflowService()
workflow_ref = WorkflowRefService.create_app_workflow_ref(app_model, workflow_id)
# Create a session and manage the transaction
with sessionmaker(db.engine).begin() as session:
try:
workflow_service.delete_workflow(
session=session,
workflow_ref=workflow_ref,
session=session, workflow_id=workflow_id, tenant_id=app_model.tenant_id
)
except WorkflowInUseError as e:
abort(400, description=str(e))
@@ -1598,8 +1515,7 @@ class DraftWorkflowTriggerRunApi(Resource):
@get_app_model(mode=[AppMode.WORKFLOW])
@with_current_user
@edit_permission_required
@with_session
def post(self, session: Session, current_user: Account, app_model: App):
def post(self, current_user: Account, app_model: App):
"""
Poll for trigger events and execute full workflow when event arrives
"""
@@ -1627,7 +1543,6 @@ class DraftWorkflowTriggerRunApi(Resource):
workflow_args[SKIP_PREPARE_USER_INPUTS_KEY] = True
return helper.compact_generate_response(
AppGenerateService.generate(
session=session,
app_model=app_model,
user=current_user,
args=workflow_args,
@@ -1750,8 +1665,7 @@ class DraftWorkflowTriggerRunAllApi(Resource):
@with_current_user
@edit_permission_required
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_TEST_AND_RUN)
@with_session
def post(self, session: Session, current_user: Account, app_model: App):
def post(self, current_user: Account, app_model: App):
"""
Full workflow debug when the start node is a trigger
"""
@@ -1783,7 +1697,6 @@ class DraftWorkflowTriggerRunAllApi(Resource):
workflow_args[SKIP_PREPARE_USER_INPUTS_KEY] = True
response = AppGenerateService.generate(
session=session,
app_model=app_model,
user=current_user,
args=workflow_args,
@@ -189,14 +189,14 @@ class WorkflowCommentReplyUpdate(ResponseModel):
register_schema_models(
console_ns,
AccountWithRole,
WorkflowCommentMentionUsersPayload,
WorkflowCommentCreatePayload,
WorkflowCommentUpdatePayload,
WorkflowCommentReplyPayload,
)
register_response_schema_models(
console_ns,
AccountWithRole,
WorkflowCommentMentionUsersPayload,
WorkflowCommentAccount,
WorkflowCommentReply,
WorkflowCommentMention,
@@ -6,7 +6,7 @@ from uuid import UUID
from flask import Response, request
from flask_restx import Resource, fields, marshal, marshal_with
from pydantic import BaseModel, ConfigDict, Field
from pydantic import BaseModel, Field
from sqlalchemy.orm import sessionmaker
from controllers.common.errors import InvalidArgumentError, NotFoundError
@@ -79,33 +79,15 @@ class WorkflowDraftVariableUpdatePayload(BaseModel):
value: Any | None = Field(default=None, description="Variable value")
class WorkflowVariableItemPayload(BaseModel):
model_config = ConfigDict(extra="allow")
id: str | None = None
name: str | None = None
value_type: str | None = None
value: Any | None = None
description: str | None = None
class ConversationVariableItemPayload(WorkflowVariableItemPayload):
pass
class EnvironmentVariableItemPayload(WorkflowVariableItemPayload):
pass
class ConversationVariableUpdatePayload(BaseModel):
conversation_variables: list[ConversationVariableItemPayload] = Field(
conversation_variables: list[dict[str, Any]] = Field(
...,
description="Conversation variables for the draft workflow",
)
class EnvironmentVariableUpdatePayload(BaseModel):
environment_variables: list[EnvironmentVariableItemPayload] = Field(
environment_variables: list[dict[str, Any]] = Field(
...,
description="Environment variables for the draft workflow",
)
@@ -132,9 +114,7 @@ register_schema_models(
console_ns,
WorkflowDraftVariableListQuery,
WorkflowDraftVariableUpdatePayload,
ConversationVariableItemPayload,
ConversationVariableUpdatePayload,
EnvironmentVariableItemPayload,
EnvironmentVariableUpdatePayload,
)
register_response_schema_models(console_ns, SimpleResultResponse, EnvironmentVariableListResponse)
@@ -635,9 +615,7 @@ class ConversationVariableCollectionApi(Resource):
workflow_service = WorkflowService()
conversation_variables_list = [
variable.model_dump(mode="json", exclude_unset=True) for variable in payload.conversation_variables
]
conversation_variables_list = payload.conversation_variables
conversation_variables = [
variable_factory.build_conversation_variable_from_mapping(obj) for obj in conversation_variables_list
]
@@ -729,9 +707,7 @@ class EnvironmentVariableCollectionApi(Resource):
workflow_service = WorkflowService()
environment_variables_list = [
variable.model_dump(mode="json", exclude_unset=True) for variable in payload.environment_variables
]
environment_variables_list = payload.environment_variables
environment_variables = [
variable_factory.build_environment_variable_from_mapping(obj) for obj in environment_variables_list
]
+1 -1
View File
@@ -23,7 +23,6 @@ from controllers.console.wraps import (
with_current_user,
)
from core.workflow.human_input_forms import load_form_tokens_by_form_id as _load_form_tokens_by_form_id
from core.workflow.nodes.human_input.pause_reason import HumanInputRequired
from extensions.ext_database import db
from fields.base import ResponseModel
from fields.workflow_run_fields import (
@@ -34,6 +33,7 @@ from fields.workflow_run_fields import (
WorkflowRunNodeExecutionResponse,
WorkflowRunPaginationResponse,
)
from graphon.entities.pause_reason import HumanInputRequired
from graphon.enums import WorkflowExecutionStatus
from libs.archive_storage import ArchiveStorageNotConfiguredError, get_archive_storage
from libs.custom_inputs import time_duration
@@ -12,7 +12,6 @@ from configs import dify_config
from controllers.common.schema import query_params_from_model, register_schema_models
from extensions.ext_database import db
from fields.base import ResponseModel
from libs.helper import dump_response
from libs.login import login_required
from models.enums import AppTriggerStatus
from models.model import App, AppMode
@@ -122,7 +121,7 @@ class WebhookTriggerApi(Resource):
if not webhook_trigger:
raise NotFound("Webhook trigger not found for this node")
return dump_response(WebhookTriggerResponse, webhook_trigger)
return WebhookTriggerResponse.model_validate(webhook_trigger, from_attributes=True).model_dump(mode="json")
@console_ns.route("/apps/<uuid:app_id>/triggers")
@@ -161,7 +160,9 @@ class AppTriggersApi(Resource):
else:
trigger.icon = "" # type: ignore
return dump_response(WorkflowTriggerListResponse, {"data": triggers})
return WorkflowTriggerListResponse.model_validate({"data": triggers}, from_attributes=True).model_dump(
mode="json"
)
@console_ns.route("/apps/<uuid:app_id>/trigger-enable")
@@ -203,4 +204,4 @@ class AppTriggerEnableApi(Resource):
else:
trigger.icon = "" # type: ignore
return dump_response(WorkflowTriggerResponse, trigger)
return WorkflowTriggerResponse.model_validate(trigger, from_attributes=True).model_dump(mode="json")
+50 -8
View File
@@ -1,26 +1,26 @@
"""Controller decorators for console app resources.
App-loading decorators prefer a session injected by
`controllers.common.session.with_session` when present, while still supporting
existing handlers that have not been migrated yet and still rely on
Flask-SQLAlchemy's scoped `db.session`.
`with_session` opens one SQLAlchemy session for a request handler and injects it
as the first argument after `self`. Handlers use a transaction by default so
migrated write paths keep commit/rollback handling; pure read handlers may opt
out with `write=False`. App-loading decorators prefer that injected session when
present, while still supporting existing handlers that have not been migrated
yet and still rely on Flask-SQLAlchemy's scoped `db.session`.
"""
from collections.abc import Callable
from functools import wraps
from typing import cast, overload
from typing import Concatenate, cast, overload
from sqlalchemy import select
from sqlalchemy.orm import Session
from controllers.common.session import with_session
from controllers.console.app.error import AppNotFoundError
from core.db.session_factory import session_factory
from extensions.ext_database import db
from libs.login import current_account_with_tenant
from models import App, AppMode
__all__ = ["get_app_model", "get_app_model_with_trial", "with_session"]
def _load_app_model(session: Session, app_id: str) -> App | None:
"""Load the tenant-scoped app row with the request session owned by `with_session`."""
@@ -45,6 +45,48 @@ def _load_app_model_with_trial(app_id: str) -> App | None:
return app_model
@overload
def with_session[T, **P, R](
view: Callable[Concatenate[T, Session, P], R],
*,
write: bool = True,
) -> Callable[Concatenate[T, P], R]: ...
@overload
def with_session[T, **P, R](
view: None = None,
*,
write: bool = True,
) -> Callable[[Callable[Concatenate[T, Session, P], R]], Callable[Concatenate[T, P], R]]: ...
def with_session[T, **P, R](
view: Callable[Concatenate[T, Session, P], R] | None = None,
*,
write: bool = True,
) -> (
Callable[Concatenate[T, P], R] | Callable[[Callable[Concatenate[T, Session, P], R]], Callable[Concatenate[T, P], R]]
):
"""Inject a request-scoped session, using a transaction only for write handlers."""
def decorator(view: Callable[Concatenate[T, Session, P], R]) -> Callable[Concatenate[T, P], R]:
@wraps(view)
def wrapper(self: T, *args: P.args, **kwargs: P.kwargs) -> R:
if write:
with session_factory.get_session_maker().begin() as session:
return view(self, session, *args, **kwargs)
with session_factory.create_session() as session:
return view(self, session, *args, **kwargs)
return wrapper
if view is None:
return decorator
return decorator(view)
def _get_injected_session(args: tuple[object, ...]) -> Session | None:
"""Return the request session inserted by `with_session`, if this handler has been migrated."""
if len(args) < 2:
+4 -10
View File
@@ -16,8 +16,6 @@ from controllers.console.wraps import (
with_current_user,
)
from enums.cloud_plan import CloudPlan
from extensions.ext_database import db
from fields.base import ResponseModel
from libs.login import login_required
from models import Account
from services.billing_service import BillingService
@@ -36,12 +34,8 @@ class BillingResponse(RootModel[dict[str, Any]]):
root: dict[str, Any]
class BillingInvoiceResponse(ResponseModel):
url: str
register_schema_models(console_ns, SubscriptionQuery, PartnerTenantsPayload)
register_response_schema_models(console_ns, BillingResponse, BillingInvoiceResponse)
register_response_schema_models(console_ns, BillingResponse)
@console_ns.route("/billing/subscription")
@@ -56,13 +50,13 @@ class Subscription(Resource):
@with_current_tenant_id
def get(self, current_tenant_id: str, current_user: Account):
args = SubscriptionQuery.model_validate(request.args.to_dict(flat=True))
BillingService.is_tenant_owner_or_admin(db.session, current_user)
BillingService.is_tenant_owner_or_admin(current_user)
return BillingService.get_subscription(args.plan, args.interval, current_user.email, current_tenant_id)
@console_ns.route("/billing/invoices")
class Invoices(Resource):
@console_ns.response(200, "Success", console_ns.models[BillingInvoiceResponse.__name__])
@console_ns.response(200, "Success", console_ns.models[BillingResponse.__name__])
@setup_required
@login_required
@account_initialization_required
@@ -70,7 +64,7 @@ class Invoices(Resource):
@with_current_user
@with_current_tenant_id
def get(self, current_tenant_id: str, current_user: Account):
BillingService.is_tenant_owner_or_admin(db.session, current_user)
BillingService.is_tenant_owner_or_admin(current_user)
return BillingService.get_invoices(current_user.email, current_tenant_id)
+66 -65
View File
@@ -243,71 +243,72 @@ class DataSourceNotionListApi(Resource):
if not credential:
raise NotFound("Credential not found.")
exist_page_ids = []
# import notion in the exist dataset
if query.dataset_id:
dataset = DatasetService.get_dataset(query.dataset_id, db.session)
if not dataset:
raise NotFound("Dataset not found.")
if dataset.data_source_type != "notion_import":
raise ValueError("Dataset is not notion type.")
with sessionmaker(db.engine).begin() as session:
# import notion in the exist dataset
if query.dataset_id:
dataset = DatasetService.get_dataset(query.dataset_id)
if not dataset:
raise NotFound("Dataset not found.")
if dataset.data_source_type != "notion_import":
raise ValueError("Dataset is not notion type.")
documents = db.session.scalars(
select(Document).where(
Document.dataset_id == query.dataset_id,
Document.tenant_id == current_tenant_id,
Document.data_source_type == "notion_import",
Document.enabled.is_(True),
)
).all()
if documents:
for document in documents:
data_source_info = json.loads(document.data_source_info)
exist_page_ids.append(data_source_info["notion_page_id"])
# get all authorized pages
from core.datasource.datasource_manager import DatasourceManager
documents = session.scalars(
select(Document).where(
Document.dataset_id == query.dataset_id,
Document.tenant_id == current_tenant_id,
Document.data_source_type == "notion_import",
Document.enabled.is_(True),
)
).all()
if documents:
for document in documents:
data_source_info = json.loads(document.data_source_info)
exist_page_ids.append(data_source_info["notion_page_id"])
# get all authorized pages
from core.datasource.datasource_manager import DatasourceManager
datasource_runtime = DatasourceManager.get_datasource_runtime(
provider_id="langgenius/notion_datasource/notion_datasource",
datasource_name="notion_datasource",
tenant_id=current_tenant_id,
datasource_type=DatasourceProviderType.ONLINE_DOCUMENT,
)
datasource_provider_service = DatasourceProviderService()
if credential:
datasource_runtime.runtime.credentials = credential
datasource_runtime = cast(OnlineDocumentDatasourcePlugin, datasource_runtime)
online_document_result: Generator[OnlineDocumentPagesMessage, None, None] = (
datasource_runtime.get_online_document_pages(
user_id=current_user.id,
datasource_parameters={},
provider_type=datasource_runtime.datasource_provider_type(),
datasource_runtime = DatasourceManager.get_datasource_runtime(
provider_id="langgenius/notion_datasource/notion_datasource",
datasource_name="notion_datasource",
tenant_id=current_tenant_id,
datasource_type=DatasourceProviderType.ONLINE_DOCUMENT,
)
)
try:
pages = []
workspace_info = {}
for message in online_document_result:
result = message.result
for info in result:
workspace_info = {
"workspace_id": info.workspace_id,
"workspace_name": info.workspace_name,
"workspace_icon": info.workspace_icon,
}
for page in info.pages:
page_info = {
"page_id": page.page_id,
"page_name": page.page_name,
"type": page.type,
"parent_id": page.parent_id,
"is_bound": page.page_id in exist_page_ids,
"page_icon": page.page_icon,
datasource_provider_service = DatasourceProviderService()
if credential:
datasource_runtime.runtime.credentials = credential
datasource_runtime = cast(OnlineDocumentDatasourcePlugin, datasource_runtime)
online_document_result: Generator[OnlineDocumentPagesMessage, None, None] = (
datasource_runtime.get_online_document_pages(
user_id=current_user.id,
datasource_parameters={},
provider_type=datasource_runtime.datasource_provider_type(),
)
)
try:
pages = []
workspace_info = {}
for message in online_document_result:
result = message.result
for info in result:
workspace_info = {
"workspace_id": info.workspace_id,
"workspace_name": info.workspace_name,
"workspace_icon": info.workspace_icon,
}
pages.append(page_info)
except Exception as e:
raise e
notion_info = [{**workspace_info, "pages": pages}] if workspace_info else []
return dump_response(NotionIntegrateInfoListResponse, {"notion_info": notion_info}), 200
for page in info.pages:
page_info = {
"page_id": page.page_id,
"page_name": page.page_name,
"type": page.type,
"parent_id": page.parent_id,
"is_bound": page.page_id in exist_page_ids,
"page_icon": page.page_icon,
}
pages.append(page_info)
except Exception as e:
raise e
notion_info = [{**workspace_info, "pages": pages}] if workspace_info else []
return dump_response(NotionIntegrateInfoListResponse, {"notion_info": notion_info}), 200
@console_ns.route("/notion/pages/<uuid:page_id>/<string:page_type>/preview")
@@ -400,11 +401,11 @@ class DataSourceNotionDatasetSyncApi(Resource):
@rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_CREATE_AND_MANAGEMENT)
def get(self, dataset_id: UUID) -> tuple[dict[str, str], int]:
dataset_id_str = str(dataset_id)
dataset = DatasetService.get_dataset(dataset_id_str, db.session)
dataset = DatasetService.get_dataset(dataset_id_str)
if dataset is None:
raise NotFound("Dataset not found.")
documents = DocumentService.get_document_by_dataset_id(dataset_id_str, db.session)
documents = DocumentService.get_document_by_dataset_id(dataset_id_str)
for document in documents:
document_indexing_sync_task.delay(dataset_id_str, document.id)
return {"result": "success"}, 200
@@ -420,11 +421,11 @@ class DataSourceNotionDocumentSyncApi(Resource):
def get(self, dataset_id: UUID, document_id: UUID) -> tuple[dict[str, str], int]:
dataset_id_str = str(dataset_id)
document_id_str = str(document_id)
dataset = DatasetService.get_dataset(dataset_id_str, db.session)
dataset = DatasetService.get_dataset(dataset_id_str)
if dataset is None:
raise NotFound("Dataset not found.")
document = DocumentService.get_document(dataset_id_str, document_id_str, session=db.session)
document = DocumentService.get_document(dataset_id_str, document_id_str)
if document is None:
raise NotFound("Document not found.")
document_indexing_sync_task.delay(dataset_id_str, document_id_str)
+32 -40
View File
@@ -6,7 +6,6 @@ from flask import request
from flask_restx import Resource
from pydantic import BaseModel, Field, field_validator, model_validator
from sqlalchemy import func, select
from sqlalchemy.orm import Session
from werkzeug.exceptions import Forbidden, NotFound
import services
@@ -16,7 +15,6 @@ from controllers.common.schema import query_params_from_model, register_response
from controllers.console import console_ns
from controllers.console.apikey import ApiKeyItem, ApiKeyList
from controllers.console.app.error import ProviderNotInitializeError
from controllers.console.app.wraps import with_session
from controllers.console.datasets.error import DatasetInUseError, DatasetNameDuplicateError, IndexingEstimateError
from controllers.console.wraps import (
RBACPermission,
@@ -540,8 +538,7 @@ class DatasetListApi(Resource):
@cloud_edition_billing_rate_limit_check("knowledge")
@with_current_user
@with_current_tenant_id
@with_session
def post(self, session: Session, current_tenant_id: str, current_user: Account):
def post(self, current_tenant_id: str, current_user: Account):
payload = DatasetCreatePayload.model_validate(console_ns.payload or {})
# The role of the current user in the ta table must be admin, owner, or editor, or dataset_operator
@@ -555,7 +552,6 @@ class DatasetListApi(Resource):
try:
dataset = DatasetService.create_empty_dataset(
session=session,
tenant_id=current_tenant_id,
name=payload.name,
description=payload.description,
@@ -570,15 +566,15 @@ class DatasetListApi(Resource):
raise DatasetNameDuplicateError()
permission_keys_map = enterprise_rbac_service.RBACService.DatasetPermissions.batch_get(
current_tenant_id,
str(current_tenant_id),
current_user.id,
[dataset.id],
[str(dataset.id)],
)
item = DatasetDetailWithPartialMembersResponse.model_validate(dataset, from_attributes=True).model_dump(
mode="json"
)
item["permission_keys"] = permission_keys_map.get(dataset.id, [])
item["permission_keys"] = permission_keys_map.get(str(dataset.id), [])
return item, 201
@@ -602,15 +598,15 @@ class DatasetApi(Resource):
@with_current_tenant_id
def get(self, current_tenant_id: str, current_user: Account, dataset_id: UUID):
dataset_id_str = str(dataset_id)
dataset = DatasetService.get_dataset(dataset_id_str, db.session)
dataset = DatasetService.get_dataset(dataset_id_str)
if dataset is None:
raise NotFound("Dataset not found.")
try:
DatasetService.check_dataset_permission(dataset, current_user, db.session)
DatasetService.check_dataset_permission(dataset, current_user)
except services.errors.account.NoPermissionError as e:
raise Forbidden(str(e))
permissions = enterprise_rbac_service.RBACService.MyPermissions.get(
current_tenant_id,
str(current_tenant_id),
current_user.id,
dataset_id=dataset_id_str,
)
@@ -622,7 +618,7 @@ class DatasetApi(Resource):
provider_id = ModelProviderID(dataset.embedding_model_provider)
data["embedding_model_provider"] = str(provider_id)
if data.get("permission") == "partial_members":
part_users_list = DatasetPermissionService.get_dataset_partial_member_list(dataset_id_str, db.session)
part_users_list = DatasetPermissionService.get_dataset_partial_member_list(dataset_id_str)
data.update({"partial_member_list": part_users_list})
# check embedding setting
@@ -663,10 +659,9 @@ class DatasetApi(Resource):
@with_current_user
@with_current_tenant_id
@rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_EDIT)
@with_session
def patch(self, session: Session, current_tenant_id: str, current_user: Account, dataset_id: UUID):
def patch(self, current_tenant_id: str, current_user: Account, dataset_id: UUID):
dataset_id_str = str(dataset_id)
dataset = DatasetService.get_dataset(dataset_id_str, db.session)
dataset = DatasetService.get_dataset(dataset_id_str)
if dataset is None:
raise NotFound("Dataset not found.")
@@ -685,16 +680,16 @@ class DatasetApi(Resource):
# The role of the current user in the ta table must be admin, owner, editor, or dataset_operator
if not dify_config.RBAC_ENABLED:
DatasetPermissionService.check_permission(
session, current_user, dataset, payload.permission, payload.partial_member_list
current_user, dataset, payload.permission, payload.partial_member_list
)
dataset = DatasetService.update_dataset(session, dataset_id_str, payload_data, current_user)
dataset = DatasetService.update_dataset(dataset_id_str, payload_data, current_user)
if dataset is None:
raise NotFound("Dataset not found.")
permission_keys_map = enterprise_rbac_service.RBACService.DatasetPermissions.batch_get(
current_tenant_id,
str(current_tenant_id),
current_user.id,
[dataset_id_str],
)
@@ -703,14 +698,12 @@ class DatasetApi(Resource):
tenant_id = current_tenant_id
if payload.partial_member_list is not None and payload.permission == DatasetPermissionEnum.PARTIAL_TEAM:
DatasetPermissionService.update_partial_member_list(
tenant_id, dataset_id_str, payload.partial_member_list, db.session
)
DatasetPermissionService.update_partial_member_list(tenant_id, dataset_id_str, payload.partial_member_list)
# clear partial member list when permission is only_me or all_team_members
elif payload.permission in {DatasetPermissionEnum.ONLY_ME, DatasetPermissionEnum.ALL_TEAM}:
DatasetPermissionService.clear_partial_member_list(dataset_id_str, db.session)
DatasetPermissionService.clear_partial_member_list(dataset_id_str)
partial_member_list = DatasetPermissionService.get_dataset_partial_member_list(dataset_id_str, db.session)
partial_member_list = DatasetPermissionService.get_dataset_partial_member_list(dataset_id_str)
result_data.update({"partial_member_list": partial_member_list})
return result_data, 200
@@ -729,8 +722,8 @@ class DatasetApi(Resource):
raise Forbidden()
try:
if DatasetService.delete_dataset(dataset_id_str, current_user, db.session):
DatasetPermissionService.clear_partial_member_list(dataset_id_str, db.session)
if DatasetService.delete_dataset(dataset_id_str, current_user):
DatasetPermissionService.clear_partial_member_list(dataset_id_str)
return "", 204
else:
raise NotFound("Dataset not found.")
@@ -755,7 +748,7 @@ class DatasetUseCheckApi(Resource):
def get(self, dataset_id: UUID):
dataset_id_str = str(dataset_id)
dataset_is_using = DatasetService.dataset_use_check(dataset_id_str, db.session)
dataset_is_using = DatasetService.dataset_use_check(dataset_id_str)
return {"is_using": dataset_is_using}, 200
@@ -776,12 +769,12 @@ class DatasetQueryApi(Resource):
@rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_READONLY)
def get(self, current_user: Account, dataset_id: UUID):
dataset_id_str = str(dataset_id)
dataset = DatasetService.get_dataset(dataset_id_str, db.session)
dataset = DatasetService.get_dataset(dataset_id_str)
if dataset is None:
raise NotFound("Dataset not found.")
try:
DatasetService.check_dataset_permission(dataset, current_user, db.session)
DatasetService.check_dataset_permission(dataset, current_user)
except services.errors.account.NoPermissionError as e:
raise Forbidden(str(e))
@@ -917,16 +910,16 @@ class DatasetRelatedAppListApi(Resource):
@rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_READONLY)
def get(self, current_user: Account, dataset_id: UUID):
dataset_id_str = str(dataset_id)
dataset = DatasetService.get_dataset(dataset_id_str, db.session)
dataset = DatasetService.get_dataset(dataset_id_str)
if dataset is None:
raise NotFound("Dataset not found.")
try:
DatasetService.check_dataset_permission(dataset, current_user, db.session)
DatasetService.check_dataset_permission(dataset, current_user)
except services.errors.account.NoPermissionError as e:
raise Forbidden(str(e))
app_dataset_joins = DatasetService.get_related_apps(dataset.id, db.session)
app_dataset_joins = DatasetService.get_related_apps(dataset.id)
related_apps = []
for app_dataset_join in app_dataset_joins:
@@ -1101,7 +1094,7 @@ class DatasetEnableApiApi(Resource):
def post(self, dataset_id: UUID, status: str):
dataset_id_str = str(dataset_id)
DatasetService.update_dataset_api_status(dataset_id_str, status == "enable", db.session)
DatasetService.update_dataset_api_status(dataset_id_str, status == "enable")
return {"result": "success"}, 200
@@ -1170,10 +1163,10 @@ class DatasetErrorDocs(Resource):
@rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_READONLY)
def get(self, dataset_id: UUID):
dataset_id_str = str(dataset_id)
dataset = DatasetService.get_dataset(dataset_id_str, db.session)
dataset = DatasetService.get_dataset(dataset_id_str)
if dataset is None:
raise NotFound("Dataset not found.")
results = DocumentService.get_error_documents_by_dataset_id(dataset_id_str, db.session)
results = DocumentService.get_error_documents_by_dataset_id(dataset_id_str)
return dump_response(ErrorDocsResponse, {"data": results, "total": len(results)}), 200
@@ -1197,15 +1190,15 @@ class DatasetPermissionUserListApi(Resource):
@rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_READONLY)
def get(self, current_user: Account, dataset_id: UUID):
dataset_id_str = str(dataset_id)
dataset = DatasetService.get_dataset(dataset_id_str, db.session)
dataset = DatasetService.get_dataset(dataset_id_str)
if dataset is None:
raise NotFound("Dataset not found.")
try:
DatasetService.check_dataset_permission(dataset, current_user, db.session)
DatasetService.check_dataset_permission(dataset, current_user)
except services.errors.account.NoPermissionError as e:
raise Forbidden(str(e))
partial_members_list = DatasetPermissionService.get_dataset_partial_member_list(dataset_id_str, db.session)
partial_members_list = DatasetPermissionService.get_dataset_partial_member_list(dataset_id_str)
return dump_response(PartialMemberListResponse, {"data": partial_members_list}), 200
@@ -1227,8 +1220,7 @@ class DatasetAutoDisableLogApi(Resource):
@rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_READONLY)
def get(self, dataset_id: UUID):
dataset_id_str = str(dataset_id)
dataset = DatasetService.get_dataset(dataset_id_str, db.session)
dataset = DatasetService.get_dataset(dataset_id_str)
if dataset is None:
raise NotFound("Dataset not found.")
auto_disable_logs = DatasetService.get_dataset_auto_disable_logs(dataset_id_str, db.session)
return dump_response(AutoDisableLogsResponse, auto_disable_logs), 200
return dump_response(AutoDisableLogsResponse, DatasetService.get_dataset_auto_disable_logs(dataset_id_str)), 200
@@ -46,11 +46,9 @@ from graphon.model_runtime.errors.invoke import InvokeAuthorizationError
from libs.datetime_utils import naive_utc_now
from libs.helper import dump_response, to_timestamp
from libs.login import login_required
from libs.pagination import paginate_query
from models import Account, DatasetProcessRule, Document, DocumentSegment, UploadFile
from models.dataset import DocumentPipelineExecutionLog
from models.enums import IndexingStatus, SegmentStatus
from services.dataset_ref_service import DatasetRefService
from services.dataset_service import DatasetService, DocumentService
from services.entities.knowledge_entities.knowledge_entities import KnowledgeConfig, ProcessRule, RetrievalModel
from services.file_service import FileService
@@ -183,16 +181,16 @@ class DocumentResource(Resource):
def get_document(
self, dataset_id: str, document_id: str, current_user: Account, current_tenant_id: str
) -> Document:
dataset = DatasetService.get_dataset(dataset_id, db.session)
dataset = DatasetService.get_dataset(dataset_id)
if not dataset:
raise NotFound("Dataset not found.")
try:
DatasetService.check_dataset_permission(dataset, current_user, db.session)
DatasetService.check_dataset_permission(dataset, current_user)
except services.errors.account.NoPermissionError as e:
raise Forbidden(str(e))
document = DocumentService.get_document(dataset_id, document_id, session=db.session)
document = DocumentService.get_document(dataset_id, document_id)
if not document:
raise NotFound("Document not found.")
@@ -203,16 +201,16 @@ class DocumentResource(Resource):
return document
def get_batch_documents(self, dataset_id: str, batch: str, current_user: Account) -> Sequence[Document]:
dataset = DatasetService.get_dataset(dataset_id, db.session)
dataset = DatasetService.get_dataset(dataset_id)
if not dataset:
raise NotFound("Dataset not found.")
try:
DatasetService.check_dataset_permission(dataset, current_user, db.session)
DatasetService.check_dataset_permission(dataset, current_user)
except services.errors.account.NoPermissionError as e:
raise Forbidden(str(e))
documents = DocumentService.get_batch_documents(dataset_id, batch, db.session)
documents = DocumentService.get_batch_documents(dataset_id, batch)
if not documents:
raise NotFound("Documents not found.")
@@ -243,13 +241,13 @@ class GetProcessRuleApi(Resource):
# get the latest process rule
document = db.get_or_404(Document, document_id)
dataset = DatasetService.get_dataset(document.dataset_id, db.session)
dataset = DatasetService.get_dataset(document.dataset_id)
if not dataset:
raise NotFound("Dataset not found.")
try:
DatasetService.check_dataset_permission(dataset, current_user, db.session)
DatasetService.check_dataset_permission(dataset, current_user)
except services.errors.account.NoPermissionError as e:
raise Forbidden(str(e))
@@ -319,12 +317,12 @@ class DatasetDocumentListApi(Resource):
)
except (ArgumentTypeError, ValueError, Exception):
fetch = False
dataset = DatasetService.get_dataset(dataset_id_str, db.session)
dataset = DatasetService.get_dataset(dataset_id_str)
if not dataset:
raise NotFound("Dataset not found.")
try:
DatasetService.check_dataset_permission(dataset, current_user, db.session)
DatasetService.check_dataset_permission(dataset, current_user)
except services.errors.account.NoPermissionError as e:
raise Forbidden(str(e))
@@ -369,7 +367,7 @@ class DatasetDocumentListApi(Resource):
desc(Document.position),
)
paginated_documents = paginate_query(query, page=page, per_page=limit, max_per_page=100)
paginated_documents = db.paginate(select=query, page=page, per_page=limit, max_per_page=100, error_out=False)
documents = paginated_documents.items
DocumentService.enrich_documents_with_summary_index_status(
@@ -423,7 +421,7 @@ class DatasetDocumentListApi(Resource):
def post(self, current_user: Account, dataset_id: UUID):
dataset_id_str = str(dataset_id)
dataset = DatasetService.get_dataset(dataset_id_str, db.session)
dataset = DatasetService.get_dataset(dataset_id_str)
if not dataset:
raise NotFound("Dataset not found.")
@@ -433,7 +431,7 @@ class DatasetDocumentListApi(Resource):
raise Forbidden()
try:
DatasetService.check_dataset_permission(dataset, current_user, db.session)
DatasetService.check_dataset_permission(dataset, current_user)
except services.errors.account.NoPermissionError as e:
raise Forbidden(str(e))
@@ -446,10 +444,8 @@ class DatasetDocumentListApi(Resource):
DocumentService.document_create_args_validate(knowledge_config)
try:
documents, batch = DocumentService.save_document_with_dataset_id(
dataset, knowledge_config, current_user, session=db.session
)
dataset = DatasetService.get_dataset(dataset_id_str, db.session)
documents, batch = DocumentService.save_document_with_dataset_id(dataset, knowledge_config, current_user)
dataset = DatasetService.get_dataset(dataset_id_str)
except ProviderTokenNotInitError as ex:
raise ProviderNotInitializeError(ex.description)
@@ -468,7 +464,7 @@ class DatasetDocumentListApi(Resource):
@rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_EDIT)
def delete(self, dataset_id: UUID):
dataset_id_str = str(dataset_id)
dataset = DatasetService.get_dataset(dataset_id_str, db.session)
dataset = DatasetService.get_dataset(dataset_id_str)
if dataset is None:
raise NotFound("Dataset not found.")
# check user's model setting
@@ -476,8 +472,7 @@ class DatasetDocumentListApi(Resource):
try:
document_ids = request.args.getlist("document_id")
dataset_ref = DatasetRefService.create_dataset_ref(dataset)
DocumentService.delete_documents(dataset_ref, document_ids, dataset.doc_form, db.session)
DocumentService.delete_documents(dataset, document_ids)
except services.errors.document.DocumentIndexingError:
raise DocumentIndexingError("Cannot delete document during indexing.")
@@ -536,7 +531,6 @@ class DatasetInitApi(Resource):
tenant_id=current_tenant_id,
knowledge_config=knowledge_config,
account=current_user,
session=db.session,
)
except ProviderTokenNotInitError as ex:
raise ProviderNotInitializeError(ex.description)
@@ -873,7 +867,7 @@ class DocumentApi(DocumentResource):
if metadata == "only":
response = {"id": document.id, "doc_type": document.doc_type, "doc_metadata": document.doc_metadata_details}
elif metadata == "without":
dataset_process_rules = DatasetService.get_process_rules(dataset_id_str, db.session)
dataset_process_rules = DatasetService.get_process_rules(dataset_id_str)
document_process_rules = document.dataset_process_rule.to_dict() if document.dataset_process_rule else {}
response = {
"id": document.id,
@@ -907,7 +901,7 @@ class DocumentApi(DocumentResource):
"need_summary": document.need_summary if document.need_summary is not None else False,
}
else:
dataset_process_rules = DatasetService.get_process_rules(dataset_id_str, db.session)
dataset_process_rules = DatasetService.get_process_rules(dataset_id_str)
document_process_rules = document.dataset_process_rule.to_dict() if document.dataset_process_rule else {}
response = {
"id": document.id,
@@ -956,7 +950,7 @@ class DocumentApi(DocumentResource):
def delete(self, current_tenant_id: str, current_user: Account, dataset_id: UUID, document_id: UUID):
dataset_id_str = str(dataset_id)
document_id_str = str(document_id)
dataset = DatasetService.get_dataset(dataset_id_str, db.session)
dataset = DatasetService.get_dataset(dataset_id_str)
if dataset is None:
raise NotFound("Dataset not found.")
# check user's model setting
@@ -965,7 +959,7 @@ class DocumentApi(DocumentResource):
document = self.get_document(dataset_id_str, document_id_str, current_user, current_tenant_id)
try:
DocumentService.delete_document(document, db.session)
DocumentService.delete_document(document)
except services.errors.document.DocumentIndexingError:
raise DocumentIndexingError("Cannot delete document during indexing.")
@@ -989,7 +983,7 @@ class DocumentDownloadApi(DocumentResource):
def get(self, current_tenant_id: str, current_user: Account, dataset_id: UUID, document_id: UUID) -> dict[str, Any]:
# Reuse the shared permission/tenant checks implemented in DocumentResource.
document = self.get_document(str(dataset_id), str(document_id), current_user, current_tenant_id)
return {"url": DocumentService.get_document_download_url(document, db.session)}
return {"url": DocumentService.get_document_download_url(document)}
@console_ns.route("/datasets/<uuid:dataset_id>/documents/download-zip")
@@ -1019,7 +1013,6 @@ class DocumentBatchDownloadZipApi(DocumentResource):
document_ids=document_ids,
tenant_id=current_tenant_id,
current_user=current_user,
session=db.session,
)
# Delegate ZIP packing to FileService, but keep Flask response+cleanup in the route.
@@ -1168,7 +1161,7 @@ class DocumentStatusApi(DocumentResource):
self, current_user: Account, dataset_id: UUID, action: Literal["enable", "disable", "archive", "un_archive"]
):
dataset_id_str = str(dataset_id)
dataset = DatasetService.get_dataset(dataset_id_str, db.session)
dataset = DatasetService.get_dataset(dataset_id_str)
if dataset is None:
raise NotFound("Dataset not found.")
@@ -1180,12 +1173,12 @@ class DocumentStatusApi(DocumentResource):
DatasetService.check_dataset_model_setting(dataset)
# check user's permission
DatasetService.check_dataset_permission(dataset, current_user, db.session)
DatasetService.check_dataset_permission(dataset, current_user)
document_ids = request.args.getlist("document_id")
try:
DocumentService.batch_update_document_status(dataset, document_ids, action, current_user, db.session)
DocumentService.batch_update_document_status(dataset, document_ids, action, current_user)
except services.errors.document.DocumentIndexingError as e:
raise InvalidActionError(str(e))
except ValueError as e:
@@ -1209,11 +1202,11 @@ class DocumentPauseApi(DocumentResource):
dataset_id_str = str(dataset_id)
document_id_str = str(document_id)
dataset = DatasetService.get_dataset(dataset_id_str, db.session)
dataset = DatasetService.get_dataset(dataset_id_str)
if not dataset:
raise NotFound("Dataset not found.")
document = DocumentService.get_document(dataset.id, document_id_str, session=db.session)
document = DocumentService.get_document(dataset.id, document_id_str)
# 404 if document not found
if document is None:
@@ -1225,7 +1218,7 @@ class DocumentPauseApi(DocumentResource):
try:
# pause document
DocumentService.pause_document(document, db.session)
DocumentService.pause_document(document)
except services.errors.document.DocumentIndexingError:
raise DocumentIndexingError("Cannot pause completed document.")
@@ -1244,10 +1237,10 @@ class DocumentRecoverApi(DocumentResource):
"""recover document."""
dataset_id_str = str(dataset_id)
document_id_str = str(document_id)
dataset = DatasetService.get_dataset(dataset_id_str, db.session)
dataset = DatasetService.get_dataset(dataset_id_str)
if not dataset:
raise NotFound("Dataset not found.")
document = DocumentService.get_document(dataset.id, document_id_str, session=db.session)
document = DocumentService.get_document(dataset.id, document_id_str)
# 404 if document not found
if document is None:
@@ -1258,7 +1251,7 @@ class DocumentRecoverApi(DocumentResource):
raise ArchivedDocumentImmutableError()
try:
# pause document
DocumentService.recover_document(document, db.session)
DocumentService.recover_document(document)
except services.errors.document.DocumentIndexingError:
raise DocumentIndexingError("Document is not in paused status.")
@@ -1278,13 +1271,13 @@ class DocumentRetryApi(DocumentResource):
"""retry document."""
payload = DocumentRetryPayload.model_validate(console_ns.payload or {})
dataset_id_str = str(dataset_id)
dataset = DatasetService.get_dataset(dataset_id_str, db.session)
dataset = DatasetService.get_dataset(dataset_id_str)
retry_documents = []
if not dataset:
raise NotFound("Dataset not found.")
for document_id in payload.document_ids:
try:
document = DocumentService.get_document(dataset.id, document_id, session=db.session)
document = DocumentService.get_document(dataset.id, document_id)
# 404 if document not found
if document is None:
@@ -1302,7 +1295,7 @@ class DocumentRetryApi(DocumentResource):
logger.exception("Failed to retry document, document id: %s", document_id)
continue
# retry document
DocumentService.retry_document(dataset_id_str, retry_documents, db.session)
DocumentService.retry_document(dataset_id_str, retry_documents)
return "", 204
@@ -1320,14 +1313,14 @@ class DocumentRenameApi(DocumentResource):
# The role of the current user in the ta table must be admin, owner, editor, or dataset_operator
if not current_user.is_dataset_editor:
raise Forbidden()
dataset = DatasetService.get_dataset(dataset_id, db.session)
dataset = DatasetService.get_dataset(dataset_id)
if not dataset:
raise NotFound("Dataset not found.")
DatasetService.check_dataset_operator_permission(current_user, dataset, session=db.session)
DatasetService.check_dataset_operator_permission(current_user, dataset)
payload = DocumentRenamePayload.model_validate(console_ns.payload or {})
try:
document = DocumentService.rename_document(str(dataset_id), str(document_id), payload.name, db.session)
document = DocumentService.rename_document(str(dataset_id), str(document_id), payload.name)
except services.errors.document.DocumentIndexingError:
raise DocumentIndexingError("Cannot delete document during indexing.")
@@ -1345,11 +1338,11 @@ class WebsiteDocumentSyncApi(DocumentResource):
def get(self, current_tenant_id: str, dataset_id: UUID, document_id: UUID):
"""sync website document."""
dataset_id_str = str(dataset_id)
dataset = DatasetService.get_dataset(dataset_id_str, db.session)
dataset = DatasetService.get_dataset(dataset_id_str)
if not dataset:
raise NotFound("Dataset not found.")
document_id_str = str(document_id)
document = DocumentService.get_document(dataset.id, document_id_str, session=db.session)
document = DocumentService.get_document(dataset.id, document_id_str)
if not document:
raise NotFound("Document not found.")
if document.tenant_id != current_tenant_id:
@@ -1360,7 +1353,7 @@ class WebsiteDocumentSyncApi(DocumentResource):
if DocumentService.check_archived(document):
raise ArchivedDocumentImmutableError()
# sync document
DocumentService.sync_website_document(dataset_id_str, document, db.session)
DocumentService.sync_website_document(dataset_id_str, document)
return {"result": "success"}, 200
@@ -1380,10 +1373,10 @@ class DocumentPipelineExecutionLogApi(DocumentResource):
dataset_id_str = str(dataset_id)
document_id_str = str(document_id)
dataset = DatasetService.get_dataset(dataset_id_str, db.session)
dataset = DatasetService.get_dataset(dataset_id_str)
if not dataset:
raise NotFound("Dataset not found.")
document = DocumentService.get_document(dataset.id, document_id_str, session=db.session)
document = DocumentService.get_document(dataset.id, document_id_str)
if not document:
raise NotFound("Document not found.")
log = db.session.scalar(
@@ -1438,7 +1431,7 @@ class DocumentGenerateSummaryApi(Resource):
dataset_id_str = str(dataset_id)
# Get dataset
dataset = DatasetService.get_dataset(dataset_id_str, db.session)
dataset = DatasetService.get_dataset(dataset_id_str)
if not dataset:
raise NotFound("Dataset not found.")
@@ -1447,7 +1440,7 @@ class DocumentGenerateSummaryApi(Resource):
raise Forbidden()
try:
DatasetService.check_dataset_permission(dataset, current_user, db.session)
DatasetService.check_dataset_permission(dataset, current_user)
except services.errors.account.NoPermissionError as e:
raise Forbidden(str(e))
@@ -1472,7 +1465,7 @@ class DocumentGenerateSummaryApi(Resource):
raise ValueError("Summary index is not enabled for this dataset. Please enable it in the dataset settings.")
# Verify all documents exist and belong to the dataset
documents = DocumentService.get_documents_by_ids(dataset_id_str, document_list, db.session)
documents = DocumentService.get_documents_by_ids(dataset_id_str, document_list)
if len(documents) != len(document_list):
found_ids = {doc.id for doc in documents}
@@ -1488,7 +1481,6 @@ class DocumentGenerateSummaryApi(Resource):
DocumentService.update_documents_need_summary(
dataset_id=dataset_id_str,
document_ids=document_ids_to_update,
session=db.session,
need_summary=True,
)
@@ -1539,13 +1531,13 @@ class DocumentSummaryStatusApi(DocumentResource):
document_id_str = str(document_id)
# Get dataset
dataset = DatasetService.get_dataset(dataset_id_str, db.session)
dataset = DatasetService.get_dataset(dataset_id_str)
if not dataset:
raise NotFound("Dataset not found.")
# Check permissions
try:
DatasetService.check_dataset_permission(dataset, current_user, db.session)
DatasetService.check_dataset_permission(dataset, current_user)
except services.errors.account.NoPermissionError as e:
raise Forbidden(str(e))
@@ -1555,7 +1547,6 @@ class DocumentSummaryStatusApi(DocumentResource):
result = SummaryIndexService.get_document_summary_status_detail(
document_id=document_id_str,
dataset_id=dataset_id_str,
session=db.session,
)
return result, 200
@@ -57,11 +57,9 @@ from fields.segment_fields import (
from graphon.model_runtime.entities.model_entities import ModelType
from libs.helper import dump_response, escape_like_pattern
from libs.login import login_required
from libs.pagination import paginate_query
from models import Account
from models.dataset import Dataset, Document, DocumentSegment
from models.dataset import ChildChunk, DocumentSegment
from models.model import UploadFile
from services.dataset_ref_service import DatasetRefService, SegmentRef
from services.dataset_service import DatasetService, DocumentService, SegmentService
from services.entities.knowledge_entities.knowledge_entities import ChildChunkUpdateArgs, SegmentUpdateArgs
from services.errors.chunk import ChildChunkDeleteIndexError as ChildChunkDeleteIndexServiceError
@@ -164,21 +162,6 @@ register_response_schema_models(
)
def _get_segment_for_document(
dataset: Dataset, document: Document, segment_id: str
) -> tuple[SegmentRef, DocumentSegment]:
dataset_ref = DatasetRefService.create_dataset_ref(dataset)
document_ref = DatasetRefService.create_document_ref(dataset_ref, document)
if document_ref is None:
raise NotFound("Document not found.")
segment_ref = DatasetRefService.create_segment_ref(document_ref, segment_id)
segment = SegmentService.get_segment_by_ref(segment_ref)
if not segment:
raise NotFound("Segment not found.")
return segment_ref, segment
@console_ns.route("/datasets/<uuid:dataset_id>/documents/<uuid:document_id>/segments")
class DatasetDocumentSegmentListApi(Resource):
@console_ns.doc(params=SegmentDocParams.DATASET_DOCUMENT)
@@ -193,16 +176,16 @@ class DatasetDocumentSegmentListApi(Resource):
def get(self, current_tenant_id: str, current_user: Account, dataset_id: UUID, document_id: UUID):
dataset_id_str = str(dataset_id)
document_id_str = str(document_id)
dataset = DatasetService.get_dataset(dataset_id_str, db.session)
dataset = DatasetService.get_dataset(dataset_id_str)
if not dataset:
raise NotFound("Dataset not found.")
try:
DatasetService.check_dataset_permission(dataset, current_user, db.session)
DatasetService.check_dataset_permission(dataset, current_user)
except services.errors.account.NoPermissionError as e:
raise Forbidden(str(e))
document = DocumentService.get_document(dataset_id_str, document_id_str, session=db.session)
document = DocumentService.get_document(dataset_id_str, document_id_str)
if not document:
raise NotFound("Document not found.")
@@ -271,7 +254,7 @@ class DatasetDocumentSegmentListApi(Resource):
elif args.enabled.lower() == "false":
query = query.where(DocumentSegment.enabled == False)
segments = paginate_query(query, page=page, per_page=limit, max_per_page=100)
segments = db.paginate(select=query, page=page, per_page=limit, max_per_page=100, error_out=False)
segment_list = list(segments.items)
segment_ids = [segment.id for segment in segment_list]
@@ -303,14 +286,14 @@ class DatasetDocumentSegmentListApi(Resource):
def delete(self, current_user: Account, dataset_id: UUID, document_id: UUID):
# check dataset
dataset_id_str = str(dataset_id)
dataset = DatasetService.get_dataset(dataset_id_str, db.session)
dataset = DatasetService.get_dataset(dataset_id_str)
if not dataset:
raise NotFound("Dataset not found.")
# check user's model setting
DatasetService.check_dataset_model_setting(dataset)
# check document
document_id_str = str(document_id)
document = DocumentService.get_document(dataset_id_str, document_id_str, session=db.session)
document = DocumentService.get_document(dataset_id_str, document_id_str)
if not document:
raise NotFound("Document not found.")
segment_ids = request.args.getlist("segment_id")
@@ -319,10 +302,10 @@ class DatasetDocumentSegmentListApi(Resource):
if not current_user.is_dataset_editor:
raise Forbidden()
try:
DatasetService.check_dataset_permission(dataset, current_user, db.session)
DatasetService.check_dataset_permission(dataset, current_user)
except services.errors.account.NoPermissionError as e:
raise Forbidden(str(e))
SegmentService.delete_segments(segment_ids, document, dataset, db.session)
SegmentService.delete_segments(segment_ids, document, dataset)
return "", 204
@@ -348,11 +331,11 @@ class DatasetDocumentSegmentApi(Resource):
action: Literal["enable", "disable"],
):
dataset_id_str = str(dataset_id)
dataset = DatasetService.get_dataset(dataset_id_str, db.session)
dataset = DatasetService.get_dataset(dataset_id_str)
if not dataset:
raise NotFound("Dataset not found.")
document_id_str = str(document_id)
document = DocumentService.get_document(dataset_id_str, document_id_str, session=db.session)
document = DocumentService.get_document(dataset_id_str, document_id_str)
if not document:
raise NotFound("Document not found.")
# check user's model setting
@@ -362,7 +345,7 @@ class DatasetDocumentSegmentApi(Resource):
raise Forbidden()
try:
DatasetService.check_dataset_permission(dataset, current_user, db.session)
DatasetService.check_dataset_permission(dataset, current_user)
except services.errors.account.NoPermissionError as e:
raise Forbidden(str(e))
if dataset.indexing_technique == IndexTechniqueType.HIGH_QUALITY:
@@ -388,7 +371,7 @@ class DatasetDocumentSegmentApi(Resource):
if cache_result is not None:
raise InvalidActionError("Document is being indexed, please try again later")
try:
SegmentService.update_segments_status(segment_ids, action, dataset, document, db.session)
SegmentService.update_segments_status(segment_ids, action, dataset, document)
except Exception as e:
raise InvalidActionError(str(e))
return dump_response(SimpleResultResponse, {"result": "success"}), 200
@@ -411,12 +394,12 @@ class DatasetDocumentSegmentAddApi(Resource):
def post(self, current_tenant_id: str, current_user: Account, dataset_id: UUID, document_id: UUID):
# check dataset
dataset_id_str = str(dataset_id)
dataset = DatasetService.get_dataset(dataset_id_str, db.session)
dataset = DatasetService.get_dataset(dataset_id_str)
if not dataset:
raise NotFound("Dataset not found.")
# check document
document_id_str = str(document_id)
document = DocumentService.get_document(dataset_id_str, document_id_str, session=db.session)
document = DocumentService.get_document(dataset_id_str, document_id_str)
if not document:
raise NotFound("Document not found.")
if not current_user.is_dataset_editor:
@@ -438,14 +421,14 @@ class DatasetDocumentSegmentAddApi(Resource):
except ProviderTokenNotInitError as ex:
raise ProviderNotInitializeError(ex.description)
try:
DatasetService.check_dataset_permission(dataset, current_user, db.session)
DatasetService.check_dataset_permission(dataset, current_user)
except services.errors.account.NoPermissionError as e:
raise Forbidden(str(e))
# validate args
payload = SegmentCreatePayload.model_validate(console_ns.payload or {})
payload_dict = payload.model_dump(exclude_none=True)
SegmentService.segment_create_args_validate(payload_dict, document)
segment = type_cast(DocumentSegment, SegmentService.create_segment(payload_dict, document, dataset, db.session))
segment = type_cast(DocumentSegment, SegmentService.create_segment(payload_dict, document, dataset))
summary = SummaryIndexService.get_segment_summary(segment_id=segment.id, dataset_id=dataset_id_str)
response = {
"data": segment_response_with_summary(segment, summary.summary_content if summary else None),
@@ -472,23 +455,16 @@ class DatasetDocumentSegmentUpdateApi(Resource):
):
# check dataset
dataset_id_str = str(dataset_id)
dataset = DatasetService.get_dataset(dataset_id_str, db.session)
dataset = DatasetService.get_dataset(dataset_id_str)
if not dataset:
raise NotFound("Dataset not found.")
# check user's model setting
DatasetService.check_dataset_model_setting(dataset)
# check document
document_id_str = str(document_id)
document = DocumentService.get_document(dataset_id_str, document_id_str, session=db.session)
document = DocumentService.get_document(dataset_id_str, document_id_str)
if not document:
raise NotFound("Document not found.")
# The role of the current user in the ta table must be admin, owner, dataset_operator, or editor
if not current_user.is_dataset_editor:
raise Forbidden()
try:
DatasetService.check_dataset_permission(dataset, current_user, db.session)
except services.errors.account.NoPermissionError as e:
raise Forbidden(str(e))
if dataset.indexing_technique == IndexTechniqueType.HIGH_QUALITY:
# check embedding model setting
try:
@@ -505,8 +481,22 @@ class DatasetDocumentSegmentUpdateApi(Resource):
)
except ProviderTokenNotInitError as ex:
raise ProviderNotInitializeError(ex.description)
# check segment
segment_id_str = str(segment_id)
_, segment = _get_segment_for_document(dataset, document, segment_id_str)
segment = db.session.scalar(
select(DocumentSegment)
.where(DocumentSegment.id == segment_id_str, DocumentSegment.tenant_id == current_tenant_id)
.limit(1)
)
if not segment:
raise NotFound("Segment not found.")
# The role of the current user in the ta table must be admin, owner, dataset_operator, or editor
if not current_user.is_dataset_editor:
raise Forbidden()
try:
DatasetService.check_dataset_permission(dataset, current_user)
except services.errors.account.NoPermissionError as e:
raise Forbidden(str(e))
# validate args
payload = SegmentUpdatePayload.model_validate(console_ns.payload or {})
payload_dict = payload.model_dump(exclude_none=True)
@@ -514,11 +504,7 @@ class DatasetDocumentSegmentUpdateApi(Resource):
# Update segment (summary update with change detection is handled in SegmentService.update_segment)
segment = SegmentService.update_segment(
SegmentUpdateArgs.model_validate(payload.model_dump(exclude_none=True)),
segment,
document,
dataset,
db.session,
SegmentUpdateArgs.model_validate(payload.model_dump(exclude_none=True)), segment, document, dataset
)
summary = SummaryIndexService.get_segment_summary(segment_id=segment.id, dataset_id=dataset_id_str)
response = {
@@ -541,26 +527,33 @@ class DatasetDocumentSegmentUpdateApi(Resource):
):
# check dataset
dataset_id_str = str(dataset_id)
dataset = DatasetService.get_dataset(dataset_id_str, db.session)
dataset = DatasetService.get_dataset(dataset_id_str)
if not dataset:
raise NotFound("Dataset not found.")
# check user's model setting
DatasetService.check_dataset_model_setting(dataset)
# check document
document_id_str = str(document_id)
document = DocumentService.get_document(dataset_id_str, document_id_str, session=db.session)
document = DocumentService.get_document(dataset_id_str, document_id_str)
if not document:
raise NotFound("Document not found.")
# check segment
segment_id_str = str(segment_id)
segment = db.session.scalar(
select(DocumentSegment)
.where(DocumentSegment.id == segment_id_str, DocumentSegment.tenant_id == current_tenant_id)
.limit(1)
)
if not segment:
raise NotFound("Segment not found.")
# The role of the current user in the ta table must be admin, owner, dataset_operator, or editor
if not current_user.is_dataset_editor:
raise Forbidden()
try:
DatasetService.check_dataset_permission(dataset, current_user, db.session)
DatasetService.check_dataset_permission(dataset, current_user)
except services.errors.account.NoPermissionError as e:
raise Forbidden(str(e))
segment_id_str = str(segment_id)
_, segment = _get_segment_for_document(dataset, document, segment_id_str)
SegmentService.delete_segment(segment, document, dataset, db.session)
SegmentService.delete_segment(segment, document, dataset)
return "", 204
@@ -583,12 +576,12 @@ class DatasetDocumentSegmentBatchImportApi(Resource):
def post(self, current_tenant_id: str, current_user: Account, dataset_id: UUID, document_id: UUID):
# check dataset
dataset_id_str = str(dataset_id)
dataset = DatasetService.get_dataset(dataset_id_str, db.session)
dataset = DatasetService.get_dataset(dataset_id_str)
if not dataset:
raise NotFound("Dataset not found.")
# check document
document_id_str = str(document_id)
document = DocumentService.get_document(dataset_id_str, document_id_str, session=db.session)
document = DocumentService.get_document(dataset_id_str, document_id_str)
if not document:
raise NotFound("Document not found.")
@@ -658,20 +651,25 @@ class ChildChunkAddApi(Resource):
):
# check dataset
dataset_id_str = str(dataset_id)
dataset = DatasetService.get_dataset(dataset_id_str, db.session)
dataset = DatasetService.get_dataset(dataset_id_str)
if not dataset:
raise NotFound("Dataset not found.")
# check document
document_id_str = str(document_id)
document = DocumentService.get_document(dataset_id_str, document_id_str, session=db.session)
document = DocumentService.get_document(dataset_id_str, document_id_str)
if not document:
raise NotFound("Document not found.")
# check segment
segment_id_str = str(segment_id)
segment = db.session.scalar(
select(DocumentSegment)
.where(DocumentSegment.id == segment_id_str, DocumentSegment.tenant_id == current_tenant_id)
.limit(1)
)
if not segment:
raise NotFound("Segment not found.")
if not current_user.is_dataset_editor:
raise Forbidden()
try:
DatasetService.check_dataset_permission(dataset, current_user, db.session)
except services.errors.account.NoPermissionError as e:
raise Forbidden(str(e))
# check embedding model setting
if dataset.indexing_technique == IndexTechniqueType.HIGH_QUALITY:
try:
@@ -688,12 +686,14 @@ class ChildChunkAddApi(Resource):
)
except ProviderTokenNotInitError as ex:
raise ProviderNotInitializeError(ex.description)
segment_id_str = str(segment_id)
_, segment = _get_segment_for_document(dataset, document, segment_id_str)
try:
DatasetService.check_dataset_permission(dataset, current_user)
except services.errors.account.NoPermissionError as e:
raise Forbidden(str(e))
# validate args
try:
payload = ChildChunkCreatePayload.model_validate(console_ns.payload or {})
child_chunk = SegmentService.create_child_chunk(payload.content, segment, document, dataset, db.session)
child_chunk = SegmentService.create_child_chunk(payload.content, segment, document, dataset)
except ChildChunkIndexingServiceError as e:
raise ChildChunkIndexingError(str(e))
return dump_response(ChildChunkDetailResponse, {"data": child_chunk}), 200
@@ -709,18 +709,25 @@ class ChildChunkAddApi(Resource):
def get(self, current_tenant_id: str, dataset_id: UUID, document_id: UUID, segment_id: UUID):
# check dataset
dataset_id_str = str(dataset_id)
dataset = DatasetService.get_dataset(dataset_id_str, db.session)
dataset = DatasetService.get_dataset(dataset_id_str)
if not dataset:
raise NotFound("Dataset not found.")
# check user's model setting
DatasetService.check_dataset_model_setting(dataset)
# check document
document_id_str = str(document_id)
document = DocumentService.get_document(dataset_id_str, document_id_str, session=db.session)
document = DocumentService.get_document(dataset_id_str, document_id_str)
if not document:
raise NotFound("Document not found.")
# check segment
segment_id_str = str(segment_id)
_get_segment_for_document(dataset, document, segment_id_str)
segment = db.session.scalar(
select(DocumentSegment)
.where(DocumentSegment.id == segment_id_str, DocumentSegment.tenant_id == current_tenant_id)
.limit(1)
)
if not segment:
raise NotFound("Segment not found.")
args = query_params_from_request(ChildChunkListQuery, use_defaults_for_malformed_ints=True)
page = args.page
@@ -759,29 +766,36 @@ class ChildChunkAddApi(Resource):
):
# check dataset
dataset_id_str = str(dataset_id)
dataset = DatasetService.get_dataset(dataset_id_str, db.session)
dataset = DatasetService.get_dataset(dataset_id_str)
if not dataset:
raise NotFound("Dataset not found.")
# check user's model setting
DatasetService.check_dataset_model_setting(dataset)
# check document
document_id_str = str(document_id)
document = DocumentService.get_document(dataset_id_str, document_id_str, session=db.session)
document = DocumentService.get_document(dataset_id_str, document_id_str)
if not document:
raise NotFound("Document not found.")
# check segment
segment_id_str = str(segment_id)
segment = db.session.scalar(
select(DocumentSegment)
.where(DocumentSegment.id == segment_id_str, DocumentSegment.tenant_id == current_tenant_id)
.limit(1)
)
if not segment:
raise NotFound("Segment not found.")
# The role of the current user in the ta table must be admin, owner, dataset_operator, or editor
if not current_user.is_dataset_editor:
raise Forbidden()
try:
DatasetService.check_dataset_permission(dataset, current_user, db.session)
DatasetService.check_dataset_permission(dataset, current_user)
except services.errors.account.NoPermissionError as e:
raise Forbidden(str(e))
segment_id_str = str(segment_id)
_, segment = _get_segment_for_document(dataset, document, segment_id_str)
# validate args
payload = ChildChunkBatchUpdatePayload.model_validate(console_ns.payload or {})
try:
child_chunks = SegmentService.update_child_chunks(payload.chunks, segment, document, dataset, db.session)
child_chunks = SegmentService.update_child_chunks(payload.chunks, segment, document, dataset)
except ChildChunkIndexingServiceError as e:
raise ChildChunkIndexingError(str(e))
return dump_response(ChildChunkBatchUpdateResponse, {"data": child_chunks}), 200
@@ -811,31 +825,48 @@ class ChildChunkUpdateApi(Resource):
):
# check dataset
dataset_id_str = str(dataset_id)
dataset = DatasetService.get_dataset(dataset_id_str, db.session)
dataset = DatasetService.get_dataset(dataset_id_str)
if not dataset:
raise NotFound("Dataset not found.")
# check user's model setting
DatasetService.check_dataset_model_setting(dataset)
# check document
document_id_str = str(document_id)
document = DocumentService.get_document(dataset_id_str, document_id_str, session=db.session)
document = DocumentService.get_document(dataset_id_str, document_id_str)
if not document:
raise NotFound("Document not found.")
# check segment
segment_id_str = str(segment_id)
segment = db.session.scalar(
select(DocumentSegment)
.where(DocumentSegment.id == segment_id_str, DocumentSegment.tenant_id == current_tenant_id)
.limit(1)
)
if not segment:
raise NotFound("Segment not found.")
# check child chunk
child_chunk_id_str = str(child_chunk_id)
child_chunk = db.session.scalar(
select(ChildChunk)
.where(
ChildChunk.id == child_chunk_id_str,
ChildChunk.tenant_id == current_tenant_id,
ChildChunk.segment_id == segment.id,
ChildChunk.document_id == document_id_str,
)
.limit(1)
)
if not child_chunk:
raise NotFound("Child chunk not found.")
# The role of the current user in the ta table must be admin, owner, dataset_operator, or editor
if not current_user.is_dataset_editor:
raise Forbidden()
try:
DatasetService.check_dataset_permission(dataset, current_user, db.session)
DatasetService.check_dataset_permission(dataset, current_user)
except services.errors.account.NoPermissionError as e:
raise Forbidden(str(e))
segment_id_str = str(segment_id)
segment_ref, _ = _get_segment_for_document(dataset, document, segment_id_str)
child_chunk_id_str = str(child_chunk_id)
child_chunk = SegmentService.get_child_chunk_by_segment_ref(child_chunk_id_str, segment_ref)
if not child_chunk:
raise NotFound("Child chunk not found.")
try:
SegmentService.delete_child_chunk(child_chunk, dataset, db.session)
SegmentService.delete_child_chunk(child_chunk, dataset)
except ChildChunkDeleteIndexServiceError as e:
raise ChildChunkDeleteIndexError(str(e))
return "", 204
@@ -862,35 +893,50 @@ class ChildChunkUpdateApi(Resource):
):
# check dataset
dataset_id_str = str(dataset_id)
dataset = DatasetService.get_dataset(dataset_id_str, db.session)
dataset = DatasetService.get_dataset(dataset_id_str)
if not dataset:
raise NotFound("Dataset not found.")
# check user's model setting
DatasetService.check_dataset_model_setting(dataset)
# check document
document_id_str = str(document_id)
document = DocumentService.get_document(dataset_id_str, document_id_str, session=db.session)
document = DocumentService.get_document(dataset_id_str, document_id_str)
if not document:
raise NotFound("Document not found.")
# check segment
segment_id_str = str(segment_id)
segment = db.session.scalar(
select(DocumentSegment)
.where(DocumentSegment.id == segment_id_str, DocumentSegment.tenant_id == current_tenant_id)
.limit(1)
)
if not segment:
raise NotFound("Segment not found.")
# check child chunk
child_chunk_id_str = str(child_chunk_id)
child_chunk = db.session.scalar(
select(ChildChunk)
.where(
ChildChunk.id == child_chunk_id_str,
ChildChunk.tenant_id == current_tenant_id,
ChildChunk.segment_id == segment.id,
ChildChunk.document_id == document_id_str,
)
.limit(1)
)
if not child_chunk:
raise NotFound("Child chunk not found.")
# The role of the current user in the ta table must be admin, owner, dataset_operator, or editor
if not current_user.is_dataset_editor:
raise Forbidden()
try:
DatasetService.check_dataset_permission(dataset, current_user, db.session)
DatasetService.check_dataset_permission(dataset, current_user)
except services.errors.account.NoPermissionError as e:
raise Forbidden(str(e))
segment_id_str = str(segment_id)
segment_ref, segment = _get_segment_for_document(dataset, document, segment_id_str)
child_chunk_id_str = str(child_chunk_id)
child_chunk = SegmentService.get_child_chunk_by_segment_ref(child_chunk_id_str, segment_ref)
if not child_chunk:
raise NotFound("Child chunk not found.")
# validate args
try:
payload = ChildChunkUpdatePayload.model_validate(console_ns.payload or {})
child_chunk = SegmentService.update_child_chunk(
payload.content, child_chunk, segment, document, dataset, db.session
)
child_chunk = SegmentService.update_child_chunk(payload.content, child_chunk, segment, document, dataset)
except ChildChunkIndexingServiceError as e:
raise ChildChunkIndexingError(str(e))
return dump_response(ChildChunkDetailResponse, {"data": child_chunk}), 200
+14 -30
View File
@@ -4,7 +4,6 @@ from uuid import UUID
from flask import request
from flask_restx import Resource, fields, marshal
from pydantic import BaseModel, Field, RootModel
from sqlalchemy.orm import Session
from werkzeug.exceptions import Forbidden, InternalServerError, NotFound
import services
@@ -16,7 +15,6 @@ from controllers.common.schema import (
register_schema_models,
)
from controllers.console import console_ns
from controllers.console.app.wraps import with_session
from controllers.console.datasets.error import DatasetNameDuplicateError
from controllers.console.wraps import (
RBACPermission,
@@ -209,8 +207,7 @@ class ExternalApiTemplateListApi(Resource):
)
@with_current_user
@with_current_tenant_id
@with_session
def post(self, session: Session, current_tenant_id: str, current_user: Account):
def post(self, current_tenant_id: str, current_user: Account):
payload = ExternalKnowledgeApiPayload.model_validate(console_ns.payload or {})
ExternalDatasetService.validate_api_list(payload.settings)
@@ -221,10 +218,7 @@ class ExternalApiTemplateListApi(Resource):
try:
external_knowledge_api = ExternalDatasetService.create_external_knowledge_api(
tenant_id=current_tenant_id,
user_id=current_user.id,
args=payload.model_dump(),
session=session,
tenant_id=current_tenant_id, user_id=current_user.id, args=payload.model_dump()
)
except services.errors.dataset.DatasetNameDuplicateError:
raise DatasetNameDuplicateError()
@@ -247,11 +241,10 @@ class ExternalApiTemplateApi(Resource):
@login_required
@account_initialization_required
@with_current_tenant_id
@with_session
def get(self, session: Session, current_tenant_id: str, external_knowledge_api_id: UUID):
def get(self, current_tenant_id: str, external_knowledge_api_id: UUID):
external_knowledge_api_id_str = str(external_knowledge_api_id)
external_knowledge_api = ExternalDatasetService.get_external_knowledge_api(
external_knowledge_api_id=external_knowledge_api_id_str, tenant_id=current_tenant_id, session=session
external_knowledge_api_id_str, current_tenant_id
)
if external_knowledge_api is None:
raise NotFound("API template not found.")
@@ -269,8 +262,7 @@ class ExternalApiTemplateApi(Resource):
@console_ns.expect(console_ns.models[ExternalKnowledgeApiPayload.__name__])
@with_current_user
@with_current_tenant_id
@with_session
def patch(self, session: Session, current_tenant_id: str, current_user: Account, external_knowledge_api_id: UUID):
def patch(self, current_tenant_id: str, current_user: Account, external_knowledge_api_id: UUID):
external_knowledge_api_id_str = str(external_knowledge_api_id)
payload = ExternalKnowledgeApiPayload.model_validate(console_ns.payload or {})
@@ -281,7 +273,6 @@ class ExternalApiTemplateApi(Resource):
user_id=current_user.id,
external_knowledge_api_id=external_knowledge_api_id_str,
args=payload.model_dump(),
session=session,
)
return external_knowledge_api.to_dict(), 200
@@ -292,14 +283,13 @@ class ExternalApiTemplateApi(Resource):
@console_ns.response(204, "External knowledge API deleted successfully")
@with_current_user
@with_current_tenant_id
@with_session
def delete(self, session: Session, current_tenant_id: str, current_user: Account, external_knowledge_api_id: UUID):
def delete(self, current_tenant_id: str, current_user: Account, external_knowledge_api_id: UUID):
external_knowledge_api_id_str = str(external_knowledge_api_id)
if not (current_user.has_edit_permission or current_user.is_dataset_operator):
raise Forbidden()
ExternalDatasetService.delete_external_knowledge_api(session, current_tenant_id, external_knowledge_api_id_str)
ExternalDatasetService.delete_external_knowledge_api(current_tenant_id, external_knowledge_api_id_str)
return "", 204
@@ -313,14 +303,11 @@ class ExternalApiUseCheckApi(Resource):
@login_required
@account_initialization_required
@with_current_tenant_id
@with_session
def get(self, session: Session, current_tenant_id: str, external_knowledge_api_id: UUID):
def get(self, current_tenant_id: str, external_knowledge_api_id: UUID):
external_knowledge_api_id_str = str(external_knowledge_api_id)
external_knowledge_api_is_using, count = ExternalDatasetService.external_knowledge_api_use_check(
session,
external_knowledge_api_id_str,
current_tenant_id,
external_knowledge_api_id_str, current_tenant_id
)
return {"is_using": external_knowledge_api_is_using, "count": count}, 200
@@ -340,8 +327,7 @@ class ExternalDatasetCreateApi(Resource):
@rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_EXTERNAL_CONNECT)
@with_current_user
@with_current_tenant_id
@with_session
def post(self, session: Session, current_tenant_id: str, current_user: Account):
def post(self, current_tenant_id: str, current_user: Account):
# The role of the current user in the ta table must be admin, owner, or editor
payload = ExternalDatasetCreatePayload.model_validate(console_ns.payload or {})
args = payload.model_dump(exclude_none=True)
@@ -355,7 +341,6 @@ class ExternalDatasetCreateApi(Resource):
tenant_id=current_tenant_id,
user_id=current_user.id,
args=args,
session=session,
)
except services.errors.dataset.DatasetNameDuplicateError:
raise DatasetNameDuplicateError()
@@ -390,15 +375,14 @@ class ExternalKnowledgeHitTestingApi(Resource):
@account_initialization_required
@with_current_user
@rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_PIPELINE_TEST)
@with_session
def post(self, session: Session, current_user: Account, dataset_id: UUID):
def post(self, current_user: Account, dataset_id: UUID):
dataset_id_str = str(dataset_id)
dataset = DatasetService.get_dataset(dataset_id_str, db.session)
dataset = DatasetService.get_dataset(dataset_id_str)
if dataset is None:
raise NotFound("Dataset not found.")
try:
DatasetService.check_dataset_permission(dataset, current_user, db.session)
DatasetService.check_dataset_permission(dataset, current_user)
except services.errors.account.NoPermissionError as e:
raise Forbidden(str(e))
@@ -407,7 +391,7 @@ class ExternalKnowledgeHitTestingApi(Resource):
try:
response = HitTestingService.external_retrieve(
session=session,
session=db.session,
dataset=dataset,
query=payload.query,
account=current_user,
@@ -3,10 +3,8 @@ from __future__ import annotations
from uuid import UUID
from flask_restx import Resource
from sqlalchemy.orm import Session
from controllers.common.schema import register_response_schema_models, register_schema_models
from controllers.console.app.wraps import with_session
from controllers.console.wraps import RBACPermission, RBACResourceScope, rbac_permission_required
from fields.hit_testing_fields import HitTestingResponse
from libs.helper import dump_response
@@ -47,10 +45,7 @@ class HitTestingApi(Resource, DatasetsHitTestingBase):
@with_current_tenant_id
@with_current_user
@rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_PIPELINE_TEST)
@with_session
def post(
self, session: Session, current_user: Account, current_tenant_id: str, dataset_id: UUID
) -> dict[str, object]:
def post(self, current_user: Account, current_tenant_id: str, dataset_id: UUID) -> dict[str, object]:
dataset_id_str = str(dataset_id)
dataset = self.get_and_validate_dataset(dataset_id_str, current_user, current_tenant_id)
@@ -59,5 +54,5 @@ class HitTestingApi(Resource, DatasetsHitTestingBase):
return dump_response(
HitTestingResponse,
self.perform_hit_testing(session, dataset, args, current_user, current_tenant_id),
self.perform_hit_testing(dataset, args, current_user, current_tenant_id),
)
@@ -2,7 +2,6 @@ import logging
from typing import Any, cast
from pydantic import BaseModel, Field
from sqlalchemy.orm import Session
from werkzeug.exceptions import Forbidden, InternalServerError, NotFound
import services
@@ -86,12 +85,12 @@ class DatasetsHitTestingBase:
dataset_id: str, current_user: Account | None = None, current_tenant_id: str | None = None
) -> Dataset:
current_user, _ = resolve_account_fallback(current_user, current_tenant_id)
dataset = DatasetService.get_dataset(dataset_id, db.session)
dataset = DatasetService.get_dataset(dataset_id)
if dataset is None:
raise NotFound("Dataset not found.")
try:
DatasetService.check_dataset_permission(dataset, current_user, db.session)
DatasetService.check_dataset_permission(dataset, current_user)
except services.errors.account.NoPermissionError as e:
raise Forbidden(str(e))
@@ -109,7 +108,6 @@ class DatasetsHitTestingBase:
@staticmethod
def perform_hit_testing(
session: Session,
dataset: Dataset,
args: dict[str, Any],
current_user: Account | None = None,
@@ -118,7 +116,7 @@ class DatasetsHitTestingBase:
try:
current_user, _ = resolve_account_fallback(current_user, current_tenant_id)
response = HitTestingService.retrieve(
session=session,
session=db.session,
dataset=dataset,
query=cast(str, args.get("query")),
account=current_user,
+11 -11
View File
@@ -61,10 +61,10 @@ class DatasetMetadataCreateApi(Resource):
metadata_args = MetadataArgs.model_validate(console_ns.payload or {})
dataset_id_str = str(dataset_id)
dataset = DatasetService.get_dataset(dataset_id_str, db.session)
dataset = DatasetService.get_dataset(dataset_id_str)
if dataset is None:
raise NotFound("Dataset not found.")
DatasetService.check_dataset_permission(dataset, current_user, db.session)
DatasetService.check_dataset_permission(dataset, current_user)
metadata = MetadataService.create_metadata(
db.session(), dataset_id_str, metadata_args, current_user, current_tenant_id
@@ -81,7 +81,7 @@ class DatasetMetadataCreateApi(Resource):
@rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_CREATE_AND_MANAGEMENT)
def get(self, dataset_id: UUID):
dataset_id_str = str(dataset_id)
dataset = DatasetService.get_dataset(dataset_id_str, db.session)
dataset = DatasetService.get_dataset(dataset_id_str)
if dataset is None:
raise NotFound("Dataset not found.")
metadata = MetadataService.get_dataset_metadatas(db.session(), dataset)
@@ -105,10 +105,10 @@ class DatasetMetadataApi(Resource):
dataset_id_str = str(dataset_id)
metadata_id_str = str(metadata_id)
dataset = DatasetService.get_dataset(dataset_id_str, db.session)
dataset = DatasetService.get_dataset(dataset_id_str)
if dataset is None:
raise NotFound("Dataset not found.")
DatasetService.check_dataset_permission(dataset, current_user, db.session)
DatasetService.check_dataset_permission(dataset, current_user)
metadata = MetadataService.update_metadata_name(
db.session(), dataset_id_str, metadata_id_str, name, current_user, current_tenant_id
@@ -125,10 +125,10 @@ class DatasetMetadataApi(Resource):
def delete(self, current_user: Account, dataset_id: UUID, metadata_id: UUID):
dataset_id_str = str(dataset_id)
metadata_id_str = str(metadata_id)
dataset = DatasetService.get_dataset(dataset_id_str, db.session)
dataset = DatasetService.get_dataset(dataset_id_str)
if dataset is None:
raise NotFound("Dataset not found.")
DatasetService.check_dataset_permission(dataset, current_user, db.session)
DatasetService.check_dataset_permission(dataset, current_user)
MetadataService.delete_metadata(db.session(), dataset_id_str, metadata_id_str)
# Frontend callers only await success and invalidate metadata caches; no response body is consumed.
@@ -162,10 +162,10 @@ class DatasetMetadataBuiltInFieldActionApi(Resource):
@rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_EDIT)
def post(self, current_user: Account, dataset_id: UUID, action: Literal["enable", "disable"]):
dataset_id_str = str(dataset_id)
dataset = DatasetService.get_dataset(dataset_id_str, db.session)
dataset = DatasetService.get_dataset(dataset_id_str)
if dataset is None:
raise NotFound("Dataset not found.")
DatasetService.check_dataset_permission(dataset, current_user, db.session)
DatasetService.check_dataset_permission(dataset, current_user)
match action:
case "enable":
@@ -191,10 +191,10 @@ class DocumentMetadataEditApi(Resource):
@rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_EDIT)
def post(self, current_user: Account, dataset_id: UUID):
dataset_id_str = str(dataset_id)
dataset = DatasetService.get_dataset(dataset_id_str, db.session)
dataset = DatasetService.get_dataset(dataset_id_str)
if dataset is None:
raise NotFound("Dataset not found.")
DatasetService.check_dataset_permission(dataset, current_user, db.session)
DatasetService.check_dataset_permission(dataset, current_user)
metadata_args = MetadataOperationData.model_validate(console_ns.payload or {})
@@ -5,7 +5,7 @@ from flask import request
from flask_restx import Resource
from pydantic import BaseModel, Field
from sqlalchemy import select
from sqlalchemy.orm import Session, sessionmaker
from sqlalchemy.orm import sessionmaker
from werkzeug.exceptions import NotFound
from controllers.common.fields import SimpleDataResponse
@@ -16,7 +16,6 @@ from controllers.common.schema import (
register_schema_models,
)
from controllers.console import console_ns
from controllers.console.app.wraps import with_session
from controllers.console.wraps import (
account_initialization_required,
enterprise_license_required,
@@ -103,13 +102,10 @@ class PipelineTemplateListApi(Resource):
@account_initialization_required
@enterprise_license_required
@with_current_tenant_id
@with_session
def get(self, session: Session, current_tenant_id: str) -> JsonResponseWithStatus:
def get(self, current_tenant_id: str) -> JsonResponseWithStatus:
query = PipelineTemplateListQuery.model_validate(request.args.to_dict(flat=True))
# get pipeline templates
pipeline_templates = RagPipelineService.get_pipeline_templates(
session, query.type, query.language, current_tenant_id
)
pipeline_templates = RagPipelineService.get_pipeline_templates(query.type, query.language, current_tenant_id)
return dump_response(PipelineTemplateListResponse, pipeline_templates), 200
@@ -121,11 +117,10 @@ class PipelineTemplateDetailApi(Resource):
@login_required
@account_initialization_required
@enterprise_license_required
@with_session
def get(self, session: Session, template_id: str) -> JsonResponseWithStatus:
def get(self, template_id: str) -> JsonResponseWithStatus:
query = PipelineTemplateDetailQuery.model_validate(request.args.to_dict(flat=True))
rag_pipeline_service = RagPipelineService()
pipeline_template = rag_pipeline_service.get_pipeline_template_detail(session, template_id, query.type)
pipeline_template = rag_pipeline_service.get_pipeline_template_detail(template_id, query.type)
if pipeline_template is None:
raise NotFound("Pipeline template not found from upstream service.")
return dump_response(PipelineTemplateDetailResponse, pipeline_template), 200
@@ -1,5 +1,6 @@
from flask_restx import Resource
from pydantic import BaseModel
from sqlalchemy.orm import Session
from werkzeug.exceptions import Forbidden
import services
@@ -65,19 +66,19 @@ class CreateRagPipelineDatasetApi(Resource):
yaml_content=payload.yaml_content,
)
try:
rag_pipeline_dsl_service = RagPipelineDslService(db.session)
import_info = rag_pipeline_dsl_service.create_rag_pipeline_dataset(
tenant_id=current_tenant_id,
rag_pipeline_dataset_create_entity=rag_pipeline_dataset_create_entity,
)
with Session(db.engine, expire_on_commit=False) as session:
rag_pipeline_dsl_service = RagPipelineDslService(session)
import_info = rag_pipeline_dsl_service.create_rag_pipeline_dataset(
tenant_id=current_tenant_id,
rag_pipeline_dataset_create_entity=rag_pipeline_dataset_create_entity,
)
session.commit()
if rag_pipeline_dataset_create_entity.permission == "partial_members":
DatasetPermissionService.update_partial_member_list(
current_tenant_id,
import_info["dataset_id"],
rag_pipeline_dataset_create_entity.partial_member_list,
db.session,
)
db.session.commit()
except services.errors.dataset.DatasetNameDuplicateError:
raise DatasetNameDuplicateError()
@@ -110,6 +111,5 @@ class CreateEmptyRagPipelineDatasetApi(Resource):
permission=DatasetPermissionEnum.ONLY_ME,
partial_member_list=None,
),
session=db.session,
)
return dump_response(DatasetDetailResponse, dataset), 201
@@ -6,7 +6,7 @@ from uuid import UUID
from flask import abort, request
from flask_restx import Resource
from pydantic import BaseModel, Field, RootModel, ValidationError
from sqlalchemy.orm import Session, sessionmaker
from sqlalchemy.orm import sessionmaker
from werkzeug.exceptions import BadRequest, Forbidden, InternalServerError, NotFound
import services
@@ -26,7 +26,6 @@ from controllers.console.app.workflow import (
WorkflowPaginationResponse,
WorkflowResponse,
)
from controllers.console.app.wraps import with_session
from controllers.console.datasets.wraps import get_rag_pipeline
from controllers.console.wraps import (
RBACPermission,
@@ -65,7 +64,6 @@ from services.rag_pipeline.pipeline_generate_service import PipelineGenerateServ
from services.rag_pipeline.rag_pipeline import RagPipelineService
from services.rag_pipeline.rag_pipeline_manage_service import RagPipelineManageService
from services.rag_pipeline.rag_pipeline_transform_service import RagPipelineTransformService
from services.workflow_ref_service import WorkflowRefService
from services.workflow_service import DraftWorkflowDeletionError, WorkflowInUseError, WorkflowService
logger = logging.getLogger(__name__)
@@ -344,8 +342,7 @@ class DraftRagPipelineRunApi(Resource):
@rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_EDIT)
@with_current_user
@get_rag_pipeline
@with_session
def post(self, session: Session, current_user: Account, pipeline: Pipeline):
def post(self, current_user: Account, pipeline: Pipeline):
"""
Run draft workflow
"""
@@ -354,7 +351,6 @@ class DraftRagPipelineRunApi(Resource):
try:
response = PipelineGenerateService.generate(
session=session,
pipeline=pipeline,
user=current_user,
args=args,
@@ -378,8 +374,7 @@ class PublishedRagPipelineRunApi(Resource):
@rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_EDIT)
@with_current_user
@get_rag_pipeline
@with_session
def post(self, session: Session, current_user: Account, pipeline: Pipeline):
def post(self, current_user: Account, pipeline: Pipeline):
"""
Run published workflow
"""
@@ -389,7 +384,6 @@ class PublishedRagPipelineRunApi(Resource):
try:
response = PipelineGenerateService.generate(
session=session,
pipeline=pipeline,
user=current_user,
args=args,
@@ -744,15 +738,15 @@ class RagPipelineByIdApi(Resource):
return {"message": "No valid fields to update"}, 400
rag_pipeline_service = RagPipelineService()
workflow_ref = WorkflowRefService.create_pipeline_workflow_ref(pipeline, workflow_id)
# Create a session and manage the transaction
with sessionmaker(db.engine, expire_on_commit=False).begin() as session:
workflow = rag_pipeline_service.update_workflow(
session=session,
workflow_id=workflow_id,
tenant_id=pipeline.tenant_id,
account_id=current_user.id,
data=update_data,
workflow_ref=workflow_ref,
)
if not workflow:
@@ -775,13 +769,13 @@ class RagPipelineByIdApi(Resource):
abort(400, description=f"Cannot delete workflow that is currently in use by pipeline '{pipeline.id}'")
workflow_service = WorkflowService()
workflow_ref = WorkflowRefService.create_pipeline_workflow_ref(pipeline, workflow_id)
with sessionmaker(db.engine).begin() as session:
try:
workflow_service.delete_workflow(
session=session,
workflow_ref=workflow_ref,
workflow_id=workflow_id,
tenant_id=pipeline.tenant_id,
)
except WorkflowInUseError as e:
abort(400, description=str(e))
@@ -1019,14 +1013,13 @@ class RagPipelineTransformApi(Resource):
@login_required
@account_initialization_required
@with_current_user
@with_session
def post(self, session: Session, current_user: Account, dataset_id: UUID):
def post(self, current_user: Account, dataset_id: UUID):
if not (current_user.has_edit_permission or current_user.is_dataset_operator):
raise Forbidden()
dataset_id_str = str(dataset_id)
rag_pipeline_transform_service = RagPipelineTransformService()
result = rag_pipeline_transform_service.transform_dataset(dataset_id_str, session)
result = rag_pipeline_transform_service.transform_dataset(dataset_id_str, db.session)
return result
+1 -12
View File
@@ -22,9 +22,7 @@ from controllers.console.explore.wraps import InstalledAppResource
from core.errors.error import ModelCurrentlyNotSupportError, ProviderTokenNotInitError, QuotaExceededError
from extensions.ext_database import db
from graphon.model_runtime.errors.invoke import InvokeError
from libs.login import current_account_with_tenant
from models.model import InstalledApp
from services.app_ref_service import AppRefService
from services.audio_service import AudioService
from services.errors.audio import (
AudioTooLargeServiceError,
@@ -101,22 +99,13 @@ class ChatTextApi(InstalledAppResource):
message_id = payload.message_id
text = payload.text
voice = payload.voice
message_ref = None
if message_id:
current_user, _ = current_account_with_tenant()
app_ref = AppRefService.create_app_ref(app_model)
message_ref = AppRefService.create_message_ref(
app_ref,
message_id,
account_id=current_user.id,
)
response = AudioService.transcript_tts(
app_model=app_model,
session=db.session,
text=text,
voice=voice,
message_ref=message_ref,
message_id=message_id,
)
return response
except services.errors.app_model_config.AppModelConfigBrokenError:
+10 -26
View File
@@ -3,11 +3,10 @@ from typing import Any, Literal
from uuid import UUID
from pydantic import BaseModel, Field, field_validator
from sqlalchemy.orm import Session
from werkzeug.exceptions import InternalServerError, NotFound
import services
from controllers.common.fields import SimpleResultResponse
from controllers.common.fields import GeneratedAppResponse, SimpleResultResponse
from controllers.common.schema import register_response_schema_models, register_schema_models
from controllers.console.app.error import (
AppUnavailableError,
@@ -17,7 +16,6 @@ from controllers.console.app.error import (
ProviderNotInitializeError,
ProviderQuotaExceededError,
)
from controllers.console.app.wraps import with_session
from controllers.console.explore.error import NotChatAppError, NotCompletionAppError
from controllers.console.explore.wraps import InstalledAppResource
from controllers.console.wraps import with_current_user, with_current_user_id
@@ -75,7 +73,7 @@ class ChatMessagePayload(BaseModel):
register_schema_models(console_ns, CompletionMessageExplorePayload, ChatMessagePayload)
register_response_schema_models(console_ns, SimpleResultResponse)
register_response_schema_models(console_ns, GeneratedAppResponse, SimpleResultResponse)
# define completion api for user
@@ -85,10 +83,9 @@ register_response_schema_models(console_ns, SimpleResultResponse)
)
class CompletionApi(InstalledAppResource):
@console_ns.expect(console_ns.models[CompletionMessageExplorePayload.__name__])
@console_ns.response(200, "Success")
@console_ns.response(200, "Success", console_ns.models[GeneratedAppResponse.__name__])
@with_current_user
@with_session
def post(self, session: Session, current_user: Account, installed_app: InstalledApp):
def post(self, current_user: Account, installed_app: InstalledApp):
app_model = installed_app.app
if app_model is None:
raise AppUnavailableError()
@@ -106,15 +103,9 @@ class CompletionApi(InstalledAppResource):
try:
response = AppGenerateService.generate(
session=session,
app_model=app_model,
user=current_user,
args=args,
invoke_from=InvokeFrom.EXPLORE,
streaming=streaming,
app_model=app_model, user=current_user, args=args, invoke_from=InvokeFrom.EXPLORE, streaming=streaming
)
# response-contract:ignore compact_generate_response
return helper.compact_generate_response(response)
except services.errors.conversation.ConversationNotExistsError:
raise NotFound("Conversation Not Exists.")
@@ -159,7 +150,7 @@ class CompletionStopApi(InstalledAppResource):
app_mode=AppMode.value_of(app_model.mode),
)
return SimpleResultResponse(result="success").model_dump(mode="json"), 200
return {"result": "success"}, 200
@console_ns.route(
@@ -168,10 +159,9 @@ class CompletionStopApi(InstalledAppResource):
)
class ChatApi(InstalledAppResource):
@console_ns.expect(console_ns.models[ChatMessagePayload.__name__])
@console_ns.response(200, "Success")
@console_ns.response(200, "Success", console_ns.models[GeneratedAppResponse.__name__])
@with_current_user
@with_session
def post(self, session: Session, current_user: Account, installed_app: InstalledApp):
def post(self, current_user: Account, installed_app: InstalledApp):
app_model = installed_app.app
if app_model is None:
raise AppUnavailableError()
@@ -189,15 +179,9 @@ class ChatApi(InstalledAppResource):
try:
response = AppGenerateService.generate(
session=session,
app_model=app_model,
user=current_user,
args=args,
invoke_from=InvokeFrom.EXPLORE,
streaming=True,
app_model=app_model, user=current_user, args=args, invoke_from=InvokeFrom.EXPLORE, streaming=True
)
# response-contract:ignore compact_generate_response
return helper.compact_generate_response(response)
except services.errors.conversation.ConversationNotExistsError:
raise NotFound("Conversation Not Exists.")
@@ -245,4 +229,4 @@ class ChatStopApi(InstalledAppResource):
app_mode=app_mode,
)
return SimpleResultResponse(result="success").model_dump(mode="json"), 200
return {"result": "success"}, 200
@@ -147,15 +147,11 @@ register_schema_models(
InstalledAppCreatePayload,
InstalledAppUpdatePayload,
InstalledAppsListQuery,
)
register_response_schema_models(
console_ns,
InstalledAppInfoResponse,
InstalledAppResponse,
InstalledAppListResponse,
SimpleMessageResponse,
SimpleResultMessageResponse,
)
register_response_schema_models(console_ns, SimpleMessageResponse, SimpleResultMessageResponse)
@console_ns.route("/installed-apps")
+6 -9
View File
@@ -4,10 +4,10 @@ from uuid import UUID
from flask import request
from pydantic import BaseModel, TypeAdapter
from sqlalchemy.orm import Session
from werkzeug.exceptions import InternalServerError, NotFound
from controllers.common.controller_schemas import MessageFeedbackPayload, MessageListQuery
from controllers.common.fields import GeneratedAppResponse
from controllers.common.schema import query_params_from_model, register_response_schema_models, register_schema_models
from controllers.console.app.error import (
AppMoreLikeThisDisabledError,
@@ -17,7 +17,6 @@ from controllers.console.app.error import (
ProviderNotInitializeError,
ProviderQuotaExceededError,
)
from controllers.console.app.wraps import with_session
from controllers.console.explore.error import (
AppSuggestedQuestionsAfterAnswerDisabledError,
NotChatAppError,
@@ -60,6 +59,7 @@ class MoreLikeThisQuery(BaseModel):
register_schema_models(console_ns, MessageListQuery, MessageFeedbackPayload, MoreLikeThisQuery)
register_response_schema_models(
console_ns,
GeneratedAppResponse,
ExploreMessageInfiniteScrollPagination,
ResultResponse,
SuggestedQuestionsResponse,
@@ -88,8 +88,8 @@ class MessageListApi(InstalledAppResource):
pagination = MessageService.pagination_by_first_id(
app_model,
current_user,
args.conversation_id,
args.first_id or None,
str(args.conversation_id),
str(args.first_id) if args.first_id else None,
args.limit,
)
adapter = TypeAdapter(ExploreMessageListItem)
@@ -142,10 +142,9 @@ class MessageFeedbackApi(InstalledAppResource):
)
class MessageMoreLikeThisApi(InstalledAppResource):
@console_ns.doc(params=query_params_from_model(MoreLikeThisQuery))
@console_ns.response(200, "Success")
@console_ns.response(200, "Success", console_ns.models[GeneratedAppResponse.__name__])
@with_current_user
@with_session
def get(self, session: Session, current_user: Account, installed_app: InstalledApp, message_id: UUID):
def get(self, current_user: Account, installed_app: InstalledApp, message_id: UUID):
app_model = installed_app.app
if app_model is None:
raise AppUnavailableError()
@@ -160,14 +159,12 @@ class MessageMoreLikeThisApi(InstalledAppResource):
try:
response = AppGenerateService.generate_more_like_this(
session=session,
app_model=app_model,
user=current_user,
message_id=message_id_str,
invoke_from=InvokeFrom.EXPLORE,
streaming=streaming,
)
# response-contract:ignore compact_generate_response
return helper.compact_generate_response(response)
except MessageNotExistsError:
raise NotFound("Message Not Exists.")
+1 -6
View File
@@ -13,12 +13,7 @@ from services.app_service import AppService
class ExploreAppMetaResponse(BaseModel):
"""Metadata consumed by the installed-app chat UI.
Built-in tool icons are URL strings; API-based tool icons are provider-defined payload objects.
"""
tool_icons: dict[str, str | dict[str, Any]] = Field(default_factory=dict)
tool_icons: dict[str, Any] = Field(default_factory=dict)
register_response_schema_models(console_ns, fields.Parameters, ExploreAppMetaResponse)
@@ -70,33 +70,19 @@ class LearnDifyAppListResponse(ResponseModel):
recommended_apps: list[RecommendedAppResponse]
class RecommendedAppDetailResponse(ResponseModel):
id: str
name: str
icon: str | None = None
icon_background: str | None = None
mode: str
export_data: str
can_trial: bool | None = None
class RecommendedAppDetailNullableResponse(RootModel[RecommendedAppDetailResponse | None]):
pass
class RecommendedAppDetailResponse(RootModel[dict[str, Any]]):
root: dict[str, Any]
register_schema_models(
console_ns,
RecommendedAppsQuery,
)
register_response_schema_models(
console_ns,
RecommendedAppInfoResponse,
RecommendedAppResponse,
RecommendedAppListResponse,
LearnDifyAppListResponse,
RecommendedAppDetailResponse,
RecommendedAppDetailNullableResponse,
)
register_response_schema_models(console_ns, RecommendedAppDetailResponse)
def _resolve_language(language: str | None, user: Account) -> str:
@@ -144,7 +130,7 @@ class LearnDifyAppListApi(Resource):
@console_ns.route("/explore/apps/<uuid:app_id>")
class RecommendedAppApi(Resource):
@console_ns.response(200, "Success", console_ns.models[RecommendedAppDetailNullableResponse.__name__])
@console_ns.response(200, "Success", console_ns.models[RecommendedAppDetailResponse.__name__])
@login_required
@account_initialization_required
def get(self, app_id: UUID):
+81 -310
View File
@@ -1,12 +1,10 @@
import logging
from datetime import datetime
from typing import Any, Literal
from typing import Any, Literal, cast
from flask import request
from flask_restx import Resource
from pydantic import AliasChoices, BaseModel, Field, field_validator
from flask_restx import Resource, fields, marshal, marshal_with
from pydantic import BaseModel, Field
from sqlalchemy import select
from sqlalchemy.orm import Session
from werkzeug.exceptions import Forbidden, InternalServerError, NotFound
import services
@@ -19,6 +17,7 @@ from controllers.common.fields import (
from controllers.common.fields import Parameters as ParametersResponse
from controllers.common.fields import Site as SiteResponse
from controllers.common.schema import (
get_or_create_model,
query_params_from_model,
register_response_schema_models,
register_schema_models,
@@ -37,7 +36,7 @@ from controllers.console.app.error import (
ProviderQuotaExceededError,
UnsupportedAudioTypeError,
)
from controllers.console.app.wraps import get_app_model_with_trial, with_session
from controllers.console.app.wraps import get_app_model_with_trial
from controllers.console.explore.error import (
AppSuggestedQuestionsAfterAnswerDisabledError,
NotChatAppError,
@@ -57,18 +56,31 @@ from core.errors.error import (
)
from extensions.ext_database import db
from extensions.ext_redis import redis_client
from fields.base import ResponseModel
from fields.app_fields import (
app_detail_fields_with_site,
deleted_tool_fields,
model_config_fields,
site_fields,
tag_fields,
)
from fields.dataset_fields import dataset_fields
from fields.member_fields import simple_account_fields
from fields.message_fields import SuggestedQuestionsResponse
from fields.workflow_fields import (
conversation_variable_fields,
pipeline_variable_fields,
workflow_fields,
workflow_partial_fields,
)
from graphon.graph_engine.manager import GraphEngineManager
from graphon.model_runtime.errors.invoke import InvokeError
from libs import helper
from libs.helper import dump_response, to_timestamp, uuid_value
from libs.helper import uuid_value
from models import Account
from models.account import TenantStatus
from models.model import AppMode, Site
from models.workflow import Workflow
from services.app_generate_service import AppGenerateService
from services.app_ref_service import AppRefService
from services.app_service import AppService
from services.audio_service import AudioService
from services.dataset_service import DatasetService
@@ -90,6 +102,48 @@ from services.recommended_app_service import RecommendedAppService
logger = logging.getLogger(__name__)
model_config_model = get_or_create_model("TrialAppModelConfig", model_config_fields)
workflow_partial_model = get_or_create_model("TrialWorkflowPartial", workflow_partial_fields)
deleted_tool_model = get_or_create_model("TrialDeletedTool", deleted_tool_fields)
tag_model = get_or_create_model("TrialTag", tag_fields)
site_model = get_or_create_model("TrialSite", site_fields)
app_detail_fields_with_site_copy = app_detail_fields_with_site.copy()
app_detail_fields_with_site_copy["model_config"] = fields.Nested(
model_config_model, attribute="app_model_config", allow_null=True
)
app_detail_fields_with_site_copy["workflow"] = fields.Nested(workflow_partial_model, allow_null=True)
app_detail_fields_with_site_copy["deleted_tools"] = fields.List(fields.Nested(deleted_tool_model))
app_detail_fields_with_site_copy["tags"] = fields.List(fields.Nested(tag_model))
app_detail_fields_with_site_copy["site"] = fields.Nested(site_model)
app_detail_with_site_model = get_or_create_model("TrialAppDetailWithSite", app_detail_fields_with_site_copy)
simple_account_model = get_or_create_model("TrialSimpleAccount", simple_account_fields)
conversation_variable_model = get_or_create_model("TrialConversationVariable", conversation_variable_fields)
pipeline_variable_model = get_or_create_model("TrialPipelineVariable", pipeline_variable_fields)
workflow_fields_copy = workflow_fields.copy()
workflow_fields_copy["created_by"] = fields.Nested(simple_account_model, attribute="created_by_account")
workflow_fields_copy["updated_by"] = fields.Nested(
simple_account_model, attribute="updated_by_account", allow_null=True
)
workflow_fields_copy["conversation_variables"] = fields.List(fields.Nested(conversation_variable_model))
workflow_fields_copy["rag_pipeline_variables"] = fields.List(fields.Nested(pipeline_variable_model))
workflow_model = get_or_create_model("TrialWorkflow", workflow_fields_copy)
dataset_model = get_or_create_model("TrialDataset", dataset_fields)
dataset_list_model = get_or_create_model(
"TrialDatasetList",
{
"data": fields.List(fields.Nested(dataset_model)),
"has_more": fields.Boolean,
"limit": fields.Integer,
"total": fields.Integer,
"page": fields.Integer,
},
)
class WorkflowRunRequest(BaseModel):
inputs: dict
files: list | None = Field(default=None)
@@ -125,259 +179,6 @@ class TrialDatasetListQuery(BaseModel):
ids: list[str] = Field(default_factory=list, description="Dataset IDs")
type TrialAppMode = Literal["chat", "agent-chat", "advanced-chat", "workflow", "completion"]
type TrialIconType = Literal["emoji", "image", "link"]
type JsonObject = dict[str, Any]
class TrialAppModel(ResponseModel):
provider: str
name: str
mode: str | None = None
completion_params: JsonObject = Field(default_factory=dict)
class TrialAppAgentMode(ResponseModel):
enabled: bool | None = None
strategy: str | None = None
tools: list[JsonObject] = Field(default_factory=list)
class TrialAppModelConfigResponse(ResponseModel):
opening_statement: str | None = None
suggested_questions: list[str] = Field(
default_factory=list,
validation_alias=AliasChoices("suggested_questions_list", "suggested_questions"),
)
suggested_questions_after_answer: JsonObject | None = Field(
default=None,
validation_alias=AliasChoices("suggested_questions_after_answer_dict", "suggested_questions_after_answer"),
)
speech_to_text: JsonObject | None = Field(
default=None, validation_alias=AliasChoices("speech_to_text_dict", "speech_to_text")
)
text_to_speech: JsonObject | None = Field(
default=None, validation_alias=AliasChoices("text_to_speech_dict", "text_to_speech")
)
retriever_resource: JsonObject | None = Field(
default=None, validation_alias=AliasChoices("retriever_resource_dict", "retriever_resource")
)
annotation_reply: JsonObject | None = Field(
default=None, validation_alias=AliasChoices("annotation_reply_dict", "annotation_reply")
)
more_like_this: JsonObject | None = Field(
default=None, validation_alias=AliasChoices("more_like_this_dict", "more_like_this")
)
sensitive_word_avoidance: JsonObject | None = Field(
default=None, validation_alias=AliasChoices("sensitive_word_avoidance_dict", "sensitive_word_avoidance")
)
external_data_tools: list[JsonObject] = Field(
default_factory=list, validation_alias=AliasChoices("external_data_tools_list", "external_data_tools")
)
model: TrialAppModel | None = Field(default=None, validation_alias=AliasChoices("model_dict", "model"))
user_input_form: list[JsonObject] = Field(
default_factory=list, validation_alias=AliasChoices("user_input_form_list", "user_input_form")
)
dataset_query_variable: str | None = None
pre_prompt: str | None = None
agent_mode: TrialAppAgentMode | None = Field(
default=None,
validation_alias=AliasChoices("agent_mode_dict", "agent_mode"),
)
prompt_type: str | None = None
chat_prompt_config: JsonObject | None = Field(
default=None, validation_alias=AliasChoices("chat_prompt_config_dict", "chat_prompt_config")
)
completion_prompt_config: JsonObject | None = Field(
default=None, validation_alias=AliasChoices("completion_prompt_config_dict", "completion_prompt_config")
)
dataset_configs: JsonObject | None = Field(
default=None,
validation_alias=AliasChoices("dataset_configs_dict", "dataset_configs"),
)
file_upload: JsonObject | None = Field(
default=None,
validation_alias=AliasChoices("file_upload_dict", "file_upload"),
)
created_by: str | None = None
created_at: int | None = None
updated_by: str | None = None
updated_at: int | None = None
@field_validator("created_at", "updated_at", mode="before")
@classmethod
def _normalize_timestamp(cls, value: datetime | int | None) -> int | None:
return to_timestamp(value)
class TrialDeletedToolResponse(ResponseModel):
type: str
tool_name: str
provider_id: str
class TrialTagResponse(ResponseModel):
id: str
name: str
type: str
class TrialSiteResponse(ResponseModel):
access_token: str | None = Field(default=None, validation_alias="code")
code: str | None = None
title: str
icon_type: TrialIconType | None = None
icon: str | None = None
icon_background: str | None = None
description: str | None = None
default_language: str
chat_color_theme: str | None = None
chat_color_theme_inverted: bool | None = None
customize_domain: str | None = None
copyright: str | None = None
privacy_policy: str | None = None
input_placeholder: str | None = None
custom_disclaimer: str | None = None
customize_token_strategy: str | None = None
prompt_public: bool | None = None
app_base_url: str | None = None
show_workflow_steps: bool | None = None
use_icon_as_answer_icon: bool | None = None
created_by: str | None = None
created_at: int | None = None
updated_by: str | None = None
updated_at: int | None = None
icon_url: str | None = None
@field_validator("icon_type", mode="before")
@classmethod
def _normalize_icon_type(cls, value: Any) -> str | None:
if hasattr(value, "value"):
return value.value
return value
@field_validator("created_at", "updated_at", mode="before")
@classmethod
def _normalize_timestamp(cls, value: datetime | int | None) -> int | None:
return to_timestamp(value)
class TrialWorkflowPartialResponse(ResponseModel):
id: str
created_by: str | None = None
created_at: int | None = None
updated_by: str | None = None
updated_at: int | None = None
@field_validator("created_at", "updated_at", mode="before")
@classmethod
def _normalize_timestamp(cls, value: datetime | int | None) -> int | None:
return to_timestamp(value)
class TrialAppDetailResponse(ResponseModel):
id: str
name: str
description: str | None = None
mode: TrialAppMode = Field(validation_alias="mode_compatible_with_agent")
icon_type: TrialIconType | None = None
icon: str | None = None
icon_background: str | None = None
icon_url: str | None = None
enable_site: bool
enable_api: bool
model_config_: TrialAppModelConfigResponse | None = Field(
default=None,
validation_alias=AliasChoices("app_model_config", "model_config"),
alias="model_config",
)
workflow: TrialWorkflowPartialResponse | None = None
api_base_url: str | None = None
use_icon_as_answer_icon: bool | None = None
max_active_requests: int | None = None
created_by: str | None = None
created_at: int | None = None
updated_by: str | None = None
updated_at: int | None = None
deleted_tools: list[TrialDeletedToolResponse] = Field(default_factory=list)
access_mode: str | None = None
tags: list[TrialTagResponse] = Field(default_factory=list)
permission_keys: list[str] = Field(default_factory=list)
site: TrialSiteResponse
@field_validator("icon_type", mode="before")
@classmethod
def _normalize_icon_type(cls, value: Any) -> str | None:
if hasattr(value, "value"):
return value.value
return value
@field_validator("created_at", "updated_at", mode="before")
@classmethod
def _normalize_timestamp(cls, value: datetime | int | None) -> int | None:
return to_timestamp(value)
class TrialDatasetResponse(ResponseModel):
id: str
name: str
description: str | None = None
permission: str | None = None
data_source_type: str | None = None
indexing_technique: str | None = None
created_by: str | None = None
created_at: int | None = None
permission_keys: list[str] = Field(default_factory=list)
@field_validator("created_at", mode="before")
@classmethod
def _normalize_timestamp(cls, value: datetime | int | None) -> int | None:
return to_timestamp(value)
class TrialDatasetListResponse(ResponseModel):
data: list[TrialDatasetResponse]
has_more: bool
limit: int
total: int
page: int
class TrialSimpleAccount(ResponseModel):
id: str
name: str | None = None
email: str | None = None
class TrialWorkflowResponse(ResponseModel):
id: str
graph: JsonObject = Field(validation_alias=AliasChoices("graph_dict", "graph"))
features: JsonObject = Field(default_factory=dict, validation_alias=AliasChoices("features_dict", "features"))
hash: str | None = Field(default=None, validation_alias=AliasChoices("unique_hash", "hash"))
version: str | None = None
marked_name: str | None = None
marked_comment: str | None = None
created_by: TrialSimpleAccount | None = Field(
default=None,
validation_alias=AliasChoices("created_by_account", "created_by"),
)
created_at: int | None = None
updated_by: TrialSimpleAccount | None = Field(
default=None,
validation_alias=AliasChoices("updated_by_account", "updated_by"),
)
updated_at: int | None = None
tool_published: bool | None = None
environment_variables: list[JsonObject] = Field(default_factory=list)
conversation_variables: list[JsonObject] = Field(default_factory=list)
rag_pipeline_variables: list[JsonObject] = Field(default_factory=list)
@field_validator("created_at", "updated_at", mode="before")
@classmethod
def _normalize_timestamp(cls, value: datetime | int | None) -> int | None:
return to_timestamp(value)
register_schema_models(
console_ns,
WorkflowRunRequest,
@@ -395,21 +196,15 @@ register_response_schema_models(
SimpleResultResponse,
SiteResponse,
SuggestedQuestionsResponse,
TrialAppDetailResponse,
TrialDatasetListResponse,
TrialWorkflowResponse,
)
simple_account_model = console_ns.models[TrialSimpleAccount.__name__]
class TrialAppWorkflowRunApi(TrialAppResource):
@trial_feature_enable
@console_ns.expect(console_ns.models[WorkflowRunRequest.__name__])
@console_ns.response(200, "Success", console_ns.models[GeneratedAppResponse.__name__])
@with_current_user
@with_session
def post(self, session: Session, current_user: Account, trial_app):
def post(self, current_user: Account, trial_app):
"""
Run workflow
"""
@@ -426,12 +221,7 @@ class TrialAppWorkflowRunApi(TrialAppResource):
app_id = app_model.id
user_id = current_user.id
response = AppGenerateService.generate(
session=session,
app_model=app_model,
user=current_user,
args=args,
invoke_from=InvokeFrom.EXPLORE,
streaming=True,
app_model=app_model, user=current_user, args=args, invoke_from=InvokeFrom.EXPLORE, streaming=True
)
RecommendedAppService.add_trial_app_record(db.session, app_id, user_id)
return helper.compact_generate_response(response)
@@ -481,8 +271,7 @@ class TrialChatApi(TrialAppResource):
@console_ns.response(200, "Success", console_ns.models[GeneratedAppResponse.__name__])
@trial_feature_enable
@with_current_user
@with_session
def post(self, session: Session, current_user: Account, trial_app):
def post(self, current_user: Account, trial_app):
app_model = trial_app
app_mode = AppMode.value_of(app_model.mode)
if app_mode not in {AppMode.CHAT, AppMode.AGENT_CHAT, AppMode.ADVANCED_CHAT}:
@@ -505,12 +294,7 @@ class TrialChatApi(TrialAppResource):
user_id = current_user.id
response = AppGenerateService.generate(
session=session,
app_model=app_model,
user=current_user,
args=args,
invoke_from=InvokeFrom.EXPLORE,
streaming=True,
app_model=app_model, user=current_user, args=args, invoke_from=InvokeFrom.EXPLORE, streaming=True
)
RecommendedAppService.add_trial_app_record(db.session, app_id, user_id)
return helper.compact_generate_response(response)
@@ -630,14 +414,6 @@ class TrialChatTextApi(TrialAppResource):
message_id = request_data.message_id
text = request_data.text
voice = request_data.voice
message_ref = None
if message_id:
app_ref = AppRefService.create_app_ref(app_model)
message_ref = AppRefService.create_message_ref(
app_ref,
message_id,
account_id=current_user.id,
)
# Get IDs before they might be detached from session
app_id = app_model.id
@@ -648,7 +424,7 @@ class TrialChatTextApi(TrialAppResource):
session=db.session,
text=text,
voice=voice,
message_ref=message_ref,
message_id=message_id,
)
RecommendedAppService.add_trial_app_record(db.session, app_id, user_id)
return response
@@ -683,8 +459,7 @@ class TrialCompletionApi(TrialAppResource):
@console_ns.response(200, "Success", console_ns.models[GeneratedAppResponse.__name__])
@trial_feature_enable
@with_current_user
@with_session
def post(self, session: Session, current_user: Account, trial_app):
def post(self, current_user: Account, trial_app):
app_model = trial_app
if app_model.mode != "completion":
raise NotCompletionAppError()
@@ -701,12 +476,7 @@ class TrialCompletionApi(TrialAppResource):
user_id = current_user.id
response = AppGenerateService.generate(
session=session,
app_model=app_model,
user=current_user,
args=args,
invoke_from=InvokeFrom.EXPLORE,
streaming=streaming,
app_model=app_model, user=current_user, args=args, invoke_from=InvokeFrom.EXPLORE, streaming=streaming
)
RecommendedAppService.add_trial_app_record(db.session, app_id, user_id)
@@ -787,35 +557,34 @@ class TrialAppParameterApi(Resource):
class AppApi(Resource):
@console_ns.response(200, "Success", console_ns.models[TrialAppDetailResponse.__name__])
@console_ns.response(200, "Success", app_detail_with_site_model)
@get_app_model_with_trial(None)
@marshal_with(app_detail_with_site_model)
def get(self, app_model):
"""Get app detail"""
app_service = AppService()
app_model = app_service.get_app(app_model)
return dump_response(TrialAppDetailResponse, app_model)
return app_model
class AppWorkflowApi(Resource):
@console_ns.response(200, "Success", console_ns.models[TrialWorkflowResponse.__name__])
@console_ns.response(200, "Success", workflow_model)
@get_app_model_with_trial(None)
@marshal_with(workflow_model)
def get(self, app_model):
"""Get workflow detail"""
if not app_model.workflow_id:
raise AppUnavailableError()
workflow = db.session.get(Workflow, app_model.workflow_id)
if workflow is None:
raise AppUnavailableError()
return dump_response(TrialWorkflowResponse, workflow)
return workflow
class DatasetListApi(Resource):
@console_ns.doc(params=query_params_from_model(TrialDatasetListQuery))
@console_ns.response(200, "Success", console_ns.models[TrialDatasetListResponse.__name__])
@console_ns.response(200, "Success", dataset_list_model)
@get_app_model_with_trial(None)
def get(self, app_model):
page = request.args.get("page", default=1, type=int)
@@ -828,8 +597,10 @@ class DatasetListApi(Resource):
else:
raise NeedAddIdsError()
response = {"data": datasets, "has_more": len(datasets) == limit, "limit": limit, "total": total, "page": page}
return dump_response(TrialDatasetListResponse, response)
data = cast(list[dict[str, Any]], marshal(datasets, dataset_fields))
response = {"data": data, "has_more": len(datasets) == limit, "limit": limit, "total": total, "page": page}
return response
console_ns.add_resource(TrialChatApi, "/trial-apps/<uuid:app_id>/chat-messages", endpoint="trial_app_chat_completion")
+6 -15
View File
@@ -1,10 +1,9 @@
import logging
from sqlalchemy.orm import Session
from werkzeug.exceptions import InternalServerError
from controllers.common.controller_schemas import WorkflowRunPayload
from controllers.common.fields import SimpleResultResponse
from controllers.common.fields import GeneratedAppResponse, SimpleResultResponse
from controllers.common.schema import register_response_schema_models, register_schema_model
from controllers.console.app.error import (
CompletionRequestError,
@@ -12,7 +11,6 @@ from controllers.console.app.error import (
ProviderNotInitializeError,
ProviderQuotaExceededError,
)
from controllers.console.app.wraps import with_session
from controllers.console.explore.error import NotWorkflowAppError
from controllers.console.explore.wraps import InstalledAppResource
from controllers.console.wraps import with_current_user
@@ -38,16 +36,15 @@ from .. import console_ns
logger = logging.getLogger(__name__)
register_schema_model(console_ns, WorkflowRunPayload)
register_response_schema_models(console_ns, SimpleResultResponse)
register_response_schema_models(console_ns, GeneratedAppResponse, SimpleResultResponse)
@console_ns.route("/installed-apps/<uuid:installed_app_id>/workflows/run")
class InstalledAppWorkflowRunApi(InstalledAppResource):
@console_ns.expect(console_ns.models[WorkflowRunPayload.__name__])
@console_ns.response(200, "Success")
@console_ns.response(200, "Success", console_ns.models[GeneratedAppResponse.__name__])
@with_current_user
@with_session
def post(self, session: Session, current_user: Account, installed_app: InstalledApp):
def post(self, current_user: Account, installed_app: InstalledApp):
"""
Run workflow
"""
@@ -62,15 +59,9 @@ class InstalledAppWorkflowRunApi(InstalledAppResource):
args = payload.model_dump(exclude_none=True)
try:
response = AppGenerateService.generate(
session=session,
app_model=app_model,
user=current_user,
args=args,
invoke_from=InvokeFrom.EXPLORE,
streaming=True,
app_model=app_model, user=current_user, args=args, invoke_from=InvokeFrom.EXPLORE, streaming=True
)
# response-contract:ignore compact_generate_response
return helper.compact_generate_response(response)
except ProviderTokenNotInitError as ex:
raise ProviderNotInitializeError(ex.description)
@@ -110,4 +101,4 @@ class InstalledAppWorkflowTaskStopApi(InstalledAppResource):
# New graph engine command channel mechanism
GraphEngineManager(redis_client).send_stop_command(task_id)
return SimpleResultResponse(result="success").model_dump(mode="json")
return {"result": "success"}
+43 -35
View File
@@ -4,18 +4,18 @@ from uuid import UUID
from flask import request
from flask_restx import Resource
from pydantic import BaseModel, Field, RootModel, field_validator
from pydantic import BaseModel, Field, TypeAdapter, field_validator
from constants import HIDDEN_VALUE
from extensions.ext_database import db
from fields.base import ResponseModel
from libs.helper import dump_response, to_timestamp
from libs.helper import to_timestamp
from libs.login import login_required
from models.api_based_extension import APIBasedExtension
from services.api_based_extension_service import APIBasedExtensionService
from services.code_based_extension_service import CodeBasedExtensionService
from ..common.schema import query_params_from_model, register_response_schema_models, register_schema_models
from ..common.schema import DEFAULT_REF_TEMPLATE_OPENAPI_3_0, query_params_from_model, register_schema_models
from . import console_ns
from .wraps import account_initialization_required, setup_required, with_current_tenant_id
@@ -61,21 +61,36 @@ class APIBasedExtensionResponse(ResponseModel):
return to_timestamp(value)
class APIBasedExtensionListResponse(RootModel[list[APIBasedExtensionResponse]]):
pass
register_schema_models(
console_ns,
CodeBasedExtensionQuery,
APIBasedExtensionPayload,
)
register_response_schema_models(
console_ns,
CodeBasedExtensionResponse,
APIBasedExtensionResponse,
APIBasedExtensionListResponse,
)
console_ns.schema_model(
"APIBasedExtensionListResponse",
TypeAdapter(list[APIBasedExtensionResponse]).json_schema(ref_template=DEFAULT_REF_TEMPLATE_OPENAPI_3_0),
)
def _serialize_api_based_extension(extension: APIBasedExtension) -> dict[str, Any]:
return APIBasedExtensionResponse.model_validate(extension, from_attributes=True).model_dump(mode="json")
def _serialize_saved_api_based_extension(extension: APIBasedExtension, api_key: str) -> dict[str, Any]:
"""Serialize a saved extension with the plaintext key used for response masking only.
APIBasedExtensionService.save mutates the ORM object to hold the encrypted token before returning it. The response
contract, however, should match list/detail responses, where api_key is masked from the decrypted token.
"""
return APIBasedExtensionResponse(
id=extension.id,
name=extension.name,
api_endpoint=extension.api_endpoint,
api_key=api_key,
created_at=to_timestamp(extension.created_at),
).model_dump(mode="json")
@console_ns.route("/code-based-extension")
@@ -104,16 +119,16 @@ class CodeBasedExtensionAPI(Resource):
class APIBasedExtensionAPI(Resource):
@console_ns.doc("get_api_based_extensions")
@console_ns.doc(description="Get all API-based extensions for current tenant")
@console_ns.response(200, "Success", console_ns.models[APIBasedExtensionListResponse.__name__])
@console_ns.response(200, "Success", console_ns.models["APIBasedExtensionListResponse"])
@setup_required
@login_required
@account_initialization_required
@with_current_tenant_id
def get(self, current_tenant_id: str):
return dump_response(
APIBasedExtensionListResponse,
APIBasedExtensionService.get_all_by_tenant_id(db.session(), current_tenant_id),
)
return [
_serialize_api_based_extension(extension)
for extension in APIBasedExtensionService.get_all_by_tenant_id(db.session(), current_tenant_id)
]
@console_ns.doc("create_api_based_extension")
@console_ns.doc(description="Create a new API-based extension")
@@ -133,14 +148,12 @@ class APIBasedExtensionAPI(Resource):
api_key=payload.api_key,
)
extension = APIBasedExtensionService.save(db.session(), extension_data)
return APIBasedExtensionResponse(
id=extension.id,
name=extension.name,
api_endpoint=extension.api_endpoint,
api_key=payload.api_key,
created_at=to_timestamp(extension.created_at),
).model_dump(mode="json"), 201
return (
_serialize_saved_api_based_extension(
APIBasedExtensionService.save(db.session(), extension_data), payload.api_key
),
201,
)
@console_ns.route("/api-based-extension/<uuid:id>")
@@ -156,9 +169,8 @@ class APIBasedExtensionDetailAPI(Resource):
def get(self, current_tenant_id: str, id: UUID):
api_based_extension_id = str(id)
return dump_response(
APIBasedExtensionResponse,
APIBasedExtensionService.get_with_tenant_id(db.session(), current_tenant_id, api_based_extension_id),
return _serialize_api_based_extension(
APIBasedExtensionService.get_with_tenant_id(db.session(), current_tenant_id, api_based_extension_id)
)
@console_ns.doc("update_api_based_extension")
@@ -187,14 +199,10 @@ class APIBasedExtensionDetailAPI(Resource):
extension_data_from_db.api_key = payload.api_key
api_key_for_response = payload.api_key
APIBasedExtensionService.save(db.session(), extension_data_from_db)
return APIBasedExtensionResponse(
id=extension_data_from_db.id,
name=extension_data_from_db.name,
api_endpoint=extension_data_from_db.api_endpoint,
api_key=api_key_for_response,
created_at=to_timestamp(extension_data_from_db.created_at),
).model_dump(mode="json")
return _serialize_saved_api_based_extension(
APIBasedExtensionService.save(db.session(), extension_data_from_db),
api_key_for_response,
)
@console_ns.doc("delete_api_based_extension")
@console_ns.doc(description="Delete API-based extension")
-17
View File
@@ -38,22 +38,6 @@ register_response_schema_models(console_ns, AllowedExtensionsResponse, TextConte
PREVIEW_WORDS_LIMIT = 3000
_FILE_UPLOAD_PARAMS = {
"file": {
"description": "File to upload",
"in": "formData",
"type": "file",
"required": True,
},
"source": {
"description": "Optional upload source",
"in": "formData",
"type": "string",
"enum": ["datasets"],
"required": False,
},
}
@console_ns.route("/files/upload")
class FileApi(Resource):
@@ -80,7 +64,6 @@ class FileApi(Resource):
@login_required
@account_initialization_required
@cloud_edition_billing_resource_check("documents")
@console_ns.doc(consumes=["multipart/form-data"], params=_FILE_UPLOAD_PARAMS)
@console_ns.response(201, "File uploaded successfully", console_ns.models[FileResponse.__name__])
@with_current_user
def post(self, current_user: Account):
+8 -11
View File
@@ -21,19 +21,14 @@ class SnippetListQuery(BaseModel):
@field_validator("creators", mode="before")
@classmethod
def parse_creators(cls, value: object) -> list[str] | None:
"""Normalize creator filters without comma-splitting list values."""
if value is None:
return None
if isinstance(value, str):
item = value.strip()
return [item] if item else None
return cls._normalize_string_list(value, "creators")
"""Normalize creators filter from query string or list input."""
return cls._normalize_string_list(value)
@field_validator("tag_ids", mode="before")
@classmethod
def parse_tag_ids(cls, value: object) -> list[str] | None:
"""Normalize and validate tag IDs from repeated query parameters."""
items = cls._normalize_string_list(value, "tag_ids")
"""Normalize and validate tag IDs from query string or list input."""
items = cls._normalize_string_list(value)
if not items:
return None
try:
@@ -42,12 +37,14 @@ class SnippetListQuery(BaseModel):
raise ValueError("Invalid UUID format in tag_ids.") from exc
@staticmethod
def _normalize_string_list(value: object, field_name: str) -> list[str] | None:
def _normalize_string_list(value: object) -> list[str] | None:
if value is None:
return None
if isinstance(value, str):
return [item.strip() for item in value.split(",") if item.strip()] or None
if isinstance(value, list):
return [str(item).strip() for item in value if str(item).strip()] or None
raise ValueError(f"Unsupported {field_name} type.")
return None
class IconInfo(BaseModel):
@@ -8,7 +8,6 @@ from pydantic import BaseModel, Field
from sqlalchemy.orm import Session, sessionmaker
from werkzeug.exceptions import BadRequest, InternalServerError, NotFound
from controllers.common.controller_schemas import WorkflowUpdatePayload
from controllers.common.fields import GeneratedAppResponse, SimpleResultResponse
from controllers.common.schema import query_params_from_model, register_response_schema_models, register_schema_models
from controllers.console import console_ns
@@ -97,7 +96,6 @@ register_schema_models(
SnippetLoopNodeRunPayload,
SnippetWorkflowListQuery,
WorkflowRunQuery,
WorkflowUpdatePayload,
PublishWorkflowPayload,
)
register_response_schema_models(
@@ -167,6 +165,7 @@ class SnippetDraftWorkflowApi(Resource):
@account_initialization_required
@get_snippet
@edit_permission_required
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.SNIPPETS_MANAGE, resource_required=False)
def get(self, snippet: CustomizedSnippet):
"""Get draft workflow for snippet."""
snippet_service = _snippet_service()
@@ -235,6 +234,7 @@ class SnippetDraftConfigApi(Resource):
@account_initialization_required
@get_snippet
@edit_permission_required
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.SNIPPETS_MANAGE, resource_required=False)
def get(self, snippet: CustomizedSnippet):
"""Get snippet draft workflow configuration limits."""
return {
@@ -256,6 +256,7 @@ class SnippetPublishedWorkflowApi(Resource):
@account_initialization_required
@get_snippet
@edit_permission_required
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.SNIPPETS_MANAGE, resource_required=False)
def get(self, snippet: CustomizedSnippet):
"""Get published workflow for snippet."""
if not snippet.is_published:
@@ -320,6 +321,7 @@ class SnippetDefaultBlockConfigsApi(Resource):
@account_initialization_required
@get_snippet
@edit_permission_required
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.SNIPPETS_MANAGE, resource_required=False)
def get(self, snippet: CustomizedSnippet):
"""Get default block configurations for snippet workflow."""
snippet_service = _snippet_service()
@@ -342,9 +344,7 @@ class SnippetPublishedAllWorkflowApi(Resource):
@account_initialization_required
@get_snippet
@edit_permission_required
@rbac_permission_required(
RBACResourceScope.WORKSPACE, RBACPermission.SNIPPETS_CREATE_AND_MODIFY, resource_required=False
)
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.SNIPPETS_MANAGE, resource_required=False)
def get(self, snippet: CustomizedSnippet):
"""Get all published workflow versions for snippet."""
args = SnippetWorkflowListQuery.model_validate(request.args.to_dict(flat=True))
@@ -413,49 +413,6 @@ class SnippetDraftWorkflowRestoreApi(Resource):
}
@console_ns.route("/snippets/<uuid:snippet_id>/workflows/<string:workflow_id>")
class SnippetWorkflowByIdApi(Resource):
@console_ns.doc("update_snippet_workflow_by_id")
@console_ns.doc(description="Update published snippet workflow attributes")
@console_ns.doc(params={"snippet_id": "Snippet ID", "workflow_id": "Workflow ID"})
@console_ns.expect(console_ns.models[WorkflowUpdatePayload.__name__])
@console_ns.response(200, "Workflow updated successfully", console_ns.models[SnippetWorkflowResponse.__name__])
@console_ns.response(400, "No valid fields to update")
@console_ns.response(404, "Workflow not found")
@setup_required
@login_required
@account_initialization_required
@with_current_user
@get_snippet
@edit_permission_required
@rbac_permission_required(
RBACResourceScope.WORKSPACE, RBACPermission.SNIPPETS_CREATE_AND_MODIFY, resource_required=False
)
def patch(self, current_user: Account, snippet: CustomizedSnippet, workflow_id: str):
"""Update a published snippet workflow version's display metadata."""
payload = WorkflowUpdatePayload.model_validate(console_ns.payload or {})
update_data = payload.model_dump(exclude_unset=True)
if not update_data:
return {"message": "No valid fields to update"}, 400
snippet_service = _snippet_service()
with _snippet_session_maker().begin() as session:
workflow = snippet_service.update_workflow(
session=session,
snippet=snippet,
workflow_id=workflow_id,
account=current_user,
data=update_data,
)
if not workflow:
raise NotFound("Workflow not found")
response = SnippetWorkflowResponse.model_validate(workflow, from_attributes=True).model_dump(mode="json")
response["input_fields"] = snippet.input_fields_list
return response
@console_ns.route("/snippets/<uuid:snippet_id>/workflow-runs")
class SnippetWorkflowRunsApi(Resource):
@console_ns.doc("list_snippet_workflow_runs")
@@ -557,6 +514,9 @@ class SnippetDraftNodeRunApi(Resource):
@with_current_user
@get_snippet
@edit_permission_required
@rbac_permission_required(
RBACResourceScope.WORKSPACE, RBACPermission.SNIPPETS_CREATE_AND_MODIFY, resource_required=False
)
def post(self, current_user: Account, snippet: CustomizedSnippet, node_id: str):
"""
Run a single node in snippet draft workflow.
@@ -645,6 +605,9 @@ class SnippetDraftRunIterationNodeApi(Resource):
@with_current_user
@get_snippet
@edit_permission_required
@rbac_permission_required(
RBACResourceScope.WORKSPACE, RBACPermission.SNIPPETS_CREATE_AND_MODIFY, resource_required=False
)
def post(self, current_user: Account, snippet: CustomizedSnippet, node_id: str):
"""
Run a draft workflow iteration node for snippet.
@@ -690,6 +653,9 @@ class SnippetDraftRunLoopNodeApi(Resource):
@with_current_user
@get_snippet
@edit_permission_required
@rbac_permission_required(
RBACResourceScope.WORKSPACE, RBACPermission.SNIPPETS_CREATE_AND_MODIFY, resource_required=False
)
def post(self, current_user: Account, snippet: CustomizedSnippet, node_id: str):
"""
Run a draft workflow loop node for snippet.
@@ -733,6 +699,9 @@ class SnippetDraftWorkflowRunApi(Resource):
@with_current_user
@get_snippet
@edit_permission_required
@rbac_permission_required(
RBACResourceScope.WORKSPACE, RBACPermission.SNIPPETS_CREATE_AND_MODIFY, resource_required=False
)
def post(self, current_user: Account, snippet: CustomizedSnippet):
"""
Run draft workflow for snippet.
@@ -771,6 +740,9 @@ class SnippetWorkflowTaskStopApi(Resource):
@account_initialization_required
@get_snippet
@edit_permission_required
@rbac_permission_required(
RBACResourceScope.WORKSPACE, RBACPermission.SNIPPETS_CREATE_AND_MODIFY, resource_required=False
)
def post(self, snippet: CustomizedSnippet, task_id: str):
"""
Stop a running snippet workflow task.
@@ -34,8 +34,11 @@ from controllers.console.app.workflow_draft_variable import (
)
from controllers.console.snippets.snippet_workflow import get_snippet
from controllers.console.wraps import (
RBACPermission,
RBACResourceScope,
account_initialization_required,
edit_permission_required,
rbac_permission_required,
setup_required,
with_current_user,
)
@@ -102,6 +105,7 @@ class SnippetWorkflowVariableCollectionApi(Resource):
)
@_snippet_draft_var_prerequisite
@marshal_with(workflow_draft_variable_list_without_value_model)
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.SNIPPETS_MANAGE, resource_required=False)
def get(self, current_user: Account, snippet: CustomizedSnippet) -> WorkflowDraftVariableList:
args = WorkflowDraftVariableListQuery.model_validate(request.args.to_dict(flat=True)) # type: ignore
@@ -125,6 +129,9 @@ class SnippetWorkflowVariableCollectionApi(Resource):
@console_ns.doc(description="Delete all draft workflow variables for the current user (snippet scope)")
@console_ns.response(204, "Workflow variables deleted successfully")
@_snippet_draft_var_prerequisite
@rbac_permission_required(
RBACResourceScope.WORKSPACE, RBACPermission.SNIPPETS_CREATE_AND_MODIFY, resource_required=False
)
def delete(self, current_user: Account, snippet: CustomizedSnippet) -> Response:
draft_var_srv = WorkflowDraftVariableService(session=db.session())
draft_var_srv.delete_user_workflow_variables(snippet.id, user_id=current_user.id)
+20 -54
View File
@@ -3,18 +3,13 @@ from uuid import UUID
from flask import request
from flask_restx import Resource
from pydantic import BaseModel, Field, RootModel, field_validator
from sqlalchemy import select
from pydantic import BaseModel, Field, field_validator
from werkzeug.exceptions import Forbidden
from configs import dify_config
from controllers.common.fields import SimpleResultResponse
from controllers.common.schema import query_params_from_model, register_response_schema_models, register_schema_models
from controllers.common.wraps import enforce_rbac_access
from controllers.console import console_ns
from controllers.console.wraps import (
RBACPermission,
RBACResourceScope,
account_initialization_required,
edit_permission_required,
setup_required,
@@ -23,11 +18,9 @@ from controllers.console.wraps import (
)
from extensions.ext_database import db
from fields.base import ResponseModel
from libs.helper import dump_response
from libs.login import current_account_with_tenant, login_required
from libs.login import login_required
from models import Account
from models.enums import TagType
from models.model import Tag
from services.tag_service import (
SaveTagPayload,
TagBindingCreatePayload,
@@ -86,10 +79,6 @@ class TagResponse(ResponseModel):
return str(value)
class TagListResponse(RootModel[list[TagResponse]]):
pass
register_schema_models(
console_ns,
TagBasePayload,
@@ -97,33 +86,9 @@ register_schema_models(
TagBindingPayload,
TagBindingRemovePayload,
TagListQueryParam,
TagResponse,
)
register_response_schema_models(console_ns, SimpleResultResponse, TagResponse, TagListResponse)
def _enforce_snippet_tag_rbac_if_needed(tag_type: TagType | str | None) -> None:
if tag_type != TagType.SNIPPET:
return
if not dify_config.RBAC_ENABLED:
return
current_user, current_tenant_id = current_account_with_tenant()
enforce_rbac_access(
tenant_id=current_tenant_id,
account_id=current_user.id,
resource_type=RBACResourceScope.WORKSPACE,
scene=RBACPermission.SNIPPETS_CREATE_AND_MODIFY,
resource_required=False,
)
def _enforce_snippet_tag_rbac_by_tag_id(tag_id: str) -> None:
if not dify_config.RBAC_ENABLED:
return
_, current_tenant_id = current_account_with_tenant()
tag_type = db.session.scalar(select(Tag.type).where(Tag.id == tag_id, Tag.tenant_id == current_tenant_id).limit(1))
_enforce_snippet_tag_rbac_if_needed(tag_type)
register_response_schema_models(console_ns, SimpleResultResponse)
@console_ns.route("/tags")
@@ -132,14 +97,18 @@ class TagListApi(Resource):
@login_required
@account_initialization_required
@console_ns.doc(params=query_params_from_model(TagListQueryParam))
@console_ns.response(200, "Success", console_ns.models[TagListResponse.__name__])
@console_ns.doc(responses={200: ("Success", [console_ns.models[TagResponse.__name__]])})
@with_current_tenant_id
def get(self, current_tenant_id: str):
raw_args = request.args.to_dict()
param = TagListQueryParam.model_validate(raw_args)
tags = TagService.get_tags(db.session(), param.type, current_tenant_id, param.keyword)
return dump_response(TagListResponse, tags), 200
serialized_tags = [
TagResponse.model_validate(tag, from_attributes=True).model_dump(mode="json") for tag in tags
]
return serialized_tags, 200
@console_ns.expect(console_ns.models[TagBasePayload.__name__])
@console_ns.response(200, "Success", console_ns.models[TagResponse.__name__])
@@ -153,10 +122,13 @@ class TagListApi(Resource):
raise Forbidden()
payload = TagBasePayload.model_validate(console_ns.payload or {})
_enforce_snippet_tag_rbac_if_needed(payload.type)
tag = TagService.save_tags(SaveTagPayload(name=payload.name, type=payload.type), db.session)
return dump_response(TagResponse, {"id": tag.id, "name": tag.name, "type": tag.type, "binding_count": 0}), 200
response = TagResponse.model_validate(
{"id": tag.id, "name": tag.name, "type": tag.type, "binding_count": 0}
).model_dump(mode="json")
return response, 200
@console_ns.route("/tags/<uuid:tag_id>")
@@ -174,18 +146,15 @@ class TagUpdateDeleteApi(Resource):
raise Forbidden()
payload = TagUpdateRequestPayload.model_validate(console_ns.payload or {})
_enforce_snippet_tag_rbac_by_tag_id(tag_id_str)
tag = TagService.update_tags(UpdateTagPayload(name=payload.name), tag_id_str, db.session)
binding_count = TagService.get_tag_binding_count(tag_id_str, db.session)
return (
dump_response(
TagResponse,
{"id": tag.id, "name": tag.name, "type": tag.type, "binding_count": binding_count},
),
200,
)
response = TagResponse.model_validate(
{"id": tag.id, "name": tag.name, "type": tag.type, "binding_count": binding_count}
).model_dump(mode="json")
return response, 200
@setup_required
@login_required
@@ -195,7 +164,6 @@ class TagUpdateDeleteApi(Resource):
def delete(self, tag_id: UUID):
tag_id_str = str(tag_id)
_enforce_snippet_tag_rbac_by_tag_id(tag_id_str)
TagService.delete_tag(tag_id_str, db.session)
return "", 204
@@ -216,7 +184,6 @@ def _create_tag_bindings(current_user: Account) -> tuple[dict[str, str], int]:
_require_tag_binding_edit_permission(current_user)
payload = TagBindingPayload.model_validate(console_ns.payload or {})
_enforce_snippet_tag_rbac_if_needed(payload.type)
TagService.save_tag_binding(
TagBindingCreatePayload(
tag_ids=payload.tag_ids,
@@ -232,7 +199,6 @@ def _remove_tag_bindings(current_user: Account) -> tuple[dict[str, str], int]:
_require_tag_binding_edit_permission(current_user)
payload = TagBindingRemovePayload.model_validate(console_ns.payload or {})
_enforce_snippet_tag_rbac_if_needed(payload.type)
TagService.delete_tag_binding(
TagBindingDeletePayload(
tag_ids=payload.tag_ids,
+2 -2
View File
@@ -84,6 +84,8 @@ class MemberActionTenantResponse(ResponseModel):
register_enum_models(console_ns, TenantAccountRole)
register_schema_models(
console_ns,
AccountWithRole,
AccountWithRoleList,
MemberInvitePayload,
MemberRoleUpdatePayload,
OwnerTransferEmailPayload,
@@ -92,8 +94,6 @@ register_schema_models(
)
register_response_schema_models(
console_ns,
AccountWithRole,
AccountWithRoleList,
SimpleResultDataResponse,
SimpleResultResponse,
VerificationTokenResponse,
@@ -18,7 +18,6 @@ from controllers.console.wraps import (
with_current_tenant_id,
with_current_user,
)
from extensions.ext_database import db
from fields.base import ResponseModel
from graphon.model_runtime.entities.model_entities import ModelType
from graphon.model_runtime.errors.validate import CredentialsValidateFailedError
@@ -353,7 +352,7 @@ class ModelProviderPaymentCheckoutUrlApi(Resource):
def get(self, current_tenant_id: str, current_user: Account, provider: str):
if provider != "anthropic":
raise ValueError(f"provider name {provider} is invalid")
BillingService.is_tenant_owner_or_admin(db.session, current_user)
BillingService.is_tenant_owner_or_admin(current_user)
data = BillingService.get_model_provider_payment_link(
provider_name=provider,
tenant_id=current_tenant_id,
+2 -19
View File
@@ -302,28 +302,11 @@ class PluginListResponse(ResponseModel):
class PluginVersionsResponse(ResponseModel):
versions: Mapping[str, PluginService.LatestPluginCache | None]
class PluginInstallationItemResponse(ResponseModel):
id: str
created_at: datetime
updated_at: datetime
tenant_id: str
endpoints_setups: int
endpoints_active: int
runtime_type: str
source: PluginInstallationSource
meta: Mapping[str, Any]
plugin_id: str
plugin_unique_identifier: str
version: str
checksum: str
declaration: PluginDeclarationResponse
versions: Any
class PluginInstallationsResponse(ResponseModel):
plugins: list[PluginInstallationItemResponse]
plugins: Any
class PluginManifestResponse(ResponseModel):
+10 -34
View File
@@ -1,6 +1,7 @@
from __future__ import annotations
from typing import Any, Literal
from enum import StrEnum
from typing import Any
from flask import request
from flask_restx import Resource
@@ -9,11 +10,10 @@ from sqlalchemy import select
from werkzeug.exceptions import NotFound
from configs import dify_config
from controllers.common.schema import query_params_from_model, register_response_schema_models, register_schema_models
from controllers.common.schema import register_response_schema_models
from controllers.console import console_ns
from controllers.console.wraps import RBACPermission, RBACResourceScope, rbac_permission_required
from core.db.session_factory import session_factory
from core.rbac import RBACResourceWhitelistScope
from libs.login import current_account_with_tenant, login_required
from models import Account
from services.enterprise import rbac_service as svc
@@ -511,8 +511,14 @@ class RBACAccessPolicyBindingUnlockApi(Resource):
# ---------------------------------------------------------------------------
class _AccessScope(StrEnum):
ALL = "all"
SPECIFIC = "specific"
ONLY_ME = "only_me"
class _ResourceAccessScopeRequest(BaseModel):
scope: RBACResourceWhitelistScope
scope: _AccessScope
class _ReplaceBindingsRequest(BaseModel):
@@ -538,20 +544,6 @@ class _DeleteMemberBindingsRequest(BaseModel):
return value
class _AccessControlLanguageQuery(BaseModel):
language: Literal["en", "ja", "zh"] | None = Field(default=None, description="Localized policy label language")
register_schema_models(
console_ns,
_ResourceAccessScopeRequest,
_ReplaceBindingsRequest,
_DeleteMemberBindingsRequest,
_AccessControlLanguageQuery,
svc.ReplaceUserAccessPolicies,
)
@console_ns.route("/workspaces/current/rbac/my-permissions")
class RBACMyPermissionsApi(Resource):
@login_required
@@ -571,7 +563,6 @@ class RBACMyPermissionsApi(Resource):
@console_ns.route("/workspaces/current/rbac/apps/<uuid:app_id>/access-policy")
class RBACAppMatrixApi(Resource):
@login_required
@console_ns.doc(params=query_params_from_model(_AccessControlLanguageQuery))
@console_ns.response(200, "Success", console_ns.models[svc.AppAccessMatrix.__name__])
def get(self, app_id):
tenant_id, account_id = _current_ids()
@@ -589,7 +580,6 @@ class RBACAppWhitelistApi(Resource):
return _dump(svc.RBACService.AppAccess.whitelist(tenant_id, account_id, str(app_id)))
@login_required
@console_ns.expect(console_ns.models[_ResourceAccessScopeRequest.__name__])
@console_ns.response(200, "Success", console_ns.models[svc.ResourceWhitelist.__name__])
def put(self, app_id):
tenant_id, account_id = _current_ids()
@@ -607,7 +597,6 @@ class RBACAppWhitelistApi(Resource):
@console_ns.route("/workspaces/current/rbac/apps/<uuid:app_id>/user-access-policies")
class RBACAppUserAccessPoliciesApi(Resource):
@login_required
@console_ns.doc(params=query_params_from_model(_AccessControlLanguageQuery))
@console_ns.response(200, "Success", console_ns.models[svc.ResourceUserAccessPoliciesResponse.__name__])
def get(self, app_id):
tenant_id, account_id = _current_ids()
@@ -619,7 +608,6 @@ class RBACAppUserAccessPoliciesApi(Resource):
@console_ns.route("/workspaces/current/rbac/apps/<uuid:app_id>/users/<uuid:target_account_id>/access-policies")
class RBACAppUserAccessPolicyAssignmentApi(Resource):
@login_required
@console_ns.expect(console_ns.models[svc.ReplaceUserAccessPolicies.__name__])
@console_ns.response(200, "Success", console_ns.models[svc.ReplaceUserAccessPoliciesResponse.__name__])
def put(self, app_id, target_account_id):
tenant_id, account_id = _current_ids()
@@ -653,7 +641,6 @@ class RBACAppMemberBindingsApi(Resource):
return _dump(svc.RBACService.AppAccess.list_member_bindings(tenant_id, account_id, str(app_id), str(policy_id)))
@login_required
@console_ns.expect(console_ns.models[_DeleteMemberBindingsRequest.__name__])
@console_ns.response(200, "Success", console_ns.models[svc.MemberBindingsResponse.__name__])
def delete(self, app_id, policy_id):
tenant_id, account_id = _current_ids()
@@ -676,7 +663,6 @@ class RBACAppMemberBindingsApi(Resource):
@console_ns.route("/workspaces/current/rbac/datasets/<uuid:dataset_id>/access-policy")
class RBACDatasetMatrixApi(Resource):
@login_required
@console_ns.doc(params=query_params_from_model(_AccessControlLanguageQuery))
@console_ns.response(200, "Success", console_ns.models[svc.DatasetAccessMatrix.__name__])
def get(self, dataset_id):
tenant_id, account_id = _current_ids()
@@ -694,7 +680,6 @@ class RBACDatasetWhitelistApi(Resource):
return _dump(svc.RBACService.DatasetAccess.whitelist(tenant_id, account_id, str(dataset_id)))
@login_required
@console_ns.expect(console_ns.models[_ResourceAccessScopeRequest.__name__])
@console_ns.response(200, "Success", console_ns.models[svc.ResourceWhitelist.__name__])
def put(self, dataset_id):
tenant_id, account_id = _current_ids()
@@ -712,7 +697,6 @@ class RBACDatasetWhitelistApi(Resource):
@console_ns.route("/workspaces/current/rbac/datasets/<uuid:dataset_id>/user-access-policies")
class RBACDatasetUserAccessPoliciesApi(Resource):
@login_required
@console_ns.doc(params=query_params_from_model(_AccessControlLanguageQuery))
@console_ns.response(200, "Success", console_ns.models[svc.ResourceUserAccessPoliciesResponse.__name__])
def get(self, dataset_id):
tenant_id, account_id = _current_ids()
@@ -724,7 +708,6 @@ class RBACDatasetUserAccessPoliciesApi(Resource):
@console_ns.route("/workspaces/current/rbac/datasets/<uuid:dataset_id>/users/<uuid:target_account_id>/access-policies")
class RBACDatasetUserAccessPolicyAssignmentApi(Resource):
@login_required
@console_ns.expect(console_ns.models[svc.ReplaceUserAccessPolicies.__name__])
@console_ns.response(200, "Success", console_ns.models[svc.ReplaceUserAccessPoliciesResponse.__name__])
def put(self, dataset_id, target_account_id):
tenant_id, account_id = _current_ids()
@@ -764,7 +747,6 @@ class RBACDatasetMemberBindingsApi(Resource):
)
@login_required
@console_ns.expect(console_ns.models[_DeleteMemberBindingsRequest.__name__])
@console_ns.response(200, "Success", console_ns.models[svc.MemberBindingsResponse.__name__])
def delete(self, dataset_id, policy_id):
tenant_id, account_id = _current_ids()
@@ -803,7 +785,6 @@ class RBACWorkspaceAppRoleBindingsApi(Resource):
@console_ns.route("/workspaces/current/rbac/workspace/apps/access-policies/<uuid:policy_id>/bindings")
class RBACWorkspaceAppBindingsApi(Resource):
@login_required
@console_ns.expect(console_ns.models[_ReplaceBindingsRequest.__name__])
@console_ns.response(200, "Success", console_ns.models[svc.AccessMatrixItem.__name__])
def put(self, policy_id):
tenant_id, account_id = _current_ids()
@@ -851,7 +832,6 @@ class RBACWorkspaceDatasetRoleBindingsApi(Resource):
@console_ns.route("/workspaces/current/rbac/workspace/datasets/access-policies/<uuid:policy_id>/bindings")
class RBACWorkspaceDatasetBindingsApi(Resource):
@login_required
@console_ns.expect(console_ns.models[_ReplaceBindingsRequest.__name__])
@console_ns.response(200, "Success", console_ns.models[svc.AccessMatrixItem.__name__])
def put(self, policy_id):
tenant_id, account_id = _current_ids()
@@ -893,9 +873,6 @@ class _ReplaceMemberRolesRequest(BaseModel):
return value
register_schema_models(console_ns, _ReplaceMemberRolesRequest)
@console_ns.route("/workspaces/current/rbac/members/<uuid:member_id>/rbac-roles")
class RBACMemberRolesApi(Resource):
@login_required
@@ -905,7 +882,6 @@ class RBACMemberRolesApi(Resource):
return _dump(svc.RBACService.MemberRoles.get(tenant_id, account_id, str(member_id)))
@login_required
@console_ns.expect(console_ns.models[_ReplaceMemberRolesRequest.__name__])
@console_ns.response(200, "Success", console_ns.models[svc.MemberRolesResponse.__name__])
def put(self, member_id):
tenant_id, account_id = _current_ids()
+50 -107
View File
@@ -1,21 +1,17 @@
import logging
from datetime import datetime
import re
from typing import Any
from urllib.parse import quote
from flask import Response, request
from flask_restx import Resource, marshal
from pydantic import Field as PydanticField
from pydantic import field_validator
from pydantic import RootModel
from sqlalchemy.orm import Session, sessionmaker
from werkzeug.datastructures import MultiDict
from werkzeug.exceptions import NotFound
from controllers.common.fields import TextFileResponse
from controllers.common.schema import (
query_params_from_model,
register_response_schema_models,
register_schema_models,
)
from controllers.common.schema import query_params_from_model, register_response_schema_models, register_schema_models
from controllers.console import console_ns
from controllers.console.snippets.payloads import (
CreateSnippetPayload,
@@ -34,31 +30,27 @@ from controllers.console.wraps import (
with_current_tenant_id,
with_current_user,
)
from core.plugin.entities.plugin import PluginDependency
from extensions.ext_database import db
from fields.base import ResponseModel
from fields.snippet_fields import snippet_fields, snippet_list_fields
from libs.helper import to_timestamp
from fields.snippet_fields import snippet_fields, snippet_list_fields, snippet_pagination_fields
from libs.login import login_required
from models import Account
from models.snippet import SnippetType
from services.snippet_dsl_service import ImportStatus, SnippetDslService
from services.app_dsl_service import ImportStatus
from services.snippet_dsl_service import SnippetDslService
from services.snippet_service import SnippetService
logger = logging.getLogger(__name__)
_TAG_IDS_BRACKET_PATTERN = re.compile(r"^tag_ids\[(\d+)\]$")
_CREATOR_IDS_BRACKET_PATTERN = re.compile(r"^creator_ids\[(\d+)\]$")
class SnippetImportResponse(ResponseModel):
id: str
status: ImportStatus
snippet_id: str | None
current_dsl_version: str
imported_dsl_version: str
error: str
class SnippetImportResponse(RootModel[dict[str, Any]]):
root: dict[str, Any]
class SnippetDependencyCheckResponse(ResponseModel):
leaked_dependencies: list[PluginDependency]
class SnippetDependencyCheckResponse(RootModel[dict[str, Any]]):
root: dict[str, Any]
class SnippetUseCountResponse(ResponseModel):
@@ -66,90 +58,36 @@ class SnippetUseCountResponse(ResponseModel):
use_count: int
class SnippetTagResponse(ResponseModel):
id: str
name: str
type: str
class SnippetAccountResponse(ResponseModel):
id: str
name: str
email: str
class SnippetListItemResponse(ResponseModel):
id: str
name: str
description: str | None
type: SnippetType
version: int
use_count: int
is_published: bool
icon_info: dict[str, Any] | None
tags: list[SnippetTagResponse]
created_by: str | None
author_name: str | None
created_at: int
updated_by: str | None
updated_at: int
@field_validator("created_at", "updated_at", mode="before")
@classmethod
def _normalize_timestamp(cls, value: datetime | int | None) -> int:
timestamp = to_timestamp(value)
if timestamp is None:
raise ValueError("timestamp is required")
return timestamp
class SnippetResponse(ResponseModel):
id: str
name: str
description: str | None
type: SnippetType
version: int
use_count: int
is_published: bool
icon_info: dict[str, Any] | None
graph: dict[str, Any] = PydanticField(validation_alias="graph_dict")
input_fields: list[dict[str, Any]] = PydanticField(validation_alias="input_fields_list")
tags: list[SnippetTagResponse]
created_by: SnippetAccountResponse | None = PydanticField(validation_alias="created_by_account")
created_at: int
updated_by: SnippetAccountResponse | None = PydanticField(validation_alias="updated_by_account")
updated_at: int
@field_validator("created_at", "updated_at", mode="before")
@classmethod
def _normalize_timestamp(cls, value: datetime | int | None) -> int:
timestamp = to_timestamp(value)
if timestamp is None:
raise ValueError("timestamp is required")
return timestamp
class SnippetPaginationResponse(ResponseModel):
data: list[SnippetListItemResponse]
page: int
limit: int
total: int
has_more: bool
def _snippet_service() -> SnippetService:
return SnippetService(sessionmaker(bind=db.engine, expire_on_commit=False))
def _snippet_list_query_from_request() -> SnippetListQuery:
query_data: dict[str, str | list[str]] = dict(request.args.to_dict())
query_data["tag_ids"] = request.args.getlist("tag_ids")
def _normalize_snippet_list_query_args(query_args: MultiDict[str, str]) -> dict[str, str | list[str]]:
normalized: dict[str, str | list[str]] = {}
indexed_tag_ids: list[tuple[int, str]] = []
indexed_creator_ids: list[tuple[int, str]] = []
creator_ids = request.args.getlist("creators") or request.args.getlist("creator_ids")
if creator_ids:
query_data["creators"] = creator_ids
for key in query_args:
match = _TAG_IDS_BRACKET_PATTERN.fullmatch(key)
if match:
indexed_tag_ids.extend((int(match.group(1)), value) for value in query_args.getlist(key))
continue
return SnippetListQuery.model_validate(query_data)
match = _CREATOR_IDS_BRACKET_PATTERN.fullmatch(key)
if match:
indexed_creator_ids.extend((int(match.group(1)), value) for value in query_args.getlist(key))
continue
value = query_args.get(key)
if value is not None:
normalized[key] = value
if indexed_tag_ids:
normalized["tag_ids"] = [value for _, value in sorted(indexed_tag_ids)]
if indexed_creator_ids:
normalized["creators"] = [value for _, value in sorted(indexed_creator_ids)]
return normalized
# Register Pydantic models with Swagger
@@ -167,24 +105,26 @@ register_response_schema_models(
SnippetImportResponse,
SnippetDependencyCheckResponse,
SnippetUseCountResponse,
SnippetListItemResponse,
SnippetResponse,
SnippetPaginationResponse,
)
# Create namespace models for marshaling
snippet_model = console_ns.model("Snippet", snippet_fields)
snippet_list_model = console_ns.model("SnippetList", snippet_list_fields)
snippet_pagination_model = console_ns.model("SnippetPagination", snippet_pagination_fields)
@console_ns.route("/workspaces/current/customized-snippets")
class CustomizedSnippetsApi(Resource):
@console_ns.doc("list_customized_snippets")
@console_ns.doc(params=query_params_from_model(SnippetListQuery))
@console_ns.response(200, "Snippets retrieved successfully", console_ns.models[SnippetPaginationResponse.__name__])
@console_ns.response(200, "Snippets retrieved successfully", snippet_pagination_model)
@setup_required
@login_required
@account_initialization_required
@with_current_tenant_id
def get(self, current_tenant_id: str):
"""List customized snippets with pagination and search."""
query = _snippet_list_query_from_request()
query = SnippetListQuery.model_validate(_normalize_snippet_list_query_args(request.args))
snippet_service = _snippet_service()
snippets, total, has_more = snippet_service.get_snippets(
@@ -208,7 +148,7 @@ class CustomizedSnippetsApi(Resource):
@console_ns.doc("create_customized_snippet")
@console_ns.expect(console_ns.models.get(CreateSnippetPayload.__name__))
@console_ns.response(201, "Snippet created successfully", console_ns.models[SnippetResponse.__name__])
@console_ns.response(201, "Snippet created successfully", snippet_model)
@console_ns.response(400, "Invalid request")
@setup_required
@login_required
@@ -251,7 +191,7 @@ class CustomizedSnippetsApi(Resource):
@console_ns.route("/workspaces/current/customized-snippets/<uuid:snippet_id>")
class CustomizedSnippetDetailApi(Resource):
@console_ns.doc("get_customized_snippet")
@console_ns.response(200, "Snippet retrieved successfully", console_ns.models[SnippetResponse.__name__])
@console_ns.response(200, "Snippet retrieved successfully", snippet_model)
@console_ns.response(404, "Snippet not found")
@setup_required
@login_required
@@ -272,7 +212,7 @@ class CustomizedSnippetDetailApi(Resource):
@console_ns.doc("update_customized_snippet")
@console_ns.expect(console_ns.models.get(UpdateSnippetPayload.__name__))
@console_ns.response(200, "Snippet updated successfully", console_ns.models[SnippetResponse.__name__])
@console_ns.response(200, "Snippet updated successfully", snippet_model)
@console_ns.response(400, "Invalid request")
@console_ns.response(404, "Snippet not found")
@setup_required
@@ -515,6 +455,9 @@ class CustomizedSnippetUseCountIncrementApi(Resource):
@login_required
@account_initialization_required
@edit_permission_required
@rbac_permission_required(
RBACResourceScope.WORKSPACE, RBACPermission.SNIPPETS_CREATE_AND_MODIFY, resource_required=False
)
@with_current_tenant_id
def post(self, current_tenant_id: str, snippet_id: str):
"""Increment snippet use count when it is inserted into a workflow."""
@@ -11,12 +11,7 @@ from werkzeug.exceptions import Forbidden
from configs import dify_config
from controllers.common.fields import BinaryFileResponse, RedirectResponse, SimpleResultResponse
from controllers.common.schema import (
query_params_from_model,
query_params_from_request,
register_response_schema_models,
register_schema_models,
)
from controllers.common.schema import query_params_from_model, register_response_schema_models, register_schema_models
from controllers.console import console_ns
from controllers.console.wraps import (
RBACPermission,
@@ -450,17 +445,16 @@ class ToolBuiltinProviderGetCredentialsApi(Resource):
def get(self, tenant_id: str, user: Account, provider: str):
# Optional list of credential IDs to include even if visibility would hide them
# (used when a workflow/agent node still references another member's only_me credential).
query = query_params_from_request(
BuiltinCredentialListQuery,
list_fields=("include_credential_ids",),
)
include_credential_ids = request.args.getlist("include_credential_ids") or [
s for s in (request.args.get("include_credential_ids") or "").split(",") if s
]
return jsonable_encoder(
BuiltinToolManageService.get_builtin_tool_provider_credentials(
tenant_id=tenant_id,
provider_name=provider,
user=user,
include_credential_ids=query.include_credential_ids or None,
include_credential_ids=include_credential_ids or None,
)
)
@@ -1055,17 +1049,16 @@ class ToolBuiltinProviderGetCredentialInfoApi(Resource):
@with_current_user
@with_current_tenant_id
def get(self, tenant_id: str, user: Account, provider: str):
query = query_params_from_request(
BuiltinCredentialListQuery,
list_fields=("include_credential_ids",),
)
include_credential_ids = request.args.getlist("include_credential_ids") or [
s for s in (request.args.get("include_credential_ids") or "").split(",") if s
]
return jsonable_encoder(
BuiltinToolManageService.get_builtin_tool_provider_credential_info(
tenant_id=tenant_id,
provider=provider,
user=user,
include_credential_ids=query.include_credential_ids or None,
include_credential_ids=include_credential_ids or None,
)
)
@@ -1,5 +1,5 @@
import logging
from typing import Any, Literal
from typing import Any
from flask import make_response, redirect, request
from flask_restx import Resource
@@ -11,19 +11,11 @@ from configs import dify_config
from controllers.common.errors import NotFoundError
from controllers.common.fields import BinaryFileResponse, RedirectResponse, SimpleResultResponse
from controllers.common.schema import register_response_schema_models, register_schema_models
from core.entities.parameter_entities import AppSelectorScope, ModelSelectorScope, ToolSelectorScope
from core.plugin.entities.plugin_daemon import CredentialType
from core.plugin.impl.oauth import OAuthHandler
from core.tools.entities.common_entities import I18nObject
from core.trigger.entities.api_entities import (
SubscriptionBuilderApiEntity,
TriggerProviderApiEntity,
TriggerProviderSubscriptionApiEntity,
)
from core.trigger.entities.entities import RequestLog, SubscriptionBuilderUpdater
from core.trigger.entities.entities import SubscriptionBuilderUpdater
from core.trigger.trigger_manager import TriggerManager
from extensions.ext_database import db
from fields.base import ResponseModel
from graphon.model_runtime.utils.encoders import jsonable_encoder
from libs.login import login_required
from models.account import Account
@@ -78,41 +70,14 @@ class TriggerOAuthClientPayload(BaseModel):
class TriggerOAuthAuthorizeResponse(BaseModel):
authorization_url: str
subscription_builder_id: str
subscription_builder: SubscriptionBuilderApiEntity
class TriggerProviderConfigOptionResponse(BaseModel):
value: str = Field(..., description="The value of the option")
label: I18nObject = Field(..., description="The label of the option")
class TriggerProviderConfigResponse(BaseModel):
type: Literal[
"secret-input",
"text-input",
"select",
"boolean",
"app-selector",
"model-selector",
"array[tools]",
] = Field(..., description="The type of the credentials")
name: str = Field(..., description="The name of the credentials")
scope: AppSelectorScope | ModelSelectorScope | ToolSelectorScope | None = None
required: bool = False
default: int | str | float | bool | None = None
options: list[TriggerProviderConfigOptionResponse] | None = None
multiple: bool = False
label: I18nObject | None = None
help: I18nObject | None = None
url: str | None = None
placeholder: I18nObject | None = None
subscription_builder: Any
class TriggerOAuthClientResponse(BaseModel):
configured: bool
system_configured: bool
custom_configured: bool
oauth_client_schema: list[TriggerProviderConfigResponse]
oauth_client_schema: Any
custom_enabled: bool
redirect_uri: str
params: dict[str, Any]
@@ -122,26 +87,6 @@ class TriggerProviderOpaqueResponse(RootModel[Any]):
root: Any
class TriggerProviderListResponse(RootModel[list[TriggerProviderApiEntity]]):
root: list[TriggerProviderApiEntity]
class TriggerSubscriptionListResponse(RootModel[list[TriggerProviderSubscriptionApiEntity]]):
root: list[TriggerProviderSubscriptionApiEntity]
class TriggerSubscriptionBuilderCreateResponse(ResponseModel):
subscription_builder: SubscriptionBuilderApiEntity
class TriggerSubscriptionBuilderVerifyResponse(ResponseModel):
verified: bool
class TriggerSubscriptionBuilderLogsResponse(ResponseModel):
logs: list[RequestLog]
register_schema_models(
console_ns,
TriggerSubscriptionBuilderCreatePayload,
@@ -157,15 +102,6 @@ register_response_schema_models(
TriggerOAuthAuthorizeResponse,
TriggerOAuthClientResponse,
TriggerProviderOpaqueResponse,
TriggerProviderApiEntity,
TriggerProviderListResponse,
TriggerProviderSubscriptionApiEntity,
TriggerSubscriptionListResponse,
SubscriptionBuilderApiEntity,
TriggerSubscriptionBuilderCreateResponse,
TriggerSubscriptionBuilderVerifyResponse,
RequestLog,
TriggerSubscriptionBuilderLogsResponse,
)
@@ -182,7 +118,7 @@ class TriggerProviderIconApi(Resource):
@console_ns.route("/workspaces/current/triggers")
class TriggerProviderListApi(Resource):
@console_ns.response(200, "Success", console_ns.models[TriggerProviderListResponse.__name__])
@console_ns.response(200, "Success", console_ns.models[TriggerProviderOpaqueResponse.__name__])
@setup_required
@login_required
@account_initialization_required
@@ -194,7 +130,7 @@ class TriggerProviderListApi(Resource):
@console_ns.route("/workspaces/current/trigger-provider/<path:provider>/info")
class TriggerProviderInfoApi(Resource):
@console_ns.response(200, "Success", console_ns.models[TriggerProviderApiEntity.__name__])
@console_ns.response(200, "Success", console_ns.models[TriggerProviderOpaqueResponse.__name__])
@setup_required
@login_required
@account_initialization_required
@@ -206,7 +142,7 @@ class TriggerProviderInfoApi(Resource):
@console_ns.route("/workspaces/current/trigger-provider/<path:provider>/subscriptions/list")
class TriggerSubscriptionListApi(Resource):
@console_ns.response(200, "Success", console_ns.models[TriggerSubscriptionListResponse.__name__])
@console_ns.response(200, "Success", console_ns.models[TriggerProviderOpaqueResponse.__name__])
@setup_required
@login_required
@edit_permission_required
@@ -236,7 +172,7 @@ class TriggerSubscriptionListApi(Resource):
)
class TriggerSubscriptionBuilderCreateApi(Resource):
@console_ns.expect(console_ns.models[TriggerSubscriptionBuilderCreatePayload.__name__])
@console_ns.response(200, "Success", console_ns.models[TriggerSubscriptionBuilderCreateResponse.__name__])
@console_ns.response(200, "Success", console_ns.models[TriggerProviderOpaqueResponse.__name__])
@setup_required
@login_required
@edit_permission_required
@@ -266,7 +202,7 @@ class TriggerSubscriptionBuilderCreateApi(Resource):
"/workspaces/current/trigger-provider/<path:provider>/subscriptions/builder/<path:subscription_builder_id>",
)
class TriggerSubscriptionBuilderGetApi(Resource):
@console_ns.response(200, "Success", console_ns.models[SubscriptionBuilderApiEntity.__name__])
@console_ns.response(200, "Success", console_ns.models[TriggerProviderOpaqueResponse.__name__])
@setup_required
@login_required
@edit_permission_required
@@ -284,7 +220,7 @@ class TriggerSubscriptionBuilderGetApi(Resource):
)
class TriggerSubscriptionBuilderVerifyApi(Resource):
@console_ns.expect(console_ns.models[TriggerSubscriptionBuilderVerifyPayload.__name__])
@console_ns.response(200, "Success", console_ns.models[TriggerSubscriptionBuilderVerifyResponse.__name__])
@console_ns.response(200, "Success", console_ns.models[TriggerProviderOpaqueResponse.__name__])
@setup_required
@login_required
@edit_permission_required
@@ -317,7 +253,7 @@ class TriggerSubscriptionBuilderVerifyApi(Resource):
)
class TriggerSubscriptionBuilderUpdateApi(Resource):
@console_ns.expect(console_ns.models[TriggerSubscriptionBuilderUpdatePayload.__name__])
@console_ns.response(200, "Success", console_ns.models[SubscriptionBuilderApiEntity.__name__])
@console_ns.response(200, "Success", console_ns.models[TriggerProviderOpaqueResponse.__name__])
@setup_required
@login_required
@edit_permission_required
@@ -350,7 +286,7 @@ class TriggerSubscriptionBuilderUpdateApi(Resource):
"/workspaces/current/trigger-provider/<path:provider>/subscriptions/builder/logs/<path:subscription_builder_id>",
)
class TriggerSubscriptionBuilderLogsApi(Resource):
@console_ns.response(200, "Success", console_ns.models[TriggerSubscriptionBuilderLogsResponse.__name__])
@console_ns.response(200, "Success", console_ns.models[TriggerProviderOpaqueResponse.__name__])
@setup_required
@login_required
@edit_permission_required
@@ -753,7 +689,7 @@ class TriggerOAuthClientManageApi(Resource):
)
class TriggerSubscriptionVerifyApi(Resource):
@console_ns.expect(console_ns.models[TriggerSubscriptionBuilderVerifyPayload.__name__])
@console_ns.response(200, "Success", console_ns.models[TriggerSubscriptionBuilderVerifyResponse.__name__])
@console_ns.response(200, "Success", console_ns.models[TriggerProviderOpaqueResponse.__name__])
@setup_required
@login_required
@edit_permission_required
@@ -33,7 +33,6 @@ from extensions.ext_database import db
from fields.base import ResponseModel
from libs.helper import OptionalTimestampField, TimestampField, dump_response, to_timestamp
from libs.login import login_required
from libs.pagination import paginate_query
from models.account import Account, Tenant, TenantAccountJoin, TenantCustomConfigDict, TenantStatus
from services.account_service import TenantService
from services.billing_service import BillingService, SubscriptionPlan
@@ -103,7 +102,6 @@ class TenantListItemResponse(ResponseModel):
plan: str | None = None
status: str | None = None
created_at: int | None = None
last_opened_at: int | None = None
current: bool
@field_validator("plan", "status", mode="before")
@@ -115,9 +113,9 @@ class TenantListItemResponse(ResponseModel):
return value
return str(getattr(value, "value", value))
@field_validator("created_at", "last_opened_at", mode="before")
@field_validator("created_at", mode="before")
@classmethod
def _normalize_timestamp(cls, value: datetime | int | None):
def _normalize_created_at(cls, value: datetime | int | None):
return to_timestamp(value)
@@ -295,7 +293,7 @@ class WorkspaceListApi(Resource):
args = WorkspaceListQuery.model_validate(payload)
stmt = select(Tenant).order_by(Tenant.created_at.desc())
tenants = paginate_query(stmt, page=args.page, per_page=args.limit)
tenants = db.paginate(select=stmt, page=args.page, per_page=args.limit, error_out=False)
has_more = False
if tenants.has_next:
+1 -2
View File
@@ -14,12 +14,11 @@ api = ExternalApi(
files_ns = Namespace("files", description="File operations", path="/")
from . import agent_drive_archive, image_preview, tool_files, upload
from . import image_preview, tool_files, upload
api.add_namespace(files_ns)
__all__ = [
"agent_drive_archive",
"api",
"bp",
"files_ns",
@@ -1,67 +0,0 @@
from urllib.parse import quote
from flask import Response, request
from flask_restx import Resource
from pydantic import BaseModel, Field
from werkzeug.exceptions import Forbidden, NotFound
from controllers.common.file_response import enforce_download_for_html
from controllers.common.schema import register_schema_models
from controllers.files import files_ns
from models.agent import AgentDriveFileKind
from services.agent_drive_service import AgentDriveError, AgentDriveService
class AgentDriveArchiveMemberQuery(BaseModel):
tenant_id: str = Field(..., description="Tenant ID")
agent_id: str = Field(..., description="Agent ID")
key: str = Field(..., description="Virtual drive key")
archive_file_kind: AgentDriveFileKind = Field(..., description="Archive file kind")
archive_file_id: str = Field(..., description="Archive file id")
member_path: str = Field(..., description="Zip member path")
timestamp: str = Field(..., description="Unix timestamp")
nonce: str = Field(..., description="Random nonce")
sign: str = Field(..., description="HMAC signature")
as_attachment: bool = Field(default=False, description="Download as attachment")
register_schema_models(files_ns, AgentDriveArchiveMemberQuery)
@files_ns.route("/agent-drive/archive-member")
class AgentDriveArchiveMemberApi(Resource):
@files_ns.doc("get_agent_drive_archive_member")
@files_ns.doc(description="Download a lazily resolved Agent Skill archive member by signed parameters")
def get(self):
args = AgentDriveArchiveMemberQuery.model_validate(request.args.to_dict(flat=True))
if not AgentDriveService.verify_archive_member_signature(
tenant_id=args.tenant_id,
agent_id=args.agent_id,
key=args.key,
archive_file_kind=args.archive_file_kind,
archive_file_id=args.archive_file_id,
member_path=args.member_path,
timestamp=args.timestamp,
nonce=args.nonce,
sign=args.sign,
):
raise Forbidden("Invalid request.")
try:
payload, mime_type, filename = AgentDriveService().load_archive_member_for_signed_request(
tenant_id=args.tenant_id,
agent_id=args.agent_id,
key=args.key,
archive_file_kind=args.archive_file_kind,
archive_file_id=args.archive_file_id,
member_path=args.member_path,
)
except AgentDriveError as exc:
raise NotFound(exc.message) from exc
response = Response(payload, mimetype=mime_type, direct_passthrough=True, headers={})
response.headers["Content-Length"] = str(len(payload))
if args.as_attachment and filename:
encoded_filename = quote(filename)
response.headers["Content-Disposition"] = f"attachment; filename*=UTF-8''{encoded_filename}"
enforce_download_for_html(response, mime_type=mime_type, filename=filename, extension="")
return response
+1 -3
View File
@@ -28,7 +28,6 @@ class PluginUploadQuery(BaseModel):
sign: str = Field(..., description="HMAC signature")
tenant_id: str = Field(..., description="Tenant identifier")
user_id: str | None = Field(default=None, description="User identifier")
conversation_id: str | None = Field(default=None, description="Conversation identifier")
register_schema_models(files_ns, PluginUploadQuery)
@@ -93,7 +92,6 @@ class PluginUploadFileApi(Resource):
mimetype=mimetype,
tenant_id=tenant_id,
user_id=user.id,
conversation_id=args.conversation_id,
timestamp=timestamp,
nonce=nonce,
sign=sign,
@@ -107,7 +105,7 @@ class PluginUploadFileApi(Resource):
file_binary=file.stream.read(),
mimetype=mimetype,
filename=filename,
conversation_id=args.conversation_id,
conversation_id=None,
)
extension = guess_extension(tool_file.mimetype) or ".bin"
-4
View File
@@ -17,10 +17,8 @@ inner_api_ns = Namespace("inner_api", description="Internal API operations", pat
from . import mail as _mail
from . import runtime_credentials as _runtime_credentials
from .agent import tools as _agent_tools
from .app import dsl as _app_dsl
from .knowledge import retrieval as _knowledge_retrieval
from .plugin import agent_config as _agent_config
from .plugin import agent_drive as _agent_drive
from .plugin import plugin as _plugin
from .workspace import workspace as _workspace
@@ -28,9 +26,7 @@ from .workspace import workspace as _workspace
api.add_namespace(inner_api_ns)
__all__ = [
"_agent_config",
"_agent_drive",
"_agent_tools",
"_app_dsl",
"_knowledge_retrieval",
"_mail",
@@ -1 +0,0 @@
"""Agent backend inner API controllers."""
-70
View File
@@ -1,70 +0,0 @@
"""Inner API endpoint for Agent core tool invocation."""
from flask_restx import Resource
from pydantic import ValidationError
from sqlalchemy.orm import Session
from controllers.common.schema import register_response_schema_models, register_schema_models
from controllers.console.app.wraps import with_session
from controllers.inner_api import inner_api_ns
from controllers.inner_api.wraps import agent_inner_api_only
from libs.exception import BaseHTTPException
from services.agent_tool_inner_service import AgentToolInnerService
from services.entities.agent_tool_inner import AgentToolInvokeRequest, AgentToolInvokeResponse
from services.errors.agent_tool_inner import AgentToolInnerServiceError
class AgentToolInvokeHttpError(BaseHTTPException):
error_code = "agent_tool_invoke_failed"
description = "Agent tool invocation failed."
code = 500
def __init__(
self,
*,
error_code: str | None = None,
description: str | None = None,
status_code: int | None = None,
) -> None:
if error_code is not None:
self.error_code = error_code
if description is not None:
self.description = description
if status_code is not None:
self.code = status_code
super().__init__(self.description)
register_schema_models(inner_api_ns, AgentToolInvokeRequest)
register_response_schema_models(inner_api_ns, AgentToolInvokeResponse)
@inner_api_ns.route("/agent/tools/invoke")
class AgentToolInvokeApi(Resource):
"""Invoke one Agent tool through the API-owned core tool runtime path."""
@agent_inner_api_only
@inner_api_ns.doc("inner_agent_tool_invoke")
@inner_api_ns.expect(inner_api_ns.models[AgentToolInvokeRequest.__name__])
@inner_api_ns.response(200, "Tool invoked successfully", inner_api_ns.models[AgentToolInvokeResponse.__name__])
@with_session
def post(self, session: Session) -> dict[str, object]:
try:
payload = AgentToolInvokeRequest.model_validate(inner_api_ns.payload or {})
except ValidationError as exc:
raise AgentToolInvokeHttpError(
error_code="invalid_request",
description=str(exc),
status_code=400,
) from exc
try:
response = AgentToolInnerService().invoke(session=session, request=payload)
except AgentToolInnerServiceError as exc:
raise AgentToolInvokeHttpError(
error_code=exc.error_code,
description=exc.description,
status_code=exc.status_code,
) from exc
return response.model_dump(mode="json")
@@ -9,13 +9,12 @@ app/dataset validation remains in the service layer.
from flask_restx import Resource
from pydantic import ValidationError
from sqlalchemy.orm import Session
from controllers.common.schema import register_response_schema_models, register_schema_models
from controllers.console.app.wraps import with_session
from controllers.inner_api import inner_api_ns
from controllers.inner_api.wraps import plugin_inner_api_only
from core.workflow.nodes.knowledge_retrieval import exc as retrieval_exc
from extensions.ext_database import db
from libs.exception import BaseHTTPException
from services.entities.knowledge_retrieval_inner import InnerKnowledgeRetrieveRequest, InnerKnowledgeRetrieveResponse
from services.errors.knowledge_retrieval import ExternalKnowledgeRetrievalError, InnerKnowledgeRetrievalServiceError
@@ -71,8 +70,7 @@ class InnerKnowledgeRetrieveApi(Resource):
500: "Unexpected knowledge retrieval failure",
}
)
@with_session
def post(self, session: Session) -> dict[str, object]:
def post(self) -> dict[str, object]:
"""Validate the payload, run retrieval, and return workflow-style sources."""
try:
payload = InnerKnowledgeRetrieveRequest.model_validate(inner_api_ns.payload or {})
@@ -84,7 +82,7 @@ class InnerKnowledgeRetrieveApi(Resource):
) from exc
try:
response = InnerKnowledgeRetrievalService().retrieve(payload, session=session)
response = InnerKnowledgeRetrievalService().retrieve(payload, session=db.session)
except InnerKnowledgeRetrievalServiceError as exc:
raise InnerKnowledgeRetrievalHttpError(
error_code=exc.error_code,
@@ -1,243 +0,0 @@
"""Inner API for Agent Soul-backed config assets.
These endpoints are called by the dify-agent server with the inner API key.
They resolve the requested Agent config version directly from Agent Soul JSON
and never expose signed download URLs or drive-owned metadata.
"""
from __future__ import annotations
import io
from flask import request, send_file
from flask_restx import Resource
from pydantic import BaseModel, ValidationError
from controllers.console.wraps import setup_required
from controllers.inner_api import inner_api_ns
from controllers.inner_api.wraps import plugin_inner_api_only
from services.agent_config_service import (
AgentConfigService,
AgentConfigServiceError,
AgentConfigVersionKind,
ConfigPushPayload,
)
class _ConfigTargetQuery(BaseModel):
tenant_id: str
user_id: str | None = None
config_version_id: str
config_version_kind: AgentConfigVersionKind
class _ConfigMutationRequest(BaseModel):
tenant_id: str
user_id: str
config_version_id: str
config_version_kind: AgentConfigVersionKind
class _ConfigPushRequest(_ConfigMutationRequest):
files: list[dict] = []
skills: list[dict] = []
env_text: str | None = None
note: str | None = None
def to_payload(self) -> ConfigPushPayload:
return ConfigPushPayload.model_validate(
{
"files": self.files,
"skills": self.skills,
"env_text": self.env_text,
"note": self.note,
}
)
class _ConfigEnvUpdateRequest(_ConfigMutationRequest):
env_text: str
class _ConfigNoteUpdateRequest(_ConfigMutationRequest):
note: str
def _target_query_from_request() -> _ConfigTargetQuery:
return _ConfigTargetQuery.model_validate(
{
"tenant_id": request.args.get("tenant_id"),
"user_id": request.args.get("user_id"),
"config_version_id": request.args.get("config_version_id"),
"config_version_kind": request.args.get("config_version_kind"),
}
)
def _error_response(exc: AgentConfigServiceError) -> tuple[dict[str, str], int]:
return {"code": exc.code, "message": exc.message}, exc.status_code
@inner_api_ns.route("/agent-config/<string:agent_id>/manifest")
class AgentConfigManifestApi(Resource):
@setup_required
@plugin_inner_api_only
@inner_api_ns.doc("agent_config_manifest")
def get(self, agent_id: str):
try:
query = _target_query_from_request()
return AgentConfigService().manifest(
tenant_id=query.tenant_id,
agent_id=agent_id,
user_id=query.user_id,
config_version_id=query.config_version_id,
config_version_kind=query.config_version_kind,
)
except ValidationError as exc:
return {"code": "invalid_request", "message": str(exc)}, 400
except AgentConfigServiceError as exc:
return _error_response(exc)
@inner_api_ns.route("/agent-config/<string:agent_id>/skills/<string:name>/pull")
class AgentConfigSkillPullApi(Resource):
@setup_required
@plugin_inner_api_only
@inner_api_ns.doc("agent_config_skill_pull")
def get(self, agent_id: str, name: str):
try:
query = _target_query_from_request()
result = AgentConfigService().pull_skill(
tenant_id=query.tenant_id,
agent_id=agent_id,
user_id=query.user_id,
config_version_id=query.config_version_id,
config_version_kind=query.config_version_kind,
name=name,
)
return send_file(
io.BytesIO(result.payload),
mimetype=result.mime_type,
as_attachment=True,
download_name=result.filename,
)
except ValidationError as exc:
return {"code": "invalid_request", "message": str(exc)}, 400
except AgentConfigServiceError as exc:
return _error_response(exc)
@inner_api_ns.route("/agent-config/<string:agent_id>/skills/<string:name>/inspect")
class AgentConfigSkillInspectApi(Resource):
@setup_required
@plugin_inner_api_only
@inner_api_ns.doc("agent_config_skill_inspect")
def get(self, agent_id: str, name: str):
try:
query = _target_query_from_request()
return AgentConfigService().inspect_skill(
tenant_id=query.tenant_id,
agent_id=agent_id,
user_id=query.user_id,
config_version_id=query.config_version_id,
config_version_kind=query.config_version_kind,
name=name,
)
except ValidationError as exc:
return {"code": "invalid_request", "message": str(exc)}, 400
except AgentConfigServiceError as exc:
return _error_response(exc)
@inner_api_ns.route("/agent-config/<string:agent_id>/files/<string:name>/pull")
class AgentConfigFilePullApi(Resource):
@setup_required
@plugin_inner_api_only
@inner_api_ns.doc("agent_config_file_pull")
def get(self, agent_id: str, name: str):
try:
query = _target_query_from_request()
result = AgentConfigService().pull_file(
tenant_id=query.tenant_id,
agent_id=agent_id,
user_id=query.user_id,
config_version_id=query.config_version_id,
config_version_kind=query.config_version_kind,
name=name,
)
return send_file(
io.BytesIO(result.payload),
mimetype=result.mime_type,
as_attachment=True,
download_name=result.filename,
)
except ValidationError as exc:
return {"code": "invalid_request", "message": str(exc)}, 400
except AgentConfigServiceError as exc:
return _error_response(exc)
@inner_api_ns.route("/agent-config/<string:agent_id>/push")
class AgentConfigPushApi(Resource):
@setup_required
@plugin_inner_api_only
@inner_api_ns.doc("agent_config_push")
def post(self, agent_id: str):
try:
body = _ConfigPushRequest.model_validate(request.get_json(silent=True) or {})
return AgentConfigService().push(
tenant_id=body.tenant_id,
agent_id=agent_id,
user_id=body.user_id,
config_version_id=body.config_version_id,
config_version_kind=body.config_version_kind,
payload=body.to_payload(),
)
except ValidationError as exc:
return {"code": "invalid_request", "message": str(exc)}, 400
except AgentConfigServiceError as exc:
return _error_response(exc)
@inner_api_ns.route("/agent-config/<string:agent_id>/env")
class AgentConfigEnvApi(Resource):
@setup_required
@plugin_inner_api_only
@inner_api_ns.doc("agent_config_env")
def patch(self, agent_id: str):
try:
body = _ConfigEnvUpdateRequest.model_validate(request.get_json(silent=True) or {})
return AgentConfigService().update_env(
tenant_id=body.tenant_id,
agent_id=agent_id,
user_id=body.user_id,
config_version_id=body.config_version_id,
config_version_kind=body.config_version_kind,
env_text=body.env_text,
)
except ValidationError as exc:
return {"code": "invalid_request", "message": str(exc)}, 400
except AgentConfigServiceError as exc:
return _error_response(exc)
@inner_api_ns.route("/agent-config/<string:agent_id>/note")
class AgentConfigNoteApi(Resource):
@setup_required
@plugin_inner_api_only
@inner_api_ns.doc("agent_config_note")
def put(self, agent_id: str):
try:
body = _ConfigNoteUpdateRequest.model_validate(request.get_json(silent=True) or {})
return AgentConfigService().update_note(
tenant_id=body.tenant_id,
agent_id=agent_id,
user_id=body.user_id,
config_version_id=body.config_version_id,
config_version_kind=body.config_version_kind,
note=body.note,
)
except ValidationError as exc:
return {"code": "invalid_request", "message": str(exc)}, 400
except AgentConfigServiceError as exc:
return _error_response(exc)
+2 -9
View File
@@ -1,7 +1,5 @@
from flask_restx import Resource
from sqlalchemy.orm import Session
from controllers.console.app.wraps import with_session
from controllers.console.wraps import setup_required
from controllers.inner_api import inner_api_ns
from controllers.inner_api.plugin.wraps import get_user_tenant, plugin_data
@@ -238,12 +236,10 @@ class PluginInvokeToolApi(Resource):
404: "Service not available",
}
)
@with_session
def post(self, session: Session, user_model: Account | EndUser, tenant_model: Tenant, payload: RequestInvokeTool):
def post(self, user_model: Account | EndUser, tenant_model: Tenant, payload: RequestInvokeTool):
def generator():
return PluginToolBackwardsInvocation.convert_to_event_stream(
PluginToolBackwardsInvocation.invoke_tool(
session=session,
tenant_id=tenant_model.id,
user_id=user_model.id,
tool_type=ToolProviderType.value_of(payload.tool_type),
@@ -338,10 +334,8 @@ class PluginInvokeAppApi(Resource):
404: "Service not available",
}
)
@with_session
def post(self, session: Session, user_model: Account | EndUser, tenant_model: Tenant, payload: RequestInvokeApp):
def post(self, user_model: Account | EndUser, tenant_model: Tenant, payload: RequestInvokeApp):
response = PluginAppBackwardsInvocation.invoke_app(
session=session,
app_id=payload.app_id,
user_id=user_model.id,
tenant_id=tenant_model.id,
@@ -434,7 +428,6 @@ class PluginUploadFileRequestApi(Resource):
mimetype=payload.mimetype,
tenant_id=tenant_model.id,
user_id=user_model.id,
conversation_id=payload.conversation_id,
)
return BaseBackwardsInvocationResponse(data={"url": url}).model_dump()
-12
View File
@@ -95,15 +95,3 @@ def plugin_inner_api_only[**P, R](view: Callable[P, R]) -> Callable[P, R]:
return view(*args, **kwargs)
return decorated
def agent_inner_api_only[**P, R](view: Callable[P, R]) -> Callable[P, R]:
"""Temporary alias for agent-backend inner API callers.
Agent tool and knowledge calls currently share the same trusted
`dify-agent -> Dify API` transport credentials as the plugin inner bridge.
Keep the wrapper name agent-specific so the controller surface does not grow
more plugin-only semantics while auth settings stay shared.
"""
return plugin_inner_api_only(view)
+3 -2
View File
@@ -236,6 +236,7 @@ class MCPAppApi(Resource):
if not end_user and isinstance(mcp_request.root, mcp_types.InitializeRequest):
client_info = mcp_request.root.params.clientInfo
client_name = f"{client_info.name}@{client_info.version}"
end_user = self._create_end_user(client_name, app.tenant_id, app.id, mcp_server.id, session)
with sessionmaker(db.engine, expire_on_commit=False).begin() as create_session:
end_user = self._create_end_user(client_name, app.tenant_id, app.id, mcp_server.id, create_session)
return handle_mcp_request(session, app, mcp_request, user_input_form, mcp_server, end_user, request_id)
return handle_mcp_request(app, mcp_request, user_input_form, mcp_server, end_user, request_id)
+14
View File
@@ -34,6 +34,7 @@ class OpenApiErrorCode(StrEnum):
# transport-generic (resolved from HTTP status for plain werkzeug raises)
BAD_REQUEST = "bad_request"
UNAUTHORIZED = "unauthorized"
TOKEN_EXPIRED = "token_expired"
FORBIDDEN = "forbidden"
NOT_FOUND = "not_found"
METHOD_NOT_ALLOWED = "method_not_allowed"
@@ -223,6 +224,19 @@ class OpenApiErrorFormatter:
return isinstance(part, (str, int)) and not isinstance(part, bool)
class InvalidBearer(OpenApiError): # noqa: N818
code = 401
error_code = OpenApiErrorCode.UNAUTHORIZED
description = "Invalid or unknown bearer token."
class SessionExpired(OpenApiError): # noqa: N818
code = 401
error_code = OpenApiErrorCode.TOKEN_EXPIRED
description = "Your session has expired."
hint = "Re-authenticate to continue (e.g. re-run your login command)."
class FilenameNotExists(OpenApiError): # noqa: N818
code = 400
error_code = OpenApiErrorCode.FILENAME_NOT_EXISTS

Some files were not shown because too many files have changed in this diff Show More