Compare commits
80
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
15c1b63474 | ||
|
|
14c50a9f09 | ||
|
|
ba5b26be03 | ||
|
|
922ed4d67d | ||
|
|
a8a83a357c | ||
|
|
302b50c4fb | ||
|
|
43254c1ded | ||
|
|
2beafdc457 | ||
|
|
0198e32447 | ||
|
|
b8f91d0e61 | ||
|
|
452dff5c37 | ||
|
|
b56ac4af4d | ||
|
|
2a0661a769 | ||
|
|
00fe96325d | ||
|
|
a3c18c561e | ||
|
|
1835e8ef31 | ||
|
|
2b8e40af32 | ||
|
|
625ed2a0e7 | ||
|
|
45711eb6dd | ||
|
|
fd703739b5 | ||
|
|
82c8741c73 | ||
|
|
78023bf50d | ||
|
|
0ad985515b | ||
|
|
da2177c7b2 | ||
|
|
8d293023cc | ||
|
|
6736f4005b | ||
|
|
bd8f0104d6 | ||
|
|
018fe7a9d3 | ||
|
|
d5788cc019 | ||
|
|
4da4fa72cd | ||
|
|
8853ed99a5 | ||
|
|
34a5ca191b | ||
|
|
1271cfcf3f | ||
|
|
2a0b1a8a63 | ||
|
|
4a2e92f43f | ||
|
|
84889b88ec | ||
|
|
6f034d2c93 | ||
|
|
f91ea49414 | ||
|
|
62db37c403 | ||
|
|
de07730548 | ||
|
|
9b2aa8b216 | ||
|
|
8ee3a7eabc | ||
|
|
0f9fffb7a2 | ||
|
|
187501f53e | ||
|
|
03fad2e041 | ||
|
|
acb5ee29e1 | ||
|
|
c5aadfe557 | ||
|
|
c76ff4c38c | ||
|
|
4b355b7039 | ||
|
|
a3654d6a7d | ||
|
|
855464e7b4 | ||
|
|
05a25943e7 | ||
|
|
c5d471dbdd | ||
|
|
ba449890a9 | ||
|
|
072953a83a | ||
|
|
7b95e2a75f | ||
|
|
01efc6eecb | ||
|
|
a4c7261bf9 | ||
|
|
0aa04f610e | ||
|
|
66afa3bf4a | ||
|
|
c0b991c5f9 | ||
|
|
d2aea76dec | ||
|
|
bc21972878 | ||
|
|
389a7608bf | ||
|
|
3abbab4798 | ||
|
|
84f3c410c4 | ||
|
|
04f9026724 | ||
|
|
3f0f57c594 | ||
|
|
79a5b31bc6 | ||
|
|
f35655a5bb | ||
|
|
97489d5a44 | ||
|
|
e9bd8741f4 | ||
|
|
33acaa558e | ||
|
|
b38caf3cdb | ||
|
|
cb2b36f1aa | ||
|
|
2d15743b96 | ||
|
|
d5e5227c6e | ||
|
|
9b432d0d29 | ||
|
|
5216ee1d20 | ||
|
|
ba8ce362d7 |
@@ -32,12 +32,11 @@ Keep this skill focused on Cucumber, Playwright, and package-level E2E guidance.
|
||||
- `e2e/` uses Cucumber for scenarios and Playwright as the browser layer.
|
||||
- `DifyWorld` is the per-scenario context object. Type `this` as `DifyWorld` and use `async function`, not arrow functions.
|
||||
- Keep glue organized by capability under `e2e/features/step-definitions/`; use `common/` only for broadly reusable steps.
|
||||
- Browser session behavior comes from `features/support/hooks.ts`:
|
||||
- default: authenticated session with shared storage state
|
||||
- `@unauthenticated`: clean browser context
|
||||
- `@authenticated`: readability/selective-run tag only unless implementation changes
|
||||
- `@fresh`: only for `e2e:full*` flows
|
||||
- Treat `e2e/AGENTS.md`, `features/support/hooks.ts`, and the Cucumber configuration as the owners of current session and tag semantics. Verify them when behavior depends on session state instead of copying a tag inventory into this skill.
|
||||
- Do not import Playwright Test runner patterns that bypass the current Cucumber + `DifyWorld` architecture unless the task is explicitly about changing that architecture.
|
||||
- Perform the behavior under test through Playwright. APIs are allowed for setup, seed preparation, persistence polling, and cleanup, but ordinary Console JSON and representable multipart operations must use the scenario- or process-owned generated oRPC client with request and response validation enabled. Keep the setup/cleanup API identity independent from an unauthenticated or logged-out behavior browser.
|
||||
- Consume generated operations directly. Do not add one-to-one API wrappers, handwritten endpoint URLs, response DTO casts, duplicate schemas, global mutable clients, or TanStack Query caching in Cucumber. Keep helpers only for real fixture construction, multi-operation orchestration, invariants, polling, derived test views, or protocol adapters.
|
||||
- Keep SSE, binary, redirect-only, external-service, and readiness exceptions centralized under their protocol owner. A contract mismatch must fail and be fixed at the backend schema owner followed by regeneration; never weaken validation to make E2E pass.
|
||||
|
||||
## Workflow
|
||||
|
||||
@@ -66,7 +65,7 @@ Keep this skill focused on Cucumber, Playwright, and package-level E2E guidance.
|
||||
- If a product element has real user-facing semantics but no accessible name, prefer fixing that accessible contract over adding a test id.
|
||||
5. Validate narrowly.
|
||||
- Run the narrowest tagged scenario or flow that exercises the change.
|
||||
- Run `vpr lint --fix --quiet` from the repository root and `pnpm -C e2e type-check`.
|
||||
- Run the package-required static checks documented in `e2e/AGENTS.md`.
|
||||
- Broaden verification only when the change affects hooks, tags, setup, or shared step semantics.
|
||||
|
||||
## Review Checklist
|
||||
@@ -77,6 +76,8 @@ Keep this skill focused on Cucumber, Playwright, and package-level E2E guidance.
|
||||
- Are locators user-facing and assertions web-first?
|
||||
- Does the change introduce hidden coupling across scenarios, tags, or instance state?
|
||||
- Does it document or implement behavior that differs from the real hooks or configuration?
|
||||
- Does setup/cleanup use the generated client directly, with any remaining helper owning more than a one-to-one endpoint forward?
|
||||
- Is every raw HTTP call a documented protocol or infrastructure exception rather than an ordinary Console operation?
|
||||
|
||||
Lead findings with correctness, flake risk, and architecture drift.
|
||||
|
||||
|
||||
+17
-44
@@ -8,7 +8,6 @@
|
||||
|
||||
# Lint bulk suppression baselines.
|
||||
/oxlint-suppressions.json
|
||||
/eslint-suppressions.json
|
||||
|
||||
# CODEOWNERS file
|
||||
/.github/CODEOWNERS @laipz8200 @crazywoola
|
||||
@@ -33,31 +32,9 @@
|
||||
# Backend (default owner, more specific rules below will override)
|
||||
/api/ @QuantumGhost
|
||||
|
||||
# Backend - MCP
|
||||
/api/core/mcp/ @Nov1c444
|
||||
/api/core/entities/mcp_provider.py @Nov1c444
|
||||
/api/services/tools/mcp_tools_manage_service.py @Nov1c444
|
||||
/api/controllers/mcp/ @Nov1c444
|
||||
/api/controllers/console/app/mcp_server.py @Nov1c444
|
||||
|
||||
# Backend - Tests
|
||||
/api/tests/ @laipz8200 @QuantumGhost
|
||||
|
||||
/api/tests/**/*mcp* @Nov1c444
|
||||
|
||||
# Backend - Workflow - Engine (Core graph execution engine)
|
||||
/api/core/workflow/graph_engine/ @laipz8200 @QuantumGhost
|
||||
/api/core/workflow/runtime/ @laipz8200 @QuantumGhost
|
||||
/api/core/workflow/graph/ @laipz8200 @QuantumGhost
|
||||
/api/core/workflow/graph_events/ @laipz8200 @QuantumGhost
|
||||
/api/core/workflow/node_events/ @laipz8200 @QuantumGhost
|
||||
|
||||
# Backend - Workflow - Nodes (Agent, Iteration, Loop, LLM)
|
||||
/api/core/workflow/nodes/agent/ @Nov1c444
|
||||
/api/core/workflow/nodes/iteration/ @Nov1c444
|
||||
/api/core/workflow/nodes/loop/ @Nov1c444
|
||||
/api/core/workflow/nodes/llm/ @Nov1c444
|
||||
|
||||
# Backend - RAG (Retrieval Augmented Generation)
|
||||
/api/core/rag/ @JohnJyong
|
||||
/api/services/rag_pipeline/ @JohnJyong
|
||||
@@ -111,7 +88,6 @@
|
||||
/api/core/app/layers/trigger_post_layer.py @CourTeous33
|
||||
/api/services/trigger/ @CourTeous33
|
||||
/api/models/trigger.py @CourTeous33
|
||||
/api/fields/workflow_trigger_fields.py @CourTeous33
|
||||
/api/repositories/workflow_trigger_log_repository.py @CourTeous33
|
||||
/api/repositories/sqlalchemy_workflow_trigger_log_repository.py @CourTeous33
|
||||
/api/libs/schedule_utils.py @CourTeous33
|
||||
@@ -136,11 +112,11 @@
|
||||
/api/controllers/console/billing/ @hj24 @zyssyz123
|
||||
|
||||
# Backend - Enterprise
|
||||
/api/configs/enterprise/ @GarfieldDai @GareArc
|
||||
/api/services/enterprise/ @GarfieldDai @GareArc
|
||||
/api/services/feature_service.py @GarfieldDai @GareArc
|
||||
/api/controllers/console/feature.py @GarfieldDai @GareArc
|
||||
/api/controllers/web/feature.py @GarfieldDai @GareArc
|
||||
/api/configs/enterprise/ @GareArc
|
||||
/api/services/enterprise/ @GareArc
|
||||
/api/services/feature_service.py @GareArc
|
||||
/api/controllers/console/feature.py @GareArc
|
||||
/api/controllers/web/feature.py @GareArc
|
||||
|
||||
# Backend - Database Migrations
|
||||
/api/migrations/ @snakevash @laipz8200 @MRZHUH
|
||||
@@ -153,7 +129,6 @@
|
||||
|
||||
# Frontend - Platform and Features
|
||||
/web/config/ @lyzno1
|
||||
/web/contract/ @lyzno1
|
||||
/web/env.ts @lyzno1
|
||||
/web/features/ @lyzno1
|
||||
/web/hooks/ @lyzno1
|
||||
@@ -212,7 +187,6 @@
|
||||
/web/app/components/rag-pipeline/store/ @iamjoel @zxhlyh
|
||||
|
||||
# Frontend - RAG - Documents List
|
||||
/web/app/components/datasets/documents/list.tsx @iamjoel @WTW0313
|
||||
/web/app/components/datasets/documents/create-from-pipeline/ @iamjoel @WTW0313
|
||||
|
||||
# Frontend - RAG - Segments List
|
||||
@@ -231,22 +205,22 @@
|
||||
/web/app/components/plugins/marketplace/ @iamjoel @Yessenia-d
|
||||
|
||||
# Frontend - Login and Registration
|
||||
/web/app/signin/ @douxc @iamjoel
|
||||
/web/app/signup/ @douxc @iamjoel
|
||||
/web/app/reset-password/ @douxc @iamjoel
|
||||
/web/app/install/ @douxc @iamjoel
|
||||
/web/app/init/ @douxc @iamjoel
|
||||
/web/app/forgot-password/ @douxc @iamjoel
|
||||
/web/app/account/ @douxc @iamjoel
|
||||
/web/app/signin/ @iamjoel
|
||||
/web/app/signup/ @iamjoel
|
||||
/web/app/reset-password/ @iamjoel
|
||||
/web/app/install/ @iamjoel
|
||||
/web/app/init/ @iamjoel
|
||||
/web/app/forgot-password/ @iamjoel
|
||||
/web/app/account/ @iamjoel
|
||||
|
||||
# Frontend - Service Authentication
|
||||
/web/service/base.ts @douxc @iamjoel
|
||||
/web/service/base.ts @iamjoel
|
||||
|
||||
# Frontend - WebApp Authentication and Access Control
|
||||
/web/app/(shareLayout)/components/ @douxc @iamjoel
|
||||
/web/app/(shareLayout)/webapp-signin/ @douxc @iamjoel
|
||||
/web/app/(shareLayout)/webapp-reset-password/ @douxc @iamjoel
|
||||
/web/app/components/app/app-access-control/ @douxc @iamjoel
|
||||
/web/app/(shareLayout)/components/ @iamjoel
|
||||
/web/app/(shareLayout)/webapp-signin/ @iamjoel
|
||||
/web/app/(shareLayout)/webapp-reset-password/ @iamjoel
|
||||
/web/app/components/app/app-access-control/ @iamjoel
|
||||
|
||||
# Frontend - Explore Page
|
||||
/web/app/components/explore/ @CodingOnStar @iamjoel
|
||||
@@ -265,7 +239,6 @@
|
||||
/web/app/components/base/**/*.spec.tsx @hyoban @CodingOnStar
|
||||
|
||||
# Frontend - Utils and Hooks
|
||||
/web/utils/classnames.ts @iamjoel @zxhlyh
|
||||
/web/utils/time.ts @iamjoel @zxhlyh
|
||||
/web/utils/format.ts @iamjoel @zxhlyh
|
||||
/web/utils/clipboard.ts @iamjoel @zxhlyh
|
||||
|
||||
@@ -335,6 +335,8 @@ jobs:
|
||||
- check-changes
|
||||
if: needs.pre_job.outputs.should_skip != 'true' && needs.check-changes.outputs.e2e-changed == 'true'
|
||||
uses: ./.github/workflows/web-e2e.yml
|
||||
with:
|
||||
run-external-runtime: false
|
||||
secrets: inherit
|
||||
|
||||
web-e2e-skip:
|
||||
|
||||
@@ -26,19 +26,30 @@ jobs:
|
||||
external_e2e:
|
||||
- 'e2e/features/agent-v2/**'
|
||||
- 'e2e/features/step-definitions/agent-v2/**'
|
||||
- 'e2e/features/step-definitions/common/**'
|
||||
- 'e2e/features/support/**'
|
||||
- 'e2e/fixtures/auth.ts'
|
||||
- 'e2e/fixtures/test-materials/**'
|
||||
- 'e2e/scripts/**'
|
||||
- 'e2e/support/**'
|
||||
- 'e2e/cucumber.config.ts'
|
||||
- 'e2e/package.json'
|
||||
- 'e2e/test-env.ts'
|
||||
- 'e2e/tsconfig.json'
|
||||
- 'e2e/tsx-register.js'
|
||||
- 'package.json'
|
||||
- 'pnpm-lock.yaml'
|
||||
- '.nvmrc'
|
||||
- '.github/workflows/post-merge.yml'
|
||||
- '.github/workflows/web-e2e.yml'
|
||||
- '.github/actions/setup-web/**'
|
||||
- 'docker/docker-compose.middleware.yaml'
|
||||
- 'docker/envs/middleware.env.example'
|
||||
- 'dify-agent/**'
|
||||
- 'dify-agent-runtime/**'
|
||||
- 'api/pyproject.toml'
|
||||
- 'api/uv.lock'
|
||||
- 'api/tests/integration_tests/.env.example'
|
||||
- 'api/clients/agent_backend/**'
|
||||
- 'api/core/app/apps/agent_app/**'
|
||||
- 'api/core/workflow/nodes/agent_v2/**'
|
||||
@@ -48,8 +59,13 @@ jobs:
|
||||
- 'api/services/plugin/**'
|
||||
- 'api/core/tools/**'
|
||||
- 'api/services/tools/**'
|
||||
- 'packages/contracts/package.json'
|
||||
- 'packages/contracts/generated/api/console/agent/**'
|
||||
- 'packages/contracts/generated/api/console/apps/**'
|
||||
- 'packages/contracts/generated/api/console/datasets/**'
|
||||
- 'packages/contracts/generated/api/console/orpc.gen.ts'
|
||||
- 'packages/contracts/generated/api/console/workspaces/**'
|
||||
- 'packages/contracts/generated/api/service/**'
|
||||
- 'web/features/agent-v2/**'
|
||||
- 'web/app/(commonLayout)/agents/**'
|
||||
- 'web/app/(commonLayout)/@detailSidebar/agents/**'
|
||||
|
||||
@@ -4,9 +4,9 @@ on:
|
||||
workflow_call:
|
||||
inputs:
|
||||
run-external-runtime:
|
||||
required: false
|
||||
description: Run only the prepared and external runtime suite instead of the core suites.
|
||||
required: true
|
||||
type: boolean
|
||||
default: false
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
@@ -46,6 +46,7 @@ jobs:
|
||||
run: uv sync --project api --dev
|
||||
|
||||
- name: Run E2E support unit tests
|
||||
if: ${{ !inputs.run-external-runtime }}
|
||||
working-directory: ./e2e
|
||||
run: vp run test:unit
|
||||
|
||||
@@ -54,6 +55,7 @@ jobs:
|
||||
run: vp run e2e:install
|
||||
|
||||
- name: Run isolated source-api and built-web Cucumber E2E tests
|
||||
if: ${{ !inputs.run-external-runtime }}
|
||||
working-directory: ./e2e
|
||||
env:
|
||||
E2E_ADMIN_EMAIL: e2e-admin@example.com
|
||||
@@ -64,7 +66,7 @@ jobs:
|
||||
run: vp run e2e:full
|
||||
|
||||
- name: Preserve Chromium E2E report and logs
|
||||
if: ${{ !cancelled() }}
|
||||
if: ${{ !cancelled() && !inputs.run-external-runtime }}
|
||||
run: |
|
||||
if [[ -d e2e/cucumber-report ]]; then
|
||||
mv e2e/cucumber-report e2e/cucumber-report-non-external
|
||||
@@ -74,6 +76,7 @@ jobs:
|
||||
fi
|
||||
|
||||
- name: Run WebKit keyboard and browser smoke tests
|
||||
if: ${{ !inputs.run-external-runtime }}
|
||||
working-directory: ./e2e
|
||||
env:
|
||||
E2E_ADMIN_EMAIL: e2e-admin@example.com
|
||||
@@ -99,7 +102,7 @@ jobs:
|
||||
vp run e2e -- --tags '@browser-smoke'
|
||||
|
||||
- name: Preserve WebKit E2E report and logs
|
||||
if: ${{ !cancelled() }}
|
||||
if: ${{ !cancelled() && !inputs.run-external-runtime }}
|
||||
run: |
|
||||
if [[ -d e2e/cucumber-report ]]; then
|
||||
mv e2e/cucumber-report e2e/cucumber-report-webkit
|
||||
|
||||
@@ -71,7 +71,7 @@ Dify is an open-source LLM app development platform. Its intuitive interface com
|
||||
|
||||
<br/>
|
||||
|
||||
The easiest way to start the Dify server is through [Docker Compose](docker/docker-compose.yaml). Before running Dify with the following commands, make sure that [Docker](https://docs.docker.com/get-docker/) and [Docker Compose](https://docs.docker.com/compose/install/) are installed on your machine:
|
||||
The easiest way to start the Dify server is through [Docker Compose](docker/docker-compose.yaml). Before running Dify with the following commands, make sure that [Docker](https://docs.docker.com/get-docker/) and Docker Compose v2.24.0 or later are installed on your machine:
|
||||
|
||||
```bash
|
||||
cd dify
|
||||
|
||||
@@ -10,6 +10,7 @@ from extensions.ext_database import db
|
||||
from libs.login import current_account_with_tenant
|
||||
from models.dataset import Dataset
|
||||
from models.model import App
|
||||
from services.agent.roster_service import AgentRosterService
|
||||
from services.enterprise.rbac_service import RBACService
|
||||
|
||||
__all__ = ["RBACPermission", "RBACResourceScope", "enforce_rbac_access", "rbac_permission_required"]
|
||||
@@ -51,7 +52,7 @@ def enforce_rbac_access(
|
||||
check_resource_type = None if resource_type == RBACResourceScope.WORKSPACE else resource_type
|
||||
resource_id = None
|
||||
if resource_required and check_resource_type:
|
||||
resource_id = _extract_resource_id(resource_type, path_args)
|
||||
resource_id = _extract_resource_id(resource_type, tenant_id, path_args)
|
||||
if _is_resource_owned_by_current_user(tenant_id, account_id, resource_type, resource_id):
|
||||
return
|
||||
allowed = RBACService.CheckAccess.check(
|
||||
@@ -131,11 +132,14 @@ def _is_resource_owned_by_current_user(
|
||||
return False
|
||||
|
||||
|
||||
def _extract_resource_id(resource_type: RBACResourceScope, path_args: dict[str, object] | None = None) -> str:
|
||||
def _extract_resource_id(
|
||||
resource_type: RBACResourceScope, tenant_id: str, path_args: dict[str, object] | None = None
|
||||
) -> str:
|
||||
"""Extract the resource ID from matched path arguments.
|
||||
|
||||
Some legacy route classes use neutral names such as ``resource_id`` for
|
||||
app/dataset resources, and Agent App routes use ``agent_id`` as the app id.
|
||||
app/dataset resources, and Agent routes carry ``agent_id``, which is
|
||||
resolved to the App backing that Agent.
|
||||
Dataset endpoints behind a rag-pipeline route contain ``pipeline_id``
|
||||
instead of ``dataset_id``. In that case we look up the associated
|
||||
``Dataset`` row via ``Dataset.pipeline_id``.
|
||||
@@ -146,10 +150,19 @@ def _extract_resource_id(resource_type: RBACResourceScope, path_args: dict[str,
|
||||
matched_args = {**view_args, **(path_args or {})}
|
||||
|
||||
if resource_type == RBACResourceScope.APP:
|
||||
app_id = matched_args.get("app_id") or matched_args.get("agent_id") or matched_args.get("resource_id")
|
||||
if not app_id:
|
||||
raise ValueError("Missing app_id in request path")
|
||||
return str(app_id)
|
||||
app_id = matched_args.get("app_id")
|
||||
if app_id:
|
||||
return str(app_id)
|
||||
|
||||
agent_id = matched_args.get("agent_id")
|
||||
if agent_id:
|
||||
authz_app_id = AgentRosterService(db.session).peek_authz_app_id(tenant_id=tenant_id, agent_id=str(agent_id))
|
||||
return authz_app_id or str(agent_id)
|
||||
|
||||
resource_id = matched_args.get("resource_id")
|
||||
if resource_id:
|
||||
return str(resource_id)
|
||||
raise ValueError("Missing app_id in request path")
|
||||
|
||||
if resource_type == RBACResourceScope.DATASET:
|
||||
dataset_id = matched_args.get("dataset_id") or matched_args.get("resource_id")
|
||||
|
||||
@@ -230,6 +230,7 @@ class WorkflowAgentComposerSaveToRosterApi(Resource):
|
||||
@account_initialization_required
|
||||
@edit_permission_required
|
||||
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_EDIT)
|
||||
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.AGENT_MANAGE, resource_required=False)
|
||||
@with_current_user_id
|
||||
@with_current_tenant_id
|
||||
@with_session
|
||||
@@ -439,6 +440,7 @@ class SnippetAgentComposerSaveToRosterApi(Resource):
|
||||
@rbac_permission_required(
|
||||
RBACResourceScope.WORKSPACE, RBACPermission.SNIPPETS_CREATE_AND_MODIFY, resource_required=False
|
||||
)
|
||||
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.AGENT_MANAGE, resource_required=False)
|
||||
@with_current_user_id
|
||||
@with_current_tenant_id
|
||||
@with_session
|
||||
@@ -478,6 +480,7 @@ class AgentComposerApi(Resource):
|
||||
@account_initialization_required
|
||||
@edit_permission_required
|
||||
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_EDIT)
|
||||
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.AGENT_MANAGE, resource_required=False)
|
||||
@with_current_user_id
|
||||
@with_current_tenant_id
|
||||
@with_session
|
||||
|
||||
@@ -62,7 +62,7 @@ from libs.datetime_utils import parse_time_range
|
||||
from libs.helper import dump_response
|
||||
from libs.login import login_required
|
||||
from models import Account
|
||||
from models.agent import Agent, AgentStatus
|
||||
from models.agent import Agent, AgentConfigDraftType, AgentStatus
|
||||
from models.agent_config_entities import AgentSoulConfig
|
||||
from models.enums import ApiTokenType
|
||||
from models.model import ApiToken, App, IconType
|
||||
@@ -266,6 +266,13 @@ class AgentDebugConversationRefreshResponse(BaseModel):
|
||||
debug_conversation_message_count: int = 0
|
||||
|
||||
|
||||
class AgentDebugConversationRefreshPayload(BaseModel):
|
||||
draft_type: AgentConfigDraftType = Field(
|
||||
default=AgentConfigDraftType.DEBUG_BUILD,
|
||||
description="Agent draft surface whose conversation should be refreshed",
|
||||
)
|
||||
|
||||
|
||||
class AgentPublishPayload(BaseModel):
|
||||
version_note: str | None = Field(default=None, description="Optional note for this published Agent version")
|
||||
|
||||
@@ -309,6 +316,7 @@ register_schema_models(
|
||||
AgentAppCopyPayload,
|
||||
AgentPublishPayload,
|
||||
AgentBuildDraftCheckoutPayload,
|
||||
AgentDebugConversationRefreshPayload,
|
||||
ComposerSavePayload,
|
||||
AgentApiStatusPayload,
|
||||
AgentInviteOptionsQuery,
|
||||
@@ -392,6 +400,7 @@ def _serialize_agent_app_detail(
|
||||
tenant_id=app_model.tenant_id,
|
||||
agent_id=agent.id,
|
||||
account_id=current_user.id,
|
||||
draft_type=AgentConfigDraftType.DEBUG_BUILD,
|
||||
commit=False,
|
||||
)
|
||||
message_count = roster_service.count_agent_app_debug_conversation_messages(
|
||||
@@ -439,6 +448,7 @@ def _serialize_agent_app_pagination(session: Session, app_pagination, *, tenant_
|
||||
tenant_id=tenant_id,
|
||||
agents=list(agents_by_app_id.values()),
|
||||
account_id=current_user.id,
|
||||
draft_type=AgentConfigDraftType.DEBUG_BUILD,
|
||||
)
|
||||
payload = AgentAppPagination.model_validate(
|
||||
app_pagination,
|
||||
@@ -529,9 +539,23 @@ def _parse_observability_time_range(start: str | None, end: str | None, account:
|
||||
|
||||
|
||||
def _query_values(name: str, alias_name: str | None = None) -> list[str]:
|
||||
values = request.args.getlist(name)
|
||||
def _get_values(field_name: str) -> list[str]:
|
||||
values = request.args.getlist(field_name)
|
||||
indexed_values: list[tuple[int, list[str]]] = []
|
||||
prefix = f"{field_name}["
|
||||
for key in request.args:
|
||||
if not key.startswith(prefix) or not key.endswith("]"):
|
||||
continue
|
||||
index = key[len(prefix) : -1]
|
||||
if index.isdigit():
|
||||
indexed_values.append((int(index), request.args.getlist(key)))
|
||||
for _, items in sorted(indexed_values):
|
||||
values.extend(items)
|
||||
return values
|
||||
|
||||
values = _get_values(name)
|
||||
if alias_name:
|
||||
values.extend(request.args.getlist(alias_name))
|
||||
values.extend(_get_values(alias_name))
|
||||
return [value.strip() for value in values if value.strip()]
|
||||
|
||||
|
||||
@@ -542,6 +566,7 @@ class AgentAppListApi(Resource):
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.AGENT_MANAGE, resource_required=False)
|
||||
@with_current_user
|
||||
@with_current_tenant_id
|
||||
@with_session
|
||||
@@ -579,6 +604,7 @@ class AgentAppListApi(Resource):
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@edit_permission_required
|
||||
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.AGENT_MANAGE, resource_required=False)
|
||||
@with_current_user
|
||||
@with_current_tenant_id
|
||||
@with_session
|
||||
@@ -620,6 +646,7 @@ class AgentAppApi(Resource):
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@edit_permission_required
|
||||
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.AGENT_MANAGE, resource_required=False)
|
||||
@with_current_user
|
||||
@with_current_tenant_id
|
||||
@with_session
|
||||
@@ -645,6 +672,7 @@ class AgentAppApi(Resource):
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@edit_permission_required
|
||||
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.AGENT_MANAGE, resource_required=False)
|
||||
@with_current_tenant_id
|
||||
@with_session
|
||||
def delete(self, session: Session, tenant_id: str, agent_id: UUID):
|
||||
@@ -655,6 +683,16 @@ class AgentAppApi(Resource):
|
||||
|
||||
@console_ns.route("/agent/<uuid:agent_id>/debug-conversation/refresh")
|
||||
class AgentDebugConversationRefreshApi(Resource):
|
||||
@console_ns.expect(console_ns.models[AgentDebugConversationRefreshPayload.__name__])
|
||||
@console_ns.doc(
|
||||
params={
|
||||
"payload": {
|
||||
"in": "body",
|
||||
"required": False,
|
||||
"schema": {"$ref": f"#/components/schemas/{AgentDebugConversationRefreshPayload.__name__}"},
|
||||
}
|
||||
}
|
||||
)
|
||||
@console_ns.response(
|
||||
200,
|
||||
"Agent debug conversation refreshed",
|
||||
@@ -669,10 +707,12 @@ class AgentDebugConversationRefreshApi(Resource):
|
||||
@with_current_tenant_id
|
||||
@with_session
|
||||
def post(self, session: Session, tenant_id: str, current_user: Account, agent_id: UUID):
|
||||
args = AgentDebugConversationRefreshPayload.model_validate(request.get_json(silent=True) or {})
|
||||
debug_conversation_id = _agent_roster_service(session).refresh_agent_app_debug_conversation_id(
|
||||
tenant_id=tenant_id,
|
||||
agent_id=str(agent_id),
|
||||
account_id=current_user.id,
|
||||
draft_type=args.draft_type,
|
||||
)
|
||||
return AgentDebugConversationRefreshResponse(
|
||||
debug_conversation_id=debug_conversation_id,
|
||||
@@ -690,6 +730,7 @@ class AgentPublishApi(Resource):
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@edit_permission_required
|
||||
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.AGENT_MANAGE, resource_required=False)
|
||||
@with_current_user
|
||||
@with_current_tenant_id
|
||||
@with_session
|
||||
@@ -712,6 +753,7 @@ class AgentBuildDraftCheckoutApi(Resource):
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@edit_permission_required
|
||||
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.AGENT_MANAGE, resource_required=False)
|
||||
@with_current_user
|
||||
@with_current_tenant_id
|
||||
@with_session
|
||||
@@ -729,6 +771,7 @@ class AgentBuildDraftCheckoutApi(Resource):
|
||||
@console_ns.route("/agent/<uuid:agent_id>/build-draft")
|
||||
class AgentBuildDraftApi(Resource):
|
||||
@console_ns.response(200, "Agent build draft", console_ns.models[AgentBuildDraftResponse.__name__])
|
||||
@console_ns.response(404, "Agent build draft not found")
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@@ -787,6 +830,7 @@ class AgentBuildDraftApplyApi(Resource):
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@edit_permission_required
|
||||
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.AGENT_MANAGE, resource_required=False)
|
||||
@with_current_user
|
||||
@with_current_tenant_id
|
||||
@with_session
|
||||
@@ -809,6 +853,7 @@ class AgentAppCopyApi(Resource):
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@edit_permission_required
|
||||
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.AGENT_MANAGE, resource_required=False)
|
||||
@with_current_user
|
||||
@with_current_tenant_id
|
||||
@with_session
|
||||
@@ -834,6 +879,7 @@ class AgentApiAccessApi(Resource):
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.AGENT_MANAGE, resource_required=False)
|
||||
@with_current_tenant_id
|
||||
@with_session(write=False)
|
||||
def get(self, session: Session, tenant_id: str, agent_id: UUID):
|
||||
@@ -850,6 +896,7 @@ class AgentApiStatusApi(Resource):
|
||||
@login_required
|
||||
@is_admin_or_owner_required
|
||||
@account_initialization_required
|
||||
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.AGENT_MANAGE, resource_required=False)
|
||||
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_RELEASE_AND_VERSION)
|
||||
@with_current_tenant_id
|
||||
@with_session
|
||||
@@ -868,6 +915,7 @@ class AgentApiKeyListApi(BaseApiKeyListResource):
|
||||
token_prefix = "app-"
|
||||
|
||||
@console_ns.response(200, "Agent service API keys", console_ns.models[ApiKeyList.__name__])
|
||||
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.AGENT_MANAGE, resource_required=False)
|
||||
@with_current_tenant_id
|
||||
@with_session(write=False)
|
||||
def get(self, session: Session, tenant_id: str, agent_id: UUID) -> dict[str, object]:
|
||||
@@ -878,6 +926,7 @@ class AgentApiKeyListApi(BaseApiKeyListResource):
|
||||
@console_ns.response(400, "Maximum keys exceeded")
|
||||
@with_current_tenant_id
|
||||
@edit_permission_required
|
||||
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.AGENT_MANAGE, resource_required=False)
|
||||
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_RELEASE_AND_VERSION)
|
||||
@with_session
|
||||
def post(self, session: Session, tenant_id: str, agent_id: UUID) -> tuple[dict[str, object], int]:
|
||||
@@ -897,6 +946,7 @@ class AgentApiKeyApi(BaseApiKeyResource):
|
||||
@console_ns.response(204, "Agent service API key deleted")
|
||||
@with_current_user
|
||||
@with_current_tenant_id
|
||||
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.AGENT_MANAGE, resource_required=False)
|
||||
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_RELEASE_AND_VERSION)
|
||||
@with_session
|
||||
def delete(
|
||||
@@ -942,6 +992,7 @@ class AgentLogsApi(Resource):
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.AGENT_MANAGE, resource_required=False)
|
||||
@with_current_user
|
||||
@with_current_tenant_id
|
||||
@with_session(write=False)
|
||||
@@ -980,6 +1031,7 @@ class AgentLogMessagesApi(Resource):
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.AGENT_MANAGE, resource_required=False)
|
||||
@with_current_user
|
||||
@with_current_tenant_id
|
||||
@with_session(write=False)
|
||||
@@ -1018,6 +1070,7 @@ class AgentLogSourcesApi(Resource):
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.AGENT_MANAGE, resource_required=False)
|
||||
@with_current_user
|
||||
@with_current_tenant_id
|
||||
@with_session(write=False)
|
||||
@@ -1038,6 +1091,7 @@ class AgentStatisticsSummaryApi(Resource):
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.AGENT_MANAGE, resource_required=False)
|
||||
@with_current_user
|
||||
@with_current_tenant_id
|
||||
@with_session(write=False)
|
||||
@@ -1063,6 +1117,7 @@ class AgentRosterVersionsApi(Resource):
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.AGENT_MANAGE, resource_required=False)
|
||||
@with_current_tenant_id
|
||||
@with_session(write=False)
|
||||
def get(self, session: Session, tenant_id: str, agent_id: UUID):
|
||||
@@ -1078,6 +1133,7 @@ class AgentRosterVersionDetailApi(Resource):
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.AGENT_MANAGE, resource_required=False)
|
||||
@with_current_tenant_id
|
||||
@with_session(write=False)
|
||||
def get(self, session: Session, tenant_id: str, agent_id: UUID, version_id: UUID):
|
||||
@@ -1098,6 +1154,7 @@ class AgentRosterVersionRestoreApi(Resource):
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@edit_permission_required
|
||||
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.AGENT_MANAGE, resource_required=False)
|
||||
@with_current_user
|
||||
@with_current_tenant_id
|
||||
@with_session
|
||||
|
||||
@@ -12,6 +12,7 @@ from werkzeug.exceptions import Forbidden
|
||||
from configs import dify_config
|
||||
from controllers.common.schema import register_response_schema_models
|
||||
from controllers.common.session import with_session
|
||||
from controllers.console.app.wraps import agent_manage_required_for_agent_app
|
||||
from fields.base import ResponseModel
|
||||
from libs.helper import dump_response, to_timestamp
|
||||
from libs.login import login_required
|
||||
@@ -194,6 +195,7 @@ class AppApiKeyListResource(BaseApiKeyListResource):
|
||||
@console_ns.doc(params={"resource_id": "App ID"})
|
||||
@console_ns.response(200, "API keys retrieved successfully", console_ns.models[ApiKeyList.__name__])
|
||||
@with_current_tenant_id
|
||||
@agent_manage_required_for_agent_app
|
||||
@with_session(write=False)
|
||||
def get(self, session: Session, current_tenant_id: str, resource_id: UUID) -> dict[str, object]:
|
||||
"""Get all API keys for an app"""
|
||||
@@ -210,6 +212,7 @@ class AppApiKeyListResource(BaseApiKeyListResource):
|
||||
@with_current_tenant_id
|
||||
@edit_permission_required
|
||||
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_RELEASE_AND_VERSION)
|
||||
@agent_manage_required_for_agent_app
|
||||
@with_session
|
||||
def post(self, session: Session, current_tenant_id: str, resource_id: UUID) -> tuple[dict[str, object], int]:
|
||||
"""Create a new API key for an app"""
|
||||
@@ -233,6 +236,7 @@ class AppApiKeyResource(BaseApiKeyResource):
|
||||
@with_current_user
|
||||
@with_current_tenant_id
|
||||
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_RELEASE_AND_VERSION)
|
||||
@agent_manage_required_for_agent_app
|
||||
@with_session
|
||||
def delete(
|
||||
self,
|
||||
|
||||
@@ -9,7 +9,7 @@ from flask_restx import Resource
|
||||
from pydantic import AliasChoices, BaseModel, Field, ValidationInfo, computed_field, field_validator, model_validator
|
||||
from sqlalchemy import select
|
||||
from sqlalchemy.orm import Session
|
||||
from werkzeug.exceptions import BadRequest, NotFound
|
||||
from werkzeug.exceptions import BadRequest, Forbidden, NotFound
|
||||
|
||||
from configs import dify_config
|
||||
from controllers.common.app_access import resolve_app_access_filter
|
||||
@@ -23,7 +23,7 @@ from controllers.common.schema import (
|
||||
register_schema_models,
|
||||
)
|
||||
from controllers.console import console_ns
|
||||
from controllers.console.app.wraps import get_app_model, with_session
|
||||
from controllers.console.app.wraps import agent_manage_required_for_agent_app, get_app_model, with_session
|
||||
from controllers.console.workspace.models import LoadBalancingPayload
|
||||
from controllers.console.wraps import (
|
||||
RBACPermission,
|
||||
@@ -75,6 +75,7 @@ from services.entities.knowledge_entities.knowledge_entities import (
|
||||
WeightModel,
|
||||
WeightVectorSetting,
|
||||
)
|
||||
from services.errors.account import NoPermissionError
|
||||
from services.feature_service import FeatureService
|
||||
from tasks.initialize_created_app_rbac_access_task import initialize_created_app_rbac_access_task
|
||||
|
||||
@@ -246,7 +247,7 @@ class ModelConfigPartial(ResponseModel):
|
||||
return to_timestamp(value)
|
||||
|
||||
|
||||
class ModelConfig(ResponseModel):
|
||||
class AppModelConfigResponse(ResponseModel):
|
||||
opening_statement: str | None = None
|
||||
suggested_questions: Any | None = Field(
|
||||
default=None, validation_alias=AliasChoices("suggested_questions_list", "suggested_questions")
|
||||
@@ -419,7 +420,7 @@ class AppDetail(AppResponseModel):
|
||||
icon_background: str | None = None
|
||||
enable_site: bool
|
||||
enable_api: bool
|
||||
model_config_: ModelConfig | None = Field(
|
||||
model_config_: AppModelConfigResponse | None = Field(
|
||||
default=None,
|
||||
validation_alias=AliasChoices("app_model_config", "model_config"),
|
||||
alias="model_config",
|
||||
@@ -525,7 +526,13 @@ def _enrich_app_list_items(session: Session, *, apps: Sequence[App], tenant_id:
|
||||
|
||||
register_enum_models(console_ns, RetrievalMethod, WorkflowExecutionStatus, DatasetPermissionEnum)
|
||||
register_response_schema_models(
|
||||
console_ns, RedirectUrlResponse, SimpleResultResponse, AppImportResponse, AppTraceResponse
|
||||
console_ns,
|
||||
RedirectUrlResponse,
|
||||
SimpleResultResponse,
|
||||
AppImportResponse,
|
||||
AppTraceResponse,
|
||||
AppModelConfigResponse,
|
||||
AppDetail,
|
||||
)
|
||||
|
||||
register_schema_models(
|
||||
@@ -544,10 +551,8 @@ register_schema_models(
|
||||
Tag,
|
||||
WorkflowPartial,
|
||||
ModelConfigPartial,
|
||||
ModelConfig,
|
||||
AppDetailSiteResponse,
|
||||
DeletedTool,
|
||||
AppDetail,
|
||||
AppExportResponse,
|
||||
Segmentation,
|
||||
PreProcessingRule,
|
||||
@@ -823,6 +828,7 @@ class AppApi(Resource):
|
||||
@account_initialization_required
|
||||
@edit_permission_required
|
||||
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_EDIT)
|
||||
@agent_manage_required_for_agent_app
|
||||
@with_session
|
||||
@get_app_model(mode=None)
|
||||
def put(self, session: Session, app_model: App):
|
||||
@@ -857,6 +863,7 @@ class AppApi(Resource):
|
||||
@account_initialization_required
|
||||
@edit_permission_required
|
||||
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_DELETE)
|
||||
@agent_manage_required_for_agent_app
|
||||
@with_session
|
||||
@get_app_model
|
||||
def delete(self, session: Session, app_model: App):
|
||||
@@ -881,6 +888,7 @@ class AppCopyApi(Resource):
|
||||
@account_initialization_required
|
||||
@edit_permission_required
|
||||
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_CREATE_AND_MANAGEMENT)
|
||||
@agent_manage_required_for_agent_app
|
||||
@with_current_user
|
||||
@with_current_tenant_id
|
||||
@get_app_model(mode=None)
|
||||
@@ -892,16 +900,19 @@ class AppCopyApi(Resource):
|
||||
with Session(db.engine, expire_on_commit=False) as session:
|
||||
import_service = AppDslService(session)
|
||||
yaml_content = import_service.export_dsl(app_model=app_model, session=session, include_secret=True)
|
||||
result = import_service.import_app(
|
||||
account=current_user,
|
||||
import_mode=ImportMode.YAML_CONTENT,
|
||||
yaml_content=yaml_content,
|
||||
name=args.name,
|
||||
description=args.description,
|
||||
icon_type=args.icon_type,
|
||||
icon=args.icon,
|
||||
icon_background=args.icon_background,
|
||||
)
|
||||
try:
|
||||
result = import_service.import_app(
|
||||
account=current_user,
|
||||
import_mode=ImportMode.YAML_CONTENT,
|
||||
yaml_content=yaml_content,
|
||||
name=args.name,
|
||||
description=args.description,
|
||||
icon_type=args.icon_type,
|
||||
icon=args.icon,
|
||||
icon_background=args.icon_background,
|
||||
)
|
||||
except NoPermissionError as e:
|
||||
raise Forbidden(str(e))
|
||||
if result.status == ImportStatus.FAILED:
|
||||
session.rollback()
|
||||
return dump_response(AppImportResponse, result), 400
|
||||
@@ -955,6 +966,7 @@ class AppExportApi(Resource):
|
||||
@account_initialization_required
|
||||
@edit_permission_required
|
||||
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_IMPORT_EXPORT_DSL)
|
||||
@agent_manage_required_for_agent_app
|
||||
@get_app_model
|
||||
def get(self, app_model: App):
|
||||
"""Export app"""
|
||||
@@ -979,6 +991,7 @@ class AppPublishToCreatorsPlatformApi(Resource):
|
||||
@account_initialization_required
|
||||
@edit_permission_required
|
||||
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_IMPORT_EXPORT_DSL)
|
||||
@agent_manage_required_for_agent_app
|
||||
@with_current_user_id
|
||||
@get_app_model(mode=None)
|
||||
def post(self, current_user_id: str, app_model: App):
|
||||
@@ -1009,6 +1022,7 @@ class AppNameApi(Resource):
|
||||
@account_initialization_required
|
||||
@edit_permission_required
|
||||
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_EDIT)
|
||||
@agent_manage_required_for_agent_app
|
||||
@with_session
|
||||
@get_app_model(mode=None)
|
||||
def post(self, session: Session, app_model: App):
|
||||
@@ -1036,6 +1050,7 @@ class AppIconApi(Resource):
|
||||
@account_initialization_required
|
||||
@edit_permission_required
|
||||
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_EDIT)
|
||||
@agent_manage_required_for_agent_app
|
||||
@with_session
|
||||
@get_app_model(mode=None)
|
||||
def post(self, session: Session, app_model: App):
|
||||
@@ -1069,6 +1084,7 @@ class AppSiteStatus(Resource):
|
||||
@account_initialization_required
|
||||
@edit_permission_required
|
||||
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_RELEASE_AND_VERSION)
|
||||
@agent_manage_required_for_agent_app
|
||||
@with_session
|
||||
@get_app_model(mode=None)
|
||||
def post(self, session: Session, app_model: App):
|
||||
@@ -1096,6 +1112,7 @@ class AppApiStatus(Resource):
|
||||
@is_admin_or_owner_required
|
||||
@account_initialization_required
|
||||
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_RELEASE_AND_VERSION)
|
||||
@agent_manage_required_for_agent_app
|
||||
@with_session
|
||||
@get_app_model(mode=None)
|
||||
def post(self, session: Session, app_model: App):
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
from flask_restx import Resource
|
||||
from pydantic import BaseModel, Field
|
||||
from sqlalchemy.orm import Session
|
||||
from werkzeug.exceptions import Forbidden
|
||||
|
||||
from configs import dify_config
|
||||
from controllers.common.schema import register_enum_models, register_schema_models
|
||||
@@ -28,6 +29,7 @@ from services.app_dsl_service import (
|
||||
)
|
||||
from services.enterprise.enterprise_service import EnterpriseService
|
||||
from services.entities.dsl_entities import CheckDependenciesResult, ImportStatus
|
||||
from services.errors.account import NoPermissionError
|
||||
from services.feature_service import FeatureService
|
||||
|
||||
from .. import console_ns
|
||||
@@ -91,18 +93,21 @@ class AppImportApi(Resource):
|
||||
import_service = AppDslService(session)
|
||||
# Import app
|
||||
account = current_user
|
||||
result = import_service.import_app(
|
||||
account=account,
|
||||
import_mode=args.mode,
|
||||
yaml_content=args.yaml_content,
|
||||
yaml_url=args.yaml_url,
|
||||
name=args.name,
|
||||
description=args.description,
|
||||
icon_type=args.icon_type,
|
||||
icon=args.icon,
|
||||
icon_background=args.icon_background,
|
||||
app_id=args.app_id,
|
||||
)
|
||||
try:
|
||||
result = import_service.import_app(
|
||||
account=account,
|
||||
import_mode=args.mode,
|
||||
yaml_content=args.yaml_content,
|
||||
yaml_url=args.yaml_url,
|
||||
name=args.name,
|
||||
description=args.description,
|
||||
icon_type=args.icon_type,
|
||||
icon=args.icon,
|
||||
icon_background=args.icon_background,
|
||||
app_id=args.app_id,
|
||||
)
|
||||
except NoPermissionError as e:
|
||||
raise Forbidden(str(e))
|
||||
if result.status == ImportStatus.FAILED:
|
||||
session.rollback()
|
||||
else:
|
||||
@@ -157,7 +162,10 @@ class AppImportConfirmApi(Resource):
|
||||
import_service = AppDslService(session)
|
||||
# Confirm import
|
||||
account = current_user
|
||||
result = import_service.confirm_import(import_id=import_id, account=account)
|
||||
try:
|
||||
result = import_service.confirm_import(import_id=import_id, account=account)
|
||||
except NoPermissionError as e:
|
||||
raise Forbidden(str(e))
|
||||
if result.status == ImportStatus.FAILED:
|
||||
session.rollback()
|
||||
else:
|
||||
|
||||
@@ -49,6 +49,7 @@ from libs import helper
|
||||
from libs.helper import uuid_value
|
||||
from libs.login import login_required
|
||||
from models import Account
|
||||
from models.agent import AgentConfigDraftType
|
||||
from models.model import App, AppMode
|
||||
from services.agent.errors import AgentNotFoundError
|
||||
from services.agent.roster_service import AgentRosterService
|
||||
@@ -343,23 +344,40 @@ class AgentChatMessageStopApi(Resource):
|
||||
|
||||
|
||||
def _resolve_current_user_agent_debug_conversation_id(
|
||||
*, session: Session, current_tenant_id: str, current_user: Account, app_model: App, agent_id: str | None
|
||||
*,
|
||||
session: Session,
|
||||
current_tenant_id: str,
|
||||
current_user: Account,
|
||||
app_model: App,
|
||||
agent_id: str | None,
|
||||
draft_type: AgentConfigDraftType,
|
||||
start_new: bool = False,
|
||||
) -> str:
|
||||
roster_service = AgentRosterService(session)
|
||||
if agent_id:
|
||||
return roster_service.get_or_create_agent_app_debug_conversation_id(
|
||||
tenant_id=current_tenant_id,
|
||||
agent_id=agent_id,
|
||||
account_id=current_user.id,
|
||||
)
|
||||
"""Resolve or rotate the current editor's conversation within one draft surface.
|
||||
|
||||
agent = roster_service.get_app_backing_agent(tenant_id=current_tenant_id, app_id=str(app_model.id))
|
||||
if agent is None:
|
||||
raise AgentNotFoundError()
|
||||
return roster_service.get_or_create_agent_app_debug_conversation_id(
|
||||
``start_new`` rotates the scoped mapping through ``AgentRosterService`` so
|
||||
the old runtime session is retired before the new conversation is used.
|
||||
Continuations and Build chat keep resolving the existing mapping.
|
||||
"""
|
||||
|
||||
roster_service = AgentRosterService(session)
|
||||
resolved_agent_id = agent_id
|
||||
if not resolved_agent_id:
|
||||
agent = roster_service.get_app_backing_agent(tenant_id=current_tenant_id, app_id=str(app_model.id))
|
||||
if agent is None:
|
||||
raise AgentNotFoundError()
|
||||
resolved_agent_id = agent.id
|
||||
|
||||
resolve_conversation = (
|
||||
roster_service.refresh_agent_app_debug_conversation_id
|
||||
if start_new
|
||||
else roster_service.get_or_create_agent_app_debug_conversation_id
|
||||
)
|
||||
return resolve_conversation(
|
||||
tenant_id=current_tenant_id,
|
||||
agent_id=agent.id,
|
||||
agent_id=resolved_agent_id,
|
||||
account_id=current_user.id,
|
||||
draft_type=draft_type,
|
||||
)
|
||||
|
||||
|
||||
@@ -376,12 +394,17 @@ def _create_chat_message(
|
||||
args = args_model.model_dump(exclude_none=True, by_alias=True)
|
||||
|
||||
if AppMode.value_of(app_model.mode) == AppMode.AGENT:
|
||||
draft_type = AgentConfigDraftType(args_model.draft_type)
|
||||
# Preview follows the normal chat contract: an omitted/empty conversation ID starts a new
|
||||
# conversation. Build chat keeps its stable mapping so build drafts and finalization stay continuous.
|
||||
debug_conversation_id = _resolve_current_user_agent_debug_conversation_id(
|
||||
session=session,
|
||||
current_tenant_id=current_tenant_id or app_model.tenant_id,
|
||||
current_user=current_user,
|
||||
app_model=app_model,
|
||||
agent_id=agent_id,
|
||||
draft_type=draft_type,
|
||||
start_new=draft_type == AgentConfigDraftType.DRAFT and not args_model.conversation_id,
|
||||
)
|
||||
if args_model.conversation_id and args_model.conversation_id != debug_conversation_id:
|
||||
raise NotFound("Conversation Not Exists.")
|
||||
@@ -418,6 +441,7 @@ def _create_build_chat_finalization_message(
|
||||
current_user=current_user,
|
||||
app_model=app_model,
|
||||
agent_id=agent_id,
|
||||
draft_type=AgentConfigDraftType.DEBUG_BUILD,
|
||||
)
|
||||
args: dict[str, Any] = {
|
||||
"query": _BUILD_CHAT_FINALIZATION_QUERY,
|
||||
|
||||
@@ -10,7 +10,7 @@ from constants.languages import supported_language
|
||||
from controllers.common.schema import register_schema_models
|
||||
from controllers.common.session import with_session
|
||||
from controllers.console import console_ns
|
||||
from controllers.console.app.wraps import get_app_model
|
||||
from controllers.console.app.wraps import agent_manage_required_for_agent_app, get_app_model
|
||||
from controllers.console.wraps import (
|
||||
RBACPermission,
|
||||
RBACResourceScope,
|
||||
@@ -93,6 +93,7 @@ class AppSite(Resource):
|
||||
@login_required
|
||||
@edit_permission_required
|
||||
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_RELEASE_AND_VERSION)
|
||||
@agent_manage_required_for_agent_app
|
||||
@account_initialization_required
|
||||
@with_current_user
|
||||
@with_session
|
||||
@@ -145,6 +146,7 @@ class AppSiteAccessTokenReset(Resource):
|
||||
@login_required
|
||||
@is_admin_or_owner_required
|
||||
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_RELEASE_AND_VERSION)
|
||||
@agent_manage_required_for_agent_app
|
||||
@account_initialization_required
|
||||
@with_current_user
|
||||
@with_session
|
||||
|
||||
@@ -359,6 +359,12 @@ class WorkflowPublishResponse(ResponseModel):
|
||||
created_at: int
|
||||
|
||||
|
||||
class SyncDraftWorkflowResponse(ResponseModel):
|
||||
result: str
|
||||
hash: str
|
||||
updated_at: int
|
||||
|
||||
|
||||
class WorkflowRestoreResponse(ResponseModel):
|
||||
result: str
|
||||
hash: str
|
||||
@@ -441,6 +447,7 @@ register_response_schema_models(
|
||||
WorkflowOnlineUsersByApp,
|
||||
WorkflowOnlineUsersResponse,
|
||||
WorkflowPublishResponse,
|
||||
SyncDraftWorkflowResponse,
|
||||
WorkflowRestoreResponse,
|
||||
DefaultBlockConfigsResponse,
|
||||
DefaultBlockConfigResponse,
|
||||
@@ -556,14 +563,7 @@ class DraftWorkflowApi(Resource):
|
||||
@console_ns.response(
|
||||
200,
|
||||
"Draft workflow synced successfully",
|
||||
console_ns.model(
|
||||
"SyncDraftWorkflowResponse",
|
||||
{
|
||||
"result": fields.String,
|
||||
"hash": fields.String,
|
||||
"updated_at": fields.String,
|
||||
},
|
||||
),
|
||||
console_ns.models[SyncDraftWorkflowResponse.__name__],
|
||||
)
|
||||
@console_ns.response(400, "Invalid workflow configuration")
|
||||
@console_ns.response(403, "Permission denied")
|
||||
@@ -618,11 +618,14 @@ class DraftWorkflowApi(Resource):
|
||||
except VariableError as e:
|
||||
raise InvalidArgumentError(description=str(e))
|
||||
|
||||
return {
|
||||
"result": "success",
|
||||
"hash": workflow.unique_hash,
|
||||
"updated_at": TimestampField().format(workflow.updated_at or workflow.created_at),
|
||||
}
|
||||
return dump_response(
|
||||
SyncDraftWorkflowResponse,
|
||||
{
|
||||
"result": "success",
|
||||
"hash": workflow.unique_hash,
|
||||
"updated_at": TimestampField().format(workflow.updated_at or workflow.created_at),
|
||||
},
|
||||
)
|
||||
|
||||
|
||||
@console_ns.route("/apps/<uuid:app_id>/advanced-chat/workflows/draft/run")
|
||||
|
||||
@@ -12,14 +12,22 @@ from typing import cast, overload
|
||||
from sqlalchemy import select
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from configs import dify_config
|
||||
from controllers.common.session import with_session
|
||||
from controllers.common.wraps import RBACPermission, RBACResourceScope, enforce_rbac_access
|
||||
from controllers.console.app.error import AppNotFoundError
|
||||
from extensions.ext_database import db
|
||||
from libs.login import current_account_with_tenant
|
||||
from models import App, AppMode, TrialApp
|
||||
from models.agent import AgentScope
|
||||
from services.recommended_app_service import RecommendedAppService
|
||||
|
||||
__all__ = ["get_app_model", "get_app_model_with_trial", "with_session"]
|
||||
__all__ = [
|
||||
"agent_manage_required_for_agent_app",
|
||||
"get_app_model",
|
||||
"get_app_model_with_trial",
|
||||
"with_session",
|
||||
]
|
||||
|
||||
|
||||
def _load_app_model(session: Session, app_id: str) -> App | None:
|
||||
@@ -48,6 +56,45 @@ def _load_app_model_with_trial(session: Session, app_id: str) -> App | None:
|
||||
return app_model
|
||||
|
||||
|
||||
def agent_manage_required_for_agent_app[**P, R](view: Callable[P, R]) -> Callable[P, R]:
|
||||
"""Gate generic app management routes that target an Agent App.
|
||||
|
||||
A hidden workflow-only backing App only reuses the App runtime and is not
|
||||
part of the general app management plane, so generic routes reject it
|
||||
outright. Managing a roster Agent App mutates the roster Agent behind it
|
||||
(rename/icon sync, archive, API enablement), so it additionally requires
|
||||
workspace ``agent.manage`` on top of the route's existing App permission
|
||||
checks when RBAC is enabled. A no-op for non-agent Apps. Must be placed
|
||||
above ``get_app_model`` so the ``app_id`` path parameter is still present.
|
||||
"""
|
||||
|
||||
@wraps(view)
|
||||
def decorated(*args: P.args, **kwargs: P.kwargs) -> R:
|
||||
raw_app_id = kwargs.get("app_id") or kwargs.get("resource_id")
|
||||
if raw_app_id is not None:
|
||||
app_model = _load_app_model_from_scoped_session(str(raw_app_id))
|
||||
binding = (
|
||||
app_model.agent_app_binding_with_session(session=db.session(), include_archived=True)
|
||||
if app_model is not None
|
||||
else None
|
||||
)
|
||||
if binding is not None:
|
||||
if binding.scope == AgentScope.WORKFLOW_ONLY:
|
||||
raise AppNotFoundError()
|
||||
if dify_config.RBAC_ENABLED:
|
||||
current_user, current_tenant_id = current_account_with_tenant()
|
||||
enforce_rbac_access(
|
||||
tenant_id=current_tenant_id,
|
||||
account_id=current_user.id,
|
||||
resource_type=RBACResourceScope.WORKSPACE,
|
||||
scene=RBACPermission.AGENT_MANAGE,
|
||||
resource_required=False,
|
||||
)
|
||||
return view(*args, **kwargs)
|
||||
|
||||
return decorated
|
||||
|
||||
|
||||
def _get_injected_session(args: tuple[object, ...]) -> Session | None:
|
||||
"""Return the request session inserted by `with_session`, if this handler has been migrated."""
|
||||
if len(args) < 2:
|
||||
|
||||
@@ -7,10 +7,13 @@ from configs import dify_config
|
||||
from constants.languages import supported_language
|
||||
from controllers.common.schema import query_params_from_model, register_schema_models
|
||||
from controllers.console import console_ns
|
||||
from controllers.console.auth.error import InvitationAccountMismatchError
|
||||
from controllers.console.error import AccountInFreezeError, AlreadyActivateError
|
||||
from extensions.ext_database import db
|
||||
from libs.datetime_utils import naive_utc_now
|
||||
from libs.helper import EmailStr, timezone
|
||||
from libs.login import current_account_with_tenant
|
||||
from libs.token import extract_access_token
|
||||
from models import AccountStatus
|
||||
from models.account import TenantAccountJoin, TenantAccountRole
|
||||
from services.account_service import RegisterService, TenantService
|
||||
@@ -136,6 +139,12 @@ class ActivateApi(Resource):
|
||||
)
|
||||
@console_ns.response(400, "Already activated or invalid token")
|
||||
def post(self):
|
||||
"""Accept an invitation without letting an existing session act for another account.
|
||||
|
||||
Token-only activation remains available for legacy clients. When the request already
|
||||
carries a console session, that session must belong to the account encoded in the
|
||||
invitation before the token is consumed or tenant membership is changed.
|
||||
"""
|
||||
args = ActivatePayload.model_validate(console_ns.payload)
|
||||
|
||||
normalized_request_email = args.email.lower() if args.email else None
|
||||
@@ -146,6 +155,11 @@ class ActivateApi(Resource):
|
||||
raise AlreadyActivateError()
|
||||
|
||||
account = invitation["account"]
|
||||
if extract_access_token(request):
|
||||
current_account, _ = current_account_with_tenant()
|
||||
if current_account.id != account.id:
|
||||
raise InvitationAccountMismatchError()
|
||||
|
||||
if dify_config.BILLING_ENABLED and BillingService.is_email_in_freeze(account.email):
|
||||
raise AccountInFreezeError()
|
||||
|
||||
|
||||
@@ -13,6 +13,12 @@ class InvalidEmailError(BaseHTTPException):
|
||||
code = 400
|
||||
|
||||
|
||||
class InvitationAccountMismatchError(BaseHTTPException):
|
||||
error_code = "invitation_account_mismatch"
|
||||
description = "This invitation was sent to another account. Please sign in with the invited account."
|
||||
code = 403
|
||||
|
||||
|
||||
class PasswordMismatchError(BaseHTTPException):
|
||||
error_code = "password_mismatch"
|
||||
description = "The passwords do not match."
|
||||
|
||||
@@ -6,6 +6,7 @@ from flask import current_app, redirect, request
|
||||
from flask_restx import Resource
|
||||
from pydantic import BaseModel, Field
|
||||
from werkzeug.exceptions import Unauthorized
|
||||
from werkzeug.wrappers import Response
|
||||
|
||||
from configs import dify_config
|
||||
from constants.languages import languages
|
||||
@@ -127,6 +128,20 @@ def _preferred_interface_language(language: str | None = None) -> str:
|
||||
return languages[0]
|
||||
|
||||
|
||||
def _redirect_with_console_session(account: Account, target_url: str) -> Response:
|
||||
"""Create a console session and attach its cookies to a redirect response."""
|
||||
token_pair = AccountService.login(
|
||||
account=account,
|
||||
session=db.session(),
|
||||
ip_address=extract_remote_ip(request),
|
||||
)
|
||||
response = redirect(target_url)
|
||||
set_access_token_to_cookie(request, response, token_pair.access_token)
|
||||
set_refresh_token_to_cookie(request, response, token_pair.refresh_token)
|
||||
set_csrf_token_to_cookie(request, response, token_pair.csrf_token)
|
||||
return response
|
||||
|
||||
|
||||
@console_ns.route("/oauth/login/<provider>")
|
||||
class OAuthLogin(Resource):
|
||||
@console_ns.doc("oauth_login")
|
||||
@@ -195,16 +210,26 @@ class OAuthCallback(Resource):
|
||||
return redirect(f"{dify_config.CONSOLE_WEB_URL}/signin?message={urllib.parse.quote(str(e))}")
|
||||
|
||||
if invite_token and RegisterService.is_valid_invite_token(invite_token):
|
||||
invitation = RegisterService.get_invitation_by_token(token=invite_token)
|
||||
if invitation:
|
||||
invitation_email = invitation.get("email", None)
|
||||
invitation_email_normalized = (
|
||||
invitation_email.lower() if isinstance(invitation_email, str) else invitation_email
|
||||
)
|
||||
if invitation_email_normalized != user_info.email.lower():
|
||||
return redirect(f"{dify_config.CONSOLE_WEB_URL}/signin?message=Invalid invitation token.")
|
||||
invitation = RegisterService.get_invitation_if_token_valid(
|
||||
None,
|
||||
None,
|
||||
invite_token,
|
||||
session=db.session(),
|
||||
)
|
||||
if not invitation:
|
||||
return redirect(f"{dify_config.CONSOLE_WEB_URL}/signin?message=Invalid invitation token.")
|
||||
if invitation["data"]["email"].lower() != user_info.email.lower():
|
||||
message = "This invitation was sent to another account. Please sign in with the invited account."
|
||||
query = urllib.parse.urlencode({"message": message, "invite_token": invite_token})
|
||||
return redirect(f"{dify_config.CONSOLE_WEB_URL}/signin?{query}")
|
||||
|
||||
return redirect(f"{dify_config.CONSOLE_WEB_URL}/signin/invite-settings?invite_token={invite_token}")
|
||||
account = invitation["account"]
|
||||
if account.status == AccountStatus.BANNED:
|
||||
return redirect(f"{dify_config.CONSOLE_WEB_URL}/signin?message=Account is banned.")
|
||||
|
||||
AccountService.link_account_integrate(provider, user_info.id, account, session=db.session())
|
||||
target_url = f"{dify_config.CONSOLE_WEB_URL}/signin/invite-settings?invite_token={invite_token}"
|
||||
return _redirect_with_console_session(account, target_url)
|
||||
|
||||
try:
|
||||
account, oauth_new_user = _generate_account(provider, user_info, timezone=timezone, language=language)
|
||||
@@ -239,21 +264,10 @@ class OAuthCallback(Resource):
|
||||
"?message=Workspace not found, please contact system admin to invite you to join in a workspace."
|
||||
)
|
||||
|
||||
token_pair = AccountService.login(
|
||||
account=account,
|
||||
session=db.session(),
|
||||
ip_address=extract_remote_ip(request),
|
||||
)
|
||||
|
||||
target_url = _get_redirect_target(redirect_url)
|
||||
query_char = "&" if "?" in target_url else "?"
|
||||
target_url = f"{target_url}{query_char}oauth_new_user={str(oauth_new_user).lower()}"
|
||||
response = redirect(target_url)
|
||||
|
||||
set_access_token_to_cookie(request, response, token_pair.access_token)
|
||||
set_refresh_token_to_cookie(request, response, token_pair.refresh_token)
|
||||
set_csrf_token_to_cookie(request, response, token_pair.csrf_token)
|
||||
return response
|
||||
return _redirect_with_console_session(account, target_url)
|
||||
|
||||
|
||||
def _get_account_by_openid_or_email(provider: str, user_info: OAuthUserInfo) -> Account | None:
|
||||
|
||||
@@ -56,6 +56,7 @@ class OAuthProviderTokenResponse(BaseModel):
|
||||
|
||||
|
||||
class OAuthProviderAccountResponse(BaseModel):
|
||||
id: str
|
||||
name: str
|
||||
email: str
|
||||
avatar: str | None = None
|
||||
@@ -251,6 +252,7 @@ class OAuthServerUserAccountApi(Resource):
|
||||
def post(self, oauth_provider_app: OAuthProviderApp, account: Account):
|
||||
return jsonable_encoder(
|
||||
{
|
||||
"id": account.id,
|
||||
"name": account.name,
|
||||
"email": account.email,
|
||||
"avatar": account.avatar,
|
||||
|
||||
@@ -8,8 +8,9 @@ can be validated explicitly against the pinned KnowledgeFS contract during devel
|
||||
Console auth and contract-specific dataset RBAC run before forwarding. Request
|
||||
bodies are capped at 64 MiB, JSON and binary responses have separate bounds,
|
||||
SSE responses remain streaming with a bounded idle read timeout, and only safe
|
||||
response headers are exposed. Upstream 401 responses become 502 so they cannot
|
||||
trigger Dify browser-session recovery; resource-level 403 responses remain 403.
|
||||
response headers are exposed. Operation-specific upstream error mappings are
|
||||
applied before Console JSON error handling; the default maps 401 to 502 so it
|
||||
cannot trigger browser-session recovery and preserves resource-level 403.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
@@ -27,9 +28,11 @@ from werkzeug.exceptions import (
|
||||
BadGateway,
|
||||
Forbidden,
|
||||
GatewayTimeout,
|
||||
HTTPException,
|
||||
NotFound,
|
||||
RequestEntityTooLarge,
|
||||
ServiceUnavailable,
|
||||
default_exceptions,
|
||||
)
|
||||
|
||||
from configs import dify_config
|
||||
@@ -41,21 +44,28 @@ from controllers.console.wraps import (
|
||||
)
|
||||
from core.helper import ssrf_proxy
|
||||
from libs.login import current_account_with_tenant, login_required
|
||||
from services.knowledge_fs_operations import KnowledgeFSMethod
|
||||
from services.knowledge_fs_proxy import (
|
||||
KnowledgeFSAccessDeniedError,
|
||||
KnowledgeFSAuthorization,
|
||||
KnowledgeFSConfigurationError,
|
||||
KnowledgeFSMethod,
|
||||
KnowledgeFSRouteNotAllowedError,
|
||||
KnowledgeFSTimeoutError,
|
||||
KnowledgeFSTransportError,
|
||||
KnowledgeFSUpstreamResponse,
|
||||
authorize_knowledge_fs_request,
|
||||
get_knowledge_fs_operation,
|
||||
proxy_authorized_knowledge_fs_request,
|
||||
proxy_knowledge_fs_request,
|
||||
)
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
type _KnowledgeFSRequestForwarder = Callable[
|
||||
[str | None, str | None, bytes | None, bytes | None],
|
||||
KnowledgeFSUpstreamResponse,
|
||||
]
|
||||
|
||||
_MAX_PROXY_BODY_BYTES = 64 * 1024 * 1024
|
||||
_RESPONSE_HEADER_ALLOWLIST = (
|
||||
"Cache-Control",
|
||||
@@ -128,27 +138,25 @@ def _translate_proxy_error(exc: Exception, *, tenant_id: str) -> NoReturn:
|
||||
|
||||
|
||||
def _knowledge_fs_operation_access_required(
|
||||
view: Callable[[KnowledgeFSMethod, str], ResponseReturnValue],
|
||||
view: Callable[[KnowledgeFSAuthorization], ResponseReturnValue],
|
||||
) -> Callable[[KnowledgeFSMethod, str], ResponseReturnValue]:
|
||||
"""Authorize one declared operation before billing and request-body work."""
|
||||
|
||||
@wraps(view)
|
||||
def decorated(method: KnowledgeFSMethod, upstream_path: str) -> ResponseReturnValue:
|
||||
try:
|
||||
operation = get_knowledge_fs_operation(method, upstream_path)
|
||||
except KnowledgeFSRouteNotAllowedError as exc:
|
||||
raise NotFound() from exc
|
||||
|
||||
current_user, tenant_id = current_account_with_tenant()
|
||||
try:
|
||||
authorize_knowledge_fs_request(
|
||||
authorization = authorize_knowledge_fs_request(
|
||||
account=current_user,
|
||||
tenant_id=tenant_id,
|
||||
operation=operation,
|
||||
method=method,
|
||||
path=upstream_path,
|
||||
)
|
||||
except KnowledgeFSRouteNotAllowedError as exc:
|
||||
raise NotFound() from exc
|
||||
except KnowledgeFSAccessDeniedError as exc:
|
||||
_translate_proxy_error(exc, tenant_id=tenant_id)
|
||||
return view(method, upstream_path)
|
||||
return view(authorization)
|
||||
|
||||
return decorated
|
||||
|
||||
@@ -190,21 +198,26 @@ def _proxy_response(
|
||||
"""Expose raw content, status, and allowlisted headers from KnowledgeFS.
|
||||
|
||||
Raises:
|
||||
BadGateway: KnowledgeFS rejects the configured server credential.
|
||||
Forbidden: KnowledgeFS denies the account access to the requested resource.
|
||||
HTTPException: KnowledgeFS returns a status normalized by the operation contract.
|
||||
"""
|
||||
upstream = upstream_result.response
|
||||
if upstream.status_code == HTTPStatus.UNAUTHORIZED:
|
||||
mapped_status = dict(upstream_result.operation.error_status_map).get(upstream.status_code)
|
||||
if mapped_status is not None:
|
||||
upstream.close()
|
||||
logger.error(
|
||||
"KnowledgeFS rejected the Dify server credential with HTTP %s for tenant_id=%s",
|
||||
upstream.status_code,
|
||||
tenant_id,
|
||||
)
|
||||
raise BadGateway("KnowledgeFS authentication failed")
|
||||
if upstream.status_code == HTTPStatus.FORBIDDEN:
|
||||
upstream.close()
|
||||
raise Forbidden()
|
||||
description = "KnowledgeFS upstream request failed"
|
||||
if upstream.status_code == HTTPStatus.UNAUTHORIZED:
|
||||
description = "KnowledgeFS authentication failed"
|
||||
logger.error(
|
||||
"KnowledgeFS rejected the Dify server credential with HTTP %s for tenant_id=%s",
|
||||
upstream.status_code,
|
||||
tenant_id,
|
||||
)
|
||||
exception_type = default_exceptions.get(mapped_status)
|
||||
if exception_type is None:
|
||||
exception = HTTPException(description)
|
||||
exception.code = mapped_status
|
||||
raise exception
|
||||
raise exception_type(description)
|
||||
|
||||
allowed_header_names = dict.fromkeys(
|
||||
name.lower() for name in (*_RESPONSE_HEADER_ALLOWLIST, *contract_response_headers)
|
||||
@@ -237,26 +250,21 @@ def _proxy_response(
|
||||
return Response(content, status=upstream.status_code, headers=headers)
|
||||
|
||||
|
||||
def _proxy_request(method: KnowledgeFSMethod, upstream_path: str) -> Response:
|
||||
"""Forward the current raw request and return its filtered upstream response.
|
||||
|
||||
The call performs one outbound KnowledgeFS request. Integration failures are
|
||||
converted to Console HTTP exceptions for the outer JSON error adapter.
|
||||
"""
|
||||
def _proxy_current_request(
|
||||
*,
|
||||
method: KnowledgeFSMethod,
|
||||
tenant_id: str,
|
||||
forward: _KnowledgeFSRequestForwarder,
|
||||
) -> Response:
|
||||
"""Forward the current raw request through one preconfigured service entry."""
|
||||
if not dify_config.KNOWLEDGE_FS_ENABLED:
|
||||
raise NotFound()
|
||||
current_user, tenant_id = current_account_with_tenant()
|
||||
try:
|
||||
proxy_result = proxy_knowledge_fs_request(
|
||||
account=current_user,
|
||||
method=method,
|
||||
path=upstream_path,
|
||||
tenant_id=tenant_id,
|
||||
accept=request.headers.get("Accept"),
|
||||
content_type=request.content_type,
|
||||
query=request.query_string or None,
|
||||
body=_request_body() if method != "GET" else None,
|
||||
request_headers=request.headers,
|
||||
proxy_result = forward(
|
||||
request.headers.get("Accept"),
|
||||
request.content_type,
|
||||
request.query_string or None,
|
||||
_request_body() if method != "GET" else None,
|
||||
)
|
||||
except (
|
||||
KnowledgeFSConfigurationError,
|
||||
@@ -274,20 +282,99 @@ def _proxy_request(method: KnowledgeFSMethod, upstream_path: str) -> Response:
|
||||
)
|
||||
|
||||
|
||||
def _proxy_request(
|
||||
method: KnowledgeFSMethod,
|
||||
upstream_path: str,
|
||||
) -> Response:
|
||||
"""Authorize and forward the current request through the combined service use case."""
|
||||
if not dify_config.KNOWLEDGE_FS_ENABLED:
|
||||
raise NotFound()
|
||||
current_user, tenant_id = current_account_with_tenant()
|
||||
|
||||
def forward(
|
||||
accept: str | None,
|
||||
content_type: str | None,
|
||||
query: bytes | None,
|
||||
body: bytes | None,
|
||||
) -> KnowledgeFSUpstreamResponse:
|
||||
return proxy_knowledge_fs_request(
|
||||
account=current_user,
|
||||
method=method,
|
||||
path=upstream_path,
|
||||
tenant_id=tenant_id,
|
||||
accept=accept,
|
||||
content_type=content_type,
|
||||
query=query,
|
||||
body=body,
|
||||
request_headers=request.headers,
|
||||
)
|
||||
|
||||
return _proxy_current_request(method=method, tenant_id=tenant_id, forward=forward)
|
||||
|
||||
|
||||
def _proxy_authorized_request(authorization: KnowledgeFSAuthorization) -> Response:
|
||||
"""Forward the current request using one previously authorized operation capability.
|
||||
|
||||
Args:
|
||||
authorization: Request-scoped capability produced before billing and body parsing.
|
||||
|
||||
Returns:
|
||||
The filtered response returned by KnowledgeFS.
|
||||
|
||||
Raises:
|
||||
HTTPException: The integration is disabled or forwarding fails.
|
||||
"""
|
||||
operation = authorization.operation
|
||||
tenant_id = authorization.tenant_id
|
||||
|
||||
def forward(
|
||||
accept: str | None,
|
||||
content_type: str | None,
|
||||
query: bytes | None,
|
||||
body: bytes | None,
|
||||
) -> KnowledgeFSUpstreamResponse:
|
||||
return proxy_authorized_knowledge_fs_request(
|
||||
authorization=authorization,
|
||||
accept=accept,
|
||||
content_type=content_type,
|
||||
query=query,
|
||||
body=body,
|
||||
request_headers=request.headers,
|
||||
)
|
||||
|
||||
return _proxy_current_request(method=operation.method, tenant_id=tenant_id, forward=forward)
|
||||
|
||||
|
||||
@_knowledge_fs_enabled
|
||||
@_knowledge_fs_operation_access_required
|
||||
@cloud_edition_billing_rate_limit_check("knowledge")
|
||||
def _proxy_knowledge_fs_non_get(
|
||||
method: KnowledgeFSMethod,
|
||||
upstream_path: str,
|
||||
authorization: KnowledgeFSAuthorization,
|
||||
) -> ResponseReturnValue:
|
||||
"""Apply knowledge billing checks to one allowlisted non-GET operation."""
|
||||
return _proxy_request(method, upstream_path)
|
||||
return _proxy_authorized_request(authorization)
|
||||
|
||||
|
||||
@bp.route(
|
||||
"/knowledge-fs/<path:upstream_path>",
|
||||
methods=["GET", "OPTIONS"],
|
||||
methods=["OPTIONS"],
|
||||
provide_automatic_options=False,
|
||||
)
|
||||
@_console_api_errors
|
||||
@_knowledge_fs_enabled
|
||||
def proxy_knowledge_fs_options(upstream_path: str) -> ResponseReturnValue:
|
||||
"""Complete a CORS preflight only for an enabled Console operation."""
|
||||
requested_method = cast(KnowledgeFSMethod, request.headers.get("Access-Control-Request-Method", "").upper())
|
||||
try:
|
||||
get_knowledge_fs_operation(requested_method, upstream_path)
|
||||
except KnowledgeFSRouteNotAllowedError as exc:
|
||||
raise NotFound() from exc
|
||||
return Response(status=HTTPStatus.NO_CONTENT)
|
||||
|
||||
|
||||
@bp.route(
|
||||
"/knowledge-fs/<path:upstream_path>",
|
||||
methods=["GET"],
|
||||
provide_automatic_options=False,
|
||||
)
|
||||
@_console_api_errors
|
||||
|
||||
@@ -67,6 +67,15 @@ from services.plugin.plugin_parameter_service import PluginParameterService
|
||||
from services.plugin.plugin_permission_service import PluginPermissionService
|
||||
from services.tools.tools_transform_service import ToolTransformService
|
||||
|
||||
_PLUGIN_PACKAGE_UPLOAD_PARAMS = {
|
||||
"pkg": {
|
||||
"description": "Plugin package to upload",
|
||||
"in": "formData",
|
||||
"type": "file",
|
||||
"required": True,
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
class AutoUpgradeSettingsResponse(TypedDict):
|
||||
strategy_setting: TenantPluginAutoUpgradeStrategySetting
|
||||
@@ -645,6 +654,7 @@ class PluginAssetApi(Resource):
|
||||
|
||||
@console_ns.route("/workspaces/current/plugin/upload/pkg")
|
||||
class PluginUploadFromPkgApi(Resource):
|
||||
@console_ns.doc(consumes=["multipart/form-data"], params=_PLUGIN_PACKAGE_UPLOAD_PARAMS)
|
||||
@console_ns.response(200, "Success", console_ns.models[PluginDecodeResponse.__name__])
|
||||
@setup_required
|
||||
@login_required
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
from typing import Any, Self
|
||||
|
||||
from pydantic import AliasChoices, Field, computed_field
|
||||
from pydantic import AliasChoices, Field
|
||||
from sqlalchemy import select
|
||||
from werkzeug.exceptions import Forbidden
|
||||
|
||||
@@ -9,11 +9,13 @@ from controllers.common.schema import register_response_schema_models
|
||||
from controllers.web import web_ns
|
||||
from controllers.web.wraps import WebApiResource
|
||||
from extensions.ext_database import db
|
||||
from extensions.storage.storage_type import StorageType
|
||||
from fields.base import ResponseModel
|
||||
from libs.helper import build_icon_url
|
||||
from models.account import Tenant, TenantStatus
|
||||
from models.model import App, EndUser, Site
|
||||
from models.model import App, EndUser, IconType, Site
|
||||
from services.feature_service import FeatureModel, FeatureService
|
||||
from services.file_service import FileService
|
||||
|
||||
|
||||
class WebSiteResponse(ResponseModel):
|
||||
@@ -32,11 +34,7 @@ class WebSiteResponse(ResponseModel):
|
||||
prompt_public: bool | None = None
|
||||
show_workflow_steps: bool | None = None
|
||||
use_icon_as_answer_icon: bool | None = None
|
||||
|
||||
@computed_field(return_type=str | None) # type: ignore[prop-decorator]
|
||||
@property
|
||||
def icon_url(self) -> str | None:
|
||||
return build_icon_url(self.icon_type, self.icon)
|
||||
icon_url: str | None = None
|
||||
|
||||
|
||||
class WebModelConfigResponse(ResponseModel):
|
||||
@@ -88,6 +86,7 @@ class WebAppSiteResponse(ResponseModel):
|
||||
end_user_id: str | None,
|
||||
features: FeatureModel,
|
||||
can_replace_logo: bool,
|
||||
icon_url: str | None = None,
|
||||
) -> Self:
|
||||
custom_config = None
|
||||
if can_replace_logo:
|
||||
@@ -102,6 +101,7 @@ class WebAppSiteResponse(ResponseModel):
|
||||
)
|
||||
|
||||
site_response = WebSiteResponse.model_validate(site, from_attributes=True)
|
||||
site_response.icon_url = icon_url if icon_url is not None else build_icon_url(site.icon_type, site.icon)
|
||||
if features.billing.enabled and not features.webapp_copyright_enabled:
|
||||
site_response.copyright = None
|
||||
site_response.input_placeholder = None
|
||||
@@ -123,6 +123,15 @@ register_response_schema_models(
|
||||
)
|
||||
|
||||
|
||||
def _build_site_icon_url(*, site: Site, tenant_id: str) -> str | None:
|
||||
"""Use direct S3 URLs only in Cloud Mode and preserve preview URLs elsewhere."""
|
||||
if site.icon_type != IconType.IMAGE or not site.icon:
|
||||
return None
|
||||
if dify_config.EDITION == "CLOUD" and StorageType(dify_config.STORAGE_TYPE) == StorageType.S3:
|
||||
return FileService(db.engine).get_file_presigned_url(file_id=site.icon, tenant_id=tenant_id)
|
||||
return build_icon_url(site.icon_type, site.icon)
|
||||
|
||||
|
||||
@web_ns.route("/site")
|
||||
class AppSiteApi(WebApiResource):
|
||||
@web_ns.doc("Get App Site Info")
|
||||
@@ -159,4 +168,5 @@ class AppSiteApi(WebApiResource):
|
||||
end_user_id=end_user.id,
|
||||
features=features,
|
||||
can_replace_logo=features.can_replace_logo,
|
||||
icon_url=_build_site_icon_url(site=site, tenant_id=tenant.id),
|
||||
).model_dump(mode="json")
|
||||
|
||||
@@ -11,6 +11,7 @@ from werkzeug.exceptions import BadRequest, NotFound, Unauthorized
|
||||
|
||||
from constants import HEADER_NAME_APP_CODE
|
||||
from controllers.web.error import WebAppAuthAccessDeniedError, WebAppAuthRequiredError
|
||||
from core.logging.context import set_identity_context
|
||||
from extensions.ext_database import db
|
||||
from libs.passport import PassportService
|
||||
from libs.token import extract_webapp_passport
|
||||
@@ -28,6 +29,11 @@ def validate_jwt_token[**P, R](
|
||||
@wraps(view)
|
||||
def decorated(*args: P.args, **kwargs: P.kwargs) -> R:
|
||||
app_model, end_user = decode_jwt_token()
|
||||
set_identity_context(
|
||||
tenant_id=end_user.tenant_id,
|
||||
user_id=end_user.id,
|
||||
user_type=end_user.type or "end_user",
|
||||
)
|
||||
return view(app_model, end_user, *args, **kwargs)
|
||||
|
||||
return decorated
|
||||
|
||||
@@ -10,12 +10,13 @@ from sqlalchemy.orm import Session
|
||||
from core.agent.entities import AgentEntity, AgentToolEntity
|
||||
from core.app.app_config.features.file_upload.manager import FileUploadConfigManager
|
||||
from core.app.apps.agent_chat.app_config_manager import AgentChatAppConfig
|
||||
from core.app.apps.base_app_queue_manager import AppQueueManager
|
||||
from core.app.apps.base_app_queue_manager import AppQueueManager, PublishFrom
|
||||
from core.app.apps.base_app_runner import AppRunner
|
||||
from core.app.entities.app_invoke_entities import (
|
||||
AgentChatAppGenerateEntity,
|
||||
ModelConfigWithCredentialsEntity,
|
||||
)
|
||||
from core.app.entities.queue_entities import QueueUIPartEvent
|
||||
from core.app.file_access import DatabaseFileAccessController
|
||||
from core.callback_handler.agent_tool_callback_handler import DifyAgentCallbackHandler
|
||||
from core.callback_handler.index_tool_callback_handler import DatasetIndexToolCallbackHandler
|
||||
@@ -23,6 +24,12 @@ from core.memory.token_buffer_memory import TokenBufferMemory
|
||||
from core.model_manager import ModelInstance
|
||||
from core.prompt.utils.extract_thread_messages import extract_thread_messages
|
||||
from core.tools.__base.tool import Tool
|
||||
from core.tools.entities.ui_entities import (
|
||||
MessageUIPart,
|
||||
ToolUIMessage,
|
||||
build_ui_part_id,
|
||||
validate_tool_ui_message_batch,
|
||||
)
|
||||
from core.tools.tool_manager import ToolManager
|
||||
from core.tools.utils.dataset_retriever_tool import DatasetRetrieverTool
|
||||
from extensions.ext_database import db
|
||||
@@ -352,6 +359,34 @@ class BaseAgentRunner(AppRunner):
|
||||
db.session.commit()
|
||||
db.session.close()
|
||||
|
||||
def publish_ui_messages(self, *, namespace: str, ui_messages: list[ToolUIMessage]) -> None:
|
||||
"""Publish validated tool UI messages without adding them to agent observations."""
|
||||
try:
|
||||
validate_tool_ui_message_batch(ui_messages)
|
||||
except ValueError:
|
||||
logger.warning(
|
||||
"Ignored tool UI batch that exceeds queue publication limits",
|
||||
extra={"message_id": self.message.id, "namespace": namespace},
|
||||
exc_info=True,
|
||||
)
|
||||
return
|
||||
|
||||
sequences: dict[str, int] = {}
|
||||
for ui_message in ui_messages:
|
||||
part_id = build_ui_part_id(namespace, ui_message.surface_id)
|
||||
sequence = sequences.get(part_id, 0) + 1
|
||||
sequences[part_id] = sequence
|
||||
self.queue_manager.publish(
|
||||
QueueUIPartEvent(
|
||||
part=MessageUIPart.from_tool_ui_message(
|
||||
part_id=part_id,
|
||||
sequence=sequence,
|
||||
ui_message=ui_message,
|
||||
)
|
||||
),
|
||||
PublishFrom.APPLICATION_MANAGER,
|
||||
)
|
||||
|
||||
def organize_agent_history(self, prompt_messages: list[PromptMessage], *, session: Session) -> list[PromptMessage]:
|
||||
"""
|
||||
Organize agent history
|
||||
|
||||
@@ -235,6 +235,7 @@ class CotAgentRunner(BaseAgentRunner, ABC):
|
||||
# action is tool call, invoke tool
|
||||
tool_invoke_response, tool_invoke_meta = self._handle_invoke_action(
|
||||
session=session,
|
||||
agent_thought_id=agent_thought_id,
|
||||
action=scratchpad.action,
|
||||
tool_instances=tool_instances,
|
||||
message_file_ids=message_file_ids,
|
||||
@@ -306,9 +307,11 @@ class CotAgentRunner(BaseAgentRunner, ABC):
|
||||
tool_instances: Mapping[str, Tool],
|
||||
message_file_ids: list[str],
|
||||
trace_manager: TraceQueueManager | None = None,
|
||||
agent_thought_id: str | None = None,
|
||||
) -> tuple[str, ToolInvokeMeta]:
|
||||
"""
|
||||
handle invoke action
|
||||
:param agent_thought_id: namespace for UI parts emitted by this action
|
||||
:param action: action
|
||||
:param tool_instances: tool instances
|
||||
:param message_file_ids: message file ids
|
||||
@@ -331,7 +334,7 @@ class CotAgentRunner(BaseAgentRunner, ABC):
|
||||
pass
|
||||
|
||||
# invoke tool
|
||||
tool_invoke_response, message_files, tool_invoke_meta = ToolEngine.agent_invoke(
|
||||
invoke_result = ToolEngine.agent_invoke(
|
||||
session=session,
|
||||
tool=tool_instance,
|
||||
tool_parameters=tool_call_args,
|
||||
@@ -342,6 +345,13 @@ class CotAgentRunner(BaseAgentRunner, ABC):
|
||||
agent_tool_callback=self.agent_callback,
|
||||
trace_manager=trace_manager,
|
||||
)
|
||||
tool_invoke_response = invoke_result.observation
|
||||
message_files = invoke_result.message_files
|
||||
tool_invoke_meta = invoke_result.meta
|
||||
self.publish_ui_messages(
|
||||
namespace=agent_thought_id or self.message.id,
|
||||
ui_messages=invoke_result.ui_messages,
|
||||
)
|
||||
session.commit()
|
||||
session.close()
|
||||
|
||||
|
||||
@@ -251,7 +251,7 @@ class FunctionCallAgentRunner(BaseAgentRunner):
|
||||
}
|
||||
else:
|
||||
# invoke tool
|
||||
tool_invoke_response, message_files, tool_invoke_meta = ToolEngine.agent_invoke(
|
||||
invoke_result = ToolEngine.agent_invoke(
|
||||
session=session,
|
||||
tool=tool_instance,
|
||||
tool_parameters=tool_call_args,
|
||||
@@ -265,6 +265,13 @@ class FunctionCallAgentRunner(BaseAgentRunner):
|
||||
message_id=self.message.id,
|
||||
conversation_id=self.conversation.id,
|
||||
)
|
||||
tool_invoke_response = invoke_result.observation
|
||||
message_files = invoke_result.message_files
|
||||
tool_invoke_meta = invoke_result.meta
|
||||
self.publish_ui_messages(
|
||||
namespace=tool_call_id,
|
||||
ui_messages=invoke_result.ui_messages,
|
||||
)
|
||||
session.commit()
|
||||
session.close()
|
||||
# publish files
|
||||
|
||||
@@ -682,42 +682,27 @@ class AgentAppGenerator(MessageBasedAppGenerator):
|
||||
if draft_type == AgentConfigDraftType.DEBUG_BUILD.value
|
||||
else AgentConfigDraftType.DRAFT
|
||||
)
|
||||
if effective_draft_type == AgentConfigDraftType.DRAFT:
|
||||
from services.agent.composer_service import AgentComposerService
|
||||
|
||||
return AgentComposerService.get_or_create_normal_agent_draft(
|
||||
session=session,
|
||||
tenant_id=tenant_id,
|
||||
agent=agent,
|
||||
created_by=agent.updated_by or agent.created_by,
|
||||
)
|
||||
if not account_id:
|
||||
raise AgentAppGeneratorError("Build draft requires an account user")
|
||||
stmt = select(AgentConfigDraft).where(
|
||||
AgentConfigDraft.tenant_id == tenant_id,
|
||||
AgentConfigDraft.agent_id == agent.id,
|
||||
AgentConfigDraft.draft_type == effective_draft_type,
|
||||
AgentConfigDraft.draft_type == AgentConfigDraftType.DEBUG_BUILD,
|
||||
AgentConfigDraft.account_id == account_id,
|
||||
)
|
||||
if effective_draft_type == AgentConfigDraftType.DEBUG_BUILD:
|
||||
if not account_id:
|
||||
raise AgentAppGeneratorError("Build draft requires an account user")
|
||||
stmt = stmt.where(AgentConfigDraft.account_id == account_id)
|
||||
else:
|
||||
stmt = stmt.where(AgentConfigDraft.account_id.is_(None))
|
||||
draft = session.scalar(stmt.order_by(AgentConfigDraft.updated_at.desc()).limit(1))
|
||||
if draft is not None:
|
||||
return draft
|
||||
if effective_draft_type == AgentConfigDraftType.DEBUG_BUILD:
|
||||
raise AgentAppGeneratorError("Agent build draft not found")
|
||||
_, snapshot, agent_soul = AgentAppGenerator._resolve_agent_by_id(
|
||||
tenant_id=tenant_id,
|
||||
agent_id=agent.id,
|
||||
snapshot_id=agent.active_config_snapshot_id,
|
||||
session=session,
|
||||
)
|
||||
draft = AgentConfigDraft(
|
||||
tenant_id=tenant_id,
|
||||
agent_id=agent.id,
|
||||
draft_type=AgentConfigDraftType.DRAFT,
|
||||
account_id=None,
|
||||
draft_owner_key="",
|
||||
base_snapshot_id=snapshot.id,
|
||||
config_snapshot=agent_soul,
|
||||
created_by=agent.created_by,
|
||||
updated_by=agent.updated_by,
|
||||
)
|
||||
session.add(draft)
|
||||
session.flush()
|
||||
return draft
|
||||
raise AgentAppGeneratorError("Agent build draft not found")
|
||||
|
||||
@staticmethod
|
||||
def _resolve_agent_by_id(
|
||||
|
||||
@@ -52,8 +52,16 @@ from core.app.entities.queue_entities import (
|
||||
QueueAgentThoughtEvent,
|
||||
QueueLLMChunkEvent,
|
||||
QueueMessageEndEvent,
|
||||
QueueUIPartEvent,
|
||||
)
|
||||
from core.repositories.human_input_repository import HumanInputFormRepository, HumanInputFormRepositoryImpl
|
||||
from core.tools.entities.ui_entities import (
|
||||
MessageUIPart,
|
||||
ToolUIMessage,
|
||||
build_ui_part_id,
|
||||
parse_tool_ui_messages,
|
||||
validate_tool_ui_message_batch,
|
||||
)
|
||||
from core.workflow.nodes.agent_v2.ask_human_hitl import AskHumanFormBuildError, create_ask_human_form
|
||||
from core.workflow.nodes.agent_v2.ask_human_resume import build_deferred_tool_results, resolve_ask_human_form
|
||||
from extensions.ext_database import db
|
||||
@@ -435,9 +443,29 @@ class _AgentProcessRecorder:
|
||||
content = part.get("content")
|
||||
if content is None:
|
||||
content = part
|
||||
self._record_tool_observation(tool_call_id=tool_call_id, tool_name=tool_name, observation=content)
|
||||
thought_id = self._record_tool_observation(
|
||||
tool_call_id=tool_call_id,
|
||||
tool_name=tool_name,
|
||||
observation=content,
|
||||
)
|
||||
|
||||
def _record_tool_observation(self, *, tool_call_id: str | None, tool_name: str | None, observation: Any) -> None:
|
||||
metadata = part.get("metadata")
|
||||
if not isinstance(metadata, Mapping) or "dify_ui_messages" not in metadata:
|
||||
return
|
||||
try:
|
||||
ui_messages = parse_tool_ui_messages(metadata["dify_ui_messages"])
|
||||
except (TypeError, ValueError):
|
||||
logger.warning("Ignored invalid dify_ui_messages metadata from Agent backend", exc_info=True)
|
||||
return
|
||||
self._publish_ui_messages(namespace=tool_call_id or thought_id, ui_messages=ui_messages)
|
||||
|
||||
def _record_tool_observation(
|
||||
self,
|
||||
*,
|
||||
tool_call_id: str | None,
|
||||
tool_name: str | None,
|
||||
observation: Any,
|
||||
) -> str:
|
||||
self._close_thinking_segments()
|
||||
thought_id = self._lookup_observation_thought(tool_call_id=tool_call_id, tool_name=tool_name)
|
||||
if thought_id is None:
|
||||
@@ -445,6 +473,34 @@ class _AgentProcessRecorder:
|
||||
else:
|
||||
self._mark_tool_observed(thought_id)
|
||||
self._update_thought(thought_id, observation=_json_or_text(observation))
|
||||
return thought_id
|
||||
|
||||
def _publish_ui_messages(self, *, namespace: str, ui_messages: list[ToolUIMessage]) -> None:
|
||||
try:
|
||||
validate_tool_ui_message_batch(ui_messages)
|
||||
except ValueError:
|
||||
logger.warning(
|
||||
"Ignored Agent backend UI batch that exceeds queue publication limits",
|
||||
extra={"message_id": self._message_id, "namespace": namespace},
|
||||
exc_info=True,
|
||||
)
|
||||
return
|
||||
|
||||
sequences: dict[str, int] = {}
|
||||
for ui_message in ui_messages:
|
||||
part_id = build_ui_part_id(namespace, ui_message.surface_id)
|
||||
sequence = sequences.get(part_id, 0) + 1
|
||||
sequences[part_id] = sequence
|
||||
self._queue_manager.publish(
|
||||
QueueUIPartEvent(
|
||||
part=MessageUIPart.from_tool_ui_message(
|
||||
part_id=part_id,
|
||||
sequence=sequence,
|
||||
ui_message=ui_message,
|
||||
)
|
||||
),
|
||||
PublishFrom.APPLICATION_MANAGER,
|
||||
)
|
||||
|
||||
def _lookup_tool_thought(self, *, index: int, tool_call_id: str | None) -> str | None:
|
||||
if tool_call_id and tool_call_id in self._tool_by_call_id:
|
||||
|
||||
@@ -7,6 +7,7 @@ from pydantic import BaseModel, ConfigDict, Field
|
||||
|
||||
from core.app.entities.agent_strategy import AgentStrategyInfo
|
||||
from core.rag.entities import RetrievalSourceMetadata
|
||||
from core.tools.entities.ui_entities import MessageUIPart
|
||||
from core.workflow.nodes.human_input.pause_reason import PauseReason
|
||||
from graphon.entities import WorkflowStartReason
|
||||
from graphon.enums import NodeType, WorkflowNodeExecutionMetadataKey
|
||||
@@ -43,6 +44,7 @@ class QueueEvent(StrEnum):
|
||||
REASONING_CHUNK = "reasoning_chunk"
|
||||
ANNOTATION_REPLY = "annotation_reply"
|
||||
AGENT_THOUGHT = "agent_thought"
|
||||
UI_PART = "ui_part"
|
||||
MESSAGE_FILE = "message_file"
|
||||
AGENT_LOG = "agent_log"
|
||||
ERROR = "error"
|
||||
@@ -457,6 +459,13 @@ class QueueAgentThoughtEvent(AppQueueEvent):
|
||||
agent_thought_id: str
|
||||
|
||||
|
||||
class QueueUIPartEvent(AppQueueEvent):
|
||||
"""A validated tool UI surface revision."""
|
||||
|
||||
event: QueueEvent = QueueEvent.UI_PART
|
||||
part: MessageUIPart
|
||||
|
||||
|
||||
class QueueMessageFileEvent(AppQueueEvent):
|
||||
"""
|
||||
QueueAgentThoughtEvent entity
|
||||
|
||||
@@ -2,10 +2,11 @@ from collections.abc import Mapping, Sequence
|
||||
from enum import StrEnum
|
||||
from typing import Any, Literal
|
||||
|
||||
from pydantic import BaseModel, ConfigDict, Field, JsonValue
|
||||
from pydantic import BaseModel, ConfigDict, Field, JsonValue, field_validator
|
||||
|
||||
from core.app.entities.agent_strategy import AgentStrategyInfo
|
||||
from core.rag.entities import RetrievalSourceMetadata
|
||||
from core.tools.entities.ui_entities import MessageUIPart, validate_ui_part_batch
|
||||
from core.workflow.nodes.human_input.entities import FormInputConfig, UserActionConfig
|
||||
from core.workflow.nodes.human_input.pause_reason import DifyHITLEventType
|
||||
from graphon.entities import WorkflowStartReason
|
||||
@@ -30,6 +31,14 @@ class TaskStateMetadata(BaseModel):
|
||||
reasoning: dict[str, str] = Field(default_factory=dict)
|
||||
"""reasoning_content per LLM node id (separated mode), accumulated across iteration/loop
|
||||
passes for that node; persisted to message_metadata"""
|
||||
ui_parts: list[MessageUIPart] = Field(default_factory=list)
|
||||
"""Latest validated revision of each tool UI part; persisted to message metadata."""
|
||||
|
||||
@field_validator("ui_parts")
|
||||
@classmethod
|
||||
def _validate_ui_parts(cls, value: list[MessageUIPart]) -> list[MessageUIPart]:
|
||||
validate_ui_part_batch(value)
|
||||
return value
|
||||
|
||||
|
||||
class TaskState(BaseModel):
|
||||
@@ -73,6 +82,7 @@ class StreamEvent(StrEnum):
|
||||
MESSAGE_FILE = "message_file"
|
||||
MESSAGE_REPLACE = "message_replace"
|
||||
AGENT_THOUGHT = "agent_thought"
|
||||
UI_PART = "ui_part"
|
||||
AGENT_MESSAGE = "agent_message"
|
||||
WORKFLOW_STARTED = "workflow_started"
|
||||
WORKFLOW_PAUSED = "workflow_paused"
|
||||
@@ -192,6 +202,14 @@ class AgentThoughtStreamResponse(StreamResponse):
|
||||
message_files: list[str] | None = None
|
||||
|
||||
|
||||
class UIPartStreamResponse(StreamResponse):
|
||||
"""A tool-owned UI surface revision."""
|
||||
|
||||
event: StreamEvent = StreamEvent.UI_PART
|
||||
id: str
|
||||
part: MessageUIPart
|
||||
|
||||
|
||||
class AgentMessageStreamResponse(StreamResponse):
|
||||
"""
|
||||
AgentMessageStreamResponse entity
|
||||
|
||||
@@ -26,6 +26,7 @@ from core.app.entities.queue_entities import (
|
||||
QueuePingEvent,
|
||||
QueueRetrieverResourcesEvent,
|
||||
QueueStopEvent,
|
||||
QueueUIPartEvent,
|
||||
)
|
||||
from core.app.entities.task_entities import (
|
||||
AgentMessageStreamResponse,
|
||||
@@ -41,6 +42,7 @@ from core.app.entities.task_entities import (
|
||||
MessageEndStreamResponse,
|
||||
StreamEvent,
|
||||
StreamResponse,
|
||||
UIPartStreamResponse,
|
||||
)
|
||||
from core.app.task_pipeline.based_generate_task_pipeline import BasedGenerateTaskPipeline
|
||||
from core.app.task_pipeline.message_cycle_manager import MessageCycleManager
|
||||
@@ -52,6 +54,7 @@ from core.ops.entities.trace_entity import TraceTaskName
|
||||
from core.ops.ops_trace_manager import TraceQueueManager, TraceTask
|
||||
from core.prompt.utils.prompt_message_util import PromptMessageUtil
|
||||
from core.prompt.utils.prompt_template_parser import PromptTemplateParser
|
||||
from core.tools.entities.ui_entities import upsert_ui_part
|
||||
from events.message_event import message_was_created
|
||||
from graphon.file import FileTransferMethod
|
||||
from graphon.model_runtime.entities.llm_entities import LLMResult, LLMResultChunk, LLMResultChunkDelta, LLMUsage
|
||||
@@ -311,6 +314,10 @@ class EasyUIBasedGenerateTaskPipeline(BasedGenerateTaskPipeline[EasyUIAppGenerat
|
||||
agent_thought_response = self._agent_thought_to_stream_response(event)
|
||||
if agent_thought_response is not None:
|
||||
yield agent_thought_response
|
||||
case QueueUIPartEvent():
|
||||
ui_part_response = self._ui_part_to_stream_response(event)
|
||||
if ui_part_response is not None:
|
||||
yield ui_part_response
|
||||
case QueueMessageFileEvent():
|
||||
response = self._message_cycle_manager.message_file_to_stream_response(event)
|
||||
if response:
|
||||
@@ -526,6 +533,27 @@ class EasyUIBasedGenerateTaskPipeline(BasedGenerateTaskPipeline[EasyUIAppGenerat
|
||||
task_id=self._application_generate_entity.task_id, id=message_id, answer=answer
|
||||
)
|
||||
|
||||
def _ui_part_to_stream_response(self, event: QueueUIPartEvent) -> UIPartStreamResponse | None:
|
||||
"""Upsert a UI part revision into persistent metadata and stream it."""
|
||||
try:
|
||||
updated_parts = upsert_ui_part(self._task_state.metadata.ui_parts, event.part)
|
||||
except ValueError:
|
||||
logger.warning(
|
||||
"Ignored UI part that exceeds assistant message limits",
|
||||
extra={"message_id": self._message_id, "part_id": event.part.part_id},
|
||||
exc_info=True,
|
||||
)
|
||||
return None
|
||||
if updated_parts is None:
|
||||
return None
|
||||
self._task_state.metadata.ui_parts = updated_parts
|
||||
|
||||
return UIPartStreamResponse(
|
||||
task_id=self._application_generate_entity.task_id,
|
||||
id=self._message_id,
|
||||
part=event.part,
|
||||
)
|
||||
|
||||
def _agent_thought_to_stream_response(self, event: QueueAgentThoughtEvent) -> AgentThoughtStreamResponse | None:
|
||||
"""
|
||||
Agent thought to stream response.
|
||||
|
||||
+46
-44
@@ -21,6 +21,7 @@ from core.model_manager import ModelInstance, ModelManager
|
||||
from core.rag.cleaner.clean_processor import CleanProcessor
|
||||
from core.rag.datasource.keyword.keyword_factory import Keyword
|
||||
from core.rag.docstore.dataset_docstore import DatasetDocumentStore
|
||||
from core.rag.embedding.token_counter import calculate_segment_token_counts
|
||||
from core.rag.extractor.entity.datasource_type import DatasourceType
|
||||
from core.rag.extractor.entity.extract_setting import ExtractSetting, NotionInfo, WebsiteInfo
|
||||
from core.rag.index_processor.constant.index_type import IndexStructureType, IndexTechniqueType
|
||||
@@ -113,17 +114,25 @@ class IndexingRunner:
|
||||
current_user=current_user,
|
||||
session=session,
|
||||
)
|
||||
token_counts = calculate_segment_token_counts(dataset=dataset, documents=documents)
|
||||
total_tokens = sum(token_counts)
|
||||
# save segment
|
||||
self._load_segments(dataset, requeried_document, documents, session)
|
||||
self._load_segments(
|
||||
session=session,
|
||||
dataset=dataset,
|
||||
dataset_document=requeried_document,
|
||||
documents=documents,
|
||||
token_counts=token_counts,
|
||||
)
|
||||
session.commit()
|
||||
|
||||
# load
|
||||
self._load(
|
||||
index_processor=index_processor,
|
||||
session=session,
|
||||
dataset=dataset,
|
||||
dataset_document=requeried_document,
|
||||
documents=documents,
|
||||
session=session,
|
||||
total_tokens=total_tokens,
|
||||
)
|
||||
except DocumentIsPausedError:
|
||||
raise DocumentIsPausedError(f"Document paused, document id: {document_id}")
|
||||
@@ -190,17 +199,25 @@ class IndexingRunner:
|
||||
current_user=current_user,
|
||||
session=session,
|
||||
)
|
||||
token_counts = calculate_segment_token_counts(dataset=dataset, documents=documents)
|
||||
total_tokens = sum(token_counts)
|
||||
# save segment
|
||||
self._load_segments(dataset, requeried_document, documents, session)
|
||||
self._load_segments(
|
||||
session=session,
|
||||
dataset=dataset,
|
||||
dataset_document=requeried_document,
|
||||
documents=documents,
|
||||
token_counts=token_counts,
|
||||
)
|
||||
session.commit()
|
||||
|
||||
# load
|
||||
self._load(
|
||||
index_processor=index_processor,
|
||||
session=session,
|
||||
dataset=dataset,
|
||||
dataset_document=requeried_document,
|
||||
documents=documents,
|
||||
session=session,
|
||||
total_tokens=total_tokens,
|
||||
)
|
||||
except DocumentIsPausedError:
|
||||
raise DocumentIsPausedError(f"Document paused, document id: {document_id}")
|
||||
@@ -225,7 +242,7 @@ class IndexingRunner:
|
||||
if not dataset:
|
||||
raise ValueError("no dataset found")
|
||||
|
||||
# get exist document_segment list and delete
|
||||
# get existing document segments
|
||||
document_segments = session.scalars(
|
||||
select(DocumentSegment).where(
|
||||
DocumentSegment.dataset_id == dataset.id,
|
||||
@@ -264,15 +281,15 @@ class IndexingRunner:
|
||||
child_documents.append(child_document)
|
||||
document.children = child_documents
|
||||
documents.append(document)
|
||||
# Preserve the full document total even when only incomplete segments are re-indexed.
|
||||
total_tokens = sum(document_segment.tokens for document_segment in document_segments)
|
||||
# build index
|
||||
index_type = requeried_document.doc_form
|
||||
index_processor = IndexProcessorFactory(index_type).init_index_processor()
|
||||
self._load(
|
||||
index_processor=index_processor,
|
||||
session=session,
|
||||
dataset=dataset,
|
||||
dataset_document=requeried_document,
|
||||
documents=documents,
|
||||
session=session,
|
||||
total_tokens=total_tokens,
|
||||
)
|
||||
except DocumentIsPausedError:
|
||||
raise DocumentIsPausedError(f"Document paused, document id: {document_id}")
|
||||
@@ -601,28 +618,16 @@ class IndexingRunner:
|
||||
|
||||
def _load(
|
||||
self,
|
||||
index_processor: BaseIndexProcessor,
|
||||
session: Session,
|
||||
dataset: Dataset,
|
||||
dataset_document: DatasetDocument,
|
||||
documents: list[Document],
|
||||
session: Session,
|
||||
):
|
||||
"""
|
||||
insert index and update document/segment status to completed
|
||||
"""
|
||||
total_tokens: int,
|
||||
) -> None:
|
||||
"""Build indexes and mark the document complete using the token total computed before hash sharding."""
|
||||
|
||||
embedding_model_instance = None
|
||||
if dataset.indexing_technique == IndexTechniqueType.HIGH_QUALITY:
|
||||
embedding_model_instance = self._get_model_manager(dataset.tenant_id).get_model_instance(
|
||||
tenant_id=dataset.tenant_id,
|
||||
provider=dataset.embedding_model_provider,
|
||||
model_type=ModelType.TEXT_EMBEDDING,
|
||||
model=dataset.embedding_model,
|
||||
)
|
||||
|
||||
# chunk nodes by chunk size
|
||||
# Build indexes using the existing hash-based worker groups.
|
||||
indexing_start_at = time.perf_counter()
|
||||
tokens = 0
|
||||
create_keyword_thread = None
|
||||
if (
|
||||
dataset_document.doc_form != IndexStructureType.PARENT_CHILD_INDEX
|
||||
@@ -659,12 +664,11 @@ class IndexingRunner:
|
||||
chunk_documents,
|
||||
dataset.id,
|
||||
dataset_document.id,
|
||||
embedding_model_instance,
|
||||
)
|
||||
)
|
||||
|
||||
for future in futures:
|
||||
tokens += future.result()
|
||||
future.result()
|
||||
if (
|
||||
dataset_document.doc_form != IndexStructureType.PARENT_CHILD_INDEX
|
||||
and dataset.indexing_technique == IndexTechniqueType.ECONOMY
|
||||
@@ -679,7 +683,7 @@ class IndexingRunner:
|
||||
document_id=dataset_document.id,
|
||||
after_indexing_status=IndexingStatus.COMPLETED,
|
||||
extra_update_params={
|
||||
DatasetDocument.tokens: tokens,
|
||||
DatasetDocument.tokens: total_tokens,
|
||||
DatasetDocument.completed_at: naive_utc_now(),
|
||||
DatasetDocument.indexing_latency: indexing_end_at - indexing_start_at,
|
||||
DatasetDocument.error: None,
|
||||
@@ -720,8 +724,7 @@ class IndexingRunner:
|
||||
chunk_documents: list[Document],
|
||||
dataset_id: str,
|
||||
dataset_document_id: str,
|
||||
embedding_model_instance: ModelInstance | None,
|
||||
):
|
||||
) -> None:
|
||||
with flask_app.app_context():
|
||||
with session_factory.create_session() as session:
|
||||
dataset = session.get(Dataset, dataset_id)
|
||||
@@ -735,11 +738,6 @@ class IndexingRunner:
|
||||
# check document is paused
|
||||
self._check_document_paused_status(dataset_document.id)
|
||||
|
||||
tokens = 0
|
||||
if embedding_model_instance:
|
||||
page_content_list = [document.page_content for document in chunk_documents]
|
||||
tokens += sum(embedding_model_instance.get_text_embedding_num_tokens(page_content_list))
|
||||
|
||||
multimodal_documents = []
|
||||
for document in chunk_documents:
|
||||
if document.attachments and dataset.is_multimodal:
|
||||
@@ -773,8 +771,6 @@ class IndexingRunner:
|
||||
|
||||
session.commit()
|
||||
|
||||
return tokens
|
||||
|
||||
@staticmethod
|
||||
def _check_document_paused_status(document_id: str):
|
||||
indexing_cache_key = f"document_{document_id}_is_paused"
|
||||
@@ -864,8 +860,14 @@ class IndexingRunner:
|
||||
return documents
|
||||
|
||||
def _load_segments(
|
||||
self, dataset: Dataset, dataset_document: DatasetDocument, documents: list[Document], session: Session
|
||||
):
|
||||
self,
|
||||
session: Session,
|
||||
dataset: Dataset,
|
||||
dataset_document: DatasetDocument,
|
||||
documents: list[Document],
|
||||
token_counts: list[int],
|
||||
) -> None:
|
||||
"""Persist transformed documents and their precomputed token counts before indexing starts."""
|
||||
# save node to document segment
|
||||
doc_store = DatasetDocumentStore(
|
||||
dataset=dataset, user_id=dataset_document.created_by, document_id=dataset_document.id
|
||||
@@ -873,9 +875,10 @@ class IndexingRunner:
|
||||
|
||||
# add document segments
|
||||
doc_store.add_documents(
|
||||
session=session,
|
||||
docs=documents,
|
||||
save_child=dataset_document.doc_form == IndexStructureType.PARENT_CHILD_INDEX,
|
||||
session=session,
|
||||
token_counts=token_counts,
|
||||
)
|
||||
|
||||
# update document status to indexing
|
||||
@@ -900,7 +903,6 @@ class IndexingRunner:
|
||||
DocumentSegment.indexing_at: naive_utc_now(),
|
||||
},
|
||||
)
|
||||
pass
|
||||
|
||||
|
||||
class DocumentIsPausedError(Exception):
|
||||
|
||||
@@ -6,9 +6,21 @@ using Python's contextvars for thread-safe and async-safe storage.
|
||||
|
||||
import uuid
|
||||
from contextvars import ContextVar
|
||||
from typing import NamedTuple
|
||||
|
||||
|
||||
class IdentityContext(NamedTuple):
|
||||
"""Immutable identity values captured for logging."""
|
||||
|
||||
tenant_id: str
|
||||
user_id: str
|
||||
user_type: str
|
||||
|
||||
|
||||
_request_id: ContextVar[str] = ContextVar("log_request_id", default="")
|
||||
_trace_id: ContextVar[str] = ContextVar("log_trace_id", default="")
|
||||
_EMPTY_IDENTITY_CONTEXT = IdentityContext(tenant_id="", user_id="", user_type="")
|
||||
_identity: ContextVar[IdentityContext] = ContextVar("log_identity", default=_EMPTY_IDENTITY_CONTEXT)
|
||||
|
||||
|
||||
def get_request_id() -> str:
|
||||
@@ -21,15 +33,35 @@ def get_trace_id() -> str:
|
||||
return _trace_id.get()
|
||||
|
||||
|
||||
def get_identity_context() -> IdentityContext:
|
||||
"""Get the immutable tenant, user, and user-type snapshot for logging."""
|
||||
return _identity.get()
|
||||
|
||||
|
||||
def set_identity_context(
|
||||
*, tenant_id: str | None = None, user_id: str | None = None, user_type: str | None = None
|
||||
) -> None:
|
||||
"""Set primitive identity values already resolved by an authentication boundary."""
|
||||
_identity.set(
|
||||
IdentityContext(
|
||||
tenant_id=tenant_id or "",
|
||||
user_id=user_id or "",
|
||||
user_type=user_type or "",
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
def init_request_context() -> None:
|
||||
"""Initialize request context. Call at start of each request."""
|
||||
"""Initialize request context and discard identity left by earlier work."""
|
||||
req_id = uuid.uuid4().hex[:10]
|
||||
trace_id = uuid.uuid5(uuid.NAMESPACE_DNS, req_id).hex
|
||||
_request_id.set(req_id)
|
||||
_trace_id.set(trace_id)
|
||||
_identity.set(_EMPTY_IDENTITY_CONTEXT)
|
||||
|
||||
|
||||
def clear_request_context() -> None:
|
||||
"""Clear request context. Call at end of request (optional)."""
|
||||
"""Clear request context at a request or task lifecycle boundary."""
|
||||
_request_id.set("")
|
||||
_trace_id.set("")
|
||||
_identity.set(_EMPTY_IDENTITY_CONTEXT)
|
||||
|
||||
@@ -4,10 +4,7 @@ import contextlib
|
||||
import logging
|
||||
from typing import override
|
||||
|
||||
import flask
|
||||
|
||||
from core.logging.context import get_request_id, get_trace_id
|
||||
from core.logging.structured_formatter import IdentityDict
|
||||
from core.logging.context import get_identity_context, get_request_id, get_trace_id
|
||||
|
||||
|
||||
class TraceContextFilter(logging.Filter):
|
||||
@@ -51,49 +48,16 @@ class TraceContextFilter(logging.Filter):
|
||||
|
||||
|
||||
class IdentityContextFilter(logging.Filter):
|
||||
"""
|
||||
Filter that adds user identity context to log records.
|
||||
Extracts tenant_id, user_id, and user_type from Flask-Login current_user.
|
||||
"""Add an identity snapshot without invoking authentication or database work.
|
||||
|
||||
Logging can run while other libraries hold internal locks, so this filter must
|
||||
only read primitive ContextVar values populated by authentication boundaries.
|
||||
"""
|
||||
|
||||
@override
|
||||
def filter(self, record: logging.LogRecord) -> bool:
|
||||
identity = self._extract_identity()
|
||||
record.tenant_id = identity.get("tenant_id", "")
|
||||
record.user_id = identity.get("user_id", "")
|
||||
record.user_type = identity.get("user_type", "")
|
||||
identity = get_identity_context()
|
||||
record.tenant_id = identity.tenant_id
|
||||
record.user_id = identity.user_id
|
||||
record.user_type = identity.user_type
|
||||
return True
|
||||
|
||||
def _extract_identity(self) -> IdentityDict:
|
||||
"""Extract identity from current_user if in request context."""
|
||||
try:
|
||||
if not flask.has_request_context():
|
||||
return {}
|
||||
from flask_login import current_user
|
||||
|
||||
# Check if user is authenticated using the proxy
|
||||
if not current_user.is_authenticated:
|
||||
return {}
|
||||
|
||||
# Access the underlying user object
|
||||
user = current_user
|
||||
|
||||
from models import Account
|
||||
from models.model import EndUser
|
||||
|
||||
identity: IdentityDict = {}
|
||||
|
||||
match user:
|
||||
case Account():
|
||||
if user.current_tenant_id:
|
||||
identity["tenant_id"] = user.current_tenant_id
|
||||
identity["user_id"] = user.id
|
||||
identity["user_type"] = "account"
|
||||
case EndUser():
|
||||
identity["tenant_id"] = user.tenant_id
|
||||
identity["user_id"] = user.id
|
||||
identity["user_type"] = user.type or "end_user"
|
||||
|
||||
return identity
|
||||
except Exception:
|
||||
return {}
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
import inspect
|
||||
import json
|
||||
import logging
|
||||
from collections.abc import Callable, Generator
|
||||
from collections.abc import Callable, Generator, Mapping
|
||||
from typing import Any, cast
|
||||
from urllib.parse import unquote
|
||||
|
||||
@@ -23,6 +23,7 @@ from core.plugin.impl.exc import (
|
||||
PluginLLMPollingUnsupportedError,
|
||||
PluginNotFoundError,
|
||||
PluginPermissionDeniedError,
|
||||
PluginRuntimeError,
|
||||
PluginUniqueIdentifierError,
|
||||
)
|
||||
from core.trigger.errors import (
|
||||
@@ -375,6 +376,18 @@ class BasePluginClient:
|
||||
# type `PluginLLMPollingUnsupportedError`.
|
||||
case PluginLLMPollingUnsupportedError.__name__:
|
||||
raise PluginLLMPollingUnsupportedError(description=error_object.get("message"))
|
||||
case PluginRuntimeError.__name__:
|
||||
args = error_object.get("args")
|
||||
lambda_request_id = args.get("request_id") if isinstance(args, Mapping) else None
|
||||
if not isinstance(lambda_request_id, str):
|
||||
lambda_request_id = None
|
||||
runtime_message = error_object.get("message")
|
||||
if not isinstance(runtime_message, str):
|
||||
runtime_message = "Plugin runtime request failed"
|
||||
raise PluginRuntimeError(
|
||||
description=runtime_message,
|
||||
lambda_request_id=lambda_request_id,
|
||||
)
|
||||
case _:
|
||||
raise PluginInvokeError(description=message)
|
||||
case PluginDaemonInternalServerError.__name__:
|
||||
|
||||
@@ -49,6 +49,18 @@ class PluginDaemonBadRequestError(PluginDaemonClientSideError):
|
||||
description: str = "Bad Request"
|
||||
|
||||
|
||||
class PluginRuntimeError(PluginDaemonInternalError):
|
||||
"""A plugin runtime failed before it could return a valid plugin response."""
|
||||
|
||||
lambda_request_id: str | None
|
||||
|
||||
def __init__(self, description: str, lambda_request_id: str | None = None) -> None:
|
||||
self.lambda_request_id = lambda_request_id
|
||||
if lambda_request_id:
|
||||
description = description.replace(f"RequestId: {lambda_request_id} Error: ", "", 1)
|
||||
super().__init__(description)
|
||||
|
||||
|
||||
class PluginInvokeError(PluginDaemonClientSideError, ValueError):
|
||||
description: str = "Invoke Error"
|
||||
|
||||
|
||||
@@ -6,10 +6,7 @@ from typing import Any
|
||||
from sqlalchemy import delete, func, select
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from core.model_manager import ModelManager
|
||||
from core.rag.index_processor.constant.index_type import IndexTechniqueType
|
||||
from core.rag.models.document import AttachmentDocument, Document
|
||||
from graphon.model_runtime.entities.model_entities import ModelType
|
||||
from models.dataset import ChildChunk, Dataset, DocumentSegment, SegmentAttachmentBinding
|
||||
from models.enums import SegmentType
|
||||
|
||||
@@ -69,34 +66,22 @@ class DatasetDocumentStore:
|
||||
|
||||
def add_documents(
|
||||
self,
|
||||
docs: Sequence[Document],
|
||||
session: Session,
|
||||
docs: Sequence[Document],
|
||||
token_counts: list[int],
|
||||
allow_update: bool = True,
|
||||
save_child: bool = False,
|
||||
):
|
||||
) -> None:
|
||||
document_token_pairs = list(zip(docs, token_counts, strict=True))
|
||||
|
||||
max_position = session.scalar(
|
||||
select(func.max(DocumentSegment.position)).where(DocumentSegment.document_id == self._document_id)
|
||||
)
|
||||
|
||||
if max_position is None:
|
||||
max_position = 0
|
||||
embedding_model = None
|
||||
if self._dataset.indexing_technique == IndexTechniqueType.HIGH_QUALITY:
|
||||
model_manager = ModelManager.for_tenant(tenant_id=self._dataset.tenant_id)
|
||||
embedding_model = model_manager.get_model_instance(
|
||||
tenant_id=self._dataset.tenant_id,
|
||||
provider=self._dataset.embedding_model_provider,
|
||||
model_type=ModelType.TEXT_EMBEDDING,
|
||||
model=self._dataset.embedding_model,
|
||||
)
|
||||
|
||||
if embedding_model:
|
||||
page_content_list = [doc.page_content for doc in docs]
|
||||
tokens_list = embedding_model.get_text_embedding_num_tokens(page_content_list)
|
||||
else:
|
||||
tokens_list = [0] * len(docs)
|
||||
|
||||
for doc, tokens in zip(docs, tokens_list):
|
||||
for doc, tokens in document_token_pairs:
|
||||
if not isinstance(doc, Document):
|
||||
raise ValueError("doc must be a Document")
|
||||
|
||||
|
||||
@@ -0,0 +1,25 @@
|
||||
"""Token counting for document segments."""
|
||||
|
||||
from core.model_manager import ModelManager
|
||||
from core.rag.index_processor.constant.index_type import IndexTechniqueType
|
||||
from core.rag.models.document import Document
|
||||
from graphon.model_runtime.entities.model_entities import ModelType
|
||||
from models.dataset import Dataset
|
||||
|
||||
|
||||
def calculate_segment_token_counts(dataset: Dataset, documents: list[Document]) -> list[int]:
|
||||
"""Return one token count per document, invoking the embedding model only for high-quality indexes."""
|
||||
if not documents:
|
||||
return []
|
||||
|
||||
if dataset.indexing_technique != IndexTechniqueType.HIGH_QUALITY:
|
||||
return [0] * len(documents)
|
||||
|
||||
model_manager = ModelManager.for_tenant(tenant_id=dataset.tenant_id)
|
||||
embedding_model = model_manager.get_model_instance(
|
||||
tenant_id=dataset.tenant_id,
|
||||
provider=dataset.embedding_model_provider,
|
||||
model_type=ModelType.TEXT_EMBEDDING,
|
||||
model=dataset.embedding_model,
|
||||
)
|
||||
return embedding_model.get_text_embedding_num_tokens([document.page_content for document in documents])
|
||||
@@ -19,6 +19,7 @@ from core.rag.cleaner.clean_processor import CleanProcessor
|
||||
from core.rag.datasource.keyword.keyword_factory import Keyword
|
||||
from core.rag.datasource.vdb.vector_factory import Vector
|
||||
from core.rag.docstore.dataset_docstore import DatasetDocumentStore
|
||||
from core.rag.embedding.token_counter import calculate_segment_token_counts
|
||||
from core.rag.entities import Rule
|
||||
from core.rag.extractor.entity.extract_setting import ExtractSetting
|
||||
from core.rag.extractor.extract_processor import ExtractProcessor
|
||||
@@ -244,10 +245,16 @@ class ParagraphIndexProcessor(BaseIndexProcessor):
|
||||
all_multimodal_documents.extend(doc.attachments)
|
||||
documents.append(doc)
|
||||
if documents:
|
||||
token_counts = calculate_segment_token_counts(dataset=dataset, documents=documents)
|
||||
# save node to document segment
|
||||
doc_store = DatasetDocumentStore(dataset=dataset, user_id=document.created_by, document_id=document.id)
|
||||
# add document segments
|
||||
doc_store.add_documents(docs=documents, save_child=False, session=session)
|
||||
doc_store.add_documents(
|
||||
session=session,
|
||||
docs=documents,
|
||||
token_counts=token_counts,
|
||||
save_child=False,
|
||||
)
|
||||
session.commit()
|
||||
if dataset.indexing_technique == IndexTechniqueType.HIGH_QUALITY:
|
||||
vector = Vector(dataset, session=session)
|
||||
|
||||
@@ -15,6 +15,7 @@ from core.model_manager import ModelInstance
|
||||
from core.rag.cleaner.clean_processor import CleanProcessor
|
||||
from core.rag.datasource.vdb.vector_factory import Vector
|
||||
from core.rag.docstore.dataset_docstore import DatasetDocumentStore
|
||||
from core.rag.embedding.token_counter import calculate_segment_token_counts
|
||||
from core.rag.entities import ParentMode, Rule
|
||||
from core.rag.extractor.entity.extract_setting import ExtractSetting
|
||||
from core.rag.extractor.extract_processor import ExtractProcessor
|
||||
@@ -304,6 +305,7 @@ class ParentChildIndexProcessor(BaseIndexProcessor):
|
||||
doc.attachments = self._get_content_files(doc, current_user=account, session=session)
|
||||
documents.append(doc)
|
||||
if documents:
|
||||
token_counts = calculate_segment_token_counts(dataset=dataset, documents=documents)
|
||||
# update document parent mode
|
||||
dataset_process_rule = DatasetProcessRule(
|
||||
dataset_id=dataset.id,
|
||||
@@ -321,7 +323,12 @@ class ParentChildIndexProcessor(BaseIndexProcessor):
|
||||
# save node to document segment
|
||||
doc_store = DatasetDocumentStore(dataset=dataset, user_id=document.created_by, document_id=document.id)
|
||||
# add document segments
|
||||
doc_store.add_documents(docs=documents, save_child=True, session=session)
|
||||
doc_store.add_documents(
|
||||
session=session,
|
||||
docs=documents,
|
||||
token_counts=token_counts,
|
||||
save_child=True,
|
||||
)
|
||||
session.commit()
|
||||
if dataset.indexing_technique == IndexTechniqueType.HIGH_QUALITY:
|
||||
all_child_documents = []
|
||||
|
||||
@@ -17,6 +17,7 @@ from core.llm_generator.llm_generator import LLMGenerator
|
||||
from core.rag.cleaner.clean_processor import CleanProcessor
|
||||
from core.rag.datasource.vdb.vector_factory import Vector
|
||||
from core.rag.docstore.dataset_docstore import DatasetDocumentStore
|
||||
from core.rag.embedding.token_counter import calculate_segment_token_counts
|
||||
from core.rag.entities import Rule
|
||||
from core.rag.extractor.entity.extract_setting import ExtractSetting
|
||||
from core.rag.extractor.extract_processor import ExtractProcessor
|
||||
@@ -205,9 +206,15 @@ class QAIndexProcessor(BaseIndexProcessor):
|
||||
doc = Document(page_content=qa_chunk.question, metadata=metadata)
|
||||
documents.append(doc)
|
||||
if documents:
|
||||
token_counts = calculate_segment_token_counts(dataset=dataset, documents=documents)
|
||||
# save node to document segment
|
||||
doc_store = DatasetDocumentStore(dataset=dataset, user_id=document.created_by, document_id=document.id)
|
||||
doc_store.add_documents(docs=documents, save_child=False, session=session)
|
||||
doc_store.add_documents(
|
||||
session=session,
|
||||
docs=documents,
|
||||
token_counts=token_counts,
|
||||
save_child=False,
|
||||
)
|
||||
session.commit()
|
||||
if dataset.indexing_technique == IndexTechniqueType.HIGH_QUALITY:
|
||||
vector = Vector(dataset, session=session)
|
||||
|
||||
@@ -61,6 +61,7 @@ class RBACPermission(StrEnum):
|
||||
WORKSPACE_ROLE_MANAGE = "workspace_role_manage"
|
||||
API_EXTENSION_MANAGE = "api_extension_manage"
|
||||
CUSTOMIZATION_MANAGE = "customization_manage"
|
||||
AGENT_MANAGE = "agent_manage"
|
||||
|
||||
SNIPPETS_CREATE_AND_MODIFY = "snippets_create_and_modify"
|
||||
SNIPPETS_MANAGE = "snippets_management"
|
||||
|
||||
@@ -17,6 +17,7 @@ from core.tools.entities.tool_entities import (
|
||||
ToolParameter,
|
||||
ToolProviderType,
|
||||
)
|
||||
from core.tools.entities.ui_entities import ToolUIMessage
|
||||
|
||||
|
||||
class Tool(ABC):
|
||||
@@ -58,7 +59,6 @@ class Tool(ABC):
|
||||
if self.runtime and self.runtime.runtime_parameters:
|
||||
tool_parameters.update(self.runtime.runtime_parameters)
|
||||
|
||||
# try parse tool parameters into the correct type
|
||||
tool_parameters = self._transform_tool_parameters_type(tool_parameters)
|
||||
|
||||
result = self._invoke(
|
||||
@@ -87,14 +87,14 @@ class Tool(ABC):
|
||||
return result
|
||||
|
||||
def _transform_tool_parameters_type(self, tool_parameters: dict[str, Any]) -> dict[str, Any]:
|
||||
"""
|
||||
Transform tool parameters type
|
||||
"""
|
||||
# Temp fix for the issue that the tool parameters will be converted to empty while validating the credentials
|
||||
"""Transform declared tool parameter values without resolving runtime schemas."""
|
||||
result = deepcopy(tool_parameters)
|
||||
for parameter in self.entity.parameters or []:
|
||||
if parameter.name in tool_parameters:
|
||||
result[parameter.name] = parameter.type.cast_value(tool_parameters[parameter.name])
|
||||
if parameter.multiple:
|
||||
result[parameter.name] = parameter.init_frontend_parameter(result.get(parameter.name))
|
||||
else:
|
||||
result[parameter.name] = parameter.type.cast_value(tool_parameters[parameter.name])
|
||||
|
||||
return result
|
||||
|
||||
@@ -196,17 +196,31 @@ class Tool(ABC):
|
||||
}:
|
||||
continue
|
||||
|
||||
parameter_schema: dict[str, Any] = (
|
||||
{
|
||||
"type": parameter.type.as_normal_type(),
|
||||
"description": parameter.llm_description or "",
|
||||
}
|
||||
if parameter.input_schema is None
|
||||
else deepcopy(parameter.input_schema)
|
||||
)
|
||||
is_multiple_select = parameter.multiple and parameter.type in {
|
||||
ToolParameter.ToolParameterType.SELECT,
|
||||
ToolParameter.ToolParameterType.DYNAMIC_SELECT,
|
||||
}
|
||||
if is_multiple_select:
|
||||
item_schema: dict[str, Any] = {"type": "string"}
|
||||
if parameter.type == ToolParameter.ToolParameterType.SELECT and parameter.options:
|
||||
item_schema["enum"] = [option.value for option in parameter.options]
|
||||
parameter_schema: dict[str, Any] = {"type": "array", "items": item_schema}
|
||||
else:
|
||||
parameter_schema = (
|
||||
{
|
||||
"type": parameter.type.as_normal_type(),
|
||||
"description": parameter.llm_description or "",
|
||||
}
|
||||
if parameter.input_schema is None
|
||||
else deepcopy(parameter.input_schema)
|
||||
)
|
||||
parameter_schema.setdefault("description", parameter.llm_description or "")
|
||||
|
||||
if parameter.type == ToolParameter.ToolParameterType.SELECT and parameter.options:
|
||||
if (
|
||||
not is_multiple_select
|
||||
and parameter.type == ToolParameter.ToolParameterType.SELECT
|
||||
and parameter.options
|
||||
):
|
||||
parameter_schema["enum"] = [option.value for option in parameter.options]
|
||||
|
||||
schema["properties"][parameter.name] = parameter_schema
|
||||
@@ -282,6 +296,13 @@ class Tool(ABC):
|
||||
message=ToolInvokeMessage.JsonMessage(json_object=object, suppress_output=suppress_output),
|
||||
)
|
||||
|
||||
def create_ui_message(self, ui_message: ToolUIMessage | dict[str, Any]) -> ToolInvokeMessage:
|
||||
"""Create a validated, model-invisible UI message for chat clients."""
|
||||
return ToolInvokeMessage(
|
||||
type=ToolInvokeMessage.MessageType.UI,
|
||||
message=ToolUIMessage.model_validate(ui_message),
|
||||
)
|
||||
|
||||
def create_variable_message(
|
||||
self, variable_name: str, variable_value: Any, stream: bool = False
|
||||
) -> ToolInvokeMessage:
|
||||
|
||||
@@ -7,6 +7,9 @@ from sqlalchemy.orm import Session
|
||||
|
||||
from core.tools.builtin_tool.tool import BuiltinTool
|
||||
from core.tools.entities.tool_entities import ToolInvokeMessage
|
||||
from core.tools.entities.ui_entities import A2UI_CATALOG_ID, A2UI_PROTOCOL_VERSION
|
||||
|
||||
_CURRENT_TIME_SURFACE_ID = "current-time"
|
||||
|
||||
|
||||
class CurrentTimeTool(BuiltinTool):
|
||||
@@ -20,19 +23,60 @@ class CurrentTimeTool(BuiltinTool):
|
||||
app_id: str | None = None,
|
||||
message_id: str | None = None,
|
||||
) -> Generator[ToolInvokeMessage, None, None]:
|
||||
"""
|
||||
invoke tools
|
||||
"""
|
||||
# get timezone
|
||||
tz = tool_parameters.get("timezone", "UTC")
|
||||
fm = tool_parameters.get("format") or "%Y-%m-%d %H:%M:%S %Z"
|
||||
if tz == "UTC":
|
||||
yield self.create_text_message(f"{datetime.now(UTC).strftime(fm)}")
|
||||
return
|
||||
"""Return the formatted time for the model and a standard time card for chat clients.
|
||||
|
||||
try:
|
||||
tz = pytz_timezone(tz)
|
||||
except Exception:
|
||||
yield self.create_text_message(f"Invalid timezone: {tz}")
|
||||
return
|
||||
yield self.create_text_message(f"{datetime.now(tz).strftime(fm)}")
|
||||
Invalid timezone values retain the existing text-only error response.
|
||||
"""
|
||||
timezone_name = tool_parameters.get("timezone", "UTC")
|
||||
fm = tool_parameters.get("format") or "%Y-%m-%d %H:%M:%S %Z"
|
||||
if timezone_name == "UTC":
|
||||
current_time = datetime.now(UTC)
|
||||
else:
|
||||
try:
|
||||
timezone_info = pytz_timezone(timezone_name)
|
||||
except Exception:
|
||||
yield self.create_text_message(f"Invalid timezone: {timezone_name}")
|
||||
return
|
||||
current_time = datetime.now(timezone_info)
|
||||
|
||||
formatted_time = current_time.strftime(fm)
|
||||
yield self.create_text_message(formatted_time)
|
||||
yield self.create_ui_message(
|
||||
{
|
||||
"messages": [
|
||||
{
|
||||
"version": A2UI_PROTOCOL_VERSION,
|
||||
"createSurface": {
|
||||
"surfaceId": _CURRENT_TIME_SURFACE_ID,
|
||||
"catalogId": A2UI_CATALOG_ID,
|
||||
},
|
||||
},
|
||||
{
|
||||
"version": A2UI_PROTOCOL_VERSION,
|
||||
"updateDataModel": {
|
||||
"surfaceId": _CURRENT_TIME_SURFACE_ID,
|
||||
"value": {"currentTime": current_time.isoformat()},
|
||||
},
|
||||
},
|
||||
{
|
||||
"version": A2UI_PROTOCOL_VERSION,
|
||||
"updateComponents": {
|
||||
"surfaceId": _CURRENT_TIME_SURFACE_ID,
|
||||
"components": [
|
||||
{
|
||||
"id": "root",
|
||||
"component": "Card",
|
||||
"children": ["time"],
|
||||
},
|
||||
{
|
||||
"id": "time",
|
||||
"component": "DateTime",
|
||||
"value": {"path": "/currentTime"},
|
||||
"format": "datetime",
|
||||
},
|
||||
],
|
||||
},
|
||||
},
|
||||
],
|
||||
}
|
||||
)
|
||||
|
||||
@@ -32,6 +32,7 @@ from core.plugin.entities.parameters import (
|
||||
from core.rag.entities import RetrievalSourceMetadata
|
||||
from core.tools.entities.common_entities import I18nObject
|
||||
from core.tools.entities.constants import TOOL_SELECTOR_MODEL_IDENTITY
|
||||
from core.tools.entities.ui_entities import ToolUIMessage
|
||||
|
||||
|
||||
class EmojiIconDict(TypedDict):
|
||||
@@ -240,13 +241,15 @@ class ToolInvokeMessage(BaseModel):
|
||||
LOG = auto()
|
||||
BLOB_CHUNK = auto()
|
||||
RETRIEVER_RESOURCES = auto()
|
||||
UI = auto()
|
||||
|
||||
type: MessageType = MessageType.TEXT
|
||||
"""
|
||||
plain text, image url or link url
|
||||
"""
|
||||
message: (
|
||||
JsonMessage
|
||||
ToolUIMessage
|
||||
| JsonMessage
|
||||
| TextMessage
|
||||
| BlobChunkMessage
|
||||
| BlobMessage
|
||||
@@ -275,6 +278,8 @@ class ToolInvokeMessage(BaseModel):
|
||||
v = {"json_object": v}
|
||||
elif msg_type == cls.MessageType.FILE:
|
||||
v = {"file_marker": "file_marker"}
|
||||
elif msg_type == cls.MessageType.UI:
|
||||
v = ToolUIMessage.model_validate(v)
|
||||
|
||||
return v
|
||||
|
||||
@@ -292,9 +297,7 @@ class ToolInvokeMessageBinary(BaseModel):
|
||||
|
||||
|
||||
class ToolParameter(PluginParameter):
|
||||
"""
|
||||
Overrides type
|
||||
"""
|
||||
"""Tool-specific parameter declaration and invocation-value normalization."""
|
||||
|
||||
class ToolParameterType(StrEnum):
|
||||
"""
|
||||
@@ -333,12 +336,28 @@ class ToolParameter(PluginParameter):
|
||||
LLM = auto() # will be set by LLM
|
||||
|
||||
type: ToolParameterType = Field(..., description="The type of the parameter")
|
||||
multiple: bool = Field(
|
||||
default=False,
|
||||
description="Whether the parameter is multiple select, only valid for select or dynamic-select type",
|
||||
)
|
||||
human_description: I18nObject | None = Field(default=None, description="The description presented to the user")
|
||||
form: ToolParameterForm = Field(..., description="The form of the parameter, schema/form/llm")
|
||||
llm_description: str | None = None
|
||||
# MCP object and array type parameters use this field to store the schema
|
||||
input_schema: dict[str, Any] | None = None
|
||||
|
||||
@model_validator(mode="after")
|
||||
def validate_multiple(self) -> ToolParameter:
|
||||
supports_multiple = self.type in {
|
||||
self.ToolParameterType.SELECT,
|
||||
self.ToolParameterType.DYNAMIC_SELECT,
|
||||
}
|
||||
if self.multiple and not supports_multiple:
|
||||
raise ValueError("multiple is only valid for select and dynamic-select parameters")
|
||||
if supports_multiple and self.default is not None and (isinstance(self.default, list) != self.multiple):
|
||||
raise ValueError("default must be a list exactly when multiple is true")
|
||||
return self
|
||||
|
||||
@classmethod
|
||||
def get_simple_instance(
|
||||
cls,
|
||||
@@ -378,8 +397,25 @@ class ToolParameter(PluginParameter):
|
||||
options=option_objs,
|
||||
)
|
||||
|
||||
def init_frontend_parameter(self, value: Any):
|
||||
return init_frontend_parameter(self, self.type, value)
|
||||
def init_frontend_parameter(self, value: Any) -> Any:
|
||||
"""Normalize a value against this tool parameter's full declaration."""
|
||||
if not self.multiple:
|
||||
return init_frontend_parameter(self, self.type, value)
|
||||
|
||||
parameter_value = self.default if value is None else value
|
||||
if parameter_value is None:
|
||||
parameter_value = []
|
||||
if not isinstance(parameter_value, list):
|
||||
raise ValueError(f"tool parameter {self.name} must be a list when multiple is true")
|
||||
if not all(isinstance(item, str) for item in parameter_value):
|
||||
raise ValueError(f"tool parameter {self.name} must contain only strings")
|
||||
if self.required and not parameter_value:
|
||||
raise ValueError(f"tool parameter {self.name} not found in tool config")
|
||||
if self.type == self.ToolParameterType.SELECT:
|
||||
options = [option.value for option in self.options]
|
||||
if any(item not in options for item in parameter_value):
|
||||
raise ValueError(f"tool parameter {self.name} value {parameter_value} not in options {options}")
|
||||
return parameter_value
|
||||
|
||||
|
||||
class ToolProviderIdentity(BaseModel):
|
||||
|
||||
@@ -0,0 +1,675 @@
|
||||
"""Validated, read-only UI messages emitted by tools.
|
||||
|
||||
The public wire format is the A2UI v0.9.1 flat message shape, narrowed to a
|
||||
Dify-owned catalog. Tool authors can bind display values to the surface data
|
||||
model, but cannot supply executable actions, HTML, styles, themes, or custom
|
||||
components. A complete ``ToolUIMessage`` describes exactly one surface and is
|
||||
validated as a bounded, self-contained component graph before it crosses into
|
||||
chat streaming. Sequential data-model patches are materialized with the same
|
||||
object/array upsert semantics as the web renderer so cumulative limits cannot
|
||||
be bypassed with individually small updates.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import re
|
||||
import uuid
|
||||
from collections.abc import Sequence
|
||||
from enum import StrEnum
|
||||
from typing import Annotated, Literal
|
||||
|
||||
from pydantic import (
|
||||
BaseModel,
|
||||
ConfigDict,
|
||||
Field,
|
||||
JsonValue,
|
||||
SerializerFunctionWrapHandler,
|
||||
field_validator,
|
||||
model_serializer,
|
||||
model_validator,
|
||||
)
|
||||
|
||||
A2UI_PROTOCOL = "a2ui"
|
||||
A2UI_PROTOCOL_VERSION = "v0.9.1"
|
||||
A2UI_CATALOG_ID = "https://dify.ai/a2ui/catalog/v1"
|
||||
DIFY_UI_JSON_ENVELOPE_KEY = "__dify_ui__"
|
||||
|
||||
MAX_UI_MESSAGES = 64
|
||||
MAX_UI_COMPONENTS = 100
|
||||
MAX_UI_STRING_LENGTH = 4096
|
||||
MAX_UI_PAYLOAD_BYTES = 128 * 1024
|
||||
MAX_DATA_MODEL_DEPTH = 16
|
||||
MAX_DATA_MODEL_NODES = 2000
|
||||
MAX_DATA_MODEL_ARRAY_INDEX = 1000
|
||||
MAX_JSON_POINTER_SEGMENTS = 16
|
||||
MAX_UI_PARTS_PER_MESSAGE = 16
|
||||
MAX_UI_PARTS_PAYLOAD_BYTES = 512 * 1024
|
||||
|
||||
_ID_PATTERN = re.compile(r"^[A-Za-z0-9][A-Za-z0-9_.:-]{0,127}$")
|
||||
_ARRAY_INDEX_PATTERN = re.compile(r"^(?:0|[1-9]\d*)$")
|
||||
_DANGEROUS_POINTER_SEGMENTS = {"__proto__", "constructor", "prototype"}
|
||||
_DANGEROUS_DATA_KEYS = _DANGEROUS_POINTER_SEGMENTS
|
||||
_MAX_HISTORY_UI_PART_CANDIDATES = MAX_UI_PARTS_PER_MESSAGE * 4
|
||||
_UI_PART_ID_NAMESPACE = uuid.uuid5(uuid.NAMESPACE_URL, "https://dify.ai/a2ui/part-id/v1")
|
||||
|
||||
|
||||
class _StrictModel(BaseModel):
|
||||
model_config = ConfigDict(
|
||||
extra="forbid",
|
||||
populate_by_name=True,
|
||||
serialize_by_alias=True,
|
||||
allow_inf_nan=False,
|
||||
)
|
||||
|
||||
|
||||
class A2UIDataBinding(_StrictModel):
|
||||
"""A JSON Pointer into the surface data model."""
|
||||
|
||||
path: str
|
||||
|
||||
@field_validator("path")
|
||||
@classmethod
|
||||
def _validate_path(cls, value: str) -> str:
|
||||
return _validate_json_pointer(value, label="data binding path")
|
||||
|
||||
|
||||
type DynamicString = str | A2UIDataBinding
|
||||
type DynamicNumber = int | float | A2UIDataBinding
|
||||
type DynamicScalar = str | int | float | bool | None | A2UIDataBinding
|
||||
|
||||
|
||||
class A2UIComponentType(StrEnum):
|
||||
CARD = "Card"
|
||||
ROW = "Row"
|
||||
COLUMN = "Column"
|
||||
TEXT = "Text"
|
||||
ICON = "Icon"
|
||||
DIVIDER = "Divider"
|
||||
BADGE = "Badge"
|
||||
METRIC = "Metric"
|
||||
DATE_TIME = "DateTime"
|
||||
PROGRESS = "Progress"
|
||||
KEY_VALUE = "KeyValue"
|
||||
|
||||
|
||||
class A2UIComponent(_StrictModel):
|
||||
"""One flat component entry from the Dify catalog."""
|
||||
|
||||
id: str
|
||||
component: A2UIComponentType
|
||||
children: list[str] | None = None
|
||||
title: DynamicString | None = None
|
||||
gap: Literal["small", "medium", "large"] | None = None
|
||||
align: Literal["start", "center", "end"] | None = None
|
||||
text: DynamicString | None = None
|
||||
variant: Literal["body", "caption"] | None = None
|
||||
name: (
|
||||
Literal[
|
||||
"clock",
|
||||
"cloud",
|
||||
"sun",
|
||||
"rain",
|
||||
"snow",
|
||||
"wind",
|
||||
"thermometer",
|
||||
"calendar",
|
||||
"location",
|
||||
]
|
||||
| None
|
||||
) = None
|
||||
tone: Literal["neutral", "info", "success", "warning", "critical"] | None = None
|
||||
label: DynamicString | None = None
|
||||
value: DynamicScalar | None = None
|
||||
unit: DynamicString | None = None
|
||||
format: Literal["date", "time", "datetime"] | None = None
|
||||
max: DynamicNumber | None = None
|
||||
|
||||
@field_validator("id")
|
||||
@classmethod
|
||||
def _validate_id(cls, value: str) -> str:
|
||||
if not _ID_PATTERN.fullmatch(value):
|
||||
raise ValueError("component id contains unsupported characters or is too long")
|
||||
return value
|
||||
|
||||
@field_validator("children")
|
||||
@classmethod
|
||||
def _validate_children(cls, value: list[str] | None) -> list[str] | None:
|
||||
if value is None:
|
||||
return None
|
||||
if len(value) > MAX_UI_COMPONENTS:
|
||||
raise ValueError("component has too many children")
|
||||
if len(value) != len(set(value)):
|
||||
raise ValueError("component children must not contain duplicate ids")
|
||||
for child_id in value:
|
||||
if not _ID_PATTERN.fullmatch(child_id):
|
||||
raise ValueError("child component id contains unsupported characters or is too long")
|
||||
return value
|
||||
|
||||
@model_validator(mode="after")
|
||||
def _validate_component_props(self) -> A2UIComponent:
|
||||
present = self.model_fields_set - {"id", "component"}
|
||||
allowed: dict[A2UIComponentType, set[str]] = {
|
||||
A2UIComponentType.CARD: {"children", "title"},
|
||||
A2UIComponentType.ROW: {"children", "gap", "align"},
|
||||
A2UIComponentType.COLUMN: {"children", "gap"},
|
||||
A2UIComponentType.TEXT: {"text", "variant"},
|
||||
A2UIComponentType.ICON: {"name"},
|
||||
A2UIComponentType.DIVIDER: set(),
|
||||
A2UIComponentType.BADGE: {"text", "tone"},
|
||||
A2UIComponentType.METRIC: {"label", "value", "unit"},
|
||||
A2UIComponentType.DATE_TIME: {"value", "format"},
|
||||
A2UIComponentType.PROGRESS: {"value", "max", "label"},
|
||||
A2UIComponentType.KEY_VALUE: {"label", "value"},
|
||||
}
|
||||
required: dict[A2UIComponentType, set[str]] = {
|
||||
A2UIComponentType.CARD: {"children"},
|
||||
A2UIComponentType.ROW: {"children"},
|
||||
A2UIComponentType.COLUMN: {"children"},
|
||||
A2UIComponentType.TEXT: {"text"},
|
||||
A2UIComponentType.ICON: {"name"},
|
||||
A2UIComponentType.DIVIDER: set(),
|
||||
A2UIComponentType.BADGE: {"text"},
|
||||
A2UIComponentType.METRIC: {"label", "value"},
|
||||
A2UIComponentType.DATE_TIME: {"value"},
|
||||
A2UIComponentType.PROGRESS: {"value", "label"},
|
||||
A2UIComponentType.KEY_VALUE: {"label", "value"},
|
||||
}
|
||||
unexpected = present - allowed[self.component]
|
||||
missing = required[self.component] - present
|
||||
if unexpected:
|
||||
raise ValueError(f"{self.component} does not support properties: {sorted(unexpected)}")
|
||||
if missing:
|
||||
raise ValueError(f"{self.component} requires properties: {sorted(missing)}")
|
||||
nullable_props = (
|
||||
{"value"} if self.component in {A2UIComponentType.METRIC, A2UIComponentType.KEY_VALUE} else set()
|
||||
)
|
||||
null_props = {prop for prop in present if getattr(self, prop) is None and prop not in nullable_props}
|
||||
if null_props:
|
||||
raise ValueError(f"{self.component} properties cannot be null: {sorted(null_props)}")
|
||||
if self.component == A2UIComponentType.DATE_TIME and not isinstance(self.value, str | A2UIDataBinding):
|
||||
raise ValueError("DateTime value must be a string or data binding")
|
||||
if self.component == A2UIComponentType.PROGRESS and (
|
||||
isinstance(self.value, bool) or not isinstance(self.value, int | float | A2UIDataBinding)
|
||||
):
|
||||
raise ValueError("Progress value must be a number or data binding")
|
||||
if (
|
||||
self.component == A2UIComponentType.PROGRESS
|
||||
and "max" in present
|
||||
and (isinstance(self.max, bool) or not isinstance(self.max, int | float | A2UIDataBinding))
|
||||
):
|
||||
raise ValueError("Progress max must be a number or data binding")
|
||||
return self
|
||||
|
||||
@model_serializer(mode="wrap")
|
||||
def _serialize_component(self, handler: SerializerFunctionWrapHandler) -> dict[str, JsonValue]:
|
||||
serialized = handler(self)
|
||||
return {
|
||||
key: value
|
||||
for key, value in serialized.items()
|
||||
if key in {"id", "component"} or key in self.model_fields_set
|
||||
}
|
||||
|
||||
|
||||
class A2UICreateSurface(_StrictModel):
|
||||
surface_id: str = Field(alias="surfaceId")
|
||||
catalog_id: Literal["https://dify.ai/a2ui/catalog/v1"] = Field(alias="catalogId")
|
||||
|
||||
@field_validator("surface_id")
|
||||
@classmethod
|
||||
def _validate_surface_id(cls, value: str) -> str:
|
||||
if not _ID_PATTERN.fullmatch(value):
|
||||
raise ValueError("surface id contains unsupported characters or is too long")
|
||||
return value
|
||||
|
||||
|
||||
class A2UIUpdateComponents(_StrictModel):
|
||||
surface_id: str = Field(alias="surfaceId")
|
||||
components: Annotated[list[A2UIComponent], Field(min_length=1, max_length=MAX_UI_COMPONENTS)]
|
||||
|
||||
@field_validator("surface_id")
|
||||
@classmethod
|
||||
def _validate_surface_id(cls, value: str) -> str:
|
||||
if not _ID_PATTERN.fullmatch(value):
|
||||
raise ValueError("surface id contains unsupported characters or is too long")
|
||||
return value
|
||||
|
||||
@field_validator("components")
|
||||
@classmethod
|
||||
def _validate_unique_component_ids(cls, value: list[A2UIComponent]) -> list[A2UIComponent]:
|
||||
ids = [component.id for component in value]
|
||||
if len(ids) != len(set(ids)):
|
||||
raise ValueError("updateComponents contains duplicate component ids")
|
||||
return value
|
||||
|
||||
|
||||
class A2UIUpdateDataModel(_StrictModel):
|
||||
surface_id: str = Field(alias="surfaceId")
|
||||
value: JsonValue
|
||||
path: str | None = None
|
||||
|
||||
@field_validator("surface_id")
|
||||
@classmethod
|
||||
def _validate_surface_id(cls, value: str) -> str:
|
||||
if not _ID_PATTERN.fullmatch(value):
|
||||
raise ValueError("surface id contains unsupported characters or is too long")
|
||||
return value
|
||||
|
||||
@field_validator("value")
|
||||
@classmethod
|
||||
def _validate_value(cls, value: JsonValue) -> JsonValue:
|
||||
_validate_data_model_value(value)
|
||||
return value
|
||||
|
||||
@field_validator("path")
|
||||
@classmethod
|
||||
def _validate_path(cls, value: str | None) -> str | None:
|
||||
if value is None:
|
||||
return value
|
||||
return _validate_json_pointer(value, label="data model path", enforce_array_index_limit=True)
|
||||
|
||||
@model_serializer(mode="wrap")
|
||||
def _serialize_update(self, handler: SerializerFunctionWrapHandler) -> dict[str, JsonValue]:
|
||||
serialized = handler(self)
|
||||
if "path" not in self.model_fields_set:
|
||||
serialized.pop("path", None)
|
||||
return serialized
|
||||
|
||||
|
||||
class A2UIDeleteSurface(_StrictModel):
|
||||
surface_id: str = Field(alias="surfaceId")
|
||||
|
||||
@field_validator("surface_id")
|
||||
@classmethod
|
||||
def _validate_surface_id(cls, value: str) -> str:
|
||||
if not _ID_PATTERN.fullmatch(value):
|
||||
raise ValueError("surface id contains unsupported characters or is too long")
|
||||
return value
|
||||
|
||||
|
||||
class A2UIMessage(_StrictModel):
|
||||
"""A single A2UI v0.9.1 server message."""
|
||||
|
||||
version: Literal["v0.9.1"]
|
||||
create_surface: A2UICreateSurface | None = Field(default=None, alias="createSurface")
|
||||
update_components: A2UIUpdateComponents | None = Field(default=None, alias="updateComponents")
|
||||
update_data_model: A2UIUpdateDataModel | None = Field(default=None, alias="updateDataModel")
|
||||
delete_surface: A2UIDeleteSurface | None = Field(default=None, alias="deleteSurface")
|
||||
|
||||
@model_validator(mode="after")
|
||||
def _validate_exactly_one_operation(self) -> A2UIMessage:
|
||||
operations = (
|
||||
self.create_surface,
|
||||
self.update_components,
|
||||
self.update_data_model,
|
||||
self.delete_surface,
|
||||
)
|
||||
if sum(operation is not None for operation in operations) != 1:
|
||||
raise ValueError("A2UI message must contain exactly one operation")
|
||||
return self
|
||||
|
||||
@model_serializer(mode="wrap")
|
||||
def _serialize_message(self, handler: SerializerFunctionWrapHandler) -> dict[str, JsonValue]:
|
||||
return {key: value for key, value in handler(self).items() if value is not None}
|
||||
|
||||
@property
|
||||
def surface_id(self) -> str:
|
||||
operation = self.create_surface or self.update_components or self.update_data_model or self.delete_surface
|
||||
assert operation is not None
|
||||
return operation.surface_id
|
||||
|
||||
|
||||
class ToolUIMessage(_StrictModel):
|
||||
"""A complete, validated UI surface emitted by one tool result.
|
||||
|
||||
Validation covers both the component graph and every materialized
|
||||
data-model revision, because clients render the message sequence in order.
|
||||
"""
|
||||
|
||||
protocol: Literal["a2ui"] = "a2ui"
|
||||
protocol_version: Literal["v0.9.1"] = "v0.9.1"
|
||||
messages: Annotated[list[A2UIMessage], Field(min_length=1, max_length=MAX_UI_MESSAGES)]
|
||||
fallback: str | None = None
|
||||
|
||||
@field_validator("fallback")
|
||||
@classmethod
|
||||
def _validate_fallback(cls, value: str | None) -> str | None:
|
||||
if value is not None and len(value) > MAX_UI_STRING_LENGTH:
|
||||
raise ValueError("UI fallback is too long")
|
||||
return value
|
||||
|
||||
@model_validator(mode="after")
|
||||
def _validate_surface(self) -> ToolUIMessage:
|
||||
first = self.messages[0]
|
||||
if first.create_surface is None:
|
||||
raise ValueError("first A2UI message must be createSurface")
|
||||
if any(message.create_surface is not None for message in self.messages[1:]):
|
||||
raise ValueError("createSurface may only appear as the first message")
|
||||
delete_indexes = [index for index, message in enumerate(self.messages) if message.delete_surface]
|
||||
if delete_indexes and delete_indexes != [len(self.messages) - 1]:
|
||||
raise ValueError("deleteSurface may only appear once as the final message")
|
||||
|
||||
surface_id = first.surface_id
|
||||
if any(message.surface_id != surface_id for message in self.messages):
|
||||
raise ValueError("all A2UI messages in a tool UI message must target the same surface")
|
||||
|
||||
components: dict[str, A2UIComponent] = {}
|
||||
data_model: JsonValue = {}
|
||||
root_seen_before_delete = False
|
||||
for message in self.messages[1:]:
|
||||
if message.update_components is not None:
|
||||
for component in message.update_components.components:
|
||||
components[component.id] = component
|
||||
if len(components) > MAX_UI_COMPONENTS:
|
||||
raise ValueError("UI surface has too many components")
|
||||
if message.update_data_model is not None:
|
||||
data_model = _apply_data_model_update(data_model, message.update_data_model)
|
||||
_validate_data_model_value(data_model)
|
||||
if message.delete_surface is not None:
|
||||
root_seen_before_delete = "root" in components
|
||||
|
||||
if "root" not in components and not root_seen_before_delete:
|
||||
raise ValueError("UI surface must define a component with id 'root'")
|
||||
|
||||
_validate_component_graph(components)
|
||||
payload = self.model_dump(mode="json", by_alias=True, exclude_none=True)
|
||||
_validate_bounded_json(payload)
|
||||
return self
|
||||
|
||||
@model_serializer(mode="wrap")
|
||||
def _serialize_ui_message(self, handler: SerializerFunctionWrapHandler) -> dict[str, JsonValue]:
|
||||
return {key: value for key, value in handler(self).items() if value is not None}
|
||||
|
||||
@property
|
||||
def surface_id(self) -> str:
|
||||
return self.messages[0].surface_id
|
||||
|
||||
|
||||
class MessageUIPart(_StrictModel):
|
||||
"""SSE/history representation of one tool-owned UI surface revision."""
|
||||
|
||||
part_id: Annotated[str, Field(min_length=1, max_length=512)]
|
||||
sequence: Annotated[int, Field(ge=1)]
|
||||
protocol: Literal["a2ui"] = "a2ui"
|
||||
protocol_version: Literal["v0.9.1"] = "v0.9.1"
|
||||
messages: Annotated[list[A2UIMessage], Field(min_length=1, max_length=MAX_UI_MESSAGES)]
|
||||
fallback: str | None = None
|
||||
|
||||
@model_validator(mode="after")
|
||||
def _validate_messages(self) -> MessageUIPart:
|
||||
ToolUIMessage(
|
||||
protocol=self.protocol,
|
||||
protocol_version=self.protocol_version,
|
||||
messages=self.messages,
|
||||
fallback=self.fallback,
|
||||
)
|
||||
return self
|
||||
|
||||
@model_serializer(mode="wrap")
|
||||
def _serialize_part(self, handler: SerializerFunctionWrapHandler) -> dict[str, JsonValue]:
|
||||
return {key: value for key, value in handler(self).items() if value is not None}
|
||||
|
||||
@classmethod
|
||||
def from_tool_ui_message(
|
||||
cls,
|
||||
*,
|
||||
part_id: str,
|
||||
sequence: int,
|
||||
ui_message: ToolUIMessage,
|
||||
) -> MessageUIPart:
|
||||
return cls(
|
||||
part_id=part_id,
|
||||
sequence=sequence,
|
||||
protocol=ui_message.protocol,
|
||||
protocol_version=ui_message.protocol_version,
|
||||
messages=ui_message.messages,
|
||||
fallback=ui_message.fallback,
|
||||
)
|
||||
|
||||
|
||||
def extract_ui_message_from_json(value: JsonValue) -> ToolUIMessage | None:
|
||||
"""Recognize the reserved compatibility envelope used by older SDKs."""
|
||||
|
||||
if not isinstance(value, dict) or set(value) != {DIFY_UI_JSON_ENVELOPE_KEY}:
|
||||
return None
|
||||
return ToolUIMessage.model_validate(value[DIFY_UI_JSON_ENVELOPE_KEY])
|
||||
|
||||
|
||||
def parse_tool_ui_messages(value: object) -> list[ToolUIMessage]:
|
||||
"""Validate a list received through the Agent backend metadata channel."""
|
||||
|
||||
if not isinstance(value, list):
|
||||
raise ValueError("dify_ui_messages metadata must be a list")
|
||||
if len(value) > MAX_UI_PARTS_PER_MESSAGE:
|
||||
raise ValueError(f"tool UI batch cannot contain more than {MAX_UI_PARTS_PER_MESSAGE} messages")
|
||||
messages = [ToolUIMessage.model_validate(item) for item in value]
|
||||
validate_tool_ui_message_batch(messages)
|
||||
return messages
|
||||
|
||||
|
||||
def build_ui_part_id(namespace: str, surface_id: str) -> str:
|
||||
"""Build a stable, bounded ID from an unambiguous tool/surface tuple."""
|
||||
if not isinstance(namespace, str) or not isinstance(surface_id, str):
|
||||
raise TypeError("UI part namespace and surface id must be strings")
|
||||
identity = json.dumps((namespace, surface_id), ensure_ascii=True, separators=(",", ":"))
|
||||
return f"ui-{uuid.uuid5(_UI_PART_ID_NAMESPACE, identity)}"
|
||||
|
||||
|
||||
def validate_tool_ui_message_batch(messages: Sequence[ToolUIMessage]) -> None:
|
||||
"""Enforce the per-tool-call UI batch budget before queue publication."""
|
||||
if len(messages) > MAX_UI_PARTS_PER_MESSAGE:
|
||||
raise ValueError(f"tool UI batch cannot contain more than {MAX_UI_PARTS_PER_MESSAGE} messages")
|
||||
if _serialized_model_list_size(messages) > MAX_UI_PARTS_PAYLOAD_BYTES:
|
||||
raise ValueError(f"tool UI batch payload cannot exceed {MAX_UI_PARTS_PAYLOAD_BYTES} bytes")
|
||||
|
||||
|
||||
def validate_ui_part_batch(parts: Sequence[MessageUIPart]) -> None:
|
||||
"""Enforce persisted/current assistant-message UI budgets."""
|
||||
if len(parts) > MAX_UI_PARTS_PER_MESSAGE:
|
||||
raise ValueError(f"assistant message cannot contain more than {MAX_UI_PARTS_PER_MESSAGE} UI parts")
|
||||
part_ids = [part.part_id for part in parts]
|
||||
if len(part_ids) != len(set(part_ids)):
|
||||
raise ValueError("assistant message UI parts must have distinct part ids")
|
||||
if _serialized_model_list_size(parts) > MAX_UI_PARTS_PAYLOAD_BYTES:
|
||||
raise ValueError(f"assistant message UI parts cannot exceed {MAX_UI_PARTS_PAYLOAD_BYTES} bytes")
|
||||
|
||||
|
||||
def upsert_ui_part(parts: Sequence[MessageUIPart], part: MessageUIPart) -> list[MessageUIPart] | None:
|
||||
"""Return a bounded new part list, or ``None`` for a stale revision."""
|
||||
candidate = list(parts)
|
||||
for index, existing_part in enumerate(candidate):
|
||||
if existing_part.part_id != part.part_id:
|
||||
continue
|
||||
if existing_part.sequence >= part.sequence:
|
||||
return None
|
||||
candidate[index] = part
|
||||
break
|
||||
else:
|
||||
candidate.append(part)
|
||||
|
||||
validate_ui_part_batch(candidate)
|
||||
return candidate
|
||||
|
||||
|
||||
def parse_history_ui_parts(value: object) -> list[MessageUIPart]:
|
||||
"""Best-effort recovery of bounded UI parts from untrusted historical metadata."""
|
||||
if not isinstance(value, list):
|
||||
return []
|
||||
|
||||
parts: list[MessageUIPart] = []
|
||||
for item in value[:_MAX_HISTORY_UI_PART_CANDIDATES]:
|
||||
try:
|
||||
part = MessageUIPart.model_validate(item)
|
||||
updated_parts = upsert_ui_part(parts, part)
|
||||
except (TypeError, ValueError):
|
||||
continue
|
||||
if updated_parts is not None:
|
||||
parts = updated_parts
|
||||
return parts
|
||||
|
||||
|
||||
def _validate_component_graph(components: dict[str, A2UIComponent]) -> None:
|
||||
for component in components.values():
|
||||
for child_id in component.children or []:
|
||||
if child_id not in components:
|
||||
raise ValueError(f"component {component.id!r} references unknown child {child_id!r}")
|
||||
|
||||
visiting: set[str] = set()
|
||||
visited: set[str] = set()
|
||||
|
||||
def visit(component_id: str) -> None:
|
||||
if component_id in visiting:
|
||||
raise ValueError("UI component graph contains a cycle")
|
||||
if component_id in visited:
|
||||
return
|
||||
visiting.add(component_id)
|
||||
for child_id in components[component_id].children or []:
|
||||
visit(child_id)
|
||||
visiting.remove(component_id)
|
||||
visited.add(component_id)
|
||||
|
||||
for component_id in components:
|
||||
visit(component_id)
|
||||
|
||||
parent_by_child: dict[str, str] = {}
|
||||
for component in components.values():
|
||||
for child_id in component.children or []:
|
||||
if child_id == "root":
|
||||
raise ValueError("root component cannot be referenced as a child")
|
||||
existing_parent = parent_by_child.get(child_id)
|
||||
if existing_parent is not None:
|
||||
raise ValueError(
|
||||
f"component {child_id!r} has multiple parents: {existing_parent!r} and {component.id!r}"
|
||||
)
|
||||
parent_by_child[child_id] = component.id
|
||||
|
||||
reachable = {"root"}
|
||||
pending = ["root"]
|
||||
while pending:
|
||||
component_id = pending.pop()
|
||||
for child_id in components[component_id].children or []:
|
||||
if child_id in reachable:
|
||||
continue
|
||||
reachable.add(child_id)
|
||||
pending.append(child_id)
|
||||
|
||||
unreachable = set(components) - reachable
|
||||
if unreachable:
|
||||
raise ValueError(f"UI component graph contains components unreachable from root: {sorted(unreachable)}")
|
||||
|
||||
|
||||
def _serialized_model_list_size(values: Sequence[BaseModel]) -> int:
|
||||
payload = [value.model_dump(mode="json", by_alias=True, exclude_none=True) for value in values]
|
||||
return len(json.dumps(payload, ensure_ascii=False, separators=(",", ":")).encode())
|
||||
|
||||
|
||||
def _validate_json_pointer(
|
||||
value: str,
|
||||
*,
|
||||
label: str,
|
||||
enforce_array_index_limit: bool = False,
|
||||
) -> str:
|
||||
if not value.startswith("/"):
|
||||
raise ValueError(f"{label} must be a JSON Pointer beginning with '/'")
|
||||
if len(value) > 256:
|
||||
raise ValueError(f"{label} is too long")
|
||||
if re.search(r"~(?![01])", value):
|
||||
raise ValueError(f"{label} contains an invalid JSON Pointer escape")
|
||||
decoded_segments = _decode_json_pointer(value)
|
||||
if len(decoded_segments) > MAX_JSON_POINTER_SEGMENTS:
|
||||
raise ValueError(f"{label} contains too many segments")
|
||||
if set(decoded_segments).intersection(_DANGEROUS_POINTER_SEGMENTS):
|
||||
raise ValueError(f"{label} contains a forbidden segment")
|
||||
if enforce_array_index_limit and any(
|
||||
segment.isdecimal() and int(segment) > MAX_DATA_MODEL_ARRAY_INDEX for segment in decoded_segments
|
||||
):
|
||||
raise ValueError(f"{label} contains an array index larger than {MAX_DATA_MODEL_ARRAY_INDEX}")
|
||||
return value
|
||||
|
||||
|
||||
def _decode_json_pointer(value: str) -> list[str]:
|
||||
if value == "/":
|
||||
return []
|
||||
return [segment.replace("~1", "/").replace("~0", "~") for segment in value.split("/")[1:]]
|
||||
|
||||
|
||||
def _apply_data_model_update(current: JsonValue, update: A2UIUpdateDataModel) -> JsonValue:
|
||||
path = update.path
|
||||
if path is None or path == "/":
|
||||
return update.value
|
||||
|
||||
segments = _decode_json_pointer(path)
|
||||
|
||||
def apply_at(node: JsonValue | None, segment_index: int) -> JsonValue:
|
||||
if segment_index == len(segments):
|
||||
return update.value
|
||||
|
||||
segment = segments[segment_index]
|
||||
is_array_segment = _ARRAY_INDEX_PATTERN.fullmatch(segment) is not None
|
||||
if isinstance(node, list) or (not isinstance(node, dict) and is_array_segment):
|
||||
if not is_array_segment:
|
||||
raise ValueError("data model array paths must use canonical non-negative indexes")
|
||||
index = int(segment)
|
||||
if index > MAX_DATA_MODEL_ARRAY_INDEX:
|
||||
raise ValueError(f"data model array index cannot exceed {MAX_DATA_MODEL_ARRAY_INDEX}")
|
||||
|
||||
next_node = list(node) if isinstance(node, list) else []
|
||||
if index > len(next_node):
|
||||
raise ValueError("data model array index cannot create a gap")
|
||||
child = next_node[index] if index < len(next_node) else None
|
||||
updated_child = apply_at(child, segment_index + 1)
|
||||
if index == len(next_node):
|
||||
next_node.append(updated_child)
|
||||
else:
|
||||
next_node[index] = updated_child
|
||||
return next_node
|
||||
|
||||
next_node = dict(node) if isinstance(node, dict) else {}
|
||||
next_node[segment] = apply_at(next_node.get(segment), segment_index + 1)
|
||||
return next_node
|
||||
|
||||
return apply_at(current, 0)
|
||||
|
||||
|
||||
def _validate_data_model_value(value: JsonValue) -> None:
|
||||
nodes = 0
|
||||
|
||||
def walk(node: JsonValue, *, depth: int) -> None:
|
||||
nonlocal nodes
|
||||
nodes += 1
|
||||
if nodes > MAX_DATA_MODEL_NODES:
|
||||
raise ValueError("data model value has too many nodes")
|
||||
if depth > MAX_DATA_MODEL_DEPTH:
|
||||
raise ValueError("data model value is nested too deeply")
|
||||
if isinstance(node, list):
|
||||
for item in node:
|
||||
walk(item, depth=depth + 1)
|
||||
return
|
||||
if isinstance(node, dict):
|
||||
unsafe_keys = set(node).intersection(_DANGEROUS_DATA_KEYS)
|
||||
if unsafe_keys:
|
||||
raise ValueError(f"data model value contains forbidden keys: {sorted(unsafe_keys)}")
|
||||
for item in node.values():
|
||||
walk(item, depth=depth + 1)
|
||||
|
||||
walk(value, depth=0)
|
||||
|
||||
|
||||
def _validate_bounded_json(value: JsonValue) -> None:
|
||||
def walk(node: JsonValue) -> None:
|
||||
if isinstance(node, str):
|
||||
if len(node) > MAX_UI_STRING_LENGTH:
|
||||
raise ValueError("UI payload contains a string that is too long")
|
||||
return
|
||||
if isinstance(node, list):
|
||||
for item in node:
|
||||
walk(item)
|
||||
return
|
||||
if isinstance(node, dict):
|
||||
for item in node.values():
|
||||
walk(item)
|
||||
|
||||
walk(value)
|
||||
encoded = json.dumps(value, ensure_ascii=False, separators=(",", ":")).encode()
|
||||
if len(encoded) > MAX_UI_PAYLOAD_BYTES:
|
||||
raise ValueError("UI payload is too large")
|
||||
@@ -3,6 +3,7 @@ import json
|
||||
import logging
|
||||
from collections.abc import Generator, Iterable
|
||||
from copy import deepcopy
|
||||
from dataclasses import dataclass
|
||||
from datetime import UTC, datetime
|
||||
from mimetypes import guess_type
|
||||
from typing import Any, Union, cast
|
||||
@@ -21,6 +22,12 @@ from core.tools.entities.tool_entities import (
|
||||
ToolInvokeMeta,
|
||||
ToolParameter,
|
||||
)
|
||||
from core.tools.entities.ui_entities import (
|
||||
DIFY_UI_JSON_ENVELOPE_KEY,
|
||||
ToolUIMessage,
|
||||
extract_ui_message_from_json,
|
||||
validate_tool_ui_message_batch,
|
||||
)
|
||||
from core.tools.errors import (
|
||||
ToolEngineInvokeError,
|
||||
ToolInvokeError,
|
||||
@@ -40,6 +47,16 @@ from models.model import Message, MessageFile
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class ToolAgentInvokeResult:
|
||||
"""Agent-facing tool result with UI kept outside the model observation."""
|
||||
|
||||
observation: str
|
||||
message_files: list[str]
|
||||
ui_messages: list[ToolUIMessage]
|
||||
meta: ToolInvokeMeta
|
||||
|
||||
|
||||
class ToolEngine:
|
||||
"""
|
||||
Tool runtime engine take care of the tool executions.
|
||||
@@ -59,9 +76,13 @@ class ToolEngine:
|
||||
conversation_id: str | None = None,
|
||||
app_id: str | None = None,
|
||||
message_id: str | None = None,
|
||||
) -> tuple[str, list[str], ToolInvokeMeta]:
|
||||
) -> ToolAgentInvokeResult:
|
||||
"""
|
||||
Agent invokes the tool with the given arguments.
|
||||
Invoke a tool for an agent.
|
||||
|
||||
UI messages are validated and returned on a dedicated channel. They are
|
||||
intentionally excluded from both the LLM observation and tracing
|
||||
callback output.
|
||||
"""
|
||||
# check if arguments is a string
|
||||
if isinstance(tool_parameters, str):
|
||||
@@ -103,7 +124,7 @@ class ToolEngine:
|
||||
conversation_id=message.conversation_id,
|
||||
)
|
||||
|
||||
message_list = list(messages)
|
||||
message_list, ui_messages = ToolEngine.collect_agent_messages(messages)
|
||||
|
||||
# extract binary data from tool invoke message
|
||||
binary_files = ToolEngine._extract_tool_response_binary_and_text(message_list)
|
||||
@@ -126,7 +147,12 @@ class ToolEngine:
|
||||
)
|
||||
|
||||
# transform tool invoke message to get LLM friendly message
|
||||
return plain_text, message_files, meta
|
||||
return ToolAgentInvokeResult(
|
||||
observation=plain_text,
|
||||
message_files=message_files,
|
||||
ui_messages=ui_messages,
|
||||
meta=meta,
|
||||
)
|
||||
except ToolProviderCredentialValidationError as e:
|
||||
logger.error(e, exc_info=True)
|
||||
error_response = "Please check your tool provider credentials"
|
||||
@@ -148,13 +174,23 @@ class ToolEngine:
|
||||
error_response = f"tool invoke error: {meta.error}"
|
||||
agent_tool_callback.on_tool_error(e)
|
||||
logger.error(e, exc_info=True)
|
||||
return error_response, [], meta
|
||||
return ToolAgentInvokeResult(
|
||||
observation=error_response,
|
||||
message_files=[],
|
||||
ui_messages=[],
|
||||
meta=meta,
|
||||
)
|
||||
except Exception as e:
|
||||
error_response = f"unknown error: {e}"
|
||||
agent_tool_callback.on_tool_error(e)
|
||||
logger.error(e, exc_info=True)
|
||||
|
||||
return error_response, [], ToolInvokeMeta.error_instance(error_response)
|
||||
return ToolAgentInvokeResult(
|
||||
observation=error_response,
|
||||
message_files=[],
|
||||
ui_messages=[],
|
||||
meta=ToolInvokeMeta.error_instance(error_response),
|
||||
)
|
||||
|
||||
@staticmethod
|
||||
def generic_invoke(
|
||||
@@ -197,7 +233,7 @@ class ToolEngine:
|
||||
tool_outputs=response,
|
||||
)
|
||||
|
||||
return response
|
||||
return ToolEngine.normalize_ui_messages(response)
|
||||
except Exception as e:
|
||||
workflow_tool_callback.on_tool_error(e)
|
||||
raise e
|
||||
@@ -266,7 +302,10 @@ class ToolEngine:
|
||||
ensure_ascii=False,
|
||||
)
|
||||
)
|
||||
elif response.type == ToolInvokeMessage.MessageType.VARIABLE:
|
||||
elif response.type in {
|
||||
ToolInvokeMessage.MessageType.VARIABLE,
|
||||
ToolInvokeMessage.MessageType.UI,
|
||||
}:
|
||||
continue
|
||||
else:
|
||||
parts.append(str(response.message))
|
||||
@@ -278,6 +317,124 @@ class ToolEngine:
|
||||
|
||||
return "".join(parts)
|
||||
|
||||
@staticmethod
|
||||
def normalize_ui_messages(messages: Iterable[ToolInvokeMessage]) -> Generator[ToolInvokeMessage, None, None]:
|
||||
"""Convert reserved variable/JSON compatibility envelopes into native UI."""
|
||||
for message in messages:
|
||||
if message.type == ToolInvokeMessage.MessageType.VARIABLE:
|
||||
variable_message = cast(ToolInvokeMessage.VariableMessage, message.message)
|
||||
if variable_message.variable_name != DIFY_UI_JSON_ENVELOPE_KEY:
|
||||
yield message
|
||||
continue
|
||||
yield ToolInvokeMessage(
|
||||
type=ToolInvokeMessage.MessageType.UI,
|
||||
message=ToolUIMessage.model_validate(variable_message.variable_value),
|
||||
meta=message.meta,
|
||||
)
|
||||
continue
|
||||
|
||||
if message.type != ToolInvokeMessage.MessageType.JSON:
|
||||
yield message
|
||||
continue
|
||||
|
||||
json_message = cast(ToolInvokeMessage.JsonMessage, message.message)
|
||||
ui_message = extract_ui_message_from_json(json_message.json_object)
|
||||
if ui_message is None:
|
||||
yield message
|
||||
continue
|
||||
yield ToolInvokeMessage(type=ToolInvokeMessage.MessageType.UI, message=ui_message, meta=message.meta)
|
||||
|
||||
@staticmethod
|
||||
def collect_agent_messages(
|
||||
messages: Iterable[ToolInvokeMessage],
|
||||
) -> tuple[list[ToolInvokeMessage], list[ToolUIMessage]]:
|
||||
"""Collect a bounded UI batch while preserving all non-UI observations.
|
||||
|
||||
A malformed or over-budget UI message invalidates the complete UI
|
||||
batch. Remaining UI is skipped without parsing, but the input iterable
|
||||
is still consumed so later text and file messages are retained.
|
||||
"""
|
||||
normalized_messages: list[ToolInvokeMessage] = []
|
||||
ui_messages: list[ToolUIMessage] = []
|
||||
ui_batch_rejected = False
|
||||
|
||||
for message in messages:
|
||||
is_ui_transport = ToolEngine._is_ui_transport_message(message)
|
||||
if ui_batch_rejected and is_ui_transport:
|
||||
continue
|
||||
|
||||
try:
|
||||
normalized = ToolEngine.normalize_ui_messages([message])
|
||||
except (TypeError, ValueError):
|
||||
if not is_ui_transport:
|
||||
raise
|
||||
# Reserved UI envelopes must never fall back to JSON/variable
|
||||
# observations, even when malformed.
|
||||
ui_batch_rejected = True
|
||||
ui_messages.clear()
|
||||
normalized_messages = [
|
||||
normalized_message
|
||||
for normalized_message in normalized_messages
|
||||
if normalized_message.type != ToolInvokeMessage.MessageType.UI
|
||||
]
|
||||
logger.warning("Ignored invalid reserved tool UI message", exc_info=True)
|
||||
continue
|
||||
|
||||
try:
|
||||
for normalized_message in normalized:
|
||||
if normalized_message.type != ToolInvokeMessage.MessageType.UI:
|
||||
normalized_messages.append(normalized_message)
|
||||
continue
|
||||
if ui_batch_rejected:
|
||||
continue
|
||||
|
||||
ui_message = cast(ToolUIMessage, normalized_message.message)
|
||||
candidate = [*ui_messages, ui_message]
|
||||
try:
|
||||
validate_tool_ui_message_batch(candidate)
|
||||
except ValueError:
|
||||
ui_batch_rejected = True
|
||||
ui_messages.clear()
|
||||
normalized_messages = [
|
||||
collected_message
|
||||
for collected_message in normalized_messages
|
||||
if collected_message.type != ToolInvokeMessage.MessageType.UI
|
||||
]
|
||||
logger.warning(
|
||||
"Ignored tool UI batch that exceeds agent invocation limits",
|
||||
exc_info=True,
|
||||
)
|
||||
continue
|
||||
|
||||
ui_messages = candidate
|
||||
normalized_messages.append(normalized_message)
|
||||
except (TypeError, ValueError):
|
||||
if not is_ui_transport:
|
||||
raise
|
||||
ui_batch_rejected = True
|
||||
ui_messages.clear()
|
||||
normalized_messages = [
|
||||
normalized_message
|
||||
for normalized_message in normalized_messages
|
||||
if normalized_message.type != ToolInvokeMessage.MessageType.UI
|
||||
]
|
||||
logger.warning("Ignored invalid reserved tool UI message", exc_info=True)
|
||||
|
||||
return normalized_messages, ui_messages
|
||||
|
||||
@staticmethod
|
||||
def _is_ui_transport_message(message: ToolInvokeMessage) -> bool:
|
||||
if message.type == ToolInvokeMessage.MessageType.UI:
|
||||
return True
|
||||
if message.type == ToolInvokeMessage.MessageType.VARIABLE:
|
||||
variable_message = cast(ToolInvokeMessage.VariableMessage, message.message)
|
||||
return variable_message.variable_name == DIFY_UI_JSON_ENVELOPE_KEY
|
||||
if message.type == ToolInvokeMessage.MessageType.JSON:
|
||||
json_message = cast(ToolInvokeMessage.JsonMessage, message.message)
|
||||
value = json_message.json_object
|
||||
return isinstance(value, dict) and set(value) == {DIFY_UI_JSON_ENVELOPE_KEY}
|
||||
return False
|
||||
|
||||
@staticmethod
|
||||
def _extract_tool_response_binary_and_text(
|
||||
tool_response: list[ToolInvokeMessage],
|
||||
|
||||
@@ -90,6 +90,7 @@ from .system_variables import SystemVariableKey, get_system_text
|
||||
if TYPE_CHECKING:
|
||||
from core.tools.__base.tool import Tool
|
||||
from core.tools.entities.tool_entities import ToolInvokeMessage as CoreToolInvokeMessage
|
||||
from core.tools.entities.ui_entities import ToolUIMessage
|
||||
from graphon.nodes.llm.file_saver import LLMFileSaver
|
||||
from graphon.nodes.tool.entities import ToolNodeData
|
||||
|
||||
@@ -666,6 +667,11 @@ class DifyToolNodeRuntime(ToolNodeRuntimeProtocol):
|
||||
) -> Generator[ToolRuntimeMessage, None, None]:
|
||||
try:
|
||||
for message in messages:
|
||||
# Tool Gen-UI is currently a chat presentation channel. Graphon
|
||||
# does not define an equivalent runtime message, so workflow
|
||||
# execution safely ignores it instead of coercing it to JSON.
|
||||
if message.type.value == "ui":
|
||||
continue
|
||||
yield self._convert_message(message)
|
||||
except Exception as exc:
|
||||
raise self._map_invocation_exception(exc, provider_name=provider_name) from exc
|
||||
@@ -686,6 +692,7 @@ class DifyToolNodeRuntime(ToolNodeRuntimeProtocol):
|
||||
| CoreToolInvokeMessage.FileMessage
|
||||
| CoreToolInvokeMessage.VariableMessage
|
||||
| CoreToolInvokeMessage.RetrieverResourceMessage
|
||||
| ToolUIMessage
|
||||
| None,
|
||||
) -> (
|
||||
ToolRuntimeMessage.TextMessage
|
||||
|
||||
@@ -2,7 +2,7 @@ from __future__ import annotations
|
||||
|
||||
from collections.abc import Mapping
|
||||
from dataclasses import dataclass
|
||||
from typing import Any, Literal, Protocol, cast
|
||||
from typing import Any, Final, Literal, Protocol
|
||||
|
||||
from dify_agent.layers.dify_core_tools import DifyCoreToolConfig, DifyCoreToolProviderType, DifyCoreToolsLayerConfig
|
||||
from dify_agent.layers.dify_plugin import (
|
||||
@@ -29,6 +29,13 @@ from models.provider_ids import ToolProviderID
|
||||
from models.tools import WorkflowToolProvider
|
||||
from services.tools.mcp_tools_manage_service import MCPToolManageService
|
||||
|
||||
_CORE_TOOL_PROVIDER_TYPES: Final[dict[ToolProviderType, DifyCoreToolProviderType]] = {
|
||||
ToolProviderType.BUILT_IN: "builtin",
|
||||
ToolProviderType.API: "api",
|
||||
ToolProviderType.WORKFLOW: "workflow",
|
||||
ToolProviderType.MCP: "mcp",
|
||||
}
|
||||
|
||||
|
||||
class WorkflowAgentDifyToolsBuildError(ValueError):
|
||||
"""Raised when Agent Soul tools cannot be prepared for Agent backend."""
|
||||
@@ -235,7 +242,7 @@ class WorkflowAgentDifyToolsBuilder:
|
||||
if tool_config.tool_name is not None:
|
||||
expanded.append(tool_config)
|
||||
continue
|
||||
provider_type = ToolProviderType.value_of(tool_config.provider_type)
|
||||
provider_type = tool_config.provider_type
|
||||
provider_id = self._provider_id(tool_config)
|
||||
try:
|
||||
tool_names = self._provider_declared_tool_names(
|
||||
@@ -279,7 +286,7 @@ class WorkflowAgentDifyToolsBuilder:
|
||||
tenant_id: str,
|
||||
tool_config: AgentSoulDifyToolConfig,
|
||||
) -> AgentSoulDifyToolConfig:
|
||||
if tool_config.provider_type != ToolProviderType.MCP.value:
|
||||
if tool_config.provider_type is not ToolProviderType.MCP:
|
||||
return tool_config
|
||||
provider_id = self._mcp_provider_id_resolver(tenant_id=tenant_id, provider_id=self._provider_id(tool_config))
|
||||
return tool_config.model_copy(update={"provider_id": provider_id, "plugin_id": None, "provider": None})
|
||||
@@ -326,7 +333,7 @@ class WorkflowAgentDifyToolsBuilder:
|
||||
def _to_agent_tool_entity(tool_config: AgentSoulDifyToolConfig) -> AgentToolEntity:
|
||||
assert tool_config.tool_name is not None
|
||||
return AgentToolEntity(
|
||||
provider_type=ToolProviderType.value_of(tool_config.provider_type),
|
||||
provider_type=tool_config.provider_type,
|
||||
provider_id=WorkflowAgentDifyToolsBuilder._provider_id(tool_config),
|
||||
tool_name=tool_config.tool_name,
|
||||
tool_parameters=dict(tool_config.runtime_parameters),
|
||||
@@ -343,24 +350,16 @@ class WorkflowAgentDifyToolsBuilder:
|
||||
|
||||
@staticmethod
|
||||
def _provider_key(tool_config: AgentSoulDifyToolConfig) -> tuple[ToolProviderType, str]:
|
||||
return (
|
||||
ToolProviderType.value_of(tool_config.provider_type),
|
||||
WorkflowAgentDifyToolsBuilder._provider_id(tool_config),
|
||||
)
|
||||
return (tool_config.provider_type, WorkflowAgentDifyToolsBuilder._provider_id(tool_config))
|
||||
|
||||
@staticmethod
|
||||
def _tool_layer_destination(tool_config: AgentSoulDifyToolConfig) -> Literal["plugin", "core"]:
|
||||
provider_type = ToolProviderType.value_of(tool_config.provider_type)
|
||||
provider_type = tool_config.provider_type
|
||||
if provider_type is ToolProviderType.PLUGIN or (
|
||||
provider_type is ToolProviderType.BUILT_IN and _is_plugin_provider_id(tool_config.provider_id)
|
||||
):
|
||||
return "plugin"
|
||||
if provider_type in {
|
||||
ToolProviderType.BUILT_IN,
|
||||
ToolProviderType.API,
|
||||
ToolProviderType.WORKFLOW,
|
||||
ToolProviderType.MCP,
|
||||
}:
|
||||
if provider_type in _CORE_TOOL_PROVIDER_TYPES:
|
||||
return "core"
|
||||
if provider_type is ToolProviderType.DATASET_RETRIEVAL:
|
||||
raise WorkflowAgentDifyToolsBuildError(
|
||||
@@ -417,7 +416,7 @@ class WorkflowAgentDifyToolsBuilder:
|
||||
) -> DifyCoreToolConfig:
|
||||
parameters = self._prepared_parameters(tool_runtime)
|
||||
return DifyCoreToolConfig(
|
||||
provider_type=cast(DifyCoreToolProviderType, tool_config.provider_type),
|
||||
provider_type=_CORE_TOOL_PROVIDER_TYPES[tool_config.provider_type],
|
||||
provider_id=self._provider_id(tool_config),
|
||||
tool_name=tool_config.tool_name or exposed_name,
|
||||
credential_id=tool_config.credential_ref.id if tool_config.credential_ref else None,
|
||||
|
||||
@@ -12,6 +12,7 @@ import json
|
||||
import subprocess
|
||||
import sys
|
||||
import tempfile
|
||||
from copy import deepcopy
|
||||
from pathlib import Path
|
||||
from typing import Any, Literal, TypedDict
|
||||
|
||||
@@ -24,6 +25,16 @@ LOCK_PATH = API_ROOT / "knowledge-fs-contract.lock.json"
|
||||
DEFAULT_REPOSITORY = WORKSPACE_ROOT.parent / "knowledge-fs"
|
||||
OPENAPI_METHODS = ("delete", "get", "head", "options", "patch", "post", "put", "trace")
|
||||
PROXY_METHODS = frozenset({"delete", "get", "patch", "post", "put"})
|
||||
CONSOLE_PROXY_ERROR_SCHEMA_NAME = "ConsoleProxyError"
|
||||
CONSOLE_PROXY_ERROR_SCHEMA: dict[str, Any] = {
|
||||
"type": "object",
|
||||
"required": ["code", "message", "status"],
|
||||
"properties": {
|
||||
"code": {"type": "string"},
|
||||
"message": {"type": "string"},
|
||||
"status": {"type": "integer"},
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
class ContractDeclaration(TypedDict):
|
||||
@@ -38,6 +49,7 @@ class ContractDeclaration(TypedDict):
|
||||
request_headers: tuple[str, ...]
|
||||
response_headers: tuple[str, ...]
|
||||
response_media_types: tuple[str, ...]
|
||||
error_status_map: tuple[tuple[int, int], ...]
|
||||
|
||||
|
||||
type DeclarationField = Literal[
|
||||
@@ -70,6 +82,7 @@ def main() -> None:
|
||||
mode.add_argument("--check", action="store_true")
|
||||
mode.add_argument("--update-lock", action="store_true")
|
||||
parser.add_argument("--repository", type=Path, default=DEFAULT_REPOSITORY)
|
||||
parser.add_argument("--output-openapi", type=Path)
|
||||
args = parser.parse_args()
|
||||
|
||||
repository = args.repository.resolve()
|
||||
@@ -101,7 +114,15 @@ def main() -> None:
|
||||
)
|
||||
|
||||
document: dict[str, Any] = json.loads(openapi_content)
|
||||
validate_declarations(document, console_contract_declarations())
|
||||
declarations = console_contract_declarations()
|
||||
validate_declarations(document, declarations)
|
||||
|
||||
if args.output_openapi:
|
||||
filtered_document = filter_openapi_document(document, declarations)
|
||||
filtered_document["x-dify-source-openapi-sha256"] = openapi_sha256
|
||||
filtered_document["x-dify-console-declarations-sha256"] = contract_declarations_sha256(declarations)
|
||||
args.output_openapi.parent.mkdir(parents=True, exist_ok=True)
|
||||
args.output_openapi.write_text(json.dumps(filtered_document, indent=2) + "\n")
|
||||
|
||||
if args.update_lock:
|
||||
LOCK_PATH.write_text(
|
||||
@@ -147,8 +168,7 @@ def validate_declarations(document: dict[str, Any], declarations: tuple[Contract
|
||||
raise ValueError(f"KnowledgeFS OpenAPI path must be absolute: {path}")
|
||||
if method not in PROXY_METHODS:
|
||||
raise ValueError(f"KnowledgeFS proxy does not support {method.upper()} {path}")
|
||||
expected: ContractDeclaration = {
|
||||
"operation_id": operation_id,
|
||||
expected: dict[DeclarationField, object] = {
|
||||
"method": method.upper(),
|
||||
"path": path[1:],
|
||||
"required_scope": required_scope(operation),
|
||||
@@ -166,11 +186,114 @@ def validate_declarations(document: dict[str, Any], declarations: tuple[Contract
|
||||
f"KnowledgeFS operation {operation_id} field {field} drifted: "
|
||||
f"expected {expected_value!r}, received {received_value!r}"
|
||||
)
|
||||
validate_error_status_map(operation_id, declaration["error_status_map"])
|
||||
|
||||
|
||||
def filter_openapi_document(
|
||||
document: dict[str, Any],
|
||||
declarations: tuple[ContractDeclaration, ...],
|
||||
) -> dict[str, Any]:
|
||||
"""Return a code-generation document containing only Console-allowlisted operations."""
|
||||
filtered_document: dict[str, Any] = {
|
||||
key: value for key, value in document.items() if key not in {"components", "paths"}
|
||||
}
|
||||
source_paths = document.get("paths", {})
|
||||
filtered_paths: dict[str, Any] = {}
|
||||
|
||||
for declaration in declarations:
|
||||
path = f"/{declaration['path']}"
|
||||
method = declaration["method"].lower()
|
||||
source_path_item = source_paths[path]
|
||||
path_metadata = {key: value for key, value in source_path_item.items() if key not in OPENAPI_METHODS}
|
||||
filtered_path_item = filtered_paths.setdefault(path, path_metadata)
|
||||
filtered_operation = deepcopy(source_path_item[method])
|
||||
_rewrite_proxy_error_responses(filtered_operation, declaration["error_status_map"])
|
||||
filtered_path_item[method] = filtered_operation
|
||||
|
||||
filtered_document["paths"] = filtered_paths
|
||||
|
||||
source_components = document.get("components", {})
|
||||
filtered_components = {key: value for key, value in source_components.items() if key != "schemas"}
|
||||
source_schemas = source_components.get("schemas", {})
|
||||
available_schemas = {**source_schemas, CONSOLE_PROXY_ERROR_SCHEMA_NAME: CONSOLE_PROXY_ERROR_SCHEMA}
|
||||
schema_names = _referenced_schema_names(filtered_paths, available_schemas)
|
||||
filtered_components["schemas"] = {
|
||||
name: schema for name, schema in available_schemas.items() if name in schema_names
|
||||
}
|
||||
filtered_document["components"] = filtered_components
|
||||
return filtered_document
|
||||
|
||||
|
||||
def validate_error_status_map(operation_id: str, error_status_map: tuple[tuple[int, int], ...]) -> None:
|
||||
"""Validate the status normalization advertised by one Console operation."""
|
||||
upstream_statuses: set[int] = set()
|
||||
for upstream_status, console_status in error_status_map:
|
||||
if upstream_status in upstream_statuses:
|
||||
raise ValueError(f"KnowledgeFS operation {operation_id} has duplicate error status: {upstream_status}")
|
||||
if not 400 <= upstream_status <= 599 or not 400 <= console_status <= 599:
|
||||
raise ValueError(f"KnowledgeFS operation {operation_id} has invalid error status mapping")
|
||||
upstream_statuses.add(upstream_status)
|
||||
|
||||
|
||||
def _rewrite_proxy_error_responses(
|
||||
operation: dict[str, Any],
|
||||
error_status_map: tuple[tuple[int, int], ...],
|
||||
) -> None:
|
||||
responses = operation.setdefault("responses", {})
|
||||
proxy_error_response = {
|
||||
"description": "Error normalized by the Dify Console KnowledgeFS proxy.",
|
||||
"content": {
|
||||
"application/json": {"schema": {"$ref": f"#/components/schemas/{CONSOLE_PROXY_ERROR_SCHEMA_NAME}"}}
|
||||
},
|
||||
}
|
||||
for upstream_status, console_status in error_status_map:
|
||||
existing_target = responses.get(str(console_status)) if upstream_status != console_status else None
|
||||
responses.pop(str(upstream_status), None)
|
||||
normalized_response: dict[str, Any] = deepcopy(proxy_error_response)
|
||||
existing_schema = (
|
||||
existing_target.get("content", {}).get("application/json", {}).get("schema")
|
||||
if isinstance(existing_target, dict)
|
||||
else None
|
||||
)
|
||||
if existing_schema is not None:
|
||||
normalized_response["content"]["application/json"]["schema"] = {
|
||||
"oneOf": [
|
||||
deepcopy(existing_schema),
|
||||
{"$ref": f"#/components/schemas/{CONSOLE_PROXY_ERROR_SCHEMA_NAME}"},
|
||||
]
|
||||
}
|
||||
responses[str(console_status)] = normalized_response
|
||||
|
||||
|
||||
def _referenced_schema_names(value: Any, schemas: dict[str, Any]) -> set[str]:
|
||||
reference_prefix = "#/components/schemas/"
|
||||
selected: set[str] = set()
|
||||
pending: list[Any] = [value]
|
||||
|
||||
while pending:
|
||||
current = pending.pop()
|
||||
if isinstance(current, list):
|
||||
pending.extend(current)
|
||||
continue
|
||||
if not isinstance(current, dict):
|
||||
continue
|
||||
|
||||
reference = current.get("$ref")
|
||||
if isinstance(reference, str) and reference.startswith(reference_prefix):
|
||||
name = reference.removeprefix(reference_prefix)
|
||||
if name not in selected:
|
||||
if name not in schemas:
|
||||
raise ValueError(f"KnowledgeFS OpenAPI references missing schema: {name}")
|
||||
selected.add(name)
|
||||
pending.append(schemas[name])
|
||||
pending.extend(current.values())
|
||||
|
||||
return selected
|
||||
|
||||
|
||||
def console_contract_declarations() -> tuple[ContractDeclaration, ...]:
|
||||
"""Return transport declarations from the runtime Console operation registry."""
|
||||
from services.knowledge_fs_proxy import KNOWLEDGE_FS_CONSOLE_OPERATIONS
|
||||
from services.knowledge_fs_operations import KNOWLEDGE_FS_CONSOLE_OPERATIONS
|
||||
|
||||
return tuple(
|
||||
{
|
||||
@@ -183,11 +306,18 @@ def console_contract_declarations() -> tuple[ContractDeclaration, ...]:
|
||||
"request_headers": operation.request_headers,
|
||||
"response_headers": operation.response_headers,
|
||||
"response_media_types": operation.response_media_types,
|
||||
"error_status_map": operation.error_status_map,
|
||||
}
|
||||
for operation in KNOWLEDGE_FS_CONSOLE_OPERATIONS
|
||||
)
|
||||
|
||||
|
||||
def contract_declarations_sha256(declarations: tuple[ContractDeclaration, ...]) -> str:
|
||||
"""Return a stable digest for the runtime Console operation declarations."""
|
||||
content = json.dumps(declarations, separators=(",", ":"), sort_keys=True).encode()
|
||||
return sha256(content)
|
||||
|
||||
|
||||
def response_kind(operation: dict[str, Any]) -> str:
|
||||
media_types = response_media_types(operation)
|
||||
if "text/event-stream" in media_types:
|
||||
|
||||
@@ -157,6 +157,7 @@ def init_app(app: DifyApp) -> Celery:
|
||||
"tasks.regenerate_summary_index_task", # summary index regeneration
|
||||
"tasks.initialize_created_app_rbac_access_task", # app access initialization
|
||||
"tasks.install_default_plugins_task", # tenant default plugin installation
|
||||
"tasks.refresh_billing_vector_space_task", # billing vector-space cache refresh
|
||||
"tasks.app_generate.resume_agent_app_task", # ENG-635: Agent v2 chat ask_human resume
|
||||
"tasks.workflow_run_archive_download_tasks", # workflow-run archive download preparation
|
||||
]
|
||||
|
||||
@@ -1,5 +1,6 @@
|
||||
import json
|
||||
from typing import cast, override
|
||||
import logging
|
||||
from typing import assert_never, cast, override
|
||||
|
||||
import flask_login
|
||||
from flask import Request, Response, request
|
||||
@@ -11,6 +12,7 @@ from werkzeug.exceptions import NotFound, Unauthorized
|
||||
from configs import dify_config
|
||||
from constants import HEADER_NAME_APP_CODE
|
||||
from core.db.session_factory import session_factory
|
||||
from core.logging.context import set_identity_context
|
||||
from dify_app import DifyApp
|
||||
from libs.passport import PassportService
|
||||
from libs.token import extract_access_token, extract_console_cookie_token, extract_webapp_passport
|
||||
@@ -19,6 +21,8 @@ from models.enums import EndUserType
|
||||
from models.model import AppMCPServer, EndUser
|
||||
from services.account_service import AccountService
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
type LoginUser = Account | EndUser
|
||||
|
||||
|
||||
@@ -156,13 +160,24 @@ def _load_user_from_request(request_from_flask_login: Request, session: Session)
|
||||
@user_logged_in.connect
|
||||
@user_loaded_from_request.connect
|
||||
def on_user_logged_in(_sender: object, user: LoginUser) -> None:
|
||||
"""Called when a user logged in.
|
||||
"""Snapshot authenticated identity into the side-effect-free logging context.
|
||||
|
||||
Note: AccountService.load_logged_in_account will populate user.current_tenant_id
|
||||
through the load_user method, which calls account.set_tenant_id_with_session().
|
||||
"""
|
||||
# tenant_id context variable removed - using current_user.current_tenant_id directly
|
||||
pass
|
||||
set_identity_context()
|
||||
try:
|
||||
match user:
|
||||
case Account():
|
||||
set_identity_context(tenant_id=user.current_tenant_id, user_id=user.id, user_type="account")
|
||||
case EndUser():
|
||||
set_identity_context(tenant_id=user.tenant_id, user_id=user.id, user_type=user.type or "end_user")
|
||||
case _ as unreachable:
|
||||
assert_never(unreachable)
|
||||
except Exception:
|
||||
# Logging enrichment must never make authentication fail.
|
||||
logger.exception("Failed to set logging identity context")
|
||||
return
|
||||
|
||||
|
||||
@login_manager.unauthorized_handler
|
||||
|
||||
@@ -119,6 +119,19 @@ class Storage:
|
||||
def delete(self, filename: str):
|
||||
return self.storage_runner.delete(filename)
|
||||
|
||||
def generate_presigned_url(
|
||||
self,
|
||||
filename: str,
|
||||
*,
|
||||
expires_in: int,
|
||||
content_type: str | None = None,
|
||||
) -> str:
|
||||
return self.storage_runner.generate_presigned_url(
|
||||
filename,
|
||||
expires_in=expires_in,
|
||||
content_type=content_type,
|
||||
)
|
||||
|
||||
def scan(self, path: str, files: bool = True, directories: bool = False) -> list[str]:
|
||||
return self.storage_runner.scan(path, files=files, directories=directories)
|
||||
|
||||
|
||||
@@ -69,12 +69,17 @@ def on_user_loaded(_sender, user: Union["Account", "EndUser"]):
|
||||
if user:
|
||||
try:
|
||||
current_span = get_current_span()
|
||||
if not current_span.is_recording():
|
||||
return
|
||||
tenant_id = extract_tenant_id(user)
|
||||
if not tenant_id:
|
||||
return
|
||||
if current_span:
|
||||
current_span.set_attribute(DifySpanAttributes.TENANT_ID, tenant_id)
|
||||
current_span.set_attribute(GenAIAttributes.USER_ID, user.id)
|
||||
current_span.set_attributes(
|
||||
{
|
||||
DifySpanAttributes.TENANT_ID: tenant_id,
|
||||
GenAIAttributes.USER_ID: user.id,
|
||||
}
|
||||
)
|
||||
except Exception:
|
||||
logger.exception("Error setting tenant and user attributes")
|
||||
pass
|
||||
|
||||
@@ -92,3 +92,21 @@ class AwsS3Storage(BaseStorage):
|
||||
@override
|
||||
def delete(self, filename: str):
|
||||
self.client.delete_object(Bucket=self.bucket_name, Key=filename)
|
||||
|
||||
@override
|
||||
def generate_presigned_url(
|
||||
self,
|
||||
filename: str,
|
||||
*,
|
||||
expires_in: int,
|
||||
content_type: str | None = None,
|
||||
) -> str:
|
||||
params = {"Bucket": self.bucket_name, "Key": filename}
|
||||
if content_type:
|
||||
params["ResponseContentType"] = content_type
|
||||
|
||||
return self.client.generate_presigned_url(
|
||||
"get_object",
|
||||
Params=params,
|
||||
ExpiresIn=expires_in,
|
||||
)
|
||||
|
||||
@@ -31,6 +31,16 @@ class BaseStorage(ABC):
|
||||
def delete(self, filename: str):
|
||||
raise NotImplementedError
|
||||
|
||||
def generate_presigned_url(
|
||||
self,
|
||||
filename: str,
|
||||
*,
|
||||
expires_in: int,
|
||||
content_type: str | None = None,
|
||||
) -> str:
|
||||
"""Generate a temporary direct-download URL when the backend supports it."""
|
||||
raise NotImplementedError("This storage backend doesn't support presigned URLs")
|
||||
|
||||
def scan(self, path, files=True, directories=False) -> list[str]:
|
||||
"""
|
||||
Scan files and directories in the given path.
|
||||
|
||||
@@ -7,6 +7,7 @@ from uuid import uuid4
|
||||
from pydantic import Field, computed_field, field_validator
|
||||
|
||||
from core.entities.execution_extra_content import ExecutionExtraContentDomainModel
|
||||
from core.tools.entities.ui_entities import MessageUIPart, parse_history_ui_parts
|
||||
from fields.base import ResponseModel
|
||||
from fields.conversation_fields import AgentThought, JSONValue, MessageFile
|
||||
from graphon.file import File
|
||||
@@ -64,6 +65,7 @@ class MessageListItem(ResponseModel):
|
||||
status: str
|
||||
error: str | None = None
|
||||
extra_contents: list[ExecutionExtraContentDomainModel]
|
||||
ui_parts: list[MessageUIPart] = Field(default_factory=list, validation_alias="message_metadata_dict")
|
||||
|
||||
@computed_field
|
||||
def total_tokens(self) -> int:
|
||||
@@ -79,6 +81,12 @@ class MessageListItem(ResponseModel):
|
||||
def _normalize_created_at(cls, value: datetime | int | None) -> int | None:
|
||||
return to_timestamp(value)
|
||||
|
||||
@field_validator("ui_parts", mode="before")
|
||||
@classmethod
|
||||
def _normalize_ui_parts(cls, value: object) -> list[MessageUIPart]:
|
||||
raw_parts = value.get("ui_parts", []) if isinstance(value, dict) else value
|
||||
return parse_history_ui_parts(raw_parts)
|
||||
|
||||
|
||||
class WebMessageListItem(MessageListItem):
|
||||
metadata: JSONValueType | None = Field(
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
{
|
||||
"commit": "4310e2d582d25e7de58183f27720afab01e123cf",
|
||||
"openapiSha256": "5827ca930ce38462bfd1b2bef387efbf37eb7ffcaedde4558af2fbaeccbfbc4b",
|
||||
"commit": "a0f50470612cc0b3656f89e4f2435aaf412e6e3b",
|
||||
"openapiSha256": "f18910e9c45a64f0855e0643a7a626fb2889021b4f943458de86c6bd2469facb",
|
||||
"repository": "https://github.com/langgenius/knowledge-fs"
|
||||
}
|
||||
|
||||
@@ -9,6 +9,8 @@ from werkzeug.http import HTTP_STATUS_CODES
|
||||
|
||||
from configs import dify_config
|
||||
from core.errors.error import AppInvokeQuotaExceededError
|
||||
from core.plugin.impl.exc import PluginRuntimeError
|
||||
from extensions.ext_logging import get_request_id
|
||||
from libs.flask_restx_compat import install_swagger_compatibility
|
||||
from libs.token import build_force_logout_cookie_headers
|
||||
|
||||
@@ -100,6 +102,20 @@ def register_external_error_handlers(api: Api, body_formatter: ErrorBodyFormatte
|
||||
data = {"code": "too_many_requests", "message": str(e), "status": status_code}
|
||||
return _finalize(e, data, status_code), status_code
|
||||
|
||||
def handle_plugin_runtime_error(e: PluginRuntimeError):
|
||||
got_request_exception.send(current_app, exception=e)
|
||||
status_code = 502
|
||||
details = {"request_id": get_request_id()}
|
||||
if e.lambda_request_id:
|
||||
details["lambda_request_id"] = e.lambda_request_id
|
||||
data = {
|
||||
"code": "plugin_runtime_error",
|
||||
"message": e.description,
|
||||
"details": details,
|
||||
"status": status_code,
|
||||
}
|
||||
return _finalize(e, data, status_code), status_code
|
||||
|
||||
def handle_general_exception(e: Exception):
|
||||
got_request_exception.send(current_app, exception=e)
|
||||
|
||||
@@ -121,6 +137,7 @@ def register_external_error_handlers(api: Api, body_formatter: ErrorBodyFormatte
|
||||
api.errorhandler(HTTPException)(handle_http_exception)
|
||||
api.errorhandler(ValueError)(handle_value_error)
|
||||
api.errorhandler(AppInvokeQuotaExceededError)(handle_quota_exceeded)
|
||||
api.errorhandler(PluginRuntimeError)(handle_plugin_runtime_error)
|
||||
api.errorhandler(Exception)(handle_general_exception)
|
||||
|
||||
|
||||
|
||||
+5
-2
@@ -221,8 +221,11 @@ def current_timestamp() -> int:
|
||||
def email(email):
|
||||
# Define a regex pattern for email addresses
|
||||
pattern = r"^[\w\.!#$%&'*+\-/=?^_`{|}~]+@([\w-]+\.)+[\w-]{2,}$"
|
||||
# Check if the email matches the pattern
|
||||
if re.match(pattern, email) is not None:
|
||||
# Use re.fullmatch instead of re.match to reject trailing newlines.
|
||||
# In Python, '$' matches at end-of-string OR just before a trailing newline,
|
||||
# so re.match accepts "user@example.com\n". re.fullmatch requires the entire
|
||||
# string to match, closing the mail header-injection vector. (#39234)
|
||||
if re.fullmatch(pattern, email) is not None:
|
||||
return email
|
||||
|
||||
error = f"{email} is not a valid email."
|
||||
|
||||
@@ -23,9 +23,9 @@ class PaginatedResult[T]:
|
||||
|
||||
@property
|
||||
def pages(self) -> int:
|
||||
if self.per_page == 0:
|
||||
if self.total == 0 or self.per_page == 0:
|
||||
return 0
|
||||
return max(1, math.ceil(self.total / self.per_page))
|
||||
return math.ceil(self.total / self.per_page)
|
||||
|
||||
@property
|
||||
def has_next(self) -> bool:
|
||||
|
||||
+77
@@ -0,0 +1,77 @@
|
||||
"""scope agent debug conversations by draft type
|
||||
|
||||
Revision ID: d2825e7b9c10
|
||||
Revises: b8c9d0e1f2a3
|
||||
Create Date: 2026-07-22 15:00:00.000000
|
||||
|
||||
"""
|
||||
|
||||
import sqlalchemy as sa
|
||||
from alembic import op
|
||||
|
||||
import models
|
||||
|
||||
# revision identifiers, used by Alembic.
|
||||
revision = "d2825e7b9c10"
|
||||
down_revision = "b8c9d0e1f2a3"
|
||||
branch_labels = None
|
||||
depends_on = None
|
||||
|
||||
|
||||
def upgrade():
|
||||
# Existing pointers have always represented Build chat because the Agent
|
||||
# detail API exposes them as ``debug_conversation_id`` for that surface.
|
||||
op.add_column(
|
||||
"agent_debug_conversations",
|
||||
sa.Column(
|
||||
"draft_type",
|
||||
sa.String(length=32),
|
||||
nullable=False,
|
||||
server_default=sa.text("'debug_build'"),
|
||||
),
|
||||
)
|
||||
op.drop_constraint(
|
||||
"agent_debug_conversation_agent_account_unique",
|
||||
"agent_debug_conversations",
|
||||
type_="unique",
|
||||
)
|
||||
op.create_unique_constraint(
|
||||
"agent_debug_conversation_agent_account_draft_type_unique",
|
||||
"agent_debug_conversations",
|
||||
["tenant_id", "agent_id", "account_id", "draft_type"],
|
||||
)
|
||||
|
||||
|
||||
def downgrade():
|
||||
debug_conversations = sa.table(
|
||||
"agent_debug_conversations",
|
||||
sa.column("tenant_id", models.types.StringUUID()),
|
||||
sa.column("agent_id", models.types.StringUUID()),
|
||||
sa.column("account_id", models.types.StringUUID()),
|
||||
sa.column("draft_type", sa.String(length=32)),
|
||||
)
|
||||
build_conversations = debug_conversations.alias("build_conversations")
|
||||
op.get_bind().execute(
|
||||
sa.delete(debug_conversations).where(
|
||||
debug_conversations.c.draft_type == "draft",
|
||||
sa.exists(
|
||||
sa.select(sa.literal(1)).where(
|
||||
build_conversations.c.tenant_id == debug_conversations.c.tenant_id,
|
||||
build_conversations.c.agent_id == debug_conversations.c.agent_id,
|
||||
build_conversations.c.account_id == debug_conversations.c.account_id,
|
||||
build_conversations.c.draft_type == "debug_build",
|
||||
)
|
||||
),
|
||||
)
|
||||
)
|
||||
op.drop_constraint(
|
||||
"agent_debug_conversation_agent_account_draft_type_unique",
|
||||
"agent_debug_conversations",
|
||||
type_="unique",
|
||||
)
|
||||
op.create_unique_constraint(
|
||||
"agent_debug_conversation_agent_account_unique",
|
||||
"agent_debug_conversations",
|
||||
["tenant_id", "agent_id", "account_id"],
|
||||
)
|
||||
op.drop_column("agent_debug_conversations", "draft_type")
|
||||
+12
-3
@@ -222,11 +222,13 @@ class Agent(DefaultFieldsMixin, Base):
|
||||
|
||||
|
||||
class AgentDebugConversation(DefaultFieldsMixin, Base):
|
||||
"""Per-account console debug conversation for an Agent App.
|
||||
"""Per-account, per-draft console debug conversation for an Agent App.
|
||||
|
||||
Agent App preview state must be isolated by editor account. The Agent row is
|
||||
shared by everyone in the workspace, so this table owns the user-specific
|
||||
conversation pointer used by console debug chat.
|
||||
conversation pointers used by console debug chat. ``draft`` is the Preview
|
||||
conversation and ``debug_build`` is the Build conversation; they must never
|
||||
share persisted messages or runtime sessions.
|
||||
"""
|
||||
|
||||
__tablename__ = "agent_debug_conversations"
|
||||
@@ -236,7 +238,8 @@ class AgentDebugConversation(DefaultFieldsMixin, Base):
|
||||
"tenant_id",
|
||||
"agent_id",
|
||||
"account_id",
|
||||
name="agent_debug_conversation_agent_account_unique",
|
||||
"draft_type",
|
||||
name="agent_debug_conversation_agent_account_draft_type_unique",
|
||||
),
|
||||
Index("agent_debug_conversation_conversation_idx", "conversation_id"),
|
||||
Index("agent_debug_conversation_account_idx", "tenant_id", "account_id"),
|
||||
@@ -246,6 +249,12 @@ class AgentDebugConversation(DefaultFieldsMixin, Base):
|
||||
agent_id: Mapped[str] = mapped_column(StringUUID, nullable=False)
|
||||
app_id: Mapped[str] = mapped_column(StringUUID, nullable=False)
|
||||
account_id: Mapped[str] = mapped_column(StringUUID, nullable=False)
|
||||
draft_type: Mapped[AgentConfigDraftType] = mapped_column(
|
||||
EnumText(AgentConfigDraftType, length=32),
|
||||
nullable=False,
|
||||
default=AgentConfigDraftType.DEBUG_BUILD,
|
||||
server_default=sa.text("'debug_build'"),
|
||||
)
|
||||
conversation_id: Mapped[str] = mapped_column(StringUUID, nullable=False)
|
||||
|
||||
|
||||
|
||||
@@ -7,6 +7,7 @@ from typing import Annotated, Any, Final, Literal, Self
|
||||
from pydantic import BaseModel, ConfigDict, Field, WithJsonSchema, field_validator, model_validator
|
||||
|
||||
from core.rag.entities.metadata_entities import ConditionValue, SupportedComparisonOperator
|
||||
from core.tools.entities.tool_entities import ToolProviderType
|
||||
from core.workflow.file_reference import is_canonical_file_reference
|
||||
from graphon.file import FileTransferMethod, FileType
|
||||
|
||||
@@ -43,18 +44,28 @@ _DECLARED_OUTPUT_CHILDREN_JSON_SCHEMA = {
|
||||
},
|
||||
"description": {"anyOf": [{"type": "string"}, {"type": "null"}]},
|
||||
"required": {"type": "boolean"},
|
||||
"file": {"type": "object", "additionalProperties": True},
|
||||
"file": {
|
||||
"anyOf": [
|
||||
{"type": "object", "additionalProperties": True},
|
||||
{"type": "null"},
|
||||
]
|
||||
},
|
||||
"array_item": {
|
||||
"type": "object",
|
||||
"additionalProperties": True,
|
||||
"properties": {
|
||||
"type": {
|
||||
"type": "string",
|
||||
"enum": [item.value for item in DeclaredOutputType],
|
||||
"anyOf": [
|
||||
{
|
||||
"type": "object",
|
||||
"additionalProperties": True,
|
||||
"properties": {
|
||||
"type": {
|
||||
"type": "string",
|
||||
"enum": [item.value for item in DeclaredOutputType],
|
||||
},
|
||||
"description": {"anyOf": [{"type": "string"}, {"type": "null"}]},
|
||||
"children": {"type": "array", "items": {"type": "object", "additionalProperties": True}},
|
||||
},
|
||||
},
|
||||
"description": {"anyOf": [{"type": "string"}, {"type": "null"}]},
|
||||
"children": {"type": "array", "items": {"type": "object", "additionalProperties": True}},
|
||||
},
|
||||
{"type": "null"},
|
||||
]
|
||||
},
|
||||
"children": {"type": "array", "items": {"type": "object", "additionalProperties": True}},
|
||||
},
|
||||
@@ -653,11 +664,7 @@ class AgentSoulDifyToolConfig(BaseModel):
|
||||
model_config = ConfigDict(extra="ignore")
|
||||
|
||||
enabled: bool = True
|
||||
# ``plugin`` remains the default for legacy Agent Soul payloads. The runtime
|
||||
# now also accepts ``builtin`` / ``api`` / ``workflow`` / ``mcp`` here and
|
||||
# routes them through ``dify.core.tools``; keeping the default narrow still
|
||||
# makes a missing field resolve against the plugin provider table.
|
||||
provider_type: str = "plugin"
|
||||
provider_type: ToolProviderType
|
||||
provider_id: str | None = Field(default=None, max_length=255)
|
||||
plugin_id: str | None = Field(default=None, max_length=255)
|
||||
provider: str | None = Field(default=None, max_length=255)
|
||||
|
||||
+32
-14
@@ -56,6 +56,7 @@ from .provider_ids import GenericProviderID
|
||||
from .types import EnumText, LongText, StringUUID
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from .agent import Agent
|
||||
from .workflow import Workflow
|
||||
|
||||
|
||||
@@ -501,25 +502,42 @@ class App(Base):
|
||||
Resolved via ``Agent.app_id`` so the console can open the Composer in
|
||||
roster-detail mode from the app id. ``None`` for non-agent apps.
|
||||
"""
|
||||
agent = self.agent_app_binding_with_session(session=session)
|
||||
return agent.id if agent else None
|
||||
|
||||
def agent_app_binding_with_session(self, *, session: Session, include_archived: bool = False) -> Agent | None:
|
||||
"""For an Agent App (mode=agent), the Agent bound to it.
|
||||
|
||||
A roster Agent is bound through ``Agent.app_id``; a workflow-only Agent
|
||||
is bound to its hidden runtime backing App through
|
||||
``Agent.backing_app_id``. Callers branch on ``Agent.scope`` to tell the
|
||||
public roster Agent App apart from the hidden backing App. Archived
|
||||
Agents are excluded unless ``include_archived`` is set (authorization
|
||||
gates must keep covering an Agent App after its Agent is archived).
|
||||
``None`` for non-agent apps and unbound agent apps.
|
||||
"""
|
||||
if self.mode != AppMode.AGENT:
|
||||
return None
|
||||
from .agent import APP_BACKED_AGENT_SOURCES, Agent, AgentScope, AgentStatus
|
||||
|
||||
agent = session.scalar(
|
||||
select(Agent).where(
|
||||
Agent.tenant_id == self.tenant_id,
|
||||
sa.or_(
|
||||
sa.and_(
|
||||
Agent.app_id == self.id,
|
||||
Agent.scope == AgentScope.ROSTER,
|
||||
Agent.source.in_(APP_BACKED_AGENT_SOURCES),
|
||||
),
|
||||
Agent.backing_app_id == self.id,
|
||||
conditions = [
|
||||
Agent.tenant_id == self.tenant_id,
|
||||
sa.or_(
|
||||
sa.and_(
|
||||
Agent.app_id == self.id,
|
||||
Agent.scope == AgentScope.ROSTER,
|
||||
Agent.source.in_(APP_BACKED_AGENT_SOURCES),
|
||||
),
|
||||
Agent.status == AgentStatus.ACTIVE,
|
||||
)
|
||||
)
|
||||
return agent.id if agent else None
|
||||
sa.and_(
|
||||
Agent.backing_app_id == self.id,
|
||||
Agent.scope == AgentScope.WORKFLOW_ONLY,
|
||||
),
|
||||
),
|
||||
]
|
||||
if not include_archived:
|
||||
conditions.append(Agent.status == AgentStatus.ACTIVE)
|
||||
|
||||
return session.scalar(select(Agent).where(*conditions).limit(1))
|
||||
|
||||
@property
|
||||
def api_base_url(self) -> str:
|
||||
|
||||
@@ -260,7 +260,12 @@ Get account avatar url
|
||||
| 200 | Success | **application/json**: [AccountResponse](#accountresponse)<br> |
|
||||
|
||||
### [POST] /activate
|
||||
**Accept an invitation without letting an existing session act for another account**
|
||||
|
||||
Activate account with invitation token
|
||||
Token-only activation remains available for legacy clients. When the request already
|
||||
carries a console session, that session must belong to the account encoded in the
|
||||
invitation before the token is consumed or tenant membership is changed.
|
||||
|
||||
#### Request Body
|
||||
|
||||
@@ -531,6 +536,7 @@ Run a build-draft Agent App turn that asks the agent to push config updates
|
||||
| Code | Description | Schema |
|
||||
| ---- | ----------- | ------ |
|
||||
| 200 | Agent build draft | **application/json**: [AgentBuildDraftResponse](#agentbuilddraftresponse)<br> |
|
||||
| 404 | Agent build draft not found | |
|
||||
|
||||
### [PUT] /agent/{agent_id}/build-draft
|
||||
#### Parameters
|
||||
@@ -958,6 +964,12 @@ Stop a running Agent App chat message generation
|
||||
| ---- | ---------- | ----------- | -------- | ------ |
|
||||
| agent_id | path | | Yes | string (uuid) |
|
||||
|
||||
#### Request Body
|
||||
|
||||
| Required | Schema |
|
||||
| -------- | ------ |
|
||||
| No | **application/json**: [AgentDebugConversationRefreshPayload](#agentdebugconversationrefreshpayload)<br> |
|
||||
|
||||
#### Responses
|
||||
|
||||
| Code | Description | Schema |
|
||||
@@ -11332,6 +11344,12 @@ Returns permission flags that control workspace features like member invitations
|
||||
| 200 | Success | **application/json**: [PluginDecodeResponse](#plugindecoderesponse)<br> |
|
||||
|
||||
### [POST] /workspaces/current/plugin/upload/pkg
|
||||
#### Request Body
|
||||
|
||||
| Required | Schema |
|
||||
| -------- | ------ |
|
||||
| Yes | **multipart/form-data**: { **"pkg"**: binary }<br> |
|
||||
|
||||
#### Responses
|
||||
|
||||
| Code | Description | Schema |
|
||||
@@ -13278,7 +13296,7 @@ Model class for AI model.
|
||||
| maintainer | string | | No |
|
||||
| max_active_requests | integer | | No |
|
||||
| mode | string | | Yes |
|
||||
| model_config | [ModelConfig](#modelconfig) | | No |
|
||||
| model_config | [AppModelConfigResponse](#appmodelconfigresponse) | | No |
|
||||
| name | string | | Yes |
|
||||
| permission_keys | [ string ] | | No |
|
||||
| role | string | | No |
|
||||
@@ -13899,6 +13917,12 @@ Stable Agent Soul reference to one normalized skill archive.
|
||||
| date | string | | Yes |
|
||||
| message_count | integer | | Yes |
|
||||
|
||||
#### AgentDebugConversationRefreshPayload
|
||||
|
||||
| Name | Type | Description | Required |
|
||||
| ---- | ---- | ----------- | -------- |
|
||||
| draft_type | [AgentConfigDraftType](#agentconfigdrafttype) | Agent draft surface whose conversation should be refreshed | No |
|
||||
|
||||
#### AgentDebugConversationRefreshResponse
|
||||
|
||||
| Name | Type | Description | Required |
|
||||
@@ -14737,7 +14761,7 @@ should send ``plugin_id`` + ``provider`` when available.
|
||||
| plugin_id | string | | No |
|
||||
| provider | string | | No |
|
||||
| provider_id | string | | No |
|
||||
| provider_type | string, <br>**Default:** plugin | | No |
|
||||
| provider_type | [ToolProviderType](#toolprovidertype) | | Yes |
|
||||
| runtime_parameters | object | | No |
|
||||
| tool_name | string | | No |
|
||||
|
||||
@@ -15348,7 +15372,6 @@ This class is used to store the schema information of an api based tool.
|
||||
| Name | Type | Description | Required |
|
||||
| ---- | ---- | ----------- | -------- |
|
||||
| access_mode | string | | No |
|
||||
| app_model_config | [ModelConfig](#modelconfig) | | No |
|
||||
| created_at | integer | | No |
|
||||
| created_by | string | | No |
|
||||
| description | string | | No |
|
||||
@@ -15358,7 +15381,8 @@ This class is used to store the schema information of an api based tool.
|
||||
| icon_background | string | | No |
|
||||
| id | string | | Yes |
|
||||
| maintainer | string | | No |
|
||||
| mode_compatible_with_agent | string | | Yes |
|
||||
| mode | string | | Yes |
|
||||
| model_config | [AppModelConfigResponse](#appmodelconfigresponse) | | No |
|
||||
| name | string | | Yes |
|
||||
| permission_keys | [ string ] | | No |
|
||||
| tags | [ [Tag](#tag) ] | | No |
|
||||
@@ -15420,7 +15444,7 @@ This class is used to store the schema information of an api based tool.
|
||||
| maintainer | string | | No |
|
||||
| max_active_requests | integer | | No |
|
||||
| mode | string | | Yes |
|
||||
| model_config | [ModelConfig](#modelconfig) | | No |
|
||||
| model_config | [AppModelConfigResponse](#appmodelconfigresponse) | | No |
|
||||
| name | string | | Yes |
|
||||
| permission_keys | [ string ] | | No |
|
||||
| site | [AppDetailSiteResponse](#appdetailsiteresponse) | | No |
|
||||
@@ -15519,6 +15543,35 @@ AppMCPServer Status Enum
|
||||
| ---- | ---- | ----------- | -------- |
|
||||
| AppMCPServerStatus | string | AppMCPServer Status Enum | |
|
||||
|
||||
#### AppModelConfigResponse
|
||||
|
||||
| Name | Type | Description | Required |
|
||||
| ---- | ---- | ----------- | -------- |
|
||||
| agent_mode | | | No |
|
||||
| annotation_reply | | | No |
|
||||
| chat_prompt_config | | | No |
|
||||
| completion_prompt_config | | | No |
|
||||
| created_at | integer | | No |
|
||||
| created_by | string | | No |
|
||||
| dataset_configs | | | No |
|
||||
| dataset_query_variable | string | | No |
|
||||
| external_data_tools | | | No |
|
||||
| file_upload | | | No |
|
||||
| model | | | No |
|
||||
| more_like_this | | | No |
|
||||
| opening_statement | string | | No |
|
||||
| pre_prompt | string | | No |
|
||||
| prompt_type | string | | No |
|
||||
| retriever_resource | | | No |
|
||||
| sensitive_word_avoidance | | | No |
|
||||
| speech_to_text | | | No |
|
||||
| suggested_questions | | | No |
|
||||
| suggested_questions_after_answer | | | No |
|
||||
| text_to_speech | | | No |
|
||||
| updated_at | integer | | No |
|
||||
| updated_by | string | | No |
|
||||
| user_input_form | | | No |
|
||||
|
||||
#### AppNamePayload
|
||||
|
||||
| Name | Type | Description | Required |
|
||||
@@ -17147,7 +17200,7 @@ about. Stage 4 §4.2.
|
||||
|
||||
| Name | Type | Description | Required |
|
||||
| ---- | ---- | ----------- | -------- |
|
||||
| children | [ { **"array_item"**: { **"children"**: [ object ], **"description"**: , **"type"**: string, <br>**Available values:** "array", "boolean", "file", "number", "object", "string" }, **"children"**: [ object ], **"description"**: , **"file"**: object, **"name"**: string, **"required"**: boolean, **"type"**: string, <br>**Available values:** "array", "boolean", "file", "number", "object", "string" } ] | | No |
|
||||
| children | [ { **"array_item"**: , **"children"**: [ object ], **"description"**: , **"file"**: , **"name"**: string, **"required"**: boolean, **"type"**: string, <br>**Available values:** "array", "boolean", "file", "number", "object", "string" } ] | | No |
|
||||
| description | string | | No |
|
||||
| type | [DeclaredOutputType](#declaredoutputtype) | | Yes |
|
||||
|
||||
@@ -17176,7 +17229,7 @@ code can call ``output.failure_strategy.on_failure`` without None-guards.
|
||||
| ---- | ---- | ----------- | -------- |
|
||||
| array_item | [DeclaredArrayItem](#declaredarrayitem) | | No |
|
||||
| check | [DeclaredOutputCheckConfig](#declaredoutputcheckconfig) | | No |
|
||||
| children | [ { **"array_item"**: { **"children"**: [ object ], **"description"**: , **"type"**: string, <br>**Available values:** "array", "boolean", "file", "number", "object", "string" }, **"children"**: [ object ], **"description"**: , **"file"**: object, **"name"**: string, **"required"**: boolean, **"type"**: string, <br>**Available values:** "array", "boolean", "file", "number", "object", "string" } ] | | No |
|
||||
| children | [ { **"array_item"**: , **"children"**: [ object ], **"description"**: , **"file"**: , **"name"**: string, **"required"**: boolean, **"type"**: string, <br>**Available values:** "array", "boolean", "file", "number", "object", "string" } ] | | No |
|
||||
| description | string | | No |
|
||||
| failure_strategy | [DeclaredOutputFailureStrategy](#declaredoutputfailurestrategy) | | No |
|
||||
| file | [DeclaredOutputFileConfig](#declaredoutputfileconfig) | | No |
|
||||
@@ -19506,6 +19559,7 @@ Coarse node-level status used by Inspector to pick a banner.
|
||||
| ---- | ---- | ----------- | -------- |
|
||||
| avatar | string | | No |
|
||||
| email | string | | Yes |
|
||||
| id | string | | Yes |
|
||||
| interface_language | string | | Yes |
|
||||
| name | string | | Yes |
|
||||
| timezone | string | | Yes |
|
||||
@@ -21954,9 +22008,9 @@ The subscription constructor of the trigger provider
|
||||
|
||||
| Name | Type | Description | Required |
|
||||
| ---- | ---- | ----------- | -------- |
|
||||
| hash | string | | No |
|
||||
| result | string | | No |
|
||||
| updated_at | string | | No |
|
||||
| hash | string | | Yes |
|
||||
| result | string | | Yes |
|
||||
| updated_at | integer | | Yes |
|
||||
|
||||
#### SystemConfigurationResponse
|
||||
|
||||
@@ -21988,6 +22042,7 @@ Model class for provider system configuration response.
|
||||
| is_allow_create_workspace | boolean | | Yes |
|
||||
| is_allow_register | boolean | | Yes |
|
||||
| is_email_setup | boolean | | Yes |
|
||||
| knowledge_fs_enabled | boolean | | Yes |
|
||||
| license | [LicenseModel](#licensemodel) | | Yes |
|
||||
| max_plugin_package_size | integer, <br>**Default:** 15728640 | | Yes |
|
||||
| plugin_installation_permission | [PluginInstallationPermissionModel](#plugininstallationpermissionmodel) | | Yes |
|
||||
@@ -22281,7 +22336,7 @@ Tool label
|
||||
|
||||
#### ToolParameter
|
||||
|
||||
Overrides type
|
||||
Tool-specific parameter declaration and invocation-value normalization.
|
||||
|
||||
| Name | Type | Description | Required |
|
||||
| ---- | ---- | ----------- | -------- |
|
||||
@@ -22294,6 +22349,7 @@ Overrides type
|
||||
| llm_description | string | | No |
|
||||
| max | number<br>integer | | No |
|
||||
| min | number<br>integer | | No |
|
||||
| multiple | boolean | Whether the parameter is multiple select, only valid for select or dynamic-select type | No |
|
||||
| name | string | The name of the parameter | Yes |
|
||||
| options | [ [PluginParameterOption](#pluginparameteroption) ] | | No |
|
||||
| placeholder | [I18nObject](#i18nobject) | The placeholder presented to the user | No |
|
||||
|
||||
@@ -1582,6 +1582,7 @@ Default configuration for form inputs.
|
||||
| is_allow_create_workspace | boolean | | Yes |
|
||||
| is_allow_register | boolean | | Yes |
|
||||
| is_email_setup | boolean | | Yes |
|
||||
| knowledge_fs_enabled | boolean | | Yes |
|
||||
| license | [LicenseModel](#licensemodel) | | Yes |
|
||||
| max_plugin_package_size | integer, <br>**Default:** 15728640 | | Yes |
|
||||
| plugin_installation_permission | [PluginInstallationPermissionModel](#plugininstallationpermissionmodel) | | Yes |
|
||||
@@ -1733,7 +1734,7 @@ in form definiton, or a variable while the workflow is running.
|
||||
| icon | string | | No |
|
||||
| icon_background | string | | No |
|
||||
| icon_type | string | | No |
|
||||
| icon_url | string | | Yes |
|
||||
| icon_url | string | | No |
|
||||
| input_placeholder | string | | No |
|
||||
| privacy_policy | string | | No |
|
||||
| prompt_public | boolean | | No |
|
||||
|
||||
@@ -1164,6 +1164,20 @@ class AgentComposerService:
|
||||
agent.active_config_is_published = True
|
||||
agent.updated_by = account_id
|
||||
binding.current_snapshot_id = version.id
|
||||
normal_draft = cls._get_agent_draft(
|
||||
session=session,
|
||||
tenant_id=tenant_id,
|
||||
agent_id=agent.id,
|
||||
draft_type=AgentConfigDraftType.DRAFT,
|
||||
account_id=None,
|
||||
)
|
||||
if normal_draft is not None and cls._rebase_workflow_only_normal_draft(
|
||||
agent=agent,
|
||||
draft=normal_draft,
|
||||
snapshot=version,
|
||||
updated_by=account_id,
|
||||
):
|
||||
session.flush()
|
||||
binding.updated_by = account_id
|
||||
return binding
|
||||
|
||||
@@ -1748,6 +1762,47 @@ class AgentComposerService:
|
||||
stmt = stmt.where(AgentConfigDraft.account_id.is_(None))
|
||||
return session.scalar(stmt.order_by(AgentConfigDraft.updated_at.desc()).limit(1))
|
||||
|
||||
@classmethod
|
||||
def get_or_create_normal_agent_draft(
|
||||
cls,
|
||||
*,
|
||||
session: Session,
|
||||
tenant_id: str,
|
||||
agent: Agent,
|
||||
created_by: str | None,
|
||||
) -> AgentConfigDraft:
|
||||
"""Resolve the shared Preview draft, rebasing inline agents when needed."""
|
||||
return cls._get_or_create_agent_draft(
|
||||
session=session,
|
||||
tenant_id=tenant_id,
|
||||
agent=agent,
|
||||
draft_type=AgentConfigDraftType.DRAFT,
|
||||
account_id=None,
|
||||
created_by=created_by,
|
||||
)
|
||||
|
||||
@staticmethod
|
||||
def _rebase_workflow_only_normal_draft(
|
||||
*,
|
||||
agent: Agent,
|
||||
draft: AgentConfigDraft,
|
||||
snapshot: AgentConfigSnapshot,
|
||||
updated_by: str | None,
|
||||
) -> bool:
|
||||
if (
|
||||
agent.scope != AgentScope.WORKFLOW_ONLY
|
||||
or draft.draft_type != AgentConfigDraftType.DRAFT
|
||||
or draft.account_id is not None
|
||||
or not agent.active_config_snapshot_id
|
||||
or draft.base_snapshot_id == agent.active_config_snapshot_id
|
||||
or snapshot.id != agent.active_config_snapshot_id
|
||||
):
|
||||
return False
|
||||
draft.base_snapshot_id = snapshot.id
|
||||
draft.config_snapshot = AgentSoulConfig.model_validate(snapshot.config_snapshot_dict)
|
||||
draft.updated_by = updated_by
|
||||
return True
|
||||
|
||||
@classmethod
|
||||
def _get_or_create_agent_draft(
|
||||
cls,
|
||||
@@ -1767,6 +1822,26 @@ class AgentComposerService:
|
||||
account_id=account_id,
|
||||
)
|
||||
if draft is not None:
|
||||
if (
|
||||
agent.scope == AgentScope.WORKFLOW_ONLY
|
||||
and draft_type == AgentConfigDraftType.DRAFT
|
||||
and draft.account_id is None
|
||||
and agent.active_config_snapshot_id
|
||||
and draft.base_snapshot_id != agent.active_config_snapshot_id
|
||||
):
|
||||
active_snapshot = cls._get_version_if_present(
|
||||
session=session,
|
||||
tenant_id=tenant_id,
|
||||
agent_id=agent.id,
|
||||
version_id=agent.active_config_snapshot_id,
|
||||
)
|
||||
if active_snapshot is not None and cls._rebase_workflow_only_normal_draft(
|
||||
agent=agent,
|
||||
draft=draft,
|
||||
snapshot=active_snapshot,
|
||||
updated_by=agent.updated_by or agent.created_by,
|
||||
):
|
||||
session.flush()
|
||||
return draft
|
||||
base_snapshot = cls._get_version_if_present(
|
||||
session=session,
|
||||
|
||||
@@ -1,8 +1,11 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
from collections.abc import Mapping
|
||||
from dataclasses import dataclass
|
||||
from datetime import datetime
|
||||
from decimal import Decimal
|
||||
from enum import Enum
|
||||
from typing import Any
|
||||
|
||||
import sqlalchemy as sa
|
||||
@@ -11,6 +14,7 @@ from sqlalchemy.orm import aliased
|
||||
|
||||
from configs import dify_config
|
||||
from core.app.entities.app_invoke_entities import InvokeFrom
|
||||
from graphon.enums import WorkflowNodeExecutionStatus
|
||||
from libs.helper import convert_datetime_to_date, escape_like_pattern, to_timestamp
|
||||
from models.agent import WorkflowAgentNodeBinding
|
||||
from models.enums import CreatorUserRole, MessageStatus
|
||||
@@ -194,11 +198,12 @@ class AgentObservabilityService:
|
||||
self, *, app: App, agent_id: str, conversation_id: str, params: AgentLogQueryParams
|
||||
) -> dict[str, Any]:
|
||||
source_filters = self.resolve_source_filters(params.sources)
|
||||
rows: list[Message] = []
|
||||
rows: list[dict[str, Any]] = []
|
||||
for source_filter in source_filters:
|
||||
if source_filter.kind in {"all", "webapp"}:
|
||||
rows.extend(
|
||||
self._list_webapp_messages(
|
||||
self.serialize_log_message(message)
|
||||
for message in self._list_webapp_messages(
|
||||
app=app,
|
||||
conversation_id=conversation_id,
|
||||
params=params,
|
||||
@@ -216,18 +221,18 @@ class AgentObservabilityService:
|
||||
)
|
||||
)
|
||||
|
||||
deduped = {message.id: message for message in rows}
|
||||
sort_column = Message.created_at if params.sort_by == "created_at" else Message.updated_at
|
||||
deduped = {row["id"]: row for row in rows}
|
||||
sort_key = "created_at" if params.sort_by == "created_at" else "updated_at"
|
||||
sorted_rows = sorted(
|
||||
deduped.values(),
|
||||
key=lambda message: (getattr(message, sort_column.key), message.id),
|
||||
key=lambda row: (row[sort_key] or 0, row["id"]),
|
||||
reverse=params.sort_order != "asc",
|
||||
)
|
||||
total = len(sorted_rows)
|
||||
start = (params.page - 1) * params.limit
|
||||
end = start + params.limit
|
||||
return {
|
||||
"data": [self.serialize_log_message(message) for message in sorted_rows[start:end]],
|
||||
"data": sorted_rows[start:end],
|
||||
"page": params.page,
|
||||
"limit": params.limit,
|
||||
"total": total,
|
||||
@@ -284,22 +289,20 @@ class AgentObservabilityService:
|
||||
workflow_app = aliased(App)
|
||||
stmt = (
|
||||
select(
|
||||
Conversation,
|
||||
WorkflowNodeExecutionModel.id.label("node_execution_id"),
|
||||
WorkflowNodeExecutionModel.title.label("node_title"),
|
||||
WorkflowNodeExecutionModel.status.label("node_status"),
|
||||
WorkflowNodeExecutionModel.created_by_role.label("node_created_by_role"),
|
||||
WorkflowNodeExecutionModel.created_by.label("node_created_by"),
|
||||
WorkflowNodeExecutionModel.created_at.label("node_created_at"),
|
||||
WorkflowNodeExecutionModel.finished_at.label("node_finished_at"),
|
||||
workflow_app,
|
||||
WorkflowAgentNodeBinding.workflow_id,
|
||||
WorkflowAgentNodeBinding.workflow_version,
|
||||
WorkflowAgentNodeBinding.node_id,
|
||||
func.count(sa.distinct(Message.id)).label("message_count"),
|
||||
func.max(Message.created_at).label("created_at"),
|
||||
func.max(Message.updated_at).label("updated_at"),
|
||||
func.sum(sa.case((Message.status == MessageStatus.PAUSED, 1), else_=0)).label("paused_count"),
|
||||
func.sum(
|
||||
sa.case((or_(Message.error.is_not(None), Message.status == MessageStatus.ERROR), 1), else_=0)
|
||||
).label("failed_count"),
|
||||
)
|
||||
.select_from(Message)
|
||||
.join(Conversation, Conversation.id == Message.conversation_id)
|
||||
.join(WorkflowRun, WorkflowRun.id == Message.workflow_run_id)
|
||||
.select_from(WorkflowNodeExecutionModel)
|
||||
.join(WorkflowRun, WorkflowRun.id == WorkflowNodeExecutionModel.workflow_run_id)
|
||||
.join(
|
||||
WorkflowAgentNodeBinding,
|
||||
and_(
|
||||
@@ -310,40 +313,32 @@ class AgentObservabilityService:
|
||||
WorkflowAgentNodeBinding.workflow_version == WorkflowRun.version,
|
||||
),
|
||||
)
|
||||
.join(
|
||||
WorkflowNodeExecutionModel,
|
||||
and_(
|
||||
WorkflowNodeExecutionModel.workflow_run_id == WorkflowRun.id,
|
||||
WorkflowNodeExecutionModel.node_id == WorkflowAgentNodeBinding.node_id,
|
||||
),
|
||||
)
|
||||
.join(workflow_app, workflow_app.id == WorkflowAgentNodeBinding.app_id)
|
||||
.where(Message.workflow_run_id.is_not(None), Conversation.app_id == WorkflowAgentNodeBinding.app_id)
|
||||
.group_by(
|
||||
Conversation.id,
|
||||
workflow_app.id,
|
||||
WorkflowAgentNodeBinding.workflow_id,
|
||||
WorkflowAgentNodeBinding.workflow_version,
|
||||
WorkflowAgentNodeBinding.node_id,
|
||||
.where(
|
||||
WorkflowNodeExecutionModel.tenant_id == app.tenant_id,
|
||||
WorkflowNodeExecutionModel.app_id == WorkflowAgentNodeBinding.app_id,
|
||||
WorkflowNodeExecutionModel.workflow_id == WorkflowAgentNodeBinding.workflow_id,
|
||||
WorkflowNodeExecutionModel.node_id == WorkflowAgentNodeBinding.node_id,
|
||||
)
|
||||
)
|
||||
stmt = self._apply_observability_filters(stmt, params=params, source_filter=source_filter)
|
||||
stmt = self._apply_workflow_node_filters(stmt, params=params, workflow_app=workflow_app)
|
||||
stmt = self._apply_workflow_source_filter(stmt, source_filter)
|
||||
rows = list(self._session.execute(stmt).all())
|
||||
return [
|
||||
self._serialize_conversation_log(
|
||||
conversation=row[0],
|
||||
message_count=row.message_count,
|
||||
paused_count=row.paused_count,
|
||||
failed_count=row.failed_count,
|
||||
self._serialize_workflow_execution_log(
|
||||
node_execution_id=row.node_execution_id,
|
||||
title=row.node_title,
|
||||
status=row.node_status,
|
||||
created_by_role=row.node_created_by_role,
|
||||
created_by=row.node_created_by,
|
||||
created_at=row.node_created_at,
|
||||
finished_at=row.node_finished_at,
|
||||
source=self._serialize_workflow_source(
|
||||
app=row[1],
|
||||
app=row[7],
|
||||
workflow_id=row.workflow_id,
|
||||
workflow_version=row.workflow_version,
|
||||
node_id=row.node_id,
|
||||
),
|
||||
created_at=row.created_at,
|
||||
updated_at=row.updated_at,
|
||||
)
|
||||
for row in rows
|
||||
]
|
||||
@@ -363,10 +358,12 @@ class AgentObservabilityService:
|
||||
conversation_id: str,
|
||||
params: AgentLogQueryParams,
|
||||
source_filter: AgentSourceFilter,
|
||||
) -> list[Message]:
|
||||
) -> list[dict[str, Any]]:
|
||||
workflow_app = aliased(App)
|
||||
stmt = (
|
||||
select(Message)
|
||||
.join(WorkflowRun, WorkflowRun.id == Message.workflow_run_id)
|
||||
select(WorkflowNodeExecutionModel)
|
||||
.select_from(WorkflowNodeExecutionModel)
|
||||
.join(WorkflowRun, WorkflowRun.id == WorkflowNodeExecutionModel.workflow_run_id)
|
||||
.join(
|
||||
WorkflowAgentNodeBinding,
|
||||
and_(
|
||||
@@ -377,18 +374,23 @@ class AgentObservabilityService:
|
||||
WorkflowAgentNodeBinding.workflow_version == WorkflowRun.version,
|
||||
),
|
||||
)
|
||||
.join(
|
||||
WorkflowNodeExecutionModel,
|
||||
and_(
|
||||
WorkflowNodeExecutionModel.workflow_run_id == WorkflowRun.id,
|
||||
WorkflowNodeExecutionModel.node_id == WorkflowAgentNodeBinding.node_id,
|
||||
),
|
||||
.join(workflow_app, workflow_app.id == WorkflowAgentNodeBinding.app_id)
|
||||
.where(
|
||||
WorkflowNodeExecutionModel.id == conversation_id,
|
||||
WorkflowNodeExecutionModel.tenant_id == app.tenant_id,
|
||||
WorkflowNodeExecutionModel.app_id == WorkflowAgentNodeBinding.app_id,
|
||||
WorkflowNodeExecutionModel.workflow_id == WorkflowAgentNodeBinding.workflow_id,
|
||||
WorkflowNodeExecutionModel.node_id == WorkflowAgentNodeBinding.node_id,
|
||||
)
|
||||
.where(Message.conversation_id == conversation_id)
|
||||
)
|
||||
stmt = self._apply_message_filters(stmt, params=params, source_filter=source_filter)
|
||||
stmt = self._apply_workflow_node_filters(stmt, params=params, workflow_app=workflow_app)
|
||||
stmt = self._apply_workflow_source_filter(stmt, source_filter)
|
||||
return list(self._session.scalars(stmt.order_by(Message.created_at.desc(), Message.id.desc())).all())
|
||||
executions = list(
|
||||
self._session.scalars(
|
||||
stmt.order_by(WorkflowNodeExecutionModel.created_at.desc(), WorkflowNodeExecutionModel.id.desc())
|
||||
).all()
|
||||
)
|
||||
return [self.serialize_workflow_node_message(execution) for execution in executions]
|
||||
|
||||
def _list_workflow_sources(self, *, app: App, agent_id: str) -> list[dict[str, Any]]:
|
||||
workflow_app = aliased(App)
|
||||
@@ -443,6 +445,62 @@ class AgentObservabilityService:
|
||||
stmt = cls._apply_status_filter(stmt, params.statuses)
|
||||
return stmt
|
||||
|
||||
@classmethod
|
||||
def _apply_workflow_node_filters(cls, stmt, *, params: AgentLogQueryParams, workflow_app):
|
||||
if params.start:
|
||||
stmt = stmt.where(WorkflowNodeExecutionModel.created_at >= params.start)
|
||||
if params.end:
|
||||
stmt = stmt.where(WorkflowNodeExecutionModel.created_at < params.end)
|
||||
if params.keyword:
|
||||
escaped_keyword = escape_like_pattern(params.keyword)
|
||||
pattern = f"%{escaped_keyword}%"
|
||||
stmt = stmt.where(
|
||||
or_(
|
||||
WorkflowNodeExecutionModel.inputs.ilike(pattern, escape="\\"),
|
||||
WorkflowNodeExecutionModel.outputs.ilike(pattern, escape="\\"),
|
||||
WorkflowNodeExecutionModel.error.ilike(pattern, escape="\\"),
|
||||
WorkflowNodeExecutionModel.title.ilike(pattern, escape="\\"),
|
||||
workflow_app.name.ilike(pattern, escape="\\"),
|
||||
)
|
||||
)
|
||||
if params.statuses:
|
||||
stmt = cls._apply_workflow_node_status_filter(stmt, params.statuses)
|
||||
return stmt
|
||||
|
||||
@staticmethod
|
||||
def _apply_workflow_node_status_filter(stmt, statuses: tuple[str, ...]):
|
||||
conditions = []
|
||||
for status in statuses:
|
||||
normalized = status.strip().lower()
|
||||
if normalized in {"success", "normal"}:
|
||||
conditions.append(WorkflowNodeExecutionModel.status == WorkflowNodeExecutionStatus.SUCCEEDED)
|
||||
elif normalized in {"failed", "error"}:
|
||||
conditions.append(
|
||||
WorkflowNodeExecutionModel.status.in_(
|
||||
(
|
||||
WorkflowNodeExecutionStatus.FAILED,
|
||||
WorkflowNodeExecutionStatus.EXCEPTION,
|
||||
WorkflowNodeExecutionStatus.STOPPED,
|
||||
)
|
||||
)
|
||||
)
|
||||
elif normalized == "paused":
|
||||
conditions.append(
|
||||
WorkflowNodeExecutionModel.status.in_(
|
||||
(
|
||||
WorkflowNodeExecutionStatus.PAUSED,
|
||||
WorkflowNodeExecutionStatus.PENDING,
|
||||
WorkflowNodeExecutionStatus.RUNNING,
|
||||
WorkflowNodeExecutionStatus.RETRY,
|
||||
)
|
||||
)
|
||||
)
|
||||
else:
|
||||
raise ValueError(f"Unsupported status: {status}")
|
||||
if not conditions:
|
||||
return stmt
|
||||
return stmt.where(or_(*conditions))
|
||||
|
||||
@staticmethod
|
||||
def _apply_workflow_source_filter(stmt, source_filter: AgentSourceFilter):
|
||||
if source_filter.app_id:
|
||||
@@ -505,6 +563,148 @@ class AgentObservabilityService:
|
||||
"updated_at": to_timestamp(updated_at or conversation.updated_at),
|
||||
}
|
||||
|
||||
@classmethod
|
||||
def _serialize_workflow_execution_log(
|
||||
cls,
|
||||
*,
|
||||
node_execution_id: str,
|
||||
title: str,
|
||||
status: object,
|
||||
created_by_role: object,
|
||||
created_by: str,
|
||||
created_at: datetime,
|
||||
finished_at: datetime | None,
|
||||
source: dict[str, Any],
|
||||
) -> dict[str, Any]:
|
||||
created_by_role_value = cls._enum_value(created_by_role)
|
||||
return {
|
||||
"id": node_execution_id,
|
||||
"conversation_id": node_execution_id,
|
||||
"title": title,
|
||||
"end_user_id": created_by if created_by_role_value == CreatorUserRole.END_USER.value else None,
|
||||
"message_count": 1,
|
||||
"user_rate": None,
|
||||
"operation_rate": None,
|
||||
"unread": False,
|
||||
"source": source,
|
||||
"status": cls._workflow_node_status(status),
|
||||
"created_at": to_timestamp(created_at),
|
||||
"updated_at": to_timestamp(finished_at or created_at),
|
||||
}
|
||||
|
||||
@classmethod
|
||||
def serialize_workflow_node_message(cls, node_execution: WorkflowNodeExecutionModel) -> dict[str, Any]:
|
||||
inputs = cls._json_mapping(node_execution.inputs)
|
||||
outputs = cls._json_mapping(node_execution.outputs)
|
||||
metadata = cls._json_mapping(node_execution.execution_metadata)
|
||||
agent_log = cls._mapping_value(metadata, "agent_log")
|
||||
agent_backend = cls._mapping_value(agent_log, "agent_backend")
|
||||
usage = cls._mapping_value(agent_backend, "usage")
|
||||
|
||||
prompt_tokens = cls._int_value(usage.get("prompt_tokens"))
|
||||
completion_tokens = cls._int_value(usage.get("completion_tokens"))
|
||||
total_tokens = cls._int_value(usage.get("total_tokens") or metadata.get("total_tokens"))
|
||||
if not total_tokens:
|
||||
total_tokens = prompt_tokens + completion_tokens
|
||||
created_by_role = cls._enum_value(node_execution.created_by_role)
|
||||
|
||||
return {
|
||||
"id": node_execution.id,
|
||||
"message_id": node_execution.id,
|
||||
"conversation_id": node_execution.id,
|
||||
"query": cls._workflow_node_query(inputs, fallback=node_execution.title),
|
||||
"answer": cls._workflow_node_answer(outputs),
|
||||
"status": cls._workflow_node_status(node_execution.status),
|
||||
"error": node_execution.error,
|
||||
"from_end_user_id": (
|
||||
node_execution.created_by if created_by_role == CreatorUserRole.END_USER.value else None
|
||||
),
|
||||
"from_account_id": (
|
||||
node_execution.created_by if created_by_role == CreatorUserRole.ACCOUNT.value else None
|
||||
),
|
||||
"message_tokens": prompt_tokens,
|
||||
"answer_tokens": completion_tokens,
|
||||
"total_tokens": total_tokens,
|
||||
"total_price": str(usage.get("total_price") or metadata.get("total_price") or Decimal(0)),
|
||||
"currency": str(usage.get("currency") or metadata.get("currency") or ""),
|
||||
"latency": float(usage.get("latency") or node_execution.elapsed_time or 0),
|
||||
"created_at": to_timestamp(node_execution.created_at),
|
||||
"updated_at": to_timestamp(node_execution.finished_at or node_execution.created_at),
|
||||
}
|
||||
|
||||
@staticmethod
|
||||
def _json_mapping(value: object) -> Mapping[str, Any]:
|
||||
if isinstance(value, Mapping):
|
||||
return value
|
||||
if not isinstance(value, str) or not value:
|
||||
return {}
|
||||
try:
|
||||
parsed = json.loads(value)
|
||||
except (TypeError, ValueError):
|
||||
return {}
|
||||
return parsed if isinstance(parsed, Mapping) else {}
|
||||
|
||||
@staticmethod
|
||||
def _mapping_value(value: Mapping[str, Any], key: str) -> Mapping[str, Any]:
|
||||
nested = value.get(key)
|
||||
return nested if isinstance(nested, Mapping) else {}
|
||||
|
||||
@staticmethod
|
||||
def _enum_value(value: object) -> str:
|
||||
return str(value.value) if isinstance(value, Enum) else str(value)
|
||||
|
||||
@staticmethod
|
||||
def _int_value(value: object) -> int:
|
||||
if not isinstance(value, (str, int, float, Decimal)):
|
||||
return 0
|
||||
try:
|
||||
return int(value)
|
||||
except (TypeError, ValueError):
|
||||
return 0
|
||||
|
||||
@classmethod
|
||||
def _workflow_node_query(cls, inputs: Mapping[str, Any], *, fallback: str) -> str:
|
||||
request_data = cls._mapping_value(inputs, "agent_backend_request")
|
||||
composition = cls._mapping_value(request_data, "composition")
|
||||
layers = composition.get("layers")
|
||||
prompts: list[str] = []
|
||||
if isinstance(layers, list):
|
||||
for layer_name in ("workflow_node_job_prompt", "workflow_user_prompt"):
|
||||
for layer in layers:
|
||||
if not isinstance(layer, Mapping) or layer.get("name") != layer_name:
|
||||
continue
|
||||
config = cls._mapping_value(layer, "config")
|
||||
user_prompt = config.get("user")
|
||||
if isinstance(user_prompt, str) and user_prompt.strip():
|
||||
prompts.append(user_prompt.strip())
|
||||
return "\n\n".join(prompts) or fallback
|
||||
|
||||
@staticmethod
|
||||
def _workflow_node_answer(outputs: Mapping[str, Any]) -> str:
|
||||
for key in ("output", "text", "answer"):
|
||||
value = outputs.get(key)
|
||||
if isinstance(value, str):
|
||||
return value
|
||||
return json.dumps(outputs, ensure_ascii=False) if outputs else ""
|
||||
|
||||
@classmethod
|
||||
def _workflow_node_status(cls, status: object) -> str:
|
||||
value = cls._enum_value(status)
|
||||
if value in {
|
||||
WorkflowNodeExecutionStatus.FAILED.value,
|
||||
WorkflowNodeExecutionStatus.EXCEPTION.value,
|
||||
WorkflowNodeExecutionStatus.STOPPED.value,
|
||||
}:
|
||||
return "failed"
|
||||
if value in {
|
||||
WorkflowNodeExecutionStatus.PAUSED.value,
|
||||
WorkflowNodeExecutionStatus.PENDING.value,
|
||||
WorkflowNodeExecutionStatus.RUNNING.value,
|
||||
WorkflowNodeExecutionStatus.RETRY.value,
|
||||
}:
|
||||
return "paused"
|
||||
return "success"
|
||||
|
||||
@staticmethod
|
||||
def _conversation_status(*, paused_count: int, failed_count: int) -> str:
|
||||
if paused_count:
|
||||
|
||||
@@ -430,7 +430,11 @@ class AgentRosterService:
|
||||
agent.active_config_has_model = agent_soul_has_model(soul)
|
||||
agent.active_config_is_published = False
|
||||
self._session.flush()
|
||||
self._get_or_create_agent_app_debug_conversation(agent=agent, account_id=account_id)
|
||||
self._get_or_create_agent_app_debug_conversation(
|
||||
agent=agent,
|
||||
account_id=account_id,
|
||||
draft_type=AgentConfigDraftType.DEBUG_BUILD,
|
||||
)
|
||||
return agent
|
||||
|
||||
def create_hidden_backing_app_for_workflow_agent(
|
||||
@@ -527,7 +531,11 @@ class AgentRosterService:
|
||||
self._session.flush()
|
||||
return backing_app.id
|
||||
|
||||
def _get_or_create_agent_app_debug_conversation(self, *, agent: Agent, account_id: str) -> str:
|
||||
def _get_or_create_agent_app_debug_conversation(
|
||||
self, *, agent: Agent, account_id: str, draft_type: AgentConfigDraftType
|
||||
) -> str:
|
||||
"""Return the editor's conversation for one Agent draft surface."""
|
||||
|
||||
backing_app_id = self._ensure_workflow_agent_backing_app(agent=agent, account_id=account_id)
|
||||
if not backing_app_id:
|
||||
raise AgentNotFoundError()
|
||||
@@ -537,6 +545,7 @@ class AgentRosterService:
|
||||
AgentDebugConversation.tenant_id == agent.tenant_id,
|
||||
AgentDebugConversation.agent_id == agent.id,
|
||||
AgentDebugConversation.account_id == account_id,
|
||||
AgentDebugConversation.draft_type == draft_type,
|
||||
)
|
||||
)
|
||||
if mapping is not None:
|
||||
@@ -570,6 +579,7 @@ class AgentRosterService:
|
||||
agent_id=agent.id,
|
||||
app_id=backing_app_id,
|
||||
account_id=account_id,
|
||||
draft_type=draft_type,
|
||||
conversation_id=conversation_id,
|
||||
)
|
||||
)
|
||||
@@ -577,9 +587,15 @@ class AgentRosterService:
|
||||
return conversation_id
|
||||
|
||||
def get_or_create_agent_app_debug_conversation_id(
|
||||
self, *, tenant_id: str, agent_id: str, account_id: str, commit: bool = True
|
||||
self,
|
||||
*,
|
||||
tenant_id: str,
|
||||
agent_id: str,
|
||||
account_id: str,
|
||||
draft_type: AgentConfigDraftType = AgentConfigDraftType.DEBUG_BUILD,
|
||||
commit: bool = True,
|
||||
) -> str:
|
||||
"""Return the current editor's debug conversation for an Agent App."""
|
||||
"""Return the current editor's Build or Preview conversation for an Agent App."""
|
||||
|
||||
agent = self._session.scalar(
|
||||
select(Agent).where(
|
||||
@@ -591,13 +607,24 @@ class AgentRosterService:
|
||||
if agent is None:
|
||||
raise AgentNotFoundError()
|
||||
|
||||
conversation_id = self._get_or_create_agent_app_debug_conversation(agent=agent, account_id=account_id)
|
||||
conversation_id = self._get_or_create_agent_app_debug_conversation(
|
||||
agent=agent,
|
||||
account_id=account_id,
|
||||
draft_type=draft_type,
|
||||
)
|
||||
if commit:
|
||||
self._session.commit()
|
||||
return conversation_id
|
||||
|
||||
def load_agent_app_debug_conversation_id(self, *, tenant_id: str, agent_id: str, account_id: str) -> str | None:
|
||||
"""Return the current editor's existing debug conversation without creating or repairing rows."""
|
||||
def load_agent_app_debug_conversation_id(
|
||||
self,
|
||||
*,
|
||||
tenant_id: str,
|
||||
agent_id: str,
|
||||
account_id: str,
|
||||
draft_type: AgentConfigDraftType = AgentConfigDraftType.DEBUG_BUILD,
|
||||
) -> str | None:
|
||||
"""Return the editor's existing scoped conversation without creating or repairing rows."""
|
||||
|
||||
return self._session.scalar(
|
||||
select(Conversation.id)
|
||||
@@ -606,6 +633,7 @@ class AgentRosterService:
|
||||
AgentDebugConversation.tenant_id == tenant_id,
|
||||
AgentDebugConversation.agent_id == agent_id,
|
||||
AgentDebugConversation.account_id == account_id,
|
||||
AgentDebugConversation.draft_type == draft_type,
|
||||
AgentDebugConversation.app_id == Conversation.app_id,
|
||||
Conversation.from_source == ConversationFromSource.CONSOLE,
|
||||
Conversation.from_account_id == account_id,
|
||||
@@ -626,16 +654,24 @@ class AgentRosterService:
|
||||
)
|
||||
|
||||
def refresh_agent_app_debug_conversation_id(
|
||||
self, *, tenant_id: str, agent_id: str, account_id: str, commit: bool = True
|
||||
self,
|
||||
*,
|
||||
tenant_id: str,
|
||||
agent_id: str,
|
||||
account_id: str,
|
||||
draft_type: AgentConfigDraftType = AgentConfigDraftType.DEBUG_BUILD,
|
||||
) -> str:
|
||||
"""Start a new console debug conversation for the current Agent App editor.
|
||||
"""Start a new scoped console conversation for the current Agent App editor.
|
||||
|
||||
If this account already has a debug conversation mapping, the previous
|
||||
conversation is abandoned first: any ACTIVE conversation-owned Agent
|
||||
runtime sessions for that old conversation are sent through best-effort
|
||||
backend cleanup and then retired locally even when enqueueing fails.
|
||||
The debug mapping is then repointed to the freshly created
|
||||
conversation.
|
||||
If this account already has a mapping for the requested draft surface, the previous
|
||||
conversation is abandoned after the replacement mapping is committed: any ACTIVE
|
||||
conversation-owned Agent runtime sessions for that old conversation are sent through
|
||||
best-effort backend cleanup and then retired locally even when enqueueing fails. This
|
||||
order prevents a failed database commit from retiring the still-current runtime session.
|
||||
The other draft surface is left untouched.
|
||||
|
||||
A user and draft surface own one current mapping. If new-conversation requests overlap,
|
||||
the last committed rotation becomes current and earlier response IDs cannot be continued.
|
||||
"""
|
||||
|
||||
agent = self._session.scalar(
|
||||
@@ -658,11 +694,13 @@ class AgentRosterService:
|
||||
app_id=backing_app_id,
|
||||
account_id=account_id,
|
||||
)
|
||||
previous_conversation: tuple[str, str] | None = None
|
||||
mapping = self._session.scalar(
|
||||
select(AgentDebugConversation).where(
|
||||
AgentDebugConversation.tenant_id == tenant_id,
|
||||
AgentDebugConversation.agent_id == agent_id,
|
||||
AgentDebugConversation.account_id == account_id,
|
||||
AgentDebugConversation.draft_type == draft_type,
|
||||
)
|
||||
)
|
||||
if mapping is None:
|
||||
@@ -672,6 +710,7 @@ class AgentRosterService:
|
||||
agent_id=agent_id,
|
||||
app_id=backing_app_id,
|
||||
account_id=account_id,
|
||||
draft_type=draft_type,
|
||||
conversation_id=conversation_id,
|
||||
)
|
||||
)
|
||||
@@ -679,18 +718,22 @@ class AgentRosterService:
|
||||
previous_app_id = mapping.app_id
|
||||
previous_conversation_id = mapping.conversation_id
|
||||
if previous_conversation_id:
|
||||
self._cleanup_debug_conversation_runtime_sessions(
|
||||
tenant_id=tenant_id,
|
||||
agent_id=agent_id,
|
||||
account_id=account_id,
|
||||
app_id=previous_app_id or backing_app_id,
|
||||
conversation_id=previous_conversation_id,
|
||||
)
|
||||
previous_conversation = (previous_app_id or backing_app_id, previous_conversation_id)
|
||||
mapping.app_id = backing_app_id
|
||||
mapping.conversation_id = conversation_id
|
||||
self._session.flush()
|
||||
if commit:
|
||||
self._session.commit()
|
||||
self._session.commit()
|
||||
|
||||
if previous_conversation:
|
||||
previous_app_id, previous_conversation_id = previous_conversation
|
||||
self._cleanup_debug_conversation_runtime_sessions(
|
||||
tenant_id=tenant_id,
|
||||
agent_id=agent_id,
|
||||
account_id=account_id,
|
||||
draft_type=draft_type,
|
||||
app_id=previous_app_id,
|
||||
conversation_id=previous_conversation_id,
|
||||
)
|
||||
return conversation_id
|
||||
|
||||
def _cleanup_debug_conversation_runtime_sessions(
|
||||
@@ -699,11 +742,12 @@ class AgentRosterService:
|
||||
tenant_id: str,
|
||||
agent_id: str,
|
||||
account_id: str,
|
||||
draft_type: AgentConfigDraftType,
|
||||
app_id: str,
|
||||
conversation_id: str,
|
||||
) -> None:
|
||||
session_store = AgentAppRuntimeSessionStore()
|
||||
try:
|
||||
session_store = AgentAppRuntimeSessionStore()
|
||||
stored_sessions = session_store.list_active_sessions_for_conversation(
|
||||
tenant_id=tenant_id,
|
||||
app_id=app_id,
|
||||
@@ -727,7 +771,8 @@ class AgentRosterService:
|
||||
session_snapshot=stored_session.session_snapshot,
|
||||
runtime_layer_specs=stored_session.runtime_layer_specs,
|
||||
idempotency_key=(
|
||||
f"{tenant_id}:{agent_id}:{account_id}:{conversation_id}:debug-session-cleanup:"
|
||||
f"{tenant_id}:{agent_id}:{account_id}:{draft_type.value}:{conversation_id}:"
|
||||
"debug-session-cleanup:"
|
||||
f"{stored_session.scope.agent_id}:"
|
||||
f"{stored_session.scope.agent_config_snapshot_id or 'no-config'}:"
|
||||
f"{stored_session.backend_run_id or 'no-run'}"
|
||||
@@ -738,6 +783,7 @@ class AgentRosterService:
|
||||
"conversation_id": stored_session.scope.conversation_id,
|
||||
"agent_id": stored_session.scope.agent_id,
|
||||
"agent_config_snapshot_id": stored_session.scope.agent_config_snapshot_id,
|
||||
"draft_type": draft_type.value,
|
||||
"previous_agent_backend_run_id": stored_session.backend_run_id,
|
||||
},
|
||||
)
|
||||
@@ -772,9 +818,14 @@ class AgentRosterService:
|
||||
)
|
||||
|
||||
def load_or_create_agent_app_debug_conversation_ids_by_agent_id(
|
||||
self, *, tenant_id: str, agents: list[Agent], account_id: str
|
||||
self,
|
||||
*,
|
||||
tenant_id: str,
|
||||
agents: list[Agent],
|
||||
account_id: str,
|
||||
draft_type: AgentConfigDraftType = AgentConfigDraftType.DEBUG_BUILD,
|
||||
) -> dict[str, str]:
|
||||
"""Return per-account debug conversations for a page of Agent Apps."""
|
||||
"""Return per-account scoped conversations for a page of Agent Apps."""
|
||||
|
||||
conversation_ids_by_agent_id: dict[str, str] = {}
|
||||
changed = False
|
||||
@@ -784,6 +835,7 @@ class AgentRosterService:
|
||||
conversation_ids_by_agent_id[agent.id] = self._get_or_create_agent_app_debug_conversation(
|
||||
agent=agent,
|
||||
account_id=account_id,
|
||||
draft_type=draft_type,
|
||||
)
|
||||
changed = True
|
||||
if changed:
|
||||
@@ -866,16 +918,14 @@ class AgentRosterService:
|
||||
raise AgentNotFoundError()
|
||||
return app
|
||||
|
||||
def get_agent_runtime_app_model(self, *, tenant_id: str, agent_id: str) -> App:
|
||||
"""Resolve the App that backs an Agent runtime surface.
|
||||
def _get_runtime_resolvable_agent(self, *, tenant_id: str, agent_id: str) -> Agent | None:
|
||||
"""Load an Agent that is eligible to resolve to a runtime backing App.
|
||||
|
||||
Roster Agents use their public Agent App. Workflow-only Agents use a
|
||||
hidden Agent App stored in ``backing_app_id`` so console chat/logs can
|
||||
reuse the app runtime without exposing the resource in workspace app
|
||||
lists.
|
||||
Shared by the runtime resolver and the read-only authorization resolver
|
||||
so both agree on what counts as a resolvable Agent.
|
||||
"""
|
||||
|
||||
agent = self._session.scalar(
|
||||
return self._session.scalar(
|
||||
select(Agent)
|
||||
.where(
|
||||
Agent.tenant_id == tenant_id,
|
||||
@@ -893,6 +943,34 @@ class AgentRosterService:
|
||||
)
|
||||
.limit(1)
|
||||
)
|
||||
|
||||
def peek_authz_app_id(self, *, tenant_id: str, agent_id: str) -> str | None:
|
||||
"""Resolve the App id whose access policy governs an Agent.
|
||||
|
||||
Roster Agents are governed by their own Agent App, while workflow-only
|
||||
Agents are governed by their parent workflow App: the hidden runtime
|
||||
backing App never receives a resource access policy, so it must not be
|
||||
used for authorization. Stays read-only — unlike
|
||||
:meth:`get_agent_runtime_app_model`, this never materializes the hidden
|
||||
backing App. Returns ``None`` when the Agent does not resolve, leaving
|
||||
the caller to decide how to treat it.
|
||||
"""
|
||||
|
||||
agent = self._get_runtime_resolvable_agent(tenant_id=tenant_id, agent_id=agent_id)
|
||||
if agent is None:
|
||||
return None
|
||||
return agent.app_id
|
||||
|
||||
def get_agent_runtime_app_model(self, *, tenant_id: str, agent_id: str) -> App:
|
||||
"""Resolve the App that backs an Agent runtime surface.
|
||||
|
||||
Roster Agents use their public Agent App. Workflow-only Agents use a
|
||||
hidden Agent App stored in ``backing_app_id`` so console chat/logs can
|
||||
reuse the app runtime without exposing the resource in workspace app
|
||||
lists.
|
||||
"""
|
||||
|
||||
agent = self._get_runtime_resolvable_agent(tenant_id=tenant_id, agent_id=agent_id)
|
||||
if agent is None:
|
||||
raise AgentNotFoundError()
|
||||
should_commit_backing_app = agent.scope == AgentScope.WORKFLOW_ONLY and not agent.backing_app_id
|
||||
|
||||
@@ -92,6 +92,7 @@ class AgentToolInnerService:
|
||||
conversation_id=request.caller.conversation_id,
|
||||
)
|
||||
)
|
||||
transformed_messages, _ = ToolEngine.collect_agent_messages(transformed_messages)
|
||||
except ToolProviderNotFoundError as exc:
|
||||
raise AgentToolInnerServiceError(
|
||||
error_code="agent_tool_declaration_not_found",
|
||||
|
||||
@@ -19,6 +19,7 @@ from configs import dify_config
|
||||
from constants.dsl_version import CURRENT_APP_DSL_VERSION
|
||||
from core.file import remote_fetcher
|
||||
from core.plugin.entities.plugin import PluginDependency
|
||||
from core.rbac import RBACPermission
|
||||
from core.trigger.constants import (
|
||||
TRIGGER_PLUGIN_NODE_TYPE,
|
||||
TRIGGER_SCHEDULE_NODE_TYPE,
|
||||
@@ -43,7 +44,9 @@ from services.agent.dsl_service import AgentDslService, AgentPackage
|
||||
from services.agent.workflow_publish_service import WorkflowAgentPublishService
|
||||
from services.dsl_content import DSL_MAX_SIZE, dsl_content_size
|
||||
from services.dsl_version import check_version_compatibility
|
||||
from services.enterprise.rbac_service import RBACService
|
||||
from services.entities.dsl_entities import CheckDependenciesResult, DslImportWarning, ImportMode, ImportStatus
|
||||
from services.errors.account import NoPermissionError
|
||||
from services.errors.app import WorkflowNotFoundError
|
||||
from services.plugin.dependencies_analysis import DependenciesAnalysisService
|
||||
from services.workflow_draft_variable_service import WorkflowDraftVariableService
|
||||
@@ -301,6 +304,9 @@ class AppDslService:
|
||||
error=f"Invalid YAML format: {str(e)}",
|
||||
)
|
||||
|
||||
except NoPermissionError:
|
||||
raise
|
||||
|
||||
except Exception as e:
|
||||
logger.exception("Failed to import app")
|
||||
return Import(
|
||||
@@ -364,6 +370,9 @@ class AppDslService:
|
||||
warnings=self._warnings,
|
||||
)
|
||||
|
||||
except NoPermissionError:
|
||||
raise
|
||||
|
||||
except Exception as e:
|
||||
logger.exception("Error confirming import")
|
||||
return Import(
|
||||
@@ -395,6 +404,21 @@ class AppDslService:
|
||||
leaked_dependencies=leaked_dependencies,
|
||||
)
|
||||
|
||||
@staticmethod
|
||||
def _ensure_agent_manage_permission(account: Account) -> None:
|
||||
"""Importing an Agent DSL creates a roster Agent, which requires ``agent.manage``."""
|
||||
if not dify_config.RBAC_ENABLED:
|
||||
return
|
||||
if account.current_tenant_id is None:
|
||||
raise ValueError("Current tenant is not set")
|
||||
allowed = RBACService.CheckAccess.check(
|
||||
account.current_tenant_id,
|
||||
account.id,
|
||||
scene=RBACPermission.AGENT_MANAGE,
|
||||
)
|
||||
if not allowed:
|
||||
raise NoPermissionError("Agent management permission is required to import an Agent App")
|
||||
|
||||
def _create_or_update_app(
|
||||
self,
|
||||
*,
|
||||
@@ -415,6 +439,8 @@ class AppDslService:
|
||||
if not app_mode:
|
||||
raise ValueError("loss app mode")
|
||||
app_mode = AppMode(app_mode)
|
||||
if app_mode == AppMode.AGENT:
|
||||
self._ensure_agent_manage_permission(account)
|
||||
|
||||
# Set icon type
|
||||
icon_type_value = icon_type or app_data.get("icon_type")
|
||||
|
||||
@@ -217,12 +217,18 @@ class BillingService:
|
||||
return _billing_info_adapter.validate_python(billing_info)
|
||||
|
||||
@classmethod
|
||||
def get_vector_space(cls, tenant_id: str) -> _VectorSpaceQuota:
|
||||
def get_vector_space(cls, tenant_id: str, bypass_cache: bool = False) -> _VectorSpaceQuota:
|
||||
params = {"tenant_id": tenant_id}
|
||||
if bypass_cache:
|
||||
params["bypass_cache"] = "true"
|
||||
return _vector_space_quota_adapter.validate_python(
|
||||
cls._send_request("GET", "/subscription/vector-space", params=params)
|
||||
)
|
||||
|
||||
@classmethod
|
||||
def invalidate_vector_space_cache(cls, tenant_id: str) -> None:
|
||||
cls.get_vector_space(tenant_id, bypass_cache=True)
|
||||
|
||||
@classmethod
|
||||
def get_tenant_feature_plan_usage_info(cls, tenant_id: str):
|
||||
"""Deprecated: Use get_quota_info instead."""
|
||||
|
||||
@@ -1974,7 +1974,10 @@ class DocumentService:
|
||||
if data_source_info and "upload_file_id" in data_source_info:
|
||||
file_id = data_source_info["upload_file_id"]
|
||||
document_was_deleted.send(
|
||||
document.id, dataset_id=document.dataset_id, doc_form=document.doc_form, file_id=file_id
|
||||
document.id,
|
||||
dataset_id=document.dataset_id,
|
||||
doc_form=document.doc_form,
|
||||
file_id=file_id,
|
||||
)
|
||||
|
||||
session.delete(document)
|
||||
@@ -2013,7 +2016,12 @@ class DocumentService:
|
||||
# Dispatch cleanup task after commit to avoid lock contention
|
||||
# Task cleans up segments, files, and vector indexes
|
||||
if deleted_document_ids and doc_form is not None:
|
||||
batch_clean_document_task.delay(deleted_document_ids, dataset_ref.dataset_id, doc_form, file_ids)
|
||||
batch_clean_document_task.delay(
|
||||
deleted_document_ids,
|
||||
dataset_ref.dataset_id,
|
||||
doc_form,
|
||||
file_ids,
|
||||
)
|
||||
|
||||
@staticmethod
|
||||
def rename_document(dataset_id: str, document_id: str, name: str, session: Session) -> Document:
|
||||
|
||||
@@ -5,6 +5,7 @@ from typing import Any
|
||||
|
||||
import httpx
|
||||
|
||||
from configs import dify_config
|
||||
from core.helper.trace_id_helper import generate_traceparent_header
|
||||
from services.errors.enterprise import (
|
||||
EnterpriseAPIBadRequestError,
|
||||
@@ -96,12 +97,14 @@ class BaseRequest:
|
||||
logger.debug("Failed to generate traceparent header", exc_info=True)
|
||||
|
||||
with httpx.Client(mounts=mounts) as client:
|
||||
# IMPORTANT:
|
||||
# - In httpx, passing timeout=None disables timeouts (infinite) and overrides the library default.
|
||||
# - To preserve httpx's default timeout behavior for existing call sites, only pass the kwarg when set.
|
||||
request_kwargs: dict[str, Any] = {"json": json, "params": params, "headers": headers}
|
||||
if timeout is not None:
|
||||
request_kwargs["timeout"] = timeout
|
||||
# Callers that pass an explicit timeout keep it; everyone else gets the
|
||||
# configured budget rather than httpx's implicit 5s default.
|
||||
request_kwargs: dict[str, Any] = {
|
||||
"json": json,
|
||||
"params": params,
|
||||
"headers": headers,
|
||||
"timeout": timeout if timeout is not None else dify_config.ENTERPRISE_REQUEST_TIMEOUT,
|
||||
}
|
||||
|
||||
response = client.request(method, url, **request_kwargs)
|
||||
|
||||
@@ -206,9 +209,8 @@ class EnterpriseRequest(BaseRequest):
|
||||
"json": json,
|
||||
"params": params,
|
||||
"headers": {"Content-Type": "application/json", cls.secret_key_header: cls.secret_key, **inner_headers},
|
||||
"timeout": timeout if timeout is not None else dify_config.ENTERPRISE_RBAC_REQUEST_TIMEOUT,
|
||||
}
|
||||
if timeout is not None:
|
||||
request_kwargs["timeout"] = timeout
|
||||
response = client.request(method, url, **request_kwargs)
|
||||
if not response.is_success:
|
||||
cls._handle_error_response(response)
|
||||
|
||||
@@ -330,6 +330,7 @@ _LEGACY_WORKSPACE_OWNER_KEYS: list[str] = [
|
||||
"snippets.management",
|
||||
"tool.manage",
|
||||
"mcp.manage",
|
||||
"agent.manage",
|
||||
]
|
||||
|
||||
_LEGACY_WORKSPACE_ADMIN_KEYS: list[str] = [
|
||||
@@ -357,6 +358,7 @@ _LEGACY_WORKSPACE_ADMIN_KEYS: list[str] = [
|
||||
"snippets.management",
|
||||
"tool.manage",
|
||||
"mcp.manage",
|
||||
"agent.manage",
|
||||
]
|
||||
|
||||
_LEGACY_WORKSPACE_EDITOR_KEYS: list[str] = [
|
||||
@@ -371,6 +373,7 @@ _LEGACY_WORKSPACE_EDITOR_KEYS: list[str] = [
|
||||
"dataset.external.connect",
|
||||
"snippets.create_and_modify",
|
||||
"tool.manage",
|
||||
"agent.manage",
|
||||
]
|
||||
|
||||
_LEGACY_WORKSPACE_NORMAL_KEYS: list[str] = [
|
||||
|
||||
@@ -93,8 +93,20 @@ class ExternalDatasetService:
|
||||
raise ValueError(f"invalid endpoint: {endpoint} must start with http:// or https://")
|
||||
else:
|
||||
raise ValueError(f"invalid endpoint: {endpoint}")
|
||||
# Send a minimal body shaped like the External Knowledge API retrieval contract so providers
|
||||
# that require a JSON payload (e.g. RAGFlow) accept the validation probe instead of rejecting
|
||||
# a body-less POST. Mirrors the request built in fetch_external_knowledge_retrieval.
|
||||
validation_payload = {
|
||||
"knowledge_id": "",
|
||||
"query": "",
|
||||
"retrieval_setting": {"top_k": 1, "score_threshold": 0.0},
|
||||
}
|
||||
try:
|
||||
response = ssrf_proxy.post(endpoint, headers={"Authorization": f"Bearer {api_key}"})
|
||||
response = ssrf_proxy.post(
|
||||
endpoint,
|
||||
headers={"Authorization": f"Bearer {api_key}", "Content-Type": "application/json"},
|
||||
data=json.dumps(validation_payload),
|
||||
)
|
||||
except Exception as e:
|
||||
raise ValueError(f"failed to connect to the endpoint: {endpoint}") from e
|
||||
if response.status_code == 502:
|
||||
|
||||
@@ -186,6 +186,7 @@ class SystemFeatureModel(FeatureResponseModel):
|
||||
enable_learn_app: bool = True
|
||||
enable_step_by_step_tour: bool = False
|
||||
rbac_enabled: bool = False
|
||||
knowledge_fs_enabled: bool = False
|
||||
|
||||
|
||||
class FeatureService:
|
||||
@@ -289,6 +290,7 @@ class FeatureService:
|
||||
system_features.enable_learn_app = dify_config.ENABLE_LEARN_APP
|
||||
system_features.webapp_auth.allow_public_access = dify_config.WEBAPP_PUBLIC_ACCESS_ENABLED
|
||||
system_features.enable_step_by_step_tour = dify_config.ENABLE_STEP_BY_STEP_TOUR
|
||||
system_features.knowledge_fs_enabled = dify_config.KNOWLEDGE_FS_ENABLED
|
||||
|
||||
@classmethod
|
||||
def _fulfill_trial_models_from_env(cls) -> list[str]:
|
||||
|
||||
@@ -141,6 +141,29 @@ class FileService:
|
||||
blob = storage.load_once(upload_file_key)
|
||||
return base64.b64encode(blob).decode()
|
||||
|
||||
def get_file_presigned_url(self, *, file_id: str, tenant_id: str) -> str:
|
||||
"""Generate a direct storage URL for a tenant-owned upload file."""
|
||||
with self._session_maker(expire_on_commit=False) as session:
|
||||
upload_file = session.scalar(
|
||||
select(UploadFile)
|
||||
.where(
|
||||
UploadFile.id == file_id,
|
||||
UploadFile.tenant_id == tenant_id,
|
||||
)
|
||||
.limit(1)
|
||||
)
|
||||
if upload_file is None:
|
||||
raise NotFound("File not found")
|
||||
|
||||
file_key = upload_file.key
|
||||
content_type = upload_file.mime_type
|
||||
|
||||
return storage.generate_presigned_url(
|
||||
file_key,
|
||||
expires_in=dify_config.FILES_ACCESS_TIMEOUT,
|
||||
content_type=content_type,
|
||||
)
|
||||
|
||||
def upload_text(self, text: str, text_name: str, user_id: str, tenant_id: str) -> UploadFile:
|
||||
if len(text_name) > 200:
|
||||
text_name = text_name[:200]
|
||||
|
||||
@@ -0,0 +1,502 @@
|
||||
"""Product-facing KnowledgeFS operation and authorization declarations.
|
||||
|
||||
This registry is Dify's explicit Console surface. Transport concerns live in
|
||||
knowledge_fs_proxy so contract review does not require reading proxy mechanics.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import Final, Literal, NamedTuple
|
||||
|
||||
from core.rbac import RBACPermission
|
||||
|
||||
type KnowledgeFSMethod = Literal["DELETE", "GET", "PATCH", "POST", "PUT"]
|
||||
type KnowledgeFSResponseKind = Literal["binary", "buffered", "stream"]
|
||||
type KnowledgeFSRequiredScope = Literal["knowledge-spaces:read", "knowledge-spaces:write"]
|
||||
type KnowledgeFSLegacyRole = Literal["reader", "dataset_editor", "admin"]
|
||||
type KnowledgeFSErrorStatusMap = tuple[tuple[int, int], ...]
|
||||
|
||||
|
||||
class KnowledgeFSOperation(NamedTuple):
|
||||
operation_id: str
|
||||
method: KnowledgeFSMethod
|
||||
path: str
|
||||
response_kind: KnowledgeFSResponseKind
|
||||
required_scope: KnowledgeFSRequiredScope
|
||||
rbac_permission: RBACPermission
|
||||
legacy_role: KnowledgeFSLegacyRole
|
||||
max_response_bytes: int
|
||||
request_headers: tuple[str, ...]
|
||||
response_headers: tuple[str, ...]
|
||||
response_media_types: tuple[str, ...]
|
||||
error_status_map: KnowledgeFSErrorStatusMap
|
||||
|
||||
|
||||
def _console_operation(
|
||||
operation_id: str,
|
||||
method: KnowledgeFSMethod,
|
||||
path: str,
|
||||
*,
|
||||
rbac_permission: RBACPermission,
|
||||
legacy_role: KnowledgeFSLegacyRole,
|
||||
max_response_bytes: int = 1_048_576,
|
||||
request_headers: tuple[str, ...] = ("x-trace-id",),
|
||||
response_kind: KnowledgeFSResponseKind = "buffered",
|
||||
response_media_types: tuple[str, ...] = ("application/json",),
|
||||
error_status_map: KnowledgeFSErrorStatusMap = ((401, 502), (403, 403)),
|
||||
) -> KnowledgeFSOperation:
|
||||
"""Declare one contract-pinned operation with an explicit Dify authorization policy."""
|
||||
is_read = method == "GET"
|
||||
return KnowledgeFSOperation(
|
||||
operation_id=operation_id,
|
||||
method=method,
|
||||
path=path,
|
||||
response_kind=response_kind,
|
||||
required_scope="knowledge-spaces:read" if is_read else "knowledge-spaces:write",
|
||||
rbac_permission=rbac_permission,
|
||||
legacy_role=legacy_role,
|
||||
max_response_bytes=max_response_bytes,
|
||||
request_headers=request_headers,
|
||||
response_headers=("x-trace-id",),
|
||||
response_media_types=response_media_types,
|
||||
error_status_map=error_status_map,
|
||||
)
|
||||
|
||||
|
||||
def _dataset_read_operation(operation_id: str, path: str) -> KnowledgeFSOperation:
|
||||
"""Declare a dataset-readable buffered JSON operation."""
|
||||
return _console_operation(
|
||||
operation_id,
|
||||
"GET",
|
||||
path,
|
||||
rbac_permission=RBACPermission.DATASET_READONLY,
|
||||
legacy_role="reader",
|
||||
)
|
||||
|
||||
|
||||
def _dataset_edit_operation(
|
||||
operation_id: str,
|
||||
method: KnowledgeFSMethod,
|
||||
path: str,
|
||||
*,
|
||||
request_headers: tuple[str, ...] = ("x-trace-id",),
|
||||
) -> KnowledgeFSOperation:
|
||||
"""Declare a dataset-editable buffered JSON operation."""
|
||||
return _console_operation(
|
||||
operation_id,
|
||||
method,
|
||||
path,
|
||||
rbac_permission=RBACPermission.DATASET_EDIT,
|
||||
legacy_role="dataset_editor",
|
||||
request_headers=request_headers,
|
||||
)
|
||||
|
||||
|
||||
def _external_source_operation(
|
||||
operation_id: str,
|
||||
method: KnowledgeFSMethod,
|
||||
path: str,
|
||||
*,
|
||||
request_headers: tuple[str, ...] = ("x-trace-id",),
|
||||
) -> KnowledgeFSOperation:
|
||||
"""Declare a source-connection operation restricted to dataset editors."""
|
||||
return _console_operation(
|
||||
operation_id,
|
||||
method,
|
||||
path,
|
||||
rbac_permission=RBACPermission.DATASET_EXTERNAL_CONNECT,
|
||||
legacy_role="dataset_editor",
|
||||
request_headers=request_headers,
|
||||
)
|
||||
|
||||
|
||||
KNOWLEDGE_FS_CONSOLE_OPERATIONS: Final[tuple[KnowledgeFSOperation, ...]] = (
|
||||
_console_operation(
|
||||
operation_id="listKnowledgeSpaces",
|
||||
method="GET",
|
||||
path="knowledge-spaces",
|
||||
rbac_permission=RBACPermission.DATASET_READONLY,
|
||||
legacy_role="reader",
|
||||
),
|
||||
_console_operation(
|
||||
operation_id="createKnowledgeSpace",
|
||||
method="POST",
|
||||
path="knowledge-spaces",
|
||||
rbac_permission=RBACPermission.DATASET_CREATE_AND_MANAGEMENT,
|
||||
legacy_role="dataset_editor",
|
||||
),
|
||||
_console_operation(
|
||||
operation_id="getKnowledgeSpacesById",
|
||||
method="GET",
|
||||
path="knowledge-spaces/{id}",
|
||||
rbac_permission=RBACPermission.DATASET_READONLY,
|
||||
legacy_role="reader",
|
||||
),
|
||||
_dataset_edit_operation("patchKnowledgeSpacesById", "PATCH", "knowledge-spaces/{id}"),
|
||||
_dataset_edit_operation(
|
||||
"deleteKnowledgeSpacesById",
|
||||
"DELETE",
|
||||
"knowledge-spaces/{id}",
|
||||
request_headers=("idempotency-key", "x-trace-id"),
|
||||
),
|
||||
_dataset_read_operation("getKnowledgeSpacesByIdStats", "knowledge-spaces/{id}/stats"),
|
||||
_console_operation(
|
||||
operation_id="getKnowledgeSpacesByIdAccessPolicy",
|
||||
method="GET",
|
||||
path="knowledge-spaces/{id}/access-policy",
|
||||
rbac_permission=RBACPermission.DATASET_READONLY,
|
||||
legacy_role="reader",
|
||||
),
|
||||
_console_operation(
|
||||
operation_id="patchKnowledgeSpacesByIdAccessPolicy",
|
||||
method="PATCH",
|
||||
path="knowledge-spaces/{id}/access-policy",
|
||||
rbac_permission=RBACPermission.DATASET_ACCESS_CONFIG,
|
||||
legacy_role="admin",
|
||||
),
|
||||
_console_operation(
|
||||
operation_id="getSourceProviders",
|
||||
method="GET",
|
||||
path="source-providers",
|
||||
rbac_permission=RBACPermission.DATASET_EXTERNAL_CONNECT,
|
||||
legacy_role="dataset_editor",
|
||||
),
|
||||
_console_operation(
|
||||
operation_id="getKnowledgeSpacesByIdSourceConnections",
|
||||
method="GET",
|
||||
path="knowledge-spaces/{id}/source-connections",
|
||||
rbac_permission=RBACPermission.DATASET_EXTERNAL_CONNECT,
|
||||
legacy_role="dataset_editor",
|
||||
),
|
||||
_console_operation(
|
||||
operation_id="postKnowledgeSpacesByIdSourceConnections",
|
||||
method="POST",
|
||||
path="knowledge-spaces/{id}/source-connections",
|
||||
rbac_permission=RBACPermission.DATASET_EXTERNAL_CONNECT,
|
||||
legacy_role="dataset_editor",
|
||||
),
|
||||
_external_source_operation(
|
||||
"postKnowledgeSpacesByIdSourceConnectionsOauth",
|
||||
"POST",
|
||||
"knowledge-spaces/{id}/source-connections/oauth",
|
||||
),
|
||||
_external_source_operation("postSourceOauthCallback", "POST", "source-oauth/callback"),
|
||||
_external_source_operation(
|
||||
"getKnowledgeSpacesByIdSourceConnectionsByConnectionId",
|
||||
"GET",
|
||||
"knowledge-spaces/{id}/source-connections/{connectionId}",
|
||||
),
|
||||
_external_source_operation(
|
||||
"deleteKnowledgeSpacesByIdSourceConnectionsByConnectionId",
|
||||
"DELETE",
|
||||
"knowledge-spaces/{id}/source-connections/{connectionId}",
|
||||
),
|
||||
_console_operation(
|
||||
operation_id="postKnowledgeSpacesByIdSourceConnectionsByConnectionIdRefresh",
|
||||
method="POST",
|
||||
path="knowledge-spaces/{id}/source-connections/{connectionId}/refresh",
|
||||
rbac_permission=RBACPermission.DATASET_EXTERNAL_CONNECT,
|
||||
legacy_role="dataset_editor",
|
||||
),
|
||||
_console_operation(
|
||||
operation_id="getKnowledgeSpacesByIdSources",
|
||||
method="GET",
|
||||
path="knowledge-spaces/{id}/sources",
|
||||
rbac_permission=RBACPermission.DATASET_READONLY,
|
||||
legacy_role="reader",
|
||||
),
|
||||
_console_operation(
|
||||
operation_id="postKnowledgeSpacesByIdSources",
|
||||
method="POST",
|
||||
path="knowledge-spaces/{id}/sources",
|
||||
rbac_permission=RBACPermission.DATASET_EXTERNAL_CONNECT,
|
||||
legacy_role="dataset_editor",
|
||||
),
|
||||
_external_source_operation(
|
||||
"getKnowledgeSpacesByIdSourcesBySourceId",
|
||||
"GET",
|
||||
"knowledge-spaces/{id}/sources/{sourceId}",
|
||||
),
|
||||
_external_source_operation(
|
||||
"patchKnowledgeSpacesByIdSourcesBySourceId",
|
||||
"PATCH",
|
||||
"knowledge-spaces/{id}/sources/{sourceId}",
|
||||
),
|
||||
_external_source_operation(
|
||||
"deleteKnowledgeSpacesByIdSourcesBySourceId",
|
||||
"DELETE",
|
||||
"knowledge-spaces/{id}/sources/{sourceId}",
|
||||
request_headers=("idempotency-key", "x-trace-id"),
|
||||
),
|
||||
_external_source_operation(
|
||||
"putKnowledgeSpacesByIdSourcesBySourceIdCredentials",
|
||||
"PUT",
|
||||
"knowledge-spaces/{id}/sources/{sourceId}/credentials",
|
||||
),
|
||||
_external_source_operation(
|
||||
"deleteKnowledgeSpacesByIdSourcesBySourceIdCredentials",
|
||||
"DELETE",
|
||||
"knowledge-spaces/{id}/sources/{sourceId}/credentials",
|
||||
),
|
||||
_external_source_operation(
|
||||
"postKnowledgeSpacesByIdSourcesBySourceIdSync",
|
||||
"POST",
|
||||
"knowledge-spaces/{id}/sources/{sourceId}/sync",
|
||||
request_headers=("idempotency-key", "x-trace-id"),
|
||||
),
|
||||
_console_operation(
|
||||
operation_id="postKnowledgeSpacesByIdSourcesBySourceIdCrawlPreview",
|
||||
method="POST",
|
||||
path="knowledge-spaces/{id}/sources/{sourceId}/crawl-preview",
|
||||
rbac_permission=RBACPermission.DATASET_EXTERNAL_CONNECT,
|
||||
legacy_role="dataset_editor",
|
||||
request_headers=("idempotency-key", "x-trace-id"),
|
||||
),
|
||||
_external_source_operation(
|
||||
"postKnowledgeSpacesByIdSourcesBySourceIdWorkflowImports",
|
||||
"POST",
|
||||
"knowledge-spaces/{id}/sources/{sourceId}/workflow-imports",
|
||||
request_headers=("idempotency-key", "x-trace-id"),
|
||||
),
|
||||
_external_source_operation(
|
||||
"getKnowledgeSpacesByIdSourcesBySourceIdPages",
|
||||
"GET",
|
||||
"knowledge-spaces/{id}/sources/{sourceId}/pages",
|
||||
),
|
||||
_external_source_operation(
|
||||
"getKnowledgeSpacesByIdSourcesBySourceIdFiles",
|
||||
"GET",
|
||||
"knowledge-spaces/{id}/sources/{sourceId}/files",
|
||||
),
|
||||
_external_source_operation(
|
||||
"postKnowledgeSpacesByIdSourcesBySourceIdCrawl",
|
||||
"POST",
|
||||
"knowledge-spaces/{id}/sources/{sourceId}/crawl",
|
||||
),
|
||||
_external_source_operation(
|
||||
"postKnowledgeSpacesByIdSourcesBySourceIdImport",
|
||||
"POST",
|
||||
"knowledge-spaces/{id}/sources/{sourceId}/import",
|
||||
),
|
||||
_external_source_operation(
|
||||
"postKnowledgeSpacesByIdSourcesBySourceIdTest",
|
||||
"POST",
|
||||
"knowledge-spaces/{id}/sources/{sourceId}/test",
|
||||
),
|
||||
_external_source_operation(
|
||||
"postKnowledgeSpacesByIdSourcesBySourceIdImportFiles",
|
||||
"POST",
|
||||
"knowledge-spaces/{id}/sources/{sourceId}/import-files",
|
||||
),
|
||||
_external_source_operation(
|
||||
"postKnowledgeSpacesByIdSourcesBulk",
|
||||
"POST",
|
||||
"knowledge-spaces/{id}/sources/bulk",
|
||||
request_headers=("idempotency-key", "x-trace-id"),
|
||||
),
|
||||
_external_source_operation(
|
||||
"getKnowledgeSpacesByIdSourceWorkflows",
|
||||
"GET",
|
||||
"knowledge-spaces/{id}/source-workflows",
|
||||
),
|
||||
_console_operation(
|
||||
operation_id="getKnowledgeSpacesByIdSourceWorkflowsByRunId",
|
||||
method="GET",
|
||||
path="knowledge-spaces/{id}/source-workflows/{runId}",
|
||||
rbac_permission=RBACPermission.DATASET_EXTERNAL_CONNECT,
|
||||
legacy_role="dataset_editor",
|
||||
),
|
||||
_external_source_operation(
|
||||
"getKnowledgeSpacesByIdSourceWorkflowsByRunIdBulkItems",
|
||||
"GET",
|
||||
"knowledge-spaces/{id}/source-workflows/{runId}/bulk-items",
|
||||
),
|
||||
_console_operation(
|
||||
operation_id="getKnowledgeSpacesByIdSourceWorkflowsByRunIdPages",
|
||||
method="GET",
|
||||
path="knowledge-spaces/{id}/source-workflows/{runId}/pages",
|
||||
rbac_permission=RBACPermission.DATASET_EXTERNAL_CONNECT,
|
||||
legacy_role="dataset_editor",
|
||||
),
|
||||
_console_operation(
|
||||
operation_id="postKnowledgeSpacesByIdSourceWorkflowsByRunIdCancel",
|
||||
method="POST",
|
||||
path="knowledge-spaces/{id}/source-workflows/{runId}/cancel",
|
||||
rbac_permission=RBACPermission.DATASET_EXTERNAL_CONNECT,
|
||||
legacy_role="dataset_editor",
|
||||
),
|
||||
_console_operation(
|
||||
operation_id="postKnowledgeSpacesByIdSourceWorkflowsByRunIdRetry",
|
||||
method="POST",
|
||||
path="knowledge-spaces/{id}/source-workflows/{runId}/retry",
|
||||
rbac_permission=RBACPermission.DATASET_EXTERNAL_CONNECT,
|
||||
legacy_role="dataset_editor",
|
||||
),
|
||||
_console_operation(
|
||||
operation_id="postKnowledgeSpacesByIdSourceWorkflowsByRunIdSelection",
|
||||
method="POST",
|
||||
path="knowledge-spaces/{id}/source-workflows/{runId}/selection",
|
||||
rbac_permission=RBACPermission.DATASET_EXTERNAL_CONNECT,
|
||||
legacy_role="dataset_editor",
|
||||
request_headers=("idempotency-key", "x-trace-id"),
|
||||
),
|
||||
_console_operation(
|
||||
operation_id="getKnowledgeSpacesByIdSourcesBySourceIdSyncPolicy",
|
||||
method="GET",
|
||||
path="knowledge-spaces/{id}/sources/{sourceId}/sync-policy",
|
||||
rbac_permission=RBACPermission.DATASET_READONLY,
|
||||
legacy_role="reader",
|
||||
),
|
||||
_console_operation(
|
||||
operation_id="putKnowledgeSpacesByIdSourcesBySourceIdSyncPolicy",
|
||||
method="PUT",
|
||||
path="knowledge-spaces/{id}/sources/{sourceId}/sync-policy",
|
||||
rbac_permission=RBACPermission.DATASET_EDIT,
|
||||
legacy_role="dataset_editor",
|
||||
),
|
||||
_dataset_read_operation("getKnowledgeSpacesByIdDocuments", "knowledge-spaces/{id}/documents"),
|
||||
_dataset_edit_operation("postKnowledgeSpacesByIdDocuments", "POST", "knowledge-spaces/{id}/documents"),
|
||||
_dataset_edit_operation(
|
||||
"deleteKnowledgeSpacesByIdDocumentsBulk",
|
||||
"DELETE",
|
||||
"knowledge-spaces/{id}/documents/bulk",
|
||||
request_headers=("idempotency-key", "x-trace-id"),
|
||||
),
|
||||
_dataset_edit_operation(
|
||||
"postKnowledgeSpacesByIdDocumentsBulk",
|
||||
"POST",
|
||||
"knowledge-spaces/{id}/documents/bulk",
|
||||
),
|
||||
_dataset_edit_operation(
|
||||
"postKnowledgeSpacesByIdDocumentsBulkReindex",
|
||||
"POST",
|
||||
"knowledge-spaces/{id}/documents/bulk/reindex",
|
||||
),
|
||||
_dataset_read_operation(
|
||||
"getKnowledgeSpacesByIdDocumentsByDocumentId",
|
||||
"knowledge-spaces/{id}/documents/{documentId}",
|
||||
),
|
||||
_dataset_edit_operation(
|
||||
"deleteKnowledgeSpacesByIdDocumentsByDocumentId",
|
||||
"DELETE",
|
||||
"knowledge-spaces/{id}/documents/{documentId}",
|
||||
request_headers=("idempotency-key", "x-trace-id"),
|
||||
),
|
||||
_console_operation(
|
||||
operation_id="getKnowledgeSpacesByIdLogicalDocuments",
|
||||
method="GET",
|
||||
path="knowledge-spaces/{id}/logical-documents",
|
||||
rbac_permission=RBACPermission.DATASET_READONLY,
|
||||
legacy_role="reader",
|
||||
),
|
||||
_dataset_edit_operation(
|
||||
"deleteKnowledgeSpacesByIdLogicalDocumentsByDocumentId",
|
||||
"DELETE",
|
||||
"knowledge-spaces/{id}/logical-documents/{documentId}",
|
||||
request_headers=("idempotency-key", "x-trace-id"),
|
||||
),
|
||||
_dataset_read_operation(
|
||||
"getKnowledgeSpacesByIdDocumentsByDocumentIdOutline",
|
||||
"knowledge-spaces/{id}/documents/{documentId}/outline",
|
||||
),
|
||||
_console_operation(
|
||||
operation_id="getKnowledgeSpacesByIdLogicalDocumentsByDocumentId",
|
||||
method="GET",
|
||||
path="knowledge-spaces/{id}/logical-documents/{documentId}",
|
||||
rbac_permission=RBACPermission.DATASET_READONLY,
|
||||
legacy_role="reader",
|
||||
),
|
||||
_console_operation(
|
||||
operation_id="getKnowledgeSpacesByIdDocumentsByDocumentIdRevisions",
|
||||
method="GET",
|
||||
path="knowledge-spaces/{id}/documents/{documentId}/revisions",
|
||||
rbac_permission=RBACPermission.DATASET_READONLY,
|
||||
legacy_role="reader",
|
||||
),
|
||||
_dataset_edit_operation(
|
||||
"postKnowledgeSpacesByIdDocumentsByDocumentIdRevisionsByRevisionRollback",
|
||||
"POST",
|
||||
"knowledge-spaces/{id}/documents/{documentId}/revisions/{revision}/rollback",
|
||||
),
|
||||
_dataset_edit_operation(
|
||||
"patchKnowledgeSpacesByIdDocumentsByDocumentIdMetadata",
|
||||
"PATCH",
|
||||
"knowledge-spaces/{id}/documents/{documentId}/metadata",
|
||||
),
|
||||
_console_operation(
|
||||
operation_id="getKnowledgeSpacesByIdDocumentsByDocumentIdRevisionsByRevisionChunks",
|
||||
method="GET",
|
||||
path="knowledge-spaces/{id}/documents/{documentId}/revisions/{revision}/chunks",
|
||||
rbac_permission=RBACPermission.DATASET_READONLY,
|
||||
legacy_role="reader",
|
||||
),
|
||||
_dataset_read_operation(
|
||||
"getKnowledgeSpacesByIdDocumentsByDocumentIdRevisionsByRevisionChunksByChunkId",
|
||||
"knowledge-spaces/{id}/documents/{documentId}/revisions/{revision}/chunks/{chunkId}",
|
||||
),
|
||||
_dataset_edit_operation(
|
||||
"postKnowledgeSpacesByIdDocumentsByDocumentIdRevisionsByRevisionChunksByChunkIdState",
|
||||
"POST",
|
||||
"knowledge-spaces/{id}/documents/{documentId}/revisions/{revision}/chunks/{chunkId}/state",
|
||||
),
|
||||
_console_operation(
|
||||
operation_id="getKnowledgeSpacesByIdProcessingTasks",
|
||||
method="GET",
|
||||
path="knowledge-spaces/{id}/processing-tasks",
|
||||
rbac_permission=RBACPermission.DATASET_READONLY,
|
||||
legacy_role="reader",
|
||||
),
|
||||
_dataset_read_operation(
|
||||
"getKnowledgeSpacesByIdDocumentsByDocumentIdProcessingTasks",
|
||||
"knowledge-spaces/{id}/documents/{documentId}/processing-tasks",
|
||||
),
|
||||
_dataset_read_operation(
|
||||
"getKnowledgeSpacesByIdDocumentsByDocumentIdProcessingTasksByTaskId",
|
||||
"knowledge-spaces/{id}/documents/{documentId}/processing-tasks/{taskId}",
|
||||
),
|
||||
_console_operation(
|
||||
operation_id="getKnowledgeSpacesByIdDocumentsByDocumentIdProcessingTasksByTaskIdEvents",
|
||||
method="GET",
|
||||
path="knowledge-spaces/{id}/documents/{documentId}/processing-tasks/{taskId}/events",
|
||||
rbac_permission=RBACPermission.DATASET_READONLY,
|
||||
legacy_role="reader",
|
||||
max_response_bytes=67_108_864,
|
||||
request_headers=("last-event-id", "x-trace-id"),
|
||||
response_kind="stream",
|
||||
response_media_types=("text/event-stream",),
|
||||
),
|
||||
_console_operation(
|
||||
operation_id="deleteKnowledgeSpacesByIdDocumentsByDocumentIdProcessingTasksByTaskId",
|
||||
method="DELETE",
|
||||
path="knowledge-spaces/{id}/documents/{documentId}/processing-tasks/{taskId}",
|
||||
rbac_permission=RBACPermission.DATASET_EDIT,
|
||||
legacy_role="dataset_editor",
|
||||
),
|
||||
_console_operation(
|
||||
operation_id="postKnowledgeSpacesByIdDocumentsByDocumentIdProcessingTasksByTaskIdRetry",
|
||||
method="POST",
|
||||
path="knowledge-spaces/{id}/documents/{documentId}/processing-tasks/{taskId}/retry",
|
||||
rbac_permission=RBACPermission.DATASET_EDIT,
|
||||
legacy_role="dataset_editor",
|
||||
),
|
||||
_dataset_read_operation(
|
||||
"getKnowledgeSpacesByIdDocumentsByDocumentIdSettings",
|
||||
"knowledge-spaces/{id}/documents/{documentId}/settings",
|
||||
),
|
||||
_dataset_edit_operation(
|
||||
"putKnowledgeSpacesByIdDocumentsByDocumentIdSettings",
|
||||
"PUT",
|
||||
"knowledge-spaces/{id}/documents/{documentId}/settings",
|
||||
),
|
||||
_dataset_read_operation("getJobsById", "jobs/{id}"),
|
||||
_dataset_edit_operation("deleteJobsById", "DELETE", "jobs/{id}"),
|
||||
_dataset_edit_operation("postJobsByIdRetry", "POST", "jobs/{id}/retry"),
|
||||
_dataset_read_operation("getDeletionJobsByJobId", "deletion-jobs/{jobId}"),
|
||||
_dataset_edit_operation(
|
||||
"postDeletionJobsByJobIdRetry",
|
||||
"POST",
|
||||
"deletion-jobs/{jobId}/retry",
|
||||
request_headers=("idempotency-key", "x-trace-id"),
|
||||
),
|
||||
_dataset_read_operation("getBulkJobsById", "bulk-jobs/{id}"),
|
||||
)
|
||||
@@ -1,33 +1,32 @@
|
||||
"""Transport-only forwarding for the explicitly enabled KnowledgeFS Console operations.
|
||||
"""Authorize and forward the explicitly enabled KnowledgeFS Console operations.
|
||||
|
||||
KnowledgeFS owns the request and response contract. This module binds short-lived
|
||||
account and workspace identities, enforces Dify's coarse workspace policy, and
|
||||
normalizes transport failures. Dify deliberately maintains a small product-facing
|
||||
operation registry instead of exposing the full upstream OpenAPI surface. The
|
||||
dedicated request path uses Dify's shared SSRF policy, never follows redirects,
|
||||
bounds buffered responses, and rejects compressed responses.
|
||||
The dedicated request path uses Dify's shared SSRF policy, never follows redirects,
|
||||
bounds buffered responses, and rejects compressed streaming responses.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from collections.abc import Iterable, Mapping
|
||||
from dataclasses import dataclass
|
||||
from datetime import UTC, datetime, timedelta
|
||||
from http import HTTPStatus
|
||||
from typing import Final, Literal, NamedTuple, Protocol
|
||||
from typing import NamedTuple, Protocol
|
||||
|
||||
import httpx
|
||||
import jwt
|
||||
|
||||
from configs import dify_config
|
||||
from core.helper import ssrf_proxy
|
||||
from core.rbac import RBACPermission, RBACResourceScope
|
||||
from core.rbac import RBACResourceScope
|
||||
from core.tools.errors import ToolSSRFError
|
||||
from models import Account
|
||||
from services.enterprise.rbac_service import RBACService
|
||||
|
||||
type KnowledgeFSMethod = Literal["DELETE", "GET", "PATCH", "POST", "PUT"]
|
||||
type KnowledgeFSResponseKind = Literal["binary", "buffered", "stream"]
|
||||
type KnowledgeFSRequiredScope = Literal["knowledge-spaces:read", "knowledge-spaces:write"]
|
||||
from services.knowledge_fs_operations import (
|
||||
KNOWLEDGE_FS_CONSOLE_OPERATIONS,
|
||||
KnowledgeFSMethod,
|
||||
KnowledgeFSOperation,
|
||||
KnowledgeFSResponseKind,
|
||||
)
|
||||
|
||||
_JWT_AUDIENCE = "knowledge-fs"
|
||||
_JWT_ISSUER = "dify"
|
||||
@@ -35,56 +34,55 @@ _JWT_TTL_SECONDS = 60
|
||||
_MAX_BUFFERED_RESPONSE_BYTES = 1024 * 1024
|
||||
|
||||
|
||||
class KnowledgeFSOperation(NamedTuple):
|
||||
operation_id: str
|
||||
method: KnowledgeFSMethod
|
||||
path: str
|
||||
response_kind: KnowledgeFSResponseKind
|
||||
required_scope: KnowledgeFSRequiredScope
|
||||
rbac_permission: RBACPermission
|
||||
requires_dataset_editor: bool
|
||||
max_response_bytes: int
|
||||
request_headers: tuple[str, ...]
|
||||
response_headers: tuple[str, ...]
|
||||
response_media_types: tuple[str, ...]
|
||||
|
||||
|
||||
KNOWLEDGE_FS_CONSOLE_OPERATIONS: Final[tuple[KnowledgeFSOperation, ...]] = (
|
||||
KnowledgeFSOperation(
|
||||
operation_id="listKnowledgeSpaces",
|
||||
method="GET",
|
||||
path="knowledge-spaces",
|
||||
response_kind="buffered",
|
||||
required_scope="knowledge-spaces:read",
|
||||
rbac_permission=RBACPermission.DATASET_READONLY,
|
||||
requires_dataset_editor=False,
|
||||
max_response_bytes=1_048_576,
|
||||
request_headers=("x-trace-id",),
|
||||
response_headers=("x-trace-id",),
|
||||
response_media_types=("application/json",),
|
||||
),
|
||||
KnowledgeFSOperation(
|
||||
operation_id="createKnowledgeSpace",
|
||||
method="POST",
|
||||
path="knowledge-spaces",
|
||||
response_kind="buffered",
|
||||
required_scope="knowledge-spaces:write",
|
||||
rbac_permission=RBACPermission.DATASET_CREATE_AND_MANAGEMENT,
|
||||
requires_dataset_editor=True,
|
||||
max_response_bytes=1_048_576,
|
||||
request_headers=("x-trace-id",),
|
||||
response_headers=("x-trace-id",),
|
||||
response_media_types=("application/json",),
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
class KnowledgeFSUpstreamResponse(NamedTuple):
|
||||
response: httpx.Response
|
||||
response_kind: KnowledgeFSResponseKind
|
||||
operation: KnowledgeFSOperation
|
||||
|
||||
|
||||
_AUTHORIZATION_MARKER = object()
|
||||
|
||||
|
||||
@dataclass(eq=False, frozen=True, init=False, slots=True)
|
||||
class KnowledgeFSAuthorization:
|
||||
"""Single-use forwarding capability created after Dify workspace policy checks.
|
||||
|
||||
Callers obtain this value from :func:`authorize_knowledge_fs_request`. Direct
|
||||
construction and repeated forwarding are rejected before outbound I/O.
|
||||
"""
|
||||
|
||||
account_id: str
|
||||
tenant_id: str
|
||||
operation: KnowledgeFSOperation
|
||||
_used: bool
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
account_id: str,
|
||||
tenant_id: str,
|
||||
operation: KnowledgeFSOperation,
|
||||
*,
|
||||
_marker: object | None = None,
|
||||
) -> None:
|
||||
if _marker is not _AUTHORIZATION_MARKER:
|
||||
raise KnowledgeFSAccessDeniedError("KnowledgeFS authorization must be created by workspace authorization")
|
||||
object.__setattr__(self, "account_id", account_id)
|
||||
object.__setattr__(self, "tenant_id", tenant_id)
|
||||
object.__setattr__(self, "operation", operation)
|
||||
object.__setattr__(self, "_used", False)
|
||||
|
||||
def consume(self) -> tuple[str, str, KnowledgeFSOperation]:
|
||||
"""Return the authorized principals and canonical operation exactly once.
|
||||
|
||||
Raises:
|
||||
KnowledgeFSAccessDeniedError: The capability was already consumed.
|
||||
"""
|
||||
if self._used:
|
||||
raise KnowledgeFSAccessDeniedError("KnowledgeFS authorization has already been used")
|
||||
object.__setattr__(self, "_used", True)
|
||||
return self.account_id, self.tenant_id, self.operation
|
||||
|
||||
|
||||
class _RequestHeaders(Protocol):
|
||||
def items(self) -> Iterable[tuple[str, str]]: ...
|
||||
|
||||
@@ -113,20 +111,29 @@ def authorize_knowledge_fs_request(
|
||||
*,
|
||||
account: Account,
|
||||
tenant_id: str,
|
||||
operation: KnowledgeFSOperation,
|
||||
) -> None:
|
||||
method: KnowledgeFSMethod,
|
||||
path: str,
|
||||
) -> KnowledgeFSAuthorization:
|
||||
"""Enforce Dify's workspace policy before KFS performs resource authorization.
|
||||
|
||||
Args:
|
||||
account: Authenticated Dify account with its current workspace role.
|
||||
tenant_id: Current Dify workspace identifier.
|
||||
operation: Dify-maintained KnowledgeFS operation and policy metadata.
|
||||
method: Requested upstream HTTP method.
|
||||
path: Requested relative KnowledgeFS path.
|
||||
|
||||
Raises:
|
||||
KnowledgeFSRouteNotAllowedError: The method and path do not resolve to a declared operation.
|
||||
KnowledgeFSAccessDeniedError: The account lacks a required legacy or enterprise permission.
|
||||
|
||||
Returns:
|
||||
A request-scoped capability binding the authorized account, workspace, and operation.
|
||||
"""
|
||||
if operation.requires_dataset_editor and not account.is_dataset_editor:
|
||||
raise KnowledgeFSAccessDeniedError("KnowledgeFS mutations require dataset edit access")
|
||||
operation = get_knowledge_fs_operation(method, path)
|
||||
if operation.legacy_role == "dataset_editor" and not account.is_dataset_editor:
|
||||
raise KnowledgeFSAccessDeniedError("KnowledgeFS operation requires dataset edit access")
|
||||
if operation.legacy_role == "admin" and not account.is_admin_or_owner:
|
||||
raise KnowledgeFSAccessDeniedError("KnowledgeFS operation requires workspace administration access")
|
||||
if not RBACService.CheckAccess.check(
|
||||
tenant_id,
|
||||
account.id,
|
||||
@@ -134,6 +141,7 @@ def authorize_knowledge_fs_request(
|
||||
resource_type=RBACResourceScope.DATASET.value,
|
||||
):
|
||||
raise KnowledgeFSAccessDeniedError("KnowledgeFS operation is denied by workspace RBAC")
|
||||
return KnowledgeFSAuthorization(account.id, tenant_id, operation, _marker=_AUTHORIZATION_MARKER)
|
||||
|
||||
|
||||
def proxy_knowledge_fs_request(
|
||||
@@ -149,20 +157,62 @@ def proxy_knowledge_fs_request(
|
||||
request_headers: _RequestHeaders | None = None,
|
||||
) -> KnowledgeFSUpstreamResponse:
|
||||
"""Authorize and forward one allowlisted KnowledgeFS request as a single use case."""
|
||||
operation = get_knowledge_fs_operation(method, path)
|
||||
authorize_knowledge_fs_request(
|
||||
authorization = authorize_knowledge_fs_request(
|
||||
account=account,
|
||||
tenant_id=tenant_id,
|
||||
operation=operation,
|
||||
method=method,
|
||||
path=path,
|
||||
)
|
||||
|
||||
return proxy_authorized_knowledge_fs_request(
|
||||
authorization=authorization,
|
||||
accept=accept,
|
||||
content_type=content_type,
|
||||
query=query,
|
||||
body=body,
|
||||
request_headers=request_headers,
|
||||
)
|
||||
|
||||
|
||||
def proxy_authorized_knowledge_fs_request(
|
||||
*,
|
||||
authorization: KnowledgeFSAuthorization,
|
||||
accept: str | None = None,
|
||||
content_type: str | None = None,
|
||||
query: bytes | None = None,
|
||||
body: bytes | None = None,
|
||||
request_headers: _RequestHeaders | None = None,
|
||||
) -> KnowledgeFSUpstreamResponse:
|
||||
"""Forward one request whose operation and workspace policy were already authorized.
|
||||
|
||||
This performs one outbound KnowledgeFS request and does not repeat Dify RBAC checks.
|
||||
|
||||
Args:
|
||||
authorization: Request-scoped capability returned by :func:`authorize_knowledge_fs_request`.
|
||||
accept: Original Accept header, when present.
|
||||
content_type: Original request Content-Type header, when present.
|
||||
query: Original encoded query string from the Console request.
|
||||
body: Original request body, when present.
|
||||
request_headers: Incoming headers; only names declared by the operation are forwarded.
|
||||
|
||||
Returns:
|
||||
The bounded KnowledgeFS response together with its transport metadata.
|
||||
|
||||
Raises:
|
||||
KnowledgeFSConfigurationError: The connection is incomplete or blocked by outbound policy.
|
||||
KnowledgeFSRouteNotAllowedError: A forwarded request header is outside the operation contract.
|
||||
KnowledgeFSTimeoutError: KnowledgeFS exceeds the configured timeout.
|
||||
KnowledgeFSTransportError: The request fails or its response violates transport bounds.
|
||||
"""
|
||||
account_id, tenant_id, operation = authorization.consume()
|
||||
incoming_request_headers = {name.lower(): value for name, value in (request_headers or {}).items()}
|
||||
contract_request_headers = {
|
||||
name: incoming_request_headers[name] for name in operation.request_headers if name in incoming_request_headers
|
||||
}
|
||||
return _forward_knowledge_fs_request(
|
||||
account_id=account.id,
|
||||
method=method,
|
||||
path=path,
|
||||
account_id=account_id,
|
||||
method=operation.method,
|
||||
path=operation.path,
|
||||
tenant_id=tenant_id,
|
||||
accept=accept,
|
||||
content_type=content_type,
|
||||
|
||||
@@ -13,6 +13,7 @@ from core.tools.utils.web_reader_tool import get_image_upload_file_ids
|
||||
from extensions.ext_storage import storage
|
||||
from models.dataset import Dataset, DatasetMetadataBinding, DocumentSegment
|
||||
from models.model import UploadFile
|
||||
from tasks.refresh_billing_vector_space_task import schedule_billing_vector_space_refresh
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
@@ -21,7 +22,12 @@ BATCH_SIZE = 1000
|
||||
|
||||
|
||||
@shared_task(queue="dataset")
|
||||
def batch_clean_document_task(document_ids: list[str], dataset_id: str, doc_form: str | None, file_ids: list[str]):
|
||||
def batch_clean_document_task(
|
||||
document_ids: list[str],
|
||||
dataset_id: str,
|
||||
doc_form: str | None,
|
||||
file_ids: list[str],
|
||||
) -> None:
|
||||
"""
|
||||
Clean document when document deleted.
|
||||
:param document_ids: document ids
|
||||
@@ -40,6 +46,7 @@ def batch_clean_document_task(document_ids: list[str], dataset_id: str, doc_form
|
||||
index_node_ids: list[str] = []
|
||||
segment_ids: list[str] = []
|
||||
total_image_upload_file_ids: list[str] = []
|
||||
dataset_tenant_id: str | None = None
|
||||
|
||||
try:
|
||||
# ============ Step 1: Query segment and file data (short read-only transaction) ============
|
||||
@@ -88,6 +95,7 @@ def batch_clean_document_task(document_ids: list[str], dataset_id: str, doc_form
|
||||
delete_summaries=True,
|
||||
session=session,
|
||||
)
|
||||
dataset_tenant_id = dataset.tenant_id
|
||||
except Exception:
|
||||
logger.exception(
|
||||
"Failed to clean vector index for dataset_id: %s, document_ids: %s, index_node_ids count: %d",
|
||||
@@ -203,6 +211,9 @@ def batch_clean_document_task(document_ids: list[str], dataset_id: str, doc_form
|
||||
dataset_id,
|
||||
)
|
||||
|
||||
if dataset_tenant_id is not None:
|
||||
schedule_billing_vector_space_refresh(dataset_tenant_id)
|
||||
|
||||
end_at = time.perf_counter()
|
||||
logger.info(
|
||||
click.style(
|
||||
|
||||
@@ -24,6 +24,7 @@ from models.dataset import (
|
||||
)
|
||||
from models.model import UploadFile
|
||||
from models.workflow import Workflow
|
||||
from tasks.refresh_billing_vector_space_task import schedule_billing_vector_space_refresh
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
@@ -52,6 +53,7 @@ def clean_dataset_task(
|
||||
"""
|
||||
logger.info(click.style(f"Start clean dataset when dataset deleted: {dataset_id}", fg="green"))
|
||||
start_at = time.perf_counter()
|
||||
vector_cleanup_succeeded = False
|
||||
|
||||
with session_factory.create_session() as session:
|
||||
try:
|
||||
@@ -93,6 +95,7 @@ def clean_dataset_task(
|
||||
try:
|
||||
index_processor = IndexProcessorFactory(doc_form).init_index_processor()
|
||||
index_processor.clean(dataset, None, with_keywords=True, delete_child_chunks=True, session=session)
|
||||
vector_cleanup_succeeded = True
|
||||
logger.info(click.style(f"Successfully cleaned vector database for dataset: {dataset_id}", fg="green"))
|
||||
except Exception:
|
||||
logger.exception(click.style(f"Failed to clean vector database for dataset {dataset_id}", fg="red"))
|
||||
@@ -186,6 +189,8 @@ def clean_dataset_task(
|
||||
session.execute(file_delete_stmt)
|
||||
|
||||
session.commit()
|
||||
if vector_cleanup_succeeded:
|
||||
schedule_billing_vector_space_refresh(dataset.tenant_id)
|
||||
end_at = time.perf_counter()
|
||||
logger.info(
|
||||
click.style(
|
||||
|
||||
@@ -11,12 +11,18 @@ from core.tools.utils.web_reader_tool import get_image_upload_file_ids
|
||||
from extensions.ext_storage import storage
|
||||
from models.dataset import Dataset, DatasetMetadataBinding, DocumentSegment, SegmentAttachmentBinding
|
||||
from models.model import UploadFile
|
||||
from tasks.refresh_billing_vector_space_task import schedule_billing_vector_space_refresh
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
@shared_task(queue="dataset")
|
||||
def clean_document_task(document_id: str, dataset_id: str, doc_form: str, file_id: str | None):
|
||||
def clean_document_task(
|
||||
document_id: str,
|
||||
dataset_id: str,
|
||||
doc_form: str,
|
||||
file_id: str | None,
|
||||
) -> None:
|
||||
"""
|
||||
Clean document when document deleted.
|
||||
:param document_id: document id
|
||||
@@ -29,6 +35,7 @@ def clean_document_task(document_id: str, dataset_id: str, doc_form: str, file_i
|
||||
logger.info(click.style(f"Start clean document when document deleted: {document_id}", fg="green"))
|
||||
start_at = time.perf_counter()
|
||||
total_attachment_files = []
|
||||
vector_cleanup_succeeded = False
|
||||
|
||||
with session_factory.create_session() as session:
|
||||
try:
|
||||
@@ -37,6 +44,7 @@ def clean_document_task(document_id: str, dataset_id: str, doc_form: str, file_i
|
||||
if not dataset:
|
||||
raise Exception("Document has no dataset")
|
||||
|
||||
dataset_tenant_id = dataset.tenant_id
|
||||
segments = session.scalars(select(DocumentSegment).where(DocumentSegment.document_id == document_id)).all()
|
||||
# Use JOIN to fetch attachments with bindings in a single query
|
||||
attachments_with_bindings = session.execute(
|
||||
@@ -82,6 +90,7 @@ def clean_document_task(document_id: str, dataset_id: str, doc_form: str, file_i
|
||||
delete_summaries=True,
|
||||
session=session,
|
||||
)
|
||||
vector_cleanup_succeeded = True
|
||||
except Exception:
|
||||
logger.exception(
|
||||
"Failed to clean vector / keyword index in clean_document_task, "
|
||||
@@ -154,6 +163,9 @@ def clean_document_task(document_id: str, dataset_id: str, doc_form: str, file_i
|
||||
)
|
||||
)
|
||||
|
||||
if vector_cleanup_succeeded:
|
||||
schedule_billing_vector_space_refresh(dataset_tenant_id)
|
||||
|
||||
end_at = time.perf_counter()
|
||||
logger.info(
|
||||
click.style(
|
||||
|
||||
@@ -0,0 +1,59 @@
|
||||
import logging
|
||||
|
||||
from celery import shared_task
|
||||
from opentelemetry import metrics
|
||||
|
||||
from configs import dify_config
|
||||
from services.billing_service import BillingService
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
_MAX_RETRIES = 3
|
||||
_RETRY_DELAY_SECONDS = 30
|
||||
_refresh_counter = metrics.get_meter(__name__).create_counter(
|
||||
"billing.vector_space_cache_refresh.count",
|
||||
description="Number of billing vector-space cache refresh outcomes",
|
||||
)
|
||||
|
||||
|
||||
@shared_task(queue="dataset", bind=True, max_retries=_MAX_RETRIES, default_retry_delay=_RETRY_DELAY_SECONDS)
|
||||
def refresh_billing_vector_space_task(self, tenant_id: str) -> None:
|
||||
"""Refresh billing vector-space usage after vector cleanup has completed."""
|
||||
if not dify_config.BILLING_ENABLED:
|
||||
return
|
||||
|
||||
try:
|
||||
BillingService.invalidate_vector_space_cache(tenant_id)
|
||||
except Exception as exc:
|
||||
if self.request.retries >= _MAX_RETRIES:
|
||||
_refresh_counter.add(1, {"outcome": "exhausted"})
|
||||
logger.exception(
|
||||
"Billing vector-space cache refresh retry budget exhausted, tenant_id=%s",
|
||||
tenant_id,
|
||||
)
|
||||
raise
|
||||
|
||||
_refresh_counter.add(1, {"outcome": "retry"})
|
||||
logger.warning(
|
||||
"Billing vector-space cache refresh failed, scheduling retry %d/%d, tenant_id=%s",
|
||||
self.request.retries + 1,
|
||||
_MAX_RETRIES,
|
||||
tenant_id,
|
||||
exc_info=True,
|
||||
)
|
||||
raise self.retry(exc=exc, countdown=_RETRY_DELAY_SECONDS * (2**self.request.retries))
|
||||
|
||||
_refresh_counter.add(1, {"outcome": "success"})
|
||||
logger.info("Billing vector-space cache refreshed, tenant_id=%s", tenant_id)
|
||||
|
||||
|
||||
def schedule_billing_vector_space_refresh(tenant_id: str) -> None:
|
||||
"""Dispatch a best-effort billing refresh without changing cleanup status."""
|
||||
if not dify_config.BILLING_ENABLED:
|
||||
return
|
||||
|
||||
try:
|
||||
refresh_billing_vector_space_task.delay(tenant_id)
|
||||
except Exception:
|
||||
_refresh_counter.add(1, {"outcome": "dispatch_failure"})
|
||||
logger.exception("Failed to dispatch billing vector-space cache refresh, tenant_id=%s", tenant_id)
|
||||
+1
@@ -318,6 +318,7 @@ def test_oauth_account_successful_retrieval(
|
||||
|
||||
assert response.status_code == 200
|
||||
assert response.get_json() == {
|
||||
"id": account.id,
|
||||
"name": "Test User",
|
||||
"email": account.email,
|
||||
"avatar": "avatar_url",
|
||||
|
||||
@@ -9,7 +9,9 @@ from flask import Flask
|
||||
from sqlalchemy.orm import Session
|
||||
from werkzeug.exceptions import Forbidden
|
||||
|
||||
from configs import dify_config
|
||||
from controllers.web.site import AppSiteApi, WebAppSiteResponse, WebModelConfigResponse
|
||||
from extensions.storage.storage_type import StorageType
|
||||
from models import Tenant, TenantStatus
|
||||
from models.account import TenantCustomConfigDict
|
||||
from models.model import App, AppMode, AppModelConfig, CustomizeTokenStrategy, EndUser, Site
|
||||
@@ -96,6 +98,39 @@ class TestAppSiteApi:
|
||||
assert result["plan"] == "basic"
|
||||
assert result["enable_site"] is True
|
||||
|
||||
@patch("controllers.web.site.FileService.get_file_presigned_url")
|
||||
@patch("controllers.web.site.FeatureService.get_features")
|
||||
def test_image_icon_uses_s3_presigned_url(
|
||||
self,
|
||||
mock_features: MagicMock,
|
||||
mock_get_file_presigned_url: MagicMock,
|
||||
app: Flask,
|
||||
db_session_with_containers: Session,
|
||||
) -> None:
|
||||
app.config["RESTX_MASK_HEADER"] = "X-Fields"
|
||||
tenant = _create_tenant(db_session_with_containers)
|
||||
app_model = _create_app(db_session_with_containers, tenant.id)
|
||||
site = _create_site(db_session_with_containers, app_model.id)
|
||||
site.icon_type = "image"
|
||||
site.icon = "11111111-1111-4111-8111-111111111111"
|
||||
db_session_with_containers.commit()
|
||||
end_user = _end_user(tenant.id, app_model.id)
|
||||
mock_features.return_value = FeatureModel(can_replace_logo=False)
|
||||
mock_get_file_presigned_url.return_value = "https://s3.example.com/icon.png?signature=test"
|
||||
|
||||
with (
|
||||
patch.object(dify_config, "EDITION", "CLOUD"),
|
||||
patch.object(dify_config, "STORAGE_TYPE", StorageType.S3),
|
||||
app.test_request_context("/site"),
|
||||
):
|
||||
result = AppSiteApi().get(app_model, end_user)
|
||||
|
||||
assert result["site"]["icon_url"] == "https://s3.example.com/icon.png?signature=test"
|
||||
mock_get_file_presigned_url.assert_called_once_with(
|
||||
file_id="11111111-1111-4111-8111-111111111111",
|
||||
tenant_id=tenant.id,
|
||||
)
|
||||
|
||||
def test_missing_site_raises_forbidden(self, app: Flask, db_session_with_containers: Session) -> None:
|
||||
app.config["RESTX_MASK_HEADER"] = "X-Fields"
|
||||
tenant = _create_tenant(db_session_with_containers)
|
||||
|
||||
@@ -212,6 +212,17 @@ def test_generate_specs_include_console_contract_shapes_for_schema_migration(tmp
|
||||
assert {"type": "null"} in app_detail_nullable_schema["anyOf"]
|
||||
assert schemas["RecommendedAppInfoResponse"]["properties"]["icon_url"]["readOnly"] is True
|
||||
assert schemas["InstalledAppInfoResponse"]["properties"]["icon_url"]["readOnly"] is True
|
||||
assert _response_schema(paths["/apps/{app_id}"]["get"])["$ref"] == "#/components/schemas/AppDetailWithSite"
|
||||
app_model_config = schemas["AppDetailWithSite"]["properties"]["model_config"]
|
||||
assert {"$ref": "#/components/schemas/AppModelConfigResponse"} in app_model_config["anyOf"]
|
||||
app_detail = schemas["AppDetail"]
|
||||
assert "mode" in app_detail["properties"]
|
||||
assert "mode_compatible_with_agent" not in app_detail["properties"]
|
||||
sync_draft_workflow = schemas["SyncDraftWorkflowResponse"]
|
||||
assert _response_schema(paths["/apps/{app_id}/workflows/draft"]["post"])["$ref"] == (
|
||||
"#/components/schemas/SyncDraftWorkflowResponse"
|
||||
)
|
||||
assert sync_draft_workflow["properties"]["updated_at"]["type"] == "integer"
|
||||
tool_icon_schema = schemas["ExploreAppMetaResponse"]["properties"]["tool_icons"]["additionalProperties"]
|
||||
assert {"type": "string"} in tool_icon_schema["anyOf"]
|
||||
assert {"additionalProperties": True, "type": "object"} in tool_icon_schema["anyOf"]
|
||||
|
||||
@@ -1,15 +1,24 @@
|
||||
"""Unit tests for the reset-encrypt-key-pair CLI command (#35396).
|
||||
"""SQLite-backed tests for the reset-encrypt-key-pair CLI command (#35396).
|
||||
|
||||
The command must purge every table that stores ciphertext encrypted with the
|
||||
tenant's asymmetric key, otherwise stale rows cause downstream API failures
|
||||
such as `/console/api/workspaces/current/tool-providers` returning 500.
|
||||
Tests bind the command-owned transaction to the fixture engine and assert the
|
||||
committed state rather than inspecting fabricated ``Session.execute`` calls.
|
||||
"""
|
||||
|
||||
from unittest.mock import MagicMock, patch
|
||||
from types import SimpleNamespace
|
||||
|
||||
import pytest
|
||||
from sqlalchemy import select
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
import commands
|
||||
from commands import system as system_commands
|
||||
from models.provider import Provider, ProviderModel
|
||||
from core.tools.entities.tool_entities import ApiProviderSchemaType
|
||||
from graphon.model_runtime.entities.model_entities import ModelType
|
||||
from models import Tenant
|
||||
from models.provider import Provider, ProviderModel, ProviderType
|
||||
from models.tools import ApiToolProvider, BuiltinToolProvider, MCPToolProvider
|
||||
|
||||
|
||||
@@ -21,17 +30,60 @@ def _invoke_reset() -> int:
|
||||
return 0
|
||||
|
||||
|
||||
def _delete_targets(session_mock: MagicMock) -> list:
|
||||
"""Extract the model class targeted by each `delete(...)` call on the session."""
|
||||
targets = []
|
||||
for call in session_mock.execute.call_args_list:
|
||||
stmt = call.args[0]
|
||||
# `delete(Foo)` constructs a `Delete` statement whose entity is `Foo`.
|
||||
try:
|
||||
targets.append(stmt.table.name)
|
||||
except AttributeError:
|
||||
targets.append(repr(stmt))
|
||||
return targets
|
||||
TENANT_ID = "11111111-1111-1111-1111-111111111111"
|
||||
OTHER_TENANT_ID = "11111111-1111-1111-1111-111111111112"
|
||||
USER_ID = "22222222-2222-2222-2222-222222222222"
|
||||
|
||||
|
||||
def _tenant(tenant_id: str, *, name: str = "Test tenant") -> Tenant:
|
||||
tenant = Tenant(name=name, encrypt_public_key="old-key")
|
||||
tenant.id = tenant_id
|
||||
return tenant
|
||||
|
||||
|
||||
def _encrypted_rows(tenant_id: str, *, suffix: str = "1") -> tuple[object, ...]:
|
||||
"""Build one persisted credential-bearing row for every purge target."""
|
||||
return (
|
||||
Provider(tenant_id=tenant_id, provider_name=f"provider-{suffix}"),
|
||||
ProviderModel(
|
||||
tenant_id=tenant_id,
|
||||
provider_name=f"provider-{suffix}",
|
||||
model_name=f"model-{suffix}",
|
||||
model_type=ModelType.LLM,
|
||||
),
|
||||
BuiltinToolProvider(
|
||||
name=f"builtin-credential-{suffix}",
|
||||
tenant_id=tenant_id,
|
||||
user_id=USER_ID,
|
||||
provider=f"builtin-{suffix}",
|
||||
encrypted_credentials="ciphertext",
|
||||
),
|
||||
ApiToolProvider(
|
||||
name=f"api-{suffix}",
|
||||
icon="icon",
|
||||
schema="{}",
|
||||
schema_type_str=ApiProviderSchemaType.OPENAPI,
|
||||
user_id=USER_ID,
|
||||
tenant_id=tenant_id,
|
||||
description="description",
|
||||
tools_str="[]",
|
||||
credentials_str="{}",
|
||||
),
|
||||
MCPToolProvider(
|
||||
name=f"mcp-{suffix}",
|
||||
server_identifier=f"server-{suffix}",
|
||||
server_url="ciphertext",
|
||||
server_url_hash=f"hash-{suffix}",
|
||||
icon=None,
|
||||
tenant_id=tenant_id,
|
||||
user_id=USER_ID,
|
||||
encrypted_credentials="ciphertext",
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
def _bind_command_to_sqlite(monkeypatch: pytest.MonkeyPatch, session: Session) -> None:
|
||||
monkeypatch.setattr(system_commands, "db", SimpleNamespace(engine=session.get_bind()))
|
||||
|
||||
|
||||
def test_reset_aborts_when_not_self_hosted(monkeypatch, capsys):
|
||||
@@ -44,65 +96,73 @@ def test_reset_aborts_when_not_self_hosted(monkeypatch, capsys):
|
||||
assert "only for SELF_HOSTED" in captured.out
|
||||
|
||||
|
||||
def test_reset_purges_provider_and_tool_tables_for_each_tenant(monkeypatch, capsys):
|
||||
@pytest.mark.parametrize(
|
||||
"sqlite_session",
|
||||
[(Tenant, Provider, ProviderModel, BuiltinToolProvider, ApiToolProvider, MCPToolProvider)],
|
||||
indirect=True,
|
||||
)
|
||||
def test_reset_purges_provider_and_tool_tables_for_each_tenant(
|
||||
monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str], sqlite_session: Session
|
||||
) -> None:
|
||||
"""The command must purge LLM provider rows AND every tool provider table
|
||||
that stores ciphertext encrypted under the tenant key (#35396)."""
|
||||
monkeypatch.setattr(system_commands.dify_config, "EDITION", "SELF_HOSTED")
|
||||
monkeypatch.setattr(system_commands, "generate_key_pair", lambda tenant_id: f"new-key-{tenant_id}")
|
||||
_bind_command_to_sqlite(monkeypatch, sqlite_session)
|
||||
|
||||
fake_tenant = MagicMock(id="tenant-abc", encrypt_public_key="old-key")
|
||||
session = MagicMock()
|
||||
session.scalars.return_value.all.return_value = [fake_tenant]
|
||||
tenant = _tenant(TENANT_ID)
|
||||
other_tenant = _tenant(OTHER_TENANT_ID, name="Other tenant")
|
||||
system_provider = Provider(
|
||||
tenant_id=TENANT_ID,
|
||||
provider_name="system-provider",
|
||||
provider_type=ProviderType.SYSTEM,
|
||||
)
|
||||
sqlite_session.add_all((tenant, other_tenant, system_provider, *_encrypted_rows(TENANT_ID)))
|
||||
sqlite_session.commit()
|
||||
|
||||
fake_sessionmaker = MagicMock()
|
||||
fake_sessionmaker.begin.return_value.__enter__.return_value = session
|
||||
fake_sessionmaker.begin.return_value.__exit__.return_value = False
|
||||
|
||||
with (
|
||||
patch.object(system_commands, "db", MagicMock()),
|
||||
patch.object(system_commands, "sessionmaker", return_value=fake_sessionmaker),
|
||||
):
|
||||
exit_code = _invoke_reset()
|
||||
exit_code = _invoke_reset()
|
||||
|
||||
captured = capsys.readouterr()
|
||||
assert exit_code == 0
|
||||
assert "tenant-abc" in captured.out
|
||||
assert TENANT_ID in captured.out
|
||||
|
||||
# New key pair generated and assigned.
|
||||
assert fake_tenant.encrypt_public_key == "new-key-tenant-abc"
|
||||
|
||||
# Every encrypted-credential table should have been purged for this tenant.
|
||||
table_names = _delete_targets(session)
|
||||
expected = {
|
||||
Provider.__tablename__,
|
||||
ProviderModel.__tablename__,
|
||||
BuiltinToolProvider.__tablename__,
|
||||
ApiToolProvider.__tablename__,
|
||||
MCPToolProvider.__tablename__,
|
||||
}
|
||||
assert expected.issubset(set(table_names)), f"missing purges: expected {expected}, got {table_names}"
|
||||
sqlite_session.expire_all()
|
||||
assert sqlite_session.get(Tenant, TENANT_ID).encrypt_public_key == f"new-key-{TENANT_ID}"
|
||||
assert sqlite_session.get(Tenant, OTHER_TENANT_ID).encrypt_public_key == f"new-key-{OTHER_TENANT_ID}"
|
||||
assert sqlite_session.scalars(select(Provider).where(Provider.provider_type == ProviderType.CUSTOM)).all() == []
|
||||
assert sqlite_session.scalars(select(ProviderModel)).all() == []
|
||||
assert sqlite_session.scalars(select(BuiltinToolProvider)).all() == []
|
||||
assert sqlite_session.scalars(select(ApiToolProvider)).all() == []
|
||||
assert sqlite_session.scalars(select(MCPToolProvider)).all() == []
|
||||
assert (
|
||||
sqlite_session.scalar(select(Provider).where(Provider.provider_type == ProviderType.SYSTEM)) is system_provider
|
||||
)
|
||||
|
||||
|
||||
def test_reset_iterates_all_tenants(monkeypatch, capsys):
|
||||
@pytest.mark.parametrize(
|
||||
"sqlite_session",
|
||||
[(Tenant, Provider, ProviderModel, BuiltinToolProvider, ApiToolProvider, MCPToolProvider)],
|
||||
indirect=True,
|
||||
)
|
||||
def test_reset_iterates_all_tenants(monkeypatch: pytest.MonkeyPatch, sqlite_session: Session) -> None:
|
||||
"""Multi-tenant deployments must purge every tenant, not just the first."""
|
||||
monkeypatch.setattr(system_commands.dify_config, "EDITION", "SELF_HOSTED")
|
||||
monkeypatch.setattr(system_commands, "generate_key_pair", lambda tenant_id: f"new-key-{tenant_id}")
|
||||
|
||||
tenants = [MagicMock(id=f"tenant-{i}", encrypt_public_key="old") for i in range(3)]
|
||||
session = MagicMock()
|
||||
session.scalars.return_value.all.return_value = tenants
|
||||
_bind_command_to_sqlite(monkeypatch, sqlite_session)
|
||||
tenant_ids = [f"11111111-1111-1111-1111-{index:012d}" for index in range(3)]
|
||||
tenants = [_tenant(tenant_id, name=f"Tenant {index}") for index, tenant_id in enumerate(tenant_ids)]
|
||||
for index, tenant in enumerate(tenants):
|
||||
sqlite_session.add(tenant)
|
||||
sqlite_session.add_all(_encrypted_rows(tenant.id, suffix=str(index)))
|
||||
sqlite_session.commit()
|
||||
|
||||
fake_sessionmaker = MagicMock()
|
||||
fake_sessionmaker.begin.return_value.__enter__.return_value = session
|
||||
fake_sessionmaker.begin.return_value.__exit__.return_value = False
|
||||
assert _invoke_reset() == 0
|
||||
|
||||
with (
|
||||
patch.object(system_commands, "db", MagicMock()),
|
||||
patch.object(system_commands, "sessionmaker", return_value=fake_sessionmaker),
|
||||
):
|
||||
_invoke_reset()
|
||||
|
||||
# Five purges per tenant × 3 tenants = 15 execute calls.
|
||||
assert session.execute.call_count == 15
|
||||
for tenant in tenants:
|
||||
assert tenant.encrypt_public_key == f"new-key-{tenant.id}"
|
||||
sqlite_session.expire_all()
|
||||
persisted_tenants = sqlite_session.scalars(select(Tenant).order_by(Tenant.id)).all()
|
||||
assert [tenant.encrypt_public_key for tenant in persisted_tenants] == [
|
||||
f"new-key-{tenant_id}" for tenant_id in tenant_ids
|
||||
]
|
||||
for model in (Provider, ProviderModel, BuiltinToolProvider, ApiToolProvider, MCPToolProvider):
|
||||
assert sqlite_session.scalars(select(model)).all() == []
|
||||
|
||||
@@ -37,6 +37,7 @@ os.environ.setdefault("STORAGE_TYPE", "opendal")
|
||||
|
||||
from core.db.session_factory import configure_session_factory, session_factory
|
||||
from extensions import ext_redis
|
||||
from models.account import Account, Tenant, TenantAccountJoin, TenantAccountRole
|
||||
from models.base import TypeBase
|
||||
|
||||
|
||||
@@ -148,32 +149,45 @@ def _configure_session_factory(_unit_test_engine):
|
||||
configure_session_factory(_unit_test_engine, expire_on_commit=False)
|
||||
|
||||
|
||||
def setup_mock_tenant_owner_execute_result(mock_db, mock_tenant, mock_owner):
|
||||
"""
|
||||
Helper to stub the tenant-owner execute result for service API app authentication.
|
||||
def persist_service_api_tenant_owner(session: Session, tenant: Tenant, owner: Account) -> TenantAccountJoin:
|
||||
"""Persist the owner identity resolved by service-API app authentication.
|
||||
|
||||
The validate_app_token decorator currently resolves the active tenant owner
|
||||
via db.session.execute(select(Tenant, Account)...).one_or_none().
|
||||
|
||||
Args:
|
||||
mock_db: The mocked db object
|
||||
mock_tenant: Mock tenant object to return
|
||||
mock_owner: Mock owner object to return from the execute result
|
||||
The legacy name is retained temporarily for consumers on independent
|
||||
conversion branches, but this helper no longer fabricates an execute result.
|
||||
"""
|
||||
membership = TenantAccountJoin(
|
||||
tenant_id=tenant.id,
|
||||
account_id=owner.id,
|
||||
role=TenantAccountRole.OWNER,
|
||||
)
|
||||
owner._current_tenant = tenant
|
||||
session.add_all([tenant, owner, membership])
|
||||
session.commit()
|
||||
return membership
|
||||
|
||||
|
||||
def persist_service_api_dataset_owner(
|
||||
session: Session,
|
||||
tenant: Tenant,
|
||||
tenant_account_join: TenantAccountJoin,
|
||||
) -> None:
|
||||
"""Persist the tenant-owner mapping resolved by dataset-token authentication."""
|
||||
session.add_all([tenant, tenant_account_join])
|
||||
session.commit()
|
||||
|
||||
|
||||
def setup_mock_tenant_owner_execute_result(mock_db: MagicMock, mock_tenant: object, mock_owner: object) -> None:
|
||||
"""Stub the legacy owner query; SQLite-backed tests use ``persist_service_api_tenant_owner``."""
|
||||
mock_db.session.execute.return_value.one_or_none.return_value = (mock_tenant, mock_owner)
|
||||
|
||||
|
||||
def setup_mock_dataset_owner_execute_result(mock_db, mock_tenant, mock_tenant_account_join):
|
||||
"""
|
||||
Helper to stub the tenant-owner execute result for dataset token authentication.
|
||||
|
||||
The validate_dataset_token decorator currently resolves the owner mapping via
|
||||
db.session.execute(select(Tenant, TenantAccountJoin)...).one_or_none(), and
|
||||
then loads the Account separately via db.session.get(...).
|
||||
|
||||
Args:
|
||||
mock_db: The mocked db object
|
||||
mock_tenant: Mock tenant object to return
|
||||
mock_tenant_account_join: Mock tenant-account join object to return
|
||||
"""
|
||||
mock_db.session.execute.return_value.one_or_none.return_value = (mock_tenant, mock_tenant_account_join)
|
||||
def setup_mock_dataset_owner_execute_result(
|
||||
mock_db: MagicMock,
|
||||
mock_tenant: object,
|
||||
mock_tenant_account_join: object,
|
||||
) -> None:
|
||||
"""Stub the legacy dataset-owner query; SQLite tests use ``persist_service_api_dataset_owner``."""
|
||||
mock_db.session.execute.return_value.one_or_none.return_value = (
|
||||
mock_tenant,
|
||||
mock_tenant_account_join,
|
||||
)
|
||||
|
||||
@@ -53,6 +53,7 @@ from controllers.console.app.message import (
|
||||
AgentMessageFeedbackApi,
|
||||
AgentMessageSuggestedQuestionApi,
|
||||
)
|
||||
from models.agent import AgentConfigDraftType
|
||||
from services.entities.agent_entities import ComposerSaveStrategy, ComposerVariant
|
||||
|
||||
|
||||
@@ -68,6 +69,20 @@ def _version_response(version_id: str = "version-1") -> dict:
|
||||
}
|
||||
|
||||
|
||||
def test_query_values_accepts_repeated_and_indexed_arrays() -> None:
|
||||
app = Flask(__name__)
|
||||
|
||||
with app.test_request_context("/?sources=webapp:app-1&sources%5B1%5D=workflow:app-2&sources%5B0%5D=workflow:app-1"):
|
||||
assert roster_controller._query_values("sources", "source") == [
|
||||
"webapp:app-1",
|
||||
"workflow:app-1",
|
||||
"workflow:app-2",
|
||||
]
|
||||
|
||||
with app.test_request_context("/?source%5B0%5D=workflow:app-3"):
|
||||
assert roster_controller._query_values("sources", "source") == ["workflow:app-3"]
|
||||
|
||||
|
||||
def _workflow_composer_response(**overrides) -> dict:
|
||||
response = {
|
||||
"variant": "workflow",
|
||||
@@ -371,6 +386,7 @@ def test_agent_app_list_and_create_use_agent_route(
|
||||
"tenant_id": "tenant-1",
|
||||
"agent_id": "agent-created",
|
||||
"account_id": account_id,
|
||||
"draft_type": AgentConfigDraftType.DEBUG_BUILD,
|
||||
"commit": False,
|
||||
}
|
||||
|
||||
@@ -544,8 +560,19 @@ def test_agent_app_copy_uses_agent_id_and_returns_agent_detail(
|
||||
}
|
||||
|
||||
|
||||
def test_agent_debug_conversation_refresh_uses_current_user(
|
||||
app: Flask, monkeypatch: pytest.MonkeyPatch, account_id: str
|
||||
@pytest.mark.parametrize(
|
||||
("payload", "expected_draft_type"),
|
||||
[
|
||||
(None, AgentConfigDraftType.DEBUG_BUILD),
|
||||
({"draft_type": "draft"}, AgentConfigDraftType.DRAFT),
|
||||
],
|
||||
)
|
||||
def test_agent_debug_conversation_refresh_uses_current_user_and_draft_type(
|
||||
app: Flask,
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
account_id: str,
|
||||
payload: dict[str, str] | None,
|
||||
expected_draft_type: AgentConfigDraftType,
|
||||
) -> None:
|
||||
agent_id = "00000000-0000-0000-0000-000000000001"
|
||||
captured: dict[str, object] = {}
|
||||
@@ -557,7 +584,9 @@ def test_agent_debug_conversation_refresh_uses_current_user(
|
||||
|
||||
monkeypatch.setattr(roster_controller, "_agent_roster_service", lambda *_args: FakeRosterService())
|
||||
with app.test_request_context(
|
||||
"/console/api/agent/00000000-0000-0000-0000-000000000001/debug-conversation/refresh", method="POST"
|
||||
"/console/api/agent/00000000-0000-0000-0000-000000000001/debug-conversation/refresh",
|
||||
method="POST",
|
||||
json=payload,
|
||||
):
|
||||
response = unwrap(AgentDebugConversationRefreshApi.post)(
|
||||
AgentDebugConversationRefreshApi(), MagicMock(), "tenant-1", SimpleNamespace(id=account_id), agent_id
|
||||
@@ -567,7 +596,12 @@ def test_agent_debug_conversation_refresh_uses_current_user(
|
||||
"debug_conversation_has_messages": False,
|
||||
"debug_conversation_message_count": 0,
|
||||
}
|
||||
assert captured == {"tenant_id": "tenant-1", "agent_id": agent_id, "account_id": account_id}
|
||||
assert captured == {
|
||||
"tenant_id": "tenant-1",
|
||||
"agent_id": agent_id,
|
||||
"account_id": account_id,
|
||||
"draft_type": expected_draft_type,
|
||||
}
|
||||
|
||||
|
||||
def test_agent_publish_and_build_draft_routes_call_composer_service(
|
||||
@@ -1456,6 +1490,7 @@ def test_build_chat_finalization_helper_forces_debug_build_and_push_prompt(
|
||||
"current_user": SimpleNamespace(id=account_id),
|
||||
"app_model": app_model,
|
||||
"agent_id": "agent-1",
|
||||
"draft_type": AgentConfigDraftType.DEBUG_BUILD,
|
||||
}
|
||||
generate_call = cast(dict[str, object], captured["generate"])
|
||||
assert generate_call["app_model"] is app_model
|
||||
@@ -1509,8 +1544,35 @@ def test_drain_streaming_generate_response_raises_when_stream_ends_early() -> No
|
||||
completion_controller._drain_streaming_generate_response(response)
|
||||
|
||||
|
||||
def test_agent_chat_helper_forces_agent_streaming_and_external_trace(
|
||||
app: Flask, monkeypatch: pytest.MonkeyPatch, account_id: str
|
||||
@pytest.mark.parametrize(
|
||||
("payload_extra", "expected_draft_type", "expected_start_new"),
|
||||
[
|
||||
({}, AgentConfigDraftType.DRAFT, True),
|
||||
({"conversation_id": None}, AgentConfigDraftType.DRAFT, True),
|
||||
({"conversation_id": ""}, AgentConfigDraftType.DRAFT, True),
|
||||
(
|
||||
{"conversation_id": "00000000-0000-0000-0000-000000000001"},
|
||||
AgentConfigDraftType.DRAFT,
|
||||
False,
|
||||
),
|
||||
({"draft_type": "debug_build"}, AgentConfigDraftType.DEBUG_BUILD, False),
|
||||
(
|
||||
{
|
||||
"draft_type": "debug_build",
|
||||
"conversation_id": "00000000-0000-0000-0000-000000000001",
|
||||
},
|
||||
AgentConfigDraftType.DEBUG_BUILD,
|
||||
False,
|
||||
),
|
||||
],
|
||||
)
|
||||
def test_agent_chat_helper_resolves_scoped_conversation_and_forces_streaming(
|
||||
app: Flask,
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
account_id: str,
|
||||
payload_extra: dict[str, str | None],
|
||||
expected_draft_type: AgentConfigDraftType,
|
||||
expected_start_new: bool,
|
||||
) -> None:
|
||||
app_model = SimpleNamespace(id="app-1", tenant_id="tenant-1", mode="agent")
|
||||
current_user = SimpleNamespace(id=account_id)
|
||||
@@ -1520,17 +1582,22 @@ def test_agent_chat_helper_forces_agent_streaming_and_external_trace(
|
||||
captured.update(kwargs)
|
||||
return {"answer": "ok"}
|
||||
|
||||
def resolve_debug_conversation(**kwargs: object) -> str:
|
||||
captured["resolve_debug_conversation"] = kwargs
|
||||
return "00000000-0000-0000-0000-000000000001"
|
||||
|
||||
monkeypatch.setattr(completion_controller.AppGenerateService, "generate", generate)
|
||||
monkeypatch.setattr(
|
||||
completion_controller,
|
||||
"_resolve_current_user_agent_debug_conversation_id",
|
||||
lambda **kwargs: "debug-conversation-1",
|
||||
resolve_debug_conversation,
|
||||
)
|
||||
monkeypatch.setattr(
|
||||
completion_controller.helper, "compact_generate_response", lambda response: {"response": response}
|
||||
)
|
||||
with app.test_request_context(
|
||||
json={"inputs": {}, "query": "hello", "response_mode": "streaming"}, headers={"X-Trace-Id": "trace-1"}
|
||||
json={"inputs": {}, "query": "hello", "response_mode": "streaming", **payload_extra},
|
||||
headers={"X-Trace-Id": "trace-1"},
|
||||
):
|
||||
result = completion_controller._create_chat_message(
|
||||
current_user=current_user, app_model=app_model, session=Mock()
|
||||
@@ -1541,9 +1608,12 @@ def test_agent_chat_helper_forces_agent_streaming_and_external_trace(
|
||||
assert captured["streaming"] is True
|
||||
args = cast(dict[str, object], captured["args"])
|
||||
assert args["response_mode"] == "streaming"
|
||||
assert args["conversation_id"] == "debug-conversation-1"
|
||||
assert args["conversation_id"] == "00000000-0000-0000-0000-000000000001"
|
||||
assert args["auto_generate_name"] is False
|
||||
assert args["external_trace_id"] == "trace-1"
|
||||
resolve_call = cast(dict[str, object], captured["resolve_debug_conversation"])
|
||||
assert resolve_call["draft_type"] == expected_draft_type
|
||||
assert resolve_call["start_new"] is expected_start_new
|
||||
|
||||
|
||||
def test_agent_chat_helper_ignores_private_exit_intent_payload_key(
|
||||
@@ -1591,14 +1661,28 @@ def test_agent_chat_helper_ignores_private_exit_intent_payload_key(
|
||||
assert completion_controller.AGENT_RUNTIME_EXIT_INTENT_ARG not in args
|
||||
|
||||
|
||||
def test_agent_chat_helper_rejects_foreign_debug_conversation(
|
||||
app: Flask, monkeypatch: pytest.MonkeyPatch, account_id: str
|
||||
@pytest.mark.parametrize(
|
||||
("payload_extra", "expected_draft_type"),
|
||||
[
|
||||
({}, AgentConfigDraftType.DRAFT),
|
||||
({"draft_type": "debug_build"}, AgentConfigDraftType.DEBUG_BUILD),
|
||||
],
|
||||
)
|
||||
def test_agent_chat_helper_rejects_foreign_debug_conversation_before_generation(
|
||||
app: Flask,
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
account_id: str,
|
||||
payload_extra: dict[str, str],
|
||||
expected_draft_type: AgentConfigDraftType,
|
||||
) -> None:
|
||||
app_model = SimpleNamespace(id="app-1", tenant_id="tenant-1", mode="agent")
|
||||
generate = MagicMock()
|
||||
resolve_debug_conversation = MagicMock(return_value="owned-conversation")
|
||||
monkeypatch.setattr(completion_controller.AppGenerateService, "generate", generate)
|
||||
monkeypatch.setattr(
|
||||
completion_controller,
|
||||
"_resolve_current_user_agent_debug_conversation_id",
|
||||
lambda **kwargs: "owned-conversation",
|
||||
resolve_debug_conversation,
|
||||
)
|
||||
with app.test_request_context(
|
||||
json={
|
||||
@@ -1606,6 +1690,7 @@ def test_agent_chat_helper_rejects_foreign_debug_conversation(
|
||||
"query": "hello",
|
||||
"response_mode": "streaming",
|
||||
"conversation_id": "00000000-0000-0000-0000-000000000001",
|
||||
**payload_extra,
|
||||
}
|
||||
):
|
||||
with pytest.raises(NotFound):
|
||||
@@ -1617,6 +1702,12 @@ def test_agent_chat_helper_rejects_foreign_debug_conversation(
|
||||
session=Mock(),
|
||||
)
|
||||
|
||||
resolve_debug_conversation.assert_called_once()
|
||||
resolve_call = resolve_debug_conversation.call_args.kwargs
|
||||
assert resolve_call["draft_type"] == expected_draft_type
|
||||
assert resolve_call["start_new"] is False
|
||||
generate.assert_not_called()
|
||||
|
||||
|
||||
def test_resolve_current_user_agent_debug_conversation_uses_agent_or_backing_app(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
@@ -1631,6 +1722,10 @@ def test_resolve_current_user_agent_debug_conversation_uses_agent_or_backing_app
|
||||
calls.append({"get_or_create": kwargs})
|
||||
return f"debug-{kwargs['agent_id']}"
|
||||
|
||||
def refresh_agent_app_debug_conversation_id(self, **kwargs: object) -> str:
|
||||
calls.append({"refresh": kwargs})
|
||||
return f"new-{kwargs['agent_id']}"
|
||||
|
||||
def get_app_backing_agent(self, **kwargs: object) -> object:
|
||||
calls.append({"get_app_backing_agent": kwargs})
|
||||
return SimpleNamespace(id="backing-agent")
|
||||
@@ -1642,6 +1737,8 @@ def test_resolve_current_user_agent_debug_conversation_uses_agent_or_backing_app
|
||||
current_user=SimpleNamespace(id="account-1"),
|
||||
app_model=SimpleNamespace(id="app-1"),
|
||||
agent_id="agent-1",
|
||||
draft_type=AgentConfigDraftType.DRAFT,
|
||||
start_new=True,
|
||||
)
|
||||
fallback_id = completion_controller._resolve_current_user_agent_debug_conversation_id(
|
||||
session="session-1", # type: ignore[arg-type]
|
||||
@@ -1649,13 +1746,45 @@ def test_resolve_current_user_agent_debug_conversation_uses_agent_or_backing_app
|
||||
current_user=SimpleNamespace(id="account-1"),
|
||||
app_model=SimpleNamespace(id="app-1"),
|
||||
agent_id=None,
|
||||
draft_type=AgentConfigDraftType.DEBUG_BUILD,
|
||||
)
|
||||
assert explicit_id == "debug-agent-1"
|
||||
fallback_preview_id = completion_controller._resolve_current_user_agent_debug_conversation_id(
|
||||
session="session-1", # type: ignore[arg-type]
|
||||
current_tenant_id="tenant-1",
|
||||
current_user=SimpleNamespace(id="account-1"),
|
||||
app_model=SimpleNamespace(id="app-1"),
|
||||
agent_id=None,
|
||||
draft_type=AgentConfigDraftType.DRAFT,
|
||||
start_new=True,
|
||||
)
|
||||
assert explicit_id == "new-agent-1"
|
||||
assert fallback_id == "debug-backing-agent"
|
||||
assert calls[1] == {"get_or_create": {"tenant_id": "tenant-1", "agent_id": "agent-1", "account_id": "account-1"}}
|
||||
assert fallback_preview_id == "new-backing-agent"
|
||||
assert calls[1] == {
|
||||
"refresh": {
|
||||
"tenant_id": "tenant-1",
|
||||
"agent_id": "agent-1",
|
||||
"account_id": "account-1",
|
||||
"draft_type": AgentConfigDraftType.DRAFT,
|
||||
}
|
||||
}
|
||||
assert calls[3] == {"get_app_backing_agent": {"tenant_id": "tenant-1", "app_id": "app-1"}}
|
||||
assert calls[4] == {
|
||||
"get_or_create": {"tenant_id": "tenant-1", "agent_id": "backing-agent", "account_id": "account-1"}
|
||||
"get_or_create": {
|
||||
"tenant_id": "tenant-1",
|
||||
"agent_id": "backing-agent",
|
||||
"account_id": "account-1",
|
||||
"draft_type": AgentConfigDraftType.DEBUG_BUILD,
|
||||
}
|
||||
}
|
||||
assert calls[6] == {"get_app_backing_agent": {"tenant_id": "tenant-1", "app_id": "app-1"}}
|
||||
assert calls[7] == {
|
||||
"refresh": {
|
||||
"tenant_id": "tenant-1",
|
||||
"agent_id": "backing-agent",
|
||||
"account_id": "account-1",
|
||||
"draft_type": AgentConfigDraftType.DRAFT,
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
@@ -0,0 +1,150 @@
|
||||
from types import SimpleNamespace
|
||||
from unittest.mock import MagicMock, patch
|
||||
|
||||
import pytest
|
||||
from werkzeug.exceptions import Forbidden
|
||||
|
||||
from controllers.console.app.error import AppNotFoundError
|
||||
from controllers.console.app.wraps import agent_manage_required_for_agent_app
|
||||
from core.rbac import RBACPermission, RBACResourceScope
|
||||
from models.agent import AgentScope
|
||||
|
||||
TENANT_ID = "tenant-1"
|
||||
ACCOUNT = SimpleNamespace(id="account-1")
|
||||
|
||||
|
||||
def _guarded_view():
|
||||
calls: list[dict[str, object]] = []
|
||||
|
||||
@agent_manage_required_for_agent_app
|
||||
def view(*args, **kwargs):
|
||||
calls.append(kwargs)
|
||||
return "ok"
|
||||
|
||||
return view, calls
|
||||
|
||||
|
||||
def _app_with_binding(binding):
|
||||
app_model = MagicMock()
|
||||
app_model.agent_app_binding_with_session.return_value = binding
|
||||
return app_model
|
||||
|
||||
|
||||
def _patch_guard(app_model, rbac_enabled: bool):
|
||||
mock_db = MagicMock()
|
||||
mock_db.session.scalar.return_value = app_model
|
||||
return (
|
||||
patch("controllers.console.app.wraps.db", mock_db),
|
||||
patch("controllers.console.app.wraps.current_account_with_tenant", return_value=(ACCOUNT, TENANT_ID)),
|
||||
patch("controllers.console.app.wraps.dify_config.RBAC_ENABLED", rbac_enabled),
|
||||
)
|
||||
|
||||
|
||||
class TestAgentManageRequiredForAgentApp:
|
||||
def test_non_agent_app_passes_through_without_workspace_check(self):
|
||||
view, calls = _guarded_view()
|
||||
patches = _patch_guard(_app_with_binding(None), rbac_enabled=True)
|
||||
|
||||
with patches[0], patches[1], patches[2], patch("controllers.console.app.wraps.enforce_rbac_access") as gate:
|
||||
assert view(app_id="app-1") == "ok"
|
||||
|
||||
gate.assert_not_called()
|
||||
assert calls == [{"app_id": "app-1"}]
|
||||
|
||||
def test_roster_agent_app_requires_agent_manage_when_rbac_enabled(self):
|
||||
view, _ = _guarded_view()
|
||||
binding = SimpleNamespace(scope=AgentScope.ROSTER)
|
||||
patches = _patch_guard(_app_with_binding(binding), rbac_enabled=True)
|
||||
|
||||
with patches[0], patches[1], patches[2], patch("controllers.console.app.wraps.enforce_rbac_access") as gate:
|
||||
assert view(app_id="app-1") == "ok"
|
||||
|
||||
gate.assert_called_once_with(
|
||||
tenant_id=TENANT_ID,
|
||||
account_id=ACCOUNT.id,
|
||||
resource_type=RBACResourceScope.WORKSPACE,
|
||||
scene=RBACPermission.AGENT_MANAGE,
|
||||
resource_required=False,
|
||||
)
|
||||
|
||||
def test_roster_agent_app_denied_without_agent_manage(self):
|
||||
view, calls = _guarded_view()
|
||||
binding = SimpleNamespace(scope=AgentScope.ROSTER)
|
||||
patches = _patch_guard(_app_with_binding(binding), rbac_enabled=True)
|
||||
|
||||
with (
|
||||
patches[0],
|
||||
patches[1],
|
||||
patches[2],
|
||||
patch("controllers.console.app.wraps.enforce_rbac_access", side_effect=Forbidden()),
|
||||
):
|
||||
with pytest.raises(Forbidden):
|
||||
view(app_id="app-1")
|
||||
|
||||
assert calls == []
|
||||
|
||||
def test_roster_agent_app_skips_workspace_check_when_rbac_disabled(self):
|
||||
view, _ = _guarded_view()
|
||||
binding = SimpleNamespace(scope=AgentScope.ROSTER)
|
||||
patches = _patch_guard(_app_with_binding(binding), rbac_enabled=False)
|
||||
|
||||
with patches[0], patches[1], patches[2], patch("controllers.console.app.wraps.enforce_rbac_access") as gate:
|
||||
assert view(app_id="app-1") == "ok"
|
||||
|
||||
gate.assert_not_called()
|
||||
|
||||
def test_hidden_backing_app_is_rejected_even_without_rbac(self):
|
||||
"""A workflow-only backing App is not part of the general app management plane."""
|
||||
view, calls = _guarded_view()
|
||||
binding = SimpleNamespace(scope=AgentScope.WORKFLOW_ONLY)
|
||||
patches = _patch_guard(_app_with_binding(binding), rbac_enabled=False)
|
||||
|
||||
with patches[0], patches[1], patches[2]:
|
||||
with pytest.raises(AppNotFoundError):
|
||||
view(app_id="app-1")
|
||||
|
||||
assert calls == []
|
||||
|
||||
def test_hidden_backing_app_is_rejected_before_workspace_check(self):
|
||||
view, calls = _guarded_view()
|
||||
binding = SimpleNamespace(scope=AgentScope.WORKFLOW_ONLY)
|
||||
patches = _patch_guard(_app_with_binding(binding), rbac_enabled=True)
|
||||
|
||||
with patches[0], patches[1], patches[2], patch("controllers.console.app.wraps.enforce_rbac_access") as gate:
|
||||
with pytest.raises(AppNotFoundError):
|
||||
view(app_id="app-1")
|
||||
|
||||
gate.assert_not_called()
|
||||
assert calls == []
|
||||
|
||||
def test_binding_lookup_covers_archived_agents(self):
|
||||
"""An Agent App stays gated after its roster Agent is archived."""
|
||||
view, _ = _guarded_view()
|
||||
app_model = _app_with_binding(SimpleNamespace(scope=AgentScope.ROSTER))
|
||||
patches = _patch_guard(app_model, rbac_enabled=True)
|
||||
|
||||
with patches[0], patches[1], patches[2], patch("controllers.console.app.wraps.enforce_rbac_access"):
|
||||
view(app_id="app-1")
|
||||
|
||||
_, call_kwargs = app_model.agent_app_binding_with_session.call_args
|
||||
assert call_kwargs["include_archived"] is True
|
||||
|
||||
def test_resource_id_path_alias_is_resolved(self):
|
||||
view, _ = _guarded_view()
|
||||
binding = SimpleNamespace(scope=AgentScope.ROSTER)
|
||||
patches = _patch_guard(_app_with_binding(binding), rbac_enabled=True)
|
||||
|
||||
with patches[0], patches[1], patches[2], patch("controllers.console.app.wraps.enforce_rbac_access") as gate:
|
||||
assert view(resource_id="app-1") == "ok"
|
||||
|
||||
gate.assert_called_once()
|
||||
|
||||
def test_unknown_app_passes_through_for_downstream_handling(self):
|
||||
view, calls = _guarded_view()
|
||||
patches = _patch_guard(None, rbac_enabled=True)
|
||||
|
||||
with patches[0], patches[1], patches[2], patch("controllers.console.app.wraps.enforce_rbac_access") as gate:
|
||||
assert view(app_id="app-1") == "ok"
|
||||
|
||||
gate.assert_not_called()
|
||||
assert calls == [{"app_id": "app-1"}]
|
||||
@@ -9,13 +9,14 @@ from unittest.mock import MagicMock
|
||||
|
||||
import pytest
|
||||
from flask import Flask
|
||||
from sqlalchemy import event
|
||||
from sqlalchemy import Engine, event
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from controllers.console.app import app_import as app_import_module
|
||||
from models.account import Account
|
||||
from models.base import TypeBase
|
||||
from models.engine import db
|
||||
from models.model import App
|
||||
from models.model import App, AppMode
|
||||
from services.app_dsl_service import ImportStatus
|
||||
from services.entities.dsl_entities import CheckDependenciesResult
|
||||
from services.feature_service import SystemFeatureModel, WebAppAuthModel
|
||||
@@ -66,6 +67,13 @@ def app() -> Iterator[Flask]:
|
||||
yield app
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def sqlite_app_engine(app: Flask) -> Engine:
|
||||
engine = db.engine
|
||||
TypeBase.metadata.create_all(engine, tables=[TypeBase.metadata.tables[App.__tablename__]])
|
||||
return engine
|
||||
|
||||
|
||||
@dataclass
|
||||
class TransactionEvents:
|
||||
commits: int = 0
|
||||
@@ -93,11 +101,34 @@ def transaction_events() -> TransactionEvents:
|
||||
event.remove(Session, "after_rollback", record_rollback)
|
||||
|
||||
|
||||
def _failed_result_after_starting_transaction(
|
||||
service: app_import_module.AppDslService, *, app_id: str | None = None
|
||||
) -> _Result:
|
||||
service._session.begin()
|
||||
return _Result(ImportStatus.FAILED, app_id=app_id)
|
||||
def _install_persisting_service_result(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
*,
|
||||
method_name: str,
|
||||
result: _Result,
|
||||
) -> str:
|
||||
app_id = result.app_id or "rolled-back-app"
|
||||
|
||||
def _return_result(import_service: app_import_module.AppDslService, *_args, **_kwargs):
|
||||
import_service._session.add(
|
||||
App(
|
||||
id=app_id,
|
||||
tenant_id="tenant-1",
|
||||
name="Imported App",
|
||||
mode=AppMode.WORKFLOW,
|
||||
enable_site=True,
|
||||
enable_api=True,
|
||||
)
|
||||
)
|
||||
return result
|
||||
|
||||
monkeypatch.setattr(app_import_module.AppDslService, method_name, _return_result)
|
||||
return app_id
|
||||
|
||||
|
||||
def _assert_app_persistence(sqlite_app_engine: Engine, app_id: str, *, persisted: bool) -> None:
|
||||
with Session(sqlite_app_engine) as session:
|
||||
assert (session.get(App, app_id) is not None) is persisted
|
||||
|
||||
|
||||
class TestAppImportApi:
|
||||
@@ -110,15 +141,16 @@ class TestAppImportApi:
|
||||
api,
|
||||
app: Flask,
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
sqlite_app_engine: Engine,
|
||||
transaction_events: TransactionEvents,
|
||||
) -> None:
|
||||
method = unwrap(api.post)
|
||||
|
||||
_install_features(monkeypatch, enabled=False)
|
||||
monkeypatch.setattr(
|
||||
app_import_module.AppDslService,
|
||||
"import_app",
|
||||
lambda service, *_args, **_kwargs: _failed_result_after_starting_transaction(service, app_id=None),
|
||||
app_id = _install_persisting_service_result(
|
||||
monkeypatch,
|
||||
method_name="import_app",
|
||||
result=_Result(ImportStatus.FAILED, app_id=None),
|
||||
)
|
||||
|
||||
with app.test_request_context("/console/api/apps/imports", method="POST", json={"mode": "yaml-content"}):
|
||||
@@ -126,6 +158,7 @@ class TestAppImportApi:
|
||||
|
||||
assert transaction_events.rollbacks == 1
|
||||
assert transaction_events.commits == 0
|
||||
_assert_app_persistence(sqlite_app_engine, app_id, persisted=False)
|
||||
assert status == 400
|
||||
assert response["status"] == ImportStatus.FAILED
|
||||
|
||||
@@ -134,15 +167,16 @@ class TestAppImportApi:
|
||||
api,
|
||||
app: Flask,
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
sqlite_app_engine: Engine,
|
||||
transaction_events: TransactionEvents,
|
||||
) -> None:
|
||||
method = unwrap(api.post)
|
||||
|
||||
_install_features(monkeypatch, enabled=False)
|
||||
monkeypatch.setattr(
|
||||
app_import_module.AppDslService,
|
||||
"import_app",
|
||||
lambda *_args, **_kwargs: _Result(ImportStatus.PENDING),
|
||||
app_id = _install_persisting_service_result(
|
||||
monkeypatch,
|
||||
method_name="import_app",
|
||||
result=_Result(ImportStatus.PENDING),
|
||||
)
|
||||
|
||||
with app.test_request_context("/console/api/apps/imports", method="POST", json={"mode": "yaml-content"}):
|
||||
@@ -150,6 +184,7 @@ class TestAppImportApi:
|
||||
|
||||
assert transaction_events.commits == 1
|
||||
assert transaction_events.rollbacks == 0
|
||||
_assert_app_persistence(sqlite_app_engine, app_id, persisted=True)
|
||||
assert status == 202
|
||||
assert response["status"] == ImportStatus.PENDING
|
||||
|
||||
@@ -158,15 +193,16 @@ class TestAppImportApi:
|
||||
api,
|
||||
app: Flask,
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
sqlite_app_engine: Engine,
|
||||
transaction_events: TransactionEvents,
|
||||
) -> None:
|
||||
method = unwrap(api.post)
|
||||
|
||||
_install_features(monkeypatch, enabled=True)
|
||||
monkeypatch.setattr(
|
||||
app_import_module.AppDslService,
|
||||
"import_app",
|
||||
lambda *_args, **_kwargs: _Result(ImportStatus.COMPLETED, app_id="app-123"),
|
||||
app_id = _install_persisting_service_result(
|
||||
monkeypatch,
|
||||
method_name="import_app",
|
||||
result=_Result(ImportStatus.COMPLETED, app_id="app-123"),
|
||||
)
|
||||
update_access = MagicMock()
|
||||
monkeypatch.setattr(app_import_module.EnterpriseService.WebAppAuth, "update_app_access_mode", update_access)
|
||||
@@ -176,6 +212,7 @@ class TestAppImportApi:
|
||||
|
||||
assert transaction_events.commits == 1
|
||||
assert transaction_events.rollbacks == 0
|
||||
_assert_app_persistence(sqlite_app_engine, app_id, persisted=True)
|
||||
update_access.assert_called_once_with("app-123", "private")
|
||||
assert status == 200
|
||||
assert response["status"] == ImportStatus.COMPLETED
|
||||
@@ -185,6 +222,7 @@ class TestAppImportApi:
|
||||
api,
|
||||
app: Flask,
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
sqlite_app_engine: Engine,
|
||||
transaction_events: TransactionEvents,
|
||||
) -> None:
|
||||
method = _unwrap(api.post)
|
||||
@@ -196,10 +234,10 @@ class TestAppImportApi:
|
||||
lambda: (_make_account(), "tenant-1"),
|
||||
)
|
||||
monkeypatch.setattr(app_import_module.dify_config, "RBAC_ENABLED", True)
|
||||
monkeypatch.setattr(
|
||||
app_import_module.AppDslService,
|
||||
"import_app",
|
||||
lambda *_args, **_kwargs: _Result(ImportStatus.COMPLETED, app_id="app-123"),
|
||||
app_id = _install_persisting_service_result(
|
||||
monkeypatch,
|
||||
method_name="import_app",
|
||||
result=_Result(ImportStatus.COMPLETED, app_id="app-123"),
|
||||
)
|
||||
monkeypatch.setattr(
|
||||
app_import_module,
|
||||
@@ -211,6 +249,7 @@ class TestAppImportApi:
|
||||
response, status = method()
|
||||
|
||||
assert transaction_events.commits == 1
|
||||
_assert_app_persistence(sqlite_app_engine, app_id, persisted=True)
|
||||
assert status == 200
|
||||
assert response["permission_keys"] == ["app.acl.view_layout", "app.acl.edit"]
|
||||
|
||||
@@ -219,6 +258,7 @@ class TestAppImportApi:
|
||||
api,
|
||||
app: Flask,
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
sqlite_app_engine: Engine,
|
||||
transaction_events: TransactionEvents,
|
||||
) -> None:
|
||||
method = _unwrap(api.post)
|
||||
@@ -230,10 +270,10 @@ class TestAppImportApi:
|
||||
lambda: (_make_account(), "tenant-1"),
|
||||
)
|
||||
monkeypatch.setattr(app_import_module.dify_config, "RBAC_ENABLED", True)
|
||||
monkeypatch.setattr(
|
||||
app_import_module.AppDslService,
|
||||
"import_app",
|
||||
lambda *_args, **_kwargs: _Result(ImportStatus.COMPLETED, app_id="app-123"),
|
||||
app_id = _install_persisting_service_result(
|
||||
monkeypatch,
|
||||
method_name="import_app",
|
||||
result=_Result(ImportStatus.COMPLETED, app_id="app-123"),
|
||||
)
|
||||
monkeypatch.setattr(
|
||||
app_import_module,
|
||||
@@ -249,6 +289,7 @@ class TestAppImportApi:
|
||||
response, status = method()
|
||||
|
||||
assert transaction_events.commits == 1
|
||||
_assert_app_persistence(sqlite_app_engine, app_id, persisted=True)
|
||||
assert status == 200
|
||||
assert response["permission_keys"] == []
|
||||
|
||||
@@ -263,14 +304,15 @@ class TestAppImportConfirmApi:
|
||||
api,
|
||||
app: Flask,
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
sqlite_app_engine: Engine,
|
||||
transaction_events: TransactionEvents,
|
||||
) -> None:
|
||||
method = unwrap(api.post)
|
||||
|
||||
monkeypatch.setattr(
|
||||
app_import_module.AppDslService,
|
||||
"confirm_import",
|
||||
lambda service, *_args, **_kwargs: _failed_result_after_starting_transaction(service),
|
||||
app_id = _install_persisting_service_result(
|
||||
monkeypatch,
|
||||
method_name="confirm_import",
|
||||
result=_Result(ImportStatus.FAILED),
|
||||
)
|
||||
|
||||
with app.test_request_context("/console/api/apps/imports/import-1/confirm", method="POST"):
|
||||
@@ -278,6 +320,7 @@ class TestAppImportConfirmApi:
|
||||
|
||||
assert transaction_events.rollbacks == 1
|
||||
assert transaction_events.commits == 0
|
||||
_assert_app_persistence(sqlite_app_engine, app_id, persisted=False)
|
||||
assert status == 400
|
||||
assert response["status"] == ImportStatus.FAILED
|
||||
|
||||
@@ -286,6 +329,7 @@ class TestAppImportConfirmApi:
|
||||
api,
|
||||
app: Flask,
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
sqlite_app_engine: Engine,
|
||||
transaction_events: TransactionEvents,
|
||||
) -> None:
|
||||
method = _unwrap(api.post)
|
||||
@@ -304,10 +348,10 @@ class TestAppImportConfirmApi:
|
||||
),
|
||||
)
|
||||
monkeypatch.setattr(app_import_module.dify_config, "RBAC_ENABLED", True)
|
||||
monkeypatch.setattr(
|
||||
app_import_module.AppDslService,
|
||||
"confirm_import",
|
||||
lambda *_args, **_kwargs: _Result(ImportStatus.COMPLETED, app_id="app-456"),
|
||||
app_id = _install_persisting_service_result(
|
||||
monkeypatch,
|
||||
method_name="confirm_import",
|
||||
result=_Result(ImportStatus.COMPLETED, app_id="app-456"),
|
||||
)
|
||||
monkeypatch.setattr(
|
||||
app_import_module,
|
||||
@@ -319,6 +363,7 @@ class TestAppImportConfirmApi:
|
||||
response, status = method(import_id="import-1")
|
||||
|
||||
assert transaction_events.commits == 1
|
||||
_assert_app_persistence(sqlite_app_engine, app_id, persisted=True)
|
||||
assert status == 200
|
||||
assert response["permission_keys"] == ["app.acl.view_layout", "app.acl.edit"]
|
||||
|
||||
@@ -327,6 +372,7 @@ class TestAppImportConfirmApi:
|
||||
api,
|
||||
app: Flask,
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
sqlite_app_engine: Engine,
|
||||
transaction_events: TransactionEvents,
|
||||
) -> None:
|
||||
method = _unwrap(api.post)
|
||||
@@ -345,10 +391,10 @@ class TestAppImportConfirmApi:
|
||||
),
|
||||
)
|
||||
monkeypatch.setattr(app_import_module.dify_config, "RBAC_ENABLED", True)
|
||||
monkeypatch.setattr(
|
||||
app_import_module.AppDslService,
|
||||
"confirm_import",
|
||||
lambda *_args, **_kwargs: _Result(ImportStatus.COMPLETED, app_id="app-456"),
|
||||
app_id = _install_persisting_service_result(
|
||||
monkeypatch,
|
||||
method_name="confirm_import",
|
||||
result=_Result(ImportStatus.COMPLETED, app_id="app-456"),
|
||||
)
|
||||
monkeypatch.setattr(
|
||||
app_import_module,
|
||||
@@ -360,6 +406,7 @@ class TestAppImportConfirmApi:
|
||||
response, status = method(import_id="import-1")
|
||||
|
||||
assert transaction_events.commits == 1
|
||||
_assert_app_persistence(sqlite_app_engine, app_id, persisted=True)
|
||||
assert status == 200
|
||||
assert response["permission_keys"] == []
|
||||
|
||||
|
||||
@@ -1,5 +1,6 @@
|
||||
import pytest
|
||||
from flask import Flask
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from controllers.console.app import generator as generator_module
|
||||
from core.errors.error import ModelCurrentlyNotSupportError, ProviderTokenNotInitError, QuotaExceededError
|
||||
@@ -102,12 +103,14 @@ def test_structured_output_generate_exceptions(app: Flask, monkeypatch: pytest.M
|
||||
method(api, "t1")
|
||||
|
||||
|
||||
def test_instruction_generate_exceptions(app: Flask, monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
@pytest.mark.parametrize("sqlite_session", [()], indirect=True)
|
||||
def test_instruction_generate_exceptions(
|
||||
app: Flask,
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
sqlite_session: Session,
|
||||
) -> None:
|
||||
api = generator_module.InstructionGenerateApi()
|
||||
method = unwrap(api.post)
|
||||
from types import SimpleNamespace
|
||||
|
||||
session = SimpleNamespace()
|
||||
|
||||
exceptions_to_test = [
|
||||
(ProviderTokenNotInitError("token error"), generator_module.ProviderNotInitializeError),
|
||||
@@ -135,4 +138,4 @@ def test_instruction_generate_exceptions(app: Flask, monkeypatch: pytest.MonkeyP
|
||||
},
|
||||
):
|
||||
with pytest.raises(expected_exception):
|
||||
method(api, session, "t1")
|
||||
method(api, sqlite_session, "t1")
|
||||
|
||||
@@ -11,7 +11,7 @@ import pytest
|
||||
from flask import Flask
|
||||
from sqlalchemy import func, select
|
||||
from sqlalchemy.engine import Engine
|
||||
from sqlalchemy.orm import object_session, sessionmaker
|
||||
from sqlalchemy.orm import Session, object_session, sessionmaker
|
||||
|
||||
from controllers.common import session as controller_session
|
||||
from controllers.console.app import model_config as model_config_module
|
||||
@@ -29,11 +29,14 @@ def _poison_implicit_app_config_properties(monkeypatch: pytest.MonkeyPatch) -> N
|
||||
|
||||
|
||||
@pytest.mark.parametrize("app_mode", [AppMode.CHAT, AppMode.COMPLETION])
|
||||
@pytest.mark.parametrize("sqlite_session", [(AppModelConfig,)], indirect=True)
|
||||
def test_post_updates_non_agent_model_config_without_implicit_properties(
|
||||
app: Flask,
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
app_mode: AppMode,
|
||||
sqlite_session: Session,
|
||||
) -> None:
|
||||
"""Flush a non-agent config through the injected session without legacy model properties."""
|
||||
api = model_config_module.ModelConfigResource()
|
||||
method = unwrap(api.post)
|
||||
|
||||
@@ -45,14 +48,16 @@ def test_post_updates_non_agent_model_config_without_implicit_properties(
|
||||
updated_at=None,
|
||||
)
|
||||
original_config = AppModelConfig(app_id="app-1", created_by="u1", updated_by="u1")
|
||||
original_config.id = "config-0"
|
||||
original_config.agent_mode = None
|
||||
sqlite_session.add(original_config)
|
||||
sqlite_session.commit()
|
||||
_poison_implicit_app_config_properties(monkeypatch)
|
||||
monkeypatch.setattr(
|
||||
model_config_module.AppModelConfigService,
|
||||
"validate_configuration",
|
||||
lambda **_kwargs: {"pre_prompt": "hi"},
|
||||
)
|
||||
session = MagicMock()
|
||||
|
||||
def _from_model_config_dict(self, model_config):
|
||||
self.pre_prompt = model_config["pre_prompt"]
|
||||
@@ -62,18 +67,16 @@ def test_post_updates_non_agent_model_config_without_implicit_properties(
|
||||
monkeypatch.setattr(AppModelConfig, "from_model_config_dict", _from_model_config_dict)
|
||||
send_mock = MagicMock()
|
||||
monkeypatch.setattr(model_config_module.app_model_config_was_updated, "send", send_mock)
|
||||
session.get.return_value = original_config
|
||||
|
||||
with app.test_request_context("/console/api/apps/app-1/model-config", method="POST", json={"pre_prompt": "hi"}):
|
||||
response = method(api, session, "t1", "u1", app_model=app_model)
|
||||
response = method(api, sqlite_session, "t1", "u1", app_model=app_model)
|
||||
|
||||
session.get.assert_called_once_with(AppModelConfig, "config-0")
|
||||
session.add.assert_called_once()
|
||||
session.flush.assert_called_once()
|
||||
session.commit.assert_not_called()
|
||||
assert send_mock.call_args.kwargs["session"] is session
|
||||
assert send_mock.call_args.kwargs["session"] is sqlite_session
|
||||
assert app_model.app_model_config_id == "config-1"
|
||||
assert app_model.mode == app_mode
|
||||
persisted_config = sqlite_session.get(AppModelConfig, "config-1")
|
||||
assert persisted_config is not None
|
||||
assert persisted_config.pre_prompt == "hi"
|
||||
assert response["result"] == "success"
|
||||
|
||||
|
||||
@@ -160,7 +163,11 @@ def test_post_uses_one_session_and_rolls_back_when_signal_fails(
|
||||
assert config_count == 1
|
||||
|
||||
|
||||
def test_post_encrypts_agent_tool_parameters(app: Flask, monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
@pytest.mark.parametrize("sqlite_session", [(AppModelConfig,)], indirect=True)
|
||||
def test_post_encrypts_agent_tool_parameters(
|
||||
app: Flask, monkeypatch: pytest.MonkeyPatch, sqlite_session: Session
|
||||
) -> None:
|
||||
"""Agent parameter encryption reads and writes persisted model configurations."""
|
||||
api = model_config_module.ModelConfigResource()
|
||||
method = unwrap(api.post)
|
||||
|
||||
@@ -174,6 +181,7 @@ def test_post_encrypts_agent_tool_parameters(app: Flask, monkeypatch: pytest.Mon
|
||||
_poison_implicit_app_config_properties(monkeypatch)
|
||||
|
||||
original_config = AppModelConfig(app_id="app-1", created_by="u1", updated_by="u1")
|
||||
original_config.id = "config-0"
|
||||
original_config.agent_mode = json.dumps(
|
||||
{
|
||||
"enabled": True,
|
||||
@@ -190,8 +198,8 @@ def test_post_encrypts_agent_tool_parameters(app: Flask, monkeypatch: pytest.Mon
|
||||
}
|
||||
)
|
||||
|
||||
session = MagicMock()
|
||||
session.scalar.return_value = original_config
|
||||
sqlite_session.add(original_config)
|
||||
sqlite_session.commit()
|
||||
|
||||
monkeypatch.setattr(
|
||||
model_config_module.AppModelConfigService,
|
||||
@@ -236,11 +244,11 @@ def test_post_encrypts_agent_tool_parameters(app: Flask, monkeypatch: pytest.Mon
|
||||
monkeypatch.setattr(model_config_module.app_model_config_was_updated, "send", send_mock)
|
||||
|
||||
with app.test_request_context("/console/api/apps/app-1/model-config", method="POST", json={"pre_prompt": "hi"}):
|
||||
response = method(api, session, "t1", "u1", app_model=app_model)
|
||||
response = method(api, sqlite_session, "t1", "u1", app_model=app_model)
|
||||
|
||||
stored_config = session.add.call_args[0][0]
|
||||
stored_config = sqlite_session.get(AppModelConfig, app_model.app_model_config_id)
|
||||
assert stored_config is not None
|
||||
stored_agent_mode = json.loads(stored_config.agent_mode)
|
||||
session.scalar.assert_called_once()
|
||||
assert app_model.mode == AppMode.AGENT_CHAT
|
||||
assert stored_agent_mode["tools"][0]["tool_parameters"]["secret"] == "encrypted"
|
||||
assert response["result"] == "success"
|
||||
|
||||
@@ -14,6 +14,7 @@ import pytest
|
||||
from flask import Flask
|
||||
|
||||
from controllers.console.auth.activate import ActivateApi, ActivateCheckApi
|
||||
from controllers.console.auth.error import InvitationAccountMismatchError
|
||||
from controllers.console.error import AccountInFreezeError, AlreadyActivateError
|
||||
from models.account import AccountStatus, TenantAccountRole
|
||||
|
||||
@@ -202,6 +203,50 @@ class TestActivateApi:
|
||||
with patch("controllers.console.auth.activate.TenantService.switch_tenant") as mock:
|
||||
yield mock
|
||||
|
||||
@patch("controllers.console.auth.activate.TenantService.create_tenant_member")
|
||||
@patch("controllers.console.auth.activate.RegisterService.get_invitation_with_case_fallback")
|
||||
@patch("controllers.console.auth.activate.RegisterService.revoke_token")
|
||||
@patch("controllers.console.auth.activate.current_account_with_tenant")
|
||||
@patch("controllers.console.auth.activate.extract_access_token", return_value="access-token")
|
||||
@patch("controllers.console.auth.activate.db")
|
||||
def test_activation_rejects_invitation_for_different_authenticated_account(
|
||||
self,
|
||||
mock_db: MagicMock,
|
||||
mock_extract_access_token: MagicMock,
|
||||
mock_current_account_with_tenant: MagicMock,
|
||||
mock_revoke_token: MagicMock,
|
||||
mock_get_invitation: MagicMock,
|
||||
mock_create_tenant_member: MagicMock,
|
||||
app: Flask,
|
||||
mock_invitation: MagicMock,
|
||||
mock_account: MagicMock,
|
||||
mock_switch_tenant: MagicMock,
|
||||
):
|
||||
"""A logged-in account cannot consume another account's invitation token."""
|
||||
current_account = MagicMock()
|
||||
current_account.id = "current-account-id"
|
||||
mock_account.id = "invited-account-id"
|
||||
mock_account.status = AccountStatus.ACTIVE
|
||||
mock_invitation["data"]["requires_setup"] = False
|
||||
mock_get_invitation.return_value = mock_invitation
|
||||
mock_current_account_with_tenant.return_value = (current_account, "current-workspace-id")
|
||||
|
||||
with app.test_request_context(
|
||||
"/activate",
|
||||
method="POST",
|
||||
json={
|
||||
"token": "valid_token",
|
||||
},
|
||||
):
|
||||
with pytest.raises(InvitationAccountMismatchError):
|
||||
ActivateApi().post()
|
||||
|
||||
mock_extract_access_token.assert_called_once()
|
||||
mock_revoke_token.assert_not_called()
|
||||
mock_create_tenant_member.assert_not_called()
|
||||
mock_switch_tenant.assert_not_called()
|
||||
mock_db.session.scalar.assert_not_called()
|
||||
|
||||
@patch("controllers.console.auth.activate.RegisterService.get_invitation_if_token_valid")
|
||||
@patch("controllers.console.auth.activate.RegisterService.revoke_token")
|
||||
@patch("controllers.console.auth.activate.db")
|
||||
|
||||
+26
-48
@@ -1,4 +1,4 @@
|
||||
"""Testcontainers integration tests for OAuth controller endpoints."""
|
||||
"""Unit tests for OAuth controller endpoints."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
@@ -16,15 +16,10 @@ from controllers.console.auth.oauth import (
|
||||
)
|
||||
from libs.oauth import OAuthUserInfo, encode_oauth_state
|
||||
from models.account import AccountStatus
|
||||
from services.account_service import AccountService
|
||||
from services.errors.account import AccountRegisterError
|
||||
|
||||
|
||||
class TestGetOAuthProviders:
|
||||
@pytest.fixture
|
||||
def app(self, flask_app_with_containers: Flask):
|
||||
return flask_app_with_containers
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("github_config", "google_config", "expected_github", "expected_google"),
|
||||
[
|
||||
@@ -65,10 +60,6 @@ class TestOAuthLogin:
|
||||
def resource(self):
|
||||
return OAuthLogin()
|
||||
|
||||
@pytest.fixture
|
||||
def app(self, flask_app_with_containers: Flask):
|
||||
return flask_app_with_containers
|
||||
|
||||
@pytest.fixture
|
||||
def mock_oauth_provider(self):
|
||||
provider = MagicMock()
|
||||
@@ -181,10 +172,6 @@ class TestOAuthCallback:
|
||||
def resource(self):
|
||||
return OAuthCallback()
|
||||
|
||||
@pytest.fixture
|
||||
def app(self, flask_app_with_containers: Flask):
|
||||
return flask_app_with_containers
|
||||
|
||||
@pytest.fixture
|
||||
def oauth_setup(self):
|
||||
"""Common OAuth setup for callback tests"""
|
||||
@@ -263,10 +250,12 @@ class TestOAuthCallback:
|
||||
@patch("controllers.console.auth.oauth.dify_config")
|
||||
@patch("controllers.console.auth.oauth.get_oauth_providers")
|
||||
@patch("controllers.console.auth.oauth.RegisterService")
|
||||
@patch("controllers.console.auth.oauth.AccountService")
|
||||
@patch("controllers.console.auth.oauth.redirect")
|
||||
def test_invitation_comparison_is_case_insensitive(
|
||||
self,
|
||||
mock_redirect,
|
||||
mock_account_service,
|
||||
mock_register_service,
|
||||
mock_get_providers,
|
||||
mock_config,
|
||||
@@ -280,13 +269,20 @@ class TestOAuthCallback:
|
||||
)
|
||||
mock_get_providers.return_value = {"github": oauth_setup["provider"]}
|
||||
mock_register_service.is_valid_invite_token.return_value = True
|
||||
mock_register_service.get_invitation_by_token.return_value = {"email": "user@example.com"}
|
||||
mock_register_service.get_invitation_if_token_valid.return_value = {
|
||||
"account": oauth_setup["account"],
|
||||
"data": {"email": "user@example.com"},
|
||||
"tenant": MagicMock(),
|
||||
}
|
||||
mock_account_service.login.return_value = oauth_setup["token_pair"]
|
||||
|
||||
state = encode_oauth_state(invite_token="invite123", timezone="Asia/Shanghai")
|
||||
with app.test_request_context(f"/auth/oauth/github/callback?code=test_code&state={state}"):
|
||||
resource.get("github")
|
||||
|
||||
mock_register_service.get_invitation_by_token.assert_called_once_with(token="invite123")
|
||||
mock_register_service.get_invitation_if_token_valid.assert_called_once_with(
|
||||
None, None, "invite123", session=ANY
|
||||
)
|
||||
mock_redirect.assert_called_once_with("http://localhost:3000/signin/invite-settings?invite_token=invite123")
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
@@ -448,10 +444,6 @@ class TestOAuthCallback:
|
||||
|
||||
|
||||
class TestAccountGeneration:
|
||||
@pytest.fixture
|
||||
def app(self, flask_app_with_containers: Flask):
|
||||
return flask_app_with_containers
|
||||
|
||||
@pytest.fixture
|
||||
def user_info(self):
|
||||
return OAuthUserInfo(id="123", name="Test User", email="test@example.com")
|
||||
@@ -468,39 +460,25 @@ class TestAccountGeneration:
|
||||
self,
|
||||
mock_account_model,
|
||||
mock_get_account,
|
||||
flask_req_ctx_with_containers,
|
||||
app: Flask,
|
||||
user_info: OAuthUserInfo,
|
||||
mock_account,
|
||||
):
|
||||
# Test OpenID found
|
||||
mock_account_model.get_by_openid.return_value = mock_account
|
||||
result = _get_account_by_openid_or_email("github", user_info)
|
||||
assert result == mock_account
|
||||
mock_account_model.get_by_openid.assert_called_once_with("github", "123")
|
||||
mock_get_account.assert_not_called()
|
||||
with app.test_request_context("/"):
|
||||
# Test OpenID found
|
||||
mock_account_model.get_by_openid.return_value = mock_account
|
||||
result = _get_account_by_openid_or_email("github", user_info)
|
||||
assert result == mock_account
|
||||
mock_account_model.get_by_openid.assert_called_once_with("github", "123")
|
||||
mock_get_account.assert_not_called()
|
||||
|
||||
# Test fallback to email lookup
|
||||
mock_account_model.get_by_openid.return_value = None
|
||||
mock_get_account.return_value = mock_account
|
||||
# Test fallback to email lookup
|
||||
mock_account_model.get_by_openid.return_value = None
|
||||
mock_get_account.return_value = mock_account
|
||||
|
||||
result = _get_account_by_openid_or_email("github", user_info)
|
||||
assert result == mock_account
|
||||
mock_get_account.assert_called_once()
|
||||
|
||||
def test_get_account_by_email_with_case_fallback_falls_back_to_lowercase(self):
|
||||
"""Test that case fallback tries lowercase when exact match fails."""
|
||||
mock_session = MagicMock()
|
||||
first_result = MagicMock()
|
||||
first_result.scalar_one_or_none.return_value = None
|
||||
expected_account = MagicMock()
|
||||
second_result = MagicMock()
|
||||
second_result.scalar_one_or_none.return_value = expected_account
|
||||
mock_session.execute.side_effect = [first_result, second_result]
|
||||
|
||||
result = AccountService.get_account_by_email_with_case_fallback("Case@Test.com", session=mock_session)
|
||||
|
||||
assert result is expected_account
|
||||
assert mock_session.execute.call_count == 2
|
||||
result = _get_account_by_openid_or_email("github", user_info)
|
||||
assert result == mock_account
|
||||
mock_get_account.assert_called_once()
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("allow_register", "existing_account", "should_create"),
|
||||
@@ -1,5 +1,5 @@
|
||||
import urllib.parse
|
||||
from unittest.mock import MagicMock, patch
|
||||
from unittest.mock import ANY, MagicMock, patch
|
||||
|
||||
import pytest
|
||||
from flask import Flask
|
||||
@@ -91,3 +91,102 @@ def test_oauth_callback_validates_redirect_url_and_appends_new_user_flag(
|
||||
assert response.headers["Location"] == (
|
||||
f"{expected_target_url}{query_char}oauth_new_user={str(oauth_new_user).lower()}"
|
||||
)
|
||||
|
||||
|
||||
def test_oauth_callback_with_invitation_establishes_console_session(app: Flask) -> None:
|
||||
oauth_provider = MagicMock()
|
||||
oauth_provider.get_access_token.return_value = "google-access-token"
|
||||
oauth_provider.get_user_info.return_value = OAuthUserInfo(
|
||||
id="google-user-123",
|
||||
name="Test User",
|
||||
email="Invitee@Example.com",
|
||||
)
|
||||
account = MagicMock()
|
||||
account.status = AccountStatus.ACTIVE
|
||||
token_pair = MagicMock()
|
||||
token_pair.access_token = "dify-access-token"
|
||||
token_pair.refresh_token = "dify-refresh-token"
|
||||
token_pair.csrf_token = "dify-csrf-token"
|
||||
state = encode_oauth_state(invite_token="invite-token")
|
||||
|
||||
with (
|
||||
patch("controllers.console.auth.oauth.get_oauth_providers", return_value={"google": oauth_provider}),
|
||||
patch("controllers.console.auth.oauth.dify_config.CONSOLE_WEB_URL", CONSOLE_WEB_URL),
|
||||
patch("controllers.console.auth.oauth.RegisterService") as register_service,
|
||||
patch("controllers.console.auth.oauth.AccountService.link_account_integrate") as link_account,
|
||||
patch("controllers.console.auth.oauth.AccountService.login", return_value=token_pair) as login,
|
||||
patch("controllers.console.auth.oauth.TenantService.create_owner_tenant_if_not_exist") as create_workspace,
|
||||
patch("controllers.console.auth.oauth.set_access_token_to_cookie") as set_access_cookie,
|
||||
patch("controllers.console.auth.oauth.set_refresh_token_to_cookie") as set_refresh_cookie,
|
||||
patch("controllers.console.auth.oauth.set_csrf_token_to_cookie") as set_csrf_cookie,
|
||||
app.test_request_context(f"/oauth/authorize/google?code=test-code&state={state}"),
|
||||
):
|
||||
register_service.is_valid_invite_token.return_value = True
|
||||
register_service.get_invitation_if_token_valid.return_value = {
|
||||
"account": account,
|
||||
"data": {
|
||||
"account_id": "account-id",
|
||||
"email": "invitee@example.com",
|
||||
"workspace_id": "workspace-id",
|
||||
},
|
||||
"tenant": MagicMock(),
|
||||
}
|
||||
|
||||
response = OAuthCallback().get("google")
|
||||
|
||||
assert response.status_code == 302
|
||||
assert response.headers["Location"] == (f"{CONSOLE_WEB_URL}/signin/invite-settings?invite_token=invite-token")
|
||||
link_account.assert_called_once_with("google", "google-user-123", account, session=ANY)
|
||||
login.assert_called_once_with(account=account, session=ANY, ip_address=ANY)
|
||||
create_workspace.assert_not_called()
|
||||
set_access_cookie.assert_called_once_with(ANY, response, "dify-access-token")
|
||||
set_refresh_cookie.assert_called_once_with(ANY, response, "dify-refresh-token")
|
||||
set_csrf_cookie.assert_called_once_with(ANY, response, "dify-csrf-token")
|
||||
|
||||
|
||||
def test_oauth_callback_with_invitation_rejects_another_account(app: Flask) -> None:
|
||||
oauth_provider = MagicMock()
|
||||
oauth_provider.get_access_token.return_value = "google-access-token"
|
||||
oauth_provider.get_user_info.return_value = OAuthUserInfo(
|
||||
id="google-user-123",
|
||||
name="Test User",
|
||||
email="another@example.com",
|
||||
)
|
||||
account = MagicMock()
|
||||
account.status = AccountStatus.ACTIVE
|
||||
state = encode_oauth_state(invite_token="invite-token")
|
||||
|
||||
with (
|
||||
patch("controllers.console.auth.oauth.get_oauth_providers", return_value={"google": oauth_provider}),
|
||||
patch("controllers.console.auth.oauth.dify_config.CONSOLE_WEB_URL", CONSOLE_WEB_URL),
|
||||
patch("controllers.console.auth.oauth.RegisterService") as register_service,
|
||||
patch("controllers.console.auth.oauth.AccountService.link_account_integrate") as link_account,
|
||||
patch("controllers.console.auth.oauth.AccountService.login") as login,
|
||||
patch("controllers.console.auth.oauth.set_access_token_to_cookie") as set_access_cookie,
|
||||
patch("controllers.console.auth.oauth.set_refresh_token_to_cookie") as set_refresh_cookie,
|
||||
patch("controllers.console.auth.oauth.set_csrf_token_to_cookie") as set_csrf_cookie,
|
||||
app.test_request_context(f"/oauth/authorize/google?code=test-code&state={state}"),
|
||||
):
|
||||
register_service.is_valid_invite_token.return_value = True
|
||||
register_service.get_invitation_if_token_valid.return_value = {
|
||||
"account": account,
|
||||
"data": {
|
||||
"account_id": "account-id",
|
||||
"email": "invitee@example.com",
|
||||
"workspace_id": "workspace-id",
|
||||
},
|
||||
"tenant": MagicMock(),
|
||||
}
|
||||
|
||||
response = OAuthCallback().get("google")
|
||||
|
||||
query = urllib.parse.parse_qs(urllib.parse.urlparse(response.headers["Location"]).query)
|
||||
assert response.status_code == 302
|
||||
assert query["message"] == ["This invitation was sent to another account. Please sign in with the invited account."]
|
||||
assert query["invite_token"] == ["invite-token"]
|
||||
link_account.assert_not_called()
|
||||
login.assert_not_called()
|
||||
register_service.revoke_token.assert_not_called()
|
||||
set_access_cookie.assert_not_called()
|
||||
set_refresh_cookie.assert_not_called()
|
||||
set_csrf_cookie.assert_not_called()
|
||||
|
||||
@@ -3,7 +3,7 @@ from __future__ import annotations
|
||||
from inspect import unwrap
|
||||
from unittest.mock import patch
|
||||
|
||||
from controllers.console.auth.oauth_server import OAuthServerUserAuthorizeApi
|
||||
from controllers.console.auth.oauth_server import OAuthServerUserAccountApi, OAuthServerUserAuthorizeApi
|
||||
from models import Account
|
||||
from models.account import AccountStatus, TenantAccountRole
|
||||
from models.model import OAuthProviderApp
|
||||
@@ -45,3 +45,13 @@ def test_oauth_authorize_uses_injected_current_user() -> None:
|
||||
|
||||
sign_oauth_authorization_code.assert_called_once_with("client-1", "account-1")
|
||||
assert response == {"code": "authorization-code"}
|
||||
|
||||
|
||||
def test_oauth_account_returns_stable_account_id() -> None:
|
||||
api = OAuthServerUserAccountApi()
|
||||
method = unwrap(api.post)
|
||||
account = _make_account()
|
||||
|
||||
response = method(api, _make_oauth_provider_app(), account)
|
||||
|
||||
assert response["id"] == "account-1"
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user