Compare commits

..
Author SHA1 Message Date
zhangx1n ee431ef0ae feat(enterprise): reject member invites that would exceed licensed seats
Batch invites only enforced seats deep inside create_account, per invitee
against a cached count, so a single batch could overshoot the cap. Count
seat-consuming invitees (emails with no existing account) upfront and
reject the whole batch before any account is created, mirroring the
workspace-members admission check. The per-invitee seats catch is folded
into the generic failure handler.
2026-07-15 15:13:24 +08:00
3060 changed files with 146997 additions and 99669 deletions
@@ -32,11 +32,12 @@ Keep this skill focused on Cucumber, Playwright, and package-level E2E guidance.
- `e2e/` uses Cucumber for scenarios and Playwright as the browser layer.
- `DifyWorld` is the per-scenario context object. Type `this` as `DifyWorld` and use `async function`, not arrow functions.
- Keep glue organized by capability under `e2e/features/step-definitions/`; use `common/` only for broadly reusable steps.
- Treat `e2e/AGENTS.md`, `features/support/hooks.ts`, and the Cucumber configuration as the owners of current session and tag semantics. Verify them when behavior depends on session state instead of copying a tag inventory into this skill.
- Browser session behavior comes from `features/support/hooks.ts`:
- default: authenticated session with shared storage state
- `@unauthenticated`: clean browser context
- `@authenticated`: readability/selective-run tag only unless implementation changes
- `@fresh`: only for `e2e:full*` flows
- Do not import Playwright Test runner patterns that bypass the current Cucumber + `DifyWorld` architecture unless the task is explicitly about changing that architecture.
- Perform the behavior under test through Playwright. APIs are allowed for setup, seed preparation, persistence polling, and cleanup, but ordinary Console JSON and representable multipart operations must use the scenario- or process-owned generated oRPC client with request and response validation enabled. Keep the setup/cleanup API identity independent from an unauthenticated or logged-out behavior browser.
- Consume generated operations directly. Do not add one-to-one API wrappers, handwritten endpoint URLs, response DTO casts, duplicate schemas, global mutable clients, or TanStack Query caching in Cucumber. Keep helpers only for real fixture construction, multi-operation orchestration, invariants, polling, derived test views, or protocol adapters.
- Keep SSE, binary, redirect-only, external-service, and readiness exceptions centralized under their protocol owner. A contract mismatch must fail and be fixed at the backend schema owner followed by regeneration; never weaken validation to make E2E pass.
## Workflow
@@ -65,7 +66,7 @@ Keep this skill focused on Cucumber, Playwright, and package-level E2E guidance.
- If a product element has real user-facing semantics but no accessible name, prefer fixing that accessible contract over adding a test id.
5. Validate narrowly.
- Run the narrowest tagged scenario or flow that exercises the change.
- Run the package-required static checks documented in `e2e/AGENTS.md`.
- Run `vpr lint --fix --quiet` from the repository root and `pnpm -C e2e type-check`.
- Broaden verification only when the change affects hooks, tags, setup, or shared step semantics.
## Review Checklist
@@ -76,8 +77,6 @@ Keep this skill focused on Cucumber, Playwright, and package-level E2E guidance.
- Are locators user-facing and assertions web-first?
- Does the change introduce hidden coupling across scenarios, tags, or instance state?
- Does it document or implement behavior that differs from the real hooks or configuration?
- Does setup/cleanup use the generated client directly, with any remaining helper owning more than a one-to-one endpoint forward?
- Is every raw HTTP call a documented protocol or infrastructure exception rather than an ordinary Console operation?
Lead findings with correctness, flake risk, and architecture drift.
+5 -9
View File
@@ -1,15 +1,15 @@
---
name: frontend-testing
description: Write, update, or review Dify frontend tests using Vitest and Testing Library. Trigger for frontend specs, test coverage requests, regressions, testability, or testing strategy under web/ or packages/dify-ui/.
description: Write, update, or review Dify frontend tests using Vitest and React Testing Library. Trigger for frontend specs, test coverage requests, regressions, testability, or testing strategy under web/.
---
# Dify Frontend Testing
Use this skill for Vitest work under `web/` and `packages/dify-ui/`. Do not use it for Python tests or Cucumber/Playwright tests under `e2e/`.
Use this skill for Vitest and React Testing Library work under `web/`. Do not use it for Python tests or Cucumber/Playwright tests under `e2e/`.
## Required Source
Before writing, changing, or reviewing frontend tests, read `web/docs/test.md` completely. It is the single source of truth. This skill provides an execution checklist and must not redefine or extend that policy.
Before writing, changing, or reviewing frontend tests, read `web/docs/test.md` completely. It is the single source of truth. This skill defines the execution workflow and must not add requirements that conflict with or duplicate that guide.
## Workflow
@@ -22,14 +22,10 @@ Before writing, changing, or reviewing frontend tests, read `web/docs/test.md` c
When reviewing existing tests, recommend deleting low-value tests as readily as adding missing behavior coverage.
Run focused tests from the owning workspace:
Run focused tests from `web/`:
```bash
# web/
vp test run path/to/spec-or-directory
# packages/dify-ui/
vp test run --project unit src/path/to/spec
```
Run Dify UI Storybook tests with `vp test --project storybook --run`. Run broader checks only after the focused behavior passes.
Run broader checks only after the focused behavior passes.
+44 -17
View File
@@ -8,6 +8,7 @@
# Lint bulk suppression baselines.
/oxlint-suppressions.json
/eslint-suppressions.json
# CODEOWNERS file
/.github/CODEOWNERS @laipz8200 @crazywoola
@@ -32,9 +33,31 @@
# Backend (default owner, more specific rules below will override)
/api/ @QuantumGhost
# Backend - MCP
/api/core/mcp/ @Nov1c444
/api/core/entities/mcp_provider.py @Nov1c444
/api/services/tools/mcp_tools_manage_service.py @Nov1c444
/api/controllers/mcp/ @Nov1c444
/api/controllers/console/app/mcp_server.py @Nov1c444
# Backend - Tests
/api/tests/ @laipz8200 @QuantumGhost
/api/tests/**/*mcp* @Nov1c444
# Backend - Workflow - Engine (Core graph execution engine)
/api/core/workflow/graph_engine/ @laipz8200 @QuantumGhost
/api/core/workflow/runtime/ @laipz8200 @QuantumGhost
/api/core/workflow/graph/ @laipz8200 @QuantumGhost
/api/core/workflow/graph_events/ @laipz8200 @QuantumGhost
/api/core/workflow/node_events/ @laipz8200 @QuantumGhost
# Backend - Workflow - Nodes (Agent, Iteration, Loop, LLM)
/api/core/workflow/nodes/agent/ @Nov1c444
/api/core/workflow/nodes/iteration/ @Nov1c444
/api/core/workflow/nodes/loop/ @Nov1c444
/api/core/workflow/nodes/llm/ @Nov1c444
# Backend - RAG (Retrieval Augmented Generation)
/api/core/rag/ @JohnJyong
/api/services/rag_pipeline/ @JohnJyong
@@ -88,6 +111,7 @@
/api/core/app/layers/trigger_post_layer.py @CourTeous33
/api/services/trigger/ @CourTeous33
/api/models/trigger.py @CourTeous33
/api/fields/workflow_trigger_fields.py @CourTeous33
/api/repositories/workflow_trigger_log_repository.py @CourTeous33
/api/repositories/sqlalchemy_workflow_trigger_log_repository.py @CourTeous33
/api/libs/schedule_utils.py @CourTeous33
@@ -112,11 +136,11 @@
/api/controllers/console/billing/ @hj24 @zyssyz123
# Backend - Enterprise
/api/configs/enterprise/ @GareArc
/api/services/enterprise/ @GareArc
/api/services/feature_service.py @GareArc
/api/controllers/console/feature.py @GareArc
/api/controllers/web/feature.py @GareArc
/api/configs/enterprise/ @GarfieldDai @GareArc
/api/services/enterprise/ @GarfieldDai @GareArc
/api/services/feature_service.py @GarfieldDai @GareArc
/api/controllers/console/feature.py @GarfieldDai @GareArc
/api/controllers/web/feature.py @GarfieldDai @GareArc
# Backend - Database Migrations
/api/migrations/ @snakevash @laipz8200 @MRZHUH
@@ -129,6 +153,7 @@
# Frontend - Platform and Features
/web/config/ @lyzno1
/web/contract/ @lyzno1
/web/env.ts @lyzno1
/web/features/ @lyzno1
/web/hooks/ @lyzno1
@@ -187,6 +212,7 @@
/web/app/components/rag-pipeline/store/ @iamjoel @zxhlyh
# Frontend - RAG - Documents List
/web/app/components/datasets/documents/list.tsx @iamjoel @WTW0313
/web/app/components/datasets/documents/create-from-pipeline/ @iamjoel @WTW0313
# Frontend - RAG - Segments List
@@ -205,22 +231,22 @@
/web/app/components/plugins/marketplace/ @iamjoel @Yessenia-d
# Frontend - Login and Registration
/web/app/signin/ @iamjoel
/web/app/signup/ @iamjoel
/web/app/reset-password/ @iamjoel
/web/app/install/ @iamjoel
/web/app/init/ @iamjoel
/web/app/forgot-password/ @iamjoel
/web/app/account/ @iamjoel
/web/app/signin/ @douxc @iamjoel
/web/app/signup/ @douxc @iamjoel
/web/app/reset-password/ @douxc @iamjoel
/web/app/install/ @douxc @iamjoel
/web/app/init/ @douxc @iamjoel
/web/app/forgot-password/ @douxc @iamjoel
/web/app/account/ @douxc @iamjoel
# Frontend - Service Authentication
/web/service/base.ts @iamjoel
/web/service/base.ts @douxc @iamjoel
# Frontend - WebApp Authentication and Access Control
/web/app/(shareLayout)/components/ @iamjoel
/web/app/(shareLayout)/webapp-signin/ @iamjoel
/web/app/(shareLayout)/webapp-reset-password/ @iamjoel
/web/app/components/app/app-access-control/ @iamjoel
/web/app/(shareLayout)/components/ @douxc @iamjoel
/web/app/(shareLayout)/webapp-signin/ @douxc @iamjoel
/web/app/(shareLayout)/webapp-reset-password/ @douxc @iamjoel
/web/app/components/app/app-access-control/ @douxc @iamjoel
# Frontend - Explore Page
/web/app/components/explore/ @CodingOnStar @iamjoel
@@ -239,6 +265,7 @@
/web/app/components/base/**/*.spec.tsx @hyoban @CodingOnStar
# Frontend - Utils and Hooks
/web/utils/classnames.ts @iamjoel @zxhlyh
/web/utils/time.ts @iamjoel @zxhlyh
/web/utils/format.ts @iamjoel @zxhlyh
/web/utils/clipboard.ts @iamjoel @zxhlyh
+3
View File
@@ -47,6 +47,9 @@ jobs:
- name: Install dependencies
run: uv sync --project api --dev
- name: Run dify config tests
run: uv run --project api pytest api/tests/unit_tests/configs/test_env_consistency.py
- name: Run Unit Tests
run: |
uv run --project api pytest \
+2 -5
View File
@@ -27,7 +27,7 @@ jobs:
steps:
- id: skip_check
continue-on-error: true
uses: fkirc/skip-duplicate-actions@b974a9395958c231af965b70070979a577efa578 # v5.3.2
uses: fkirc/skip-duplicate-actions@f75f66ce1886f00957d99748a42c724f4330bdcf # v5.3.1
with:
cancel_others: 'true'
concurrent_skipping: same_content_newer
@@ -72,7 +72,6 @@ jobs:
- 'docker/volumes/sandbox/conf/**'
cli:
- 'cli/**'
- 'packages/contracts/**'
- 'packages/tsconfig/**'
- 'package.json'
- 'pnpm-lock.yaml'
@@ -81,6 +80,7 @@ jobs:
- '.npmrc'
- '.nvmrc'
- '.github/workflows/cli-tests.yml'
- '.github/workflows/cli-docker-build.yml'
- '.github/actions/setup-web/**'
web:
- 'web/**'
@@ -105,7 +105,6 @@ jobs:
- 'docker/docker-compose.middleware.yaml'
- 'docker/envs/middleware.env.example'
- '.github/workflows/web-e2e.yml'
- '.github/workflows/main-ci.yml'
- '.github/actions/setup-web/**'
vdb:
- 'api/core/rag/datasource/**'
@@ -335,8 +334,6 @@ jobs:
- check-changes
if: needs.pre_job.outputs.should_skip != 'true' && needs.check-changes.outputs.e2e-changed == 'true'
uses: ./.github/workflows/web-e2e.yml
with:
run-external-runtime: false
secrets: inherit
web-e2e-skip:
-16
View File
@@ -26,30 +26,19 @@ jobs:
external_e2e:
- 'e2e/features/agent-v2/**'
- 'e2e/features/step-definitions/agent-v2/**'
- 'e2e/features/step-definitions/common/**'
- 'e2e/features/support/**'
- 'e2e/fixtures/auth.ts'
- 'e2e/fixtures/test-materials/**'
- 'e2e/scripts/**'
- 'e2e/support/**'
- 'e2e/cucumber.config.ts'
- 'e2e/package.json'
- 'e2e/test-env.ts'
- 'e2e/tsconfig.json'
- 'e2e/tsx-register.js'
- 'package.json'
- 'pnpm-lock.yaml'
- '.nvmrc'
- '.github/workflows/post-merge.yml'
- '.github/workflows/web-e2e.yml'
- '.github/actions/setup-web/**'
- 'docker/docker-compose.middleware.yaml'
- 'docker/envs/middleware.env.example'
- 'dify-agent/**'
- 'dify-agent-runtime/**'
- 'api/pyproject.toml'
- 'api/uv.lock'
- 'api/tests/integration_tests/.env.example'
- 'api/clients/agent_backend/**'
- 'api/core/app/apps/agent_app/**'
- 'api/core/workflow/nodes/agent_v2/**'
@@ -59,13 +48,8 @@ jobs:
- 'api/services/plugin/**'
- 'api/core/tools/**'
- 'api/services/tools/**'
- 'packages/contracts/package.json'
- 'packages/contracts/generated/api/console/agent/**'
- 'packages/contracts/generated/api/console/apps/**'
- 'packages/contracts/generated/api/console/datasets/**'
- 'packages/contracts/generated/api/console/orpc.gen.ts'
- 'packages/contracts/generated/api/console/workspaces/**'
- 'packages/contracts/generated/api/service/**'
- 'web/features/agent-v2/**'
- 'web/app/(commonLayout)/agents/**'
- 'web/app/(commonLayout)/@detailSidebar/agents/**'
+4 -4
View File
@@ -27,7 +27,7 @@ jobs:
persist-credentials: false
- name: Setup Go
uses: actions/setup-go@b7ad1dad31e06c5925ef5d2fc7ad053ef454303e # v7.0.0
uses: actions/setup-go@d35c59abb061a4a6fb18e82ac0862c26744d6ab5 # v5.5.0
with:
go-version-file: dify-agent-runtime/go.mod
cache-dependency-path: dify-agent-runtime/go.sum
@@ -51,13 +51,13 @@ jobs:
persist-credentials: false
- name: Setup Go
uses: actions/setup-go@b7ad1dad31e06c5925ef5d2fc7ad053ef454303e # v7.0.0
uses: actions/setup-go@d35c59abb061a4a6fb18e82ac0862c26744d6ab5 # v5.5.0
with:
go-version-file: dify-agent-runtime/go.mod
cache-dependency-path: dify-agent-runtime/go.sum
- name: Run golangci-lint
uses: golangci/golangci-lint-action@ba0d7d2ec06a0ea1cb5fa41b2e4a3ab91d21278a # v6.5.0
uses: golangci/golangci-lint-action@4afd733a84b1f43292c63897423277bb7f4313a9 # v6.5.0
with:
working-directory: dify-agent-runtime
version: latest
@@ -78,7 +78,7 @@ jobs:
persist-credentials: false
- name: Setup Go
uses: actions/setup-go@b7ad1dad31e06c5925ef5d2fc7ad053ef454303e # v7.0.0
uses: actions/setup-go@d35c59abb061a4a6fb18e82ac0862c26744d6ab5 # v5.5.0
with:
go-version-file: dify-agent-runtime/go.mod
cache-dependency-path: dify-agent-runtime/go.sum
+1 -1
View File
@@ -29,7 +29,7 @@ jobs:
persist-credentials: false
- name: Use Node.js
uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0
with:
node-version: 22
cache: ''
+1 -1
View File
@@ -158,7 +158,7 @@ jobs:
- name: Run Claude Code for Translation Sync
if: steps.context.outputs.CHANGED_FILES != ''
uses: anthropics/claude-code-action@af0559ee4f514d1ef21826982bed13f7edc3c35e # v1.0.178
uses: anthropics/claude-code-action@e90deca47693f9457b72f2b53c17d7c445a87342 # v1.0.171
with:
anthropic_api_key: ${{ secrets.ANTHROPIC_API_KEY }}
github_token: ${{ secrets.GITHUB_TOKEN }}
+18 -77
View File
@@ -4,9 +4,9 @@ on:
workflow_call:
inputs:
run-external-runtime:
description: Run only the prepared and external runtime suite instead of the core suites.
required: true
required: false
type: boolean
default: false
permissions:
contents: read
@@ -19,7 +19,6 @@ jobs:
test:
name: Web Full-Stack E2E
runs-on: depot-ubuntu-24.04-4
timeout-minutes: 120
defaults:
run:
shell: bash
@@ -46,7 +45,6 @@ jobs:
run: uv sync --project api --dev
- name: Run E2E support unit tests
if: ${{ !inputs.run-external-runtime }}
working-directory: ./e2e
run: vp run test:unit
@@ -55,7 +53,6 @@ jobs:
run: vp run e2e:install
- name: Run isolated source-api and built-web Cucumber E2E tests
if: ${{ !inputs.run-external-runtime }}
working-directory: ./e2e
env:
E2E_ADMIN_EMAIL: e2e-admin@example.com
@@ -65,53 +62,7 @@ jobs:
E2E_INIT_PASSWORD: E2eInit12345
run: vp run e2e:full
- name: Preserve Chromium E2E report and logs
if: ${{ !cancelled() && !inputs.run-external-runtime }}
run: |
if [[ -d e2e/cucumber-report ]]; then
mv e2e/cucumber-report e2e/cucumber-report-non-external
fi
if [[ -d e2e/.logs ]]; then
mv e2e/.logs e2e/.logs-non-external
fi
- name: Run WebKit keyboard and browser smoke tests
if: ${{ !inputs.run-external-runtime }}
working-directory: ./e2e
env:
E2E_ADMIN_EMAIL: e2e-admin@example.com
E2E_ADMIN_NAME: E2E Admin
E2E_ADMIN_PASSWORD: E2eAdmin12345
E2E_BROWSER: webkit
E2E_INIT_PASSWORD: E2eInit12345
run: |
teardown_webkit_smoke() {
local run_status=$?
trap - EXIT
if ! vp run e2e:middleware:down; then
echo "::error title=E2E teardown failed::WebKit smoke middleware did not shut down cleanly."
if [[ "$run_status" -eq 0 ]]; then
run_status=1
fi
fi
exit "$run_status"
}
trap teardown_webkit_smoke EXIT
vp run e2e:middleware:up
vp run e2e -- --tags '@browser-smoke'
- name: Preserve WebKit E2E report and logs
if: ${{ !cancelled() && !inputs.run-external-runtime }}
run: |
if [[ -d e2e/cucumber-report ]]; then
mv e2e/cucumber-report e2e/cucumber-report-webkit
fi
if [[ -d e2e/.logs ]]; then
mv e2e/.logs e2e/.logs-webkit
fi
- name: Run prepared and external runtime E2E tests
- name: Run external runtime E2E tests
if: ${{ inputs.run-external-runtime }}
working-directory: ./e2e
env:
@@ -121,10 +72,13 @@ jobs:
E2E_AGENT_DECISION_MODEL_NAME: ${{ vars.E2E_AGENT_DECISION_MODEL_NAME || 'gpt-5.5' }}
E2E_AGENT_DECISION_MODEL_PROVIDER: ${{ vars.E2E_AGENT_DECISION_MODEL_PROVIDER || 'openai' }}
E2E_AGENT_DECISION_MODEL_TYPE: ${{ vars.E2E_AGENT_DECISION_MODEL_TYPE || 'llm' }}
E2E_EXTERNAL_RUNTIME_SEED_SPECS: ${{ vars.E2E_EXTERNAL_RUNTIME_SEED_SPECS }}
E2E_EXTERNAL_RUNTIME_TAGS: ${{ vars.E2E_EXTERNAL_RUNTIME_TAGS }}
E2E_FORCE_WEB_BUILD: "1"
E2E_INIT_PASSWORD: E2eInit12345
E2E_MARKETPLACE_API_URL: ${{ vars.E2E_MARKETPLACE_API_URL }}
E2E_MARKETPLACE_PLUGIN_IDS: ${{ vars.E2E_MARKETPLACE_PLUGIN_IDS }}
E2E_MARKETPLACE_PLUGIN_UNIQUE_IDENTIFIERS: ${{ vars.E2E_MARKETPLACE_PLUGIN_UNIQUE_IDENTIFIERS }}
E2E_MODEL_PROVIDER_CREDENTIALS_JSON: ${{ secrets.E2E_MODEL_PROVIDER_CREDENTIALS_JSON }}
E2E_SPEECH_TO_TEXT_MODEL_NAME: ${{ vars.E2E_SPEECH_TO_TEXT_MODEL_NAME || 'gpt-4o-mini-transcribe' }}
E2E_SPEECH_TO_TEXT_MODEL_PROVIDER: ${{ vars.E2E_SPEECH_TO_TEXT_MODEL_PROVIDER || 'openai' }}
@@ -138,22 +92,20 @@ jobs:
exit 1
fi
teardown_external_runtime() {
local run_status=$?
trap - EXIT
if ! vp run e2e:middleware:down; then
echo "::error title=E2E teardown failed::External runtime middleware did not shut down cleanly."
if [[ "$run_status" -eq 0 ]]; then
run_status=1
fi
fi
exit "$run_status"
}
if [[ -d cucumber-report ]]; then
rm -rf cucumber-report-non-external
mv cucumber-report cucumber-report-non-external
fi
trap teardown_external_runtime EXIT
if [[ -d .logs ]]; then
rm -rf .logs-non-external
mv .logs .logs-non-external
fi
trap 'vp run e2e:middleware:down' EXIT
vp run e2e:middleware:up
vp run e2e:post-merge:prepare
vp run e2e:post-merge
vp run e2e:external:prepare
vp run e2e:external
- name: Upload Cucumber report
if: ${{ !cancelled() }}
@@ -163,7 +115,6 @@ jobs:
path: |
e2e/cucumber-report
e2e/cucumber-report-non-external
e2e/cucumber-report-webkit
retention-days: 7
- name: Upload E2E logs
@@ -174,15 +125,5 @@ jobs:
path: |
e2e/.logs/*.log
e2e/.logs-non-external/*.log
e2e/.logs-webkit/*.log
include-hidden-files: true
retention-days: 7
- name: Upload E2E seed report
if: ${{ !cancelled() && inputs.run-external-runtime }}
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: e2e-seed-report
path: e2e/seed-report
if-no-files-found: ignore
retention-days: 7
+2
View File
@@ -17,6 +17,8 @@ jobs:
test:
name: Web Tests (${{ matrix.shardIndex }}/${{ matrix.shardTotal }})
runs-on: depot-ubuntu-24.04-4
env:
VITEST_COVERAGE_SCOPE: app-components
strategy:
fail-fast: false
matrix:
+18 -4
View File
@@ -5,9 +5,7 @@
"name": "Python: API (gevent)",
"type": "debugpy",
"request": "launch",
"module": "gevent.monkey",
"args": ["--module", "app"],
"gevent": true,
"program": "${workspaceFolder}/api/app.py",
"jinja": true,
"justMyCode": true,
"cwd": "${workspaceFolder}/api",
@@ -35,6 +33,22 @@
"justMyCode": false,
"cwd": "${workspaceFolder}/api",
"python": "${workspaceFolder}/api/.venv/bin/python"
}
},
{
"name": "Next.js: debug full stack",
"type": "node",
"request": "launch",
"program": "${workspaceFolder}/web/node_modules/next/dist/bin/next",
"runtimeArgs": ["--inspect"],
"skipFiles": ["<node_internals>/**"],
"serverReadyAction": {
"action": "debugWithChrome",
"killOnServerStop": true,
"pattern": "- Local:.+(https?://.+)",
"uriFormat": "%s",
"webRoot": "${workspaceFolder}/web"
},
"cwd": "${workspaceFolder}/web"
}
]
}
-2
View File
@@ -107,7 +107,6 @@ test:
echo "Target: $(TARGET_TESTS)"; \
uv run --project api --dev pytest $(TARGET_TESTS); \
else \
set -e; \
echo "Running backend unit tests"; \
uv run --project api --dev pytest -p no:benchmark --timeout "$${PYTEST_TIMEOUT:-20}" -n auto \
api/tests/unit_tests \
@@ -125,7 +124,6 @@ test-all:
echo "Target: $(TARGET_TESTS)"; \
uv run --project api --dev pytest $(TARGET_TESTS); \
else \
set -e; \
echo "Running backend unit tests"; \
uv run --project api --dev pytest -p no:benchmark --timeout "$${PYTEST_TIMEOUT:-20}" -n auto \
api/tests/unit_tests \
+4 -11
View File
@@ -15,7 +15,7 @@
<a href="https://discord.gg/FngNHpbcY7" target="_blank">
<img src="https://img.shields.io/discord/1082486657678311454?logo=discord&labelColor=%20%235462eb&logoColor=%20%23f5f5f5&color=%20%235462eb"
alt="chat on Discord"></a>
<a href="https://reddit.com/r/difyai" target="_blank">
<a href="https://reddit.com/r/difyai" target="_blank">
<img src="https://img.shields.io/reddit/subreddit-subscribers/difyai?style=plastic&logo=reddit&label=r%2Fdifyai&labelColor=white"
alt="join Reddit"></a>
<a href="https://twitter.com/intent/follow?screen_name=dify_ai" target="_blank">
@@ -71,7 +71,7 @@ Dify is an open-source LLM app development platform. Its intuitive interface com
<br/>
The easiest way to start the Dify server is through [Docker Compose](docker/docker-compose.yaml). Before running Dify with the following commands, make sure that [Docker](https://docs.docker.com/get-docker/) and Docker Compose v2.24.0 or later are installed on your machine:
The easiest way to start the Dify server is through [Docker Compose](docker/docker-compose.yaml). Before running Dify with the following commands, make sure that [Docker](https://docs.docker.com/get-docker/) and [Docker Compose](https://docs.docker.com/compose/install/) are installed on your machine:
```bash
cd dify
@@ -207,16 +207,9 @@ At the same time, please consider supporting Dify by sharing it on social media
<img src="https://contrib.rocks/image?repo=langgenius/dify" />
</a>
## Star History
## Star history
<!-- GitHub token name: star-history -->
<a href="https://www.star-history.com/?repos=langgenius%2Fdify&type=date&legend=top-left">
<picture>
<source media="(prefers-color-scheme: dark)" srcset="https://api.star-history.com/chart?repos=langgenius/dify&type=date&theme=dark&legend=top-left&sealed_token=p-SWD-UXZEDc5a2d0EMfgMmyCwVyRlSof0Qox68v7k4PvQPKRlCx0jDIlNztw7mbA6DEn96R50DojO9pCi5LQUlDBAoIhRswt8-GuC8K3rZQ3naJUXbuHqR_oItIW2F0NNM-7npevzw5SXp7L5mwixgqJncAvAGzuGq5zmhQfiDRSW_Jd6y8TQCZMJbt" />
<source media="(prefers-color-scheme: light)" srcset="https://api.star-history.com/chart?repos=langgenius/dify&type=date&legend=top-left&sealed_token=p-SWD-UXZEDc5a2d0EMfgMmyCwVyRlSof0Qox68v7k4PvQPKRlCx0jDIlNztw7mbA6DEn96R50DojO9pCi5LQUlDBAoIhRswt8-GuC8K3rZQ3naJUXbuHqR_oItIW2F0NNM-7npevzw5SXp7L5mwixgqJncAvAGzuGq5zmhQfiDRSW_Jd6y8TQCZMJbt" />
<img alt="Star History Chart" src="https://api.star-history.com/chart?repos=langgenius/dify&type=date&legend=top-left&sealed_token=p-SWD-UXZEDc5a2d0EMfgMmyCwVyRlSof0Qox68v7k4PvQPKRlCx0jDIlNztw7mbA6DEn96R50DojO9pCi5LQUlDBAoIhRswt8-GuC8K3rZQ3naJUXbuHqR_oItIW2F0NNM-7npevzw5SXp7L5mwixgqJncAvAGzuGq5zmhQfiDRSW_Jd6y8TQCZMJbt" />
</picture>
</a>
[![Star History Chart](https://api.star-history.com/svg?repos=langgenius/dify&type=Date)](https://star-history.com/#langgenius/dify&Date)
## Security disclosure
-19
View File
@@ -561,8 +561,6 @@ WORKFLOW_MAX_EXECUTION_STEPS=500
WORKFLOW_MAX_EXECUTION_TIME=1200
WORKFLOW_CALL_MAX_DEPTH=5
MAX_VARIABLE_SIZE=204800
# Maximum concurrent node-builder LLM calls per workflow generation request
WORKFLOW_GENERATOR_NODE_BUILDER_MAX_WORKERS=6
# GraphEngine Worker Pool Configuration
# Minimum number of workers per GraphEngine instance (default: 1)
@@ -667,27 +665,10 @@ PLUGIN_REMOTE_INSTALL_HOST=localhost
PLUGIN_MAX_PACKAGE_SIZE=15728640
PLUGIN_MODEL_SCHEMA_CACHE_TTL=3600
PLUGIN_MODEL_PROVIDERS_CACHE_TTL=86400
# Comma-separated marketplace plugin IDs whose latest versions are installed for newly registered users.
# Example: langgenius/openai,langgenius/gemini
NEW_USER_DEFAULT_PLUGIN_IDS=
# Comma-separated model_type:provider:model entries assigned after default plugins finish installing.
# Example: llm:langgenius/openai/openai:gpt-4o-mini,text-embedding:langgenius/openai/openai:text-embedding-3-small
NEW_USER_DEFAULT_MODELS=
INNER_API_KEY_FOR_PLUGIN=QaHbTe77CtuXmsfyhR7+vRjI/+XbV1AaFy691iy+kGDv2Jvy0/eAh8Y1
# Dify Agent backend
AGENT_BACKEND_BASE_URL=http://localhost:5050
AGENT_BACKEND_STREAM_READ_TIMEOUT_SECONDS=30
AGENT_BACKEND_STREAM_MAX_RECONNECTS=3
AGENT_BACKEND_RUN_TIMEOUT_SECONDS=1200
# KnowledgeFS (Dataset 2.0)
KNOWLEDGE_FS_ENABLED=false
KNOWLEDGE_FS_BASE_URL=
# Shared with KnowledgeFS; use at least 32 random characters.
KNOWLEDGE_FS_JWT_SECRET=
KNOWLEDGE_FS_SSE_READ_TIMEOUT_SECONDS=300
KNOWLEDGE_FS_TIMEOUT_SECONDS=10
# Marketplace configuration
MARKETPLACE_ENABLED=true
-12
View File
@@ -109,18 +109,6 @@ class Example:
- Reuse existing helpers in `core/`, `services/`, and `libs/` before creating new abstractions.
- Optimise for observability: deterministic control flow, clear logging, actionable errors.
### Owner-Bound Resource References
- Resolve and validate the outer owner before binding a nested resource ID.
- For stable single-parent chains, use immutable nested `NamedTuple` refs.
- Root refs carry tenant plus root ID; child refs carry the parent ref.
- In production, construct refs through the domain ref service.
- Python allowing direct construction does not grant authorization.
- Scope every consuming query with complete owner predicates; refs are not security tokens.
- Keep polymorphic owners flat until explicit nominal owner types exist.
- Do not add generic ref bases or compatibility fields only for uniformity.
- Reconstruct internal refs from validated database state after payload or async boundaries.
### Logging & Errors
- Never use `print`; use a module-level logger:
-19
View File
@@ -1,24 +1,5 @@
from __future__ import annotations
# ``python -m app`` (docker DEBUG=true, or IDE debugging) serves through the
# gevent pywsgi server at the bottom of this file, so the stdlib must be
# monkey-patched BEFORE any other import pulls in sockets or locks. Without
# this, every request runs as a greenlet on one OS thread while blocking
# calls (LLM invokes, ``Future.result`` waits, DB I/O) pin that thread — the
# whole process freezes until the call returns. Gunicorn and Celery apply
# their own patching (see gunicorn.conf.py / celery_entrypoint.py), and
# ``flask run`` uses real Werkzeug threads, so both skip this branch.
if __name__ == "__main__":
from gevent import monkey
monkey.patch_all()
import psycogreen.gevent as psycogreen_gevent
from grpc.experimental import gevent as grpc_gevent
grpc_gevent.init_gevent()
psycogreen_gevent.patch_psycopg()
import logging
import sys
from typing import TYPE_CHECKING, cast
+6 -40
View File
@@ -8,7 +8,7 @@ creating another wire contract.
from __future__ import annotations
from collections.abc import Callable, Iterator
from collections.abc import Iterator
from typing import Protocol
from dify_agent.client import (
@@ -45,13 +45,7 @@ class AgentBackendRunClient(Protocol):
def cancel_run(self, run_id: str, request: CancelRunRequest | None = None) -> CancelRunResponse:
"""Request explicit cancellation for one Agent backend run."""
def stream_events(
self,
run_id: str,
*,
after: str | None = None,
should_stop: Callable[[], bool] | None = None,
) -> Iterator[RunEvent]:
def stream_events(self, run_id: str, *, after: str | None = None) -> Iterator[RunEvent]:
"""Yield public ``dify-agent`` run events in stream order."""
def wait_run(self, run_id: str, *, timeout_seconds: float | None = None) -> RunStatusResponse:
@@ -67,15 +61,7 @@ class _DifyAgentSyncClient(Protocol):
def cancel_run_sync(self, run_id: str, request: CancelRunRequest | None = None) -> CancelRunResponse:
"""Cancel one run synchronously."""
def stream_events_sync(
self,
run_id: str,
*,
after: str | None = None,
max_reconnects: int | None = None,
timeout_seconds: float | None = None,
should_stop: Callable[[], bool] | None = None,
) -> Iterator[RunEvent]:
def stream_events_sync(self, run_id: str, *, after: str | None = None) -> Iterator[RunEvent]:
"""Stream run events synchronously."""
def wait_run_sync(self, run_id: str, *, timeout_seconds: float | None = None) -> RunStatusResponse:
@@ -87,16 +73,8 @@ class DifyAgentBackendRunClient:
client: _DifyAgentSyncClient
def __init__(
self,
client: _DifyAgentSyncClient,
*,
stream_max_reconnects: int = 3,
stream_timeout_seconds: float = 1200,
) -> None:
def __init__(self, client: _DifyAgentSyncClient) -> None:
self.client = client
self._stream_max_reconnects = stream_max_reconnects
self._stream_timeout_seconds = stream_timeout_seconds
def create_run(self, request: CreateRunRequest) -> CreateRunResponse:
"""Create one run through ``POST /runs`` and normalize client exceptions."""
@@ -112,22 +90,10 @@ class DifyAgentBackendRunClient:
except Exception as exc:
raise _normalize_dify_agent_error(exc) from exc
def stream_events(
self,
run_id: str,
*,
after: str | None = None,
should_stop: Callable[[], bool] | None = None,
) -> Iterator[RunEvent]:
def stream_events(self, run_id: str, *, after: str | None = None) -> Iterator[RunEvent]:
"""Stream run events from ``/events/sse`` with the wrapped client's reconnect policy."""
try:
yield from self.client.stream_events_sync(
run_id,
after=after,
max_reconnects=self._stream_max_reconnects,
timeout_seconds=self._stream_timeout_seconds,
should_stop=should_stop,
)
yield from self.client.stream_events_sync(run_id, after=after)
except Exception as exc:
raise _normalize_dify_agent_error(exc) from exc
+1 -8
View File
@@ -13,17 +13,10 @@ def create_agent_backend_run_client(
base_url: str | None = None,
use_fake: bool = False,
fake_scenario: str | FakeAgentBackendScenario = FakeAgentBackendScenario.SUCCESS,
stream_read_timeout_seconds: float = 30,
stream_max_reconnects: int = 3,
stream_run_timeout_seconds: float = 1200,
) -> AgentBackendRunClient:
"""Create the API-side run client without hiding the ``dify-agent`` protocol."""
if use_fake:
return FakeAgentBackendRunClient(scenario=FakeAgentBackendScenario(fake_scenario))
if base_url is None:
raise ValueError("base_url is required when creating a real Agent backend client")
return DifyAgentBackendRunClient(
Client(base_url=base_url, stream_timeout=stream_read_timeout_seconds),
stream_max_reconnects=stream_max_reconnects,
stream_timeout_seconds=stream_run_timeout_seconds,
)
return DifyAgentBackendRunClient(Client(base_url=base_url))
+2 -10
View File
@@ -7,7 +7,7 @@ separate ``agent-backend.v1`` event stream.
from __future__ import annotations
from collections.abc import Callable, Iterator
from collections.abc import Iterator
from datetime import UTC, datetime
from enum import StrEnum
@@ -69,17 +69,9 @@ class FakeAgentBackendRunClient:
del request
return CancelRunResponse(run_id=run_id, status="cancelled")
def stream_events(
self,
run_id: str,
*,
after: str | None = None,
should_stop: Callable[[], bool] | None = None,
) -> Iterator[RunEvent]:
def stream_events(self, run_id: str, *, after: str | None = None) -> Iterator[RunEvent]:
"""Yield the deterministic public ``RunEvent`` sequence for ``run_id``."""
for event in self._events(run_id):
if should_stop is not None and should_stop():
return
if after is not None and event.id is not None and event.id <= after:
continue
yield event
+5 -3
View File
@@ -9,7 +9,6 @@ from sqlalchemy import delete, func, select
from sqlalchemy.engine import CursorResult
from configs import dify_config
from core.db.session_factory import session_factory
from core.helper import encrypter
from core.plugin.entities.plugin_daemon import CredentialType
from core.plugin.impl.plugin import PluginInstaller
@@ -579,6 +578,9 @@ def install_rag_pipeline_plugins(input_file, output_file, workers):
"""
click.echo(click.style("Installing rag pipeline plugins", fg="yellow"))
plugin_migration = PluginMigration()
with session_factory.create_session() as session:
plugin_migration.install_rag_pipeline_plugins(input_file, output_file, workers, session=session)
plugin_migration.install_rag_pipeline_plugins(
input_file,
output_file,
workers,
)
click.echo(click.style("Installing rag pipeline plugins successfully", fg="green"))
+63 -209
View File
@@ -1,8 +1,6 @@
import datetime
import logging
import re
import time
import uuid
from collections.abc import Callable
from typing import TypedDict
@@ -23,7 +21,6 @@ from tasks.remove_app_and_related_data_task import delete_draft_variables_batch
logger = logging.getLogger(__name__)
_HEX_PREFIXES = tuple("0123456789abcdef")
_TARGET_MONTH_PATTERN = re.compile(r"^\d{4}-(0[1-9]|1[0-2])$")
class WorkflowRunArchivePlanRow(TypedDict):
@@ -69,7 +66,6 @@ def _parse_tenant_prefixes(prefixes: str | None) -> list[str]:
def _parse_comma_separated_ids(raw_ids: str | None, *, param_name: str) -> list[str] | None:
"""Keep an omitted scope unset while rejecting an explicitly empty scope."""
if raw_ids is None:
return None
parsed = sorted({raw_id.strip() for raw_id in raw_ids.split(",") if raw_id.strip()})
@@ -78,27 +74,6 @@ def _parse_comma_separated_ids(raw_ids: str | None, *, param_name: str) -> list[
return parsed
def _parse_archive_target_month(target_month: str) -> tuple[int, int]:
"""Validate the V2 catalog month selector and return its numeric components."""
if not _TARGET_MONTH_PATTERN.fullmatch(target_month):
raise click.BadParameter("target-month must use YYYY-MM format", param_hint="--target-month")
year_text, month_text = target_month.split("-", maxsplit=1)
return int(year_text), int(month_text)
def _parse_archive_catalog_cursor(after_catalog_id: str | None) -> str | None:
"""Normalize the exclusive V2 catalog keyset cursor when one is provided."""
if after_catalog_id is None:
return None
try:
return str(uuid.UUID(after_catalog_id))
except ValueError as exc:
raise click.BadParameter(
"after-catalog-id must be a UUID returned by the same V2 operation and scope",
param_hint="--after-catalog-id",
) from exc
def _get_archive_candidate_tenant_ids_by_prefix(
session: Session,
prefix: str,
@@ -835,11 +810,9 @@ def backfill_workflow_run_archive_bundles(
click.echo(click.style(f" ... and {len(summary.errors) - 10} more failures", fg="red"))
def _echo_bundle_archive_operation_summary(summary, *, dry_run: bool) -> None:
def _echo_bundle_archive_operation_summary(summary) -> None:
status = "completed successfully" if summary.bundles_failed == 0 else "completed with failures"
fg = "green" if summary.bundles_failed == 0 else "red"
cursor_label = "preview_next_catalog_id" if dry_run else "next_catalog_id"
cursor_value = summary.preview_next_catalog_id if dry_run else summary.next_catalog_id
click.echo(
click.style(
f"{summary.operation} {status}. "
@@ -848,12 +821,10 @@ def _echo_bundle_archive_operation_summary(summary, *, dry_run: bool) -> None:
f"archive_bytes={summary.archive_bytes} duration={summary.elapsed_time:.2f}s "
f"validation_time={summary.validation_time:.2f}s "
f"runs_per_second={summary.runs_per_second:.2f} rows_per_second={summary.rows_per_second:.2f} "
f"bytes_per_second={summary.bytes_per_second:.2f} {cursor_label}={cursor_value or 'none'}",
f"bytes_per_second={summary.bytes_per_second:.2f}",
fg=fg,
)
)
if dry_run:
click.echo(click.style("Dry-run cursor is preview-only; do not persist it for a destructive run.", fg="yellow"))
click.echo(click.style("table,row_count", fg="white"))
for table_name in [
"workflow_runs",
@@ -871,8 +842,7 @@ def _echo_bundle_archive_operation_summary(summary, *, dry_run: bool) -> None:
click.style(
f" bundle={result.bundle_id} tenant={result.tenant_id} runs={result.run_count} "
f"rows={result.row_count} archive_bytes={result.archive_bytes} "
f"catalog_id={result.catalog_id} time={result.elapsed_time:.2f}s "
f"validation={result.validation_time:.2f}s",
f"time={result.elapsed_time:.2f}s validation={result.validation_time:.2f}s",
fg="white",
)
)
@@ -880,7 +850,7 @@ def _echo_bundle_archive_operation_summary(summary, *, dry_run: bool) -> None:
click.echo(
click.style(
f" failed bundle={result.bundle_id} tenant={result.tenant_id} "
f"catalog_id={result.catalog_id} object_prefix={result.object_prefix} error={result.error}",
f"object_prefix={result.object_prefix} error={result.error}",
fg="red",
)
)
@@ -897,24 +867,25 @@ def _echo_bundle_archive_operation_summary(summary, *, dry_run: bool) -> None:
)
@click.option("--run-id", required=False, help="Workflow run ID to restore.")
@click.option(
"--target-month",
metavar="YYYY-MM",
"--start-from",
type=click.DateTime(formats=["%Y-%m-%d", "%Y-%m-%dT%H:%M:%S"]),
default=None,
help="V2 catalog month to restore; required unless --run-id is used.",
help="Optional lower bound (inclusive) for created_at; must be paired with --end-before.",
)
@click.option(
"--after-catalog-id",
"--end-before",
type=click.DateTime(formats=["%Y-%m-%d", "%Y-%m-%dT%H:%M:%S"]),
default=None,
help="Exclusive V2 cursor from the same restore month and tenant scope.",
help="Optional upper bound (exclusive) for created_at; must be paired with --start-from.",
)
@click.option("--workers", default=1, show_default=True, type=int, help="V1 --run-id compatibility only.")
@click.option("--limit", type=click.IntRange(min=1), default=100, show_default=True, help="Maximum V2 catalog rows.")
@click.option("--limit", type=int, default=100, show_default=True, help="Maximum number of V2 bundles to restore.")
@click.option("--dry-run", is_flag=True, help="Preview without restoring.")
def restore_workflow_runs(
tenant_ids: str | None,
run_id: str | None,
target_month: str | None,
after_catalog_id: str | None,
start_from: datetime.datetime | None,
end_before: datetime.datetime | None,
workers: int,
limit: int,
dry_run: bool,
@@ -934,20 +905,23 @@ def restore_workflow_runs(
from services.retention.workflow_run.bundle_archive_maintenance import WorkflowRunBundleArchiveMaintenance
from services.retention.workflow_run.restore_archived_workflow_run import WorkflowRunRestore
parsed_tenant_ids = _parse_comma_separated_ids(tenant_ids, param_name="tenant-ids")
parsed_tenant_ids = None
if tenant_ids:
parsed_tenant_ids = [tid.strip() for tid in tenant_ids.split(",") if tid.strip()]
if not parsed_tenant_ids:
raise click.BadParameter("tenant-ids must not be empty")
if (start_from is None) ^ (end_before is None):
raise click.UsageError("--start-from and --end-before must be provided together.")
if run_id is None and (start_from is None or end_before is None):
raise click.UsageError("--start-from and --end-before are required for batch restore.")
if workers < 1:
raise click.BadParameter("workers must be at least 1")
if run_id is not None and (target_month is not None or after_catalog_id is not None):
raise click.UsageError("--target-month and --after-catalog-id are only valid for V2 batch restore.")
if run_id is None and target_month is None:
raise click.UsageError("--target-month is required for V2 batch restore.")
start_time = datetime.datetime.now(datetime.UTC)
target_desc = f"workflow run {run_id}" if run_id else f"workflow archive catalog month {target_month}"
click.echo(
click.style(
f"Starting restore of {target_desc} at {start_time.isoformat()}.",
f"Starting restore of workflow run {run_id} at {start_time.isoformat()}.",
fg="white",
)
)
@@ -981,20 +955,17 @@ def restore_workflow_runs(
click.echo(
click.style("--workers is ignored for V2 bundle restore; bundles are processed serially.", fg="yellow")
)
assert target_month is not None
target_year, target_month_number = _parse_archive_target_month(target_month)
catalog_cursor = _parse_archive_catalog_cursor(after_catalog_id)
assert start_from is not None
assert end_before is not None
bundle_restorer = WorkflowRunBundleArchiveMaintenance(dry_run=dry_run, strict_content_validation=True)
summary = bundle_restorer.restore_batch(
tenant_ids=parsed_tenant_ids,
target_year=target_year,
target_month=target_month_number,
after_catalog_id=catalog_cursor,
start_date=start_from,
end_date=end_before,
limit=limit,
)
_echo_bundle_archive_operation_summary(summary, dry_run=dry_run)
if summary.bundles_failed:
raise click.exceptions.Exit(1)
_echo_bundle_archive_operation_summary(summary)
return
@click.command(
@@ -1008,41 +979,23 @@ def restore_workflow_runs(
)
@click.option("--run-id", required=False, help="Workflow run ID to delete.")
@click.option(
"--target-month",
metavar="YYYY-MM",
"--start-from",
type=click.DateTime(formats=["%Y-%m-%d", "%Y-%m-%dT%H:%M:%S"]),
default=None,
help="V2 catalog month to delete; required unless --run-id is used.",
help="Optional lower bound (inclusive) for created_at; must be paired with --end-before.",
)
@click.option(
"--after-catalog-id",
"--end-before",
type=click.DateTime(formats=["%Y-%m-%d", "%Y-%m-%dT%H:%M:%S"]),
default=None,
help="Exclusive V2 cursor from the same delete month and tenant scope.",
)
@click.option(
"--run-shard-index",
default=None,
type=click.IntRange(min=0),
help="Zero-based archive shard index. Must be paired with --run-shard-total.",
)
@click.option(
"--run-shard-total",
default=None,
type=click.IntRange(min=1, max=16),
help="Total archive shard count. Must be paired with --run-shard-index.",
)
@click.option("--all-pages", is_flag=True, help="Process catalog pages until an empty page is reached.")
@click.option(
"--limit",
type=click.IntRange(min=1),
default=100,
show_default=True,
help="Maximum V2 catalog rows per page.",
help="Optional upper bound (exclusive) for created_at; must be paired with --start-from.",
)
@click.option("--limit", type=int, default=100, show_default=True, help="Maximum number of V2 bundles to delete.")
@click.option("--dry-run", is_flag=True, help="Preview without deleting.")
@click.option(
"--skip-bad-archives",
is_flag=True,
help="V1 --run-id only: continue when one archive object fails validation.",
help="Continue batch deletion when one archive object fails validation.",
)
@click.option(
"--restore-sample-interval",
@@ -1054,11 +1007,8 @@ def restore_workflow_runs(
def delete_archived_workflow_runs(
tenant_ids: str | None,
run_id: str | None,
target_month: str | None,
after_catalog_id: str | None,
run_shard_index: int | None,
run_shard_total: int | None,
all_pages: bool,
start_from: datetime.datetime | None,
end_before: datetime.datetime | None,
limit: int,
dry_run: bool,
skip_bad_archives: bool,
@@ -1068,38 +1018,26 @@ def delete_archived_workflow_runs(
Delete archived workflow runs from the database.
Batch delete uses V2 bundle metadata and validates object existence, manifest schema, object size, checksum, row
counts, and source/archive content checksums before deleting source rows. Parallel workers may select one exact
archive shard; all-pages mode keeps only the current bounded page in memory. `--run-id` keeps the V1 per-run path.
counts, and source/archive content checksums before deleting source rows. `--run-id` keeps the V1 per-run path.
"""
from services.retention.workflow_run.bundle_archive_maintenance import WorkflowRunBundleArchiveMaintenance
from services.retention.workflow_run.delete_archived_workflow_run import ArchivedWorkflowRunDeletion
parsed_tenant_ids = _parse_comma_separated_ids(tenant_ids, param_name="tenant-ids")
parsed_tenant_ids = None
if tenant_ids:
parsed_tenant_ids = [tid.strip() for tid in tenant_ids.split(",") if tid.strip()]
if not parsed_tenant_ids:
raise click.BadParameter("tenant-ids must not be empty")
if (start_from is None) ^ (end_before is None):
raise click.UsageError("--start-from and --end-before must be provided together.")
if run_id is None and (start_from is None or end_before is None):
raise click.UsageError("--start-from and --end-before are required for batch delete.")
if restore_sample_interval < 0:
raise click.BadParameter("restore-sample-interval must be >= 0")
if run_id is not None and (
target_month is not None
or after_catalog_id is not None
or run_shard_index is not None
or run_shard_total is not None
or all_pages
):
raise click.UsageError(
"--target-month, --after-catalog-id, --run-shard-index, --run-shard-total, and --all-pages "
"are only valid for V2 batch delete."
)
if run_id is None and target_month is None:
raise click.UsageError("--target-month is required for V2 batch delete.")
if run_id is None and skip_bad_archives:
raise click.UsageError("--skip-bad-archives is not supported for V2 catalog batches; they fail fast.")
if (run_shard_index is None) ^ (run_shard_total is None):
raise click.UsageError("--run-shard-index and --run-shard-total must be provided together.")
if run_shard_index is not None and run_shard_total is not None and run_shard_index >= run_shard_total:
raise click.UsageError("--run-shard-index must be less than --run-shard-total.")
start_time = datetime.datetime.now(datetime.UTC)
target_desc = f"workflow run {run_id}" if run_id else f"workflow archive catalog month {target_month}"
target_desc = f"workflow run {run_id}" if run_id else "workflow runs"
click.echo(
click.style(
f"Starting delete of {target_desc} at {start_time.isoformat()}.",
@@ -1172,104 +1110,20 @@ def delete_archived_workflow_runs(
if restore_sample_interval:
click.echo(click.style("--restore-sample-interval is ignored for V2 bundle delete.", fg="yellow"))
assert target_month is not None
target_year, target_month_number = _parse_archive_target_month(target_month)
catalog_cursor = _parse_archive_catalog_cursor(after_catalog_id)
shard = (
f"{run_shard_index:02d}-of-{run_shard_total:02d}"
if run_shard_index is not None and run_shard_total is not None
else None
assert start_from is not None
assert end_before is not None
bundle_deleter = WorkflowRunBundleArchiveMaintenance(
dry_run=dry_run,
strict_content_validation=True,
stop_on_error=not skip_bad_archives,
)
bundle_deleter = WorkflowRunBundleArchiveMaintenance(dry_run=dry_run, strict_content_validation=True)
if run_shard_total is not None:
try:
bundle_deleter.validate_catalog_shards(
target_year=target_year,
target_month=target_month_number,
shard_total=run_shard_total,
tenant_ids=parsed_tenant_ids,
)
except ValueError as exc:
logger.exception(
"Archive catalog shard preflight failed: target_month=%s shard=%s",
target_month,
shard,
)
raise click.ClickException(
f"Archive catalog shard preflight failed for target_month={target_month} shard={shard}: {exc}"
) from exc
initial_catalog_cursor = catalog_cursor
pages_processed = 0
bundles_succeeded = 0
runs_processed = 0
rows_processed = 0
archive_bytes = 0
while True:
summary = bundle_deleter.delete_batch(
tenant_ids=parsed_tenant_ids,
target_year=target_year,
target_month=target_month_number,
after_catalog_id=catalog_cursor,
limit=limit,
shard=shard,
)
_echo_bundle_archive_operation_summary(summary, dry_run=dry_run)
if summary.bundles_failed:
failed_result = next((result for result in summary.results if not result.success), None)
failed_catalog_id = failed_result.catalog_id if failed_result is not None else "unknown"
page_resume_cursor = summary.preview_next_catalog_id if dry_run else summary.next_catalog_id
resume_cursor = page_resume_cursor or catalog_cursor
if dry_run:
cursor_details = (
f"preview_after_catalog_id={resume_cursor or 'none'} "
f"destructive_retry_after_catalog_id={initial_catalog_cursor or 'none'}"
)
else:
cursor_details = f"resume_after_catalog_id={resume_cursor or 'none'}"
click.echo(
click.style(
f"Delete stopped: target_month={target_month} shard={shard or 'all'} "
f"failed_catalog_id={failed_catalog_id} "
f"{cursor_details}",
fg="red",
)
)
raise click.exceptions.Exit(1)
if not all_pages:
break
if summary.bundles_processed == 0:
break
pages_processed += 1
bundles_succeeded += summary.bundles_succeeded
runs_processed += summary.runs_processed
rows_processed += summary.rows_processed
archive_bytes += summary.archive_bytes
next_catalog_id = summary.preview_next_catalog_id if dry_run else summary.next_catalog_id
if next_catalog_id is None or (catalog_cursor is not None and next_catalog_id <= catalog_cursor):
click.echo(
click.style(
f"Delete cursor did not advance: target_month={target_month} shard={shard or 'all'} "
f"after_catalog_id={catalog_cursor or 'none'} next_catalog_id={next_catalog_id or 'none'}",
fg="red",
)
)
raise click.exceptions.Exit(1)
catalog_cursor = next_catalog_id
if all_pages:
final_cursor_label = "preview_final_catalog_id" if dry_run else "final_catalog_id"
click.echo(
click.style(
f"Delete all-pages completed successfully. target_month={target_month} shard={shard or 'all'} "
f"pages={pages_processed} bundles_success={bundles_succeeded} runs={runs_processed} "
f"rows={rows_processed} archive_bytes={archive_bytes} "
f"{final_cursor_label}={catalog_cursor or 'none'}",
fg="green",
)
)
summary = bundle_deleter.delete_batch(
tenant_ids=parsed_tenant_ids,
start_date=start_from,
end_date=end_before,
limit=limit,
)
_echo_bundle_archive_operation_summary(summary)
def _find_orphaned_draft_variables(batch_size: int = 1000) -> list[str]:
+4 -7
View File
@@ -188,26 +188,23 @@ where sites.id is null limit 1000"""
if app_id in failed_app_ids:
continue
session = db.session()
try:
app = session.scalar(select(App).where(App.id == app_id))
app = db.session.scalar(select(App).where(App.id == app_id))
if not app:
logger.info("App %s not found", app_id)
continue
tenant = session.get(Tenant, app.tenant_id)
tenant = app.tenant
if tenant:
accounts = tenant.get_accounts(session=session)
accounts = tenant.get_accounts()
if not accounts:
logger.info("Fix failed for app %s", app.id)
continue
account = accounts[0]
logger.info("Fixing missing site for app %s", app.id)
app_was_created.send(app, account=account, session=session)
session.commit()
app_was_created.send(app, account=account)
except Exception:
session.rollback()
failed_app_ids.append(app_id)
click.echo(click.style(f"Failed to fix missing site for app {app_id}", fg="red"))
logger.exception("Failed to fix app related site missing issue, app_id: %s", app_id)
+6 -12
View File
@@ -1,11 +1,10 @@
import json
from typing import cast
import click
from flask import current_app
from sqlalchemy import select
from sqlalchemy.exc import SQLAlchemyError
from sqlalchemy.orm import Session, sessionmaker
from sqlalchemy.orm import sessionmaker
from configs import dify_config
from core.rag.datasource.vdb.vector_factory import Vector
@@ -102,8 +101,7 @@ def migrate_annotation_vector_database():
)
documents.append(document)
with Session(db.engine) as session:
vector = Vector(dataset, attributes=["doc_id", "annotation_id", "app_id"], session=session)
vector = Vector(dataset, attributes=["doc_id", "annotation_id", "app_id"])
click.echo(f"Migrating annotations for app: {app.id}.")
try:
@@ -178,7 +176,6 @@ def migrate_knowledge_vector_database():
VectorType.OCEANBASE,
}
page = 1
db_session = db.session()
while True:
try:
stmt = (
@@ -187,7 +184,7 @@ def migrate_knowledge_vector_database():
.order_by(Dataset.created_at.desc())
)
datasets = paginate_query(stmt, page=page, per_page=50, max_per_page=50, session=db_session)
datasets = paginate_query(stmt, page=page, per_page=50, max_per_page=50)
if not datasets.items:
break
except SQLAlchemyError:
@@ -230,8 +227,7 @@ def migrate_knowledge_vector_database():
index_struct_dict = {"type": vector_type, "vector_store": {"class_prefix": collection_name}}
dataset.index_struct = json.dumps(index_struct_dict)
with Session(db.engine) as session:
vector = Vector(dataset, session=session)
vector = Vector(dataset)
click.echo(f"Migrating dataset {dataset.id}.")
try:
@@ -278,7 +274,7 @@ def migrate_knowledge_vector_database():
},
)
if dataset_document.doc_form == IndexStructureType.PARENT_CHILD_INDEX:
child_chunks = segment.get_child_chunks(session=db_session)
child_chunks = segment.get_child_chunks()
if child_chunks:
child_documents = []
for child_chunk in child_chunks:
@@ -414,9 +410,7 @@ def old_metadata_migration():
.where(DatasetDocument.doc_metadata.is_not(None))
.order_by(DatasetDocument.created_at.desc())
)
documents = paginate_query(
stmt, page=page, per_page=50, max_per_page=50, session=cast(Session, db.session())
)
documents = paginate_query(stmt, page=page, per_page=50, max_per_page=50)
except SQLAlchemyError:
raise
if not documents:
-6
View File
@@ -14,12 +14,6 @@ class EnterpriseFeatureConfig(BaseSettings):
default=False,
)
WEBAPP_PUBLIC_ACCESS_ENABLED: bool = Field(
description="Whether admins are allowed to set a webapp's access mode to public (anyone with the link, "
"no auth). Disable in security-sensitive on-prem deployments.",
default=True,
)
CAN_REPLACE_LOGO: bool = Field(
description="Allow customization of the enterprise logo.",
default=False,
-2
View File
@@ -1,6 +1,5 @@
from configs.extra.agent_backend_config import AgentBackendConfig
from configs.extra.archive_config import ArchiveStorageConfig
from configs.extra.knowledge_fs_config import KnowledgeFSConfig
from configs.extra.notion_config import NotionConfig
from configs.extra.sentry_config import SentryConfig
@@ -9,7 +8,6 @@ class ExtraServiceConfig(
# place the configs in alphabet order
AgentBackendConfig,
ArchiveStorageConfig,
KnowledgeFSConfig,
NotionConfig,
SentryConfig,
):
+1 -16
View File
@@ -1,4 +1,4 @@
from pydantic import Field, NonNegativeFloat, NonNegativeInt, PositiveFloat
from pydantic import Field, NonNegativeFloat
from pydantic_settings import BaseSettings
@@ -22,21 +22,6 @@ class AgentBackendConfig(BaseSettings):
default="success",
)
AGENT_BACKEND_STREAM_READ_TIMEOUT_SECONDS: PositiveFloat = Field(
description="Read timeout for one Agent backend SSE connection.",
default=30,
)
AGENT_BACKEND_STREAM_MAX_RECONNECTS: NonNegativeInt = Field(
description="Maximum Agent backend SSE reconnects before failing the run.",
default=3,
)
AGENT_BACKEND_RUN_TIMEOUT_SECONDS: PositiveFloat = Field(
description="Total deadline for one Agent backend run event stream.",
default=1200,
)
AGENT_SHELL_ENABLED: bool = Field(
description=(
"Inject the dify.shell layer (sandboxed bash workspace) into Agent runs. "
-64
View File
@@ -1,64 +0,0 @@
"""Configuration for the optional KnowledgeFS Console bridge."""
from urllib.parse import urlsplit
from pydantic import Field, PositiveFloat, SecretStr, field_validator, model_validator
from pydantic_settings import BaseSettings
class KnowledgeFSConfig(BaseSettings):
"""Server-only settings for the KnowledgeFS production connection."""
KNOWLEDGE_FS_ENABLED: bool = Field(
default=False,
description="Enable the private KnowledgeFS Console bridge.",
)
KNOWLEDGE_FS_BASE_URL: str | None = Field(default=None, description="KnowledgeFS gateway base URL.")
KNOWLEDGE_FS_JWT_SECRET: SecretStr | None = Field(
default=None,
min_length=32,
description="Shared secret used to sign short-lived KnowledgeFS service JWTs.",
)
KNOWLEDGE_FS_SSE_READ_TIMEOUT_SECONDS: PositiveFloat = Field(default=300.0, le=3600.0, allow_inf_nan=False)
KNOWLEDGE_FS_TIMEOUT_SECONDS: PositiveFloat = Field(default=10.0, le=60.0, allow_inf_nan=False)
@field_validator(
"KNOWLEDGE_FS_BASE_URL",
"KNOWLEDGE_FS_JWT_SECRET",
mode="before",
)
@classmethod
def normalize_optional_string(cls, value: object) -> object:
if isinstance(value, SecretStr):
normalized = value.get_secret_value().strip()
return SecretStr(normalized) if normalized else None
if isinstance(value, str):
normalized = value.strip()
return normalized or None
return value
@field_validator("KNOWLEDGE_FS_BASE_URL")
@classmethod
def validate_base_url(cls, value: str | None) -> str | None:
if value is None:
return None
parsed = urlsplit(value)
if parsed.scheme not in {"http", "https"} or not parsed.netloc:
raise ValueError("KNOWLEDGE_FS_BASE_URL must be an absolute HTTP(S) URL")
try:
_ = parsed.port
except ValueError as exc:
raise ValueError("KNOWLEDGE_FS_BASE_URL must include a valid port") from exc
if parsed.username or parsed.password or parsed.query or parsed.fragment:
raise ValueError("KNOWLEDGE_FS_BASE_URL must not include credentials, query, or fragment")
return value.rstrip("/")
@model_validator(mode="after")
def validate_enabled_connection(self) -> "KnowledgeFSConfig":
if not self.KNOWLEDGE_FS_ENABLED:
return self
if bool(self.KNOWLEDGE_FS_BASE_URL) != bool(self.KNOWLEDGE_FS_JWT_SECRET):
raise ValueError("KNOWLEDGE_FS_BASE_URL and KNOWLEDGE_FS_JWT_SECRET must be configured together")
if not self.KNOWLEDGE_FS_BASE_URL:
raise ValueError("KnowledgeFS connection settings are required when the integration is enabled")
return self
+1 -79
View File
@@ -1,4 +1,4 @@
from datetime import datetime, timedelta
from datetime import timedelta
from enum import StrEnum
from typing import Literal
@@ -11,7 +11,6 @@ from pydantic import (
PositiveFloat,
PositiveInt,
computed_field,
field_validator,
)
from pydantic_settings import BaseSettings
@@ -276,63 +275,6 @@ class PluginConfig(BaseSettings):
default=50 * 1024 * 1024,
)
NEW_USER_DEFAULT_PLUGIN_IDS: str = Field(
description="Comma-separated marketplace plugin IDs whose latest versions are installed for new users",
default="",
)
@field_validator("PLUGIN_REMOTE_INSTALL_PORT", mode="before")
@classmethod
def _reject_host_port_shaped_plugin_remote_install_port(cls, v):
"""Reject ``host:port``-shaped values with an actionable hint.
``EXPOSE_PLUGIN_DEBUGGING_PORT`` is overloaded: it feeds both the
plugin_daemon ``ports:`` mapping (where ``127.0.0.1:5003`` is valid
compose syntax) and this integer app setting advertised in the console.
Without this guard a loopback bind spec crashloops the api container
with an opaque ``int_parsing`` traceback. See issue #39323.
"""
if isinstance(v, str) and ":" in v.strip():
raise ValueError(
"PLUGIN_REMOTE_INSTALL_PORT must be a bare port number, got "
f"{v!r}. A 'host:port' value usually means "
"EXPOSE_PLUGIN_DEBUGGING_PORT was set to a compose publish spec "
"like '127.0.0.1:5003'; bind loopback via a "
"docker-compose.override.yaml instead of overloading this var."
)
return v
@property
def NEW_USER_DEFAULT_PLUGIN_ID_LIST(self) -> list[str]:
return [item.strip() for item in self.NEW_USER_DEFAULT_PLUGIN_IDS.split(",") if item.strip()]
NEW_USER_DEFAULT_MODELS: str = Field(
description=("Comma-separated default models for new users in 'model_type:provider:model' format"),
default="",
)
@property
def NEW_USER_DEFAULT_MODEL_LIST(self) -> list[tuple[str, str, str]]:
default_models: list[tuple[str, str, str]] = []
configured_model_types: set[str] = set()
for item in self.NEW_USER_DEFAULT_MODELS.split(","):
if not item.strip():
continue
parts = tuple(part.strip() for part in item.split(":", 2))
if len(parts) != 3 or not all(parts):
raise ValueError("NEW_USER_DEFAULT_MODELS entries must use 'model_type:provider:model' format")
model_type, provider, model = parts
if model_type in configured_model_types:
raise ValueError(f"NEW_USER_DEFAULT_MODELS contains duplicate model type: {model_type}")
configured_model_types.add(model_type)
default_models.append((model_type, provider, model))
return default_models
class MarketplaceConfig(BaseSettings):
"""
@@ -842,11 +784,6 @@ class WorkflowConfig(BaseSettings):
default=500,
)
WORKFLOW_GENERATOR_NODE_BUILDER_MAX_WORKERS: PositiveInt = Field(
description="Maximum concurrent node-builder LLM calls per workflow generation request",
default=6,
)
WORKFLOW_MAX_EXECUTION_TIME: PositiveInt = Field(
description="Maximum execution time in seconds for a single workflow",
default=1200,
@@ -1160,16 +1097,6 @@ class HomepageConfig(BaseSettings):
default=True,
)
ENABLE_STEP_BY_STEP_TOUR: bool = Field(
description="Enable account-level Step-by-step Tour eligibility checks",
default=False,
)
STEP_BY_STEP_TOUR_ROLLOUT_STARTED_AT: datetime | None = Field(
description="UTC timestamp after which newly initialized accounts are eligible for Step-by-step Tour",
default=None,
)
class RagEtlConfig(BaseSettings):
"""
@@ -1497,11 +1424,6 @@ class LoginConfig(BaseSettings):
class AccountConfig(BaseSettings):
ENABLE_CHANGE_EMAIL: bool = Field(
description="whether users can change their email address",
default=True,
)
ACCOUNT_DELETION_TOKEN_EXPIRY_MINUTES: PositiveInt = Field(
description="Duration in minutes for which a account deletion token remains valid",
default=5,
+6 -13
View File
@@ -1,29 +1,27 @@
from typing import Any
from sqlalchemy import select
from sqlalchemy.orm import Session
from core.app.apps.agent_app.app_feature_projection import merge_agent_app_features
from core.app.apps.agent_app.app_variable_projection import agent_app_variables_to_user_input_form
from core.app.apps.agent_app.errors import AgentAppGeneratorError, AgentAppNotPublishedError
from extensions.ext_database import db
from models.agent import Agent, AgentConfigSnapshot, AgentStatus
from models.agent_config_entities import AgentSoulConfig
from models.model import App, load_annotation_reply_config
from models.model import App
def get_published_agent_app_feature_dict_and_user_input_form(
app_model: App,
*,
session: Session,
) -> tuple[dict[str, Any], list[dict[str, Any]]]:
"""Return public Agent App parameters backed by the published Agent Soul."""
app_model_config = app_model.app_model_config_with_session(session=session)
app_model_config = app_model.app_model_config
agent_id = app_model.bound_agent_id
if not agent_id:
raise AgentAppGeneratorError("Agent App has no bound Agent")
agent = session.scalar(
agent = db.session.scalar(
select(Agent)
.where(
Agent.tenant_id == app_model.tenant_id,
@@ -39,7 +37,7 @@ def get_published_agent_app_feature_dict_and_user_input_form(
if not agent.active_config_snapshot_id:
raise AgentAppNotPublishedError("Agent has not been published")
snapshot = session.scalar(
snapshot = db.session.scalar(
select(AgentConfigSnapshot)
.where(
AgentConfigSnapshot.tenant_id == app_model.tenant_id,
@@ -52,10 +50,5 @@ def get_published_agent_app_feature_dict_and_user_input_form(
raise AgentAppGeneratorError("Agent published version not found")
agent_soul = AgentSoulConfig.model_validate(snapshot.config_snapshot_dict)
annotation_reply = load_annotation_reply_config(session, app_model.id) if app_model_config else None
features_dict = merge_agent_app_features(
agent_soul=agent_soul,
app_model_config=app_model_config,
annotation_reply=annotation_reply,
)
features_dict = merge_agent_app_features(agent_soul=agent_soul, app_model_config=app_model_config)
return features_dict, agent_app_variables_to_user_input_form(agent_soul.app_variables)
+2 -4
View File
@@ -4,8 +4,7 @@ from collections.abc import Sequence
from dataclasses import dataclass
from typing import TYPE_CHECKING
from sqlalchemy.orm import Session
from extensions.ext_database import db
from services.enterprise import rbac_service as enterprise_rbac_service
if TYPE_CHECKING:
@@ -69,7 +68,6 @@ def resolve_app_access_filter(
tenant_id: str,
account_id: str,
*,
session: Session,
permissions: MyPermissionsResponse | None = None,
) -> AppAccessFilter:
"""Compute the RBAC app-access filter for ``account_id`` in ``tenant_id``.
@@ -79,7 +77,7 @@ def resolve_app_access_filter(
inner-API round trip; otherwise it is fetched here.
"""
if permissions is None:
permissions = enterprise_rbac_service.RBACService.MyPermissions.get(tenant_id, account_id, session=session)
permissions = enterprise_rbac_service.RBACService.MyPermissions.get(tenant_id, account_id, session=db.session())
whitelist_scope = enterprise_rbac_service.RBACService.AppAccess.whitelist_resources(tenant_id, account_id)
can_manage_own_apps = _MANAGE_OWN_APPS_PERMISSION_KEY in permissions.workspace.permission_keys
+5 -13
View File
@@ -2,10 +2,8 @@
`with_session` is an HTTP controller helper: it opens one SQLAlchemy session
for a Resource handler and injects it as the first argument after `self`.
Write handlers commit on success and roll back on failure. They use a regular
Session context so existing services may commit an intermediate unit and keep
using the same Session through SQLAlchemy's autobegin behavior. Pure read
handlers may opt out with `write=False`.
Handlers use a transaction by default so migrated write paths keep
commit/rollback handling; pure read handlers may opt out with `write=False`.
"""
from collections.abc import Callable
@@ -40,20 +38,14 @@ def with_session[T, **P, R](
) -> (
Callable[Concatenate[T, P], R] | Callable[[Callable[Concatenate[T, Session, P], R]], Callable[Concatenate[T, P], R]]
):
"""Inject a request-scoped session and finalize write handlers."""
"""Inject a request-scoped session, using a transaction only for write handlers."""
def decorator(view: Callable[Concatenate[T, Session, P], R]) -> Callable[Concatenate[T, P], R]:
@wraps(view)
def wrapper(self: T, *args: P.args, **kwargs: P.kwargs) -> R:
if write:
with session_factory.create_session() as session:
try:
result = view(self, session, *args, **kwargs)
session.commit()
return result
except Exception:
session.rollback() # noqa: no-new-controller-sqlalchemy decorator owns transaction rollback
raise
with session_factory.get_session_maker().begin() as session:
return view(self, session, *args, **kwargs)
with session_factory.create_session() as session:
return view(self, session, *args, **kwargs)
+7 -20
View File
@@ -10,7 +10,6 @@ from extensions.ext_database import db
from libs.login import current_account_with_tenant
from models.dataset import Dataset
from models.model import App
from services.agent.roster_service import AgentRosterService
from services.enterprise.rbac_service import RBACService
__all__ = ["RBACPermission", "RBACResourceScope", "enforce_rbac_access", "rbac_permission_required"]
@@ -52,7 +51,7 @@ def enforce_rbac_access(
check_resource_type = None if resource_type == RBACResourceScope.WORKSPACE else resource_type
resource_id = None
if resource_required and check_resource_type:
resource_id = _extract_resource_id(resource_type, tenant_id, path_args)
resource_id = _extract_resource_id(resource_type, path_args)
if _is_resource_owned_by_current_user(tenant_id, account_id, resource_type, resource_id):
return
allowed = RBACService.CheckAccess.check(
@@ -132,14 +131,11 @@ def _is_resource_owned_by_current_user(
return False
def _extract_resource_id(
resource_type: RBACResourceScope, tenant_id: str, path_args: dict[str, object] | None = None
) -> str:
def _extract_resource_id(resource_type: RBACResourceScope, path_args: dict[str, object] | None = None) -> str:
"""Extract the resource ID from matched path arguments.
Some legacy route classes use neutral names such as ``resource_id`` for
app/dataset resources, and Agent routes carry ``agent_id``, which is
resolved to the App backing that Agent.
app/dataset resources, and Agent App routes use ``agent_id`` as the app id.
Dataset endpoints behind a rag-pipeline route contain ``pipeline_id``
instead of ``dataset_id``. In that case we look up the associated
``Dataset`` row via ``Dataset.pipeline_id``.
@@ -150,19 +146,10 @@ def _extract_resource_id(
matched_args = {**view_args, **(path_args or {})}
if resource_type == RBACResourceScope.APP:
app_id = matched_args.get("app_id")
if app_id:
return str(app_id)
agent_id = matched_args.get("agent_id")
if agent_id:
authz_app_id = AgentRosterService(db.session).peek_authz_app_id(tenant_id=tenant_id, agent_id=str(agent_id))
return authz_app_id or str(agent_id)
resource_id = matched_args.get("resource_id")
if resource_id:
return str(resource_id)
raise ValueError("Missing app_id in request path")
app_id = matched_args.get("app_id") or matched_args.get("agent_id") or matched_args.get("resource_id")
if not app_id:
raise ValueError("Missing app_id in request path")
return str(app_id)
if resource_type == RBACResourceScope.DATASET:
dataset_id = matched_args.get("dataset_id") or matched_args.get("resource_id")
-4
View File
@@ -38,9 +38,7 @@ from . import (
feature,
human_input_form,
init_validate,
knowledge_fs_proxy,
notification,
onboarding,
ping,
setup,
spec,
@@ -197,7 +195,6 @@ __all__ = [
"human_input_form",
"init_validate",
"installed_app",
"knowledge_fs_proxy",
"load_balancing_config",
"login",
"mcp_server",
@@ -210,7 +207,6 @@ __all__ = [
"notification",
"oauth",
"oauth_server",
"onboarding",
"ops_trace",
"parameter",
"ping",
+5 -6
View File
@@ -1,21 +1,20 @@
from uuid import UUID
from sqlalchemy.orm import Session
from extensions.ext_database import db
from models.model import App
from services.agent.roster_service import AgentRosterService
def resolve_agent_app_model(*, session: Session, tenant_id: str, agent_id: UUID) -> App:
def resolve_agent_app_model(*, tenant_id: str, agent_id: UUID) -> App:
"""Resolve a roster Agent's public Agent App."""
return AgentRosterService(session).get_agent_app_model(tenant_id=tenant_id, agent_id=str(agent_id))
return AgentRosterService(db.session).get_agent_app_model(tenant_id=tenant_id, agent_id=str(agent_id))
def resolve_agent_runtime_app_model(*, session: Session, tenant_id: str, agent_id: UUID) -> App:
def resolve_agent_runtime_app_model(*, tenant_id: str, agent_id: UUID) -> App:
"""Resolve the App that backs an Agent runtime surface.
This accepts both roster Agent Apps and workflow-only inline Agents with a
hidden backing App.
"""
return AgentRosterService(session).get_agent_runtime_app_model(tenant_id=tenant_id, agent_id=str(agent_id))
return AgentRosterService(db.session).get_agent_runtime_app_model(tenant_id=tenant_id, agent_id=str(agent_id))
+59 -87
View File
@@ -2,11 +2,9 @@ from uuid import UUID
from flask import request
from flask_restx import Resource
from sqlalchemy.orm import Session
from werkzeug.exceptions import NotFound
from controllers.common.schema import query_params_from_model, register_response_schema_models, register_schema_models
from controllers.common.session import with_session
from controllers.console import console_ns
from controllers.console.app.wraps import get_app_model
from controllers.console.wraps import (
@@ -19,6 +17,7 @@ from controllers.console.wraps import (
with_current_tenant_id,
with_current_user_id,
)
from extensions.ext_database import db
from fields.agent_fields import (
AgentAppComposerResponse,
AgentComposerCandidatesResponse,
@@ -60,21 +59,20 @@ class WorkflowAgentComposerApi(Resource):
@setup_required
@login_required
@account_initialization_required
@get_app_model(mode=[AppMode.WORKFLOW, AppMode.ADVANCED_CHAT])
@with_current_user_id
@with_current_tenant_id
@with_session
@get_app_model(mode=[AppMode.WORKFLOW, AppMode.ADVANCED_CHAT])
def get(self, session: Session, tenant_id: str, account_id: str, app_model: App, node_id: str):
def get(self, tenant_id: str, account_id: str, app_model: App, node_id: str):
query = WorkflowAgentComposerQuery.model_validate(request.args.to_dict(flat=True))
return dump_response(
WorkflowAgentComposerResponse,
AgentComposerService.load_workflow_composer(
session=session,
tenant_id=tenant_id,
app_id=app_model.id,
node_id=node_id,
account_id=account_id,
snapshot_id=query.snapshot_id,
session=db.session(),
),
)
@@ -87,21 +85,20 @@ class WorkflowAgentComposerApi(Resource):
@account_initialization_required
@edit_permission_required
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_EDIT)
@get_app_model(mode=[AppMode.WORKFLOW, AppMode.ADVANCED_CHAT])
@with_current_user_id
@with_current_tenant_id
@with_session
@get_app_model(mode=[AppMode.WORKFLOW, AppMode.ADVANCED_CHAT])
def put(self, session: Session, tenant_id: str, account_id: str, app_model: App, node_id: str):
def put(self, tenant_id: str, account_id: str, app_model: App, node_id: str):
payload = ComposerSavePayload.model_validate(console_ns.payload or {})
return dump_response(
WorkflowAgentComposerResponse,
AgentComposerService.save_workflow_composer(
session=session,
tenant_id=tenant_id,
app_id=app_model.id,
node_id=node_id,
account_id=account_id,
payload=payload,
session=db.session(),
),
)
@@ -119,16 +116,14 @@ class WorkflowAgentComposerCopyFromRosterApi(Resource):
@account_initialization_required
@edit_permission_required
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_EDIT)
@get_app_model(mode=[AppMode.WORKFLOW, AppMode.ADVANCED_CHAT])
@with_current_user_id
@with_current_tenant_id
@with_session
@get_app_model(mode=[AppMode.WORKFLOW, AppMode.ADVANCED_CHAT])
def post(self, session: Session, tenant_id: str, account_id: str, app_model: App, node_id: str):
def post(self, tenant_id: str, account_id: str, app_model: App, node_id: str):
payload = WorkflowComposerCopyFromRosterPayload.model_validate(console_ns.payload or {})
return dump_response(
WorkflowAgentComposerResponse,
AgentComposerService.copy_workflow_composer_from_roster(
session=session,
tenant_id=tenant_id,
app_id=app_model.id,
node_id=node_id,
@@ -136,6 +131,7 @@ class WorkflowAgentComposerCopyFromRosterApi(Resource):
source_agent_id=payload.source_agent_id,
source_snapshot_id=payload.source_snapshot_id,
idempotency_key=payload.idempotency_key,
session=db.session(),
),
)
@@ -149,22 +145,19 @@ class WorkflowAgentComposerValidateApi(Resource):
@setup_required
@login_required
@account_initialization_required
@with_current_tenant_id
@with_session(write=False)
@get_app_model(mode=[AppMode.WORKFLOW, AppMode.ADVANCED_CHAT])
def post(self, session: Session, tenant_id: str, app_model: App, node_id: str):
@with_current_tenant_id
def post(self, tenant_id: str, app_model: App, node_id: str):
payload = ComposerSavePayload.model_validate(console_ns.payload or {})
ComposerConfigValidator.validate_publish_payload(payload)
AgentComposerService.validate_knowledge_datasets(
session=session, tenant_id=tenant_id, agent_soul=payload.agent_soul
)
AgentComposerService.validate_knowledge_datasets(tenant_id=tenant_id, agent_soul=payload.agent_soul)
findings = AgentComposerService.collect_validation_findings(
session=session,
tenant_id=tenant_id,
payload=payload,
agent_id=AgentComposerService.resolve_workflow_node_agent_id(
session=session, tenant_id=tenant_id, app_id=app_model.id, node_id=node_id
tenant_id=tenant_id, app_id=app_model.id, node_id=node_id, session=db.session()
),
session=db.session(),
)
return dump_response(AgentComposerValidateResponse, {"result": "success", "errors": [], **findings})
@@ -177,19 +170,18 @@ class WorkflowAgentComposerCandidatesApi(Resource):
@setup_required
@login_required
@account_initialization_required
@get_app_model(mode=[AppMode.WORKFLOW, AppMode.ADVANCED_CHAT])
@with_current_user_id
@with_current_tenant_id
@with_session(write=False)
@get_app_model(mode=[AppMode.WORKFLOW, AppMode.ADVANCED_CHAT])
def get(self, session: Session, tenant_id: str, current_user_id: str, app_model: App, node_id: str):
def get(self, tenant_id: str, current_user_id: str, app_model: App, node_id: str):
return dump_response(
AgentComposerCandidatesResponse,
AgentComposerService.get_workflow_candidates(
session=session,
tenant_id=tenant_id,
app_id=app_model.id,
node_id=node_id,
user_id=current_user_id,
session=db.session(),
),
)
@@ -201,10 +193,9 @@ class WorkflowAgentComposerImpactApi(Resource):
@setup_required
@login_required
@account_initialization_required
@with_current_tenant_id
@with_session(write=False)
@get_app_model(mode=[AppMode.WORKFLOW, AppMode.ADVANCED_CHAT])
def post(self, session: Session, tenant_id: str, app_model: App, node_id: str):
@with_current_tenant_id
def post(self, tenant_id: str, app_model: App, node_id: str):
payload = ComposerSavePayload.model_validate(console_ns.payload or {})
current_snapshot_id = payload.binding.current_snapshot_id if payload.binding else None
if not current_snapshot_id:
@@ -214,7 +205,7 @@ class WorkflowAgentComposerImpactApi(Resource):
return dump_response(
AgentComposerImpactResponse,
AgentComposerService.calculate_impact(
session=session, tenant_id=tenant_id, current_snapshot_id=current_snapshot_id
tenant_id=tenant_id, current_snapshot_id=current_snapshot_id, session=db.session()
),
)
@@ -230,28 +221,26 @@ class WorkflowAgentComposerSaveToRosterApi(Resource):
@account_initialization_required
@edit_permission_required
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_EDIT)
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.AGENT_MANAGE, resource_required=False)
@get_app_model(mode=[AppMode.WORKFLOW, AppMode.ADVANCED_CHAT])
@with_current_user_id
@with_current_tenant_id
@with_session
@get_app_model(mode=[AppMode.WORKFLOW, AppMode.ADVANCED_CHAT])
def post(self, session: Session, tenant_id: str, account_id: str, app_model: App, node_id: str):
def post(self, tenant_id: str, account_id: str, app_model: App, node_id: str):
payload = ComposerSavePayload.model_validate(console_ns.payload or {})
return dump_response(
WorkflowAgentComposerResponse,
AgentComposerService.save_workflow_composer(
session=session,
tenant_id=tenant_id,
app_id=app_model.id,
node_id=node_id,
account_id=account_id,
payload=payload,
session=db.session(),
),
)
def _require_snippet_app_id(*, session: Session, tenant_id: str, snippet_id: UUID) -> str:
snippet = SnippetService(session=session).get_snippet_by_id(
def _require_snippet_app_id(*, tenant_id: str, snippet_id: UUID) -> str:
snippet = SnippetService(session=db.session()).get_snippet_by_id(
snippet_id=str(snippet_id),
tenant_id=tenant_id,
)
@@ -269,18 +258,17 @@ class SnippetAgentComposerApi(Resource):
@account_initialization_required
@with_current_user_id
@with_current_tenant_id
@with_session
def get(self, session: Session, tenant_id: str, account_id: str, snippet_id: UUID, node_id: str):
def get(self, tenant_id: str, account_id: str, snippet_id: UUID, node_id: str):
query = WorkflowAgentComposerQuery.model_validate(request.args.to_dict(flat=True))
return dump_response(
WorkflowAgentComposerResponse,
AgentComposerService.load_workflow_composer(
session=session,
tenant_id=tenant_id,
app_id=_require_snippet_app_id(session=session, tenant_id=tenant_id, snippet_id=snippet_id),
app_id=_require_snippet_app_id(tenant_id=tenant_id, snippet_id=snippet_id),
node_id=node_id,
account_id=account_id,
snapshot_id=query.snapshot_id,
session=db.session(),
),
)
@@ -295,18 +283,17 @@ class SnippetAgentComposerApi(Resource):
)
@with_current_user_id
@with_current_tenant_id
@with_session
def put(self, session: Session, tenant_id: str, account_id: str, snippet_id: UUID, node_id: str):
def put(self, tenant_id: str, account_id: str, snippet_id: UUID, node_id: str):
payload = ComposerSavePayload.model_validate(console_ns.payload or {})
return dump_response(
WorkflowAgentComposerResponse,
AgentComposerService.save_workflow_composer(
session=session,
tenant_id=tenant_id,
app_id=_require_snippet_app_id(session=session, tenant_id=tenant_id, snippet_id=snippet_id),
app_id=_require_snippet_app_id(tenant_id=tenant_id, snippet_id=snippet_id),
node_id=node_id,
account_id=account_id,
payload=payload,
session=db.session(),
),
)
@@ -326,20 +313,19 @@ class SnippetAgentComposerCopyFromRosterApi(Resource):
)
@with_current_user_id
@with_current_tenant_id
@with_session
def post(self, session: Session, tenant_id: str, account_id: str, snippet_id: UUID, node_id: str):
def post(self, tenant_id: str, account_id: str, snippet_id: UUID, node_id: str):
payload = WorkflowComposerCopyFromRosterPayload.model_validate(console_ns.payload or {})
return dump_response(
WorkflowAgentComposerResponse,
AgentComposerService.copy_workflow_composer_from_roster(
session=session,
tenant_id=tenant_id,
app_id=_require_snippet_app_id(session=session, tenant_id=tenant_id, snippet_id=snippet_id),
app_id=_require_snippet_app_id(tenant_id=tenant_id, snippet_id=snippet_id),
node_id=node_id,
account_id=account_id,
source_agent_id=payload.source_agent_id,
source_snapshot_id=payload.source_snapshot_id,
idempotency_key=payload.idempotency_key,
session=db.session(),
),
)
@@ -354,24 +340,21 @@ class SnippetAgentComposerValidateApi(Resource):
@login_required
@account_initialization_required
@with_current_tenant_id
@with_session(write=False)
def post(self, session: Session, tenant_id: str, snippet_id: UUID, node_id: str):
app_id = _require_snippet_app_id(session=session, tenant_id=tenant_id, snippet_id=snippet_id)
def post(self, tenant_id: str, snippet_id: UUID, node_id: str):
app_id = _require_snippet_app_id(tenant_id=tenant_id, snippet_id=snippet_id)
payload = ComposerSavePayload.model_validate(console_ns.payload or {})
ComposerConfigValidator.validate_publish_payload(payload)
AgentComposerService.validate_knowledge_datasets(
session=session, tenant_id=tenant_id, agent_soul=payload.agent_soul
)
AgentComposerService.validate_knowledge_datasets(tenant_id=tenant_id, agent_soul=payload.agent_soul)
findings = AgentComposerService.collect_validation_findings(
session=session,
tenant_id=tenant_id,
payload=payload,
agent_id=AgentComposerService.resolve_workflow_node_agent_id(
session=session,
tenant_id=tenant_id,
app_id=app_id,
node_id=node_id,
session=db.session(),
),
session=db.session(),
)
return dump_response(AgentComposerValidateResponse, {"result": "success", "errors": [], **findings})
@@ -386,16 +369,15 @@ class SnippetAgentComposerCandidatesApi(Resource):
@account_initialization_required
@with_current_user_id
@with_current_tenant_id
@with_session(write=False)
def get(self, session: Session, tenant_id: str, current_user_id: str, snippet_id: UUID, node_id: str):
def get(self, tenant_id: str, current_user_id: str, snippet_id: UUID, node_id: str):
return dump_response(
AgentComposerCandidatesResponse,
AgentComposerService.get_workflow_candidates(
session=session,
tenant_id=tenant_id,
app_id=_require_snippet_app_id(session=session, tenant_id=tenant_id, snippet_id=snippet_id),
app_id=_require_snippet_app_id(tenant_id=tenant_id, snippet_id=snippet_id),
node_id=node_id,
user_id=current_user_id,
session=db.session(),
),
)
@@ -408,9 +390,8 @@ class SnippetAgentComposerImpactApi(Resource):
@login_required
@account_initialization_required
@with_current_tenant_id
@with_session(write=False)
def post(self, session: Session, tenant_id: str, snippet_id: UUID, node_id: str):
_require_snippet_app_id(session=session, tenant_id=tenant_id, snippet_id=snippet_id)
def post(self, tenant_id: str, snippet_id: UUID, node_id: str):
_require_snippet_app_id(tenant_id=tenant_id, snippet_id=snippet_id)
payload = ComposerSavePayload.model_validate(console_ns.payload or {})
current_snapshot_id = payload.binding.current_snapshot_id if payload.binding else None
if not current_snapshot_id:
@@ -420,9 +401,9 @@ class SnippetAgentComposerImpactApi(Resource):
return dump_response(
AgentComposerImpactResponse,
AgentComposerService.calculate_impact(
session=session,
tenant_id=tenant_id,
current_snapshot_id=current_snapshot_id,
session=db.session(),
),
)
@@ -440,21 +421,19 @@ class SnippetAgentComposerSaveToRosterApi(Resource):
@rbac_permission_required(
RBACResourceScope.WORKSPACE, RBACPermission.SNIPPETS_CREATE_AND_MODIFY, resource_required=False
)
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.AGENT_MANAGE, resource_required=False)
@with_current_user_id
@with_current_tenant_id
@with_session
def post(self, session: Session, tenant_id: str, account_id: str, snippet_id: UUID, node_id: str):
def post(self, tenant_id: str, account_id: str, snippet_id: UUID, node_id: str):
payload = ComposerSavePayload.model_validate(console_ns.payload or {})
return dump_response(
WorkflowAgentComposerResponse,
AgentComposerService.save_workflow_composer(
session=session,
tenant_id=tenant_id,
app_id=_require_snippet_app_id(session=session, tenant_id=tenant_id, snippet_id=snippet_id),
app_id=_require_snippet_app_id(tenant_id=tenant_id, snippet_id=snippet_id),
node_id=node_id,
account_id=account_id,
payload=payload,
session=db.session(),
),
)
@@ -466,11 +445,10 @@ class AgentComposerApi(Resource):
@login_required
@account_initialization_required
@with_current_tenant_id
@with_session
def get(self, session: Session, tenant_id: str, agent_id: UUID):
def get(self, tenant_id: str, agent_id: UUID):
return dump_response(
AgentAppComposerResponse,
AgentComposerService.load_agent_composer(session=session, tenant_id=tenant_id, agent_id=str(agent_id)),
AgentComposerService.load_agent_composer(tenant_id=tenant_id, agent_id=str(agent_id), session=db.session()),
)
@console_ns.expect(console_ns.models[ComposerSavePayload.__name__])
@@ -480,20 +458,18 @@ class AgentComposerApi(Resource):
@account_initialization_required
@edit_permission_required
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_EDIT)
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.AGENT_MANAGE, resource_required=False)
@with_current_user_id
@with_current_tenant_id
@with_session
def put(self, session: Session, tenant_id: str, account_id: str, agent_id: UUID):
def put(self, tenant_id: str, account_id: str, agent_id: UUID):
payload = ComposerSavePayload.model_validate(console_ns.payload or {})
return dump_response(
AgentAppComposerResponse,
AgentComposerService.save_agent_composer(
session=session,
tenant_id=tenant_id,
agent_id=str(agent_id),
account_id=account_id,
payload=payload,
session=db.session(),
),
)
@@ -508,19 +484,16 @@ class AgentComposerValidateApi(Resource):
@login_required
@account_initialization_required
@with_current_tenant_id
@with_session
def post(self, session: Session, tenant_id: str, agent_id: UUID):
AgentComposerService.load_agent_composer(session=session, tenant_id=tenant_id, agent_id=str(agent_id))
def post(self, tenant_id: str, agent_id: UUID):
AgentComposerService.load_agent_composer(tenant_id=tenant_id, agent_id=str(agent_id), session=db.session())
payload = ComposerSavePayload.model_validate(console_ns.payload or {})
ComposerConfigValidator.validate_publish_payload(payload)
AgentComposerService.validate_knowledge_datasets(
session=session, tenant_id=tenant_id, agent_soul=payload.agent_soul
)
AgentComposerService.validate_knowledge_datasets(tenant_id=tenant_id, agent_soul=payload.agent_soul)
findings = AgentComposerService.collect_validation_findings(
session=session,
tenant_id=tenant_id,
payload=payload,
agent_id=str(agent_id),
session=db.session(),
)
return dump_response(AgentComposerValidateResponse, {"result": "success", "errors": [], **findings})
@@ -535,14 +508,13 @@ class AgentComposerCandidatesApi(Resource):
@account_initialization_required
@with_current_user_id
@with_current_tenant_id
@with_session(write=False)
def get(self, session: Session, tenant_id: str, current_user_id: str, agent_id: UUID):
def get(self, tenant_id: str, current_user_id: str, agent_id: UUID):
return dump_response(
AgentComposerCandidatesResponse,
AgentComposerService.get_agent_app_candidates(
session=session,
tenant_id=tenant_id,
agent_id=str(agent_id),
user_id=current_user_id,
session=db.session(),
),
)
+91 -195
View File
@@ -4,7 +4,6 @@ from flask import abort, request
from flask_restx import Resource
from pydantic import AliasChoices, BaseModel, Field, field_validator
from sqlalchemy import func, select
from sqlalchemy.orm import Session
from controllers.common.schema import (
query_params_from_model,
@@ -12,8 +11,8 @@ from controllers.common.schema import (
register_response_schema_models,
register_schema_models,
)
from controllers.common.session import with_session
from controllers.console import console_ns
from controllers.console.agent.app_helpers import resolve_agent_app_model, resolve_agent_runtime_app_model
from controllers.console.apikey import ApiKeyItem, ApiKeyList, BaseApiKeyListResource, BaseApiKeyResource
from controllers.console.app.app import (
APP_LIST_QUERY_ARRAY_FIELDS,
@@ -43,6 +42,7 @@ from controllers.console.wraps import (
with_current_tenant_id,
with_current_user,
)
from extensions.ext_database import db
from fields.agent_fields import (
AgentConfigDraftSummaryResponse,
AgentConfigSnapshotDetailResponse,
@@ -62,7 +62,7 @@ from libs.datetime_utils import parse_time_range
from libs.helper import dump_response
from libs.login import login_required
from models import Account
from models.agent import Agent, AgentConfigDraftType, AgentStatus
from models.agent import Agent, AgentStatus
from models.agent_config_entities import AgentSoulConfig
from models.enums import ApiTokenType
from models.model import ApiToken, App, IconType
@@ -177,7 +177,7 @@ class AgentLogsQuery(BaseModel):
default_factory=list,
description=(
"Filter by one or more source IDs, e.g. webapp:<app_id> "
"or workflow:<app_id>. Exact workflow:<app_id>:<workflow_id>:<version>:<node_id> IDs remain supported."
"or workflow:<app_id>:<workflow_id>:<version>:<node_id>"
),
)
sort_by: str = Field(default="updated_at", description="Sort by created_at or updated_at")
@@ -221,10 +221,7 @@ class AgentLogsQuery(BaseModel):
class AgentStatisticsQuery(BaseModel):
source: str | None = Field(
default=None,
description=(
"Filter by a structured webapp:<app_id> or workflow:<app_id> source ID. "
"Legacy invoke sources and exact workflow version/node source IDs remain supported."
),
description="Filter by all, console/explore, api/service-api, web-app, debugger, openapi, or trigger",
)
start: str | None = Field(default=None, description="Start date (YYYY-MM-DD HH:MM)")
end: str | None = Field(default=None, description="End date (YYYY-MM-DD HH:MM)")
@@ -244,7 +241,6 @@ class AgentAppPartial(GenericAppPartial):
debug_conversation_id: str | None = None
role: str | None = None
active_config_is_published: bool = False
reference_count: int | None = None
published_reference_count: int = 0
published_references: list[AgentAppPublishedReferenceResponse] = Field(default_factory=list)
@@ -266,13 +262,6 @@ class AgentDebugConversationRefreshResponse(BaseModel):
debug_conversation_message_count: int = 0
class AgentDebugConversationRefreshPayload(BaseModel):
draft_type: AgentConfigDraftType = Field(
default=AgentConfigDraftType.DEBUG_BUILD,
description="Agent draft surface whose conversation should be refreshed",
)
class AgentPublishPayload(BaseModel):
version_note: str | None = Field(default=None, description="Optional note for this published Agent version")
@@ -316,7 +305,6 @@ register_schema_models(
AgentAppCopyPayload,
AgentPublishPayload,
AgentBuildDraftCheckoutPayload,
AgentDebugConversationRefreshPayload,
ComposerSavePayload,
AgentApiStatusPayload,
AgentInviteOptionsQuery,
@@ -351,13 +339,11 @@ register_response_schema_models(
)
def _agent_roster_service(session: Session) -> AgentRosterService:
return AgentRosterService(session)
def _agent_roster_service() -> AgentRosterService:
return AgentRosterService(db.session)
def _serialize_agent_app_detail(
session: Session, app_model, *, current_user: Account, agent_id: str | None = None
) -> dict:
def _serialize_agent_app_detail(app_model, *, current_user: Account, agent_id: str | None = None) -> dict:
"""Serialize an Agent App detail using roster-only DTOs.
`/agent` responses are roster-shaped rather than raw app-shaped: `id`
@@ -367,19 +353,15 @@ def _serialize_agent_app_detail(
roster persona fields without widening the shared /apps detail schema.
"""
app_model = AppService().get_app(app_model, session=session)
app_model = AppService().get_app(app_model)
if FeatureService.get_system_features().webapp_auth.enabled:
app_setting = EnterpriseService.WebAppAuth.get_app_access_mode_by_id(app_id=str(app_model.id))
app_model.access_mode = app_setting.access_mode # type: ignore[attr-defined]
roster_service = _agent_roster_service(session)
payload = AgentAppDetailWithSite.model_validate(
app_model,
from_attributes=True,
context={"session": session},
).model_dump(mode="json")
roster_service = _agent_roster_service()
payload = AgentAppDetailWithSite.model_validate(app_model, from_attributes=True).model_dump(mode="json")
agent = (
session.scalar(
db.session.scalar(
select(Agent).where(
Agent.tenant_id == app_model.tenant_id,
Agent.id == agent_id,
@@ -400,8 +382,6 @@ def _serialize_agent_app_detail(
tenant_id=app_model.tenant_id,
agent_id=agent.id,
account_id=current_user.id,
draft_type=AgentConfigDraftType.DEBUG_BUILD,
commit=False,
)
message_count = roster_service.count_agent_app_debug_conversation_messages(
conversation_id=debug_conversation_id,
@@ -417,7 +397,7 @@ def _serialize_agent_app_detail(
return payload
def _serialize_agent_app_pagination(session: Session, app_pagination, *, tenant_id: str, current_user: Account) -> dict:
def _serialize_agent_app_pagination(app_pagination, *, tenant_id: str, current_user: Account) -> dict:
"""Serialize Agent App lists with roster-shaped items.
Each item starts from the shared App list shape, then drops
@@ -427,7 +407,7 @@ def _serialize_agent_app_pagination(session: Session, app_pagination, *, tenant_
"""
app_ids = [str(app.id) for app in app_pagination.items]
roster_service = _agent_roster_service(session)
roster_service = _agent_roster_service()
agents_by_app_id = roster_service.load_app_backing_agents_by_app_id(
tenant_id=tenant_id,
app_ids=app_ids,
@@ -440,21 +420,12 @@ def _serialize_agent_app_pagination(session: Session, app_pagination, *, tenant_
tenant_id=tenant_id,
agent_ids=[agent.id for agent in agents_by_app_id.values()],
)
reference_counts_by_agent_id = roster_service.load_reference_counts_by_agent_id(
tenant_id=tenant_id,
agent_ids=[agent.id for agent in agents_by_app_id.values()],
)
debug_conversation_ids_by_agent_id = roster_service.load_or_create_agent_app_debug_conversation_ids_by_agent_id(
tenant_id=tenant_id,
agents=list(agents_by_app_id.values()),
account_id=current_user.id,
draft_type=AgentConfigDraftType.DEBUG_BUILD,
)
payload = AgentAppPagination.model_validate(
app_pagination,
from_attributes=True,
context={"session": session},
).model_dump(mode="json")
payload = AgentAppPagination.model_validate(app_pagination, from_attributes=True).model_dump(mode="json")
for item in payload["data"]:
app_id = item["id"]
item.pop("bound_agent_id", None)
@@ -467,7 +438,6 @@ def _serialize_agent_app_pagination(session: Session, app_pagination, *, tenant_
item["debug_conversation_id"] = debug_conversation_ids_by_agent_id.get(agent.id)
item["role"] = agent.role or ""
item["active_config_is_published"] = active_config_is_published_by_agent_id.get(agent.id, False)
item["reference_count"] = reference_counts_by_agent_id.get(agent.id, 0)
published_references = published_references_by_agent_id.get(agent.id, [])
item["published_reference_count"] = len(published_references)
item["published_references"] = [
@@ -486,17 +456,13 @@ def _serialize_agent_app_pagination(session: Session, app_pagination, *, tenant_
)
def _resolve_agent_app_model(session: Session, *, tenant_id: str, agent_id: UUID) -> App:
return _agent_roster_service(session).get_agent_app_model(tenant_id=tenant_id, agent_id=str(agent_id))
def _resolve_agent_app_model(*, tenant_id: str, agent_id: UUID):
return resolve_agent_app_model(tenant_id=tenant_id, agent_id=agent_id)
def _resolve_agent_runtime_app_model(session: Session, *, tenant_id: str, agent_id: UUID) -> App:
return _agent_roster_service(session).get_agent_runtime_app_model(tenant_id=tenant_id, agent_id=str(agent_id))
def _agent_api_key_count(session: Session, app_id: str) -> int:
def _agent_api_key_count(app_id: str) -> int:
return (
session.scalar(
db.session.scalar(
select(func.count(ApiToken.id)).where(
ApiToken.type == ApiTokenType.APP,
ApiToken.app_id == app_id,
@@ -506,7 +472,7 @@ def _agent_api_key_count(session: Session, app_id: str) -> int:
)
def _serialize_agent_api_access(session: Session, app_model: App) -> dict:
def _serialize_agent_api_access(app_model: App) -> dict:
base_url = app_model.api_base_url
response = AgentApiAccessResponse(
enabled=bool(app_model.enable_api),
@@ -521,13 +487,13 @@ def _serialize_agent_api_access(session: Session, app_model: App) -> dict:
meta_endpoint=f"{base_url}/meta",
api_rpm=app_model.api_rpm or 0,
api_rph=app_model.api_rph or 0,
api_key_count=_agent_api_key_count(session, str(app_model.id)),
api_key_count=_agent_api_key_count(str(app_model.id)),
)
return response.model_dump(mode="json")
def _agent_observability_service(session: Session) -> AgentObservabilityService:
return AgentObservabilityService(session)
def _agent_observability_service() -> AgentObservabilityService:
return AgentObservabilityService(db.session)
def _parse_observability_time_range(start: str | None, end: str | None, account: Account):
@@ -552,11 +518,9 @@ class AgentAppListApi(Resource):
@setup_required
@login_required
@account_initialization_required
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.AGENT_MANAGE, resource_required=False)
@with_current_user
@with_current_tenant_id
@with_session
def get(self, session: Session, current_tenant_id: str, current_user: Account):
def get(self, current_tenant_id: str, current_user: Account):
args = query_params_from_request(AppListQuery, list_fields=APP_LIST_QUERY_ARRAY_FIELDS)
params = AppListParams(
page=args.page,
@@ -570,13 +534,12 @@ class AgentAppListApi(Resource):
status="normal",
)
app_pagination = AppService().get_paginate_apps(current_user.id, current_tenant_id, params, session)
app_pagination = AppService().get_paginate_apps(current_user.id, current_tenant_id, params, db.session())
if app_pagination is None:
empty = AgentAppPagination(page=args.page, limit=args.limit, total=0, has_more=False, data=[])
return empty.model_dump(mode="json")
return _serialize_agent_app_pagination(
session,
app_pagination,
tenant_id=current_tenant_id,
current_user=current_user,
@@ -590,11 +553,9 @@ class AgentAppListApi(Resource):
@login_required
@account_initialization_required
@edit_permission_required
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.AGENT_MANAGE, resource_required=False)
@with_current_user
@with_current_tenant_id
@with_session
def post(self, session: Session, current_tenant_id: str, current_user: Account):
def post(self, current_tenant_id: str, current_user: Account):
args = AgentAppCreatePayload.model_validate(console_ns.payload)
params = CreateAppParams(
name=args.name,
@@ -606,8 +567,8 @@ class AgentAppListApi(Resource):
icon_background=args.icon_background,
)
app = AppService().create_app(current_tenant_id, params, current_user, session=session)
return _serialize_agent_app_detail(session, app, current_user=current_user), 201
app = AppService().create_app(current_tenant_id, params, current_user, session=db.session())
return _serialize_agent_app_detail(app, current_user=current_user), 201
@console_ns.route("/agent/<uuid:agent_id>")
@@ -619,10 +580,9 @@ class AgentAppApi(Resource):
@enterprise_license_required
@with_current_user
@with_current_tenant_id
@with_session
def get(self, session: Session, tenant_id: str, current_user: Account, agent_id: UUID):
app_model = _resolve_agent_runtime_app_model(session, tenant_id=tenant_id, agent_id=agent_id)
return _serialize_agent_app_detail(session, app_model, current_user=current_user, agent_id=str(agent_id))
def get(self, tenant_id: str, current_user: Account, agent_id: UUID):
app_model = resolve_agent_runtime_app_model(tenant_id=tenant_id, agent_id=agent_id)
return _serialize_agent_app_detail(app_model, current_user=current_user, agent_id=str(agent_id))
@console_ns.expect(console_ns.models[AgentAppUpdatePayload.__name__])
@console_ns.response(200, "Agent app updated successfully", console_ns.models[AgentAppDetailWithSite.__name__])
@@ -632,12 +592,10 @@ class AgentAppApi(Resource):
@login_required
@account_initialization_required
@edit_permission_required
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.AGENT_MANAGE, resource_required=False)
@with_current_user
@with_current_tenant_id
@with_session
def put(self, session: Session, tenant_id: str, current_user: Account, agent_id: UUID):
app_model = _resolve_agent_app_model(session, tenant_id=tenant_id, agent_id=agent_id)
def put(self, tenant_id: str, current_user: Account, agent_id: UUID):
app_model = _resolve_agent_app_model(tenant_id=tenant_id, agent_id=agent_id)
args = AgentAppUpdatePayload.model_validate(console_ns.payload)
args_dict: AppService.ArgsDict = {
"name": args.name,
@@ -649,8 +607,8 @@ class AgentAppApi(Resource):
"max_active_requests": args.max_active_requests or 0,
"role": args.role,
}
updated = AppService().update_app(app_model, args_dict, session=session)
return _serialize_agent_app_detail(session, updated, current_user=current_user)
updated = AppService().update_app(app_model, args_dict, session=db.session())
return _serialize_agent_app_detail(updated, current_user=current_user)
@console_ns.response(204, "Agent app deleted successfully")
@console_ns.response(403, "Insufficient permissions")
@@ -658,27 +616,15 @@ class AgentAppApi(Resource):
@login_required
@account_initialization_required
@edit_permission_required
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.AGENT_MANAGE, resource_required=False)
@with_current_tenant_id
@with_session
def delete(self, session: Session, tenant_id: str, agent_id: UUID):
app_model = _resolve_agent_app_model(session, tenant_id=tenant_id, agent_id=agent_id)
AppService().delete_app(app_model, session=session)
def delete(self, tenant_id: str, agent_id: UUID):
app_model = _resolve_agent_app_model(tenant_id=tenant_id, agent_id=agent_id)
AppService().delete_app(app_model, session=db.session())
return "", 204
@console_ns.route("/agent/<uuid:agent_id>/debug-conversation/refresh")
class AgentDebugConversationRefreshApi(Resource):
@console_ns.expect(console_ns.models[AgentDebugConversationRefreshPayload.__name__])
@console_ns.doc(
params={
"payload": {
"in": "body",
"required": False,
"schema": {"$ref": f"#/components/schemas/{AgentDebugConversationRefreshPayload.__name__}"},
}
}
)
@console_ns.response(
200,
"Agent debug conversation refreshed",
@@ -691,14 +637,11 @@ class AgentDebugConversationRefreshApi(Resource):
@edit_permission_required
@with_current_user
@with_current_tenant_id
@with_session
def post(self, session: Session, tenant_id: str, current_user: Account, agent_id: UUID):
args = AgentDebugConversationRefreshPayload.model_validate(request.get_json(silent=True) or {})
debug_conversation_id = _agent_roster_service(session).refresh_agent_app_debug_conversation_id(
def post(self, tenant_id: str, current_user: Account, agent_id: UUID):
debug_conversation_id = _agent_roster_service().refresh_agent_app_debug_conversation_id(
tenant_id=tenant_id,
agent_id=str(agent_id),
account_id=current_user.id,
draft_type=args.draft_type,
)
return AgentDebugConversationRefreshResponse(
debug_conversation_id=debug_conversation_id,
@@ -716,18 +659,16 @@ class AgentPublishApi(Resource):
@login_required
@account_initialization_required
@edit_permission_required
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.AGENT_MANAGE, resource_required=False)
@with_current_user
@with_current_tenant_id
@with_session
def post(self, session: Session, tenant_id: str, current_user: Account, agent_id: UUID):
def post(self, tenant_id: str, current_user: Account, agent_id: UUID):
args = AgentPublishPayload.model_validate(console_ns.payload or {})
return AgentComposerService.publish_agent_app_draft(
session=session,
tenant_id=tenant_id,
agent_id=str(agent_id),
account_id=current_user.id,
version_note=args.version_note,
session=db.session(),
)
@@ -739,38 +680,34 @@ class AgentBuildDraftCheckoutApi(Resource):
@login_required
@account_initialization_required
@edit_permission_required
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.AGENT_MANAGE, resource_required=False)
@with_current_user
@with_current_tenant_id
@with_session
def post(self, session: Session, tenant_id: str, current_user: Account, agent_id: UUID):
def post(self, tenant_id: str, current_user: Account, agent_id: UUID):
args = AgentBuildDraftCheckoutPayload.model_validate(console_ns.payload or {})
return AgentComposerService.checkout_agent_app_build_draft(
session=session,
tenant_id=tenant_id,
agent_id=str(agent_id),
account_id=current_user.id,
force=args.force,
session=db.session(),
)
@console_ns.route("/agent/<uuid:agent_id>/build-draft")
class AgentBuildDraftApi(Resource):
@console_ns.response(200, "Agent build draft", console_ns.models[AgentBuildDraftResponse.__name__])
@console_ns.response(404, "Agent build draft not found")
@setup_required
@login_required
@account_initialization_required
@edit_permission_required
@with_current_user
@with_current_tenant_id
@with_session(write=False)
def get(self, session: Session, tenant_id: str, current_user: Account, agent_id: UUID):
def get(self, tenant_id: str, current_user: Account, agent_id: UUID):
return AgentComposerService.load_agent_app_build_draft(
session=session,
tenant_id=tenant_id,
agent_id=str(agent_id),
account_id=current_user.id,
session=db.session(),
)
@console_ns.expect(console_ns.models[ComposerSavePayload.__name__])
@@ -781,15 +718,14 @@ class AgentBuildDraftApi(Resource):
@edit_permission_required
@with_current_user
@with_current_tenant_id
@with_session
def put(self, session: Session, tenant_id: str, current_user: Account, agent_id: UUID):
def put(self, tenant_id: str, current_user: Account, agent_id: UUID):
payload = ComposerSavePayload.model_validate(console_ns.payload or {})
return AgentComposerService.save_agent_app_build_draft(
session=session,
tenant_id=tenant_id,
agent_id=str(agent_id),
account_id=current_user.id,
payload=payload,
session=db.session(),
)
@console_ns.response(200, "Agent build draft discarded", console_ns.models[AgentSimpleResultResponse.__name__])
@@ -799,13 +735,12 @@ class AgentBuildDraftApi(Resource):
@edit_permission_required
@with_current_user
@with_current_tenant_id
@with_session
def delete(self, session: Session, tenant_id: str, current_user: Account, agent_id: UUID):
def delete(self, tenant_id: str, current_user: Account, agent_id: UUID):
return AgentComposerService.discard_agent_app_build_draft(
session=session,
tenant_id=tenant_id,
agent_id=str(agent_id),
account_id=current_user.id,
session=db.session(),
)
@@ -816,16 +751,14 @@ class AgentBuildDraftApplyApi(Resource):
@login_required
@account_initialization_required
@edit_permission_required
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.AGENT_MANAGE, resource_required=False)
@with_current_user
@with_current_tenant_id
@with_session
def post(self, session: Session, tenant_id: str, current_user: Account, agent_id: UUID):
def post(self, tenant_id: str, current_user: Account, agent_id: UUID):
return AgentComposerService.apply_agent_app_build_draft(
session=session,
tenant_id=tenant_id,
agent_id=str(agent_id),
account_id=current_user.id,
session=db.session(),
)
@@ -839,13 +772,11 @@ class AgentAppCopyApi(Resource):
@login_required
@account_initialization_required
@edit_permission_required
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.AGENT_MANAGE, resource_required=False)
@with_current_user
@with_current_tenant_id
@with_session
def post(self, session: Session, tenant_id: str, current_user: Account, agent_id: UUID):
def post(self, tenant_id: str, current_user: Account, agent_id: UUID):
args = AgentAppCopyPayload.model_validate(console_ns.payload or {})
copied_app = _agent_roster_service(session).duplicate_agent_app(
copied_app = _agent_roster_service().duplicate_agent_app(
tenant_id=tenant_id,
agent_id=str(agent_id),
account=current_user,
@@ -856,7 +787,7 @@ class AgentAppCopyApi(Resource):
icon=args.icon,
icon_background=args.icon_background,
)
return _serialize_agent_app_detail(session, copied_app, current_user=current_user), 201
return _serialize_agent_app_detail(copied_app, current_user=current_user), 201
@console_ns.route("/agent/<uuid:agent_id>/api-access")
@@ -865,12 +796,10 @@ class AgentApiAccessApi(Resource):
@setup_required
@login_required
@account_initialization_required
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.AGENT_MANAGE, resource_required=False)
@with_current_tenant_id
@with_session(write=False)
def get(self, session: Session, tenant_id: str, agent_id: UUID):
app_model = _resolve_agent_app_model(session, tenant_id=tenant_id, agent_id=agent_id)
return _serialize_agent_api_access(session, app_model)
def get(self, tenant_id: str, agent_id: UUID):
app_model = _resolve_agent_app_model(tenant_id=tenant_id, agent_id=agent_id)
return _serialize_agent_api_access(app_model)
@console_ns.route("/agent/<uuid:agent_id>/api-enable")
@@ -882,15 +811,13 @@ class AgentApiStatusApi(Resource):
@login_required
@is_admin_or_owner_required
@account_initialization_required
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.AGENT_MANAGE, resource_required=False)
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_RELEASE_AND_VERSION)
@with_current_tenant_id
@with_session
def post(self, session: Session, tenant_id: str, agent_id: UUID):
app_model = _resolve_agent_app_model(session, tenant_id=tenant_id, agent_id=agent_id)
def post(self, tenant_id: str, agent_id: UUID):
app_model = _resolve_agent_app_model(tenant_id=tenant_id, agent_id=agent_id)
args = AgentApiStatusPayload.model_validate(console_ns.payload)
app_model = AppService().update_app_api_status(app_model, args.enable_api, session=session)
return _serialize_agent_api_access(session, app_model)
app_model = AppService().update_app_api_status(app_model, args.enable_api, session=db.session())
return _serialize_agent_api_access(app_model)
@console_ns.route("/agent/<uuid:agent_id>/api-keys")
@@ -901,26 +828,19 @@ class AgentApiKeyListApi(BaseApiKeyListResource):
token_prefix = "app-"
@console_ns.response(200, "Agent service API keys", console_ns.models[ApiKeyList.__name__])
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.AGENT_MANAGE, resource_required=False)
@with_current_tenant_id
@with_session(write=False)
def get(self, session: Session, tenant_id: str, agent_id: UUID) -> dict[str, object]:
app_model = _resolve_agent_app_model(session, tenant_id=tenant_id, agent_id=agent_id)
return dump_response(ApiKeyList, self._get_api_key_list(str(app_model.id), tenant_id, session=session))
def get(self, tenant_id: str, agent_id: UUID) -> dict[str, object]:
app_model = _resolve_agent_app_model(tenant_id=tenant_id, agent_id=agent_id)
return dump_response(ApiKeyList, self._get_api_key_list(str(app_model.id), tenant_id))
@console_ns.response(201, "Agent service API key created", console_ns.models[ApiKeyItem.__name__])
@console_ns.response(400, "Maximum keys exceeded")
@with_current_tenant_id
@edit_permission_required
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.AGENT_MANAGE, resource_required=False)
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_RELEASE_AND_VERSION)
@with_session
def post(self, session: Session, tenant_id: str, agent_id: UUID) -> tuple[dict[str, object], int]:
app_model = _resolve_agent_app_model(session, tenant_id=tenant_id, agent_id=agent_id)
return dump_response(
ApiKeyItem,
self._create_api_key(str(app_model.id), tenant_id, session=session),
), 201
def post(self, tenant_id: str, agent_id: UUID) -> tuple[dict[str, object], int]:
app_model = _resolve_agent_app_model(tenant_id=tenant_id, agent_id=agent_id)
return dump_response(ApiKeyItem, self._create_api_key(str(app_model.id), tenant_id)), 201
@console_ns.route("/agent/<uuid:agent_id>/api-keys/<uuid:api_key_id>")
@@ -932,19 +852,10 @@ class AgentApiKeyApi(BaseApiKeyResource):
@console_ns.response(204, "Agent service API key deleted")
@with_current_user
@with_current_tenant_id
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.AGENT_MANAGE, resource_required=False)
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_RELEASE_AND_VERSION)
@with_session
def delete(
self,
session: Session,
tenant_id: str,
current_user: Account,
agent_id: UUID,
api_key_id: UUID,
) -> tuple[str, int]:
app_model = _resolve_agent_app_model(session, tenant_id=tenant_id, agent_id=agent_id)
self._delete_api_key(str(app_model.id), str(api_key_id), tenant_id, current_user, session=session)
def delete(self, tenant_id: str, current_user: Account, agent_id: UUID, api_key_id: UUID) -> tuple[str, int]:
app_model = _resolve_agent_app_model(tenant_id=tenant_id, agent_id=agent_id)
self._delete_api_key(str(app_model.id), str(api_key_id), tenant_id, current_user)
return "", 204
@@ -956,12 +867,11 @@ class AgentInviteOptionsApi(Resource):
@login_required
@account_initialization_required
@with_current_tenant_id
@with_session(write=False)
def get(self, session: Session, tenant_id: str):
def get(self, tenant_id: str):
query = AgentInviteOptionsQuery.model_validate(request.args.to_dict(flat=True))
return dump_response(
AgentInviteOptionsResponse,
_agent_roster_service(session).list_invite_options(
_agent_roster_service().list_invite_options(
tenant_id=tenant_id,
page=query.page,
limit=query.limit,
@@ -978,19 +888,17 @@ class AgentLogsApi(Resource):
@setup_required
@login_required
@account_initialization_required
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.AGENT_MANAGE, resource_required=False)
@with_current_user
@with_current_tenant_id
@with_session(write=False)
def get(self, session: Session, tenant_id: str, current_user: Account, agent_id: UUID):
app_model = _resolve_agent_runtime_app_model(session, tenant_id=tenant_id, agent_id=agent_id)
def get(self, tenant_id: str, current_user: Account, agent_id: UUID):
app_model = resolve_agent_runtime_app_model(tenant_id=tenant_id, agent_id=agent_id)
query_data: dict[str, object] = dict(request.args.to_dict(flat=True))
query_data["sources"] = _query_values("sources", "source")
query_data["statuses"] = _query_values("statuses", "status")
query = AgentLogsQuery.model_validate(query_data)
start, end = _parse_observability_time_range(query.start, query.end, current_user)
try:
payload = _agent_observability_service(session).list_logs(
payload = _agent_observability_service().list_logs(
app=app_model,
agent_id=str(agent_id),
params=AgentLogQueryParams(
@@ -1017,19 +925,17 @@ class AgentLogMessagesApi(Resource):
@setup_required
@login_required
@account_initialization_required
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.AGENT_MANAGE, resource_required=False)
@with_current_user
@with_current_tenant_id
@with_session(write=False)
def get(self, session: Session, tenant_id: str, current_user: Account, agent_id: UUID, conversation_id: UUID):
app_model = _resolve_agent_runtime_app_model(session, tenant_id=tenant_id, agent_id=agent_id)
def get(self, tenant_id: str, current_user: Account, agent_id: UUID, conversation_id: UUID):
app_model = resolve_agent_runtime_app_model(tenant_id=tenant_id, agent_id=agent_id)
query_data: dict[str, object] = dict(request.args.to_dict(flat=True))
query_data["sources"] = _query_values("sources", "source")
query_data["statuses"] = _query_values("statuses", "status")
query = AgentLogsQuery.model_validate(query_data)
start, end = _parse_observability_time_range(query.start, query.end, current_user)
try:
payload = _agent_observability_service(session).list_log_messages(
payload = _agent_observability_service().list_log_messages(
app=app_model,
agent_id=str(agent_id),
conversation_id=str(conversation_id),
@@ -1056,13 +962,11 @@ class AgentLogSourcesApi(Resource):
@setup_required
@login_required
@account_initialization_required
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.AGENT_MANAGE, resource_required=False)
@with_current_user
@with_current_tenant_id
@with_session(write=False)
def get(self, session: Session, tenant_id: str, current_user: Account, agent_id: UUID):
app_model = _resolve_agent_runtime_app_model(session, tenant_id=tenant_id, agent_id=agent_id)
payload = _agent_observability_service(session).list_log_sources(app=app_model, agent_id=str(agent_id))
def get(self, tenant_id: str, current_user: Account, agent_id: UUID):
app_model = resolve_agent_runtime_app_model(tenant_id=tenant_id, agent_id=agent_id)
payload = _agent_observability_service().list_log_sources(app=app_model, agent_id=str(agent_id))
return dump_response(AgentLogSourceListResponse, payload)
@@ -1077,17 +981,15 @@ class AgentStatisticsSummaryApi(Resource):
@setup_required
@login_required
@account_initialization_required
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.AGENT_MANAGE, resource_required=False)
@with_current_user
@with_current_tenant_id
@with_session(write=False)
def get(self, session: Session, tenant_id: str, current_user: Account, agent_id: UUID):
app_model = _resolve_agent_runtime_app_model(session, tenant_id=tenant_id, agent_id=agent_id)
def get(self, tenant_id: str, current_user: Account, agent_id: UUID):
app_model = resolve_agent_runtime_app_model(tenant_id=tenant_id, agent_id=agent_id)
query = AgentStatisticsQuery.model_validate(request.args.to_dict(flat=True))
timezone = current_user.timezone or "UTC"
start, end = _parse_observability_time_range(query.start, query.end, current_user)
try:
payload = _agent_observability_service(session).get_statistics_summary(
payload = _agent_observability_service().get_statistics_summary(
app=app_model,
agent_id=str(agent_id),
params=AgentStatisticsQueryParams(source=query.source, start=start, end=end, timezone=timezone),
@@ -1103,13 +1005,11 @@ class AgentRosterVersionsApi(Resource):
@setup_required
@login_required
@account_initialization_required
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.AGENT_MANAGE, resource_required=False)
@with_current_tenant_id
@with_session(write=False)
def get(self, session: Session, tenant_id: str, agent_id: UUID):
def get(self, tenant_id: str, agent_id: UUID):
return dump_response(
AgentConfigSnapshotListResponse,
{"data": _agent_roster_service(session).list_agent_versions(tenant_id=tenant_id, agent_id=str(agent_id))},
{"data": _agent_roster_service().list_agent_versions(tenant_id=tenant_id, agent_id=str(agent_id))},
)
@@ -1119,13 +1019,11 @@ class AgentRosterVersionDetailApi(Resource):
@setup_required
@login_required
@account_initialization_required
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.AGENT_MANAGE, resource_required=False)
@with_current_tenant_id
@with_session(write=False)
def get(self, session: Session, tenant_id: str, agent_id: UUID, version_id: UUID):
def get(self, tenant_id: str, agent_id: UUID, version_id: UUID):
return dump_response(
AgentConfigSnapshotDetailResponse,
_agent_roster_service(session).get_agent_version_detail(
_agent_roster_service().get_agent_version_detail(
tenant_id=tenant_id,
agent_id=str(agent_id),
version_id=str(version_id),
@@ -1140,14 +1038,12 @@ class AgentRosterVersionRestoreApi(Resource):
@login_required
@account_initialization_required
@edit_permission_required
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.AGENT_MANAGE, resource_required=False)
@with_current_user
@with_current_tenant_id
@with_session
def post(self, session: Session, tenant_id: str, current_user: Account, agent_id: UUID, version_id: UUID):
def post(self, tenant_id: str, current_user: Account, agent_id: UUID, version_id: UUID):
return dump_response(
AgentConfigSnapshotRestoreResponse,
_agent_roster_service(session).restore_agent_version(
_agent_roster_service().restore_agent_version(
tenant_id=tenant_id,
agent_id=str(agent_id),
version_id=str(version_id),
+38 -97
View File
@@ -6,13 +6,12 @@ from flask_restx import Resource
from flask_restx._http import HTTPStatus
from pydantic import field_validator
from sqlalchemy import delete, func, select
from sqlalchemy.orm import Session
from sqlalchemy.orm import sessionmaker
from werkzeug.exceptions import Forbidden
from configs import dify_config
from controllers.common.schema import register_response_schema_models
from controllers.common.session import with_session
from controllers.console.app.wraps import agent_manage_required_for_agent_app
from extensions.ext_database import db
from fields.base import ResponseModel
from libs.helper import dump_response, to_timestamp
from libs.login import login_required
@@ -55,10 +54,11 @@ class ApiKeyList(ResponseModel):
register_response_schema_models(console_ns, ApiKeyItem, ApiKeyList)
def _get_resource(resource_id, tenant_id, resource_model, *, session: Session):
resource = session.execute(
select(resource_model).filter_by(id=resource_id, tenant_id=tenant_id)
).scalar_one_or_none()
def _get_resource(resource_id, tenant_id, resource_model):
with sessionmaker(db.engine).begin() as session:
resource = session.execute(
select(resource_model).filter_by(id=resource_id, tenant_id=tenant_id)
).scalar_one_or_none()
if resource is None:
flask_restx.abort(HTTPStatus.NOT_FOUND, message=f"{resource_model.__name__} not found.")
@@ -75,18 +75,14 @@ class BaseApiKeyListResource(Resource):
token_prefix: str | None = None
max_keys = 10
@with_session(write=False)
def get(self, session: Session, resource_id: str, current_tenant_id: str) -> dict[str, object]:
return dump_response(
ApiKeyList,
self._get_api_key_list(resource_id, current_tenant_id, session=session),
)
def get(self, resource_id: str, current_tenant_id: str) -> dict[str, object]:
return dump_response(ApiKeyList, self._get_api_key_list(resource_id, current_tenant_id))
def _get_api_key_list(self, resource_id: str, current_tenant_id: str, *, session: Session) -> ApiKeyList:
def _get_api_key_list(self, resource_id: str, current_tenant_id: str) -> ApiKeyList:
assert self.resource_id_field is not None, "resource_id_field must be set"
_get_resource(resource_id, current_tenant_id, self.resource_model, session=session)
keys = session.scalars(
_get_resource(resource_id, current_tenant_id, self.resource_model)
keys = db.session.scalars(
select(ApiToken).where(
ApiToken.type == self.resource_type, getattr(ApiToken, self.resource_id_field) == resource_id
)
@@ -94,18 +90,14 @@ class BaseApiKeyListResource(Resource):
return ApiKeyList.model_validate({"data": keys}, from_attributes=True)
@edit_permission_required
@with_session
def post(self, session: Session, resource_id: str, current_tenant_id: str) -> tuple[dict[str, object], int]:
return dump_response(
ApiKeyItem,
self._create_api_key(resource_id, current_tenant_id, session=session),
), 201
def post(self, resource_id: str, current_tenant_id: str) -> tuple[dict[str, object], int]:
return dump_response(ApiKeyItem, self._create_api_key(resource_id, current_tenant_id)), 201
def _create_api_key(self, resource_id: str, current_tenant_id: str, *, session: Session) -> ApiToken:
def _create_api_key(self, resource_id: str, current_tenant_id: str) -> ApiToken:
assert self.resource_id_field is not None, "resource_id_field must be set"
_get_resource(resource_id, current_tenant_id, self.resource_model, session=session)
_get_resource(resource_id, current_tenant_id, self.resource_model)
current_key_count: int = (
session.scalar(
db.session.scalar(
select(func.count(ApiToken.id)).where(
ApiToken.type == self.resource_type, getattr(ApiToken, self.resource_id_field) == resource_id
)
@@ -120,15 +112,15 @@ class BaseApiKeyListResource(Resource):
custom="max_keys_exceeded",
)
key = ApiToken.generate_api_key(self.token_prefix or "", 24, session=session)
key = ApiToken.generate_api_key(self.token_prefix or "", 24)
assert self.resource_type is not None, "resource_type must be set"
api_token = ApiToken()
setattr(api_token, self.resource_id_field, resource_id)
api_token.tenant_id = current_tenant_id
api_token.token = key
api_token.type = self.resource_type
session.add(api_token)
session.commit()
db.session.add(api_token)
db.session.commit()
return api_token
@@ -139,16 +131,10 @@ class BaseApiKeyResource(Resource):
resource_model: type | None = None
resource_id_field: str | None = None
@with_session
def delete(
self,
session: Session,
resource_id: str,
api_key_id: str,
current_tenant_id: str,
current_user: Account,
self, resource_id: str, api_key_id: str, current_tenant_id: str, current_user: Account
) -> tuple[str, int]:
self._delete_api_key(resource_id, api_key_id, current_tenant_id, current_user, session=session)
self._delete_api_key(resource_id, api_key_id, current_tenant_id, current_user)
return "", 204
def _delete_api_key(
@@ -157,16 +143,14 @@ class BaseApiKeyResource(Resource):
api_key_id: str,
current_tenant_id: str,
current_user: Account,
*,
session: Session,
) -> None:
assert self.resource_id_field is not None, "resource_id_field must be set"
_get_resource(resource_id, current_tenant_id, self.resource_model, session=session)
_get_resource(resource_id, current_tenant_id, self.resource_model)
if not dify_config.RBAC_ENABLED and not current_user.is_admin_or_owner:
raise Forbidden()
key = session.scalar(
key = db.session.scalar(
select(ApiToken)
.where(
getattr(ApiToken, self.resource_id_field) == resource_id,
@@ -184,8 +168,8 @@ class BaseApiKeyResource(Resource):
assert key is not None # nosec - for type checker only
ApiTokenCache.delete(key.token, key.type)
session.execute(delete(ApiToken).where(ApiToken.id == api_key_id))
session.commit()
db.session.execute(delete(ApiToken).where(ApiToken.id == api_key_id))
db.session.commit()
@console_ns.route("/apps/<uuid:resource_id>/api-keys")
@@ -195,14 +179,9 @@ class AppApiKeyListResource(BaseApiKeyListResource):
@console_ns.doc(params={"resource_id": "App ID"})
@console_ns.response(200, "API keys retrieved successfully", console_ns.models[ApiKeyList.__name__])
@with_current_tenant_id
@agent_manage_required_for_agent_app
@with_session(write=False)
def get(self, session: Session, current_tenant_id: str, resource_id: UUID) -> dict[str, object]:
def get(self, current_tenant_id: str, resource_id: UUID) -> dict[str, object]:
"""Get all API keys for an app"""
return dump_response(
ApiKeyList,
self._get_api_key_list(str(resource_id), current_tenant_id, session=session),
)
return dump_response(ApiKeyList, self._get_api_key_list(str(resource_id), current_tenant_id))
@console_ns.doc("create_app_api_key")
@console_ns.doc(description="Create a new API key for an app")
@@ -212,14 +191,9 @@ class AppApiKeyListResource(BaseApiKeyListResource):
@with_current_tenant_id
@edit_permission_required
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_RELEASE_AND_VERSION)
@agent_manage_required_for_agent_app
@with_session
def post(self, session: Session, current_tenant_id: str, resource_id: UUID) -> tuple[dict[str, object], int]:
def post(self, current_tenant_id: str, resource_id: UUID) -> tuple[dict[str, object], int]:
"""Create a new API key for an app"""
return dump_response(
ApiKeyItem,
self._create_api_key(str(resource_id), current_tenant_id, session=session),
), 201
return dump_response(ApiKeyItem, self._create_api_key(str(resource_id), current_tenant_id)), 201
resource_type = ApiTokenType.APP
resource_model = App
@@ -236,24 +210,11 @@ class AppApiKeyResource(BaseApiKeyResource):
@with_current_user
@with_current_tenant_id
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_RELEASE_AND_VERSION)
@agent_manage_required_for_agent_app
@with_session
def delete(
self,
session: Session,
current_tenant_id: str,
current_user: Account,
resource_id: UUID,
api_key_id: UUID,
self, current_tenant_id: str, current_user: Account, resource_id: UUID, api_key_id: UUID
) -> tuple[str, int]:
"""Delete an API key for an app"""
self._delete_api_key(
str(resource_id),
str(api_key_id),
current_tenant_id,
current_user,
session=session,
)
self._delete_api_key(str(resource_id), str(api_key_id), current_tenant_id, current_user)
return "", 204
resource_type = ApiTokenType.APP
@@ -268,13 +229,9 @@ class DatasetApiKeyListResource(BaseApiKeyListResource):
@console_ns.doc(params={"resource_id": "Dataset ID"})
@console_ns.response(200, "API keys retrieved successfully", console_ns.models[ApiKeyList.__name__])
@with_current_tenant_id
@with_session(write=False)
def get(self, session: Session, current_tenant_id: str, resource_id: UUID) -> dict[str, object]:
def get(self, current_tenant_id: str, resource_id: UUID) -> dict[str, object]:
"""Get all API keys for a dataset"""
return dump_response(
ApiKeyList,
self._get_api_key_list(str(resource_id), current_tenant_id, session=session),
)
return dump_response(ApiKeyList, self._get_api_key_list(str(resource_id), current_tenant_id))
@console_ns.doc("create_dataset_api_key")
@console_ns.doc(description="Create a new API key for a dataset")
@@ -284,13 +241,9 @@ class DatasetApiKeyListResource(BaseApiKeyListResource):
@with_current_tenant_id
@edit_permission_required
@rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_API_KEY_MANAGE)
@with_session
def post(self, session: Session, current_tenant_id: str, resource_id: UUID) -> tuple[dict[str, object], int]:
def post(self, current_tenant_id: str, resource_id: UUID) -> tuple[dict[str, object], int]:
"""Create a new API key for a dataset"""
return dump_response(
ApiKeyItem,
self._create_api_key(str(resource_id), current_tenant_id, session=session),
), 201
return dump_response(ApiKeyItem, self._create_api_key(str(resource_id), current_tenant_id)), 201
resource_type = ApiTokenType.DATASET
resource_model = Dataset
@@ -307,23 +260,11 @@ class DatasetApiKeyResource(BaseApiKeyResource):
@with_current_user
@with_current_tenant_id
@rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_API_KEY_MANAGE)
@with_session
def delete(
self,
session: Session,
current_tenant_id: str,
current_user: Account,
resource_id: UUID,
api_key_id: UUID,
self, current_tenant_id: str, current_user: Account, resource_id: UUID, api_key_id: UUID
) -> tuple[str, int]:
"""Delete an API key for a dataset"""
self._delete_api_key(
str(resource_id),
str(api_key_id),
current_tenant_id,
current_user,
session=session,
)
self._delete_api_key(str(resource_id), str(api_key_id), current_tenant_id, current_user)
return "", 204
resource_type = ApiTokenType.DATASET
+51 -71
View File
@@ -5,7 +5,6 @@ from flask import request
from flask_restx import Resource
from pydantic import BaseModel, Field, field_validator
from sqlalchemy import select
from sqlalchemy.orm import Session
from controllers.common.schema import (
query_params_from_model,
@@ -13,7 +12,6 @@ from controllers.common.schema import (
register_response_schema_models,
register_schema_models,
)
from controllers.common.session import with_session
from controllers.console import console_ns
from controllers.console.agent.app_helpers import resolve_agent_runtime_app_model
from controllers.console.app.wraps import get_app_model
@@ -26,6 +24,7 @@ from controllers.console.wraps import (
with_current_tenant_id,
with_current_user,
)
from extensions.ext_database import db
from fields.base import ResponseModel
from libs.helper import uuid_value
from libs.login import login_required
@@ -170,23 +169,23 @@ register_response_schema_models(
)
def _resolve_agent_id(session: Session, app_model: App, node_id: str | None) -> str | None:
def _resolve_agent_id(app_model: App, node_id: str | None) -> str | None:
if node_id and app_model.mode != AppMode.AGENT:
return AgentComposerService.resolve_workflow_node_agent_id(
session=session, tenant_id=app_model.tenant_id, app_id=app_model.id, node_id=node_id
tenant_id=app_model.tenant_id, app_id=app_model.id, node_id=node_id, session=db.session()
)
return app_model.bound_agent_id_with_session(session=session)
return app_model.bound_agent_id
def _agent_not_bound() -> tuple[dict[str, str], int]:
return {"code": "agent_not_bound", "message": "no agent is bound for this app/node"}, 400
def _upload_skill_for_app(*, session: Session, current_user: Account, app_model: App):
def _upload_skill_for_app(*, current_user: Account, app_model: App):
"""Upload one skill package and commit its normalized files into the agent drive."""
query = query_params_from_request(AgentDriveMutationQuery)
agent_id = _resolve_agent_id(session, app_model, query.node_id)
agent_id = _resolve_agent_id(app_model, query.node_id)
if not agent_id:
return _agent_not_bound()
if "file" not in request.files:
@@ -203,22 +202,22 @@ def _upload_skill_for_app(*, session: Session, current_user: Account, app_model:
tenant_id=app_model.tenant_id,
user_id=current_user.id,
agent_id=agent_id,
session=session,
session=db.session(),
)
except (SkillPackageError, AgentDriveError) as exc:
return {"code": exc.code, "message": exc.message}, exc.status_code
return result, 201
def _commit_drive_file_for_app(*, session: Session, current_user: Account, app_model: App, allow_node_id: bool = True):
def _commit_drive_file_for_app(*, current_user: Account, app_model: App, allow_node_id: bool = True):
query = query_params_from_request(AgentDriveMutationQuery)
node_id = query.node_id if allow_node_id else None
agent_id = _resolve_agent_id(session, app_model, node_id)
agent_id = _resolve_agent_id(app_model, node_id)
if not agent_id:
return _agent_not_bound()
payload = AgentDriveFilePayload.model_validate(console_ns.payload or {})
upload_file = session.scalar(
upload_file = db.session.scalar(
select(UploadFile).where(
UploadFile.id == payload.upload_file_id,
UploadFile.tenant_id == app_model.tenant_id,
@@ -242,7 +241,7 @@ def _commit_drive_file_for_app(*, session: Session, current_user: Account, app_m
value_owned_by_drive=True,
)
],
session=session,
session=db.session(),
)
except AgentDriveError as exc:
return {"code": exc.code, "message": exc.message}, exc.status_code
@@ -259,10 +258,10 @@ def _commit_drive_file_for_app(*, session: Session, current_user: Account, app_m
}, 201
def _delete_drive_file_for_app(*, session: Session, current_user: Account, app_model: App, allow_node_id: bool = True):
def _delete_drive_file_for_app(*, current_user: Account, app_model: App, allow_node_id: bool = True):
query = query_params_from_request(AgentDriveDeleteFileQuery)
node_id = query.node_id if allow_node_id else None
agent_id = _resolve_agent_id(session, app_model, node_id)
agent_id = _resolve_agent_id(app_model, node_id)
if not agent_id:
return _agent_not_bound()
try:
@@ -276,7 +275,7 @@ def _delete_drive_file_for_app(*, session: Session, current_user: Account, app_m
user_id=current_user.id,
agent_id=agent_id,
items=[DriveCommitItem(key=key, file_ref=None)],
session=session,
session=db.session(),
)
except AgentDriveError as exc:
return {"code": exc.code, "message": exc.message}, exc.status_code
@@ -284,12 +283,10 @@ def _delete_drive_file_for_app(*, session: Session, current_user: Account, app_m
return {"result": "success", "removed_keys": removed_keys}
def _delete_skill_for_app(
*, session: Session, current_user: Account, app_model: App, slug: str, allow_node_id: bool = True
):
def _delete_skill_for_app(*, current_user: Account, app_model: App, slug: str, allow_node_id: bool = True):
query = query_params_from_request(AgentDriveMutationQuery)
node_id = query.node_id if allow_node_id else None
agent_id = _resolve_agent_id(session, app_model, node_id)
agent_id = _resolve_agent_id(app_model, node_id)
if not agent_id:
return _agent_not_bound()
if "/" in slug or not slug.strip():
@@ -304,7 +301,7 @@ def _delete_skill_for_app(
DriveCommitItem(key=f"{slug}/SKILL.md", file_ref=None),
DriveCommitItem(key=f"{slug}/.DIFY-SKILL-FULL.zip", file_ref=None),
],
session=session,
session=db.session(),
)
except AgentDriveError as exc:
return {"code": exc.code, "message": exc.message}, exc.status_code
@@ -312,16 +309,16 @@ def _delete_skill_for_app(
return {"result": "success", "removed_keys": removed_keys}
def _infer_skill_tools_for_app(*, session: Session, app_model: App, slug: str):
def _infer_skill_tools_for_app(*, app_model: App, slug: str):
query = query_params_from_request(AgentDriveMutationQuery)
agent_id = _resolve_agent_id(session, app_model, query.node_id)
agent_id = _resolve_agent_id(app_model, query.node_id)
if not agent_id:
return _agent_not_bound()
if "/" in slug or not slug.strip():
return {"code": "drive_key_invalid", "message": "skill slug must be a single path segment"}, 400
try:
return SkillToolInferenceService().infer(
tenant_id=app_model.tenant_id, agent_id=agent_id, slug=slug, session=session
tenant_id=app_model.tenant_id, agent_id=agent_id, slug=slug, session=db.session()
)
except SkillToolInferenceError as exc:
return {"code": exc.code, "message": exc.message}, exc.status_code
@@ -339,13 +336,12 @@ class AgentLogApi(Resource):
@login_required
@account_initialization_required
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_VIEW_LAYOUT)
@with_session(write=False)
@get_app_model(mode=[AppMode.AGENT_CHAT])
def get(self, session: Session, app_model: App):
def get(self, app_model: App):
"""Get agent logs"""
args = AgentLogQuery.model_validate(request.args.to_dict(flat=True))
return AgentService.get_agent_logs(app_model, args.conversation_id, args.message_id, session)
return AgentService.get_agent_logs(app_model, args.conversation_id, args.message_id, db.session())
@console_ns.route("/agent/<uuid:agent_id>/skills/upload")
@@ -360,10 +356,9 @@ class AgentSkillUploadByAgentApi(Resource):
@account_initialization_required
@with_current_user
@with_current_tenant_id
@with_session
def post(self, session: Session, tenant_id: str, current_user: Account, agent_id: UUID):
app_model = resolve_agent_runtime_app_model(session=session, tenant_id=tenant_id, agent_id=agent_id)
return _upload_skill_for_app(session=session, current_user=current_user, app_model=app_model)
def post(self, tenant_id: str, current_user: Account, agent_id: UUID):
app_model = resolve_agent_runtime_app_model(tenant_id=tenant_id, agent_id=agent_id)
return _upload_skill_for_app(current_user=current_user, app_model=app_model)
@console_ns.route("/apps/<uuid:app_id>/agent/skills/upload")
@@ -383,12 +378,11 @@ class AgentSkillUploadApi(Resource):
@setup_required
@login_required
@account_initialization_required
@with_current_user
@with_session
@get_app_model(mode=_WORKFLOW_AGENT_DRIVE_APP_MODES)
def post(self, session: Session, current_user: Account, app_model: App):
@with_current_user
def post(self, current_user: Account, app_model: App):
"""Upload a Skill, validate it, and commit drive-backed skill files."""
return _upload_skill_for_app(session=session, current_user=current_user, app_model=app_model)
return _upload_skill_for_app(current_user=current_user, app_model=app_model)
@console_ns.route("/agent/<uuid:agent_id>/files")
@@ -405,12 +399,9 @@ class AgentDriveFilesByAgentApi(Resource):
@account_initialization_required
@with_current_user
@with_current_tenant_id
@with_session
def post(self, session: Session, tenant_id: str, current_user: Account, agent_id: UUID):
app_model = resolve_agent_runtime_app_model(session=session, tenant_id=tenant_id, agent_id=agent_id)
return _commit_drive_file_for_app(
session=session, current_user=current_user, app_model=app_model, allow_node_id=False
)
def post(self, tenant_id: str, current_user: Account, agent_id: UUID):
app_model = resolve_agent_runtime_app_model(tenant_id=tenant_id, agent_id=agent_id)
return _commit_drive_file_for_app(current_user=current_user, app_model=app_model, allow_node_id=False)
@console_ns.doc("delete_agent_drive_file_by_agent")
@console_ns.doc(description="Delete one Agent App drive file by key")
@@ -421,12 +412,9 @@ class AgentDriveFilesByAgentApi(Resource):
@account_initialization_required
@with_current_user
@with_current_tenant_id
@with_session
def delete(self, session: Session, tenant_id: str, current_user: Account, agent_id: UUID):
app_model = resolve_agent_runtime_app_model(session=session, tenant_id=tenant_id, agent_id=agent_id)
return _delete_drive_file_for_app(
session=session, current_user=current_user, app_model=app_model, allow_node_id=False
)
def delete(self, tenant_id: str, current_user: Account, agent_id: UUID):
app_model = resolve_agent_runtime_app_model(tenant_id=tenant_id, agent_id=agent_id)
return _delete_drive_file_for_app(current_user=current_user, app_model=app_model, allow_node_id=False)
@console_ns.route("/apps/<uuid:app_id>/agent/files")
@@ -441,12 +429,11 @@ class AgentDriveFilesApi(Resource):
@setup_required
@login_required
@account_initialization_required
@with_current_user
@with_session
@get_app_model(mode=_WORKFLOW_AGENT_DRIVE_APP_MODES)
def post(self, session: Session, current_user: Account, app_model: App):
@with_current_user
def post(self, current_user: Account, app_model: App):
"""ADD FILE: commit one uploaded file into the bound agent's drive."""
return _commit_drive_file_for_app(session=session, current_user=current_user, app_model=app_model)
return _commit_drive_file_for_app(current_user=current_user, app_model=app_model)
@console_ns.doc("delete_agent_drive_file")
@console_ns.doc(description="Delete one drive file by key via drive commit-null semantics")
@@ -455,11 +442,10 @@ class AgentDriveFilesApi(Resource):
@setup_required
@login_required
@account_initialization_required
@with_current_user
@with_session
@get_app_model(mode=_WORKFLOW_AGENT_DRIVE_APP_MODES)
def delete(self, session: Session, current_user: Account, app_model: App):
return _delete_drive_file_for_app(session=session, current_user=current_user, app_model=app_model)
@with_current_user
def delete(self, current_user: Account, app_model: App):
return _delete_drive_file_for_app(current_user=current_user, app_model=app_model)
@console_ns.route("/agent/<uuid:agent_id>/skills/<string:slug>")
@@ -473,12 +459,9 @@ class AgentSkillByAgentApi(Resource):
@account_initialization_required
@with_current_user
@with_current_tenant_id
@with_session
def delete(self, session: Session, tenant_id: str, current_user: Account, agent_id: UUID, slug: str):
app_model = resolve_agent_runtime_app_model(session=session, tenant_id=tenant_id, agent_id=agent_id)
return _delete_skill_for_app(
session=session, current_user=current_user, app_model=app_model, slug=slug, allow_node_id=False
)
def delete(self, tenant_id: str, current_user: Account, agent_id: UUID, slug: str):
app_model = resolve_agent_runtime_app_model(tenant_id=tenant_id, agent_id=agent_id)
return _delete_skill_for_app(current_user=current_user, app_model=app_model, slug=slug, allow_node_id=False)
@console_ns.route("/apps/<uuid:app_id>/agent/skills/<string:slug>")
@@ -496,11 +479,10 @@ class AgentSkillApi(Resource):
@setup_required
@login_required
@account_initialization_required
@with_current_user
@with_session
@get_app_model(mode=_WORKFLOW_AGENT_DRIVE_APP_MODES)
def delete(self, session: Session, current_user: Account, app_model: App, slug: str):
return _delete_skill_for_app(session=session, current_user=current_user, app_model=app_model, slug=slug)
@with_current_user
def delete(self, current_user: Account, app_model: App, slug: str):
return _delete_skill_for_app(current_user=current_user, app_model=app_model, slug=slug)
@console_ns.route("/agent/<uuid:agent_id>/skills/<string:slug>/infer-tools")
@@ -517,10 +499,9 @@ class AgentSkillInferToolsByAgentApi(Resource):
@login_required
@account_initialization_required
@with_current_tenant_id
@with_session(write=False)
def post(self, session: Session, tenant_id: str, agent_id: UUID, slug: str):
app_model = resolve_agent_runtime_app_model(session=session, tenant_id=tenant_id, agent_id=agent_id)
return _infer_skill_tools_for_app(session=session, app_model=app_model, slug=slug)
def post(self, tenant_id: str, agent_id: UUID, slug: str):
app_model = resolve_agent_runtime_app_model(tenant_id=tenant_id, agent_id=agent_id)
return _infer_skill_tools_for_app(app_model=app_model, slug=slug)
@console_ns.route("/apps/<uuid:app_id>/agent/skills/<string:slug>/infer-tools")
@@ -544,8 +525,7 @@ class AgentSkillInferToolsApi(Resource):
@setup_required
@login_required
@account_initialization_required
@with_session(write=False)
@get_app_model(mode=_WORKFLOW_AGENT_DRIVE_APP_MODES)
def post(self, session: Session, app_model: App, slug: str):
def post(self, app_model: App, slug: str):
"""Suggest CLI tools/env for a skill. Saving still goes through composer validation."""
return _infer_skill_tools_for_app(session=session, app_model=app_model, slug=slug)
return _infer_skill_tools_for_app(app_model=app_model, slug=slug)
@@ -9,13 +9,12 @@ from uuid import UUID
from flask_restx import Resource
from pydantic import Field
from sqlalchemy.orm import Session
from controllers.common.schema import register_response_schema_models
from controllers.common.session import with_session
from controllers.console import console_ns
from controllers.console.agent.app_helpers import resolve_agent_app_model
from controllers.console.wraps import account_initialization_required, setup_required, with_current_tenant_id
from extensions.ext_database import db
from fields.base import ResponseModel
from libs.login import login_required
from services.agent.roster_service import AgentRosterService
@@ -56,10 +55,9 @@ class AgentAppReferencingWorkflowsResource(Resource):
@login_required
@account_initialization_required
@with_current_tenant_id
@with_session(write=False)
def get(self, session: Session, tenant_id: str, agent_id: UUID):
app_model = resolve_agent_app_model(session=session, tenant_id=tenant_id, agent_id=agent_id)
workflows = AgentRosterService(session).list_workflows_referencing_app_agent(
def get(self, tenant_id: str, agent_id: UUID):
app_model = resolve_agent_app_model(tenant_id=tenant_id, agent_id=agent_id)
workflows = AgentRosterService(db.session).list_workflows_referencing_app_agent(
tenant_id=tenant_id, app_id=app_model.id
)
return AgentReferencingWorkflowsResponse(
@@ -13,11 +13,9 @@ from uuid import UUID
from flask_restx import Resource
from pydantic import BaseModel, Field
from sqlalchemy.orm import Session
from controllers.common.fields import SimpleResultResponse
from controllers.common.schema import register_response_schema_models, register_schema_models
from controllers.common.session import with_session
from controllers.console import console_ns
from controllers.console.agent.app_helpers import resolve_agent_runtime_app_model
from controllers.console.wraps import (
@@ -31,6 +29,7 @@ from controllers.console.wraps import (
with_current_user,
)
from events.app_event import app_model_config_was_updated
from extensions.ext_database import db
from libs.login import login_required
from models import Account
from models.agent_config_entities import (
@@ -86,23 +85,17 @@ class AgentAppFeatureConfigResource(Resource):
@account_initialization_required
@with_current_user
@with_current_tenant_id
@with_session
def post(self, session: Session, tenant_id: str, current_user: Account, agent_id: UUID):
app_model = resolve_agent_runtime_app_model(session=session, tenant_id=tenant_id, agent_id=agent_id)
def post(self, tenant_id: str, current_user: Account, agent_id: UUID):
app_model = resolve_agent_runtime_app_model(tenant_id=tenant_id, agent_id=agent_id)
args = AgentAppFeaturesPayload.model_validate(console_ns.payload or {})
new_app_model_config = AgentAppFeatureConfigService.update_features(
app_model=app_model,
account=current_user,
config=args.model_dump(exclude_none=True),
session=session,
session=db.session(),
)
app_model_config_was_updated.send(
app_model,
app_model_config=new_app_model_config,
session=session,
)
session.commit()
app_model_config_was_updated.send(app_model, app_model_config=new_app_model_config)
return SimpleResultResponse(result="success").model_dump(mode="json")
@@ -14,7 +14,6 @@ from dify_agent.client import DifyAgentClientError, DifyAgentHTTPError, DifyAgen
from flask import request
from flask_restx import Resource
from pydantic import BaseModel, Field
from sqlalchemy.orm import Session
from controllers.common.schema import (
query_params_from_model,
@@ -22,7 +21,6 @@ from controllers.common.schema import (
register_response_schema_models,
register_schema_models,
)
from controllers.common.session import with_session
from controllers.console import console_ns
from controllers.console.agent.app_helpers import resolve_agent_runtime_app_model
from controllers.console.app.wraps import get_app_model
@@ -155,9 +153,8 @@ class AgentAppSandboxInfoResource(Resource):
@login_required
@account_initialization_required
@with_current_tenant_id
@with_session(write=False)
def get(self, session: Session, tenant_id: str, agent_id: UUID):
app_model = resolve_agent_runtime_app_model(session=session, tenant_id=tenant_id, agent_id=agent_id)
def get(self, tenant_id: str, agent_id: UUID):
app_model = resolve_agent_runtime_app_model(tenant_id=tenant_id, agent_id=agent_id)
query = query_params_from_request(AgentSandboxInfoQuery)
try:
result = AgentAppSandboxService().get_info(
@@ -180,9 +177,8 @@ class AgentAppSandboxListResource(Resource):
@login_required
@account_initialization_required
@with_current_tenant_id
@with_session(write=False)
def get(self, session: Session, tenant_id: str, agent_id: UUID):
app_model = resolve_agent_runtime_app_model(session=session, tenant_id=tenant_id, agent_id=agent_id)
def get(self, tenant_id: str, agent_id: UUID):
app_model = resolve_agent_runtime_app_model(tenant_id=tenant_id, agent_id=agent_id)
query = query_params_from_request(AgentSandboxListQuery)
try:
result = AgentAppSandboxService().list_files(
@@ -206,9 +202,8 @@ class AgentAppSandboxReadResource(Resource):
@login_required
@account_initialization_required
@with_current_tenant_id
@with_session(write=False)
def get(self, session: Session, tenant_id: str, agent_id: UUID):
app_model = resolve_agent_runtime_app_model(session=session, tenant_id=tenant_id, agent_id=agent_id)
def get(self, tenant_id: str, agent_id: UUID):
app_model = resolve_agent_runtime_app_model(tenant_id=tenant_id, agent_id=agent_id)
query = query_params_from_request(AgentSandboxFileQuery)
try:
result = AgentAppSandboxService().read_file(
@@ -232,9 +227,8 @@ class AgentAppSandboxUploadResource(Resource):
@login_required
@account_initialization_required
@with_current_tenant_id
@with_session(write=False)
def post(self, session: Session, tenant_id: str, agent_id: UUID):
app_model = resolve_agent_runtime_app_model(session=session, tenant_id=tenant_id, agent_id=agent_id)
def post(self, tenant_id: str, agent_id: UUID):
app_model = resolve_agent_runtime_app_model(tenant_id=tenant_id, agent_id=agent_id)
payload = AgentSandboxUploadPayload.model_validate(request.get_json(silent=True) or {})
try:
result = AgentAppSandboxService().upload_file(
@@ -13,7 +13,6 @@ from uuid import UUID
from flask import Response, request, send_file, url_for
from flask_restx import Resource
from pydantic import BaseModel, Field
from sqlalchemy.orm import Session
from controllers.common.schema import (
query_params_from_model,
@@ -21,7 +20,6 @@ from controllers.common.schema import (
register_response_schema_models,
register_schema_models,
)
from controllers.common.session import with_session
from controllers.console import console_ns
from controllers.console.agent.app_helpers import resolve_agent_runtime_app_model
from controllers.console.app.wraps import get_app_model
@@ -35,6 +33,7 @@ from controllers.console.wraps import (
with_current_tenant_id,
with_current_user,
)
from extensions.ext_database import db
from fields.base import ResponseModel
from libs.login import login_required
from models.account import Account
@@ -100,7 +99,6 @@ class AgentConfigSkillItemResponse(ResponseModel):
id: str
name: str
file_id: str | None = None
is_missing: bool = False
description: str = ""
size: int | None = None
mime_type: str | None = None
@@ -111,7 +109,6 @@ class AgentConfigFileItemResponse(ResponseModel):
id: str
name: str
file_id: str | None = None
is_missing: bool = False
size: int | None = None
mime_type: str | None = None
hash: str | None = None
@@ -250,15 +247,15 @@ def _service() -> AgentConfigService:
return AgentConfigService()
def _resolve_agent_id(session: Session, app_model: App, node_id: str | None) -> str | None:
def _resolve_agent_id(app_model: App, node_id: str | None) -> str | None:
if node_id:
return AgentComposerService.resolve_workflow_node_agent_id(
session=session,
tenant_id=app_model.tenant_id,
app_id=app_model.id,
node_id=node_id,
session=db.session(),
)
return app_model.bound_agent_id_with_session(session=session)
return app_model.bound_agent_id
def _agent_not_bound() -> tuple[dict[str, object], int]:
@@ -278,7 +275,6 @@ def _json_response(data: Mapping[str, Any]) -> Response:
def _resolve_console_version(
*,
session: Session,
tenant_id: str,
agent_id: str,
account_id: str,
@@ -290,24 +286,26 @@ def _resolve_console_version(
try:
if draft_type == "debug_build":
state = AgentComposerService.load_agent_app_build_draft(
session=session,
tenant_id=tenant_id,
agent_id=agent_id,
account_id=account_id,
session=db.session(),
)
draft = state.get("draft") or {}
draft_id = draft.get("id")
if isinstance(draft_id, str) and draft_id:
return draft_id, AgentConfigVersionKind.BUILD_DRAFT
else:
state = AgentComposerService.load_agent_composer(session=session, tenant_id=tenant_id, agent_id=agent_id)
state = AgentComposerService.load_agent_composer(
tenant_id=tenant_id, agent_id=agent_id, session=db.session()
)
draft = state.get("draft") or {}
draft_id = draft.get("id")
if isinstance(draft_id, str) and draft_id:
# load_agent_composer creates the normal draft on first access.
# Config asset services use their own SQLAlchemy session, so the
# draft must be visible before we hand its id across that boundary.
session.commit()
db.session.commit()
return draft_id, AgentConfigVersionKind.DRAFT
except AgentVersionNotFoundError as exc:
raise AgentConfigServiceError(
@@ -324,7 +322,6 @@ def _resolve_console_version(
def _resolve_target(
*,
session: Session,
tenant_id: str,
agent_id: str,
account_id: str,
@@ -332,7 +329,6 @@ def _resolve_target(
draft_type: str | None,
) -> _ResolvedConsoleTarget:
resolved_version_id, version_kind = _resolve_console_version(
session=session,
tenant_id=tenant_id,
agent_id=agent_id,
account_id=account_id,
@@ -350,15 +346,13 @@ def _resolve_target(
def _resolve_agent_route_target(
*,
session: Session,
tenant_id: str,
agent_id: UUID,
current_user: Account,
query: AgentConfigByAgentQuery,
) -> _ResolvedConsoleTarget:
resolve_agent_runtime_app_model(session=session, tenant_id=tenant_id, agent_id=agent_id)
resolve_agent_runtime_app_model(tenant_id=tenant_id, agent_id=agent_id)
return _resolve_target(
session=session,
tenant_id=tenant_id,
agent_id=str(agent_id),
account_id=current_user.id,
@@ -369,16 +363,14 @@ def _resolve_agent_route_target(
def _resolve_app_route_target(
*,
session: Session,
app_model: App,
current_user: Account,
query: AgentConfigQuery,
) -> _ResolvedConsoleTarget | tuple[dict[str, object], int]:
agent_id = _resolve_agent_id(session, app_model, query.node_id)
agent_id = _resolve_agent_id(app_model, query.node_id)
if not agent_id:
return _agent_not_bound()
return _resolve_target(
session=session,
tenant_id=app_model.tenant_id,
agent_id=agent_id,
account_id=current_user.id,
@@ -389,7 +381,6 @@ def _resolve_app_route_target(
def _with_agent_route_target(
*,
session: Session,
tenant_id: str,
agent_id: UUID,
current_user: Account,
@@ -398,7 +389,6 @@ def _with_agent_route_target(
query = query_params_from_request(AgentConfigByAgentQuery)
try:
target = _resolve_agent_route_target(
session=session,
tenant_id=tenant_id,
agent_id=agent_id,
current_user=current_user,
@@ -411,14 +401,13 @@ def _with_agent_route_target(
def _with_app_route_target(
*,
session: Session,
app_model: App,
current_user: Account,
action: Callable[[_ResolvedConsoleTarget], Any],
) -> Any:
query = query_params_from_request(AgentConfigQuery)
try:
target = _resolve_app_route_target(session=session, app_model=app_model, current_user=current_user, query=query)
target = _resolve_app_route_target(app_model=app_model, current_user=current_user, query=query)
if isinstance(target, tuple):
return target
return action(target)
@@ -660,10 +649,8 @@ class AgentConfigManifestByAgentApi(Resource):
@account_initialization_required
@with_current_user
@with_current_tenant_id
@with_session
def get(self, session: Session, tenant_id: str, current_user: Account, agent_id: UUID):
def get(self, tenant_id: str, current_user: Account, agent_id: UUID):
return _with_agent_route_target(
session=session,
tenant_id=tenant_id,
agent_id=agent_id,
current_user=current_user,
@@ -679,13 +666,10 @@ class AgentConfigManifestApi(Resource):
@setup_required
@login_required
@account_initialization_required
@with_current_user
@with_session
@get_app_model(mode=_WORKFLOW_APP_MODES)
def get(self, session: Session, current_user: Account, app_model: App):
return _with_app_route_target(
session=session, app_model=app_model, current_user=current_user, action=_manifest_response
)
@with_current_user
def get(self, current_user: Account, app_model: App):
return _with_app_route_target(app_model=app_model, current_user=current_user, action=_manifest_response)
@console_ns.route("/agent/<uuid:agent_id>/config/skills/upload")
@@ -705,10 +689,8 @@ class AgentConfigSkillUploadByAgentApi(Resource):
@account_initialization_required
@with_current_user
@with_current_tenant_id
@with_session
def post(self, session: Session, tenant_id: str, current_user: Account, agent_id: UUID):
def post(self, tenant_id: str, current_user: Account, agent_id: UUID):
return _with_agent_route_target(
session=session,
tenant_id=tenant_id,
agent_id=agent_id,
current_user=current_user,
@@ -733,13 +715,10 @@ class AgentConfigSkillUploadApi(Resource):
@account_initialization_required
@edit_permission_required
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_EDIT)
@with_current_user
@with_session
@get_app_model(mode=_WORKFLOW_APP_MODES)
def post(self, session: Session, current_user: Account, app_model: App):
return _with_app_route_target(
session=session, app_model=app_model, current_user=current_user, action=_skill_upload_response
)
@with_current_user
def post(self, current_user: Account, app_model: App):
return _with_app_route_target(app_model=app_model, current_user=current_user, action=_skill_upload_response)
@console_ns.route("/agent/<uuid:agent_id>/config/skills")
@@ -752,10 +731,8 @@ class AgentConfigSkillsByAgentApi(Resource):
@account_initialization_required
@with_current_user
@with_current_tenant_id
@with_session
def get(self, session: Session, tenant_id: str, current_user: Account, agent_id: UUID):
def get(self, tenant_id: str, current_user: Account, agent_id: UUID):
return _with_agent_route_target(
session=session,
tenant_id=tenant_id,
agent_id=agent_id,
current_user=current_user,
@@ -771,13 +748,10 @@ class AgentConfigSkillsApi(Resource):
@setup_required
@login_required
@account_initialization_required
@with_current_user
@with_session
@get_app_model(mode=_WORKFLOW_APP_MODES)
def get(self, session: Session, current_user: Account, app_model: App):
return _with_app_route_target(
session=session, app_model=app_model, current_user=current_user, action=_skill_list_response
)
@with_current_user
def get(self, current_user: Account, app_model: App):
return _with_app_route_target(app_model=app_model, current_user=current_user, action=_skill_list_response)
@console_ns.route("/agent/<uuid:agent_id>/config/files")
@@ -790,10 +764,8 @@ class AgentConfigFilesByAgentApi(Resource):
@account_initialization_required
@with_current_user
@with_current_tenant_id
@with_session
def get(self, session: Session, tenant_id: str, current_user: Account, agent_id: UUID):
def get(self, tenant_id: str, current_user: Account, agent_id: UUID):
return _with_agent_route_target(
session=session,
tenant_id=tenant_id,
agent_id=agent_id,
current_user=current_user,
@@ -809,11 +781,9 @@ class AgentConfigFilesByAgentApi(Resource):
@account_initialization_required
@with_current_user
@with_current_tenant_id
@with_session
def post(self, session: Session, tenant_id: str, current_user: Account, agent_id: UUID):
def post(self, tenant_id: str, current_user: Account, agent_id: UUID):
payload = AgentConfigFileUploadPayload.model_validate(console_ns.payload or {})
return _with_agent_route_target(
session=session,
tenant_id=tenant_id,
agent_id=agent_id,
current_user=current_user,
@@ -829,13 +799,10 @@ class AgentConfigFilesApi(Resource):
@setup_required
@login_required
@account_initialization_required
@with_current_user
@with_session
@get_app_model(mode=_WORKFLOW_APP_MODES)
def get(self, session: Session, current_user: Account, app_model: App):
return _with_app_route_target(
session=session, app_model=app_model, current_user=current_user, action=_file_list_response
)
@with_current_user
def get(self, current_user: Account, app_model: App):
return _with_app_route_target(app_model=app_model, current_user=current_user, action=_file_list_response)
@console_ns.doc("upload_agent_config_file")
@console_ns.doc(params={"app_id": "Application ID", **query_params_from_model(AgentConfigQuery)})
@@ -846,13 +813,11 @@ class AgentConfigFilesApi(Resource):
@account_initialization_required
@edit_permission_required
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_EDIT)
@with_current_user
@with_session
@get_app_model(mode=_WORKFLOW_APP_MODES)
def post(self, session: Session, current_user: Account, app_model: App):
@with_current_user
def post(self, current_user: Account, app_model: App):
payload = AgentConfigFileUploadPayload.model_validate(console_ns.payload or {})
return _with_app_route_target(
session=session,
app_model=app_model,
current_user=current_user,
action=lambda target: _file_upload_response(target, payload),
@@ -871,10 +836,8 @@ class AgentConfigSkillInspectByAgentApi(Resource):
@account_initialization_required
@with_current_user
@with_current_tenant_id
@with_session
def get(self, session: Session, tenant_id: str, current_user: Account, agent_id: UUID, name: str):
def get(self, tenant_id: str, current_user: Account, agent_id: UUID, name: str):
return _with_agent_route_target(
session=session,
tenant_id=tenant_id,
agent_id=agent_id,
current_user=current_user,
@@ -892,12 +855,10 @@ class AgentConfigSkillInspectApi(Resource):
@setup_required
@login_required
@account_initialization_required
@with_current_user
@with_session
@get_app_model(mode=_WORKFLOW_APP_MODES)
def get(self, session: Session, current_user: Account, app_model: App, name: str):
@with_current_user
def get(self, current_user: Account, app_model: App, name: str):
return _with_app_route_target(
session=session,
app_model=app_model,
current_user=current_user,
action=lambda target: _skill_inspect_response(target, name),
@@ -922,12 +883,10 @@ class AgentConfigSkillFilePreviewByAgentApi(Resource):
@account_initialization_required
@with_current_user
@with_current_tenant_id
@with_session
def get(self, session: Session, tenant_id: str, current_user: Account, agent_id: UUID, name: str):
def get(self, tenant_id: str, current_user: Account, agent_id: UUID, name: str):
query = query_params_from_request(AgentConfigSkillFileByAgentQuery)
try:
target = _resolve_agent_route_target(
session=session,
tenant_id=tenant_id,
agent_id=agent_id,
current_user=current_user,
@@ -954,15 +913,12 @@ class AgentConfigSkillFilePreviewApi(Resource):
@setup_required
@login_required
@account_initialization_required
@with_current_user
@with_session
@get_app_model(mode=_WORKFLOW_APP_MODES)
def get(self, session: Session, current_user: Account, app_model: App, name: str):
@with_current_user
def get(self, current_user: Account, app_model: App, name: str):
query = query_params_from_request(AgentConfigSkillFileQuery)
try:
target = _resolve_app_route_target(
session=session, app_model=app_model, current_user=current_user, query=query
)
target = _resolve_app_route_target(app_model=app_model, current_user=current_user, query=query)
if isinstance(target, tuple):
return target
return _skill_file_preview_response(target, name, query.path)
@@ -982,10 +938,8 @@ class AgentConfigSkillDownloadByAgentApi(Resource):
@account_initialization_required
@with_current_user
@with_current_tenant_id
@with_session
def get(self, session: Session, tenant_id: str, current_user: Account, agent_id: UUID, name: str):
def get(self, tenant_id: str, current_user: Account, agent_id: UUID, name: str):
return _with_agent_route_target(
session=session,
tenant_id=tenant_id,
agent_id=agent_id,
current_user=current_user,
@@ -1003,12 +957,10 @@ class AgentConfigSkillDownloadApi(Resource):
@setup_required
@login_required
@account_initialization_required
@with_current_user
@with_session
@get_app_model(mode=_WORKFLOW_APP_MODES)
def get(self, session: Session, current_user: Account, app_model: App, name: str):
@with_current_user
def get(self, current_user: Account, app_model: App, name: str):
return _with_app_route_target(
session=session,
app_model=app_model,
current_user=current_user,
action=lambda target: _skill_download_response(target, name),
@@ -1031,12 +983,10 @@ class AgentConfigSkillFileDownloadByAgentApi(Resource):
@account_initialization_required
@with_current_user
@with_current_tenant_id
@with_session
def get(self, session: Session, tenant_id: str, current_user: Account, agent_id: UUID, name: str):
def get(self, tenant_id: str, current_user: Account, agent_id: UUID, name: str):
query = query_params_from_request(AgentConfigSkillFileByAgentQuery)
try:
target = _resolve_agent_route_target(
session=session,
tenant_id=tenant_id,
agent_id=agent_id,
current_user=current_user,
@@ -1067,15 +1017,12 @@ class AgentConfigSkillFileDownloadApi(Resource):
@setup_required
@login_required
@account_initialization_required
@with_current_user
@with_session
@get_app_model(mode=_WORKFLOW_APP_MODES)
def get(self, session: Session, current_user: Account, app_model: App, name: str):
@with_current_user
def get(self, current_user: Account, app_model: App, name: str):
query = query_params_from_request(AgentConfigSkillFileQuery)
try:
target = _resolve_app_route_target(
session=session, app_model=app_model, current_user=current_user, query=query
)
target = _resolve_app_route_target(app_model=app_model, current_user=current_user, query=query)
if isinstance(target, tuple):
return target
return _skill_file_download_response(
@@ -1100,12 +1047,10 @@ class AgentConfigSkillFileDownloadContentByAgentApi(Resource):
@account_initialization_required
@with_current_user
@with_current_tenant_id
@with_session
def get(self, session: Session, tenant_id: str, current_user: Account, agent_id: UUID, name: str):
def get(self, tenant_id: str, current_user: Account, agent_id: UUID, name: str):
query = query_params_from_request(AgentConfigSkillFileByAgentQuery)
try:
target = _resolve_agent_route_target(
session=session,
tenant_id=tenant_id,
agent_id=agent_id,
current_user=current_user,
@@ -1124,15 +1069,12 @@ class AgentConfigSkillFileDownloadContentApi(Resource):
@setup_required
@login_required
@account_initialization_required
@with_current_user
@with_session
@get_app_model(mode=_WORKFLOW_APP_MODES)
def get(self, session: Session, current_user: Account, app_model: App, name: str):
@with_current_user
def get(self, current_user: Account, app_model: App, name: str):
query = query_params_from_request(AgentConfigSkillFileQuery)
try:
target = _resolve_app_route_target(
session=session, app_model=app_model, current_user=current_user, query=query
)
target = _resolve_app_route_target(app_model=app_model, current_user=current_user, query=query)
if isinstance(target, tuple):
return target
return _skill_file_raw_download_response(target, name, query.path)
@@ -1152,10 +1094,8 @@ class AgentConfigSkillByAgentApi(Resource):
@account_initialization_required
@with_current_user
@with_current_tenant_id
@with_session
def delete(self, session: Session, tenant_id: str, current_user: Account, agent_id: UUID, name: str):
def delete(self, tenant_id: str, current_user: Account, agent_id: UUID, name: str):
return _with_agent_route_target(
session=session,
tenant_id=tenant_id,
agent_id=agent_id,
current_user=current_user,
@@ -1175,12 +1115,10 @@ class AgentConfigSkillApi(Resource):
@account_initialization_required
@edit_permission_required
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_EDIT)
@with_current_user
@with_session
@get_app_model(mode=_WORKFLOW_APP_MODES)
def delete(self, session: Session, current_user: Account, app_model: App, name: str):
@with_current_user
def delete(self, current_user: Account, app_model: App, name: str):
return _with_app_route_target(
session=session,
app_model=app_model,
current_user=current_user,
action=lambda target: _skill_delete_response(target, name),
@@ -1199,10 +1137,8 @@ class AgentConfigFilePreviewByAgentApi(Resource):
@account_initialization_required
@with_current_user
@with_current_tenant_id
@with_session
def get(self, session: Session, tenant_id: str, current_user: Account, agent_id: UUID, name: str):
def get(self, tenant_id: str, current_user: Account, agent_id: UUID, name: str):
return _with_agent_route_target(
session=session,
tenant_id=tenant_id,
agent_id=agent_id,
current_user=current_user,
@@ -1220,12 +1156,10 @@ class AgentConfigFilePreviewApi(Resource):
@setup_required
@login_required
@account_initialization_required
@with_current_user
@with_session
@get_app_model(mode=_WORKFLOW_APP_MODES)
def get(self, session: Session, current_user: Account, app_model: App, name: str):
@with_current_user
def get(self, current_user: Account, app_model: App, name: str):
return _with_app_route_target(
session=session,
app_model=app_model,
current_user=current_user,
action=lambda target: _file_preview_response(target, name),
@@ -1244,10 +1178,8 @@ class AgentConfigFileDownloadByAgentApi(Resource):
@account_initialization_required
@with_current_user
@with_current_tenant_id
@with_session
def get(self, session: Session, tenant_id: str, current_user: Account, agent_id: UUID, name: str):
def get(self, tenant_id: str, current_user: Account, agent_id: UUID, name: str):
return _with_agent_route_target(
session=session,
tenant_id=tenant_id,
agent_id=agent_id,
current_user=current_user,
@@ -1265,12 +1197,10 @@ class AgentConfigFileDownloadApi(Resource):
@setup_required
@login_required
@account_initialization_required
@with_current_user
@with_session
@get_app_model(mode=_WORKFLOW_APP_MODES)
def get(self, session: Session, current_user: Account, app_model: App, name: str):
@with_current_user
def get(self, current_user: Account, app_model: App, name: str):
return _with_app_route_target(
session=session,
app_model=app_model,
current_user=current_user,
action=lambda target: _file_download_response(target, name),
@@ -1289,10 +1219,8 @@ class AgentConfigFileByAgentApi(Resource):
@account_initialization_required
@with_current_user
@with_current_tenant_id
@with_session
def delete(self, session: Session, tenant_id: str, current_user: Account, agent_id: UUID, name: str):
def delete(self, tenant_id: str, current_user: Account, agent_id: UUID, name: str):
return _with_agent_route_target(
session=session,
tenant_id=tenant_id,
agent_id=agent_id,
current_user=current_user,
@@ -1312,12 +1240,10 @@ class AgentConfigFileApi(Resource):
@account_initialization_required
@edit_permission_required
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_EDIT)
@with_current_user
@with_session
@get_app_model(mode=_WORKFLOW_APP_MODES)
def delete(self, session: Session, current_user: Account, app_model: App, name: str):
@with_current_user
def delete(self, current_user: Account, app_model: App, name: str):
return _with_app_route_target(
session=session,
app_model=app_model,
current_user=current_user,
action=lambda target: _file_delete_response(target, name),
@@ -18,18 +18,17 @@ from uuid import UUID
from flask import Response
from flask_restx import Resource
from pydantic import BaseModel, Field
from sqlalchemy.orm import Session
from controllers.common.schema import (
query_params_from_model,
query_params_from_request,
register_response_schema_models,
)
from controllers.common.session import with_session
from controllers.console import console_ns
from controllers.console.agent.app_helpers import resolve_agent_runtime_app_model
from controllers.console.app.wraps import get_app_model
from controllers.console.wraps import account_initialization_required, setup_required, with_current_tenant_id
from extensions.ext_database import db
from fields.base import ResponseModel
from libs.login import login_required
from models.model import App, AppMode
@@ -145,13 +144,13 @@ register_response_schema_models(
)
def _resolve_agent_id(session: Session, app_model: App, node_id: str | None) -> str | None:
def _resolve_agent_id(app_model: App, node_id: str | None) -> str | None:
"""Agent identity for the drive: app-bound agent, or the workflow node binding."""
if node_id:
return AgentComposerService.resolve_workflow_node_agent_id(
session=session, tenant_id=app_model.tenant_id, app_id=app_model.id, node_id=node_id
tenant_id=app_model.tenant_id, app_id=app_model.id, node_id=node_id, session=db.session()
)
return app_model.bound_agent_id_with_session(session=session)
return app_model.bound_agent_id
def _agent_not_bound() -> tuple[dict[str, object], int]:
@@ -182,13 +181,12 @@ class AgentDriveListByAgentApi(Resource):
@login_required
@account_initialization_required
@with_current_tenant_id
@with_session(write=False)
def get(self, session: Session, tenant_id: str, agent_id: UUID):
def get(self, tenant_id: str, agent_id: UUID):
query = query_params_from_request(AgentDriveListByAgentQuery)
resolve_agent_runtime_app_model(session=session, tenant_id=tenant_id, agent_id=agent_id)
resolve_agent_runtime_app_model(tenant_id=tenant_id, agent_id=agent_id)
try:
items = AgentDriveService().manifest(
tenant_id=tenant_id, agent_id=str(agent_id), prefix=query.prefix, session=session
tenant_id=tenant_id, agent_id=str(agent_id), prefix=query.prefix, session=db.session()
)
except AgentDriveError as exc:
return _handle(exc)
@@ -205,11 +203,10 @@ class AgentDriveSkillListByAgentApi(Resource):
@login_required
@account_initialization_required
@with_current_tenant_id
@with_session(write=False)
def get(self, session: Session, tenant_id: str, agent_id: UUID):
resolve_agent_runtime_app_model(session=session, tenant_id=tenant_id, agent_id=agent_id)
def get(self, tenant_id: str, agent_id: UUID):
resolve_agent_runtime_app_model(tenant_id=tenant_id, agent_id=agent_id)
try:
items = AgentDriveService().list_skills(tenant_id=tenant_id, agent_id=str(agent_id), session=session)
items = AgentDriveService().list_skills(tenant_id=tenant_id, agent_id=str(agent_id), session=db.session())
except AgentDriveError as exc:
return _handle(exc)
return {"items": items}
@@ -225,16 +222,15 @@ class AgentDriveSkillInspectByAgentApi(Resource):
@login_required
@account_initialization_required
@with_current_tenant_id
@with_session(write=False)
def get(self, session: Session, tenant_id: str, agent_id: UUID, skill_path: str):
resolve_agent_runtime_app_model(session=session, tenant_id=tenant_id, agent_id=agent_id)
def get(self, tenant_id: str, agent_id: UUID, skill_path: str):
resolve_agent_runtime_app_model(tenant_id=tenant_id, agent_id=agent_id)
try:
return _json_response(
AgentDriveService().inspect_skill(
tenant_id=tenant_id,
agent_id=str(agent_id),
skill_path=skill_path,
session=session,
session=db.session(),
)
)
except AgentDriveError as exc:
@@ -251,13 +247,12 @@ class AgentDrivePreviewByAgentApi(Resource):
@login_required
@account_initialization_required
@with_current_tenant_id
@with_session(write=False)
def get(self, session: Session, tenant_id: str, agent_id: UUID):
def get(self, tenant_id: str, agent_id: UUID):
query = query_params_from_request(AgentDriveFileByAgentQuery)
resolve_agent_runtime_app_model(session=session, tenant_id=tenant_id, agent_id=agent_id)
resolve_agent_runtime_app_model(tenant_id=tenant_id, agent_id=agent_id)
try:
return AgentDriveService().preview(
tenant_id=tenant_id, agent_id=str(agent_id), key=query.key, session=session
tenant_id=tenant_id, agent_id=str(agent_id), key=query.key, session=db.session()
)
except AgentDriveError as exc:
return _handle(exc)
@@ -273,13 +268,12 @@ class AgentDriveDownloadByAgentApi(Resource):
@login_required
@account_initialization_required
@with_current_tenant_id
@with_session(write=False)
def get(self, session: Session, tenant_id: str, agent_id: UUID):
def get(self, tenant_id: str, agent_id: UUID):
query = query_params_from_request(AgentDriveFileByAgentQuery)
resolve_agent_runtime_app_model(session=session, tenant_id=tenant_id, agent_id=agent_id)
resolve_agent_runtime_app_model(tenant_id=tenant_id, agent_id=agent_id)
try:
url = AgentDriveService().download_url(
tenant_id=tenant_id, agent_id=str(agent_id), key=query.key, session=session
tenant_id=tenant_id, agent_id=str(agent_id), key=query.key, session=db.session()
)
except AgentDriveError as exc:
return _handle(exc)
@@ -295,16 +289,15 @@ class AgentDriveListApi(Resource):
@setup_required
@login_required
@account_initialization_required
@with_session(write=False)
@get_app_model(mode=_WORKFLOW_APP_MODES)
def get(self, session: Session, app_model: App):
def get(self, app_model: App):
query = query_params_from_request(AgentDriveListQuery)
agent_id = _resolve_agent_id(session, app_model, query.node_id)
agent_id = _resolve_agent_id(app_model, query.node_id)
if not agent_id:
return _agent_not_bound()
try:
items = AgentDriveService().manifest(
tenant_id=app_model.tenant_id, agent_id=agent_id, prefix=query.prefix, session=session
tenant_id=app_model.tenant_id, agent_id=agent_id, prefix=query.prefix, session=db.session()
)
except AgentDriveError as exc:
return _handle(exc)
@@ -322,15 +315,16 @@ class AgentDriveSkillListApi(Resource):
@setup_required
@login_required
@account_initialization_required
@with_session(write=False)
@get_app_model(mode=_WORKFLOW_APP_MODES)
def get(self, session: Session, app_model: App):
def get(self, app_model: App):
query = query_params_from_request(AgentDriveListQuery)
agent_id = _resolve_agent_id(session, app_model, query.node_id)
agent_id = _resolve_agent_id(app_model, query.node_id)
if not agent_id:
return _agent_not_bound()
try:
items = AgentDriveService().list_skills(tenant_id=app_model.tenant_id, agent_id=agent_id, session=session)
items = AgentDriveService().list_skills(
tenant_id=app_model.tenant_id, agent_id=agent_id, session=db.session()
)
except AgentDriveError as exc:
return _handle(exc)
return {"items": items}
@@ -351,11 +345,10 @@ class AgentDriveSkillInspectApi(Resource):
@setup_required
@login_required
@account_initialization_required
@with_session(write=False)
@get_app_model(mode=_WORKFLOW_APP_MODES)
def get(self, session: Session, app_model: App, skill_path: str):
def get(self, app_model: App, skill_path: str):
query = query_params_from_request(AgentDriveSkillInspectQuery)
agent_id = _resolve_agent_id(session, app_model, query.node_id)
agent_id = _resolve_agent_id(app_model, query.node_id)
if not agent_id:
return _agent_not_bound()
try:
@@ -364,7 +357,7 @@ class AgentDriveSkillInspectApi(Resource):
tenant_id=app_model.tenant_id,
agent_id=agent_id,
skill_path=skill_path,
session=session,
session=db.session(),
)
)
except AgentDriveError as exc:
@@ -380,16 +373,15 @@ class AgentDrivePreviewApi(Resource):
@setup_required
@login_required
@account_initialization_required
@with_session(write=False)
@get_app_model(mode=_WORKFLOW_APP_MODES)
def get(self, session: Session, app_model: App):
def get(self, app_model: App):
query = query_params_from_request(AgentDriveFileQuery)
agent_id = _resolve_agent_id(session, app_model, query.node_id)
agent_id = _resolve_agent_id(app_model, query.node_id)
if not agent_id:
return _agent_not_bound()
try:
return AgentDriveService().preview(
tenant_id=app_model.tenant_id, agent_id=agent_id, key=query.key, session=session
tenant_id=app_model.tenant_id, agent_id=agent_id, key=query.key, session=db.session()
)
except AgentDriveError as exc:
return _handle(exc)
@@ -404,16 +396,15 @@ class AgentDriveDownloadApi(Resource):
@setup_required
@login_required
@account_initialization_required
@with_session(write=False)
@get_app_model(mode=_WORKFLOW_APP_MODES)
def get(self, session: Session, app_model: App):
def get(self, app_model: App):
query = query_params_from_request(AgentDriveFileQuery)
agent_id = _resolve_agent_id(session, app_model, query.node_id)
agent_id = _resolve_agent_id(app_model, query.node_id)
if not agent_id:
return _agent_not_bound()
try:
url = AgentDriveService().download_url(
tenant_id=app_model.tenant_id, agent_id=agent_id, key=query.key, session=session
tenant_id=app_model.tenant_id, agent_id=agent_id, key=query.key, session=db.session()
)
except AgentDriveError as exc:
return _handle(exc)
+30 -39
View File
@@ -5,12 +5,10 @@ from flask import abort, request
from flask_restx import Resource
from pydantic import BaseModel, Field, TypeAdapter, field_validator
from sqlalchemy import select
from sqlalchemy.orm import Session
from werkzeug.exceptions import NotFound
from controllers.common.errors import NoFileUploadedError, TooManyFilesError
from controllers.common.schema import query_params_from_model, register_response_schema_models, register_schema_models
from controllers.common.session import with_session
from controllers.console import console_ns
from controllers.console.wraps import (
RBACPermission,
@@ -23,6 +21,7 @@ from controllers.console.wraps import (
rbac_permission_required,
setup_required,
)
from extensions.ext_database import db
from extensions.ext_redis import redis_client
from fields.annotation_fields import (
Annotation,
@@ -47,9 +46,9 @@ from services.annotation_service import (
from services.app_ref_service import AppRef, AppRefService
def _get_app_ref(session: Session, app_id: str) -> AppRef:
def _get_app_ref(app_id: str) -> AppRef:
_, current_tenant_id = current_account_with_tenant()
app = session.scalar(
app = db.session.scalar(
select(App).where(App.id == app_id, App.tenant_id == current_tenant_id, App.status == "normal").limit(1)
)
if app is None:
@@ -211,9 +210,8 @@ class AppAnnotationSettingDetailApi(Resource):
@account_initialization_required
@edit_permission_required
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_VIEW_LAYOUT)
@with_session(write=False)
def get(self, session: Session, app_id: UUID):
result = AppAnnotationService.get_app_annotation_setting_by_app_id(str(app_id), session)
def get(self, app_id: UUID):
result = AppAnnotationService.get_app_annotation_setting_by_app_id(str(app_id), session=db.session())
return dump_response(AnnotationSettingResponse, result), 200
@@ -230,15 +228,14 @@ class AppAnnotationSettingUpdateApi(Resource):
@account_initialization_required
@edit_permission_required
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_EDIT)
@with_session
def post(self, session: Session, app_id: UUID, annotation_setting_id: UUID):
def post(self, app_id: UUID, annotation_setting_id: UUID):
annotation_setting_id_str = str(annotation_setting_id)
args = AnnotationSettingUpdatePayload.model_validate(console_ns.payload)
setting_args: UpdateAnnotationSettingArgs = {"score_threshold": args.score_threshold}
result = AppAnnotationService.update_app_annotation_setting(
str(app_id), annotation_setting_id_str, setting_args, session
str(app_id), annotation_setting_id_str, setting_args, session=db.session()
)
return dump_response(AnnotationSettingResponse, result), 200
@@ -289,15 +286,14 @@ class AnnotationApi(Resource):
@account_initialization_required
@edit_permission_required
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_VIEW_LAYOUT)
@with_session(write=False)
def get(self, session: Session, app_id: UUID):
def get(self, app_id: UUID):
args = AnnotationListQuery.model_validate(request.args.to_dict(flat=True))
page = args.page
limit = args.limit
keyword = args.keyword
annotation_list, total = AppAnnotationService.get_annotation_list_by_app_id(
str(app_id), page, limit, keyword, session
str(app_id), page, limit, keyword, session=db.session()
)
annotation_models = TypeAdapter(list[Annotation]).validate_python(annotation_list, from_attributes=True)
return AnnotationList(
@@ -316,8 +312,7 @@ class AnnotationApi(Resource):
@cloud_edition_billing_resource_check("annotation")
@edit_permission_required
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_EDIT)
@with_session
def post(self, session: Session, app_id: UUID):
def post(self, app_id: UUID):
args = CreateAnnotationPayload.model_validate(console_ns.payload)
upsert_args: UpsertAnnotationArgs = {}
if args.answer is not None:
@@ -328,7 +323,9 @@ class AnnotationApi(Resource):
upsert_args["message_id"] = args.message_id
if args.question is not None:
upsert_args["question"] = args.question
annotation = AppAnnotationService.up_insert_app_annotation_from_message(upsert_args, str(app_id), session)
annotation = AppAnnotationService.up_insert_app_annotation_from_message(
upsert_args, str(app_id), session=db.session()
)
return dump_response(Annotation, annotation), 201
@setup_required
@@ -337,8 +334,7 @@ class AnnotationApi(Resource):
@edit_permission_required
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_CREATE_AND_MANAGEMENT)
@console_ns.response(204, "Annotations deleted successfully")
@with_session
def delete(self, session: Session, app_id: UUID):
def delete(self, app_id: UUID):
# Use request.args.getlist to get annotation_ids array directly
annotation_ids = request.args.getlist("annotation_id")
@@ -352,12 +348,12 @@ class AnnotationApi(Resource):
"message": "annotation_ids are required if the parameter is provided.",
}, 400
app_ref = _get_app_ref(session, str(app_id))
AppAnnotationService.delete_app_annotations_in_batch(app_ref, annotation_ids, session)
app_ref = _get_app_ref(str(app_id))
AppAnnotationService.delete_app_annotations_in_batch(app_ref, annotation_ids, session=db.session())
return "", 204
# If no annotation_ids are provided, handle clearing all annotations
else:
AppAnnotationService.clear_all_annotations(str(app_id), session)
AppAnnotationService.clear_all_annotations(str(app_id), session=db.session())
return "", 204
@@ -377,9 +373,8 @@ class AnnotationExportApi(Resource):
@account_initialization_required
@edit_permission_required
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_VIEW_LAYOUT)
@with_session(write=False)
def get(self, session: Session, app_id: UUID):
annotation_list = AppAnnotationService.export_annotation_list_by_app_id(str(app_id), session)
def get(self, app_id: UUID):
annotation_list = AppAnnotationService.export_annotation_list_by_app_id(str(app_id), session=db.session())
annotation_models = TypeAdapter(list[Annotation]).validate_python(annotation_list, from_attributes=True)
return (
AnnotationExportList(data=annotation_models).model_dump(mode="json"),
@@ -406,17 +401,16 @@ class AnnotationUpdateDeleteApi(Resource):
@cloud_edition_billing_resource_check("annotation")
@edit_permission_required
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_EDIT)
@with_session
def post(self, session: Session, app_id: UUID, annotation_id: UUID):
def post(self, app_id: UUID, annotation_id: UUID):
args = UpdateAnnotationPayload.model_validate(console_ns.payload)
update_args: UpdateAnnotationArgs = {}
if args.answer is not None:
update_args["answer"] = args.answer
if args.question is not None:
update_args["question"] = args.question
app_ref = _get_app_ref(session, str(app_id))
app_ref = _get_app_ref(str(app_id))
annotation_ref = AppRefService.create_annotation_ref(app_ref, str(annotation_id))
annotation = AppAnnotationService.update_app_annotation_directly(update_args, annotation_ref, session)
annotation = AppAnnotationService.update_app_annotation_directly(update_args, annotation_ref, db.session())
return Annotation.model_validate(annotation, from_attributes=True).model_dump(mode="json")
@setup_required
@@ -425,11 +419,10 @@ class AnnotationUpdateDeleteApi(Resource):
@edit_permission_required
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_EDIT)
@console_ns.response(204, "Annotation deleted successfully")
@with_session
def delete(self, session: Session, app_id: UUID, annotation_id: UUID):
app_ref = _get_app_ref(session, str(app_id))
def delete(self, app_id: UUID, annotation_id: UUID):
app_ref = _get_app_ref(str(app_id))
annotation_ref = AppRefService.create_annotation_ref(app_ref, str(annotation_id))
AppAnnotationService.delete_app_annotation(annotation_ref, session)
AppAnnotationService.delete_app_annotation(annotation_ref, db.session())
return "", 204
@@ -453,8 +446,7 @@ class AnnotationBatchImportApi(Resource):
@annotation_import_concurrency_limit
@edit_permission_required
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_EDIT)
@with_session
def post(self, session: Session, app_id: UUID):
def post(self, app_id: UUID):
from configs import dify_config
# check file
@@ -489,7 +481,7 @@ class AnnotationBatchImportApi(Resource):
return dump_response(
AnnotationBatchImportResponse,
AppAnnotationService.batch_import_app_annotations(str(app_id), file, session),
AppAnnotationService.batch_import_app_annotations(str(app_id), file, session=db.session()),
)
@@ -541,17 +533,16 @@ class AnnotationHitHistoryListApi(Resource):
@account_initialization_required
@edit_permission_required
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_VIEW_LAYOUT)
@with_session(write=False)
def get(self, session: Session, app_id: UUID, annotation_id: UUID):
def get(self, app_id: UUID, annotation_id: UUID):
page = request.args.get("page", default=1, type=int)
limit = request.args.get("limit", default=20, type=int)
app_ref = _get_app_ref(session, str(app_id))
app_ref = _get_app_ref(str(app_id))
annotation_ref = AppRefService.create_annotation_ref(app_ref, str(annotation_id))
annotation_hit_history_list, total = AppAnnotationService.get_annotation_hit_histories(
annotation_ref,
page,
limit,
session,
session=db.session(),
)
history_models = TypeAdapter(list[AnnotationHitHistory]).validate_python(
annotation_hit_history_list, from_attributes=True
+66 -146
View File
@@ -6,10 +6,10 @@ from typing import Any, Literal
from flask import request
from flask_restx import Resource
from pydantic import AliasChoices, BaseModel, Field, ValidationInfo, computed_field, field_validator, model_validator
from pydantic import AliasChoices, BaseModel, Field, computed_field, field_validator
from sqlalchemy import select
from sqlalchemy.orm import Session
from werkzeug.exceptions import BadRequest, Forbidden, NotFound
from werkzeug.exceptions import BadRequest, NotFound
from configs import dify_config
from controllers.common.app_access import resolve_app_access_filter
@@ -23,7 +23,7 @@ from controllers.common.schema import (
register_schema_models,
)
from controllers.console import console_ns
from controllers.console.app.wraps import agent_manage_required_for_agent_app, get_app_model, with_session
from controllers.console.app.wraps import get_app_model, with_session
from controllers.console.workspace.models import LoadBalancingPayload
from controllers.console.wraps import (
RBACPermission,
@@ -52,14 +52,7 @@ from libs.login import login_required
from models import Account, App, DatasetPermissionEnum, Workflow
from models.model import IconType
from services.app_dsl_service import AppDslService
from services.app_service import (
AppListParams,
AppListSortBy,
AppResponseView,
AppService,
CreateAppParams,
StarredAppListParams,
)
from services.app_service import AppListParams, AppListSortBy, AppService, CreateAppParams, StarredAppListParams
from services.enterprise import rbac_service as enterprise_rbac_service
from services.enterprise.enterprise_service import EnterpriseService
from services.entities.dsl_entities import DslImportWarning, ImportMode, ImportStatus
@@ -75,7 +68,6 @@ from services.entities.knowledge_entities.knowledge_entities import (
WeightModel,
WeightVectorSetting,
)
from services.errors.account import NoPermissionError
from services.feature_service import FeatureService
from tasks.initialize_created_app_rbac_access_task import initialize_created_app_rbac_access_task
@@ -247,7 +239,7 @@ class ModelConfigPartial(ResponseModel):
return to_timestamp(value)
class AppModelConfigResponse(ResponseModel):
class ModelConfig(ResponseModel):
opening_statement: str | None = None
suggested_questions: Any | None = Field(
default=None, validation_alias=AliasChoices("suggested_questions_list", "suggested_questions")
@@ -356,18 +348,7 @@ class DeletedTool(ResponseModel):
provider_id: str
class AppResponseModel(ResponseModel):
@model_validator(mode="before")
@classmethod
def _use_request_session(cls, value: Any, info: ValidationInfo) -> Any:
if not isinstance(value, App):
return value
if info.context is None or "session" not in info.context:
raise ValueError("session context is required to serialize an App")
return AppResponseView(value, session=info.context["session"])
class AppPartial(AppResponseModel):
class AppPartial(ResponseModel):
id: str
name: str
max_active_requests: int | None = None
@@ -411,7 +392,7 @@ class AppPartial(AppResponseModel):
return to_timestamp(value)
class AppDetail(AppResponseModel):
class AppDetail(ResponseModel):
id: str
name: str
description: str | None = None
@@ -420,7 +401,7 @@ class AppDetail(AppResponseModel):
icon_background: str | None = None
enable_site: bool
enable_api: bool
model_config_: AppModelConfigResponse | None = Field(
model_config_: ModelConfig | None = Field(
default=None,
validation_alias=AliasChoices("app_model_config", "model_config"),
alias="model_config",
@@ -526,13 +507,7 @@ def _enrich_app_list_items(session: Session, *, apps: Sequence[App], tenant_id:
register_enum_models(console_ns, RetrievalMethod, WorkflowExecutionStatus, DatasetPermissionEnum)
register_response_schema_models(
console_ns,
RedirectUrlResponse,
SimpleResultResponse,
AppImportResponse,
AppTraceResponse,
AppModelConfigResponse,
AppDetail,
console_ns, RedirectUrlResponse, SimpleResultResponse, AppImportResponse, AppTraceResponse
)
register_schema_models(
@@ -551,8 +526,10 @@ register_schema_models(
Tag,
WorkflowPartial,
ModelConfigPartial,
ModelConfig,
AppDetailSiteResponse,
DeletedTool,
AppDetail,
AppExportResponse,
Segmentation,
PreProcessingRule,
@@ -610,13 +587,12 @@ class AppListApi(Resource):
permissions = enterprise_rbac_service.RBACService.MyPermissions.get(
str(current_tenant_id),
current_user_id,
session=session,
session=db.session(),
)
if dify_config.RBAC_ENABLED:
access_filter = resolve_app_access_filter(
str(current_tenant_id),
current_user_id,
session=session,
permissions=permissions,
)
access_filter.apply_to_params(params)
@@ -632,11 +608,7 @@ class AppListApi(Resource):
permission_keys_map = permissions.app.permission_keys_by_resource_ids(app_ids)
_enrich_app_list_items(session, apps=app_pagination.items, tenant_id=current_tenant_id)
pagination_model = AppPagination.model_validate(
app_pagination,
from_attributes=True,
context={"session": session},
)
pagination_model = AppPagination.model_validate(app_pagination, from_attributes=True)
if app_pagination.items:
pagination_model = pagination_model.model_copy(
update={
@@ -662,8 +634,7 @@ class AppListApi(Resource):
@edit_permission_required
@with_current_user
@with_current_tenant_id
@with_session
def post(self, session: Session, current_tenant_id: str, current_user: Account):
def post(self, current_tenant_id: str, current_user: Account):
"""Create app"""
args = CreateAppPayload.model_validate(console_ns.payload)
params = CreateAppParams(
@@ -676,7 +647,7 @@ class AppListApi(Resource):
)
app_service = AppService()
app = app_service.create_app(current_tenant_id, params, current_user, session=session)
app = app_service.create_app(current_tenant_id, params, current_user, session=db.session())
if dify_config.RBAC_ENABLED:
enterprise_rbac_service.RBACService.AppAccess.replace_whitelist(
tenant_id=str(current_tenant_id),
@@ -689,13 +660,11 @@ class AppListApi(Resource):
str(current_tenant_id),
current_user.id,
[str(app.id)],
session=session,
session=db.session(),
)
app_detail = AppDetailWithSite.model_validate(app, from_attributes=True).model_copy(
update={"permission_keys": permission_keys_map.get(str(app.id), [])}
)
app_detail = AppDetailWithSite.model_validate(
app,
from_attributes=True,
context={"session": session},
).model_copy(update={"permission_keys": permission_keys_map.get(str(app.id), [])})
return app_detail.model_dump(mode="json"), 201
@@ -731,14 +700,7 @@ class StarredAppListApi(Resource):
return empty.model_dump(mode="json"), 200
_enrich_app_list_items(session, apps=app_pagination.items, tenant_id=current_tenant_id)
return (
AppPagination.model_validate(
app_pagination,
from_attributes=True,
context={"session": session},
).model_dump(mode="json"),
200,
)
return AppPagination.model_validate(app_pagination, from_attributes=True).model_dump(mode="json"), 200
@console_ns.route("/apps/<uuid:app_id>/star")
@@ -789,13 +751,12 @@ class AppApi(Resource):
@with_current_user
@with_current_tenant_id
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_VIEW_LAYOUT)
@with_session(write=False)
@get_app_model(mode=None)
def get(self, session: Session, current_tenant_id: str, current_user: Account, app_model: App):
def get(self, current_tenant_id: str, current_user: Account, app_model: App):
"""Get app detail"""
app_service = AppService()
app_model = app_service.get_app(app_model, session=session)
app_model = app_service.get_app(app_model)
if FeatureService.get_system_features().webapp_auth.enabled:
app_setting = EnterpriseService.WebAppAuth.get_app_access_mode_by_id(app_id=str(app_model.id))
@@ -805,15 +766,13 @@ class AppApi(Resource):
str(current_tenant_id),
current_user.id,
app_id=str(app_model.id),
session=session,
session=db.session(),
)
permission_keys_map = permissions.app.permission_keys_by_resource_ids([str(app_model.id)])
response_model = AppDetailWithSite.model_validate(
app_model,
from_attributes=True,
context={"session": session},
).model_copy(update={"permission_keys": permission_keys_map.get(str(app_model.id), [])})
response_model = AppDetailWithSite.model_validate(app_model, from_attributes=True).model_copy(
update={"permission_keys": permission_keys_map.get(str(app_model.id), [])}
)
return response_model.model_dump(mode="json")
@console_ns.doc("update_app")
@@ -828,10 +787,8 @@ class AppApi(Resource):
@account_initialization_required
@edit_permission_required
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_EDIT)
@agent_manage_required_for_agent_app
@with_session
@get_app_model(mode=None)
def put(self, session: Session, app_model: App):
def put(self, app_model: App):
"""Update app"""
args = UpdateAppPayload.model_validate(console_ns.payload)
@@ -846,12 +803,8 @@ class AppApi(Resource):
"use_icon_as_answer_icon": args.use_icon_as_answer_icon or False,
"max_active_requests": args.max_active_requests or 0,
}
app_model = app_service.update_app(app_model, args_dict, session=session)
return AppDetailWithSite.model_validate(
app_model,
from_attributes=True,
context={"session": session},
).model_dump(mode="json")
app_model = app_service.update_app(app_model, args_dict, session=db.session())
return dump_response(AppDetailWithSite, app_model)
@console_ns.doc("delete_app")
@console_ns.doc(description="Delete application")
@@ -863,13 +816,11 @@ class AppApi(Resource):
@account_initialization_required
@edit_permission_required
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_DELETE)
@agent_manage_required_for_agent_app
@with_session
@get_app_model
def delete(self, session: Session, app_model: App):
def delete(self, app_model: App):
"""Delete app"""
app_service = AppService()
app_service.delete_app(app_model, session=session)
app_service.delete_app(app_model, session=db.session())
return "", 204
@@ -888,7 +839,6 @@ class AppCopyApi(Resource):
@account_initialization_required
@edit_permission_required
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_CREATE_AND_MANAGEMENT)
@agent_manage_required_for_agent_app
@with_current_user
@with_current_tenant_id
@get_app_model(mode=None)
@@ -900,19 +850,16 @@ class AppCopyApi(Resource):
with Session(db.engine, expire_on_commit=False) as session:
import_service = AppDslService(session)
yaml_content = import_service.export_dsl(app_model=app_model, session=session, include_secret=True)
try:
result = import_service.import_app(
account=current_user,
import_mode=ImportMode.YAML_CONTENT,
yaml_content=yaml_content,
name=args.name,
description=args.description,
icon_type=args.icon_type,
icon=args.icon,
icon_background=args.icon_background,
)
except NoPermissionError as e:
raise Forbidden(str(e))
result = import_service.import_app(
account=current_user,
import_mode=ImportMode.YAML_CONTENT,
yaml_content=yaml_content,
name=args.name,
description=args.description,
icon_type=args.icon_type,
icon=args.icon,
icon_background=args.icon_background,
)
if result.status == ImportStatus.FAILED:
session.rollback()
return dump_response(AppImportResponse, result), 400
@@ -936,21 +883,20 @@ class AppCopyApi(Resource):
stmt = select(App).where(App.id == result.app_id)
app = session.scalar(stmt)
if not app:
raise NotFound("App not found")
permission_keys_map = enterprise_rbac_service.RBACService.AppPermissions.batch_get(
str(current_tenant_id),
current_user.id,
[str(app.id)],
session=session,
)
response_model = AppDetailWithSite.model_validate(
app,
from_attributes=True,
context={"session": session},
).model_copy(update={"permission_keys": permission_keys_map.get(str(app.id), [])})
return response_model.model_dump(mode="json"), 201
if not app:
raise NotFound("App not found")
permission_keys_map = enterprise_rbac_service.RBACService.AppPermissions.batch_get(
str(current_tenant_id),
current_user.id,
[str(app.id)],
session=db.session(),
)
response_model = AppDetailWithSite.model_validate(app, from_attributes=True).model_copy(
update={"permission_keys": permission_keys_map.get(str(app.id), [])}
)
return response_model.model_dump(mode="json"), 201
@console_ns.route("/apps/<uuid:app_id>/export")
@@ -966,7 +912,6 @@ class AppExportApi(Resource):
@account_initialization_required
@edit_permission_required
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_IMPORT_EXPORT_DSL)
@agent_manage_required_for_agent_app
@get_app_model
def get(self, app_model: App):
"""Export app"""
@@ -991,7 +936,6 @@ class AppPublishToCreatorsPlatformApi(Resource):
@account_initialization_required
@edit_permission_required
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_IMPORT_EXPORT_DSL)
@agent_manage_required_for_agent_app
@with_current_user_id
@get_app_model(mode=None)
def post(self, current_user_id: str, app_model: App):
@@ -1022,19 +966,13 @@ class AppNameApi(Resource):
@account_initialization_required
@edit_permission_required
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_EDIT)
@agent_manage_required_for_agent_app
@with_session
@get_app_model(mode=None)
def post(self, session: Session, app_model: App):
def post(self, app_model: App):
args = AppNamePayload.model_validate(console_ns.payload)
app_service = AppService()
app_model = app_service.update_app_name(app_model, args.name, session=session)
return AppDetail.model_validate(
app_model,
from_attributes=True,
context={"session": session},
).model_dump(mode="json")
app_model = app_service.update_app_name(app_model, args.name, session=db.session())
return dump_response(AppDetail, app_model)
@console_ns.route("/apps/<uuid:app_id>/icon")
@@ -1050,10 +988,8 @@ class AppIconApi(Resource):
@account_initialization_required
@edit_permission_required
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_EDIT)
@agent_manage_required_for_agent_app
@with_session
@get_app_model(mode=None)
def post(self, session: Session, app_model: App):
def post(self, app_model: App):
args = AppIconPayload.model_validate(console_ns.payload or {})
app_service = AppService()
@@ -1062,13 +998,9 @@ class AppIconApi(Resource):
args.icon or "",
args.icon_background or "",
args.icon_type,
session=session,
session=db.session(),
)
return AppDetail.model_validate(
app_model,
from_attributes=True,
context={"session": session},
).model_dump(mode="json")
return dump_response(AppDetail, app_model)
@console_ns.route("/apps/<uuid:app_id>/site-enable")
@@ -1084,19 +1016,13 @@ class AppSiteStatus(Resource):
@account_initialization_required
@edit_permission_required
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_RELEASE_AND_VERSION)
@agent_manage_required_for_agent_app
@with_session
@get_app_model(mode=None)
def post(self, session: Session, app_model: App):
def post(self, app_model: App):
args = AppSiteStatusPayload.model_validate(console_ns.payload)
app_service = AppService()
app_model = app_service.update_app_site_status(app_model, args.enable_site, session=session)
return AppDetail.model_validate(
app_model,
from_attributes=True,
context={"session": session},
).model_dump(mode="json")
app_model = app_service.update_app_site_status(app_model, args.enable_site, session=db.session())
return dump_response(AppDetail, app_model)
@console_ns.route("/apps/<uuid:app_id>/api-enable")
@@ -1112,19 +1038,13 @@ class AppApiStatus(Resource):
@is_admin_or_owner_required
@account_initialization_required
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_RELEASE_AND_VERSION)
@agent_manage_required_for_agent_app
@with_session
@get_app_model(mode=None)
def post(self, session: Session, app_model: App):
def post(self, app_model: App):
args = AppApiStatusPayload.model_validate(console_ns.payload)
app_service = AppService()
app_model = app_service.update_app_api_status(app_model, args.enable_api, session=session)
return AppDetail.model_validate(
app_model,
from_attributes=True,
context={"session": session},
).model_dump(mode="json")
app_model = app_service.update_app_api_status(app_model, args.enable_api, session=db.session())
return dump_response(AppDetail, app_model)
@console_ns.route("/apps/<uuid:app_id>/trace")
+13 -21
View File
@@ -1,7 +1,6 @@
from flask_restx import Resource
from pydantic import BaseModel, Field
from sqlalchemy.orm import Session
from werkzeug.exceptions import Forbidden
from configs import dify_config
from controllers.common.schema import register_enum_models, register_schema_models
@@ -29,7 +28,6 @@ from services.app_dsl_service import (
)
from services.enterprise.enterprise_service import EnterpriseService
from services.entities.dsl_entities import CheckDependenciesResult, ImportStatus
from services.errors.account import NoPermissionError
from services.feature_service import FeatureService
from .. import console_ns
@@ -93,21 +91,18 @@ class AppImportApi(Resource):
import_service = AppDslService(session)
# Import app
account = current_user
try:
result = import_service.import_app(
account=account,
import_mode=args.mode,
yaml_content=args.yaml_content,
yaml_url=args.yaml_url,
name=args.name,
description=args.description,
icon_type=args.icon_type,
icon=args.icon,
icon_background=args.icon_background,
app_id=args.app_id,
)
except NoPermissionError as e:
raise Forbidden(str(e))
result = import_service.import_app(
account=account,
import_mode=args.mode,
yaml_content=args.yaml_content,
yaml_url=args.yaml_url,
name=args.name,
description=args.description,
icon_type=args.icon_type,
icon=args.icon,
icon_background=args.icon_background,
app_id=args.app_id,
)
if result.status == ImportStatus.FAILED:
session.rollback()
else:
@@ -162,10 +157,7 @@ class AppImportConfirmApi(Resource):
import_service = AppDslService(session)
# Confirm import
account = current_user
try:
result = import_service.confirm_import(import_id=import_id, account=account)
except NoPermissionError as e:
raise Forbidden(str(e))
result = import_service.confirm_import(import_id=import_id, account=account)
if result.status == ImportStatus.FAILED:
session.rollback()
else:
+2 -14
View File
@@ -127,7 +127,6 @@ def _transcribe_audio_to_text(
*,
app_model: App,
file: FileStorage | None,
session: Session,
agent_soul: AgentSoulConfig | None = None,
) -> dict[str, str]:
try:
@@ -135,7 +134,6 @@ def _transcribe_audio_to_text(
response = AudioService.transcript_asr(
app_model=app_model,
file=file,
session=session,
end_user=None,
)
else:
@@ -143,7 +141,6 @@ def _transcribe_audio_to_text(
app_model=app_model,
agent_soul=agent_soul,
file=file,
session=session,
end_user=None,
)
return dump_response(AudioTranscriptResponse, response)
@@ -197,11 +194,7 @@ class ChatMessageAudioApi(Resource):
@account_initialization_required
@get_app_model(mode=_CONSOLE_AUDIO_TRANSCRIPT_APP_MODES)
def post(self, app_model: App):
return _transcribe_audio_to_text(
app_model=app_model,
file=request.files.get("file"),
session=db.session(),
)
return _transcribe_audio_to_text(app_model=app_model, file=request.files.get("file"))
@console_ns.route("/agent/<uuid:agent_id>/audio-to-text")
@@ -235,11 +228,7 @@ class AgentChatMessageAudioApi(Resource):
agent_id: UUID,
):
payload = AgentAudioTranscriptFormPayload.model_validate(request.form.to_dict(flat=True))
app_model = resolve_agent_runtime_app_model(
session=session,
tenant_id=current_tenant_id,
agent_id=agent_id,
)
app_model = resolve_agent_runtime_app_model(tenant_id=current_tenant_id, agent_id=agent_id)
# Agent routes expose Agent ids, while APP RBAC is keyed by the resolved runtime App id.
enforce_rbac_access(
tenant_id=current_tenant_id,
@@ -259,7 +248,6 @@ class AgentChatMessageAudioApi(Resource):
app_model=app_model,
agent_soul=agent_soul,
file=request.files.get("file"),
session=session,
)
+9 -32
View File
@@ -44,12 +44,12 @@ from core.errors.error import (
QuotaExceededError,
)
from core.helper.trace_id_helper import get_external_trace_id
from extensions.ext_database import db
from graphon.model_runtime.errors.invoke import InvokeError
from libs import helper
from libs.helper import uuid_value
from libs.login import login_required
from models import Account
from models.agent import AgentConfigDraftType
from models.model import App, AppMode
from services.agent.errors import AgentNotFoundError
from services.agent.roster_service import AgentRosterService
@@ -158,8 +158,8 @@ class CompletionMessageApi(Resource):
@account_initialization_required
@with_current_user
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_TEST_AND_RUN)
@with_session
@get_app_model(mode=AppMode.COMPLETION)
@with_session
def post(self, session: Session, current_user: Account, app_model: App):
args_model = CompletionMessagePayload.model_validate(console_ns.payload)
args = args_model.model_dump(exclude_none=True, by_alias=True)
@@ -240,8 +240,8 @@ class ChatMessageApi(Resource):
@with_current_user
@with_current_tenant_id
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_TEST_AND_RUN)
@with_session
@get_app_model(mode=[AppMode.CHAT, AppMode.AGENT_CHAT, AppMode.AGENT])
@with_session
def post(self, session: Session, current_tenant_id: str, current_user: Account, app_model: App):
return _create_chat_message(
session=session, current_tenant_id=current_tenant_id, current_user=current_user, app_model=app_model
@@ -266,9 +266,7 @@ class AgentChatMessageApi(Resource):
@with_current_tenant_id
@with_session
def post(self, session: Session, current_tenant_id: str, current_user: Account, agent_id: UUID):
app_model = AgentRosterService(session).get_agent_runtime_app_model(
tenant_id=current_tenant_id, agent_id=str(agent_id)
)
app_model = resolve_agent_runtime_app_model(tenant_id=current_tenant_id, agent_id=agent_id)
return _create_chat_message(
session=session,
current_tenant_id=current_tenant_id,
@@ -295,9 +293,7 @@ class AgentBuildChatFinalizeApi(Resource):
@with_current_tenant_id
@with_session
def post(self, session: Session, current_tenant_id: str, current_user: Account, agent_id: UUID):
app_model = AgentRosterService(session).get_agent_runtime_app_model(
tenant_id=current_tenant_id, agent_id=str(agent_id)
)
app_model = resolve_agent_runtime_app_model(tenant_id=current_tenant_id, agent_id=agent_id)
return _create_build_chat_finalization_message(
session=session,
current_tenant_id=current_tenant_id,
@@ -333,34 +329,20 @@ class AgentChatMessageStopApi(Resource):
@account_initialization_required
@with_current_user_id
@with_current_tenant_id
@with_session(write=False)
def post(self, session: Session, current_tenant_id: str, current_user_id: str, agent_id: UUID, task_id: str):
app_model = resolve_agent_runtime_app_model(
session=session,
tenant_id=current_tenant_id,
agent_id=agent_id,
)
def post(self, current_tenant_id: str, current_user_id: str, agent_id: UUID, task_id: str):
app_model = resolve_agent_runtime_app_model(tenant_id=current_tenant_id, agent_id=agent_id)
return _stop_chat_message(current_user_id=current_user_id, app_model=app_model, task_id=task_id)
def _resolve_current_user_agent_debug_conversation_id(
*,
session: Session,
current_tenant_id: str,
current_user: Account,
app_model: App,
agent_id: str | None,
draft_type: AgentConfigDraftType,
*, current_tenant_id: str, current_user: Account, app_model: App, agent_id: str | None
) -> str:
"""Resolve the current editor's conversation without crossing draft surfaces."""
roster_service = AgentRosterService(session)
roster_service = AgentRosterService(db.session)
if agent_id:
return roster_service.get_or_create_agent_app_debug_conversation_id(
tenant_id=current_tenant_id,
agent_id=agent_id,
account_id=current_user.id,
draft_type=draft_type,
)
agent = roster_service.get_app_backing_agent(tenant_id=current_tenant_id, app_id=str(app_model.id))
@@ -370,7 +352,6 @@ def _resolve_current_user_agent_debug_conversation_id(
tenant_id=current_tenant_id,
agent_id=agent.id,
account_id=current_user.id,
draft_type=draft_type,
)
@@ -388,12 +369,10 @@ def _create_chat_message(
if AppMode.value_of(app_model.mode) == AppMode.AGENT:
debug_conversation_id = _resolve_current_user_agent_debug_conversation_id(
session=session,
current_tenant_id=current_tenant_id or app_model.tenant_id,
current_user=current_user,
app_model=app_model,
agent_id=agent_id,
draft_type=AgentConfigDraftType(args_model.draft_type),
)
if args_model.conversation_id and args_model.conversation_id != debug_conversation_id:
raise NotFound("Conversation Not Exists.")
@@ -425,12 +404,10 @@ def _create_build_chat_finalization_message(
*, session: Session, current_user: Account, app_model: App, current_tenant_id: str, agent_id: str
):
debug_conversation_id = _resolve_current_user_agent_debug_conversation_id(
session=session,
current_tenant_id=current_tenant_id,
current_user=current_user,
app_model=app_model,
agent_id=agent_id,
draft_type=AgentConfigDraftType.DEBUG_BUILD,
)
args: dict[str, Any] = {
"query": _BUILD_CHAT_FINALIZATION_QUERY,
+21 -52
View File
@@ -6,11 +6,10 @@ from flask import abort, request
from flask_restx import Resource
from pydantic import BaseModel, Field, field_validator
from sqlalchemy import func, or_
from sqlalchemy.orm import Session, selectinload
from sqlalchemy.orm import selectinload
from werkzeug.exceptions import NotFound
from controllers.common.schema import query_params_from_model, register_response_schema_models, register_schema_models
from controllers.common.session import with_session
from controllers.console import console_ns
from controllers.console.app.wraps import get_app_model
from controllers.console.wraps import (
@@ -23,6 +22,7 @@ from controllers.console.wraps import (
with_current_user,
)
from core.app.entities.app_invoke_entities import InvokeFrom
from extensions.ext_database import db
from fields.conversation_fields import (
Conversation as ConversationResponse,
)
@@ -35,7 +35,6 @@ from fields.conversation_fields import (
from fields.conversation_fields import (
ConversationPagination as ConversationPaginationResponse,
)
from fields.conversation_fields import ConversationResponseSource
from fields.conversation_fields import (
ConversationWithSummaryPagination as ConversationWithSummaryPaginationResponse,
)
@@ -106,9 +105,8 @@ class CompletionConversationApi(Resource):
@edit_permission_required
@with_current_user
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_VIEW_LAYOUT)
@with_session(write=False)
@get_app_model(mode=AppMode.COMPLETION)
def get(self, session: Session, current_user: Account, app_model: App):
def get(self, current_user: Account, app_model: App):
args = CompletionConversationQuery.model_validate(request.args.to_dict(flat=True))
query = sa.select(Conversation).where(
@@ -159,18 +157,9 @@ class CompletionConversationApi(Resource):
query = query.order_by(Conversation.created_at.desc())
conversations = paginate_query(query, session=session, page=args.page, per_page=args.limit)
conversations = paginate_query(query, page=args.page, per_page=args.limit)
return dump_response(
ConversationPaginationResponse,
{
"page": conversations.page,
"per_page": conversations.per_page,
"total": conversations.total,
"has_next": conversations.has_next,
"items": [ConversationResponseSource(item, session=session) for item in conversations.items],
},
)
return dump_response(ConversationPaginationResponse, conversations)
@console_ns.route("/apps/<uuid:app_id>/completion-conversations/<uuid:conversation_id>")
@@ -187,15 +176,11 @@ class CompletionConversationDetailApi(Resource):
@edit_permission_required
@with_current_user
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_VIEW_LAYOUT)
@with_session
@get_app_model(mode=AppMode.COMPLETION)
def get(self, session: Session, current_user: Account, app_model: App, conversation_id: UUID):
def get(self, current_user: Account, app_model: App, conversation_id: UUID):
conversation_id_str = str(conversation_id)
return dump_response(
ConversationMessageDetailResponse,
ConversationResponseSource(
_get_conversation(session, current_user, app_model, conversation_id_str), session=session
),
ConversationMessageDetailResponse, _get_conversation(current_user, app_model, conversation_id_str)
)
@console_ns.doc("delete_completion_conversation")
@@ -210,13 +195,12 @@ class CompletionConversationDetailApi(Resource):
@edit_permission_required
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_EDIT)
@with_current_user
@with_session
@get_app_model(mode=AppMode.COMPLETION)
def delete(self, session: Session, current_user: Account, app_model: App, conversation_id: UUID):
def delete(self, current_user: Account, app_model: App, conversation_id: UUID):
conversation_id_str = str(conversation_id)
try:
ConversationService.delete(app_model, conversation_id_str, current_user, session=session)
ConversationService.delete(app_model, conversation_id_str, current_user, session=db.session())
except ConversationNotExistsError:
raise NotFound("Conversation Not Exists.")
@@ -236,9 +220,8 @@ class ChatConversationApi(Resource):
@edit_permission_required
@with_current_user
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_VIEW_LAYOUT)
@with_session(write=False)
@get_app_model(mode=[AppMode.CHAT, AppMode.AGENT_CHAT, AppMode.ADVANCED_CHAT, AppMode.AGENT])
def get(self, session: Session, current_user: Account, app_model: App):
def get(self, current_user: Account, app_model: App):
args = ChatConversationQuery.model_validate(request.args.to_dict(flat=True))
subquery = (
@@ -328,18 +311,9 @@ class ChatConversationApi(Resource):
case _:
query = query.order_by(Conversation.created_at.desc())
conversations = paginate_query(query, session=session, page=args.page, per_page=args.limit)
conversations = paginate_query(query, page=args.page, per_page=args.limit)
return dump_response(
ConversationWithSummaryPaginationResponse,
{
"page": conversations.page,
"per_page": conversations.per_page,
"total": conversations.total,
"has_next": conversations.has_next,
"items": [ConversationResponseSource(item, session=session) for item in conversations.items],
},
)
return dump_response(ConversationWithSummaryPaginationResponse, conversations)
@console_ns.route("/apps/<uuid:app_id>/chat-conversations/<uuid:conversation_id>")
@@ -356,15 +330,11 @@ class ChatConversationDetailApi(Resource):
@edit_permission_required
@with_current_user
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_VIEW_LAYOUT)
@with_session
@get_app_model(mode=[AppMode.CHAT, AppMode.AGENT_CHAT, AppMode.ADVANCED_CHAT, AppMode.AGENT])
def get(self, session: Session, current_user: Account, app_model: App, conversation_id: UUID):
def get(self, current_user: Account, app_model: App, conversation_id: UUID):
conversation_id_str = str(conversation_id)
return dump_response(
ConversationDetailResponse,
ConversationResponseSource(
_get_conversation(session, current_user, app_model, conversation_id_str), session=session
),
ConversationDetailResponse, _get_conversation(current_user, app_model, conversation_id_str)
)
@console_ns.doc("delete_chat_conversation")
@@ -379,28 +349,27 @@ class ChatConversationDetailApi(Resource):
@edit_permission_required
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_EDIT)
@with_current_user
@with_session
@get_app_model(mode=[AppMode.CHAT, AppMode.AGENT_CHAT, AppMode.ADVANCED_CHAT, AppMode.AGENT])
def delete(self, session: Session, current_user: Account, app_model: App, conversation_id: UUID):
def delete(self, current_user: Account, app_model: App, conversation_id: UUID):
conversation_id_str = str(conversation_id)
try:
ConversationService.delete(app_model, conversation_id_str, current_user, session=session)
ConversationService.delete(app_model, conversation_id_str, current_user, session=db.session())
except ConversationNotExistsError:
raise NotFound("Conversation Not Exists.")
return "", 204
def _get_conversation(session: Session, current_user: Account, app_model, conversation_id):
conversation = session.scalar(
def _get_conversation(current_user: Account, app_model, conversation_id):
conversation = db.session.scalar(
sa.select(Conversation).where(Conversation.id == conversation_id, Conversation.app_id == app_model.id).limit(1)
)
if not conversation:
raise NotFound("Conversation Not Exists.")
session.execute(
db.session.execute(
sa.update(Conversation)
.where(Conversation.id == conversation_id, Conversation.read_at.is_(None))
# Keep updated_at unchanged when only marking a conversation as read.
@@ -410,7 +379,7 @@ def _get_conversation(session: Session, current_user: Account, app_model, conver
updated_at=Conversation.updated_at,
)
)
session.flush()
session.refresh(conversation)
db.session.commit()
db.session.refresh(conversation)
return conversation
+2 -2
View File
@@ -153,8 +153,8 @@ class AppMCPServerController(Resource):
select(AppMCPServer)
.where(
AppMCPServer.id == server_ref.server_id,
AppMCPServer.tenant_id == server_ref.app.tenant_id,
AppMCPServer.app_id == server_ref.app.app_id,
AppMCPServer.tenant_id == server_ref.tenant_id,
AppMCPServer.app_id == server_ref.app_id,
)
.limit(1)
)
+41 -74
View File
@@ -6,13 +6,11 @@ from flask import request
from flask_restx import Resource
from pydantic import BaseModel, Field, field_validator
from sqlalchemy import exists, func, select
from sqlalchemy.orm import Session
from werkzeug.exceptions import InternalServerError, NotFound
from controllers.common.controller_schemas import MessageFeedbackPayload as _MessageFeedbackPayloadBase
from controllers.common.fields import SimpleResultResponse, TextFileResponse
from controllers.common.schema import query_params_from_model, register_response_schema_models, register_schema_models
from controllers.common.session import with_session
from controllers.console import console_ns
from controllers.console.agent.app_helpers import resolve_agent_runtime_app_model
from controllers.console.app.error import (
@@ -41,7 +39,6 @@ from fields.base import ResponseModel
from fields.conversation_fields import (
MessageDetail as BaseMessageDetailResponse,
)
from fields.conversation_fields import MessageResponseSource
from graphon.model_runtime.errors.invoke import InvokeError
from libs.helper import dump_response, uuid_value
from libs.infinite_scroll_pagination import InfiniteScrollPagination
@@ -155,10 +152,9 @@ class ChatMessageListApi(Resource):
@edit_permission_required
@with_current_user
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_VIEW_LAYOUT)
@with_session(write=False)
@get_app_model(mode=[AppMode.CHAT, AppMode.AGENT_CHAT, AppMode.ADVANCED_CHAT, AppMode.AGENT])
def get(self, session: Session, current_user: Account, app_model: App):
return _list_chat_messages(session=session, app_model=app_model, current_user=current_user)
def get(self, current_user: Account, app_model: App):
return _list_chat_messages(app_model=app_model, current_user=current_user)
@console_ns.route("/agent/<uuid:agent_id>/chat-messages")
@@ -176,14 +172,9 @@ class AgentChatMessageListApi(Resource):
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_VIEW_LAYOUT)
@with_current_user
@with_current_tenant_id
@with_session(write=False)
def get(self, session: Session, current_tenant_id: str, current_user: Account, agent_id: UUID):
app_model = resolve_agent_runtime_app_model(
session=session,
tenant_id=current_tenant_id,
agent_id=agent_id,
)
return _list_chat_messages(session=session, app_model=app_model, current_user=current_user)
def get(self, current_tenant_id: str, current_user: Account, agent_id: UUID):
app_model = resolve_agent_runtime_app_model(tenant_id=current_tenant_id, agent_id=agent_id)
return _list_chat_messages(app_model=app_model, current_user=current_user)
@console_ns.route("/apps/<uuid:app_id>/feedbacks")
@@ -199,10 +190,9 @@ class MessageFeedbackApi(Resource):
@login_required
@account_initialization_required
@with_current_user
@with_session
@get_app_model
def post(self, session: Session, current_user: Account, app_model: App):
return _update_message_feedback(session=session, current_user=current_user, app_model=app_model)
def post(self, current_user: Account, app_model: App):
return _update_message_feedback(current_user=current_user, app_model=app_model)
@console_ns.route("/agent/<uuid:agent_id>/feedbacks")
@@ -218,14 +208,9 @@ class AgentMessageFeedbackApi(Resource):
@account_initialization_required
@with_current_user
@with_current_tenant_id
@with_session
def post(self, session: Session, current_tenant_id: str, current_user: Account, agent_id: UUID):
app_model = resolve_agent_runtime_app_model(
session=session,
tenant_id=current_tenant_id,
agent_id=agent_id,
)
return _update_message_feedback(session=session, current_user=current_user, app_model=app_model)
def post(self, current_tenant_id: str, current_user: Account, agent_id: UUID):
app_model = resolve_agent_runtime_app_model(tenant_id=current_tenant_id, agent_id=agent_id)
return _update_message_feedback(current_user=current_user, app_model=app_model)
@console_ns.route("/apps/<uuid:app_id>/annotations/count")
@@ -267,12 +252,9 @@ class MessageSuggestedQuestionApi(Resource):
@account_initialization_required
@with_current_user
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_VIEW_LAYOUT)
@with_session(write=False)
@get_app_model(mode=[AppMode.CHAT, AppMode.AGENT_CHAT, AppMode.ADVANCED_CHAT, AppMode.AGENT])
def get(self, session: Session, current_user: Account, app_model: App, message_id: UUID):
return _get_message_suggested_questions(
session=session, current_user=current_user, app_model=app_model, message_id=message_id
)
def get(self, current_user: Account, app_model: App, message_id: UUID):
return _get_message_suggested_questions(current_user=current_user, app_model=app_model, message_id=message_id)
@console_ns.route("/agent/<uuid:agent_id>/chat-messages/<uuid:message_id>/suggested-questions")
@@ -291,16 +273,9 @@ class AgentMessageSuggestedQuestionApi(Resource):
@account_initialization_required
@with_current_user
@with_current_tenant_id
@with_session(write=False)
def get(self, session: Session, current_tenant_id: str, current_user: Account, agent_id: UUID, message_id: UUID):
app_model = resolve_agent_runtime_app_model(
session=session,
tenant_id=current_tenant_id,
agent_id=agent_id,
)
return _get_message_suggested_questions(
session=session, current_user=current_user, app_model=app_model, message_id=message_id
)
def get(self, current_tenant_id: str, current_user: Account, agent_id: UUID, message_id: UUID):
app_model = resolve_agent_runtime_app_model(tenant_id=current_tenant_id, agent_id=agent_id)
return _get_message_suggested_questions(current_user=current_user, app_model=app_model, message_id=message_id)
@console_ns.route("/apps/<uuid:app_id>/feedbacks/export")
@@ -358,10 +333,9 @@ class MessageApi(Resource):
@login_required
@account_initialization_required
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_VIEW_LAYOUT)
@with_session(write=False)
@get_app_model
def get(self, session: Session, app_model: App, message_id: UUID):
return _get_message_detail(session=session, app_model=app_model, message_id=message_id)
def get(self, app_model: App, message_id: UUID):
return _get_message_detail(app_model=app_model, message_id=message_id)
@console_ns.route("/agent/<uuid:agent_id>/messages/<uuid:message_id>")
@@ -375,17 +349,12 @@ class AgentMessageApi(Resource):
@login_required
@account_initialization_required
@with_current_tenant_id
@with_session(write=False)
def get(self, session: Session, current_tenant_id: str, agent_id: UUID, message_id: UUID):
app_model = resolve_agent_runtime_app_model(
session=session,
tenant_id=current_tenant_id,
agent_id=agent_id,
)
return _get_message_detail(session=session, app_model=app_model, message_id=message_id)
def get(self, current_tenant_id: str, agent_id: UUID, message_id: UUID):
app_model = resolve_agent_runtime_app_model(tenant_id=current_tenant_id, agent_id=agent_id)
return _get_message_detail(app_model=app_model, message_id=message_id)
def _list_chat_messages(*, session: Session, app_model: App, current_user: Account | None = None):
def _list_chat_messages(*, app_model: App, current_user: Account | None = None):
args = ChatMessagesQuery.model_validate(request.args.to_dict())
if AppMode.value_of(app_model.mode) == AppMode.AGENT and current_user is not None:
@@ -394,12 +363,12 @@ def _list_chat_messages(*, session: Session, app_model: App, current_user: Accou
app_model=app_model,
conversation_id=args.conversation_id,
user=current_user,
session=session,
session=db.session(),
)
except ConversationNotExistsError:
raise NotFound("Conversation Not Exists.")
else:
conversation = session.scalar(
conversation = db.session.scalar(
select(Conversation)
.where(Conversation.id == args.conversation_id, Conversation.app_id == app_model.id)
.limit(1)
@@ -409,14 +378,14 @@ def _list_chat_messages(*, session: Session, app_model: App, current_user: Accou
raise NotFound("Conversation Not Exists.")
if args.first_id:
first_message = session.scalar(
first_message = db.session.scalar(
select(Message).where(Message.conversation_id == conversation.id, Message.id == args.first_id).limit(1)
)
if not first_message:
raise NotFound("First message not found")
history_messages = session.scalars(
history_messages = db.session.scalars(
select(Message)
.where(
Message.conversation_id == conversation.id,
@@ -427,7 +396,7 @@ def _list_chat_messages(*, session: Session, app_model: App, current_user: Accou
.limit(args.limit)
).all()
else:
history_messages = session.scalars(
history_messages = db.session.scalars(
select(Message)
.where(Message.conversation_id == conversation.id)
.order_by(Message.created_at.desc())
@@ -438,7 +407,7 @@ def _list_chat_messages(*, session: Session, app_model: App, current_user: Accou
if len(history_messages) == args.limit:
current_page_first_message = history_messages[-1]
# Check if there are more messages before the current page
has_more = session.scalar(
has_more = db.session.scalar(
select(
exists().where(
Message.conversation_id == conversation.id,
@@ -456,28 +425,26 @@ def _list_chat_messages(*, session: Session, app_model: App, current_user: Accou
return dump_response(
MessageInfiniteScrollPaginationResponse,
InfiniteScrollPagination(
data=[MessageResponseSource(message, session=session) for message in history_messages],
limit=args.limit,
has_more=has_more,
),
InfiniteScrollPagination(data=history_messages, limit=args.limit, has_more=has_more),
)
def _update_message_feedback(*, session: Session, current_user: Account, app_model: App):
def _update_message_feedback(*, current_user: Account, app_model: App):
args = MessageFeedbackPayload.model_validate(console_ns.payload)
message_id = args.message_id
message = session.scalar(select(Message).where(Message.id == message_id, Message.app_id == app_model.id).limit(1))
message = db.session.scalar(
select(Message).where(Message.id == message_id, Message.app_id == app_model.id).limit(1)
)
if not message:
raise NotFound("Message Not Exists.")
feedback = message.admin_feedback_with_session(session=session)
feedback = message.admin_feedback
if not args.rating and feedback:
session.delete(feedback)
db.session.delete(feedback)
elif args.rating and feedback:
feedback.rating = FeedbackRating(args.rating)
feedback.content = args.content
@@ -496,14 +463,14 @@ def _update_message_feedback(*, session: Session, current_user: Account, app_mod
from_source=FeedbackFromSource.ADMIN,
from_account_id=current_user.id,
)
session.add(feedback)
db.session.add(feedback)
session.commit()
db.session.commit()
return SimpleResultResponse(result="success").model_dump(mode="json")
def _get_message_suggested_questions(*, session: Session, current_user: Account, app_model: App, message_id: UUID):
def _get_message_suggested_questions(*, current_user: Account, app_model: App, message_id: UUID):
message_id_str = str(message_id)
try:
@@ -512,7 +479,7 @@ def _get_message_suggested_questions(*, session: Session, current_user: Account,
message_id=message_id_str,
user=current_user,
invoke_from=InvokeFrom.DEBUGGER,
session=session,
session=db.session(),
)
except MessageNotExistsError:
raise NotFound("Message not found")
@@ -535,10 +502,10 @@ def _get_message_suggested_questions(*, session: Session, current_user: Account,
return dump_response(SuggestedQuestionsResponse, {"data": questions})
def _get_message_detail(*, session: Session, app_model: App, message_id: UUID):
def _get_message_detail(*, app_model: App, message_id: UUID):
message_id_str = str(message_id)
message = session.scalar(
message = db.session.scalar(
select(Message).where(Message.id == message_id_str, Message.app_id == app_model.id).limit(1)
)
@@ -546,4 +513,4 @@ def _get_message_detail(*, session: Session, app_model: App, message_id: UUID):
raise NotFound("Message Not Exists.")
attach_message_extra_contents([message])
return dump_response(MessageDetailResponse, MessageResponseSource(message, session=session))
return dump_response(MessageDetailResponse, message)
+10 -15
View File
@@ -4,11 +4,9 @@ from typing import Any, cast
from flask import request
from flask_restx import Resource
from pydantic import BaseModel, Field
from sqlalchemy.orm import Session
from controllers.common.fields import SimpleResultResponse
from controllers.common.schema import register_response_schema_models, register_schema_models
from controllers.common.session import with_session
from controllers.console import console_ns
from controllers.console.app.wraps import get_app_model
from controllers.console.wraps import (
@@ -25,6 +23,7 @@ from core.agent.entities import AgentToolEntity
from core.tools.tool_manager import ToolManager
from core.tools.utils.configuration import ToolParameterConfigurationManager
from events.app_event import app_model_config_was_updated
from extensions.ext_database import db
from libs.datetime_utils import naive_utc_now
from libs.login import login_required
from models.model import App, AppMode, AppModelConfig
@@ -94,16 +93,14 @@ class ModelConfigResource(Resource):
@account_initialization_required
@with_current_user_id
@with_current_tenant_id
@with_session
@get_app_model(mode=[AppMode.AGENT_CHAT, AppMode.CHAT, AppMode.COMPLETION])
def post(self, session: Session, current_tenant_id: str, current_user_id: str, app_model: App):
"""Modify the app model config and dataset joins in one request transaction."""
def post(self, current_tenant_id: str, current_user_id: str, app_model: App):
"""Modify app model config"""
# validate config
model_configuration = AppModelConfigService.validate_configuration(
tenant_id=current_tenant_id,
config=cast(dict, request.json),
app_mode=AppMode.value_of(app_model.mode),
session=session,
)
new_app_model_config = AppModelConfig(
@@ -113,8 +110,9 @@ class ModelConfigResource(Resource):
)
new_app_model_config = new_app_model_config.from_model_config_dict(model_configuration)
if app_model.mode == AppMode.AGENT_CHAT or app_model.is_agent_with_session(session=session):
original_app_model_config = app_model.app_model_config_with_session(session=session)
if app_model.mode == AppMode.AGENT_CHAT or app_model.is_agent:
# get original app model config
original_app_model_config = db.session.get(AppModelConfig, app_model.app_model_config_id)
if original_app_model_config is None:
raise ValueError("Original app model config not found")
agent_mode = original_app_model_config.agent_mode_dict
@@ -206,17 +204,14 @@ class ModelConfigResource(Resource):
# update app model config
new_app_model_config.agent_mode = json.dumps(agent_mode)
session.add(new_app_model_config)
session.flush()
db.session.add(new_app_model_config)
db.session.flush()
app_model.app_model_config_id = new_app_model_config.id
app_model.updated_by = current_user_id
app_model.updated_at = naive_utc_now()
db.session.commit()
app_model_config_was_updated.send(
app_model,
app_model_config=new_app_model_config,
session=session,
)
app_model_config_was_updated.send(app_model, app_model_config=new_app_model_config)
return {"result": "success"}
+9 -14
View File
@@ -3,14 +3,12 @@ from typing import Literal
from flask_restx import Resource
from pydantic import BaseModel, Field, field_validator
from sqlalchemy import select
from sqlalchemy.orm import Session
from werkzeug.exceptions import NotFound
from constants.languages import supported_language
from controllers.common.schema import register_schema_models
from controllers.common.session import with_session
from controllers.console import console_ns
from controllers.console.app.wraps import agent_manage_required_for_agent_app, get_app_model
from controllers.console.app.wraps import get_app_model
from controllers.console.wraps import (
RBACPermission,
RBACResourceScope,
@@ -21,6 +19,7 @@ from controllers.console.wraps import (
setup_required,
with_current_user,
)
from extensions.ext_database import db
from fields.base import ResponseModel
from libs.datetime_utils import naive_utc_now
from libs.helper import dump_response
@@ -93,14 +92,12 @@ class AppSite(Resource):
@login_required
@edit_permission_required
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_RELEASE_AND_VERSION)
@agent_manage_required_for_agent_app
@account_initialization_required
@with_current_user
@with_session
@get_app_model
def post(self, session: Session, current_user: Account, app_model: App):
def post(self, current_user: Account, app_model: App):
args = AppSiteUpdatePayload.model_validate(console_ns.payload or {})
site = session.scalar(select(Site).where(Site.app_id == app_model.id).limit(1))
site = db.session.scalar(select(Site).where(Site.app_id == app_model.id).limit(1))
if not site:
raise NotFound
@@ -129,7 +126,7 @@ class AppSite(Resource):
site.updated_by = current_user.id
site.updated_at = naive_utc_now()
session.flush()
db.session.commit()
return dump_response(AppSiteResponse, site)
@@ -146,20 +143,18 @@ class AppSiteAccessTokenReset(Resource):
@login_required
@is_admin_or_owner_required
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_RELEASE_AND_VERSION)
@agent_manage_required_for_agent_app
@account_initialization_required
@with_current_user
@with_session
@get_app_model
def post(self, session: Session, current_user: Account, app_model: App):
site = session.scalar(select(Site).where(Site.app_id == app_model.id).limit(1))
def post(self, current_user: Account, app_model: App):
site = db.session.scalar(select(Site).where(Site.app_id == app_model.id).limit(1))
if not site:
raise NotFound
site.code = Site.generate_code(16, session=session)
site.code = Site.generate_code(16)
site.updated_by = current_user.id
site.updated_at = naive_utc_now()
session.flush()
db.session.commit()
return dump_response(AppSiteResponse, site)
+36 -50
View File
@@ -58,7 +58,6 @@ from extensions.ext_database import db
from extensions.ext_redis import redis_client
from factories import file_factory, variable_factory
from fields.base import ResponseModel
from fields.conversation_variable_fields import WorkflowConversationVariableResponse
from fields.member_fields import SimpleAccount
from fields.workflow_run_fields import WorkflowRunNodeExecutionResponse
from graphon.enums import NodeType
@@ -105,7 +104,6 @@ class SyncDraftWorkflowPayload(BaseModel):
graph: dict[str, Any]
features: dict[str, Any]
hash: str | None = None
is_collaborative: bool = Field(default=False, alias="_is_collaborative")
environment_variables: list[dict[str, Any]] = Field(
default_factory=list,
)
@@ -240,6 +238,21 @@ class WorkflowOnlineUsersPayload(BaseModel):
return list(dict.fromkeys(app_id.strip() for app_id in app_ids if app_id.strip()))
class WorkflowConversationVariableResponse(ResponseModel):
id: str
name: str
value_type: str
value: Any
description: str
@field_validator("value_type", mode="before")
@classmethod
def _serialize_value_type(cls, value: Any) -> str:
if hasattr(value, "exposed_type"):
return str(value.exposed_type())
return str(value)
class PipelineVariableResponse(ResponseModel):
label: str
variable: str
@@ -311,27 +324,6 @@ class WorkflowResponse(ResponseModel):
return [_serialize_environment_variable(item) for item in value]
class _WorkflowResponseSource:
def __init__(self, workflow: Workflow, *, session: Session) -> None:
self._workflow = workflow
self._session = session
def __getattr__(self, name: str) -> object:
return getattr(self._workflow, name) # noqa: no-new-getattr response adapter delegates model fields
@property
def created_by_account(self) -> Account | None:
return self._workflow.get_created_by_account(session=self._session)
@property
def updated_by_account(self) -> Account | None:
return self._workflow.get_updated_by_account(session=self._session)
@property
def tool_published(self) -> bool:
return self._workflow.get_tool_published(session=self._session)
class WorkflowPaginationResponse(ResponseModel):
items: list[WorkflowResponse]
page: int
@@ -359,12 +351,6 @@ class WorkflowPublishResponse(ResponseModel):
created_at: int
class SyncDraftWorkflowResponse(ResponseModel):
result: str
hash: str
updated_at: int
class WorkflowRestoreResponse(ResponseModel):
result: str
hash: str
@@ -447,7 +433,6 @@ register_response_schema_models(
WorkflowOnlineUsersByApp,
WorkflowOnlineUsersResponse,
WorkflowPublishResponse,
SyncDraftWorkflowResponse,
WorkflowRestoreResponse,
DefaultBlockConfigsResponse,
DefaultBlockConfigResponse,
@@ -563,7 +548,14 @@ class DraftWorkflowApi(Resource):
@console_ns.response(
200,
"Draft workflow synced successfully",
console_ns.models[SyncDraftWorkflowResponse.__name__],
console_ns.model(
"SyncDraftWorkflowResponse",
{
"result": fields.String,
"hash": fields.String,
"updated_at": fields.String,
},
),
)
@console_ns.response(400, "Invalid workflow configuration")
@console_ns.response(403, "Permission denied")
@@ -611,21 +603,17 @@ class DraftWorkflowApi(Resource):
environment_variables=environment_variables,
conversation_variables=conversation_variables,
session=db.session(),
graph_only=args["is_collaborative"],
)
except WorkflowHashNotEqualError:
raise DraftWorkflowNotSync()
except VariableError as e:
raise InvalidArgumentError(description=str(e))
return dump_response(
SyncDraftWorkflowResponse,
{
"result": "success",
"hash": workflow.unique_hash,
"updated_at": TimestampField().format(workflow.updated_at or workflow.created_at),
},
)
return {
"result": "success",
"hash": workflow.unique_hash,
"updated_at": TimestampField().format(workflow.updated_at or workflow.created_at),
}
@console_ns.route("/apps/<uuid:app_id>/advanced-chat/workflows/draft/run")
@@ -641,10 +629,10 @@ class AdvancedChatDraftWorkflowRunApi(Resource):
@login_required
@account_initialization_required
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_TEST_AND_RUN)
@get_app_model(mode=[AppMode.ADVANCED_CHAT])
@with_current_user
@edit_permission_required
@with_session
@get_app_model(mode=[AppMode.ADVANCED_CHAT])
def post(self, session: Session, current_user: Account, app_model: App):
"""
Run draft workflow
@@ -1086,10 +1074,10 @@ class DraftWorkflowRunApi(Resource):
@login_required
@account_initialization_required
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_TEST_AND_RUN)
@get_app_model(mode=[AppMode.WORKFLOW])
@with_current_user
@edit_permission_required
@with_session
@get_app_model(mode=[AppMode.WORKFLOW])
def post(self, session: Session, current_user: Account, app_model: App):
"""
Run draft workflow
@@ -1450,7 +1438,7 @@ class PublishedAllWorkflowApi(Resource):
)
return WorkflowPaginationResponse.model_validate(
{
"items": [_WorkflowResponseSource(workflow, session=session) for workflow in workflows],
"items": workflows,
"page": page,
"limit": limit,
"has_more": has_more,
@@ -1544,9 +1532,7 @@ class WorkflowByIdApi(Resource):
if not workflow:
raise NotFound("Workflow not found")
response = dump_response(WorkflowResponse, _WorkflowResponseSource(workflow, session=session))
return response
return dump_response(WorkflowResponse, workflow)
@setup_required
@login_required
@@ -1640,10 +1626,10 @@ class DraftWorkflowTriggerRunApi(Resource):
@login_required
@account_initialization_required
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_TEST_AND_RUN)
@get_app_model(mode=[AppMode.WORKFLOW])
@with_current_user
@edit_permission_required
@with_session
@get_app_model(mode=[AppMode.WORKFLOW])
def post(self, session: Session, current_user: Account, app_model: App):
"""
Poll for trigger events and execute full workflow when event arrives
@@ -1651,7 +1637,7 @@ class DraftWorkflowTriggerRunApi(Resource):
args = DraftWorkflowTriggerRunPayload.model_validate(console_ns.payload or {})
node_id = args.node_id
workflow_service = WorkflowService()
draft_workflow = workflow_service.get_draft_workflow(app_model, session=session)
draft_workflow = workflow_service.get_draft_workflow(app_model, session=db.session())
if not draft_workflow:
raise ValueError("Workflow not found")
@@ -1791,11 +1777,11 @@ class DraftWorkflowTriggerRunAllApi(Resource):
@setup_required
@login_required
@account_initialization_required
@get_app_model(mode=[AppMode.WORKFLOW])
@with_current_user
@edit_permission_required
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_TEST_AND_RUN)
@with_session
@get_app_model(mode=[AppMode.WORKFLOW])
def post(self, session: Session, current_user: Account, app_model: App):
"""
Full workflow debug when the start node is a trigger
@@ -1804,7 +1790,7 @@ class DraftWorkflowTriggerRunAllApi(Resource):
args = DraftWorkflowTriggerRunAllPayload.model_validate(console_ns.payload or {})
node_ids = args.node_ids
workflow_service = WorkflowService()
draft_workflow = workflow_service.get_draft_workflow(app_model, session=session)
draft_workflow = workflow_service.get_draft_workflow(app_model, session=db.session())
if not draft_workflow:
raise ValueError("Workflow not found")
+10 -59
View File
@@ -1,8 +1,9 @@
"""Controller decorators for console app resources.
`get_app_model` still supports legacy handlers backed by Flask-SQLAlchemy's
scoped session. Trial app handlers compose `get_app_model_with_trial` under
`controllers.common.session.with_session` and always reuse that request session.
App-loading decorators prefer a session injected by
`controllers.common.session.with_session` when present, while still supporting
existing handlers that have not been migrated yet and still rely on
Flask-SQLAlchemy's scoped `db.session`.
"""
from collections.abc import Callable
@@ -12,22 +13,14 @@ from typing import cast, overload
from sqlalchemy import select
from sqlalchemy.orm import Session
from configs import dify_config
from controllers.common.session import with_session
from controllers.common.wraps import RBACPermission, RBACResourceScope, enforce_rbac_access
from controllers.console.app.error import AppNotFoundError
from extensions.ext_database import db
from libs.login import current_account_with_tenant
from models import App, AppMode, TrialApp
from models.agent import AgentScope
from services.recommended_app_service import RecommendedAppService
__all__ = [
"agent_manage_required_for_agent_app",
"get_app_model",
"get_app_model_with_trial",
"with_session",
]
__all__ = ["get_app_model", "get_app_model_with_trial", "with_session"]
def _load_app_model(session: Session, app_id: str) -> App | None:
@@ -48,53 +41,14 @@ def _load_app_model_from_scoped_session(app_id: str) -> App | None:
return app_model
def _load_app_model_with_trial(session: Session, app_id: str) -> App | None:
def _load_app_model_with_trial(app_id: str) -> App | None:
"""Load a normal app through its trial registration without applying current-tenant scope."""
app_model = session.scalar(
app_model = db.session.scalar(
select(App).join(TrialApp, TrialApp.app_id == App.id).where(App.id == app_id, App.status == "normal").limit(1)
)
return app_model
def agent_manage_required_for_agent_app[**P, R](view: Callable[P, R]) -> Callable[P, R]:
"""Gate generic app management routes that target an Agent App.
A hidden workflow-only backing App only reuses the App runtime and is not
part of the general app management plane, so generic routes reject it
outright. Managing a roster Agent App mutates the roster Agent behind it
(rename/icon sync, archive, API enablement), so it additionally requires
workspace ``agent.manage`` on top of the route's existing App permission
checks when RBAC is enabled. A no-op for non-agent Apps. Must be placed
above ``get_app_model`` so the ``app_id`` path parameter is still present.
"""
@wraps(view)
def decorated(*args: P.args, **kwargs: P.kwargs) -> R:
raw_app_id = kwargs.get("app_id") or kwargs.get("resource_id")
if raw_app_id is not None:
app_model = _load_app_model_from_scoped_session(str(raw_app_id))
binding = (
app_model.agent_app_binding_with_session(session=db.session(), include_archived=True)
if app_model is not None
else None
)
if binding is not None:
if binding.scope == AgentScope.WORKFLOW_ONLY:
raise AppNotFoundError()
if dify_config.RBAC_ENABLED:
current_user, current_tenant_id = current_account_with_tenant()
enforce_rbac_access(
tenant_id=current_tenant_id,
account_id=current_user.id,
resource_type=RBACResourceScope.WORKSPACE,
scene=RBACPermission.AGENT_MANAGE,
resource_required=False,
)
return view(*args, **kwargs)
return decorated
def _get_injected_session(args: tuple[object, ...]) -> Session | None:
"""Return the request session inserted by `with_session`, if this handler has been migrated."""
if len(args) < 2:
@@ -203,7 +157,7 @@ def get_app_model_with_trial[**P, R](
*,
mode: AppMode | list[AppMode] | None = None,
) -> Callable[P, R] | Callable[[Callable[P, R]], Callable[P, R]]:
"""Inject a trial-registered or recommended App using the Session supplied by `with_session`."""
"""Inject an app registered for trial or available from the recommended catalog."""
def decorator(view_func: Callable[P, R]) -> Callable[P, R]:
@wraps(view_func)
@@ -216,12 +170,9 @@ def get_app_model_with_trial[**P, R](
del kwargs["app_id"]
session = _get_injected_session(args)
if session is None:
raise RuntimeError("get_app_model_with_trial requires @with_session")
app_model = _load_app_model_with_trial(session, app_id)
app_model = _load_app_model_with_trial(app_id)
if app_model is None:
app_model = RecommendedAppService.get_app(app_id, session=session)
app_model = RecommendedAppService.get_app(app_id, session=db.session())
if not app_model:
raise AppNotFoundError()
-14
View File
@@ -7,13 +7,10 @@ from configs import dify_config
from constants.languages import supported_language
from controllers.common.schema import query_params_from_model, register_schema_models
from controllers.console import console_ns
from controllers.console.auth.error import InvitationAccountMismatchError
from controllers.console.error import AccountInFreezeError, AlreadyActivateError
from extensions.ext_database import db
from libs.datetime_utils import naive_utc_now
from libs.helper import EmailStr, timezone
from libs.login import current_account_with_tenant
from libs.token import extract_access_token
from models import AccountStatus
from models.account import TenantAccountJoin, TenantAccountRole
from services.account_service import RegisterService, TenantService
@@ -139,12 +136,6 @@ class ActivateApi(Resource):
)
@console_ns.response(400, "Already activated or invalid token")
def post(self):
"""Accept an invitation without letting an existing session act for another account.
Token-only activation remains available for legacy clients. When the request already
carries a console session, that session must belong to the account encoded in the
invitation before the token is consumed or tenant membership is changed.
"""
args = ActivatePayload.model_validate(console_ns.payload)
normalized_request_email = args.email.lower() if args.email else None
@@ -155,11 +146,6 @@ class ActivateApi(Resource):
raise AlreadyActivateError()
account = invitation["account"]
if extract_access_token(request):
current_account, _ = current_account_with_tenant()
if current_account.id != account.id:
raise InvitationAccountMismatchError()
if dify_config.BILLING_ENABLED and BillingService.is_email_in_freeze(account.email):
raise AccountInFreezeError()
-6
View File
@@ -13,12 +13,6 @@ class InvalidEmailError(BaseHTTPException):
code = 400
class InvitationAccountMismatchError(BaseHTTPException):
error_code = "invitation_account_mismatch"
description = "This invitation was sent to another account. Please sign in with the invited account."
code = 403
class PasswordMismatchError(BaseHTTPException):
error_code = "password_mismatch"
description = "The passwords do not match."
@@ -16,6 +16,7 @@ from controllers.console.auth.error import (
)
from controllers.console.error import AccountNotFound, EmailSendIpLimitError
from controllers.console.wraps import email_password_login_enabled, setup_required
from events.tenant_event import tenant_was_created
from extensions.ext_database import db
from libs.helper import EmailStr, extract_remote_ip
from libs.password import hash_password
@@ -200,4 +201,7 @@ class ForgotPasswordResetApi(Resource):
not TenantService.get_join_tenants(account, session=db.session())
and FeatureService.get_system_features().is_allow_create_workspace
):
TenantService.create_owner_tenant(account, session=db.session())
tenant = TenantService.create_tenant(f"{account.name}'s Workspace", session=db.session())
TenantService.create_tenant_member(tenant, account, db.session(), role="owner")
account.current_tenant = tenant
tenant_was_created.send(tenant)
+7 -6
View File
@@ -4,7 +4,6 @@ import flask_login
from flask import make_response, request
from flask_restx import Resource
from pydantic import BaseModel, Field, field_validator
from sqlalchemy.orm import Session
from werkzeug.exceptions import Unauthorized
import services
@@ -17,7 +16,6 @@ from controllers.common.fields import (
SimpleResultResponse,
)
from controllers.common.schema import register_response_schema_models, register_schema_models
from controllers.common.session import with_session
from controllers.console import console_ns
from controllers.console.auth.error import (
AuthenticationFailedError,
@@ -42,6 +40,7 @@ from controllers.console.wraps import (
setup_required,
with_current_user,
)
from events.tenant_event import tenant_was_created
from extensions.ext_database import db
from libs.helper import EmailStr, extract_remote_ip
from libs.helper import timezone as validate_timezone_string
@@ -316,7 +315,10 @@ class EmailCodeLoginApi(Resource):
if not FeatureService.get_system_features().is_allow_create_workspace:
raise NotAllowedCreateWorkspace()
else:
TenantService.create_owner_tenant(account, session=db.session())
new_tenant = TenantService.create_tenant(f"{account.name}'s Workspace", session=db.session())
TenantService.create_tenant_member(new_tenant, account, db.session(), role="owner")
account.current_tenant = new_tenant
tenant_was_created.send(new_tenant)
if account is None:
try:
@@ -354,8 +356,7 @@ class EmailCodeLoginApi(Resource):
class RefreshTokenApi(Resource):
@console_ns.response(200, "Success", console_ns.models[SimpleResultResponse.__name__])
@console_ns.response(401, "Unauthorized", console_ns.models[SimpleResultMessageResponse.__name__])
@with_session(write=False)
def post(self, session: Session):
def post(self):
# Get refresh token from cookie instead of request body
refresh_token = extract_refresh_token(request)
@@ -365,7 +366,7 @@ class RefreshTokenApi(Resource):
), 401
try:
new_token_pair = AccountService.refresh_token(refresh_token, session=session)
new_token_pair = AccountService.refresh_token(refresh_token, session=db.session())
except Unauthorized as exc:
return SimpleResultMessageResponse(result="fail", message=exc.description or "Unauthorized.").model_dump(
mode="json"
+26 -36
View File
@@ -6,12 +6,12 @@ from flask import current_app, redirect, request
from flask_restx import Resource
from pydantic import BaseModel, Field
from werkzeug.exceptions import Unauthorized
from werkzeug.wrappers import Response
from configs import dify_config
from constants.languages import languages
from controllers.common.fields import RedirectResponse
from controllers.common.schema import query_params_from_model, register_response_schema_model, register_schema_models
from events.tenant_event import tenant_was_created
from extensions.ext_database import db
from libs.datetime_utils import naive_utc_now
from libs.helper import extract_remote_ip
@@ -128,20 +128,6 @@ def _preferred_interface_language(language: str | None = None) -> str:
return languages[0]
def _redirect_with_console_session(account: Account, target_url: str) -> Response:
"""Create a console session and attach its cookies to a redirect response."""
token_pair = AccountService.login(
account=account,
session=db.session(),
ip_address=extract_remote_ip(request),
)
response = redirect(target_url)
set_access_token_to_cookie(request, response, token_pair.access_token)
set_refresh_token_to_cookie(request, response, token_pair.refresh_token)
set_csrf_token_to_cookie(request, response, token_pair.csrf_token)
return response
@console_ns.route("/oauth/login/<provider>")
class OAuthLogin(Resource):
@console_ns.doc("oauth_login")
@@ -210,26 +196,16 @@ class OAuthCallback(Resource):
return redirect(f"{dify_config.CONSOLE_WEB_URL}/signin?message={urllib.parse.quote(str(e))}")
if invite_token and RegisterService.is_valid_invite_token(invite_token):
invitation = RegisterService.get_invitation_if_token_valid(
None,
None,
invite_token,
session=db.session(),
)
if not invitation:
return redirect(f"{dify_config.CONSOLE_WEB_URL}/signin?message=Invalid invitation token.")
if invitation["data"]["email"].lower() != user_info.email.lower():
message = "This invitation was sent to another account. Please sign in with the invited account."
query = urllib.parse.urlencode({"message": message, "invite_token": invite_token})
return redirect(f"{dify_config.CONSOLE_WEB_URL}/signin?{query}")
invitation = RegisterService.get_invitation_by_token(token=invite_token)
if invitation:
invitation_email = invitation.get("email", None)
invitation_email_normalized = (
invitation_email.lower() if isinstance(invitation_email, str) else invitation_email
)
if invitation_email_normalized != user_info.email.lower():
return redirect(f"{dify_config.CONSOLE_WEB_URL}/signin?message=Invalid invitation token.")
account = invitation["account"]
if account.status == AccountStatus.BANNED:
return redirect(f"{dify_config.CONSOLE_WEB_URL}/signin?message=Account is banned.")
AccountService.link_account_integrate(provider, user_info.id, account, session=db.session())
target_url = f"{dify_config.CONSOLE_WEB_URL}/signin/invite-settings?invite_token={invite_token}"
return _redirect_with_console_session(account, target_url)
return redirect(f"{dify_config.CONSOLE_WEB_URL}/signin/invite-settings?invite_token={invite_token}")
try:
account, oauth_new_user = _generate_account(provider, user_info, timezone=timezone, language=language)
@@ -264,10 +240,21 @@ class OAuthCallback(Resource):
"?message=Workspace not found, please contact system admin to invite you to join in a workspace."
)
token_pair = AccountService.login(
account=account,
session=db.session(),
ip_address=extract_remote_ip(request),
)
target_url = _get_redirect_target(redirect_url)
query_char = "&" if "?" in target_url else "?"
target_url = f"{target_url}{query_char}oauth_new_user={str(oauth_new_user).lower()}"
return _redirect_with_console_session(account, target_url)
response = redirect(target_url)
set_access_token_to_cookie(request, response, token_pair.access_token)
set_refresh_token_to_cookie(request, response, token_pair.refresh_token)
set_csrf_token_to_cookie(request, response, token_pair.csrf_token)
return response
def _get_account_by_openid_or_email(provider: str, user_info: OAuthUserInfo) -> Account | None:
@@ -295,7 +282,10 @@ def _generate_account(
if not FeatureService.get_system_features().is_allow_create_workspace:
raise WorkSpaceNotAllowedCreateError()
else:
TenantService.create_owner_tenant(account, session=db.session())
new_tenant = TenantService.create_tenant(f"{account.name}'s Workspace", session=db.session())
TenantService.create_tenant_member(new_tenant, account, db.session(), role="owner")
account.current_tenant = new_tenant
tenant_was_created.send(new_tenant)
if not account:
normalized_email = user_info.email.lower()
+4 -5
View File
@@ -10,7 +10,7 @@ from werkzeug.exceptions import BadRequest, NotFound
from controllers.common.schema import register_response_schema_models, register_schema_models
from controllers.console.wraps import account_initialization_required, setup_required, with_current_user
from core.db.session_factory import session_factory
from extensions.ext_database import db
from graphon.model_runtime.utils.encoders import jsonable_encoder
from libs.login import login_required
from models import Account
@@ -132,10 +132,9 @@ def oauth_server_access_token_required[T, **P, R](
response.headers["WWW-Authenticate"] = "Bearer"
return response
with session_factory.create_session() as session:
account = OAuthServerService.validate_oauth_access_token(
oauth_provider_app.client_id, access_token, session
)
account = OAuthServerService.validate_oauth_access_token(
oauth_provider_app.client_id, access_token, db.session()
)
if not account:
response = jsonify({"error": "access_token or client_id is invalid"})
response.status_code = 401
+27 -29
View File
@@ -8,12 +8,11 @@ from flask import request
from flask_restx import Resource
from pydantic import BaseModel, Field, field_serializer
from sqlalchemy import select
from sqlalchemy.orm import Session
from sqlalchemy.orm import sessionmaker
from werkzeug.exceptions import NotFound
from controllers.common.fields import SimpleResultResponse, TextContentResponse
from controllers.common.schema import query_params_from_model, register_response_schema_models, register_schema_models
from controllers.common.session import with_session
from core.datasource.entities.datasource_entities import DatasourceProviderType, OnlineDocumentPagesMessage
from core.datasource.online_document.online_document_plugin import OnlineDocumentDatasourcePlugin
from core.entities.knowledge_entities import IndexingEstimate
@@ -189,16 +188,16 @@ class DataSourceApi(Resource):
@account_initialization_required
@console_ns.response(200, "Success", console_ns.models[SimpleResultResponse.__name__])
@with_current_tenant_id
@with_session
def patch(
self, session: Session, current_tenant_id: str, binding_id: UUID, action: Literal["enable", "disable"]
self, current_tenant_id: str, binding_id: UUID, action: Literal["enable", "disable"]
) -> tuple[dict[str, str], int]:
binding_id_str = str(binding_id)
data_source_binding = session.scalar(
select(DataSourceOauthBinding).where(
DataSourceOauthBinding.id == binding_id_str, DataSourceOauthBinding.tenant_id == current_tenant_id
)
)
with sessionmaker(db.engine, expire_on_commit=False).begin() as session:
data_source_binding = session.execute(
select(DataSourceOauthBinding).where(
DataSourceOauthBinding.id == binding_id_str, DataSourceOauthBinding.tenant_id == current_tenant_id
)
).scalar_one_or_none()
if data_source_binding is None:
raise NotFound("Data source binding not found.")
# enable binding
@@ -207,6 +206,8 @@ class DataSourceApi(Resource):
if data_source_binding.disabled:
data_source_binding.disabled = False
data_source_binding.updated_at = naive_utc_now()
db.session.add(data_source_binding)
db.session.commit()
else:
raise ValueError("Data source is not disabled.")
# disable binding
@@ -214,6 +215,8 @@ class DataSourceApi(Resource):
if not data_source_binding.disabled:
data_source_binding.disabled = True
data_source_binding.updated_at = naive_utc_now()
db.session.add(data_source_binding)
db.session.commit()
else:
raise ValueError("Data source is disabled.")
return {"result": "success"}, 200
@@ -228,8 +231,7 @@ class DataSourceNotionListApi(Resource):
@console_ns.response(200, "Success", console_ns.models[NotionIntegrateInfoListResponse.__name__])
@with_current_user
@with_current_tenant_id
@with_session(write=False)
def get(self, session: Session, current_tenant_id: str, current_user: Account) -> tuple[dict[str, Any], int]:
def get(self, current_tenant_id: str, current_user: Account) -> tuple[dict[str, Any], int]:
query = DataSourceNotionListQuery.model_validate(request.args.to_dict(flat=True))
datasource_provider_service = DatasourceProviderService()
credential = datasource_provider_service.get_datasource_credentials(
@@ -243,13 +245,13 @@ class DataSourceNotionListApi(Resource):
exist_page_ids = []
# import notion in the exist dataset
if query.dataset_id:
dataset = DatasetService.get_dataset(query.dataset_id, session)
dataset = DatasetService.get_dataset(query.dataset_id, db.session())
if not dataset:
raise NotFound("Dataset not found.")
if dataset.data_source_type != "notion_import":
raise ValueError("Dataset is not notion type.")
documents = session.scalars(
documents = db.session.scalars(
select(Document).where(
Document.dataset_id == query.dataset_id,
Document.tenant_id == current_tenant_id,
@@ -353,8 +355,7 @@ class DataSourceNotionIndexingEstimateApi(Resource):
@console_ns.expect(console_ns.models[NotionEstimatePayload.__name__])
@console_ns.response(200, "Success", console_ns.models[IndexingEstimate.__name__])
@with_current_tenant_id
@with_session
def post(self, session: Session, current_tenant_id: str) -> tuple[dict[str, Any], int]:
def post(self, current_tenant_id: str) -> tuple[dict[str, Any], int]:
payload = NotionEstimatePayload.model_validate(console_ns.payload or {})
args = payload.model_dump()
# validate args
@@ -381,12 +382,11 @@ class DataSourceNotionIndexingEstimateApi(Resource):
extract_settings.append(extract_setting)
indexing_runner = IndexingRunner()
response = indexing_runner.indexing_estimate(
tenant_id=current_tenant_id,
extract_settings=extract_settings,
tmp_processing_rule=args["process_rule"],
doc_form=args["doc_form"],
doc_language=args["doc_language"],
session=session,
current_tenant_id,
extract_settings,
args["process_rule"],
args["doc_form"],
args["doc_language"],
)
return dump_response(IndexingEstimate, response), 200
@@ -398,14 +398,13 @@ class DataSourceNotionDatasetSyncApi(Resource):
@account_initialization_required
@console_ns.response(200, "Success", console_ns.models[SimpleResultResponse.__name__])
@rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_CREATE_AND_MANAGEMENT)
@with_session(write=False)
def get(self, session: Session, dataset_id: UUID) -> tuple[dict[str, str], int]:
def get(self, dataset_id: UUID) -> tuple[dict[str, str], int]:
dataset_id_str = str(dataset_id)
dataset = DatasetService.get_dataset(dataset_id_str, session)
dataset = DatasetService.get_dataset(dataset_id_str, db.session())
if dataset is None:
raise NotFound("Dataset not found.")
documents = DocumentService.get_document_by_dataset_id(dataset_id_str, session)
documents = DocumentService.get_document_by_dataset_id(dataset_id_str, db.session())
for document in documents:
document_indexing_sync_task.delay(dataset_id_str, document.id)
return {"result": "success"}, 200
@@ -418,15 +417,14 @@ class DataSourceNotionDocumentSyncApi(Resource):
@account_initialization_required
@console_ns.response(200, "Success", console_ns.models[SimpleResultResponse.__name__])
@rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_CREATE_AND_MANAGEMENT)
@with_session(write=False)
def get(self, session: Session, dataset_id: UUID, document_id: UUID) -> tuple[dict[str, str], int]:
def get(self, dataset_id: UUID, document_id: UUID) -> tuple[dict[str, str], int]:
dataset_id_str = str(dataset_id)
document_id_str = str(document_id)
dataset = DatasetService.get_dataset(dataset_id_str, session)
dataset = DatasetService.get_dataset(dataset_id_str, db.session())
if dataset is None:
raise NotFound("Dataset not found.")
document = DocumentService.get_document(dataset_id_str, document_id_str, session=session)
document = DocumentService.get_document(dataset_id_str, document_id_str, session=db.session())
if document is None:
raise NotFound("Document not found.")
document_indexing_sync_task.delay(dataset_id_str, document_id_str)
+77 -136
View File
@@ -1,4 +1,3 @@
from dataclasses import dataclass
from datetime import datetime
from typing import Any
from uuid import UUID
@@ -14,10 +13,10 @@ import services
from configs import dify_config
from controllers.common.fields import ApiBaseUrlResponse, SimpleResultResponse, UsageCheckResponse
from controllers.common.schema import query_params_from_model, register_response_schema_models, register_schema_models
from controllers.common.session import with_session
from controllers.console import console_ns
from controllers.console.apikey import ApiKeyItem, ApiKeyList
from controllers.console.app.error import ProviderNotInitializeError
from controllers.console.app.wraps import with_session
from controllers.console.datasets.error import DatasetInUseError, DatasetNameDuplicateError, IndexingEstimateError
from controllers.console.wraps import (
RBACPermission,
@@ -40,18 +39,18 @@ from core.rag.extractor.entity.datasource_type import DatasourceType
from core.rag.extractor.entity.extract_setting import ExtractSetting, NotionInfo, WebsiteInfo
from core.rag.index_processor.constant.index_type import IndexTechniqueType
from core.rag.retrieval.retrieval_methods import RetrievalMethod
from extensions.ext_database import db
from fields.base import ResponseModel
from fields.dataset_fields import DatasetDetailResponse, dataset_detail_response_source
from fields.dataset_fields import DatasetDetailResponse
from graphon.model_runtime.entities.model_entities import ModelType
from libs.helper import build_icon_url, dump_response, to_timestamp
from libs.login import login_required
from libs.url_utils import normalize_api_base_url
from models import Account, ApiToken, App, Dataset, Document, DocumentSegment, UploadFile
from models.dataset import DatasetPermission, DatasetPermissionEnum, DatasetQuery
from models import Account, ApiToken, Dataset, Document, DocumentSegment, UploadFile
from models.dataset import DatasetPermission, DatasetPermissionEnum
from models.enums import ApiTokenType, SegmentStatus
from models.provider_ids import ModelProviderID
from services.api_token_service import ApiTokenCache
from services.app_service import AppService
from services.dataset_service import DatasetPermissionService, DatasetService, DocumentService
from services.enterprise import rbac_service as enterprise_rbac_service
from services.enterprise.rbac_service import RBACResourceWhitelistScope, ReplaceMemberBindings
@@ -206,21 +205,6 @@ class DatasetQueryDetailResponse(ResponseModel):
return to_timestamp(value)
@dataclass(frozen=True)
class _DatasetQueryResponseSource:
"""Expose query content through the request's database session."""
query: DatasetQuery
session: Session
@property
def queries(self) -> list[dict[str, Any]]:
return self.query.get_queries(session=self.session)
def __getattr__(self, name: str) -> Any:
return getattr(self.query, name) # noqa: no-new-getattr response adapter delegates model fields
class DatasetQueryListResponse(ResponseModel):
data: list[DatasetQueryDetailResponse]
has_more: bool
@@ -245,21 +229,6 @@ class RelatedAppResponse(ResponseModel):
return self
@dataclass(frozen=True)
class _RelatedAppResponseSource:
"""Expose the compatible app mode through the request's database session."""
app: App
session: Session
@property
def mode_compatible_with_agent(self) -> str:
return self.app.mode_compatible_with_agent_with_session(session=self.session)
def __getattr__(self, name: str) -> Any:
return getattr(self.app, name) # noqa: no-new-getattr response adapter delegates model fields
class RelatedAppListResponse(ResponseModel):
data: list[RelatedAppResponse]
total: int
@@ -428,8 +397,7 @@ class DatasetListApi(Resource):
@enterprise_license_required
@with_current_user
@with_current_tenant_id
@with_session(write=False)
def get(self, session: Session, current_tenant_id: str, current_user: Account):
def get(self, current_tenant_id: str, current_user: Account):
# Convert query parameters to dict, handling list parameters correctly
query_params: dict[str, str | list[str]] = dict(request.args.to_dict())
# Handle ids and tag_ids as lists (Flask request.args.getlist returns list even for single value)
@@ -442,7 +410,7 @@ class DatasetListApi(Resource):
permissions = enterprise_rbac_service.RBACService.MyPermissions.get(
str(current_tenant_id),
current_user.id,
session=session,
session=db.session(),
)
accessible_dataset_ids: list[str] | None = None
@@ -481,13 +449,12 @@ class DatasetListApi(Resource):
user=current_user,
accessible_dataset_ids=accessible_dataset_ids,
include_own_datasets=include_own_datasets,
session=session,
)
else:
datasets, total = DatasetService.get_datasets(
query.page,
query.limit,
session,
db.session(),
current_tenant_id,
current_user,
query.keyword,
@@ -512,14 +479,11 @@ class DatasetListApi(Resource):
for embedding_model in embedding_models:
model_names.append(f"{embedding_model.model}:{embedding_model.provider.provider}")
data = [
dump_response(DatasetDetailResponse, dataset_detail_response_source(dataset, session=session))
for dataset in datasets
]
data = [dump_response(DatasetDetailResponse, dataset) for dataset in datasets]
dataset_ids = [item["id"] for item in data if item.get("permission") == "partial_members"]
partial_members_map: dict[str, list[str]] = {}
if dataset_ids:
partial_member_rows = session.execute(
partial_member_rows = db.session.execute(
select(DatasetPermission.dataset_id, DatasetPermission.account_id).where(
DatasetPermission.dataset_id.in_(dataset_ids)
)
@@ -607,13 +571,6 @@ class DatasetListApi(Resource):
ReplaceMemberBindings(scope=RBACResourceWhitelistScope.ALL),
)
initialize_created_app_rbac_access_task.delay(current_tenant_id, current_user.id, dataset_id=dataset.id)
else:
enterprise_rbac_service.RBACService.DatasetAccess.replace_whitelist(
current_tenant_id,
current_user.id,
dataset.id,
ReplaceMemberBindings(scope=RBACResourceWhitelistScope.SPECIFIC),
)
permission_keys_map = enterprise_rbac_service.RBACService.DatasetPermissions.batch_get(
current_tenant_id,
@@ -622,9 +579,9 @@ class DatasetListApi(Resource):
session=session,
)
item = DatasetDetailWithPartialMembersResponse.model_validate(
dataset_detail_response_source(dataset, session=session), from_attributes=True
).model_dump(mode="json")
item = DatasetDetailWithPartialMembersResponse.model_validate(dataset, from_attributes=True).model_dump(
mode="json"
)
item["permission_keys"] = permission_keys_map.get(dataset.id, [])
return item, 201
@@ -647,31 +604,30 @@ class DatasetApi(Resource):
@rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_READONLY)
@with_current_user
@with_current_tenant_id
@with_session(write=False)
def get(self, session: Session, current_tenant_id: str, current_user: Account, dataset_id: UUID):
def get(self, current_tenant_id: str, current_user: Account, dataset_id: UUID):
dataset_id_str = str(dataset_id)
dataset = DatasetService.get_dataset(dataset_id_str, session)
dataset = DatasetService.get_dataset(dataset_id_str, db.session())
if dataset is None:
raise NotFound("Dataset not found.")
try:
DatasetService.check_dataset_permission(dataset, current_user, session)
DatasetService.check_dataset_permission(dataset, current_user, db.session())
except services.errors.account.NoPermissionError as e:
raise Forbidden(str(e))
permissions = enterprise_rbac_service.RBACService.MyPermissions.get(
current_tenant_id,
current_user.id,
dataset_id=dataset_id_str,
session=session,
session=db.session(),
)
permission_keys_map = permissions.dataset.permission_keys_by_resource_ids([dataset_id_str])
data = dump_response(DatasetDetailResponse, dataset_detail_response_source(dataset, session=session))
data = dump_response(DatasetDetailResponse, dataset)
data["permission_keys"] = permission_keys_map.get(dataset_id_str, [])
if dataset.indexing_technique == IndexTechniqueType.HIGH_QUALITY:
if dataset.embedding_model_provider:
provider_id = ModelProviderID(dataset.embedding_model_provider)
data["embedding_model_provider"] = str(provider_id)
if data.get("permission") == "partial_members":
part_users_list = DatasetPermissionService.get_dataset_partial_member_list(dataset_id_str, session)
part_users_list = DatasetPermissionService.get_dataset_partial_member_list(dataset_id_str, db.session())
data.update({"partial_member_list": part_users_list})
# check embedding setting
@@ -715,7 +671,7 @@ class DatasetApi(Resource):
@with_session
def patch(self, session: Session, current_tenant_id: str, current_user: Account, dataset_id: UUID):
dataset_id_str = str(dataset_id)
dataset = DatasetService.get_dataset(dataset_id_str, session)
dataset = DatasetService.get_dataset(dataset_id_str, db.session())
if dataset is None:
raise NotFound("Dataset not found.")
@@ -748,19 +704,19 @@ class DatasetApi(Resource):
[dataset_id_str],
session=session,
)
result_data = dump_response(DatasetDetailResponse, dataset_detail_response_source(dataset, session=session))
result_data = dump_response(DatasetDetailResponse, dataset)
result_data["permission_keys"] = permission_keys_map.get(dataset_id_str, [])
tenant_id = current_tenant_id
if payload.partial_member_list is not None and payload.permission == DatasetPermissionEnum.PARTIAL_TEAM:
DatasetPermissionService.update_partial_member_list(
tenant_id, dataset_id_str, payload.partial_member_list, session
tenant_id, dataset_id_str, payload.partial_member_list, db.session()
)
# clear partial member list when permission is only_me or all_team_members
elif payload.permission in {DatasetPermissionEnum.ONLY_ME, DatasetPermissionEnum.ALL_TEAM}:
DatasetPermissionService.clear_partial_member_list(dataset_id_str, session)
DatasetPermissionService.clear_partial_member_list(dataset_id_str, db.session())
partial_member_list = DatasetPermissionService.get_dataset_partial_member_list(dataset_id_str, session)
partial_member_list = DatasetPermissionService.get_dataset_partial_member_list(dataset_id_str, db.session())
result_data.update({"partial_member_list": partial_member_list})
return dump_response(DatasetDetailWithPartialMembersResponse, result_data), 200
@@ -772,16 +728,15 @@ class DatasetApi(Resource):
@console_ns.response(204, "Dataset deleted successfully")
@with_current_user
@rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_EDIT)
@with_session
def delete(self, session: Session, current_user: Account, dataset_id: UUID):
def delete(self, current_user: Account, dataset_id: UUID):
dataset_id_str = str(dataset_id)
if not (current_user.has_edit_permission or current_user.is_dataset_operator):
raise Forbidden()
try:
if DatasetService.delete_dataset(dataset_id_str, current_user, session):
DatasetPermissionService.clear_partial_member_list(dataset_id_str, session)
if DatasetService.delete_dataset(dataset_id_str, current_user, db.session()):
DatasetPermissionService.clear_partial_member_list(dataset_id_str, db.session())
return "", 204
else:
raise NotFound("Dataset not found.")
@@ -803,11 +758,10 @@ class DatasetUseCheckApi(Resource):
@login_required
@account_initialization_required
@rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_READONLY)
@with_session(write=False)
def get(self, session: Session, dataset_id: UUID):
def get(self, dataset_id: UUID):
dataset_id_str = str(dataset_id)
dataset_is_using = DatasetService.dataset_use_check(dataset_id_str, session)
dataset_is_using = DatasetService.dataset_use_check(dataset_id_str, db.session())
return UsageCheckResponse(is_using=dataset_is_using).model_dump(mode="json"), 200
@@ -826,27 +780,24 @@ class DatasetQueryApi(Resource):
@account_initialization_required
@with_current_user
@rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_READONLY)
@with_session(write=False)
def get(self, session: Session, current_user: Account, dataset_id: UUID):
def get(self, current_user: Account, dataset_id: UUID):
dataset_id_str = str(dataset_id)
dataset = DatasetService.get_dataset(dataset_id_str, session)
dataset = DatasetService.get_dataset(dataset_id_str, db.session())
if dataset is None:
raise NotFound("Dataset not found.")
try:
DatasetService.check_dataset_permission(dataset, current_user, session)
DatasetService.check_dataset_permission(dataset, current_user, db.session())
except services.errors.account.NoPermissionError as e:
raise Forbidden(str(e))
page = request.args.get("page", default=1, type=int)
limit = request.args.get("limit", default=20, type=int)
dataset_queries, total = DatasetService.get_dataset_queries(
dataset_id=dataset.id, page=page, per_page=limit, session=session
)
dataset_queries, total = DatasetService.get_dataset_queries(dataset_id=dataset.id, page=page, per_page=limit)
response = {
"data": [_DatasetQueryResponseSource(query=query, session=session) for query in dataset_queries],
"data": dataset_queries,
"has_more": len(dataset_queries) == limit,
"limit": limit,
"total": total,
@@ -869,8 +820,7 @@ class DatasetIndexingEstimateApi(Resource):
@account_initialization_required
@console_ns.expect(console_ns.models[IndexingEstimatePayload.__name__])
@with_current_tenant_id
@with_session
def post(self, session: Session, current_tenant_id: str):
def post(self, current_tenant_id: str):
payload = IndexingEstimatePayload.model_validate(console_ns.payload or {})
args = payload.model_dump()
# validate args
@@ -879,10 +829,10 @@ class DatasetIndexingEstimateApi(Resource):
match args["info_list"]["data_source_type"]:
case "upload_file":
file_ids = args["info_list"]["file_info_list"]["file_ids"]
file_details = session.scalars(
file_details = db.session.scalars(
select(UploadFile).where(UploadFile.tenant_id == current_tenant_id, UploadFile.id.in_(file_ids))
).all()
if not file_details:
if file_details is None:
raise NotFound("File not found.")
if file_details:
@@ -936,14 +886,13 @@ class DatasetIndexingEstimateApi(Resource):
indexing_runner = IndexingRunner()
try:
response = indexing_runner.indexing_estimate(
tenant_id=current_tenant_id,
extract_settings=extract_settings,
tmp_processing_rule=args["process_rule"],
doc_form=args["doc_form"],
doc_language=args["doc_language"],
dataset_id=args["dataset_id"],
indexing_technique=args["indexing_technique"],
session=session,
current_tenant_id,
extract_settings,
args["process_rule"],
args["doc_form"],
args["doc_language"],
args["dataset_id"],
args["indexing_technique"],
)
except LLMBadRequestError:
raise ProviderNotInitializeError(
@@ -982,25 +931,24 @@ class DatasetRelatedAppListApi(Resource):
@account_initialization_required
@with_current_user
@rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_READONLY)
@with_session(write=False)
def get(self, session: Session, current_user: Account, dataset_id: UUID):
def get(self, current_user: Account, dataset_id: UUID):
dataset_id_str = str(dataset_id)
dataset = DatasetService.get_dataset(dataset_id_str, session)
dataset = DatasetService.get_dataset(dataset_id_str, db.session())
if dataset is None:
raise NotFound("Dataset not found.")
try:
DatasetService.check_dataset_permission(dataset, current_user, session)
DatasetService.check_dataset_permission(dataset, current_user, db.session())
except services.errors.account.NoPermissionError as e:
raise Forbidden(str(e))
app_dataset_joins = DatasetService.get_related_apps(dataset.id, session)
app_dataset_joins = DatasetService.get_related_apps(dataset.id, db.session())
related_apps = []
for app_dataset_join in app_dataset_joins:
app_model = AppService.get_app_by_id(app_dataset_join.app_id, session)
app_model = app_dataset_join.app
if app_model:
related_apps.append(_RelatedAppResponseSource(app=app_model, session=session))
related_apps.append(app_model)
return dump_response(RelatedAppListResponse, {"data": related_apps, "total": len(related_apps)}), 200
@@ -1020,16 +968,15 @@ class DatasetIndexingStatusApi(Resource):
@account_initialization_required
@with_current_tenant_id
@rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_READONLY)
@with_session(write=False)
def get(self, session: Session, current_tenant_id: str, dataset_id: UUID):
def get(self, current_tenant_id: str, dataset_id: UUID):
dataset_id_str = str(dataset_id)
documents = session.scalars(
documents = db.session.scalars(
select(Document).where(Document.dataset_id == dataset_id_str, Document.tenant_id == current_tenant_id)
).all()
documents_status = []
for document in documents:
completed_segments = (
session.scalar(
db.session.scalar(
select(func.count(DocumentSegment.id)).where(
DocumentSegment.completed_at.isnot(None),
DocumentSegment.document_id == str(document.id),
@@ -1039,7 +986,7 @@ class DatasetIndexingStatusApi(Resource):
or 0
)
total_segments = (
session.scalar(
db.session.scalar(
select(func.count(DocumentSegment.id)).where(
DocumentSegment.document_id == str(document.id),
DocumentSegment.status != SegmentStatus.RE_SEGMENT,
@@ -1079,9 +1026,8 @@ class DatasetApiKeyApi(Resource):
@login_required
@account_initialization_required
@with_current_tenant_id
@with_session(write=False)
def get(self, session: Session, current_tenant_id: str):
keys = session.scalars(
def get(self, current_tenant_id: str):
keys = db.session.scalars(
select(ApiToken).where(ApiToken.type == self.resource_type, ApiToken.tenant_id == current_tenant_id)
).all()
return dump_response(ApiKeyList, {"data": keys})
@@ -1094,10 +1040,9 @@ class DatasetApiKeyApi(Resource):
@rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_API_KEY_MANAGE, resource_required=False)
@account_initialization_required
@with_current_tenant_id
@with_session
def post(self, session: Session, current_tenant_id: str):
def post(self, current_tenant_id: str):
current_key_count = (
session.scalar(
db.session.scalar(
select(func.count(ApiToken.id)).where(
ApiToken.type == self.resource_type, ApiToken.tenant_id == current_tenant_id
)
@@ -1112,13 +1057,13 @@ class DatasetApiKeyApi(Resource):
custom="max_keys_exceeded",
)
key = ApiToken.generate_api_key(self.token_prefix, 24, session=session)
key = ApiToken.generate_api_key(self.token_prefix, 24)
api_token = ApiToken()
api_token.tenant_id = current_tenant_id
api_token.token = key
api_token.type = self.resource_type
session.add(api_token)
session.flush()
db.session.add(api_token)
db.session.commit()
return dump_response(ApiKeyItem, api_token), 200
@@ -1136,10 +1081,9 @@ class DatasetApiDeleteApi(Resource):
@rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_API_KEY_MANAGE, resource_required=False)
@account_initialization_required
@with_current_tenant_id
@with_session
def delete(self, session: Session, current_tenant_id: str, api_key_id: UUID):
def delete(self, current_tenant_id: str, api_key_id: UUID):
api_key_id_str = str(api_key_id)
key = session.scalar(
key = db.session.scalar(
select(ApiToken)
.where(
ApiToken.tenant_id == current_tenant_id,
@@ -1157,7 +1101,8 @@ class DatasetApiDeleteApi(Resource):
assert key is not None # nosec - for type checker only
ApiTokenCache.delete(key.token, key.type)
session.delete(key)
db.session.delete(key)
db.session.commit()
return "", 204
@@ -1169,11 +1114,10 @@ class DatasetEnableApiApi(Resource):
@account_initialization_required
@console_ns.response(200, "Success", console_ns.models[SimpleResultResponse.__name__])
@rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_EDIT)
@with_session
def post(self, session: Session, dataset_id: UUID, status: str):
def post(self, dataset_id: UUID, status: str):
dataset_id_str = str(dataset_id)
DatasetService.update_dataset_api_status(dataset_id_str, status == "enable", session)
DatasetService.update_dataset_api_status(dataset_id_str, status == "enable", db.session())
return SimpleResultResponse(result="success").model_dump(mode="json"), 200
@@ -1240,13 +1184,12 @@ class DatasetErrorDocs(Resource):
@login_required
@account_initialization_required
@rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_READONLY)
@with_session(write=False)
def get(self, session: Session, dataset_id: UUID):
def get(self, dataset_id: UUID):
dataset_id_str = str(dataset_id)
dataset = DatasetService.get_dataset(dataset_id_str, session)
dataset = DatasetService.get_dataset(dataset_id_str, db.session())
if dataset is None:
raise NotFound("Dataset not found.")
results = DocumentService.get_error_documents_by_dataset_id(dataset_id_str, session)
results = DocumentService.get_error_documents_by_dataset_id(dataset_id_str, db.session())
return dump_response(ErrorDocsResponse, {"data": results, "total": len(results)}), 200
@@ -1268,18 +1211,17 @@ class DatasetPermissionUserListApi(Resource):
@account_initialization_required
@with_current_user
@rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_READONLY)
@with_session(write=False)
def get(self, session: Session, current_user: Account, dataset_id: UUID):
def get(self, current_user: Account, dataset_id: UUID):
dataset_id_str = str(dataset_id)
dataset = DatasetService.get_dataset(dataset_id_str, session)
dataset = DatasetService.get_dataset(dataset_id_str, db.session())
if dataset is None:
raise NotFound("Dataset not found.")
try:
DatasetService.check_dataset_permission(dataset, current_user, session)
DatasetService.check_dataset_permission(dataset, current_user, db.session())
except services.errors.account.NoPermissionError as e:
raise Forbidden(str(e))
partial_members_list = DatasetPermissionService.get_dataset_partial_member_list(dataset_id_str, session)
partial_members_list = DatasetPermissionService.get_dataset_partial_member_list(dataset_id_str, db.session())
return dump_response(PartialMemberListResponse, {"data": partial_members_list}), 200
@@ -1299,11 +1241,10 @@ class DatasetAutoDisableLogApi(Resource):
@login_required
@account_initialization_required
@rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_READONLY)
@with_session(write=False)
def get(self, session: Session, dataset_id: UUID):
def get(self, dataset_id: UUID):
dataset_id_str = str(dataset_id)
dataset = DatasetService.get_dataset(dataset_id_str, session)
dataset = DatasetService.get_dataset(dataset_id_str, db.session())
if dataset is None:
raise NotFound("Dataset not found.")
auto_disable_logs = DatasetService.get_dataset_auto_disable_logs(dataset_id_str, session)
auto_disable_logs = DatasetService.get_dataset_auto_disable_logs(dataset_id_str, db.session())
return dump_response(AutoDisableLogsResponse, auto_disable_logs), 200
@@ -1,7 +1,7 @@
import json
import logging
from argparse import ArgumentTypeError
from collections.abc import Mapping, Sequence
from collections.abc import Sequence
from contextlib import ExitStack
from datetime import datetime
from typing import Any, Literal, cast
@@ -12,14 +12,12 @@ from flask import request, send_file
from flask_restx import Resource
from pydantic import BaseModel, Field, JsonValue, field_validator
from sqlalchemy import asc, desc, func, select
from sqlalchemy.orm import Session
from werkzeug.exceptions import Forbidden, NotFound
import services
from controllers.common.controller_schemas import DocumentBatchDownloadZipPayload
from controllers.common.fields import SimpleResultMessageResponse, SimpleResultResponse, UrlResponse
from controllers.common.schema import register_response_schema_models, register_schema_models
from controllers.common.session import with_session
from controllers.console import console_ns
from controllers.console.wraps import RBACPermission, RBACResourceScope, rbac_permission_required
from core.entities.knowledge_entities import IndexingEstimate
@@ -36,15 +34,13 @@ from core.rag.entities import Rule
from core.rag.extractor.entity.datasource_type import DatasourceType
from core.rag.extractor.entity.extract_setting import ExtractSetting, NotionInfo, WebsiteInfo
from core.rag.index_processor.constant.index_type import IndexTechniqueType
from extensions.ext_database import db
from fields.base import ResponseModel
from fields.document_fields import (
DocumentMetadataResponse,
DocumentResponse,
DocumentStatusListResponse,
DocumentStatusResponse,
DocumentWithSession,
document_response,
document_responses,
normalize_enum,
)
from graphon.model_runtime.entities.model_entities import ModelType
@@ -53,7 +49,7 @@ from libs.datetime_utils import naive_utc_now
from libs.helper import dump_response, to_timestamp
from libs.login import login_required
from libs.pagination import paginate_query
from models import Account, Document, DocumentSegment, UploadFile
from models import Account, DatasetProcessRule, Document, DocumentSegment, UploadFile
from models.dataset import DocumentPipelineExecutionLog
from models.enums import IndexingStatus, ProcessRuleMode, SegmentStatus
from services.dataset_ref_service import DatasetRefService
@@ -114,24 +110,6 @@ class DocumentWithSegmentsResponse(DocumentResponse):
total_segments: int | None = Field(default=None, exclude_if=lambda value: value is None)
class DocumentWithSegmentsSession(DocumentWithSession):
@property
def process_rule_dict(self) -> Any:
process_rule = self.document.get_dataset_process_rule(session=self.session)
return process_rule.to_dict() if process_rule else None
def document_with_segments_responses(
documents: Sequence[Document], *, session: Session
) -> list[DocumentWithSegmentsResponse]:
return [
DocumentWithSegmentsResponse.model_validate(
DocumentWithSegmentsSession(document=document, session=session), from_attributes=True
)
for document in documents
]
class DatasetAndDocumentResponse(ResponseModel):
dataset: DatasetResponse
documents: list[DocumentResponse]
@@ -291,18 +269,18 @@ register_response_schema_models(
class DocumentResource(Resource):
def get_document(
self, session: Session, dataset_id: str, document_id: str, current_user: Account, current_tenant_id: str
self, dataset_id: str, document_id: str, current_user: Account, current_tenant_id: str
) -> Document:
dataset = DatasetService.get_dataset(dataset_id, session)
dataset = DatasetService.get_dataset(dataset_id, db.session())
if not dataset:
raise NotFound("Dataset not found.")
try:
DatasetService.check_dataset_permission(dataset, current_user, session)
DatasetService.check_dataset_permission(dataset, current_user, db.session())
except services.errors.account.NoPermissionError as e:
raise Forbidden(str(e))
document = DocumentService.get_document(dataset_id, document_id, session=session)
document = DocumentService.get_document(dataset_id, document_id, session=db.session())
if not document:
raise NotFound("Document not found.")
@@ -312,19 +290,17 @@ class DocumentResource(Resource):
return document
def get_batch_documents(
self, session: Session, dataset_id: str, batch: str, current_user: Account
) -> Sequence[Document]:
dataset = DatasetService.get_dataset(dataset_id, session)
def get_batch_documents(self, dataset_id: str, batch: str, current_user: Account) -> Sequence[Document]:
dataset = DatasetService.get_dataset(dataset_id, db.session())
if not dataset:
raise NotFound("Dataset not found.")
try:
DatasetService.check_dataset_permission(dataset, current_user, session)
DatasetService.check_dataset_permission(dataset, current_user, db.session())
except services.errors.account.NoPermissionError as e:
raise Forbidden(str(e))
documents = DocumentService.get_batch_documents(dataset_id, batch, session)
documents = DocumentService.get_batch_documents(dataset_id, batch, db.session())
if not documents:
raise NotFound("Documents not found.")
@@ -342,8 +318,7 @@ class GetProcessRuleApi(Resource):
@login_required
@account_initialization_required
@with_current_user
@with_session(write=False)
def get(self, session: Session, current_user: Account):
def get(self, current_user: Account):
req_data = request.args
document_id = req_data.get("document_id")
@@ -353,21 +328,26 @@ class GetProcessRuleApi(Resource):
rules = DocumentService.DEFAULT_RULES["rules"]
limits = DocumentService.DEFAULT_RULES["limits"]
if document_id:
document = DocumentService.get_document_by_id(document_id, session)
if document is None:
raise NotFound("Document not found.")
# get the latest process rule
document = db.get_or_404(Document, document_id)
dataset = DatasetService.get_dataset(document.dataset_id, session)
dataset = DatasetService.get_dataset(document.dataset_id, db.session())
if not dataset:
raise NotFound("Dataset not found.")
try:
DatasetService.check_dataset_permission(dataset, current_user, session)
DatasetService.check_dataset_permission(dataset, current_user, db.session())
except services.errors.account.NoPermissionError as e:
raise Forbidden(str(e))
dataset_process_rule = dataset.get_latest_process_rule(session=session)
# get the latest process rule
dataset_process_rule = db.session.scalar(
select(DatasetProcessRule)
.where(DatasetProcessRule.dataset_id == document.dataset_id)
.order_by(DatasetProcessRule.created_at.desc())
.limit(1)
)
if dataset_process_rule:
mode = dataset_process_rule.mode
rules = dataset_process_rule.rules_dict
@@ -401,8 +381,7 @@ class DatasetDocumentListApi(Resource):
@with_current_user
@with_current_tenant_id
@rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_CREATE_AND_MANAGEMENT)
@with_session(write=False)
def get(self, session: Session, current_tenant_id: str, current_user: Account, dataset_id: UUID):
def get(self, current_tenant_id: str, current_user: Account, dataset_id: UUID):
dataset_id_str = str(dataset_id)
raw_args = request.args.to_dict()
param = DocumentDatasetListParam.model_validate(raw_args)
@@ -428,12 +407,12 @@ class DatasetDocumentListApi(Resource):
)
except (ArgumentTypeError, ValueError, Exception):
fetch = False
dataset = DatasetService.get_dataset(dataset_id_str, session)
dataset = DatasetService.get_dataset(dataset_id_str, db.session())
if not dataset:
raise NotFound("Dataset not found.")
try:
DatasetService.check_dataset_permission(dataset, current_user, session)
DatasetService.check_dataset_permission(dataset, current_user, db.session())
except services.errors.account.NoPermissionError as e:
raise Forbidden(str(e))
@@ -478,20 +457,20 @@ class DatasetDocumentListApi(Resource):
desc(Document.position),
)
paginated_documents = paginate_query(query, session=session, page=page, per_page=limit, max_per_page=100)
paginated_documents = paginate_query(query, page=page, per_page=limit, max_per_page=100)
documents = paginated_documents.items
DocumentService.enrich_documents_with_summary_index_status(
documents=documents,
dataset=dataset,
tenant_id=current_tenant_id,
session=session,
session=db.session(),
)
if fetch:
for document in documents:
completed_segments = (
session.scalar(
db.session.scalar(
select(func.count(DocumentSegment.id)).where(
DocumentSegment.completed_at.isnot(None),
DocumentSegment.document_id == str(document.id),
@@ -501,7 +480,7 @@ class DatasetDocumentListApi(Resource):
or 0
)
total_segments = (
session.scalar(
db.session.scalar(
select(func.count(DocumentSegment.id)).where(
DocumentSegment.document_id == str(document.id),
DocumentSegment.status != SegmentStatus.RE_SEGMENT,
@@ -512,7 +491,7 @@ class DatasetDocumentListApi(Resource):
document.completed_segments = completed_segments
document.total_segments = total_segments
response = {
"data": document_with_segments_responses(documents, session=session),
"data": documents,
"has_more": len(documents) == limit,
"limit": limit,
"total": paginated_documents.total,
@@ -530,11 +509,10 @@ class DatasetDocumentListApi(Resource):
@console_ns.response(200, "Documents created successfully", console_ns.models[DatasetAndDocumentResponse.__name__])
@with_current_user
@rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_EDIT)
@with_session
def post(self, session: Session, current_user: Account, dataset_id: UUID):
def post(self, current_user: Account, dataset_id: UUID):
dataset_id_str = str(dataset_id)
dataset = DatasetService.get_dataset(dataset_id_str, session)
dataset = DatasetService.get_dataset(dataset_id_str, db.session())
if not dataset:
raise NotFound("Dataset not found.")
@@ -544,7 +522,7 @@ class DatasetDocumentListApi(Resource):
raise Forbidden()
try:
DatasetService.check_dataset_permission(dataset, current_user, session)
DatasetService.check_dataset_permission(dataset, current_user, db.session())
except services.errors.account.NoPermissionError as e:
raise Forbidden(str(e))
@@ -558,9 +536,9 @@ class DatasetDocumentListApi(Resource):
try:
documents, batch = DocumentService.save_document_with_dataset_id(
dataset, knowledge_config, current_user, session=session
dataset, knowledge_config, current_user, session=db.session()
)
dataset = DatasetService.get_dataset(dataset_id_str, session)
dataset = DatasetService.get_dataset(dataset_id_str, db.session())
except ProviderTokenNotInitError as ex:
raise ProviderNotInitializeError(ex.description)
@@ -569,10 +547,7 @@ class DatasetDocumentListApi(Resource):
except ModelCurrentlyNotSupportError:
raise ProviderModelCurrentlyNotSupportError()
return dump_response(
DatasetAndDocumentResponse,
{"dataset": dataset, "documents": document_responses(documents, session=session), "batch": batch},
)
return dump_response(DatasetAndDocumentResponse, {"dataset": dataset, "documents": documents, "batch": batch})
@setup_required
@login_required
@@ -580,10 +555,9 @@ class DatasetDocumentListApi(Resource):
@cloud_edition_billing_rate_limit_check("knowledge")
@console_ns.response(204, "Documents deleted successfully")
@rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_EDIT)
@with_session
def delete(self, session: Session, dataset_id: UUID):
def delete(self, dataset_id: UUID):
dataset_id_str = str(dataset_id)
dataset = DatasetService.get_dataset(dataset_id_str, session)
dataset = DatasetService.get_dataset(dataset_id_str, db.session())
if dataset is None:
raise NotFound("Dataset not found.")
# check user's model setting
@@ -592,7 +566,7 @@ class DatasetDocumentListApi(Resource):
try:
document_ids = request.args.getlist("document_id")
dataset_ref = DatasetRefService.create_dataset_ref(dataset)
DocumentService.delete_documents(dataset_ref, document_ids, dataset.get_doc_form(session=session), session)
DocumentService.delete_documents(dataset_ref, document_ids, dataset.doc_form, db.session())
except services.errors.document.DocumentIndexingError:
raise DocumentIndexingError("Cannot delete document during indexing.")
@@ -615,8 +589,7 @@ class DatasetInitApi(Resource):
@cloud_edition_billing_rate_limit_check("knowledge")
@with_current_user
@with_current_tenant_id
@with_session
def post(self, session: Session, current_tenant_id: str, current_user: Account):
def post(self, current_tenant_id: str, current_user: Account):
# The role of the current user in the ta table must be admin, owner, dataset_operator, or editor
if not current_user.is_dataset_editor:
raise Forbidden()
@@ -652,7 +625,7 @@ class DatasetInitApi(Resource):
tenant_id=current_tenant_id,
knowledge_config=knowledge_config,
account=current_user,
session=session,
session=db.session(),
)
except ProviderTokenNotInitError as ex:
raise ProviderNotInitializeError(ex.description)
@@ -661,10 +634,7 @@ class DatasetInitApi(Resource):
except ModelCurrentlyNotSupportError:
raise ProviderModelCurrentlyNotSupportError()
return dump_response(
DatasetAndDocumentResponse,
{"dataset": dataset, "documents": document_responses(documents, session=session), "batch": batch},
)
return dump_response(DatasetAndDocumentResponse, {"dataset": dataset, "documents": documents, "batch": batch})
@console_ns.route("/datasets/<uuid:dataset_id>/documents/<uuid:document_id>/indexing-estimate")
@@ -685,24 +655,23 @@ class DocumentIndexingEstimateApi(DocumentResource):
@with_current_user
@with_current_tenant_id
@rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_CREATE_AND_MANAGEMENT)
@with_session
def get(self, session: Session, current_tenant_id: str, current_user: Account, dataset_id: UUID, document_id: UUID):
def get(self, current_tenant_id: str, current_user: Account, dataset_id: UUID, document_id: UUID):
dataset_id_str = str(dataset_id)
document_id_str = str(document_id)
document = self.get_document(session, dataset_id_str, document_id_str, current_user, current_tenant_id)
document = self.get_document(dataset_id_str, document_id_str, current_user, current_tenant_id)
if document.indexing_status in {IndexingStatus.COMPLETED, IndexingStatus.ERROR}:
raise DocumentAlreadyFinishedError()
data_process_rule = document.get_dataset_process_rule(session=session)
data_process_rule_dict: Mapping[str, Any] = data_process_rule.to_dict() if data_process_rule else {}
data_process_rule = document.dataset_process_rule
data_process_rule_dict = data_process_rule.to_dict() if data_process_rule else {}
if document.data_source_type == "upload_file":
data_source_info = document.data_source_info_dict
if data_source_info and "upload_file_id" in data_source_info:
file_id = data_source_info["upload_file_id"]
file = session.scalar(
file = db.session.scalar(
select(UploadFile)
.where(UploadFile.tenant_id == document.tenant_id, UploadFile.id == file_id)
.limit(1)
@@ -720,13 +689,12 @@ class DocumentIndexingEstimateApi(DocumentResource):
try:
estimate_response = indexing_runner.indexing_estimate(
tenant_id=current_tenant_id,
extract_settings=[extract_setting],
tmp_processing_rule=data_process_rule_dict,
doc_form=document.doc_form,
doc_language="English",
dataset_id=dataset_id_str,
session=session,
current_tenant_id,
[extract_setting],
data_process_rule_dict,
document.doc_form,
"English",
dataset_id_str,
)
return (
# TODO: why using zero here? the same for the below endpoint
@@ -777,10 +745,9 @@ class DocumentBatchIndexingEstimateApi(DocumentResource):
@with_current_user
@with_current_tenant_id
@rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_CREATE_AND_MANAGEMENT)
@with_session
def get(self, session: Session, current_tenant_id: str, current_user: Account, dataset_id: UUID, batch: str):
def get(self, current_tenant_id: str, current_user: Account, dataset_id: UUID, batch: str):
dataset_id_str = str(dataset_id)
documents = self.get_batch_documents(session, dataset_id_str, batch, current_user)
documents = self.get_batch_documents(dataset_id_str, batch, current_user)
if not documents:
return (
IndexingEstimateResponse(
@@ -792,8 +759,8 @@ class DocumentBatchIndexingEstimateApi(DocumentResource):
).model_dump(mode="json", exclude_none=True),
200,
)
data_process_rule = documents[0].get_dataset_process_rule(session=session)
data_process_rule_dict: Mapping[str, Any] = data_process_rule.to_dict() if data_process_rule else {}
data_process_rule = documents[0].dataset_process_rule
data_process_rule_dict = data_process_rule.to_dict() if data_process_rule else {}
extract_settings = []
for document in documents:
if document.indexing_status in {IndexingStatus.COMPLETED, IndexingStatus.ERROR}:
@@ -804,7 +771,7 @@ class DocumentBatchIndexingEstimateApi(DocumentResource):
if not data_source_info:
continue
file_id = data_source_info["upload_file_id"]
file_detail = session.scalar(
file_detail = db.session.scalar(
select(UploadFile)
.where(UploadFile.tenant_id == current_tenant_id, UploadFile.id == file_id)
.limit(1)
@@ -858,13 +825,12 @@ class DocumentBatchIndexingEstimateApi(DocumentResource):
indexing_runner = IndexingRunner()
try:
response = indexing_runner.indexing_estimate(
tenant_id=current_tenant_id,
extract_settings=extract_settings,
tmp_processing_rule=data_process_rule_dict,
doc_form=document.doc_form,
doc_language="English",
dataset_id=dataset_id_str,
session=session,
current_tenant_id,
extract_settings,
data_process_rule_dict,
document.doc_form,
"English",
dataset_id_str,
)
return (
IndexingEstimateResponse(
@@ -899,14 +865,13 @@ class DocumentBatchIndexingStatusApi(DocumentResource):
@account_initialization_required
@with_current_user
@rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_CREATE_AND_MANAGEMENT)
@with_session(write=False)
def get(self, session: Session, current_user: Account, dataset_id: UUID, batch: str):
def get(self, current_user: Account, dataset_id: UUID, batch: str):
dataset_id_str = str(dataset_id)
documents = self.get_batch_documents(session, dataset_id_str, batch, current_user)
documents = self.get_batch_documents(dataset_id_str, batch, current_user)
documents_status = []
for document in documents:
completed_segments = (
session.scalar(
db.session.scalar(
select(func.count(DocumentSegment.id)).where(
DocumentSegment.completed_at.isnot(None),
DocumentSegment.document_id == str(document.id),
@@ -916,7 +881,7 @@ class DocumentBatchIndexingStatusApi(DocumentResource):
or 0
)
total_segments = (
session.scalar(
db.session.scalar(
select(func.count(DocumentSegment.id)).where(
DocumentSegment.document_id == str(document.id),
DocumentSegment.status != SegmentStatus.RE_SEGMENT,
@@ -958,14 +923,13 @@ class DocumentIndexingStatusApi(DocumentResource):
@with_current_user
@with_current_tenant_id
@rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_CREATE_AND_MANAGEMENT)
@with_session(write=False)
def get(self, session: Session, current_tenant_id: str, current_user: Account, dataset_id: UUID, document_id: UUID):
def get(self, current_tenant_id: str, current_user: Account, dataset_id: UUID, document_id: UUID):
dataset_id_str = str(dataset_id)
document_id_str = str(document_id)
document = self.get_document(session, dataset_id_str, document_id_str, current_user, current_tenant_id)
document = self.get_document(dataset_id_str, document_id_str, current_user, current_tenant_id)
completed_segments = (
session.scalar(
db.session.scalar(
select(func.count(DocumentSegment.id)).where(
DocumentSegment.completed_at.isnot(None),
DocumentSegment.document_id == document_id_str,
@@ -975,7 +939,7 @@ class DocumentIndexingStatusApi(DocumentResource):
or 0
)
total_segments = (
session.scalar(
db.session.scalar(
select(func.count(DocumentSegment.id)).where(
DocumentSegment.document_id == document_id_str,
DocumentSegment.status != SegmentStatus.RE_SEGMENT,
@@ -1023,11 +987,10 @@ class DocumentApi(DocumentResource):
@with_current_user
@with_current_tenant_id
@rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_CREATE_AND_MANAGEMENT)
@with_session(write=False)
def get(self, session: Session, current_tenant_id: str, current_user: Account, dataset_id: UUID, document_id: UUID):
def get(self, current_tenant_id: str, current_user: Account, dataset_id: UUID, document_id: UUID):
dataset_id_str = str(dataset_id)
document_id_str = str(document_id)
document = self.get_document(session, dataset_id_str, document_id_str, current_user, current_tenant_id)
document = self.get_document(dataset_id_str, document_id_str, current_user, current_tenant_id)
metadata = request.args.get("metadata", "all")
if metadata not in self.METADATA_CHOICES:
@@ -1039,22 +1002,20 @@ class DocumentApi(DocumentResource):
{
"id": document.id,
"doc_type": document.doc_type,
"doc_metadata": document.get_doc_metadata_details(session=session),
"doc_metadata": document.doc_metadata_details,
}
)
return response.model_dump(mode="json", include={"id", *metadata_fields}, exclude_unset=True), 200
dataset_process_rules = DatasetService.get_process_rules(dataset_id_str, session)
document_process_rule = document.get_dataset_process_rule(session=session)
document_process_rules: Mapping[str, Any] = document_process_rule.to_dict() if document_process_rule else {}
segment_count = document.get_segment_count(session=session)
dataset_process_rules = DatasetService.get_process_rules(dataset_id_str, db.session())
document_process_rules = document.dataset_process_rule.to_dict() if document.dataset_process_rule else {}
response = DocumentDetailResponse.model_validate(
{
"id": document.id,
"position": document.position,
"data_source_type": document.data_source_type,
"data_source_info": document.data_source_info_dict,
"data_source_detail_dict": document.get_data_source_detail_dict(session=session),
"data_source_detail_dict": document.data_source_detail_dict,
"dataset_process_rule_id": document.dataset_process_rule_id,
"dataset_process_rule": dataset_process_rules,
"document_process_rule": document_process_rules,
@@ -1073,10 +1034,10 @@ class DocumentApi(DocumentResource):
"disabled_by": document.disabled_by,
"archived": document.archived,
"doc_type": document.doc_type,
"doc_metadata": document.get_doc_metadata_details(session=session),
"segment_count": segment_count,
"average_segment_length": (document.word_count or 0) // segment_count if segment_count else 0,
"hit_count": document.get_hit_count(session=session),
"doc_metadata": document.doc_metadata_details,
"segment_count": document.segment_count,
"average_segment_length": document.average_segment_length,
"hit_count": document.hit_count,
"display_status": document.display_status,
"doc_form": document.doc_form,
"doc_language": document.doc_language,
@@ -1094,22 +1055,19 @@ class DocumentApi(DocumentResource):
@with_current_user
@with_current_tenant_id
@rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_EDIT)
@with_session
def delete(
self, session: Session, current_tenant_id: str, current_user: Account, dataset_id: UUID, document_id: UUID
):
def delete(self, current_tenant_id: str, current_user: Account, dataset_id: UUID, document_id: UUID):
dataset_id_str = str(dataset_id)
document_id_str = str(document_id)
dataset = DatasetService.get_dataset(dataset_id_str, session)
dataset = DatasetService.get_dataset(dataset_id_str, db.session())
if dataset is None:
raise NotFound("Dataset not found.")
# check user's model setting
DatasetService.check_dataset_model_setting(dataset)
document = self.get_document(session, dataset_id_str, document_id_str, current_user, current_tenant_id)
document = self.get_document(dataset_id_str, document_id_str, current_user, current_tenant_id)
try:
DocumentService.delete_document(document, session)
DocumentService.delete_document(document, db.session())
except services.errors.document.DocumentIndexingError:
raise DocumentIndexingError("Cannot delete document during indexing.")
@@ -1130,13 +1088,12 @@ class DocumentDownloadApi(DocumentResource):
@with_current_user
@with_current_tenant_id
@rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_DOCUMENT_DOWNLOAD)
@with_session(write=False)
def get(
self, session: Session, current_tenant_id: str, current_user: Account, dataset_id: UUID, document_id: UUID
) -> dict[str, Any]:
def get(self, current_tenant_id: str, current_user: Account, dataset_id: UUID, document_id: UUID) -> dict[str, Any]:
# Reuse the shared permission/tenant checks implemented in DocumentResource.
document = self.get_document(session, str(dataset_id), str(document_id), current_user, current_tenant_id)
return UrlResponse(url=DocumentService.get_document_download_url(document, session)).model_dump(mode="json")
document = self.get_document(str(dataset_id), str(document_id), current_user, current_tenant_id)
return UrlResponse(url=DocumentService.get_document_download_url(document, db.session())).model_dump(
mode="json"
)
@console_ns.route("/datasets/<uuid:dataset_id>/documents/download-zip")
@@ -1154,8 +1111,7 @@ class DocumentBatchDownloadZipApi(DocumentResource):
@with_current_user
@with_current_tenant_id
@rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_EDIT)
@with_session(write=False)
def post(self, session: Session, current_tenant_id: str, current_user: Account, dataset_id: UUID):
def post(self, current_tenant_id: str, current_user: Account, dataset_id: UUID):
"""Stream a ZIP archive containing the requested uploaded documents."""
# Parse and validate request payload.
payload = DocumentBatchDownloadZipPayload.model_validate(console_ns.payload or {})
@@ -1167,7 +1123,7 @@ class DocumentBatchDownloadZipApi(DocumentResource):
document_ids=document_ids,
tenant_id=current_tenant_id,
current_user=current_user,
session=session,
session=db.session(),
)
# Delegate ZIP packing to FileService, but keep Flask response+cleanup in the route.
@@ -1206,10 +1162,8 @@ class DocumentProcessingApi(DocumentResource):
@with_current_user
@with_current_tenant_id
@rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_EDIT)
@with_session
def patch(
self,
session: Session,
current_tenant_id: str,
current_user: Account,
dataset_id: UUID,
@@ -1218,7 +1172,7 @@ class DocumentProcessingApi(DocumentResource):
):
dataset_id_str = str(dataset_id)
document_id_str = str(document_id)
document = self.get_document(session, dataset_id_str, document_id_str, current_user, current_tenant_id)
document = self.get_document(dataset_id_str, document_id_str, current_user, current_tenant_id)
# The role of the current user in the ta table must be admin, owner, dataset_operator, or editor
if not current_user.is_dataset_editor:
@@ -1232,6 +1186,7 @@ class DocumentProcessingApi(DocumentResource):
document.paused_by = current_user.id
document.paused_at = naive_utc_now()
document.is_paused = True
db.session.commit()
case "resume":
if document.indexing_status not in {IndexingStatus.PAUSED, IndexingStatus.ERROR}:
@@ -1240,6 +1195,7 @@ class DocumentProcessingApi(DocumentResource):
document.paused_by = None
document.paused_at = None
document.is_paused = False
db.session.commit()
return SimpleResultResponse(result="success").model_dump(mode="json"), 200
@@ -1263,11 +1219,10 @@ class DocumentMetadataApi(DocumentResource):
@with_current_user
@with_current_tenant_id
@rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_EDIT)
@with_session
def put(self, session: Session, current_tenant_id: str, current_user: Account, dataset_id: UUID, document_id: UUID):
def put(self, current_tenant_id: str, current_user: Account, dataset_id: UUID, document_id: UUID):
dataset_id_str = str(dataset_id)
document_id_str = str(document_id)
document = self.get_document(session, dataset_id_str, document_id_str, current_user, current_tenant_id)
document = self.get_document(dataset_id_str, document_id_str, current_user, current_tenant_id)
req_data = DocumentMetadataUpdatePayload.model_validate(request.get_json() or {})
@@ -1299,6 +1254,7 @@ class DocumentMetadataApi(DocumentResource):
document.doc_type = doc_type
document.updated_at = naive_utc_now()
db.session.commit()
return SimpleResultMessageResponse(result="success", message="Document metadata updated.").model_dump(
mode="json"
@@ -1315,16 +1271,11 @@ class DocumentStatusApi(DocumentResource):
@console_ns.response(200, "Success", console_ns.models[SimpleResultResponse.__name__])
@with_current_user
@rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_EDIT)
@with_session
def patch(
self,
session: Session,
current_user: Account,
dataset_id: UUID,
action: Literal["enable", "disable", "archive", "un_archive"],
self, current_user: Account, dataset_id: UUID, action: Literal["enable", "disable", "archive", "un_archive"]
):
dataset_id_str = str(dataset_id)
dataset = DatasetService.get_dataset(dataset_id_str, session)
dataset = DatasetService.get_dataset(dataset_id_str, db.session())
if dataset is None:
raise NotFound("Dataset not found.")
@@ -1336,12 +1287,12 @@ class DocumentStatusApi(DocumentResource):
DatasetService.check_dataset_model_setting(dataset)
# check user's permission
DatasetService.check_dataset_permission(dataset, current_user, session)
DatasetService.check_dataset_permission(dataset, current_user, db.session())
document_ids = request.args.getlist("document_id")
try:
DocumentService.batch_update_document_status(dataset, document_ids, action, current_user, session)
DocumentService.batch_update_document_status(dataset, document_ids, action, current_user, db.session())
except services.errors.document.DocumentIndexingError as e:
raise InvalidActionError(str(e))
except ValueError as e:
@@ -1360,17 +1311,16 @@ class DocumentPauseApi(DocumentResource):
@cloud_edition_billing_rate_limit_check("knowledge")
@console_ns.response(204, "Document paused successfully")
@rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_EDIT)
@with_session
def patch(self, session: Session, dataset_id: UUID, document_id: UUID):
def patch(self, dataset_id: UUID, document_id: UUID):
"""pause document."""
dataset_id_str = str(dataset_id)
document_id_str = str(document_id)
dataset = DatasetService.get_dataset(dataset_id_str, session)
dataset = DatasetService.get_dataset(dataset_id_str, db.session())
if not dataset:
raise NotFound("Dataset not found.")
document = DocumentService.get_document(dataset.id, document_id_str, session=session)
document = DocumentService.get_document(dataset.id, document_id_str, session=db.session())
# 404 if document not found
if document is None:
@@ -1382,7 +1332,7 @@ class DocumentPauseApi(DocumentResource):
try:
# pause document
DocumentService.pause_document(document, session)
DocumentService.pause_document(document, db.session())
except services.errors.document.DocumentIndexingError:
raise DocumentIndexingError("Cannot pause completed document.")
@@ -1397,15 +1347,14 @@ class DocumentRecoverApi(DocumentResource):
@cloud_edition_billing_rate_limit_check("knowledge")
@console_ns.response(204, "Document resumed successfully")
@rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_EDIT)
@with_session
def patch(self, session: Session, dataset_id: UUID, document_id: UUID):
def patch(self, dataset_id: UUID, document_id: UUID):
"""recover document."""
dataset_id_str = str(dataset_id)
document_id_str = str(document_id)
dataset = DatasetService.get_dataset(dataset_id_str, session)
dataset = DatasetService.get_dataset(dataset_id_str, db.session())
if not dataset:
raise NotFound("Dataset not found.")
document = DocumentService.get_document(dataset.id, document_id_str, session=session)
document = DocumentService.get_document(dataset.id, document_id_str, session=db.session())
# 404 if document not found
if document is None:
@@ -1416,7 +1365,7 @@ class DocumentRecoverApi(DocumentResource):
raise ArchivedDocumentImmutableError()
try:
# pause document
DocumentService.recover_document(document, session)
DocumentService.recover_document(document, db.session())
except services.errors.document.DocumentIndexingError:
raise DocumentIndexingError("Document is not in paused status.")
@@ -1432,18 +1381,17 @@ class DocumentRetryApi(DocumentResource):
@console_ns.expect(console_ns.models[DocumentRetryPayload.__name__])
@console_ns.response(204, "Documents retry started successfully")
@rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_EDIT)
@with_session
def post(self, session: Session, dataset_id: UUID):
def post(self, dataset_id: UUID):
"""retry document."""
payload = DocumentRetryPayload.model_validate(console_ns.payload or {})
dataset_id_str = str(dataset_id)
dataset = DatasetService.get_dataset(dataset_id_str, session)
dataset = DatasetService.get_dataset(dataset_id_str, db.session())
retry_documents = []
if not dataset:
raise NotFound("Dataset not found.")
for document_id in payload.document_ids:
try:
document = DocumentService.get_document(dataset.id, document_id, session=session)
document = DocumentService.get_document(dataset.id, document_id, session=db.session())
# 404 if document not found
if document is None:
@@ -1461,7 +1409,7 @@ class DocumentRetryApi(DocumentResource):
logger.exception("Failed to retry document, document id: %s", document_id)
continue
# retry document
DocumentService.retry_document(dataset_id_str, retry_documents, session)
DocumentService.retry_document(dataset_id_str, retry_documents, db.session())
return "", 204
@@ -1475,23 +1423,22 @@ class DocumentRenameApi(DocumentResource):
@console_ns.expect(console_ns.models[DocumentRenamePayload.__name__])
@with_current_user
@rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_EDIT)
@with_session
def post(self, session: Session, current_user: Account, dataset_id: UUID, document_id: UUID):
def post(self, current_user: Account, dataset_id: UUID, document_id: UUID):
# The role of the current user in the ta table must be admin, owner, editor, or dataset_operator
if not current_user.is_dataset_editor:
raise Forbidden()
dataset = DatasetService.get_dataset(dataset_id, session)
dataset = DatasetService.get_dataset(str(dataset_id), db.session())
if not dataset:
raise NotFound("Dataset not found.")
DatasetService.check_dataset_operator_permission(current_user, dataset, session=session)
DatasetService.check_dataset_operator_permission(current_user, dataset, session=db.session())
payload = DocumentRenamePayload.model_validate(console_ns.payload or {})
try:
document = DocumentService.rename_document(str(dataset_id), str(document_id), payload.name, session)
document = DocumentService.rename_document(str(dataset_id), str(document_id), payload.name, db.session())
except services.errors.document.DocumentIndexingError:
raise DocumentIndexingError("Cannot delete document during indexing.")
return dump_response(DocumentResponse, document_response(document, session=session))
return dump_response(DocumentResponse, document)
@console_ns.route("/datasets/<uuid:dataset_id>/documents/<uuid:document_id>/website-sync")
@@ -1502,15 +1449,14 @@ class WebsiteDocumentSyncApi(DocumentResource):
@console_ns.response(200, "Success", console_ns.models[SimpleResultResponse.__name__])
@with_current_tenant_id
@rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_CREATE_AND_MANAGEMENT)
@with_session
def get(self, session: Session, current_tenant_id: str, dataset_id: UUID, document_id: UUID):
def get(self, current_tenant_id: str, dataset_id: UUID, document_id: UUID):
"""sync website document."""
dataset_id_str = str(dataset_id)
dataset = DatasetService.get_dataset(dataset_id_str, session)
dataset = DatasetService.get_dataset(dataset_id_str, db.session())
if not dataset:
raise NotFound("Dataset not found.")
document_id_str = str(document_id)
document = DocumentService.get_document(dataset.id, document_id_str, session=session)
document = DocumentService.get_document(dataset.id, document_id_str, session=db.session())
if not document:
raise NotFound("Document not found.")
if document.tenant_id != current_tenant_id:
@@ -1521,7 +1467,7 @@ class WebsiteDocumentSyncApi(DocumentResource):
if DocumentService.check_archived(document):
raise ArchivedDocumentImmutableError()
# sync document
DocumentService.sync_website_document(dataset_id_str, document, session)
DocumentService.sync_website_document(dataset_id_str, document, db.session())
return SimpleResultResponse(result="success").model_dump(mode="json"), 200
@@ -1537,18 +1483,17 @@ class DocumentPipelineExecutionLogApi(DocumentResource):
@login_required
@account_initialization_required
@rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_CREATE_AND_MANAGEMENT)
@with_session(write=False)
def get(self, session: Session, dataset_id: UUID, document_id: UUID):
def get(self, dataset_id: UUID, document_id: UUID):
dataset_id_str = str(dataset_id)
document_id_str = str(document_id)
dataset = DatasetService.get_dataset(dataset_id_str, session)
dataset = DatasetService.get_dataset(dataset_id_str, db.session())
if not dataset:
raise NotFound("Dataset not found.")
document = DocumentService.get_document(dataset.id, document_id_str, session=session)
document = DocumentService.get_document(dataset.id, document_id_str, session=db.session())
if not document:
raise NotFound("Document not found.")
log = session.scalar(
log = db.session.scalar(
select(DocumentPipelineExecutionLog)
.where(DocumentPipelineExecutionLog.document_id == document_id_str)
.order_by(DocumentPipelineExecutionLog.created_at.desc())
@@ -1587,8 +1532,7 @@ class DocumentGenerateSummaryApi(Resource):
@cloud_edition_billing_rate_limit_check("knowledge")
@with_current_user
@rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_EDIT)
@with_session
def post(self, session: Session, current_user: Account, dataset_id: UUID):
def post(self, current_user: Account, dataset_id: UUID):
"""
Generate summary index for specified documents.
@@ -1599,7 +1543,7 @@ class DocumentGenerateSummaryApi(Resource):
dataset_id_str = str(dataset_id)
# Get dataset
dataset = DatasetService.get_dataset(dataset_id_str, session)
dataset = DatasetService.get_dataset(dataset_id_str, db.session())
if not dataset:
raise NotFound("Dataset not found.")
@@ -1608,7 +1552,7 @@ class DocumentGenerateSummaryApi(Resource):
raise Forbidden()
try:
DatasetService.check_dataset_permission(dataset, current_user, session)
DatasetService.check_dataset_permission(dataset, current_user, db.session())
except services.errors.account.NoPermissionError as e:
raise Forbidden(str(e))
@@ -1633,7 +1577,7 @@ class DocumentGenerateSummaryApi(Resource):
raise ValueError("Summary index is not enabled for this dataset. Please enable it in the dataset settings.")
# Verify all documents exist and belong to the dataset
documents = DocumentService.get_documents_by_ids(dataset_id_str, document_list, session)
documents = DocumentService.get_documents_by_ids(dataset_id_str, document_list, db.session())
if len(documents) != len(document_list):
found_ids = {doc.id for doc in documents}
@@ -1649,7 +1593,7 @@ class DocumentGenerateSummaryApi(Resource):
DocumentService.update_documents_need_summary(
dataset_id=dataset_id_str,
document_ids=document_ids_to_update,
session=session,
session=db.session(),
need_summary=True,
)
@@ -1687,8 +1631,7 @@ class DocumentSummaryStatusApi(DocumentResource):
@account_initialization_required
@with_current_user
@rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_CREATE_AND_MANAGEMENT)
@with_session(write=False)
def get(self, session: Session, current_user: Account, dataset_id: UUID, document_id: UUID):
def get(self, current_user: Account, dataset_id: UUID, document_id: UUID):
"""
Get summary index generation status for a document.
@@ -1706,13 +1649,13 @@ class DocumentSummaryStatusApi(DocumentResource):
document_id_str = str(document_id)
# Get dataset
dataset = DatasetService.get_dataset(dataset_id_str, session)
dataset = DatasetService.get_dataset(dataset_id_str, db.session())
if not dataset:
raise NotFound("Dataset not found.")
# Check permissions
try:
DatasetService.check_dataset_permission(dataset, current_user, session)
DatasetService.check_dataset_permission(dataset, current_user, db.session())
except services.errors.account.NoPermissionError as e:
raise Forbidden(str(e))
@@ -1722,7 +1665,7 @@ class DocumentSummaryStatusApi(DocumentResource):
result = SummaryIndexService.get_document_summary_status_detail(
document_id=document_id_str,
dataset_id=dataset_id_str,
session=session,
session=db.session(),
)
return dump_response(DocumentSummaryStatusResponse, result), 200
@@ -8,7 +8,6 @@ from flask_restx import Resource
from pydantic import BaseModel, Field
from sqlalchemy import String, case, cast, func, literal, or_, select
from sqlalchemy.dialects.postgresql import JSONB
from sqlalchemy.orm import Session
from werkzeug.exceptions import Forbidden, NotFound
import services
@@ -21,7 +20,6 @@ from controllers.common.schema import (
register_response_schema_models,
register_schema_models,
)
from controllers.common.session import with_session
from controllers.console import console_ns
from controllers.console.app.error import ProviderNotInitializeError
from controllers.console.datasets.error import (
@@ -44,6 +42,7 @@ from controllers.console.wraps import (
from core.errors.error import LLMBadRequestError, ProviderTokenNotInitError
from core.model_manager import ModelManager
from core.rag.index_processor.constant.index_type import IndexTechniqueType
from extensions.ext_database import db
from extensions.ext_redis import redis_client
from fields.base import ResponseModel
from fields.segment_fields import (
@@ -166,7 +165,7 @@ register_response_schema_models(
def _get_segment_for_document(
session: Session, dataset: Dataset, document: Document, segment_id: str
dataset: Dataset, document: Document, segment_id: str
) -> tuple[SegmentRef, DocumentSegment]:
dataset_ref = DatasetRefService.create_dataset_ref(dataset)
document_ref = DatasetRefService.create_document_ref(dataset_ref, document)
@@ -174,7 +173,7 @@ def _get_segment_for_document(
raise NotFound("Document not found.")
segment_ref = DatasetRefService.create_segment_ref(document_ref, segment_id)
segment = SegmentService.get_segment_by_ref(segment_ref, session=session)
segment = SegmentService.get_segment_by_ref(segment_ref, db.session())
if not segment:
raise NotFound("Segment not found.")
return segment_ref, segment
@@ -191,20 +190,19 @@ class DatasetDocumentSegmentListApi(Resource):
@with_current_user
@with_current_tenant_id
@rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_READONLY)
@with_session(write=False)
def get(self, session: Session, current_tenant_id: str, current_user: Account, dataset_id: UUID, document_id: UUID):
def get(self, current_tenant_id: str, current_user: Account, dataset_id: UUID, document_id: UUID):
dataset_id_str = str(dataset_id)
document_id_str = str(document_id)
dataset = DatasetService.get_dataset(dataset_id_str, session)
dataset = DatasetService.get_dataset(dataset_id_str, db.session())
if not dataset:
raise NotFound("Dataset not found.")
try:
DatasetService.check_dataset_permission(dataset, current_user, session)
DatasetService.check_dataset_permission(dataset, current_user, db.session())
except services.errors.account.NoPermissionError as e:
raise Forbidden(str(e))
document = DocumentService.get_document(dataset_id_str, document_id_str, session=session)
document = DocumentService.get_document(dataset_id_str, document_id_str, session=db.session())
if not document:
raise NotFound("Document not found.")
@@ -273,19 +271,19 @@ class DatasetDocumentSegmentListApi(Resource):
elif args.enabled.lower() == "false":
query = query.where(DocumentSegment.enabled == False)
segments = paginate_query(query, session=session, page=page, per_page=limit, max_per_page=100)
segments = paginate_query(query, page=page, per_page=limit, max_per_page=100)
segment_list = list(segments.items)
segment_ids = [segment.id for segment in segment_list]
summaries: dict[str, str | None] = {}
if segment_ids:
summary_records = SummaryIndexService.get_segments_summaries(
segment_ids=segment_ids, dataset_id=dataset_id_str, session=session
segment_ids=segment_ids, dataset_id=dataset_id_str, session=db.session()
)
summaries = {chunk_id: summary.summary_content for chunk_id, summary in summary_records.items()}
response = {
"data": segment_responses_with_summaries(segment_list, summaries, session=session),
"data": segment_responses_with_summaries(segment_list, summaries),
"limit": limit,
"total": segments.total,
"total_pages": segments.pages,
@@ -302,18 +300,17 @@ class DatasetDocumentSegmentListApi(Resource):
@console_ns.response(204, "Segments deleted successfully")
@with_current_user
@rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_EDIT)
@with_session
def delete(self, session: Session, current_user: Account, dataset_id: UUID, document_id: UUID):
def delete(self, current_user: Account, dataset_id: UUID, document_id: UUID):
# check dataset
dataset_id_str = str(dataset_id)
dataset = DatasetService.get_dataset(dataset_id_str, session)
dataset = DatasetService.get_dataset(dataset_id_str, db.session())
if not dataset:
raise NotFound("Dataset not found.")
# check user's model setting
DatasetService.check_dataset_model_setting(dataset)
# check document
document_id_str = str(document_id)
document = DocumentService.get_document(dataset_id_str, document_id_str, session=session)
document = DocumentService.get_document(dataset_id_str, document_id_str, session=db.session())
if not document:
raise NotFound("Document not found.")
segment_ids = request.args.getlist("segment_id")
@@ -322,10 +319,10 @@ class DatasetDocumentSegmentListApi(Resource):
if not current_user.is_dataset_editor:
raise Forbidden()
try:
DatasetService.check_dataset_permission(dataset, current_user, session)
DatasetService.check_dataset_permission(dataset, current_user, db.session())
except services.errors.account.NoPermissionError as e:
raise Forbidden(str(e))
SegmentService.delete_segments(segment_ids, document, dataset, session)
SegmentService.delete_segments(segment_ids, document, dataset, db.session())
return "", 204
@@ -342,10 +339,8 @@ class DatasetDocumentSegmentApi(Resource):
@with_current_user
@with_current_tenant_id
@rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_EDIT)
@with_session
def patch(
self,
session: Session,
current_tenant_id: str,
current_user: Account,
dataset_id: UUID,
@@ -353,11 +348,11 @@ class DatasetDocumentSegmentApi(Resource):
action: Literal["enable", "disable"],
):
dataset_id_str = str(dataset_id)
dataset = DatasetService.get_dataset(dataset_id_str, session)
dataset = DatasetService.get_dataset(dataset_id_str, db.session())
if not dataset:
raise NotFound("Dataset not found.")
document_id_str = str(document_id)
document = DocumentService.get_document(dataset_id_str, document_id_str, session=session)
document = DocumentService.get_document(dataset_id_str, document_id_str, session=db.session())
if not document:
raise NotFound("Document not found.")
# check user's model setting
@@ -367,7 +362,7 @@ class DatasetDocumentSegmentApi(Resource):
raise Forbidden()
try:
DatasetService.check_dataset_permission(dataset, current_user, session)
DatasetService.check_dataset_permission(dataset, current_user, db.session())
except services.errors.account.NoPermissionError as e:
raise Forbidden(str(e))
if dataset.indexing_technique == IndexTechniqueType.HIGH_QUALITY:
@@ -393,7 +388,7 @@ class DatasetDocumentSegmentApi(Resource):
if cache_result is not None:
raise InvalidActionError("Document is being indexed, please try again later")
try:
SegmentService.update_segments_status(segment_ids, action, dataset, document, session)
SegmentService.update_segments_status(segment_ids, action, dataset, document, db.session())
except Exception as e:
raise InvalidActionError(str(e))
return SimpleResultResponse(result="success").model_dump(mode="json"), 200
@@ -413,23 +408,15 @@ class DatasetDocumentSegmentAddApi(Resource):
@with_current_user
@with_current_tenant_id
@rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_EDIT)
@with_session
def post(
self,
session: Session,
current_tenant_id: str,
current_user: Account,
dataset_id: UUID,
document_id: UUID,
):
def post(self, current_tenant_id: str, current_user: Account, dataset_id: UUID, document_id: UUID):
# check dataset
dataset_id_str = str(dataset_id)
dataset = DatasetService.get_dataset(dataset_id_str, session)
dataset = DatasetService.get_dataset(dataset_id_str, db.session())
if not dataset:
raise NotFound("Dataset not found.")
# check document
document_id_str = str(document_id)
document = DocumentService.get_document(dataset_id_str, document_id_str, session=session)
document = DocumentService.get_document(dataset_id_str, document_id_str, session=db.session())
if not document:
raise NotFound("Document not found.")
if not current_user.is_dataset_editor:
@@ -451,21 +438,22 @@ class DatasetDocumentSegmentAddApi(Resource):
except ProviderTokenNotInitError as ex:
raise ProviderNotInitializeError(ex.description)
try:
DatasetService.check_dataset_permission(dataset, current_user, session)
DatasetService.check_dataset_permission(dataset, current_user, db.session())
except services.errors.account.NoPermissionError as e:
raise Forbidden(str(e))
# validate args
payload = SegmentCreatePayload.model_validate(console_ns.payload or {})
payload_dict = payload.model_dump(exclude_none=True)
SegmentService.segment_create_args_validate(payload_dict, document)
segment = type_cast(DocumentSegment, SegmentService.create_segment(payload_dict, document, dataset, session))
segment = type_cast(
DocumentSegment,
SegmentService.create_segment(payload_dict, document, dataset, db.session()),
)
summary = SummaryIndexService.get_segment_summary(
segment_id=segment.id, dataset_id=dataset_id_str, session=session
segment_id=segment.id, dataset_id=dataset_id_str, session=db.session()
)
response = {
"data": segment_response_with_summary(
segment, summary.summary_content if summary else None, session=session
),
"data": segment_response_with_summary(segment, summary.summary_content if summary else None),
"doc_form": document.doc_form,
}
return dump_response(SegmentDetailResponse, response), 200
@@ -484,33 +472,26 @@ class DatasetDocumentSegmentUpdateApi(Resource):
@with_current_user
@with_current_tenant_id
@rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_EDIT)
@with_session
def patch(
self,
session: Session,
current_tenant_id: str,
current_user: Account,
dataset_id: UUID,
document_id: UUID,
segment_id: UUID,
self, current_tenant_id: str, current_user: Account, dataset_id: UUID, document_id: UUID, segment_id: UUID
):
# check dataset
dataset_id_str = str(dataset_id)
dataset = DatasetService.get_dataset(dataset_id_str, session)
dataset = DatasetService.get_dataset(dataset_id_str, db.session())
if not dataset:
raise NotFound("Dataset not found.")
# check user's model setting
DatasetService.check_dataset_model_setting(dataset)
# check document
document_id_str = str(document_id)
document = DocumentService.get_document(dataset_id_str, document_id_str, session=session)
document = DocumentService.get_document(dataset_id_str, document_id_str, session=db.session())
if not document:
raise NotFound("Document not found.")
# The role of the current user in the ta table must be admin, owner, dataset_operator, or editor
if not current_user.is_dataset_editor:
raise Forbidden()
try:
DatasetService.check_dataset_permission(dataset, current_user, session)
DatasetService.check_dataset_permission(dataset, current_user, db.session())
except services.errors.account.NoPermissionError as e:
raise Forbidden(str(e))
if dataset.indexing_technique == IndexTechniqueType.HIGH_QUALITY:
@@ -530,7 +511,7 @@ class DatasetDocumentSegmentUpdateApi(Resource):
except ProviderTokenNotInitError as ex:
raise ProviderNotInitializeError(ex.description)
segment_id_str = str(segment_id)
_, segment = _get_segment_for_document(session, dataset, document, segment_id_str)
_, segment = _get_segment_for_document(dataset, document, segment_id_str)
# validate args
payload = SegmentUpdatePayload.model_validate(console_ns.payload or {})
payload_dict = payload.model_dump(exclude_none=True)
@@ -542,15 +523,13 @@ class DatasetDocumentSegmentUpdateApi(Resource):
segment,
document,
dataset,
session,
db.session(),
)
summary = SummaryIndexService.get_segment_summary(
segment_id=segment.id, dataset_id=dataset_id_str, session=session
segment_id=segment.id, dataset_id=dataset_id_str, session=db.session()
)
response = {
"data": segment_response_with_summary(
segment, summary.summary_content if summary else None, session=session
),
"data": segment_response_with_summary(segment, summary.summary_content if summary else None),
"doc_form": document.doc_form,
}
return dump_response(SegmentDetailResponse, response), 200
@@ -564,38 +543,31 @@ class DatasetDocumentSegmentUpdateApi(Resource):
@with_current_user
@with_current_tenant_id
@rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_EDIT)
@with_session
def delete(
self,
session: Session,
current_tenant_id: str,
current_user: Account,
dataset_id: UUID,
document_id: UUID,
segment_id: UUID,
self, current_tenant_id: str, current_user: Account, dataset_id: UUID, document_id: UUID, segment_id: UUID
):
# check dataset
dataset_id_str = str(dataset_id)
dataset = DatasetService.get_dataset(dataset_id_str, session)
dataset = DatasetService.get_dataset(dataset_id_str, db.session())
if not dataset:
raise NotFound("Dataset not found.")
# check user's model setting
DatasetService.check_dataset_model_setting(dataset)
# check document
document_id_str = str(document_id)
document = DocumentService.get_document(dataset_id_str, document_id_str, session=session)
document = DocumentService.get_document(dataset_id_str, document_id_str, session=db.session())
if not document:
raise NotFound("Document not found.")
# The role of the current user in the ta table must be admin, owner, dataset_operator, or editor
if not current_user.is_dataset_editor:
raise Forbidden()
try:
DatasetService.check_dataset_permission(dataset, current_user, session)
DatasetService.check_dataset_permission(dataset, current_user, db.session())
except services.errors.account.NoPermissionError as e:
raise Forbidden(str(e))
segment_id_str = str(segment_id)
_, segment = _get_segment_for_document(session, dataset, document, segment_id_str)
SegmentService.delete_segment(segment, document, dataset, session)
_, segment = _get_segment_for_document(dataset, document, segment_id_str)
SegmentService.delete_segment(segment, document, dataset, db.session())
return "", 204
@@ -615,30 +587,22 @@ class DatasetDocumentSegmentBatchImportApi(Resource):
@with_current_user
@with_current_tenant_id
@rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_EDIT)
@with_session
def post(
self,
session: Session,
current_tenant_id: str,
current_user: Account,
dataset_id: UUID,
document_id: UUID,
):
def post(self, current_tenant_id: str, current_user: Account, dataset_id: UUID, document_id: UUID):
# check dataset
dataset_id_str = str(dataset_id)
dataset = DatasetService.get_dataset(dataset_id_str, session)
dataset = DatasetService.get_dataset(dataset_id_str, db.session())
if not dataset:
raise NotFound("Dataset not found.")
# check document
document_id_str = str(document_id)
document = DocumentService.get_document(dataset_id_str, document_id_str, session=session)
document = DocumentService.get_document(dataset_id_str, document_id_str, session=db.session())
if not document:
raise NotFound("Document not found.")
payload = BatchImportPayload.model_validate(console_ns.payload or {})
upload_file_id = payload.upload_file_id
upload_file = session.scalar(select(UploadFile).where(UploadFile.id == upload_file_id).limit(1))
upload_file = db.session.scalar(select(UploadFile).where(UploadFile.id == upload_file_id).limit(1))
if not upload_file:
raise NotFound("UploadFile not found.")
@@ -696,30 +660,23 @@ class ChildChunkAddApi(Resource):
@with_current_user
@with_current_tenant_id
@rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_EDIT)
@with_session
def post(
self,
session: Session,
current_tenant_id: str,
current_user: Account,
dataset_id: UUID,
document_id: UUID,
segment_id: UUID,
self, current_tenant_id: str, current_user: Account, dataset_id: UUID, document_id: UUID, segment_id: UUID
):
# check dataset
dataset_id_str = str(dataset_id)
dataset = DatasetService.get_dataset(dataset_id_str, session)
dataset = DatasetService.get_dataset(dataset_id_str, db.session())
if not dataset:
raise NotFound("Dataset not found.")
# check document
document_id_str = str(document_id)
document = DocumentService.get_document(dataset_id_str, document_id_str, session=session)
document = DocumentService.get_document(dataset_id_str, document_id_str, session=db.session())
if not document:
raise NotFound("Document not found.")
if not current_user.is_dataset_editor:
raise Forbidden()
try:
DatasetService.check_dataset_permission(dataset, current_user, session)
DatasetService.check_dataset_permission(dataset, current_user, db.session())
except services.errors.account.NoPermissionError as e:
raise Forbidden(str(e))
# check embedding model setting
@@ -739,11 +696,11 @@ class ChildChunkAddApi(Resource):
except ProviderTokenNotInitError as ex:
raise ProviderNotInitializeError(ex.description)
segment_id_str = str(segment_id)
_, segment = _get_segment_for_document(session, dataset, document, segment_id_str)
_, segment = _get_segment_for_document(dataset, document, segment_id_str)
# validate args
try:
payload = ChildChunkCreatePayload.model_validate(console_ns.payload or {})
child_chunk = SegmentService.create_child_chunk(payload.content, segment, document, dataset, session)
child_chunk = SegmentService.create_child_chunk(payload.content, segment, document, dataset, db.session())
except ChildChunkIndexingServiceError as e:
raise ChildChunkIndexingError(str(e))
return dump_response(ChildChunkDetailResponse, {"data": child_chunk}), 200
@@ -756,22 +713,21 @@ class ChildChunkAddApi(Resource):
@account_initialization_required
@with_current_tenant_id
@rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_READONLY)
@with_session(write=False)
def get(self, session: Session, current_tenant_id: str, dataset_id: UUID, document_id: UUID, segment_id: UUID):
def get(self, current_tenant_id: str, dataset_id: UUID, document_id: UUID, segment_id: UUID):
# check dataset
dataset_id_str = str(dataset_id)
dataset = DatasetService.get_dataset(dataset_id_str, session)
dataset = DatasetService.get_dataset(dataset_id_str, db.session())
if not dataset:
raise NotFound("Dataset not found.")
# check user's model setting
DatasetService.check_dataset_model_setting(dataset)
# check document
document_id_str = str(document_id)
document = DocumentService.get_document(dataset_id_str, document_id_str, session=session)
document = DocumentService.get_document(dataset_id_str, document_id_str, session=db.session())
if not document:
raise NotFound("Document not found.")
segment_id_str = str(segment_id)
_get_segment_for_document(session, dataset, document, segment_id_str)
_get_segment_for_document(dataset, document, segment_id_str)
args = query_params_from_request(ChildChunkListQuery, use_defaults_for_malformed_ints=True)
page = args.page
@@ -779,13 +735,7 @@ class ChildChunkAddApi(Resource):
keyword = args.keyword
child_chunks = SegmentService.get_child_chunks(
segment_id_str,
document_id_str,
dataset_id_str,
page,
limit,
keyword,
session=session,
segment_id_str, document_id_str, dataset_id_str, page, limit, keyword
)
response = {
"data": child_chunks.items,
@@ -811,41 +761,34 @@ class ChildChunkAddApi(Resource):
@with_current_user
@with_current_tenant_id
@rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_EDIT)
@with_session
def patch(
self,
session: Session,
current_tenant_id: str,
current_user: Account,
dataset_id: UUID,
document_id: UUID,
segment_id: UUID,
self, current_tenant_id: str, current_user: Account, dataset_id: UUID, document_id: UUID, segment_id: UUID
):
# check dataset
dataset_id_str = str(dataset_id)
dataset = DatasetService.get_dataset(dataset_id_str, session)
dataset = DatasetService.get_dataset(dataset_id_str, db.session())
if not dataset:
raise NotFound("Dataset not found.")
# check user's model setting
DatasetService.check_dataset_model_setting(dataset)
# check document
document_id_str = str(document_id)
document = DocumentService.get_document(dataset_id_str, document_id_str, session=session)
document = DocumentService.get_document(dataset_id_str, document_id_str, session=db.session())
if not document:
raise NotFound("Document not found.")
# The role of the current user in the ta table must be admin, owner, dataset_operator, or editor
if not current_user.is_dataset_editor:
raise Forbidden()
try:
DatasetService.check_dataset_permission(dataset, current_user, session)
DatasetService.check_dataset_permission(dataset, current_user, db.session())
except services.errors.account.NoPermissionError as e:
raise Forbidden(str(e))
segment_id_str = str(segment_id)
_, segment = _get_segment_for_document(session, dataset, document, segment_id_str)
_, segment = _get_segment_for_document(dataset, document, segment_id_str)
# validate args
payload = ChildChunkBatchUpdatePayload.model_validate(console_ns.payload or {})
try:
child_chunks = SegmentService.update_child_chunks(payload.chunks, segment, document, dataset, session)
child_chunks = SegmentService.update_child_chunks(payload.chunks, segment, document, dataset, db.session())
except ChildChunkIndexingServiceError as e:
raise ChildChunkIndexingError(str(e))
return dump_response(ChildChunkBatchUpdateResponse, {"data": child_chunks}), 200
@@ -864,10 +807,8 @@ class ChildChunkUpdateApi(Resource):
@with_current_user
@with_current_tenant_id
@rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_EDIT)
@with_session
def delete(
self,
session: Session,
current_tenant_id: str,
current_user: Account,
dataset_id: UUID,
@@ -877,31 +818,31 @@ class ChildChunkUpdateApi(Resource):
):
# check dataset
dataset_id_str = str(dataset_id)
dataset = DatasetService.get_dataset(dataset_id_str, session)
dataset = DatasetService.get_dataset(dataset_id_str, db.session())
if not dataset:
raise NotFound("Dataset not found.")
# check user's model setting
DatasetService.check_dataset_model_setting(dataset)
# check document
document_id_str = str(document_id)
document = DocumentService.get_document(dataset_id_str, document_id_str, session=session)
document = DocumentService.get_document(dataset_id_str, document_id_str, session=db.session())
if not document:
raise NotFound("Document not found.")
# The role of the current user in the ta table must be admin, owner, dataset_operator, or editor
if not current_user.is_dataset_editor:
raise Forbidden()
try:
DatasetService.check_dataset_permission(dataset, current_user, session)
DatasetService.check_dataset_permission(dataset, current_user, db.session())
except services.errors.account.NoPermissionError as e:
raise Forbidden(str(e))
segment_id_str = str(segment_id)
segment_ref, _ = _get_segment_for_document(session, dataset, document, segment_id_str)
segment_ref, _ = _get_segment_for_document(dataset, document, segment_id_str)
child_chunk_id_str = str(child_chunk_id)
child_chunk = SegmentService.get_child_chunk_by_segment_ref(child_chunk_id_str, segment_ref, session=session)
child_chunk = SegmentService.get_child_chunk_by_segment_ref(child_chunk_id_str, segment_ref, db.session())
if not child_chunk:
raise NotFound("Child chunk not found.")
try:
SegmentService.delete_child_chunk(child_chunk, dataset, session)
SegmentService.delete_child_chunk(child_chunk, dataset, db.session())
except ChildChunkDeleteIndexServiceError as e:
raise ChildChunkDeleteIndexError(str(e))
return "", 204
@@ -917,10 +858,8 @@ class ChildChunkUpdateApi(Resource):
@with_current_user
@with_current_tenant_id
@rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_EDIT)
@with_session
def patch(
self,
session: Session,
current_tenant_id: str,
current_user: Account,
dataset_id: UUID,
@@ -930,34 +869,34 @@ class ChildChunkUpdateApi(Resource):
):
# check dataset
dataset_id_str = str(dataset_id)
dataset = DatasetService.get_dataset(dataset_id_str, session)
dataset = DatasetService.get_dataset(dataset_id_str, db.session())
if not dataset:
raise NotFound("Dataset not found.")
# check user's model setting
DatasetService.check_dataset_model_setting(dataset)
# check document
document_id_str = str(document_id)
document = DocumentService.get_document(dataset_id_str, document_id_str, session=session)
document = DocumentService.get_document(dataset_id_str, document_id_str, session=db.session())
if not document:
raise NotFound("Document not found.")
# The role of the current user in the ta table must be admin, owner, dataset_operator, or editor
if not current_user.is_dataset_editor:
raise Forbidden()
try:
DatasetService.check_dataset_permission(dataset, current_user, session)
DatasetService.check_dataset_permission(dataset, current_user, db.session())
except services.errors.account.NoPermissionError as e:
raise Forbidden(str(e))
segment_id_str = str(segment_id)
segment_ref, segment = _get_segment_for_document(session, dataset, document, segment_id_str)
segment_ref, segment = _get_segment_for_document(dataset, document, segment_id_str)
child_chunk_id_str = str(child_chunk_id)
child_chunk = SegmentService.get_child_chunk_by_segment_ref(child_chunk_id_str, segment_ref, session=session)
child_chunk = SegmentService.get_child_chunk_by_segment_ref(child_chunk_id_str, segment_ref, db.session())
if not child_chunk:
raise NotFound("Child chunk not found.")
# validate args
try:
payload = ChildChunkUpdatePayload.model_validate(console_ns.payload or {})
child_chunk = SegmentService.update_child_chunk(
payload.content, child_chunk, segment, document, dataset, session
payload.content, child_chunk, segment, document, dataset, db.session()
)
except ChildChunkIndexingServiceError as e:
raise ChildChunkIndexingError(str(e))
+10 -41
View File
@@ -1,4 +1,3 @@
from dataclasses import dataclass
from datetime import datetime
from typing import Any
from uuid import UUID
@@ -11,13 +10,9 @@ from werkzeug.exceptions import Forbidden, InternalServerError, NotFound
import services
from controllers.common.fields import UsageCountResponse
from controllers.common.schema import (
query_params_from_model,
register_response_schema_models,
register_schema_models,
)
from controllers.common.session import with_session
from controllers.common.schema import query_params_from_model, register_response_schema_models, register_schema_models
from controllers.console import console_ns
from controllers.console.app.wraps import with_session
from controllers.console.datasets.error import DatasetNameDuplicateError
from controllers.console.wraps import (
RBACPermission,
@@ -30,11 +25,10 @@ from controllers.console.wraps import (
with_current_user,
)
from fields.base import ResponseModel
from fields.dataset_fields import DatasetDetailResponse, dataset_detail_response_source
from fields.dataset_fields import DatasetDetailResponse
from libs.helper import dump_response
from libs.login import login_required
from models import Account
from models.dataset import ExternalKnowledgeApis
from services.dataset_service import DatasetService
from services.enterprise import rbac_service as enterprise_rbac_service
from services.external_knowledge_service import ExternalDatasetService
@@ -96,28 +90,6 @@ class ExternalKnowledgeApiResponse(ResponseModel):
return value
@dataclass(frozen=True)
class ExternalKnowledgeApiResponseSource:
external_knowledge_api: ExternalKnowledgeApis
session: Session
@property
def dataset_bindings(self) -> Any:
return self.external_knowledge_api.get_dataset_bindings(session=self.session)
def __getattr__(self, name: str) -> Any:
return getattr(self.external_knowledge_api, name) # noqa: no-new-getattr response adapter delegates model fields
def external_knowledge_api_response(
external_knowledge_api: ExternalKnowledgeApis, *, session: Session
) -> ExternalKnowledgeApiResponse:
return ExternalKnowledgeApiResponse.model_validate(
ExternalKnowledgeApiResponseSource(external_knowledge_api=external_knowledge_api, session=session),
from_attributes=True,
)
class ExternalKnowledgeApiListResponse(ResponseModel):
data: list[ExternalKnowledgeApiResponse]
has_more: bool
@@ -190,15 +162,14 @@ class ExternalApiTemplateListApi(Resource):
@login_required
@with_current_tenant_id
@account_initialization_required
@with_session(write=False)
def get(self, session: Session, current_tenant_id: str):
def get(self, current_tenant_id: str):
query = ExternalApiTemplateListQuery.model_validate(request.args.to_dict())
external_knowledge_apis, total = ExternalDatasetService.get_external_knowledge_apis(
query.page, query.limit, current_tenant_id, query.keyword, session=session
query.page, query.limit, current_tenant_id, query.keyword
)
return ExternalKnowledgeApiListResponse(
data=[external_knowledge_api_response(item, session=session) for item in external_knowledge_apis],
data=[ExternalKnowledgeApiResponse.model_validate(item) for item in external_knowledge_apis],
has_more=len(external_knowledge_apis) == query.limit,
limit=query.limit,
total=total,
@@ -239,7 +210,7 @@ class ExternalApiTemplateListApi(Resource):
except services.errors.dataset.DatasetNameDuplicateError:
raise DatasetNameDuplicateError()
return external_knowledge_api_response(external_knowledge_api, session=session).model_dump(mode="json"), 201
return dump_response(ExternalKnowledgeApiResponse, external_knowledge_api), 201
@console_ns.route("/datasets/external-knowledge-api/<uuid:external_knowledge_api_id>")
@@ -266,7 +237,7 @@ class ExternalApiTemplateApi(Resource):
if external_knowledge_api is None:
raise NotFound("API template not found.")
return external_knowledge_api_response(external_knowledge_api, session=session).model_dump(mode="json"), 200
return dump_response(ExternalKnowledgeApiResponse, external_knowledge_api), 200
@console_ns.doc("update_external_api_template")
@console_ns.doc(description="Update external knowledge API template")
@@ -298,7 +269,7 @@ class ExternalApiTemplateApi(Resource):
session=session,
)
return external_knowledge_api_response(external_knowledge_api, session=session).model_dump(mode="json"), 200
return dump_response(ExternalKnowledgeApiResponse, external_knowledge_api), 200
@setup_required
@login_required
@@ -383,9 +354,7 @@ class ExternalDatasetCreateApi(Resource):
[dataset_id_str],
session=session,
)
data = DatasetDetailResponse.model_validate(
dataset_detail_response_source(dataset, session=session)
).model_dump(mode="json")
data = DatasetDetailResponse.model_validate(dataset).model_dump(mode="json")
data["permission_keys"] = permission_keys_map.get(dataset_id_str, [])
return data, 201
@@ -53,7 +53,7 @@ class HitTestingApi(Resource, DatasetsHitTestingBase):
) -> dict[str, object]:
dataset_id_str = str(dataset_id)
dataset = self.get_and_validate_dataset(session, dataset_id_str, current_user, current_tenant_id)
dataset = self.get_and_validate_dataset(dataset_id_str, current_user, current_tenant_id)
args = self.parse_args(console_ns.payload)
self.hit_testing_args_check(args)
@@ -19,6 +19,7 @@ from core.errors.error import (
ProviderTokenNotInitError,
QuotaExceededError,
)
from extensions.ext_database import db
from graphon.model_runtime.errors.invoke import InvokeError
from libs.login import resolve_account_fallback
from models.account import Account
@@ -82,18 +83,15 @@ class DatasetsHitTestingBase:
@staticmethod
def get_and_validate_dataset(
session: Session,
dataset_id: str,
current_user: Account | None = None,
current_tenant_id: str | None = None,
dataset_id: str, current_user: Account | None = None, current_tenant_id: str | None = None
) -> Dataset:
current_user, _ = resolve_account_fallback(current_user, current_tenant_id)
dataset = DatasetService.get_dataset(dataset_id, session)
dataset = DatasetService.get_dataset(dataset_id, db.session())
if dataset is None:
raise NotFound("Dataset not found.")
try:
DatasetService.check_dataset_permission(dataset, current_user, session)
DatasetService.check_dataset_permission(dataset, current_user, db.session())
except services.errors.account.NoPermissionError as e:
raise Forbidden(str(e))
+25 -39
View File
@@ -2,12 +2,10 @@ from typing import Literal
from uuid import UUID
from flask_restx import Resource
from sqlalchemy.orm import Session
from werkzeug.exceptions import NotFound
from controllers.common.controller_schemas import MetadataUpdatePayload
from controllers.common.schema import register_response_schema_models, register_schema_models
from controllers.common.session import with_session
from controllers.console import console_ns
from controllers.console.wraps import (
RBACPermission,
@@ -19,6 +17,7 @@ from controllers.console.wraps import (
with_current_tenant_id,
with_current_user,
)
from extensions.ext_database import db
from fields.dataset_fields import (
DatasetMetadataBuiltInFieldsResponse,
DatasetMetadataListResponse,
@@ -58,18 +57,17 @@ class DatasetMetadataCreateApi(Resource):
@with_current_user
@with_current_tenant_id
@rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_EDIT)
@with_session
def post(self, session: Session, current_tenant_id: str, current_user: Account, dataset_id: UUID):
def post(self, current_tenant_id: str, current_user: Account, dataset_id: UUID):
metadata_args = MetadataArgs.model_validate(console_ns.payload or {})
dataset_id_str = str(dataset_id)
dataset = DatasetService.get_dataset(dataset_id_str, session)
dataset = DatasetService.get_dataset(dataset_id_str, db.session())
if dataset is None:
raise NotFound("Dataset not found.")
DatasetService.check_dataset_permission(dataset, current_user, session)
DatasetService.check_dataset_permission(dataset, current_user, db.session())
metadata = MetadataService.create_metadata(
dataset_id_str, metadata_args, current_user, current_tenant_id, session=session
dataset_id_str, metadata_args, current_user, current_tenant_id, session=db.session()
)
return dump_response(DatasetMetadataResponse, metadata), 201
@@ -81,13 +79,12 @@ class DatasetMetadataCreateApi(Resource):
200, "Metadata retrieved successfully", console_ns.models[DatasetMetadataListResponse.__name__]
)
@rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_CREATE_AND_MANAGEMENT)
@with_session(write=False)
def get(self, session: Session, dataset_id: UUID):
def get(self, dataset_id: UUID):
dataset_id_str = str(dataset_id)
dataset = DatasetService.get_dataset(dataset_id_str, session)
dataset = DatasetService.get_dataset(dataset_id_str, db.session())
if dataset is None:
raise NotFound("Dataset not found.")
metadata = MetadataService.get_dataset_metadatas(dataset, session)
metadata = MetadataService.get_dataset_metadatas(dataset, session=db.session())
return dump_response(DatasetMetadataListResponse, metadata), 200
@@ -102,27 +99,19 @@ class DatasetMetadataApi(Resource):
@with_current_user
@with_current_tenant_id
@rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_EDIT)
@with_session
def patch(
self,
session: Session,
current_tenant_id: str,
current_user: Account,
dataset_id: UUID,
metadata_id: UUID,
):
def patch(self, current_tenant_id: str, current_user: Account, dataset_id: UUID, metadata_id: UUID):
payload = MetadataUpdatePayload.model_validate(console_ns.payload or {})
name = payload.name
dataset_id_str = str(dataset_id)
metadata_id_str = str(metadata_id)
dataset = DatasetService.get_dataset(dataset_id_str, session)
dataset = DatasetService.get_dataset(dataset_id_str, db.session())
if dataset is None:
raise NotFound("Dataset not found.")
DatasetService.check_dataset_permission(dataset, current_user, session)
DatasetService.check_dataset_permission(dataset, current_user, db.session())
metadata = MetadataService.update_metadata_name(
dataset_id_str, metadata_id_str, name, current_user, current_tenant_id, session=session
dataset_id_str, metadata_id_str, name, current_user, current_tenant_id, session=db.session()
)
return dump_response(DatasetMetadataResponse, metadata), 200
@@ -133,16 +122,15 @@ class DatasetMetadataApi(Resource):
@console_ns.response(204, "Metadata deleted successfully")
@with_current_user
@rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_EDIT)
@with_session
def delete(self, session: Session, current_user: Account, dataset_id: UUID, metadata_id: UUID):
def delete(self, current_user: Account, dataset_id: UUID, metadata_id: UUID):
dataset_id_str = str(dataset_id)
metadata_id_str = str(metadata_id)
dataset = DatasetService.get_dataset(dataset_id_str, session)
dataset = DatasetService.get_dataset(dataset_id_str, db.session())
if dataset is None:
raise NotFound("Dataset not found.")
DatasetService.check_dataset_permission(dataset, current_user, session)
DatasetService.check_dataset_permission(dataset, current_user, db.session())
MetadataService.delete_metadata(dataset_id_str, metadata_id_str, session)
MetadataService.delete_metadata(dataset_id_str, metadata_id_str, session=db.session())
# Frontend callers only await success and invalidate metadata caches; no response body is consumed.
return "", 204
@@ -172,19 +160,18 @@ class DatasetMetadataBuiltInFieldActionApi(Resource):
@console_ns.response(204, "Action completed successfully")
@with_current_user
@rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_EDIT)
@with_session
def post(self, session: Session, current_user: Account, dataset_id: UUID, action: Literal["enable", "disable"]):
def post(self, current_user: Account, dataset_id: UUID, action: Literal["enable", "disable"]):
dataset_id_str = str(dataset_id)
dataset = DatasetService.get_dataset(dataset_id_str, session)
dataset = DatasetService.get_dataset(dataset_id_str, db.session())
if dataset is None:
raise NotFound("Dataset not found.")
DatasetService.check_dataset_permission(dataset, current_user, session)
DatasetService.check_dataset_permission(dataset, current_user, db.session())
match action:
case "enable":
MetadataService.enable_built_in_field(dataset, session)
MetadataService.enable_built_in_field(dataset, session=db.session())
case "disable":
MetadataService.disable_built_in_field(dataset, session)
MetadataService.disable_built_in_field(dataset, session=db.session())
# Frontend callers only await success and invalidate metadata caches; no response body is consumed.
return "", 204
@@ -202,17 +189,16 @@ class DocumentMetadataEditApi(Resource):
)
@with_current_user
@rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_EDIT)
@with_session
def post(self, session: Session, current_user: Account, dataset_id: UUID):
def post(self, current_user: Account, dataset_id: UUID):
dataset_id_str = str(dataset_id)
dataset = DatasetService.get_dataset(dataset_id_str, session)
dataset = DatasetService.get_dataset(dataset_id_str, db.session())
if dataset is None:
raise NotFound("Dataset not found.")
DatasetService.check_dataset_permission(dataset, current_user, session)
DatasetService.check_dataset_permission(dataset, current_user, db.session())
metadata_args = MetadataOperationData.model_validate(console_ns.payload or {})
MetadataService.update_documents_metadata(dataset, metadata_args, current_user, session=session)
MetadataService.update_documents_metadata(dataset, metadata_args, current_user, session=db.session())
# Frontend callers only await success and invalidate caches; no response body is consumed.
return "", 204
@@ -27,7 +27,7 @@ from controllers.console.app.workflow import (
WorkflowResponse,
)
from controllers.console.app.wraps import with_session
from controllers.console.datasets.wraps import get_rag_pipeline, load_rag_pipeline
from controllers.console.datasets.wraps import get_rag_pipeline
from controllers.console.wraps import (
RBACPermission,
RBACResourceScope,
@@ -344,11 +344,11 @@ class DraftRagPipelineRunApi(Resource):
@rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_EDIT)
@with_current_user
@with_session
def post(self, session: Session, current_user: Account, pipeline_id: UUID):
@get_rag_pipeline
def post(self, session: Session, current_user: Account, pipeline: Pipeline):
"""
Run draft workflow
"""
pipeline = load_rag_pipeline(session, str(pipeline_id))
payload = DraftWorkflowRunPayload.model_validate(console_ns.payload or {})
args = payload.model_dump()
@@ -378,11 +378,11 @@ class PublishedRagPipelineRunApi(Resource):
@rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_EDIT)
@with_current_user
@with_session
def post(self, session: Session, current_user: Account, pipeline_id: UUID):
@get_rag_pipeline
def post(self, session: Session, current_user: Account, pipeline: Pipeline):
"""
Run published workflow
"""
pipeline = load_rag_pipeline(session, str(pipeline_id))
payload = PublishedWorkflowRunPayload.model_validate(console_ns.payload or {})
args = payload.model_dump(exclude_none=True)
streaming = payload.response_mode == "streaming"
+13 -10
View File
@@ -1,21 +1,13 @@
from collections.abc import Callable
from functools import wraps
from sqlalchemy import select
from sqlalchemy.orm import Session
from controllers.console.datasets.error import PipelineNotFoundError
from extensions.ext_database import db
from libs.login import current_account_with_tenant
from models.dataset import Pipeline
from services.rag_pipeline.rag_pipeline import RagPipelineService
def load_rag_pipeline(session: Session, pipeline_id: str) -> Pipeline:
_, current_tenant_id = current_account_with_tenant()
pipeline = RagPipelineService.get_pipeline_by_id(pipeline_id, current_tenant_id, session=session)
if not pipeline:
raise PipelineNotFoundError()
return pipeline
def get_rag_pipeline[**P, R](view_func: Callable[P, R]) -> Callable[P, R]:
@@ -24,11 +16,22 @@ def get_rag_pipeline[**P, R](view_func: Callable[P, R]) -> Callable[P, R]:
if not kwargs.get("pipeline_id"):
raise ValueError("missing pipeline_id in path parameters")
_, current_tenant_id = current_account_with_tenant()
pipeline_id = kwargs.get("pipeline_id")
pipeline_id = str(pipeline_id)
del kwargs["pipeline_id"]
kwargs["pipeline"] = load_rag_pipeline(db.session(), pipeline_id)
stmt = select(Pipeline).where(Pipeline.id == pipeline_id, Pipeline.tenant_id == current_tenant_id).limit(1)
# Migrated handlers pass the request Session as args[1]; legacy handlers still use db.session.
session = args[1] if len(args) > 1 and isinstance(args[1], Session) else db.session
pipeline = session.scalar(stmt)
if not pipeline:
raise PipelineNotFoundError()
kwargs["pipeline"] = pipeline
return view_func(*args, **kwargs)
+3 -8
View File
@@ -50,19 +50,14 @@ register_response_schema_models(console_ns, AudioBinaryResponse, AudioTranscript
class ChatAudioApi(InstalledAppResource):
@console_ns.response(200, "Success", console_ns.models[AudioTranscriptResponse.__name__])
def post(self, installed_app: InstalledApp):
app_model = installed_app.app_with_session(session=db.session())
app_model = installed_app.app
if app_model is None:
raise AppUnavailableError()
file = request.files["file"]
try:
response = AudioService.transcript_asr(
app_model=app_model,
file=file,
session=db.session(),
end_user=None,
)
response = AudioService.transcript_asr(app_model=app_model, file=file, end_user=None)
return response
except services.errors.app_model_config.AppModelConfigBrokenError:
@@ -101,7 +96,7 @@ class ChatTextApi(InstalledAppResource):
@console_ns.expect(console_ns.models[TextToAudioPayload.__name__])
@console_ns.response(200, "Success", console_ns.models[AudioBinaryResponse.__name__])
def post(self, installed_app: InstalledApp):
app_model = installed_app.app_with_session(session=db.session())
app_model = installed_app.app
if app_model is None:
raise AppUnavailableError()
try:
@@ -90,7 +90,7 @@ class CompletionApi(InstalledAppResource):
@with_current_user
@with_session
def post(self, session: Session, current_user: Account, installed_app: InstalledApp):
app_model = installed_app.app_with_session(session=session)
app_model = installed_app.app
if app_model is None:
raise AppUnavailableError()
if app_model.mode != AppMode.COMPLETION:
@@ -146,9 +146,8 @@ class CompletionApi(InstalledAppResource):
class CompletionStopApi(InstalledAppResource):
@console_ns.response(200, "Success", console_ns.models[SimpleResultResponse.__name__])
@with_current_user_id
@with_session(write=False)
def post(self, session: Session, current_user_id: str, installed_app: InstalledApp, task_id: str):
app_model = installed_app.app_with_session(session=session)
def post(self, current_user_id: str, installed_app: InstalledApp, task_id: str):
app_model = installed_app.app
if app_model is None:
raise AppUnavailableError()
if app_model.mode != AppMode.COMPLETION:
@@ -174,7 +173,7 @@ class ChatApi(InstalledAppResource):
@with_current_user
@with_session
def post(self, session: Session, current_user: Account, installed_app: InstalledApp):
app_model = installed_app.app_with_session(session=session)
app_model = installed_app.app
if app_model is None:
raise AppUnavailableError()
app_mode = AppMode.value_of(app_model.mode)
@@ -196,7 +195,7 @@ class ChatApi(InstalledAppResource):
app_model=app_model,
conversation_id=payload.conversation_id,
user=current_user,
session=session,
session=db.session(),
)
response = AppGenerateService.generate(
@@ -241,9 +240,8 @@ class ChatApi(InstalledAppResource):
class ChatStopApi(InstalledAppResource):
@console_ns.response(200, "Success", console_ns.models[SimpleResultResponse.__name__])
@with_current_user_id
@with_session(write=False)
def post(self, session: Session, current_user_id: str, installed_app: InstalledApp, task_id: str):
app_model = installed_app.app_with_session(session=session)
def post(self, current_user_id: str, installed_app: InstalledApp, task_id: str):
app_model = installed_app.app
if app_model is None:
raise AppUnavailableError()
app_mode = AppMode.value_of(app_model.mode)
@@ -16,7 +16,6 @@ from core.app.entities.app_invoke_entities import InvokeFrom
from extensions.ext_database import db
from fields.conversation_fields import (
ConversationInfiniteScrollPagination,
ConversationResponseSource,
ResultResponse,
SimpleConversation,
)
@@ -54,7 +53,7 @@ class ConversationListApi(InstalledAppResource):
@console_ns.response(200, "Success", console_ns.models[ConversationInfiniteScrollPagination.__name__])
@with_current_user
def get(self, current_user: Account, installed_app: InstalledApp):
app_model = installed_app.app_with_session(session=db.session())
app_model = installed_app.app
if app_model is None:
raise AppUnavailableError()
app_mode = AppMode.value_of(app_model.mode)
@@ -85,13 +84,7 @@ class ConversationListApi(InstalledAppResource):
pinned=args.pinned,
)
adapter = TypeAdapter(SimpleConversation)
conversations = [
adapter.validate_python(
ConversationResponseSource(item, session=session),
from_attributes=True,
)
for item in pagination.data
]
conversations = [adapter.validate_python(item, from_attributes=True) for item in pagination.data]
return ConversationInfiniteScrollPagination(
limit=pagination.limit,
has_more=pagination.has_more,
@@ -109,7 +102,7 @@ class ConversationApi(InstalledAppResource):
@console_ns.response(204, "Conversation deleted successfully")
@with_current_user
def delete(self, current_user: Account, installed_app: InstalledApp, c_id: UUID):
app_model = installed_app.app_with_session(session=db.session())
app_model = installed_app.app
if app_model is None:
raise AppUnavailableError()
app_mode = AppMode.value_of(app_model.mode)
@@ -134,7 +127,7 @@ class ConversationRenameApi(InstalledAppResource):
@console_ns.response(200, "Conversation renamed successfully", console_ns.models[SimpleConversation.__name__])
@with_current_user
def post(self, current_user: Account, installed_app: InstalledApp, c_id: UUID):
app_model = installed_app.app_with_session(session=db.session())
app_model = installed_app.app
if app_model is None:
raise AppUnavailableError()
app_mode = AppMode.value_of(app_model.mode)
@@ -146,13 +139,12 @@ class ConversationRenameApi(InstalledAppResource):
payload = ConversationRenamePayload.model_validate(console_ns.payload or {})
try:
session = db.session()
conversation = ConversationService.rename(
app_model, conversation_id, current_user, payload.name, payload.auto_generate, session=session
app_model, conversation_id, current_user, payload.name, payload.auto_generate, session=db.session()
)
return (
TypeAdapter(SimpleConversation)
.validate_python(ConversationResponseSource(conversation, session=session), from_attributes=True)
.validate_python(conversation, from_attributes=True)
.model_dump(mode="json")
)
except ConversationNotExistsError:
@@ -167,7 +159,7 @@ class ConversationPinApi(InstalledAppResource):
@console_ns.response(200, "Success", console_ns.models[ResultResponse.__name__])
@with_current_user
def patch(self, current_user: Account, installed_app: InstalledApp, c_id: UUID):
app_model = installed_app.app_with_session(session=db.session())
app_model = installed_app.app
if app_model is None:
raise AppUnavailableError()
app_mode = AppMode.value_of(app_model.mode)
@@ -192,7 +184,7 @@ class ConversationUnPinApi(InstalledAppResource):
@console_ns.response(200, "Success", console_ns.models[ResultResponse.__name__])
@with_current_user
def patch(self, current_user: Account, installed_app: InstalledApp, c_id: UUID):
app_model = installed_app.app_with_session(session=db.session())
app_model = installed_app.app
if app_model is None:
raise AppUnavailableError()
app_mode = AppMode.value_of(app_model.mode)
+7 -11
View File
@@ -28,7 +28,7 @@ from controllers.console.wraps import with_current_user
from core.app.entities.app_invoke_entities import InvokeFrom
from core.errors.error import ModelCurrentlyNotSupportError, ProviderTokenNotInitError, QuotaExceededError
from extensions.ext_database import db
from fields.conversation_fields import MessageResponseSource, ResultResponse
from fields.conversation_fields import ResultResponse
from fields.message_fields import (
ExploreMessageInfiniteScrollPagination,
ExploreMessageListItem,
@@ -76,8 +76,7 @@ class MessageListApi(InstalledAppResource):
@console_ns.response(200, "Success", console_ns.models[ExploreMessageInfiniteScrollPagination.__name__])
@with_current_user
def get(self, current_user: Account, installed_app: InstalledApp):
session = db.session()
app_model = installed_app.app_with_session(session=session)
app_model = installed_app.app
if app_model is None:
raise AppUnavailableError()
@@ -93,13 +92,10 @@ class MessageListApi(InstalledAppResource):
args.conversation_id,
args.first_id or None,
args.limit,
session=session,
session=db.session(),
)
adapter = TypeAdapter(ExploreMessageListItem)
items = [
adapter.validate_python(MessageResponseSource(message, session=session), from_attributes=True)
for message in pagination.data
]
items = [adapter.validate_python(message, from_attributes=True) for message in pagination.data]
return ExploreMessageInfiniteScrollPagination(
limit=pagination.limit,
has_more=pagination.has_more,
@@ -120,7 +116,7 @@ class MessageFeedbackApi(InstalledAppResource):
@console_ns.response(200, "Feedback submitted successfully", console_ns.models[ResultResponse.__name__])
@with_current_user
def post(self, current_user: Account, installed_app: InstalledApp, message_id: UUID):
app_model = installed_app.app_with_session(session=db.session())
app_model = installed_app.app
if app_model is None:
raise AppUnavailableError()
@@ -153,7 +149,7 @@ class MessageMoreLikeThisApi(InstalledAppResource):
@with_current_user
@with_session
def get(self, session: Session, current_user: Account, installed_app: InstalledApp, message_id: UUID):
app_model = installed_app.app_with_session(session=session)
app_model = installed_app.app
if app_model is None:
raise AppUnavailableError()
if app_model.mode != "completion":
@@ -203,7 +199,7 @@ class MessageSuggestedQuestionApi(InstalledAppResource):
@console_ns.response(200, "Success", console_ns.models[SuggestedQuestionsResponse.__name__])
@with_current_user
def get(self, current_user: Account, installed_app: InstalledApp, message_id: UUID):
app_model = installed_app.app_with_session(session=db.session())
app_model = installed_app.app
if app_model is None:
raise AppUnavailableError()
app_mode = AppMode.value_of(app_model.mode)
+6 -11
View File
@@ -9,7 +9,7 @@ from controllers.console.app.error import AppUnavailableError
from controllers.console.explore.wraps import InstalledAppResource
from core.app.app_config.common.parameters_mapping import get_parameters_from_feature_dict
from extensions.ext_database import db
from models.model import AppMode, InstalledApp, load_annotation_reply_config
from models.model import AppMode, InstalledApp
from services.app_service import AppService
@@ -32,29 +32,24 @@ class AppParameterApi(InstalledAppResource):
@console_ns.response(200, "Success", console_ns.models[fields.Parameters.__name__])
def get(self, installed_app: InstalledApp):
"""Retrieve app parameters."""
session = db.session()
app_model = installed_app.app_with_session(session=session)
app_model = installed_app.app
if app_model is None:
raise AppUnavailableError()
if app_model.mode in {AppMode.ADVANCED_CHAT, AppMode.WORKFLOW}:
workflow = app_model.workflow_with_session(session=session)
workflow = app_model.workflow
if workflow is None:
raise AppUnavailableError()
features_dict: dict[str, Any] = workflow.features_dict
user_input_form = workflow.user_input_form(to_old_structure=True)
else:
app_model_config = app_model.app_model_config_with_session(session=session)
app_model_config = app_model.app_model_config
if app_model_config is None:
raise AppUnavailableError()
annotation_reply = load_annotation_reply_config(session, app_model.id)
features_dict = cast(
dict[str, Any],
app_model_config.to_dict(annotation_reply=annotation_reply),
)
features_dict = cast(dict[str, Any], app_model_config.to_dict())
user_input_form = features_dict.get("user_input_form", [])
@@ -67,7 +62,7 @@ class ExploreAppMetaApi(InstalledAppResource):
@console_ns.response(200, "Success", console_ns.models[ExploreAppMetaResponse.__name__])
def get(self, installed_app: InstalledApp):
"""Get app meta"""
app_model = installed_app.app_with_session(session=db.session())
app_model = installed_app.app
if not app_model:
raise ValueError("App not found")
return AppService().get_app_meta(app_model, session=db.session())
@@ -12,7 +12,7 @@ from controllers.console.explore.error import NotCompletionAppError
from controllers.console.explore.wraps import InstalledAppResource
from controllers.console.wraps import with_current_user
from extensions.ext_database import db
from fields.conversation_fields import MessageResponseSource, ResultResponse
from fields.conversation_fields import ResultResponse
from fields.message_fields import SavedMessageInfiniteScrollPagination, SavedMessageItem
from models import Account
from models.model import InstalledApp
@@ -29,8 +29,7 @@ class SavedMessageListApi(InstalledAppResource):
@console_ns.response(200, "Success", console_ns.models[SavedMessageInfiniteScrollPagination.__name__])
@with_current_user
def get(self, current_user: Account, installed_app: InstalledApp):
session = db.session()
app_model = installed_app.app_with_session(session=session)
app_model = installed_app.app
if app_model is None:
raise AppUnavailableError()
if app_model.mode != "completion":
@@ -39,13 +38,10 @@ class SavedMessageListApi(InstalledAppResource):
args = SavedMessageListQuery.model_validate(request.args.to_dict())
pagination = SavedMessageService.pagination_by_last_id(
app_model, current_user, str(args.last_id) if args.last_id else None, args.limit, session=session
app_model, current_user, str(args.last_id) if args.last_id else None, args.limit, session=db.session()
)
adapter = TypeAdapter(SavedMessageItem)
items = [
adapter.validate_python(MessageResponseSource(message, session=session), from_attributes=True)
for message in pagination.data
]
items = [adapter.validate_python(message, from_attributes=True) for message in pagination.data]
return SavedMessageInfiniteScrollPagination(
limit=pagination.limit,
has_more=pagination.has_more,
@@ -56,7 +52,7 @@ class SavedMessageListApi(InstalledAppResource):
@console_ns.response(200, "Success", console_ns.models[ResultResponse.__name__])
@with_current_user
def post(self, current_user: Account, installed_app: InstalledApp):
app_model = installed_app.app_with_session(session=db.session())
app_model = installed_app.app
if app_model is None:
raise AppUnavailableError()
if app_model.mode != "completion":
@@ -79,7 +75,7 @@ class SavedMessageApi(InstalledAppResource):
@console_ns.response(204, "Saved message deleted successfully")
@with_current_user
def delete(self, current_user: Account, installed_app: InstalledApp, message_id: UUID):
app_model = installed_app.app_with_session(session=db.session())
app_model = installed_app.app
if app_model is None:
raise AppUnavailableError()
+23 -112
View File
@@ -1,6 +1,4 @@
import logging
from collections.abc import Mapping
from dataclasses import dataclass
from datetime import datetime
from typing import Any, Literal
@@ -47,9 +45,7 @@ from controllers.console.explore.error import (
NotWorkflowAppError,
)
from controllers.console.explore.wraps import TrialAppResource, trial_feature_enable
from controllers.console.files import FILE_UPLOAD_PARAMS, upload_file_from_request
from controllers.console.remote_files import RemoteFileUploadPayload, upload_remote_file_from_request
from controllers.console.wraps import cloud_edition_billing_resource_check, with_current_user
from controllers.console.wraps import with_current_user
from controllers.web.error import InvokeRateLimitError as InvokeRateLimitHttpError
from core.app.app_config.common.parameters_mapping import get_parameters_from_feature_dict
from core.app.apps.base_app_queue_manager import AppQueueManager
@@ -62,21 +58,18 @@ from core.errors.error import (
from extensions.ext_database import db
from extensions.ext_redis import redis_client
from fields.base import ResponseModel
from fields.conversation_variable_fields import WorkflowConversationVariableResponse
from fields.file_fields import FileResponse, FileWithSignedUrl
from fields.message_fields import SuggestedQuestionsResponse
from graphon.graph_engine.manager import GraphEngineManager
from graphon.model_runtime.errors.invoke import InvokeError
from libs import helper
from libs.helper import dump_response, to_timestamp, uuid_value
from models import Account, App
from models import Account
from models.account import TenantStatus
from models.model import AppMode, Site, load_annotation_reply_config
from models.model import AppMode, Site
from models.workflow import Workflow
from services.account_service import TenantService
from services.app_generate_service import AppGenerateService
from services.app_ref_service import AppRefService
from services.app_service import AppResponseView, AppService
from services.app_service import AppService
from services.audio_service import AudioService
from services.dataset_service import DatasetService
from services.errors.audio import (
@@ -377,7 +370,7 @@ class TrialWorkflowResponse(ResponseModel):
updated_at: int | None = None
tool_published: bool | None = None
environment_variables: list[JsonObject] = Field(default_factory=list)
conversation_variables: list[WorkflowConversationVariableResponse] = Field(default_factory=list)
conversation_variables: list[JsonObject] = Field(default_factory=list)
rag_pipeline_variables: list[JsonObject] = Field(default_factory=list)
@field_validator("created_at", "updated_at", mode="before")
@@ -386,27 +379,6 @@ class TrialWorkflowResponse(ResponseModel):
return to_timestamp(value)
@dataclass(frozen=True)
class TrialWorkflowResponseSource:
workflow: Workflow
session: Session
@property
def created_by_account(self) -> Account | None:
return self.workflow.get_created_by_account(session=self.session)
@property
def updated_by_account(self) -> Account | None:
return self.workflow.get_updated_by_account(session=self.session)
@property
def tool_published(self) -> bool:
return self.workflow.get_tool_published(session=self.session)
def __getattr__(self, name: str) -> Any:
return getattr(self.workflow, name) # noqa: no-new-getattr response adapter delegates model fields
register_schema_models(
console_ns,
WorkflowRunRequest,
@@ -431,36 +403,6 @@ register_response_schema_models(
simple_account_model = console_ns.models[TrialSimpleAccount.__name__]
class TrialAppFileUploadApi(TrialAppResource):
@trial_feature_enable
@cloud_edition_billing_resource_check("documents")
@console_ns.doc(consumes=["multipart/form-data"], params=FILE_UPLOAD_PARAMS)
@console_ns.response(201, "File uploaded successfully", console_ns.models[FileResponse.__name__])
@with_current_user
def post(self, current_user: Account, app_model: App):
"""Upload a file into the tenant that owns the trial app."""
upload_file = upload_file_from_request(
current_user=current_user,
resource_tenant_id=app_model.tenant_id,
)
return dump_response(FileResponse, upload_file), 201
class TrialAppRemoteFileUploadApi(TrialAppResource):
@trial_feature_enable
@cloud_edition_billing_resource_check("documents")
@console_ns.expect(console_ns.models[RemoteFileUploadPayload.__name__])
@console_ns.response(201, "File uploaded successfully", console_ns.models[FileWithSignedUrl.__name__])
@with_current_user
def post(self, current_user: Account, app_model: App):
"""Upload a remote file into the tenant that owns the trial app."""
remote_file = upload_remote_file_from_request(
current_user=current_user,
resource_tenant_id=app_model.tenant_id,
)
return remote_file.model_dump(mode="json"), 201
class TrialAppWorkflowRunApi(TrialAppResource):
@trial_feature_enable
@console_ns.expect(console_ns.models[WorkflowRunRequest.__name__])
@@ -645,19 +587,14 @@ class TrialChatAudioApi(TrialAppResource):
def post(self, current_user: Account, trial_app):
app_model = trial_app
file = request.files.get("file")
file = request.files["file"]
try:
# Get IDs before they might be detached from session
app_id = app_model.id
user_id = current_user.id
response = AudioService.transcript_asr(
app_model=app_model,
file=file,
session=db.session(),
end_user=None,
)
response = AudioService.transcript_asr(app_model=app_model, file=file, end_user=None)
RecommendedAppService.add_trial_app_record(app_id, user_id, session=db.session())
return response
except services.errors.app_model_config.AppModelConfigBrokenError:
@@ -809,21 +746,19 @@ class TrialSitApi(Resource):
"""Resource for trial app sites."""
@console_ns.response(200, "Success", console_ns.models[SiteResponse.__name__])
@with_session(write=False)
@get_app_model_with_trial(None)
def get(self, session: Session, app_model):
def get(self, app_model):
"""Retrieve app site info.
Returns the site configuration for the application including theme, icons, and text.
"""
site = session.scalar(select(Site).where(Site.app_id == app_model.id).limit(1))
site = db.session.scalar(select(Site).where(Site.app_id == app_model.id).limit(1))
if not site:
raise Forbidden()
tenant = TenantService.get_tenant_by_id(app_model.tenant_id, session=session)
assert tenant
if tenant.status == TenantStatus.ARCHIVE:
assert app_model.tenant
if app_model.tenant.status == TenantStatus.ARCHIVE:
raise Forbidden()
return SiteResponse.model_validate(site).model_dump(mode="json")
@@ -833,29 +768,26 @@ class TrialAppParameterApi(Resource):
"""Resource for app variables."""
@console_ns.response(200, "Success", console_ns.models[ParametersResponse.__name__])
@with_session(write=False)
@get_app_model_with_trial(None)
def get(self, session: Session, app_model):
def get(self, app_model):
"""Retrieve app parameters."""
if app_model is None:
raise AppUnavailableError()
features_dict: Mapping[str, Any]
if app_model.mode in {AppMode.ADVANCED_CHAT, AppMode.WORKFLOW}:
workflow = app_model.workflow_with_session(session=session)
workflow = app_model.workflow
if workflow is None:
raise AppUnavailableError()
features_dict = workflow.features_dict
user_input_form = workflow.user_input_form(to_old_structure=True)
else:
app_model_config = app_model.app_model_config_with_session(session=session)
app_model_config = app_model.app_model_config
if app_model_config is None:
raise AppUnavailableError()
annotation_reply = load_annotation_reply_config(session, app_model_config.app_id)
features_dict = app_model_config.to_dict(annotation_reply=annotation_reply)
features_dict = app_model_config.to_dict()
user_input_form = features_dict.get("user_input_form", [])
@@ -865,52 +797,43 @@ class TrialAppParameterApi(Resource):
class AppApi(Resource):
@console_ns.response(200, "Success", console_ns.models[TrialAppDetailResponse.__name__])
@with_session(write=False)
@get_app_model_with_trial(None)
def get(self, session: Session, app_model):
def get(self, app_model):
"""Get app detail"""
app_service = AppService()
app_model = app_service.get_app(app_model, session=session)
app_model = app_service.get_app(app_model)
return TrialAppDetailResponse.model_validate(
AppResponseView(app_model, session=session),
from_attributes=True,
).model_dump(mode="json")
return dump_response(TrialAppDetailResponse, app_model)
class AppWorkflowApi(Resource):
@console_ns.response(200, "Success", console_ns.models[TrialWorkflowResponse.__name__])
@with_session(write=False)
@get_app_model_with_trial(None)
def get(self, session: Session, app_model):
def get(self, app_model):
"""Get workflow detail"""
if not app_model.workflow_id:
raise AppUnavailableError()
workflow = app_model.workflow_with_session(session=session)
workflow = db.session.get(Workflow, app_model.workflow_id)
if workflow is None:
raise AppUnavailableError()
return TrialWorkflowResponse.model_validate(
TrialWorkflowResponseSource(workflow=workflow, session=session),
from_attributes=True,
).model_dump(mode="json")
return dump_response(TrialWorkflowResponse, workflow)
class DatasetListApi(Resource):
@console_ns.doc(params=query_params_from_model(TrialDatasetListQuery))
@console_ns.response(200, "Success", console_ns.models[TrialDatasetListResponse.__name__])
@with_session(write=False)
@get_app_model_with_trial(None)
def get(self, session: Session, app_model):
def get(self, app_model):
page = request.args.get("page", default=1, type=int)
limit = request.args.get("limit", default=20, type=int)
ids = request.args.getlist("ids")
tenant_id = app_model.tenant_id
if ids:
datasets, total = DatasetService.get_datasets_by_ids(ids, tenant_id, session=session)
datasets, total = DatasetService.get_datasets_by_ids(ids, tenant_id)
else:
raise NeedAddIdsError()
@@ -920,18 +843,6 @@ class DatasetListApi(Resource):
console_ns.add_resource(TrialChatApi, "/trial-apps/<uuid:app_id>/chat-messages", endpoint="trial_app_chat_completion")
console_ns.add_resource(
TrialAppFileUploadApi,
"/trial-apps/<uuid:app_id>/files/upload",
endpoint="trial_app_file_upload",
)
console_ns.add_resource(
TrialAppRemoteFileUploadApi,
"/trial-apps/<uuid:app_id>/remote-files/upload",
endpoint="trial_app_remote_file_upload",
)
console_ns.add_resource(
TrialMessageSuggestedQuestionApi,
"/trial-apps/<uuid:app_id>/messages/<uuid:message_id>/suggested-questions",
+3 -4
View File
@@ -51,7 +51,7 @@ class InstalledAppWorkflowRunApi(InstalledAppResource):
"""
Run workflow
"""
app_model = installed_app.app_with_session(session=session)
app_model = installed_app.app
if not app_model:
raise NotWorkflowAppError()
app_mode = AppMode.value_of(app_model.mode)
@@ -92,12 +92,11 @@ class InstalledAppWorkflowRunApi(InstalledAppResource):
@console_ns.route("/installed-apps/<uuid:installed_app_id>/workflows/tasks/<string:task_id>/stop")
class InstalledAppWorkflowTaskStopApi(InstalledAppResource):
@console_ns.response(200, "Success", console_ns.models[SimpleResultResponse.__name__])
@with_session(write=False)
def post(self, session: Session, installed_app: InstalledApp, task_id: str):
def post(self, installed_app: InstalledApp, task_id: str):
"""
Stop workflow task
"""
app_model = installed_app.app_with_session(session=session)
app_model = installed_app.app
if not app_model:
raise NotWorkflowAppError()
app_mode = AppMode.value_of(app_model.mode)
+4 -5
View File
@@ -30,7 +30,7 @@ def installed_app_required[**P, R](view: Callable[Concatenate[InstalledApp, P],
if installed_app is None:
raise NotFound("Installed app not found")
if not installed_app.app_with_session(session=db.session()):
if not installed_app.app:
db.session.delete(installed_app)
db.session.commit()
@@ -74,18 +74,17 @@ def trial_app_required[**P, R](view: Callable[Concatenate[App, P], R] | None = N
@wraps(view)
def decorated(app_id: str, *args: P.args, **kwargs: P.kwargs):
current_user, _ = current_account_with_tenant()
session = db.session()
trial_app = session.scalar(select(TrialApp).where(TrialApp.app_id == str(app_id)).limit(1))
trial_app = db.session.scalar(select(TrialApp).where(TrialApp.app_id == str(app_id)).limit(1))
if trial_app is None:
raise TrialAppNotAllowed()
app = trial_app.app_with_session(session=session)
app = trial_app.app
if app is None:
raise TrialAppNotAllowed()
account_trial_app_record = session.scalar(
account_trial_app_record = db.session.scalar(
select(AccountTrialAppRecord)
.where(AccountTrialAppRecord.account_id == current_user.id, AccountTrialAppRecord.app_id == app_id)
.limit(1)
+35 -41
View File
@@ -29,7 +29,7 @@ from extensions.ext_database import db
from fields.file_fields import FileResponse, UploadConfig
from libs.helper import dump_response
from libs.login import login_required
from models import Account, UploadFile
from models import Account
from services.file_service import FileService
from . import console_ns
@@ -39,7 +39,7 @@ register_response_schema_models(console_ns, AllowedExtensionsResponse, TextConte
PREVIEW_WORDS_LIMIT = 3000
FILE_UPLOAD_PARAMS = {
_FILE_UPLOAD_PARAMS = {
"file": {
"description": "File to upload",
"in": "formData",
@@ -56,43 +56,6 @@ FILE_UPLOAD_PARAMS = {
}
def upload_file_from_request(*, current_user: Account, resource_tenant_id: str | None = None) -> UploadFile:
"""Validate the multipart request and persist the file under the requested resource tenant."""
source_str = request.form.get("source")
source: Literal["datasets"] | None = "datasets" if source_str == "datasets" else None
if "file" not in request.files:
raise NoFileUploadedError()
if len(request.files) > 1:
raise TooManyFilesError()
file = request.files["file"]
if not file.filename:
raise FilenameNotExistsError
if source == "datasets" and not current_user.is_dataset_editor:
raise Forbidden()
if source not in ("datasets", None):
source = None
try:
return FileService(db.engine).upload_file(
filename=file.filename,
content=file.stream.read(),
mimetype=file.mimetype,
user=current_user,
tenant_id=resource_tenant_id,
source=source,
)
except services.errors.file.FileTooLargeError as file_too_large_error:
raise FileTooLargeError(file_too_large_error.description)
except services.errors.file.UnsupportedFileTypeError:
raise UnsupportedFileTypeError()
except services.errors.file.BlockedFileExtensionError as blocked_extension_error:
raise BlockedFileExtensionError(blocked_extension_error.description)
@console_ns.route("/files/upload")
class FileApi(Resource):
@setup_required
@@ -118,11 +81,42 @@ class FileApi(Resource):
@login_required
@account_initialization_required
@cloud_edition_billing_resource_check("documents")
@console_ns.doc(consumes=["multipart/form-data"], params=FILE_UPLOAD_PARAMS)
@console_ns.doc(consumes=["multipart/form-data"], params=_FILE_UPLOAD_PARAMS)
@console_ns.response(201, "File uploaded successfully", console_ns.models[FileResponse.__name__])
@with_current_user
def post(self, current_user: Account):
upload_file = upload_file_from_request(current_user=current_user)
source_str = request.form.get("source")
source: Literal["datasets"] | None = "datasets" if source_str == "datasets" else None
if "file" not in request.files:
raise NoFileUploadedError()
if len(request.files) > 1:
raise TooManyFilesError()
file = request.files["file"]
if not file.filename:
raise FilenameNotExistsError
if source == "datasets" and not current_user.is_dataset_editor:
raise Forbidden()
if source not in ("datasets", None):
source = None
try:
upload_file = FileService(db.engine).upload_file(
filename=file.filename,
content=file.stream.read(),
mimetype=file.mimetype,
user=current_user,
source=source,
)
except services.errors.file.FileTooLargeError as file_too_large_error:
raise FileTooLargeError(file_too_large_error.description)
except services.errors.file.UnsupportedFileTypeError:
raise UnsupportedFileTypeError()
except services.errors.file.BlockedFileExtensionError as blocked_extension_error:
raise BlockedFileExtensionError(blocked_extension_error.description)
return dump_response(FileResponse, upload_file), 201
@@ -1,419 +0,0 @@
"""Authenticated transport adapter for the Console-to-KnowledgeFS proxy.
These raw Blueprint routes deliberately stay outside Dify's OpenAPI surface:
KnowledgeFS owns the wire contract consumed by the frontend. The catch-all path
avoids resource-specific Dify controllers, while the forwarding module consumes
only the operations explicitly enabled by Dify's product registry. The registry
can be validated explicitly against the pinned KnowledgeFS contract during development.
Console auth and contract-specific dataset RBAC run before forwarding. Request
bodies are capped at 64 MiB, JSON and binary responses have separate bounds,
SSE responses remain streaming with a bounded idle read timeout, and only safe
response headers are exposed. Operation-specific upstream error mappings are
applied before Console JSON error handling; the default maps 401 to 502 so it
cannot trigger browser-session recovery and preserves resource-level 403.
"""
from __future__ import annotations
import logging
from collections.abc import Callable, Iterator
from functools import wraps
from http import HTTPStatus
from typing import NoReturn, cast
import httpx
from flask import Response, request, stream_with_context
from flask.typing import ResponseReturnValue
from werkzeug.exceptions import (
BadGateway,
Forbidden,
GatewayTimeout,
HTTPException,
NotFound,
RequestEntityTooLarge,
ServiceUnavailable,
default_exceptions,
)
from configs import dify_config
from controllers.console import api, bp
from controllers.console.wraps import (
account_initialization_required,
cloud_edition_billing_rate_limit_check,
setup_required,
)
from core.helper import ssrf_proxy
from libs.login import current_account_with_tenant, login_required
from services.knowledge_fs_operations import KnowledgeFSMethod
from services.knowledge_fs_proxy import (
KnowledgeFSAccessDeniedError,
KnowledgeFSAuthorization,
KnowledgeFSConfigurationError,
KnowledgeFSRouteNotAllowedError,
KnowledgeFSTimeoutError,
KnowledgeFSTransportError,
KnowledgeFSUpstreamResponse,
authorize_knowledge_fs_request,
get_knowledge_fs_operation,
proxy_authorized_knowledge_fs_request,
proxy_knowledge_fs_request,
)
logger = logging.getLogger(__name__)
type _KnowledgeFSRequestForwarder = Callable[
[str | None, str | None, bytes | None, bytes | None],
KnowledgeFSUpstreamResponse,
]
_MAX_PROXY_BODY_BYTES = 64 * 1024 * 1024
_RESPONSE_HEADER_ALLOWLIST = (
"Cache-Control",
"Content-Disposition",
"Content-Type",
"Retry-After",
"X-Trace-Id",
)
_RESPONSE_HEADER_DENYLIST = frozenset(
{
"authorization",
"connection",
"cookie",
"keep-alive",
"proxy-authenticate",
"proxy-authorization",
"set-cookie",
"te",
"trailer",
"transfer-encoding",
"upgrade",
}
)
def _console_api_errors[**P](
view: Callable[P, ResponseReturnValue],
) -> Callable[P, ResponseReturnValue]:
"""Route raw Blueprint exceptions through the Console API JSON handlers."""
@wraps(view)
def decorated(*args: P.args, **kwargs: P.kwargs) -> ResponseReturnValue:
try:
return view(*args, **kwargs)
except Exception as exc:
return api.handle_error(exc)
return decorated
def _knowledge_fs_enabled[**P](
view: Callable[P, ResponseReturnValue],
) -> Callable[P, ResponseReturnValue]:
"""Hide the complete KnowledgeFS route surface while the bridge is disabled."""
@wraps(view)
def decorated(*args: P.args, **kwargs: P.kwargs) -> ResponseReturnValue:
if not dify_config.KNOWLEDGE_FS_ENABLED:
raise NotFound()
return view(*args, **kwargs)
return decorated
def _translate_proxy_error(exc: Exception, *, tenant_id: str) -> NoReturn:
"""Map forwarding failures to the stable Console HTTP error surface."""
if isinstance(exc, KnowledgeFSRouteNotAllowedError):
raise NotFound() from exc
if isinstance(exc, KnowledgeFSAccessDeniedError):
raise Forbidden() from exc
if isinstance(exc, KnowledgeFSConfigurationError):
logger.error("KnowledgeFS request was blocked by invalid configuration for tenant_id=%s", tenant_id)
raise ServiceUnavailable("KnowledgeFS integration is misconfigured") from exc
if isinstance(exc, KnowledgeFSTimeoutError):
raise GatewayTimeout("KnowledgeFS request timed out") from exc
if isinstance(exc, KnowledgeFSTransportError):
logger.warning("KnowledgeFS transport request failed for tenant_id=%s", tenant_id)
raise BadGateway("KnowledgeFS is unavailable") from exc
raise exc
def _knowledge_fs_operation_access_required(
view: Callable[[KnowledgeFSAuthorization], ResponseReturnValue],
) -> Callable[[KnowledgeFSMethod, str], ResponseReturnValue]:
"""Authorize one declared operation before billing and request-body work."""
@wraps(view)
def decorated(method: KnowledgeFSMethod, upstream_path: str) -> ResponseReturnValue:
current_user, tenant_id = current_account_with_tenant()
try:
authorization = authorize_knowledge_fs_request(
account=current_user,
tenant_id=tenant_id,
method=method,
path=upstream_path,
)
except KnowledgeFSRouteNotAllowedError as exc:
raise NotFound() from exc
except KnowledgeFSAccessDeniedError as exc:
_translate_proxy_error(exc, tenant_id=tenant_id)
return view(authorization)
return decorated
def _request_body() -> bytes:
"""Read the raw body up to the proxy limit or raise RequestEntityTooLarge."""
body = request.stream.read(_MAX_PROXY_BODY_BYTES + 1)
if len(body) > _MAX_PROXY_BODY_BYTES:
raise RequestEntityTooLarge("KnowledgeFS proxy request body is too large")
return body
def _stream_response_body(
upstream: httpx.Response,
*,
tenant_id: str,
max_response_bytes: int,
) -> Iterator[bytes]:
"""Yield one bounded SSE response and always release its pooled connection."""
total_bytes = 0
try:
for chunk in upstream.iter_bytes():
total_bytes += len(chunk)
if total_bytes > max_response_bytes:
logger.warning("KnowledgeFS stream exceeded the proxy limit for tenant_id=%s", tenant_id)
raise ssrf_proxy.ResponseTooLargeError(f"response exceeded {max_response_bytes} bytes")
yield chunk
finally:
upstream.close()
def _proxy_response(
upstream_result: KnowledgeFSUpstreamResponse,
*,
tenant_id: str,
contract_response_headers: tuple[str, ...],
max_response_bytes: int,
) -> Response:
"""Expose raw content, status, and allowlisted headers from KnowledgeFS.
Raises:
HTTPException: KnowledgeFS returns a status normalized by the operation contract.
"""
upstream = upstream_result.response
mapped_status = dict(upstream_result.operation.error_status_map).get(upstream.status_code)
if mapped_status is not None:
upstream.close()
description = "KnowledgeFS upstream request failed"
if upstream.status_code == HTTPStatus.UNAUTHORIZED:
description = "KnowledgeFS authentication failed"
logger.error(
"KnowledgeFS rejected the Dify server credential with HTTP %s for tenant_id=%s",
upstream.status_code,
tenant_id,
)
exception_type = default_exceptions.get(mapped_status)
if exception_type is None:
exception = HTTPException(description)
exception.code = mapped_status
raise exception
raise exception_type(description)
allowed_header_names = dict.fromkeys(
name.lower() for name in (*_RESPONSE_HEADER_ALLOWLIST, *contract_response_headers)
)
headers = {
name: value
for name in allowed_header_names
if name not in _RESPONSE_HEADER_DENYLIST
if (value := upstream.headers.get(name)) is not None
}
if upstream_result.response_kind == "stream":
response = Response(
stream_with_context( # pyrefly: ignore[no-matching-overload]
_stream_response_body(
upstream,
tenant_id=tenant_id,
max_response_bytes=max_response_bytes,
)
),
status=upstream.status_code,
headers=headers,
)
response.call_on_close(upstream.close)
return response
try:
content = upstream.content
finally:
upstream.close()
return Response(content, status=upstream.status_code, headers=headers)
def _proxy_current_request(
*,
method: KnowledgeFSMethod,
tenant_id: str,
forward: _KnowledgeFSRequestForwarder,
) -> Response:
"""Forward the current raw request through one preconfigured service entry."""
if not dify_config.KNOWLEDGE_FS_ENABLED:
raise NotFound()
try:
proxy_result = forward(
request.headers.get("Accept"),
request.content_type,
request.query_string or None,
_request_body() if method != "GET" else None,
)
except (
KnowledgeFSConfigurationError,
KnowledgeFSAccessDeniedError,
KnowledgeFSRouteNotAllowedError,
KnowledgeFSTimeoutError,
KnowledgeFSTransportError,
) as exc:
_translate_proxy_error(exc, tenant_id=tenant_id)
return _proxy_response(
proxy_result,
tenant_id=tenant_id,
contract_response_headers=proxy_result.operation.response_headers,
max_response_bytes=proxy_result.operation.max_response_bytes,
)
def _proxy_request(
method: KnowledgeFSMethod,
upstream_path: str,
) -> Response:
"""Authorize and forward the current request through the combined service use case."""
if not dify_config.KNOWLEDGE_FS_ENABLED:
raise NotFound()
current_user, tenant_id = current_account_with_tenant()
def forward(
accept: str | None,
content_type: str | None,
query: bytes | None,
body: bytes | None,
) -> KnowledgeFSUpstreamResponse:
return proxy_knowledge_fs_request(
account=current_user,
method=method,
path=upstream_path,
tenant_id=tenant_id,
accept=accept,
content_type=content_type,
query=query,
body=body,
request_headers=request.headers,
)
return _proxy_current_request(method=method, tenant_id=tenant_id, forward=forward)
def _proxy_authorized_request(authorization: KnowledgeFSAuthorization) -> Response:
"""Forward the current request using one previously authorized operation capability.
Args:
authorization: Request-scoped capability produced before billing and body parsing.
Returns:
The filtered response returned by KnowledgeFS.
Raises:
HTTPException: The integration is disabled or forwarding fails.
"""
operation = authorization.operation
tenant_id = authorization.tenant_id
def forward(
accept: str | None,
content_type: str | None,
query: bytes | None,
body: bytes | None,
) -> KnowledgeFSUpstreamResponse:
return proxy_authorized_knowledge_fs_request(
authorization=authorization,
accept=accept,
content_type=content_type,
query=query,
body=body,
request_headers=request.headers,
)
return _proxy_current_request(method=operation.method, tenant_id=tenant_id, forward=forward)
@_knowledge_fs_enabled
@_knowledge_fs_operation_access_required
@cloud_edition_billing_rate_limit_check("knowledge")
def _proxy_knowledge_fs_non_get(
authorization: KnowledgeFSAuthorization,
) -> ResponseReturnValue:
"""Apply knowledge billing checks to one allowlisted non-GET operation."""
return _proxy_authorized_request(authorization)
@bp.route(
"/knowledge-fs/<path:upstream_path>",
methods=["OPTIONS"],
provide_automatic_options=False,
)
@_console_api_errors
@_knowledge_fs_enabled
def proxy_knowledge_fs_options(upstream_path: str) -> ResponseReturnValue:
"""Complete a CORS preflight only for an enabled Console operation."""
requested_method = cast(KnowledgeFSMethod, request.headers.get("Access-Control-Request-Method", "").upper())
try:
get_knowledge_fs_operation(requested_method, upstream_path)
except KnowledgeFSRouteNotAllowedError as exc:
raise NotFound() from exc
return Response(status=HTTPStatus.NO_CONTENT)
@bp.route(
"/knowledge-fs/<path:upstream_path>",
methods=["GET"],
provide_automatic_options=False,
)
@_console_api_errors
@_knowledge_fs_enabled
@setup_required
@login_required
@account_initialization_required
def proxy_knowledge_fs_get(upstream_path: str) -> ResponseReturnValue:
"""Forward one authenticated, dataset-readable GET request.
Args:
upstream_path: Relative KFS path captured after the Console proxy prefix.
Returns:
The filtered raw KnowledgeFS response or a Console JSON error response.
"""
if request.method != "GET":
raise NotFound()
return _proxy_request("GET", upstream_path)
@bp.route(
"/knowledge-fs/<path:upstream_path>",
methods=["DELETE", "PATCH", "POST", "PUT"],
provide_automatic_options=False,
)
@_console_api_errors
@_knowledge_fs_enabled
@setup_required
@login_required
@account_initialization_required
def proxy_knowledge_fs_write(upstream_path: str) -> ResponseReturnValue:
"""Forward one authenticated non-GET request under its contract access policy.
Args:
upstream_path: Relative KFS path captured after the Console proxy prefix.
Returns:
The filtered raw KnowledgeFS response or a Console JSON error response.
"""
method = cast(KnowledgeFSMethod, request.method)
return _proxy_knowledge_fs_non_get(method, upstream_path)
-106
View File
@@ -1,106 +0,0 @@
"""Console onboarding APIs.
This module keeps Step-by-step Tour persistence account-scoped. Workspace IDs
are accepted only as presentation overrides; UI-only state such as minimized
panels or the currently active task stays on the frontend. PATCH requests are
action-based so callers do not replace server-side arrays with stale snapshots.
"""
from datetime import datetime
from typing import Literal, cast
from flask_restx import Resource
from pydantic import BaseModel, ConfigDict, Field, model_validator
from controllers.common.schema import register_response_schema_models, register_schema_models
from extensions.ext_database import db
from fields.base import ResponseModel
from libs.helper import dump_response
from libs.login import login_required
from models import Account
from services.step_by_step_tour_service import StepByStepTourPatch, StepByStepTourService
from . import console_ns
from .wraps import account_initialization_required, setup_required, with_current_tenant_id, with_current_user
StepByStepTourAction = Literal[
"skip",
"complete_task",
"uncomplete_task",
"enable_current_workspace",
"disable_current_workspace",
]
StepByStepTourTaskId = Literal["home", "studio", "knowledge", "integration"]
class StepByStepTourStatePatchPayload(BaseModel):
action: StepByStepTourAction = Field(description="State update action")
task_id: StepByStepTourTaskId | None = Field(default=None, description="Task ID for task actions")
model_config = ConfigDict(extra="forbid")
@model_validator(mode="after")
def validate_patch_shape(self) -> "StepByStepTourStatePatchPayload":
task_actions = {"complete_task", "uncomplete_task"}
if self.action in task_actions and self.task_id is None:
raise ValueError("task_id is required for task actions")
if self.action not in task_actions and self.task_id is not None:
raise ValueError("task_id is only supported for task actions")
return self
class StepByStepTourStateResponse(ResponseModel):
first_workspace_id: str | None = None
skipped: bool = False
completed_task_ids: list[StepByStepTourTaskId] = Field(default_factory=list)
manually_enabled_workspace_ids: list[str] = Field(default_factory=list)
manually_disabled_workspace_ids: list[str] = Field(default_factory=list)
updated_at: datetime | None = None
register_schema_models(console_ns, StepByStepTourStatePatchPayload)
register_response_schema_models(console_ns, StepByStepTourStateResponse)
@console_ns.route("/onboarding/step-by-step-tour/state")
class StepByStepTourStateApi(Resource):
@console_ns.doc("get_step_by_step_tour_state")
@console_ns.doc(description="Get account-level Step-by-step Tour state")
@console_ns.response(200, "Success", console_ns.models[StepByStepTourStateResponse.__name__])
@setup_required
@login_required
@account_initialization_required
@with_current_user
@with_current_tenant_id
def get(self, current_tenant_id: str, current_user: Account):
return dump_response(
StepByStepTourStateResponse,
StepByStepTourService.get_state(
account=current_user,
current_tenant_id=current_tenant_id,
session=db.session,
),
)
@console_ns.doc("patch_step_by_step_tour_state")
@console_ns.doc(description="Update account-level Step-by-step Tour state")
@console_ns.expect(console_ns.models[StepByStepTourStatePatchPayload.__name__])
@console_ns.response(200, "Success", console_ns.models[StepByStepTourStateResponse.__name__])
@setup_required
@login_required
@account_initialization_required
@with_current_user
@with_current_tenant_id
def patch(self, current_tenant_id: str, current_user: Account):
payload = StepByStepTourStatePatchPayload.model_validate(console_ns.payload or {})
patch = cast(StepByStepTourPatch, payload.model_dump(exclude_unset=True, exclude_none=True))
return dump_response(
StepByStepTourStateResponse,
StepByStepTourService.patch_state(
account=current_user,
current_tenant_id=current_tenant_id,
patch=patch,
session=db.session,
),
)
+47 -57
View File
@@ -46,61 +46,6 @@ class GetRemoteFileInfo(Resource):
).model_dump(mode="json")
def upload_remote_file_from_request(
*,
current_user: Account,
resource_tenant_id: str | None = None,
) -> FileWithSignedUrl:
"""Validate the JSON request, fetch its remote file, and persist it under the requested tenant."""
payload = RemoteFileUploadPayload.model_validate(console_ns.payload)
url = payload.url
# Try to fetch remote file metadata/content first
try:
resp = remote_fetcher.make_request("HEAD", url=url)
if resp.status_code != httpx.codes.OK:
resp = remote_fetcher.make_request("GET", url=url, timeout=3, follow_redirects=True)
if resp.status_code != httpx.codes.OK:
# Normalize into a user-friendly error message expected by tests
raise RemoteFileUploadError(f"Failed to fetch file from {url}: {resp.text}")
except httpx.RequestError as e:
raise RemoteFileUploadError(f"Failed to fetch file from {url}: {str(e)}")
file_info = helpers.guess_file_info_from_response(resp)
# Enforce file size limit with 400 (Bad Request) per tests' expectation
if not FileService.is_file_size_within_limit(extension=file_info.extension, file_size=file_info.size):
raise FileTooLargeError()
# Load content if needed
content = resp.content if resp.request.method == "GET" else remote_fetcher.make_request("GET", url).content
try:
upload_file = FileService(db.engine).upload_file(
filename=file_info.filename,
content=content,
mimetype=file_info.mimetype,
user=current_user,
tenant_id=resource_tenant_id,
source_url=url,
)
except services.errors.file.FileTooLargeError as file_too_large_error:
raise FileTooLargeError(file_too_large_error.description)
except services.errors.file.UnsupportedFileTypeError:
raise UnsupportedFileTypeError()
return FileWithSignedUrl(
id=upload_file.id,
name=upload_file.name,
size=upload_file.size,
extension=upload_file.extension,
url=file_helpers.get_signed_file_url(upload_file_id=upload_file.id),
mime_type=upload_file.mime_type,
created_by=upload_file.created_by,
created_at=int(upload_file.created_at.timestamp()),
)
@console_ns.route("/remote-files/upload")
class RemoteFileUpload(Resource):
@console_ns.expect(console_ns.models[RemoteFileUploadPayload.__name__])
@@ -108,8 +53,53 @@ class RemoteFileUpload(Resource):
@login_required
@with_current_user
def post(self, current_user: Account):
remote_file = upload_remote_file_from_request(current_user=current_user)
payload = RemoteFileUploadPayload.model_validate(console_ns.payload)
url = payload.url
# Try to fetch remote file metadata/content first
try:
resp = remote_fetcher.make_request("HEAD", url=url)
if resp.status_code != httpx.codes.OK:
resp = remote_fetcher.make_request("GET", url=url, timeout=3, follow_redirects=True)
if resp.status_code != httpx.codes.OK:
# Normalize into a user-friendly error message expected by tests
raise RemoteFileUploadError(f"Failed to fetch file from {url}: {resp.text}")
except httpx.RequestError as e:
raise RemoteFileUploadError(f"Failed to fetch file from {url}: {str(e)}")
file_info = helpers.guess_file_info_from_response(resp)
# Enforce file size limit with 400 (Bad Request) per tests' expectation
if not FileService.is_file_size_within_limit(extension=file_info.extension, file_size=file_info.size):
raise FileTooLargeError()
# Load content if needed
content = resp.content if resp.request.method == "GET" else remote_fetcher.make_request("GET", url).content
try:
upload_file = FileService(db.engine).upload_file(
filename=file_info.filename,
content=content,
mimetype=file_info.mimetype,
user=current_user,
source_url=url,
)
except services.errors.file.FileTooLargeError as file_too_large_error:
raise FileTooLargeError(file_too_large_error.description)
except services.errors.file.UnsupportedFileTypeError:
raise UnsupportedFileTypeError()
# Success: return created resource with 201 status
return (
remote_file.model_dump(mode="json"),
FileWithSignedUrl(
id=upload_file.id,
name=upload_file.name,
size=upload_file.size,
extension=upload_file.extension,
url=file_helpers.get_signed_file_url(upload_file_id=upload_file.id),
mime_type=upload_file.mime_type,
created_by=upload_file.created_by,
created_at=int(upload_file.created_at.timestamp()),
).model_dump(mode="json"),
201,
)
+5 -6
View File
@@ -4,7 +4,7 @@ from typing import cast
from flask import Request as FlaskRequest
from core.db.session_factory import session_factory
from extensions.ext_database import db
from extensions.ext_socketio import sio
from libs.passport import PassportService
from libs.token import extract_access_token
@@ -43,8 +43,8 @@ def socket_connect(sid, environ, auth):
logging.warning("Socket connect rejected: missing user_id (sid=%s)", sid)
return False
with sio.app.app_context(), session_factory.create_session() as session:
user = AccountService.load_logged_in_account(account_id=user_id, session=session)
with sio.app.app_context():
user = AccountService.load_logged_in_account(account_id=user_id, session=db.session())
if not user:
logging.warning("Socket connect rejected: user not found (user_id=%s, sid=%s)", user_id, sid)
return False
@@ -69,8 +69,8 @@ def handle_user_connect(sid, data):
if not workflow_id:
return {"msg": "workflow_id is required"}, 400
with sio.app.app_context(), session_factory.create_session() as session:
result = collaboration_service.authorize_and_join_workflow_room(workflow_id, sid, session=session)
with sio.app.app_context():
result = collaboration_service.authorize_and_join_workflow_room(workflow_id, sid, session=db.session())
if not result:
return {"msg": "unauthorized"}, 401
@@ -98,7 +98,6 @@ def handle_collaboration_event(sid, data):
6. workflow_update
7. comments_update
8. node_panel_presence
9. graph_view_state (session reports tab visibility; drives leader election)
"""
return collaboration_service.relay_collaboration_event(sid, data)
+27 -10
View File
@@ -4,16 +4,20 @@ from http import HTTPStatus
from flask import redirect
from flask_restx import Resource
from pydantic import BaseModel, Field
from werkzeug.exceptions import Conflict, Forbidden, NotFound
from werkzeug.exceptions import Conflict, NotFound
from controllers.common.fields import RedirectResponse
from controllers.common.schema import register_response_schema_models, register_schema_models
from controllers.console import console_ns
from controllers.console.wraps import (
RBACPermission,
RBACResourceScope,
account_initialization_required,
cloud_edition_billing_enabled,
cloud_edition_billing_paid_plan_required,
is_admin_or_owner_required,
only_edition_cloud,
rbac_permission_required,
setup_required,
)
from extensions.ext_database import db
@@ -21,7 +25,6 @@ from fields.base import ResponseModel
from libs.archive_storage import get_export_storage
from libs.helper import dump_response
from libs.login import current_account_with_tenant, login_required
from models import TenantAccountRole
from services.retention.workflow_run.archive_download_preparation import ARCHIVE_DOWNLOAD_MIME_TYPE
from services.retention.workflow_run.archive_download_task_cache import (
WorkflowRunArchiveDownloadStatus,
@@ -95,13 +98,11 @@ register_response_schema_models(
)
def _current_owner_or_admin_ids() -> tuple[str, str]:
"""Return current Cloud workspace IDs for an owner or admin, independently of enterprise RBAC."""
def _current_ids() -> tuple[str, str]:
"""Return current `(tenant_id, account_id)` or raise when no workspace is selected."""
current_user, current_tenant_id = current_account_with_tenant()
if not current_tenant_id:
raise NotFound("Current workspace not found")
if not TenantAccountRole.is_privileged_role(current_user.current_role):
raise Forbidden()
return current_tenant_id, current_user.id
@@ -123,8 +124,12 @@ class WorkflowRunArchivesApi(Resource):
@only_edition_cloud
@cloud_edition_billing_enabled
@cloud_edition_billing_paid_plan_required
@is_admin_or_owner_required
@rbac_permission_required(
RBACResourceScope.WORKSPACE, RBACPermission.WORKSPACE_ROLE_MANAGE, resource_required=False
)
def get(self):
tenant_id, _ = _current_owner_or_admin_ids()
tenant_id, _ = _current_ids()
return dump_response(WorkflowRunArchiveListResponse, list_workflow_run_archives(db.session(), tenant_id))
@@ -144,8 +149,12 @@ class WorkflowRunArchiveDownloadsApi(Resource):
@only_edition_cloud
@cloud_edition_billing_enabled
@cloud_edition_billing_paid_plan_required
@is_admin_or_owner_required
@rbac_permission_required(
RBACResourceScope.WORKSPACE, RBACPermission.WORKSPACE_ROLE_MANAGE, resource_required=False
)
def post(self):
tenant_id, account_id = _current_owner_or_admin_ids()
tenant_id, account_id = _current_ids()
payload = WorkflowRunArchiveDownloadPayload.model_validate(console_ns.payload or {})
try:
task = create_workflow_run_archive_download_task(
@@ -171,8 +180,12 @@ class WorkflowRunArchiveDownloadApi(Resource):
@only_edition_cloud
@cloud_edition_billing_enabled
@cloud_edition_billing_paid_plan_required
@is_admin_or_owner_required
@rbac_permission_required(
RBACResourceScope.WORKSPACE, RBACPermission.WORKSPACE_ROLE_MANAGE, resource_required=False
)
def get(self, download_id: str):
tenant_id, _ = _current_owner_or_admin_ids()
tenant_id, _ = _current_ids()
try:
task = get_workflow_run_archive_download_task(tenant_id=tenant_id, download_id=download_id)
except WorkflowRunArchiveDownloadTaskNotFoundError as exc:
@@ -196,8 +209,12 @@ class WorkflowRunArchiveDownloadFileApi(Resource):
@only_edition_cloud
@cloud_edition_billing_enabled
@cloud_edition_billing_paid_plan_required
@is_admin_or_owner_required
@rbac_permission_required(
RBACResourceScope.WORKSPACE, RBACPermission.WORKSPACE_ROLE_MANAGE, resource_required=False
)
def get(self, download_id: str):
tenant_id, _ = _current_owner_or_admin_ids()
tenant_id, _ = _current_ids()
try:
task = get_ready_workflow_run_archive_download_task(tenant_id=tenant_id, download_id=download_id)
except WorkflowRunArchiveDownloadTaskNotFoundError as exc:
@@ -67,15 +67,6 @@ from services.plugin.plugin_parameter_service import PluginParameterService
from services.plugin.plugin_permission_service import PluginPermissionService
from services.tools.tools_transform_service import ToolTransformService
_PLUGIN_PACKAGE_UPLOAD_PARAMS = {
"pkg": {
"description": "Plugin package to upload",
"in": "formData",
"type": "file",
"required": True,
}
}
class AutoUpgradeSettingsResponse(TypedDict):
strategy_setting: TenantPluginAutoUpgradeStrategySetting
@@ -654,7 +645,6 @@ class PluginAssetApi(Resource):
@console_ns.route("/workspaces/current/plugin/upload/pkg")
class PluginUploadFromPkgApi(Resource):
@console_ns.doc(consumes=["multipart/form-data"], params=_PLUGIN_PACKAGE_UPLOAD_PARAMS)
@console_ns.response(200, "Success", console_ns.models[PluginDecodeResponse.__name__])
@setup_required
@login_required
+21 -34
View File
@@ -6,8 +6,7 @@ from flask import request
from flask_restx import Resource
from pydantic import BaseModel, Field, field_validator
from sqlalchemy import select
from sqlalchemy.orm import Session
from werkzeug.exceptions import NotFound, Unauthorized
from werkzeug.exceptions import Unauthorized
import services
from configs import dify_config
@@ -24,7 +23,6 @@ from controllers.common.schema import (
register_response_schema_models,
register_schema_models,
)
from controllers.common.session import with_session
from controllers.console import console_ns
from controllers.console.admin import admin_required
from controllers.console.error import AccountNotLinkTenantError
@@ -88,7 +86,6 @@ class TenantInfoResponse(ResponseModel):
custom_config: WorkspaceCustomConfigResponse | None = None
trial_credits: int | None = None
trial_credits_used: int | None = None
trial_credits_exhausted_at: int | None = None
next_credit_reset_date: int | None = None
@field_validator("plan", "status", "trial_end_reason", mode="before")
@@ -223,11 +220,10 @@ class TenantListApi(Resource):
@account_initialization_required
@with_current_user
@with_current_tenant_id
@with_session(write=False)
def get(self, session: Session, current_tenant_id: str, current_user: Account):
def get(self, current_tenant_id: str, current_user: Account):
tenant_rows: list[tuple[Tenant, TenantAccountJoin]] = [
(tenant, membership)
for tenant, membership in TenantService.get_workspaces_for_account(current_user.id, session=session)
for tenant, membership in TenantService.get_workspaces_for_account(current_user.id, session=db.session())
if tenant.status == TenantStatus.NORMAL
]
tenants = [tenant for tenant, _ in tenant_rows]
@@ -278,12 +274,11 @@ class WorkspaceListApi(Resource):
@console_ns.response(HTTPStatus.OK, "Success", console_ns.models[WorkspacePaginationResponse.__name__])
@setup_required
@admin_required
@with_session(write=False)
def get(self, session: Session):
def get(self):
args = query_params_from_request(WorkspaceListQuery)
stmt = select(Tenant).order_by(Tenant.created_at.desc())
tenants = paginate_query(stmt, session=session, page=args.page, per_page=args.limit)
tenants = paginate_query(stmt, page=args.page, per_page=args.limit)
has_more = False
if tenants.has_next:
@@ -302,8 +297,7 @@ class TenantApi(Resource):
@account_initialization_required
@console_ns.response(HTTPStatus.OK, "Success", console_ns.models[TenantInfoResponse.__name__])
@with_current_user
@with_session
def post(self, session: Session, current_user: Account):
def post(self, current_user: Account):
if request.path == "/info":
logger.warning("Deprecated URL /info was used.")
@@ -312,17 +306,17 @@ class TenantApi(Resource):
raise ValueError("No current tenant")
if tenant.status == TenantStatus.ARCHIVE:
tenants = TenantService.get_join_tenants(current_user, session=session)
tenants = TenantService.get_join_tenants(current_user, session=db.session())
# if there is any tenant, switch to the first one
if len(tenants) > 0:
TenantService.switch_tenant(current_user, tenants[0].id, session=session)
TenantService.switch_tenant(current_user, tenants[0].id, session=db.session())
tenant = tenants[0]
# else, raise Unauthorized
else:
raise Unauthorized("workspace is archived")
return (
dump_response(TenantInfoResponse, WorkspaceService.get_tenant_info(tenant, session=session)),
dump_response(TenantInfoResponse, WorkspaceService.get_tenant_info(tenant, session=db.session())),
HTTPStatus.OK,
)
@@ -335,23 +329,22 @@ class SwitchWorkspaceApi(Resource):
@login_required
@account_initialization_required
@with_current_user
@with_session
def post(self, session: Session, current_user: Account):
def post(self, current_user: Account):
payload = console_ns.payload or {}
args = SwitchWorkspacePayload.model_validate(payload)
# Check whether the tenant_id belongs to the current account.
try:
TenantService.switch_tenant(current_user, args.tenant_id, session=session)
TenantService.switch_tenant(current_user, args.tenant_id, session=db.session())
except Exception:
raise AccountNotLinkTenantError("Account not link tenant")
new_tenant = TenantService.get_tenant_by_id(args.tenant_id, session=session)
new_tenant = db.session.get(Tenant, args.tenant_id) # Get new tenant
if new_tenant is None:
raise ValueError("Tenant not found")
return SwitchWorkspaceResponse(
result="success", new_tenant=WorkspaceService.get_tenant_info(new_tenant, session=session)
result="success", new_tenant=WorkspaceService.get_tenant_info(new_tenant, session=db.session())
).model_dump(mode="json")
@@ -364,13 +357,10 @@ class CustomConfigWorkspaceApi(Resource):
@account_initialization_required
@cloud_edition_billing_resource_check("workspace_custom")
@with_current_tenant_id
@with_session
def post(self, session: Session, current_tenant_id: str):
def post(self, current_tenant_id: str):
payload = console_ns.payload or {}
args = WorkspaceCustomConfigPayload.model_validate(payload)
tenant = TenantService.get_tenant_by_id(current_tenant_id, session=session)
if tenant is None:
raise NotFound()
tenant = db.get_or_404(Tenant, current_tenant_id)
custom_config_dict: TenantCustomConfigDict = {
"remove_webapp_brand": args.remove_webapp_brand
@@ -382,10 +372,10 @@ class CustomConfigWorkspaceApi(Resource):
}
tenant.custom_config_dict = custom_config_dict
session.commit()
db.session.commit()
return WorkspaceTenantResultResponse(
result="success", tenant=WorkspaceService.get_tenant_info(tenant, session=session)
result="success", tenant=WorkspaceService.get_tenant_info(tenant, session=db.session())
).model_dump(mode="json")
@@ -440,21 +430,18 @@ class WorkspaceInfoApi(Resource):
@account_initialization_required
# Change workspace name
@with_current_tenant_id
@with_session
def post(self, session: Session, current_tenant_id: str):
def post(self, current_tenant_id: str):
payload = console_ns.payload or {}
args = WorkspaceInfoPayload.model_validate(payload)
if not current_tenant_id:
raise ValueError("No current tenant")
tenant = TenantService.get_tenant_by_id(current_tenant_id, session=session)
if tenant is None:
raise NotFound()
tenant = db.get_or_404(Tenant, current_tenant_id)
tenant.name = args.name
session.commit()
db.session.commit()
return WorkspaceTenantResultResponse(
result="success", tenant=WorkspaceService.get_tenant_info(tenant, session=session)
result="success", tenant=WorkspaceService.get_tenant_info(tenant, session=db.session())
).model_dump(mode="json")
+2 -1
View File
@@ -54,8 +54,9 @@ class EnterpriseAppDSLImport(Resource):
if account is None:
return {"message": f"account '{args.creator_email}' not found or inactive"}, 404
account.set_tenant_id(workspace_id)
with Session(db.engine, expire_on_commit=False) as session:
account.set_tenant_id_with_session(workspace_id, session=session)
dsl_service = AppDslService(session)
result = dsl_service.import_app(
account=account,
@@ -47,12 +47,10 @@ class EnterpriseWorkspace(Resource):
if account is None:
return {"message": "owner account not found."}, 404
tenant = TenantService.create_owner_tenant(
account,
name=args.name,
is_from_dashboard=True,
session=db.session(),
)
tenant = TenantService.create_tenant(args.name, is_from_dashboard=True, session=db.session())
TenantService.create_tenant_member(tenant, account, db.session(), role="owner")
tenant_was_created.send(tenant)
resp = {
"id": tenant.id,
+7 -10
View File
@@ -4,11 +4,9 @@ from __future__ import annotations
from typing import Any, cast
from sqlalchemy.orm import Session
from controllers.service_api.app.error import AppUnavailableError
from models import App
from models.model import AppMode, load_annotation_reply_config
from models.model import AppMode
JSON_SCHEMA_DRAFT = "https://json-schema.org/draft/2020-12/schema"
@@ -91,13 +89,13 @@ def _form_to_jsonschema(form: list[dict[str, Any]]) -> tuple[dict[str, Any], lis
return properties, required
def resolve_app_config(app: App, *, session: Session) -> tuple[dict[str, Any], list[dict[str, Any]]]:
def resolve_app_config(app: App) -> tuple[dict[str, Any], list[dict[str, Any]]]:
"""Resolve `(features_dict, user_input_form)` for parameters / schema derivation.
Raises `AppUnavailableError` on misconfigured apps.
"""
if app.mode in {AppMode.ADVANCED_CHAT, AppMode.WORKFLOW}:
workflow = app.workflow_with_session(session=session)
workflow = app.workflow
if workflow is None:
raise AppUnavailableError()
return (
@@ -105,22 +103,21 @@ def resolve_app_config(app: App, *, session: Session) -> tuple[dict[str, Any], l
cast(list[dict[str, Any]], workflow.user_input_form(to_old_structure=True)),
)
app_model_config = app.app_model_config_with_session(session=session)
app_model_config = app.app_model_config
if app_model_config is None:
raise AppUnavailableError()
annotation_reply = load_annotation_reply_config(session, app_model_config.app_id)
features_dict = cast(dict[str, Any], app_model_config.to_dict(annotation_reply=annotation_reply))
features_dict = cast(dict[str, Any], app_model_config.to_dict())
return features_dict, cast(list[dict[str, Any]], features_dict.get("user_input_form", []))
def build_input_schema(app: App, *, session: Session) -> dict[str, Any]:
def build_input_schema(app: App) -> dict[str, Any]:
"""Derive Draft 2020-12 JSON Schema from `user_input_form` + app mode.
chat / agent-chat / advanced-chat: top-level `query` (required, minLength=1) + `inputs` object.
completion / workflow: `inputs` object only.
Raises `AppUnavailableError` on misconfigured apps.
"""
_, user_input_form = resolve_app_config(app, session=session)
_, user_input_form = resolve_app_config(app)
inputs_props, inputs_required = _form_to_jsonschema(user_input_form)
properties: dict[str, Any] = {}
+13 -16
View File
@@ -3,10 +3,8 @@ from __future__ import annotations
from datetime import UTC, datetime
from flask_restx import Resource
from sqlalchemy.orm import Session
from werkzeug.exceptions import NotFound
from controllers.common.session import with_session
from controllers.openapi import openapi_ns
from controllers.openapi._contract import accepts, returns
from controllers.openapi._models import (
@@ -20,6 +18,7 @@ from controllers.openapi._models import (
)
from controllers.openapi.auth.composition import auth_router
from controllers.openapi.auth.data import AuthData
from extensions.ext_database import db
from extensions.ext_redis import redis_client
from libs.oauth_bearer import (
Scope,
@@ -42,13 +41,14 @@ from services.oauth_device_flow import (
class AccountApi(Resource):
@auth_router.guard(scope=Scope.FULL, allowed_token_types=frozenset({TokenType.OAUTH_ACCOUNT}))
@returns(200, AccountResponse, description="Account info")
@with_session(write=False)
def get(self, session: Session, *, auth_data: AuthData):
def get(self, *, auth_data: AuthData):
enforce(LIMIT_ME_PER_ACCOUNT, key=f"account:{auth_data.account_id}")
account_id_str = str(auth_data.account_id) if auth_data.account_id else None
account = AccountService.get_account_by_id(account_id_str, session=session) if account_id_str else None
memberships = TenantService.get_account_memberships(account_id_str, session=session) if account_id_str else []
account = AccountService.get_account_by_id(account_id_str, session=db.session()) if account_id_str else None
memberships = (
TenantService.get_account_memberships(account_id_str, session=db.session()) if account_id_str else []
)
default_ws_id = _pick_default_workspace(memberships)
return AccountResponse(
@@ -64,9 +64,8 @@ class AccountApi(Resource):
class AccountSessionsSelfApi(Resource):
@auth_router.guard(scope=Scope.FULL, allowed_token_types=frozenset({TokenType.OAUTH_ACCOUNT}))
@returns(200, RevokeResponse, description="Session revoked")
@with_session
def delete(self, session: Session, *, auth_data: AuthData):
revoke_oauth_token(redis_client, str(auth_data.token_id), session=session)
def delete(self, *, auth_data: AuthData):
revoke_oauth_token(redis_client, str(auth_data.token_id), session=db.session())
return RevokeResponse(status="revoked")
@@ -75,8 +74,7 @@ class AccountSessionsApi(Resource):
@auth_router.guard(scope=Scope.FULL, allowed_token_types=frozenset({TokenType.OAUTH_ACCOUNT}))
@returns(200, SessionListResponse, description="Session list")
@accepts(query=SessionListQuery)
@with_session(write=False)
def get(self, session: Session, *, auth_data: AuthData, query: SessionListQuery):
def get(self, *, auth_data: AuthData, query: SessionListQuery):
# SessionListQuery enforces the advertised bounds (extra='forbid', page>=1,
# 1<=limit<=MAX_PAGE_LIMIT) so the server rejects out-of-range paging rather
# than silently coercing (e.g. page=0 -> empty slice).
@@ -85,7 +83,7 @@ class AccountSessionsApi(Resource):
page = query.page
limit = query.limit
all_rows = list_active_sessions(ctx, now, session=session)
all_rows = list_active_sessions(ctx, now, session=db.session())
total = len(all_rows)
sliced = all_rows[(page - 1) * limit : page * limit]
@@ -116,16 +114,15 @@ class AccountSessionsApi(Resource):
class AccountSessionByIdApi(Resource):
@auth_router.guard(scope=Scope.FULL, allowed_token_types=frozenset({TokenType.OAUTH_ACCOUNT}))
@returns(200, RevokeResponse, description="Session revoked")
@with_session
def delete(self, session: Session, session_id: str, *, auth_data: AuthData):
def delete(self, session_id: str, *, auth_data: AuthData):
ctx = get_auth_ctx()
# 404 (not 403) on cross-subject so the endpoint doesn't leak
# token IDs that belong to other subjects.
if not token_belongs_to_subject(session_id, ctx, session=session):
if not token_belongs_to_subject(session_id, ctx, session=db.session()):
raise NotFound("session not found")
revoke_oauth_token(redis_client, session_id, session=session)
revoke_oauth_token(redis_client, session_id, session=db.session())
return RevokeResponse(status="revoked")
+18 -25
View File
@@ -6,13 +6,11 @@ import uuid as _uuid
from typing import Any, cast
from flask_restx import Resource
from sqlalchemy.orm import Session
from werkzeug.exceptions import Conflict, NotFound, UnprocessableEntity
from configs import dify_config
from controllers.common.app_access import AppAccessFilter, resolve_app_access_filter
from controllers.common.fields import Parameters
from controllers.common.session import with_session
from controllers.common.wraps import RBACPermission, RBACResourceScope
from controllers.openapi import openapi_ns
from controllers.openapi._contract import accepts, returns
@@ -30,6 +28,7 @@ from controllers.openapi.auth.composition import auth_router
from controllers.openapi.auth.data import AuthData, CallerKind, RBACRequirement
from controllers.service_api.app.error import AppUnavailableError
from core.app.app_config.common.parameters_mapping import get_parameters_from_feature_dict
from extensions.ext_database import db
from libs.oauth_bearer import Scope, TokenType
from models import App
from models.enums import AppStatus
@@ -57,7 +56,7 @@ _EMPTY_PARAMETERS: dict[str, Any] = {
class AppReadResource(Resource):
"""Base for per-app read endpoints; subclasses call `_load()` for membership/exists checks."""
def _load(self, session: Session, app_id: str, workspace_id: str | None = None) -> App:
def _load(self, app_id: str, workspace_id: str | None = None) -> App:
try:
parsed_uuid = _uuid.UUID(app_id)
is_uuid = True
@@ -67,13 +66,13 @@ class AppReadResource(Resource):
if is_uuid:
# ``str(parsed_uuid)`` normalises to the canonical dashed form.
app = AppService.get_visible_app_by_id(str(parsed_uuid), session)
app = AppService.get_visible_app_by_id(str(parsed_uuid), session=db.session())
if app is None:
raise NotFound("app not found")
else:
if not workspace_id:
raise UnprocessableEntity("workspace_id is required for name-based lookup")
matches = AppService.find_visible_apps_by_name(session, name=app_id, tenant_id=workspace_id)
matches = AppService.find_visible_apps_by_name(name=app_id, tenant_id=workspace_id, session=db.session())
if len(matches) == 0:
raise NotFound("app not found")
if len(matches) > 1:
@@ -87,14 +86,14 @@ class AppReadResource(Resource):
return app
def parameters_payload(app: App, *, session: Session) -> dict:
def parameters_payload(app: App) -> dict:
"""Mirrors service_api/app/app.py::AppParameterApi response body."""
features_dict, user_input_form = resolve_app_config(app, session=session)
features_dict, user_input_form = resolve_app_config(app)
parameters = get_parameters_from_feature_dict(features_dict=features_dict, user_input_form=user_input_form)
return Parameters.model_validate(parameters).model_dump(mode="json")
def build_app_describe_response(app: App, fields: set[str] | None, *, session: Session) -> AppDescribeResponse:
def build_app_describe_response(app: App, fields: set[str] | None) -> AppDescribeResponse:
"""Public projection of an app (name / params / input schema) — never internal config."""
want_info = fields is None or "info" in fields
want_params = fields is None or "parameters" in fields
@@ -118,12 +117,12 @@ def build_app_describe_response(app: App, fields: set[str] | None, *, session: S
input_schema: dict[str, Any] | None = None
if want_params:
try:
parameters = parameters_payload(app, session=session)
parameters = parameters_payload(app)
except AppUnavailableError:
parameters = dict(_EMPTY_PARAMETERS)
if want_schema:
try:
input_schema = build_input_schema(app, session=session)
input_schema = build_input_schema(app)
except AppUnavailableError:
input_schema = dict(EMPTY_INPUT_SCHEMA)
@@ -139,11 +138,10 @@ class AppDescribeApi(AppReadResource):
)
@returns(200, AppDescribeResponse, description="App description")
@accepts(query=AppDescribeQuery)
@with_session(write=False)
def get(self, session: Session, app_id: str, *, auth_data: AuthData, query: AppDescribeQuery):
def get(self, app_id: str, *, auth_data: AuthData, query: AppDescribeQuery):
# describe is UUID-only (workspace_id query param dropped in #37212).
app = self._load(session, app_id)
return build_app_describe_response(app, query.fields, session=session)
app = self._load(app_id)
return build_app_describe_response(app, query.fields)
@openapi_ns.route("/apps")
@@ -151,8 +149,7 @@ class AppListApi(Resource):
@auth_router.guard_workspace(scope=Scope.APPS_READ, allowed_token_types=frozenset({TokenType.OAUTH_ACCOUNT}))
@returns(200, AppListResponse, description="App list")
@accepts(query=AppListQuery)
@with_session(write=False)
def get(self, session: Session, *, auth_data: AuthData, query: AppListQuery):
def get(self, *, auth_data: AuthData, query: AppListQuery):
workspace_id = query.workspace_id
empty = AppListResponse(page=query.page, limit=query.limit, total=0, has_more=False, data=[])
@@ -176,15 +173,11 @@ class AppListApi(Resource):
)
access_filter = AppAccessFilter.unrestricted()
if apply_rbac_filter:
access_filter = resolve_app_access_filter(
workspace_id,
str(auth_data.account_id),
session=session,
)
access_filter = resolve_app_access_filter(workspace_id, str(auth_data.account_id))
tenant_name: str | None = None
if parsed_uuid is not None:
app: App | None = AppService.get_visible_app_by_id(str(parsed_uuid), session)
app: App | None = AppService.get_visible_app_by_id(str(parsed_uuid), session=db.session())
if app is None or str(app.tenant_id) != workspace_id:
return empty
if not _is_listable(app):
@@ -195,7 +188,7 @@ class AppListApi(Resource):
str(app.id), str(app.maintainer) if app.maintainer else None, str(auth_data.account_id)
):
return empty
tenant_name = TenantService.get_tenant_name(workspace_id, session=session)
tenant_name = TenantService.get_tenant_name(workspace_id, session=db.session())
item = AppListRow(
id=str(app.id),
name=app.name,
@@ -222,13 +215,13 @@ class AppListApi(Resource):
if apply_rbac_filter:
access_filter.apply_to_params(params)
pagination = AppService().get_paginate_apps(str(auth_data.account_id), workspace_id, params, session)
pagination = AppService().get_paginate_apps(str(auth_data.account_id), workspace_id, params, db.session())
if pagination is None:
return empty
tenant_name = None
if pagination.items:
tenant_name = TenantService.get_tenant_name(workspace_id, session=session)
tenant_name = TenantService.get_tenant_name(workspace_id, session=db.session())
items = [
AppListRow(
@@ -8,10 +8,8 @@ EE blueprint chain so this module is unreachable there.
from __future__ import annotations
from flask_restx import Resource
from sqlalchemy.orm import Session
from werkzeug.exceptions import NotFound
from controllers.common.session import with_session
from controllers.openapi import openapi_ns
from controllers.openapi._contract import accepts, returns
from controllers.openapi._models import (
@@ -24,6 +22,7 @@ from controllers.openapi._models import (
from controllers.openapi.apps import build_app_describe_response
from controllers.openapi.auth.composition import auth_router
from controllers.openapi.auth.data import AuthData, Edition
from extensions.ext_database import db
from libs.oauth_bearer import Scope, TokenType
from models import App
from models.enums import AppStatus
@@ -41,8 +40,7 @@ class PermittedExternalAppsListApi(Resource):
)
@returns(200, PermittedExternalAppsListResponse, description="Permitted external apps list")
@accepts(query=PermittedExternalAppsListQuery)
@with_session(write=False)
def get(self, session: Session, *, auth_data: AuthData, query: PermittedExternalAppsListQuery):
def get(self, *, auth_data: AuthData, query: PermittedExternalAppsListQuery):
page_result = list_permitted_apps(
page=query.page,
limit=query.limit,
@@ -57,10 +55,10 @@ class PermittedExternalAppsListApi(Resource):
return env
apps_by_id: dict[str, App] = {
str(a.id): a for a in AppService.find_visible_apps_by_ids(page_result.app_ids, session)
str(a.id): a for a in AppService.find_visible_apps_by_ids(page_result.app_ids, session=db.session())
}
tenant_ids = list({str(a.tenant_id) for a in apps_by_id.values()})
tenants_by_id = {str(t.id): t for t in TenantService.get_tenants_by_ids(tenant_ids, session=session)}
tenants_by_id = {str(t.id): t for t in TenantService.get_tenants_by_ids(tenant_ids, session=db.session())}
items: list[AppListRow] = []
for app_id in page_result.app_ids:
@@ -98,10 +96,9 @@ class PermittedExternalAppDescribeApi(Resource):
)
@returns(200, AppDescribeResponse, description="Permitted external app description")
@accepts(query=AppDescribeQuery)
@with_session(write=False)
def get(self, session: Session, app_id: str, *, auth_data: AuthData, query: AppDescribeQuery):
def get(self, app_id: str, *, auth_data: AuthData, query: AppDescribeQuery):
# App already loaded and ACL-checked by the external_sso pipeline; project it.
app = auth_data.app
if app is None:
raise NotFound("app not found")
return build_app_describe_response(app, query.fields, session=session)
return build_app_describe_response(app, query.fields)
+10 -15
View File
@@ -6,7 +6,7 @@ from flask import request
from werkzeug.exceptions import Forbidden, InternalServerError, NotFound, Unauthorized
from controllers.openapi.auth.data import AuthData, CallerKind
from core.db.session_factory import session_factory
from extensions.ext_database import db
from models.account import AccountStatus, TenantStatus
from models.enums import AppStatus, EndUserType
from services.account_service import AccountService, TenantService
@@ -23,8 +23,7 @@ def load_app(data: AuthData) -> None:
uuid.UUID(app_id)
except ValueError:
raise NotFound("app not found")
with session_factory.create_session() as session:
app = AppService.get_app_by_id(app_id, session)
app = AppService.get_app_by_id(app_id, session=db.session())
if not app or app.status != AppStatus.NORMAL:
raise NotFound("app not found")
data.app = app
@@ -35,8 +34,7 @@ def load_tenant(data: AuthData) -> None:
return
if data.app is None:
raise InternalServerError("pipeline_invariant_violated: app not loaded before load_tenant")
with session_factory.create_session() as session:
tenant = TenantService.get_tenant_by_id(str(data.app.tenant_id), session=session)
tenant = TenantService.get_tenant_by_id(str(data.app.tenant_id), session=db.session())
if tenant is None or tenant.status == TenantStatus.ARCHIVE:
raise Forbidden("workspace unavailable")
data.tenant = tenant
@@ -52,8 +50,7 @@ def load_tenant_from_request(data: AuthData) -> None:
uuid.UUID(workspace_id)
except ValueError:
raise NotFound("workspace not found")
with session_factory.create_session() as session:
tenant = TenantService.get_tenant_by_id(workspace_id, session=session)
tenant = TenantService.get_tenant_by_id(workspace_id, session=db.session())
if tenant is None or tenant.status == TenantStatus.ARCHIVE:
raise NotFound("workspace not found")
data.tenant = tenant
@@ -62,12 +59,11 @@ def load_tenant_from_request(data: AuthData) -> None:
def load_account(data: AuthData) -> None:
if data.caller is not None:
return
with session_factory.create_session() as session:
account = AccountService.get_account_by_id(str(data.account_id), session=session)
if account is None:
raise Unauthorized("account not found")
if data.tenant:
account.set_current_tenant_with_session(data.tenant, session=session)
account = AccountService.get_account_by_id(str(data.account_id), session=db.session())
if account is None:
raise Unauthorized("account not found")
if data.tenant:
account.current_tenant = data.tenant
data.caller = account
data.caller_kind = CallerKind.ACCOUNT
@@ -79,8 +75,7 @@ def load_workspace_role(data: AuthData) -> None:
return
if data.caller is not None and getattr(data.caller, "status", None) != AccountStatus.ACTIVE:
return
with session_factory.create_session() as session:
role = TenantService.get_account_role_in_tenant(str(data.account_id), str(data.tenant.id), session=session)
role = TenantService.get_account_role_in_tenant(str(data.account_id), str(data.tenant.id), session=db.session())
if role is None:
return
data.tenant_role = role
+31 -47
View File
@@ -15,11 +15,9 @@ from itertools import starmap
from urllib import parse
from flask_restx import Resource
from sqlalchemy.orm import Session
from werkzeug.exceptions import BadRequest, NotFound
from configs import dify_config
from controllers.common.session import with_session
from controllers.openapi import openapi_ns
from controllers.openapi._contract import accepts, returns
from controllers.openapi._errors import MemberLicenseExceeded, MemberLimitExceeded
@@ -37,6 +35,7 @@ from controllers.openapi._models import (
)
from controllers.openapi.auth.composition import auth_router
from controllers.openapi.auth.data import AuthData
from extensions.ext_database import db
from libs.oauth_bearer import Scope, TokenType
from models import Account, Tenant, TenantAccountJoin
from models.account import TenantAccountRole, TenantStatus
@@ -65,15 +64,15 @@ def _member_response(account: Account) -> MemberResponse:
)
def _load_tenant(session: Session, workspace_id: str) -> Tenant:
tenant = TenantService.get_tenant_by_id(workspace_id, session=session)
def _load_tenant(workspace_id: str) -> Tenant:
tenant = TenantService.get_tenant_by_id(workspace_id, session=db.session())
if tenant is None or tenant.status != TenantStatus.NORMAL:
raise NotFound("workspace not found")
return tenant
def _load_account(session: Session, account_id: object) -> Account:
account = AccountService.get_account_by_id(str(account_id), session=session) if account_id else None
def _load_account(account_id: object) -> Account:
account = AccountService.get_account_by_id(str(account_id), session=db.session()) if account_id else None
if account is None:
raise RuntimeError("authenticated account_id has no Account row")
return account
@@ -95,9 +94,8 @@ def _check_member_invite_quota(tenant_id: str) -> None:
class WorkspacesApi(Resource):
@auth_router.guard(scope=Scope.WORKSPACE_READ, allowed_token_types=frozenset({TokenType.OAUTH_ACCOUNT}))
@returns(200, WorkspaceListResponse, description="Workspace list")
@with_session(write=False)
def get(self, session: Session, *, auth_data: AuthData):
rows = TenantService.get_workspaces_for_account(str(auth_data.account_id), session=session)
def get(self, *, auth_data: AuthData):
rows = TenantService.get_workspaces_for_account(str(auth_data.account_id), session=db.session())
return WorkspaceListResponse(workspaces=list(starmap(_workspace_summary, rows)))
@@ -106,9 +104,8 @@ class WorkspacesApi(Resource):
class WorkspaceByIdApi(Resource):
@auth_router.guard(scope=Scope.WORKSPACE_READ, allowed_token_types=frozenset({TokenType.OAUTH_ACCOUNT}))
@returns(200, WorkspaceDetailResponse, description="Workspace detail")
@with_session(write=False)
def get(self, session: Session, workspace_id: str, *, auth_data: AuthData):
row = TenantService.find_workspace_for_account(str(auth_data.account_id), workspace_id, session=session)
def get(self, workspace_id: str, *, auth_data: AuthData):
row = TenantService.find_workspace_for_account(str(auth_data.account_id), workspace_id, session=db.session())
# 404 (not 403) on non-member so workspace IDs don't leak across tenants.
if row is None:
raise NotFound("workspace not found")
@@ -128,16 +125,15 @@ class WorkspaceSwitchApi(Resource):
@auth_router.guard_workspace(scope=Scope.WORKSPACE_READ, allowed_token_types=frozenset({TokenType.OAUTH_ACCOUNT}))
@returns(200, WorkspaceDetailResponse, description="Workspace detail")
@with_session
def post(self, session: Session, workspace_id: str, *, auth_data: AuthData):
account = _load_account(session, auth_data.account_id)
def post(self, workspace_id: str, *, auth_data: AuthData):
account = _load_account(auth_data.account_id)
try:
TenantService.switch_tenant(account, workspace_id, session=session)
TenantService.switch_tenant(account, workspace_id, session=db.session())
except AccountNotLinkTenantError:
raise NotFound("workspace not found")
row = TenantService.find_workspace_for_account(str(auth_data.account_id), workspace_id, session=session)
row = TenantService.find_workspace_for_account(str(auth_data.account_id), workspace_id, session=db.session())
if row is None:
raise NotFound("workspace not found")
tenant, membership = row
@@ -155,10 +151,9 @@ class WorkspaceMembersApi(Resource):
@auth_router.guard_workspace(scope=Scope.WORKSPACE_READ, allowed_token_types=frozenset({TokenType.OAUTH_ACCOUNT}))
@returns(200, MemberListResponse, description="Member list")
@accepts(query=MemberListQuery)
@with_session(write=False)
def get(self, session: Session, workspace_id: str, *, auth_data: AuthData, query: MemberListQuery):
tenant = _load_tenant(session, workspace_id)
members = TenantService.get_tenant_members(tenant, session=session)
def get(self, workspace_id: str, *, auth_data: AuthData, query: MemberListQuery):
tenant = _load_tenant(workspace_id)
members = TenantService.get_tenant_members(tenant, session=db.session())
total = len(members)
start = (query.page - 1) * query.limit
page_items = members[start : start + query.limit]
@@ -177,10 +172,9 @@ class WorkspaceMembersApi(Resource):
)
@returns(201, MemberInviteResponse, description="Member invited")
@accepts(body=MemberInvitePayload)
@with_session
def post(self, session: Session, workspace_id: str, *, auth_data: AuthData, body: MemberInvitePayload):
inviter = _load_account(session, auth_data.account_id)
tenant = _load_tenant(session, workspace_id)
def post(self, workspace_id: str, *, auth_data: AuthData, body: MemberInvitePayload):
inviter = _load_account(auth_data.account_id)
tenant = _load_tenant(workspace_id)
_check_member_invite_quota(str(tenant.id))
@@ -191,7 +185,7 @@ class WorkspaceMembersApi(Resource):
language=None,
role=body.role,
inviter=inviter,
session=session,
session=db.session(),
)
except AccountAlreadyInTenantError as exc:
raise BadRequest(str(exc))
@@ -203,7 +197,7 @@ class WorkspaceMembersApi(Resource):
raise BadRequest(str(exc))
normalized_email = body.email.lower()
member = AccountService.get_account_by_email_with_case_fallback(normalized_email, session=session)
member = AccountService.get_account_by_email_with_case_fallback(normalized_email, session=db.session())
if member is None:
# invite_new_member just created or fetched this account.
raise RuntimeError("invited member missing from DB after invite")
@@ -235,16 +229,15 @@ class WorkspaceMemberApi(Resource):
allowed_roles=frozenset({TenantAccountRole.OWNER, TenantAccountRole.ADMIN}),
)
@returns(200, MemberActionResponse, description="Member removed")
@with_session
def delete(self, session: Session, workspace_id: str, member_id: str, *, auth_data: AuthData):
operator = _load_account(session, auth_data.account_id)
tenant = _load_tenant(session, workspace_id)
member = AccountService.get_account_by_id(member_id, session=session)
def delete(self, workspace_id: str, member_id: str, *, auth_data: AuthData):
operator = _load_account(auth_data.account_id)
tenant = _load_tenant(workspace_id)
member = AccountService.get_account_by_id(member_id, session=db.session())
if member is None:
raise NotFound("member not found")
try:
TenantService.remove_member_from_tenant(tenant, member, operator, session=session)
TenantService.remove_member_from_tenant(tenant, member, operator, session=db.session())
except CannotOperateSelfError as exc:
raise BadRequest(str(exc))
except NoPermissionError as exc:
@@ -261,24 +254,15 @@ class WorkspaceMemberApi(Resource):
)
@returns(200, MemberActionResponse, description="Role updated")
@accepts(body=MemberRoleUpdatePayload)
@with_session
def patch(
self,
session: Session,
workspace_id: str,
member_id: str,
*,
auth_data: AuthData,
body: MemberRoleUpdatePayload,
):
operator = _load_account(session, auth_data.account_id)
tenant = _load_tenant(session, workspace_id)
member = AccountService.get_account_by_id(member_id, session=session)
def patch(self, workspace_id: str, member_id: str, *, auth_data: AuthData, body: MemberRoleUpdatePayload):
operator = _load_account(auth_data.account_id)
tenant = _load_tenant(workspace_id)
member = AccountService.get_account_by_id(member_id, session=db.session())
if member is None:
raise NotFound("member not found")
try:
TenantService.update_member_role(tenant, member, body.role, operator, session=session)
TenantService.update_member_role(tenant, member, body.role, operator, session=db.session())
except CannotOperateSelfError as exc:
raise BadRequest(str(exc))
except NoPermissionError as exc:
+11 -14
View File
@@ -5,13 +5,12 @@ from flask import request
from flask_restx import Resource
from flask_restx.api import HTTPStatus
from pydantic import BaseModel, Field, TypeAdapter
from sqlalchemy.orm import Session
from controllers.common.schema import query_params_from_model, register_response_schema_models, register_schema_models
from controllers.common.session import with_session
from controllers.console.wraps import edit_permission_required
from controllers.service_api import service_api_ns
from controllers.service_api.wraps import validate_app_token
from extensions.ext_database import db
from extensions.ext_redis import redis_client
from fields.annotation_fields import (
Annotation,
@@ -206,13 +205,12 @@ class AnnotationListApi(Resource):
service_api_ns.models[AnnotationList.__name__],
)
@validate_app_token
@with_session(write=False)
def get(self, session: Session, app_model: App):
def get(self, app_model: App):
"""List annotations for the application."""
query = AnnotationListQuery.model_validate(request.args.to_dict(flat=True))
annotation_list, total = AppAnnotationService.get_annotation_list_by_app_id(
app_model.id, query.page, query.limit, query.keyword, session
app_model.id, query.page, query.limit, query.keyword, session=db.session()
)
annotation_models = TypeAdapter(list[Annotation]).validate_python(annotation_list, from_attributes=True)
return AnnotationList(
@@ -249,12 +247,13 @@ class AnnotationListApi(Resource):
service_api_ns.models[Annotation.__name__],
)
@validate_app_token
@with_session
def post(self, session: Session, app_model: App):
def post(self, app_model: App):
"""Create a new annotation."""
payload = AnnotationCreatePayload.model_validate(service_api_ns.payload or {})
insert_args: InsertAnnotationArgs = {"question": payload.question, "answer": payload.answer}
annotation = AppAnnotationService.insert_app_annotation_directly(insert_args, app_model.id, session)
annotation = AppAnnotationService.insert_app_annotation_directly(
insert_args, app_model.id, session=db.session()
)
return dump_response(Annotation, annotation), HTTPStatus.CREATED
@@ -288,15 +287,14 @@ class AnnotationUpdateDeleteApi(Resource):
service_api_ns.models[Annotation.__name__],
)
@validate_app_token
@with_session
@edit_permission_required
def put(self, session: Session, app_model: App, annotation_id: UUID):
def put(self, app_model: App, annotation_id: UUID):
"""Update an existing annotation."""
payload = AnnotationCreatePayload.model_validate(service_api_ns.payload or {})
update_args: UpdateAnnotationArgs = {"question": payload.question, "answer": payload.answer}
app_ref = AppRefService.create_app_ref(app_model)
annotation_ref = AppRefService.create_annotation_ref(app_ref, str(annotation_id))
annotation = AppAnnotationService.update_app_annotation_directly(update_args, annotation_ref, session)
annotation = AppAnnotationService.update_app_annotation_directly(update_args, annotation_ref, db.session())
return dump_response(Annotation, annotation)
@service_api_ns.doc(
@@ -321,11 +319,10 @@ class AnnotationUpdateDeleteApi(Resource):
}
)
@validate_app_token
@with_session
@edit_permission_required
def delete(self, session: Session, app_model: App, annotation_id: UUID):
def delete(self, app_model: App, annotation_id: UUID):
"""Delete an annotation."""
app_ref = AppRefService.create_app_ref(app_model)
annotation_ref = AppRefService.create_annotation_ref(app_ref, str(annotation_id))
AppAnnotationService.delete_app_annotation(annotation_ref, session)
AppAnnotationService.delete_app_annotation(annotation_ref, db.session())
return "", 204
+7 -20
View File
@@ -2,7 +2,6 @@ from typing import Any, cast
from flask_restx import Resource
from pydantic import Field
from sqlalchemy.orm import Session
from controllers.common.agent_app_parameters import get_published_agent_app_feature_dict_and_user_input_form
from controllers.common.fields import Parameters
@@ -14,7 +13,7 @@ from core.app.app_config.common.parameters_mapping import get_parameters_from_fe
from core.app.apps.agent_app.errors import AgentAppGeneratorError, AgentAppNotPublishedError
from extensions.ext_database import db
from fields.base import ResponseModel
from models.model import App, AppMode, load_annotation_reply_config
from models.model import App, AppMode
from services.app_service import AppService
@@ -33,13 +32,9 @@ class AppMetaResponse(ResponseModel):
register_response_schema_models(service_api_ns, Parameters, AppMetaResponse, AppInfoResponse)
def _get_agent_app_feature_dict_and_user_input_form(
app_model: App,
*,
session: Session,
) -> tuple[dict[str, Any], list[dict[str, Any]]]:
def _get_agent_app_feature_dict_and_user_input_form(app_model: App) -> tuple[dict[str, Any], list[dict[str, Any]]]:
try:
return get_published_agent_app_feature_dict_and_user_input_form(app_model, session=session)
return get_published_agent_app_feature_dict_and_user_input_form(app_model)
except AgentAppNotPublishedError:
raise AgentNotPublishedError()
except AgentAppGeneratorError:
@@ -78,31 +73,23 @@ class AppParameterApi(Resource):
Returns the input form parameters and configuration for the application.
"""
session = db.session()
features_dict: dict[str, Any]
user_input_form: list[dict[str, Any]]
if app_model.mode == AppMode.AGENT:
features_dict, user_input_form = _get_agent_app_feature_dict_and_user_input_form(
app_model,
session=session,
)
features_dict, user_input_form = _get_agent_app_feature_dict_and_user_input_form(app_model)
elif app_model.mode in {AppMode.ADVANCED_CHAT, AppMode.WORKFLOW}:
workflow = app_model.workflow_with_session(session=session)
workflow = app_model.workflow
if workflow is None:
raise AppUnavailableError()
features_dict = workflow.features_dict
user_input_form = workflow.user_input_form(to_old_structure=True)
else:
app_model_config = app_model.app_model_config_with_session(session=session)
app_model_config = app_model.app_model_config
if app_model_config is None:
raise AppUnavailableError()
annotation_reply = load_annotation_reply_config(session, app_model.id)
features_dict = cast(
dict[str, Any],
app_model_config.to_dict(annotation_reply=annotation_reply),
)
features_dict = cast(dict[str, Any], app_model_config.to_dict())
user_input_form = features_dict.get("user_input_form", [])
+2 -7
View File
@@ -101,15 +101,10 @@ class AudioApi(Resource):
Accepts an audio file upload and returns the transcribed text.
"""
file = request.files.get("file")
file = request.files["file"]
try:
response = AudioService.transcript_asr(
app_model=app_model,
file=file,
session=db.session(),
end_user=end_user.id,
)
response = AudioService.transcript_asr(app_model=app_model, file=file, end_user=end_user.id)
return dump_response(AudioTranscriptResponse, response)
except services.errors.app_model_config.AppModelConfigBrokenError:

Some files were not shown because too many files have changed in this diff Show More