Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
ee431ef0ae |
@@ -32,11 +32,12 @@ Keep this skill focused on Cucumber, Playwright, and package-level E2E guidance.
|
||||
- `e2e/` uses Cucumber for scenarios and Playwright as the browser layer.
|
||||
- `DifyWorld` is the per-scenario context object. Type `this` as `DifyWorld` and use `async function`, not arrow functions.
|
||||
- Keep glue organized by capability under `e2e/features/step-definitions/`; use `common/` only for broadly reusable steps.
|
||||
- Treat `e2e/AGENTS.md`, `features/support/hooks.ts`, and the Cucumber configuration as the owners of current session and tag semantics. Verify them when behavior depends on session state instead of copying a tag inventory into this skill.
|
||||
- Browser session behavior comes from `features/support/hooks.ts`:
|
||||
- default: authenticated session with shared storage state
|
||||
- `@unauthenticated`: clean browser context
|
||||
- `@authenticated`: readability/selective-run tag only unless implementation changes
|
||||
- `@fresh`: only for `e2e:full*` flows
|
||||
- Do not import Playwright Test runner patterns that bypass the current Cucumber + `DifyWorld` architecture unless the task is explicitly about changing that architecture.
|
||||
- Perform the behavior under test through Playwright. APIs are allowed for setup, seed preparation, persistence polling, and cleanup, but ordinary Console JSON and representable multipart operations must use the scenario- or process-owned generated oRPC client with request and response validation enabled. Keep the setup/cleanup API identity independent from an unauthenticated or logged-out behavior browser.
|
||||
- Consume generated operations directly. Do not add one-to-one API wrappers, handwritten endpoint URLs, response DTO casts, duplicate schemas, global mutable clients, or TanStack Query caching in Cucumber. Keep helpers only for real fixture construction, multi-operation orchestration, invariants, polling, derived test views, or protocol adapters.
|
||||
- Keep SSE, binary, redirect-only, external-service, and readiness exceptions centralized under their protocol owner. A contract mismatch must fail and be fixed at the backend schema owner followed by regeneration; never weaken validation to make E2E pass.
|
||||
|
||||
## Workflow
|
||||
|
||||
@@ -65,7 +66,7 @@ Keep this skill focused on Cucumber, Playwright, and package-level E2E guidance.
|
||||
- If a product element has real user-facing semantics but no accessible name, prefer fixing that accessible contract over adding a test id.
|
||||
5. Validate narrowly.
|
||||
- Run the narrowest tagged scenario or flow that exercises the change.
|
||||
- Run the package-required static checks documented in `e2e/AGENTS.md`.
|
||||
- Run `vpr lint --fix --quiet` from the repository root and `pnpm -C e2e type-check`.
|
||||
- Broaden verification only when the change affects hooks, tags, setup, or shared step semantics.
|
||||
|
||||
## Review Checklist
|
||||
@@ -76,8 +77,6 @@ Keep this skill focused on Cucumber, Playwright, and package-level E2E guidance.
|
||||
- Are locators user-facing and assertions web-first?
|
||||
- Does the change introduce hidden coupling across scenarios, tags, or instance state?
|
||||
- Does it document or implement behavior that differs from the real hooks or configuration?
|
||||
- Does setup/cleanup use the generated client directly, with any remaining helper owning more than a one-to-one endpoint forward?
|
||||
- Is every raw HTTP call a documented protocol or infrastructure exception rather than an ordinary Console operation?
|
||||
|
||||
Lead findings with correctness, flake risk, and architecture drift.
|
||||
|
||||
|
||||
@@ -1,15 +1,15 @@
|
||||
---
|
||||
name: frontend-testing
|
||||
description: Write, update, or review Dify frontend tests using Vitest and Testing Library. Trigger for frontend specs, test coverage requests, regressions, testability, or testing strategy under web/ or packages/dify-ui/.
|
||||
description: Write, update, or review Dify frontend tests using Vitest and React Testing Library. Trigger for frontend specs, test coverage requests, regressions, testability, or testing strategy under web/.
|
||||
---
|
||||
|
||||
# Dify Frontend Testing
|
||||
|
||||
Use this skill for Vitest work under `web/` and `packages/dify-ui/`. Do not use it for Python tests or Cucumber/Playwright tests under `e2e/`.
|
||||
Use this skill for Vitest and React Testing Library work under `web/`. Do not use it for Python tests or Cucumber/Playwright tests under `e2e/`.
|
||||
|
||||
## Required Source
|
||||
|
||||
Before writing, changing, or reviewing frontend tests, read `web/docs/test.md` completely. It is the single source of truth. This skill provides an execution checklist and must not redefine or extend that policy.
|
||||
Before writing, changing, or reviewing frontend tests, read `web/docs/test.md` completely. It is the single source of truth. This skill defines the execution workflow and must not add requirements that conflict with or duplicate that guide.
|
||||
|
||||
## Workflow
|
||||
|
||||
@@ -22,14 +22,10 @@ Before writing, changing, or reviewing frontend tests, read `web/docs/test.md` c
|
||||
|
||||
When reviewing existing tests, recommend deleting low-value tests as readily as adding missing behavior coverage.
|
||||
|
||||
Run focused tests from the owning workspace:
|
||||
Run focused tests from `web/`:
|
||||
|
||||
```bash
|
||||
# web/
|
||||
vp test run path/to/spec-or-directory
|
||||
|
||||
# packages/dify-ui/
|
||||
vp test run --project unit src/path/to/spec
|
||||
```
|
||||
|
||||
Run Dify UI Storybook tests with `vp test --project storybook --run`. Run broader checks only after the focused behavior passes.
|
||||
Run broader checks only after the focused behavior passes.
|
||||
|
||||
+44
-17
@@ -8,6 +8,7 @@
|
||||
|
||||
# Lint bulk suppression baselines.
|
||||
/oxlint-suppressions.json
|
||||
/eslint-suppressions.json
|
||||
|
||||
# CODEOWNERS file
|
||||
/.github/CODEOWNERS @laipz8200 @crazywoola
|
||||
@@ -32,9 +33,31 @@
|
||||
# Backend (default owner, more specific rules below will override)
|
||||
/api/ @QuantumGhost
|
||||
|
||||
# Backend - MCP
|
||||
/api/core/mcp/ @Nov1c444
|
||||
/api/core/entities/mcp_provider.py @Nov1c444
|
||||
/api/services/tools/mcp_tools_manage_service.py @Nov1c444
|
||||
/api/controllers/mcp/ @Nov1c444
|
||||
/api/controllers/console/app/mcp_server.py @Nov1c444
|
||||
|
||||
# Backend - Tests
|
||||
/api/tests/ @laipz8200 @QuantumGhost
|
||||
|
||||
/api/tests/**/*mcp* @Nov1c444
|
||||
|
||||
# Backend - Workflow - Engine (Core graph execution engine)
|
||||
/api/core/workflow/graph_engine/ @laipz8200 @QuantumGhost
|
||||
/api/core/workflow/runtime/ @laipz8200 @QuantumGhost
|
||||
/api/core/workflow/graph/ @laipz8200 @QuantumGhost
|
||||
/api/core/workflow/graph_events/ @laipz8200 @QuantumGhost
|
||||
/api/core/workflow/node_events/ @laipz8200 @QuantumGhost
|
||||
|
||||
# Backend - Workflow - Nodes (Agent, Iteration, Loop, LLM)
|
||||
/api/core/workflow/nodes/agent/ @Nov1c444
|
||||
/api/core/workflow/nodes/iteration/ @Nov1c444
|
||||
/api/core/workflow/nodes/loop/ @Nov1c444
|
||||
/api/core/workflow/nodes/llm/ @Nov1c444
|
||||
|
||||
# Backend - RAG (Retrieval Augmented Generation)
|
||||
/api/core/rag/ @JohnJyong
|
||||
/api/services/rag_pipeline/ @JohnJyong
|
||||
@@ -88,6 +111,7 @@
|
||||
/api/core/app/layers/trigger_post_layer.py @CourTeous33
|
||||
/api/services/trigger/ @CourTeous33
|
||||
/api/models/trigger.py @CourTeous33
|
||||
/api/fields/workflow_trigger_fields.py @CourTeous33
|
||||
/api/repositories/workflow_trigger_log_repository.py @CourTeous33
|
||||
/api/repositories/sqlalchemy_workflow_trigger_log_repository.py @CourTeous33
|
||||
/api/libs/schedule_utils.py @CourTeous33
|
||||
@@ -112,11 +136,11 @@
|
||||
/api/controllers/console/billing/ @hj24 @zyssyz123
|
||||
|
||||
# Backend - Enterprise
|
||||
/api/configs/enterprise/ @GareArc
|
||||
/api/services/enterprise/ @GareArc
|
||||
/api/services/feature_service.py @GareArc
|
||||
/api/controllers/console/feature.py @GareArc
|
||||
/api/controllers/web/feature.py @GareArc
|
||||
/api/configs/enterprise/ @GarfieldDai @GareArc
|
||||
/api/services/enterprise/ @GarfieldDai @GareArc
|
||||
/api/services/feature_service.py @GarfieldDai @GareArc
|
||||
/api/controllers/console/feature.py @GarfieldDai @GareArc
|
||||
/api/controllers/web/feature.py @GarfieldDai @GareArc
|
||||
|
||||
# Backend - Database Migrations
|
||||
/api/migrations/ @snakevash @laipz8200 @MRZHUH
|
||||
@@ -129,6 +153,7 @@
|
||||
|
||||
# Frontend - Platform and Features
|
||||
/web/config/ @lyzno1
|
||||
/web/contract/ @lyzno1
|
||||
/web/env.ts @lyzno1
|
||||
/web/features/ @lyzno1
|
||||
/web/hooks/ @lyzno1
|
||||
@@ -187,6 +212,7 @@
|
||||
/web/app/components/rag-pipeline/store/ @iamjoel @zxhlyh
|
||||
|
||||
# Frontend - RAG - Documents List
|
||||
/web/app/components/datasets/documents/list.tsx @iamjoel @WTW0313
|
||||
/web/app/components/datasets/documents/create-from-pipeline/ @iamjoel @WTW0313
|
||||
|
||||
# Frontend - RAG - Segments List
|
||||
@@ -205,22 +231,22 @@
|
||||
/web/app/components/plugins/marketplace/ @iamjoel @Yessenia-d
|
||||
|
||||
# Frontend - Login and Registration
|
||||
/web/app/signin/ @iamjoel
|
||||
/web/app/signup/ @iamjoel
|
||||
/web/app/reset-password/ @iamjoel
|
||||
/web/app/install/ @iamjoel
|
||||
/web/app/init/ @iamjoel
|
||||
/web/app/forgot-password/ @iamjoel
|
||||
/web/app/account/ @iamjoel
|
||||
/web/app/signin/ @douxc @iamjoel
|
||||
/web/app/signup/ @douxc @iamjoel
|
||||
/web/app/reset-password/ @douxc @iamjoel
|
||||
/web/app/install/ @douxc @iamjoel
|
||||
/web/app/init/ @douxc @iamjoel
|
||||
/web/app/forgot-password/ @douxc @iamjoel
|
||||
/web/app/account/ @douxc @iamjoel
|
||||
|
||||
# Frontend - Service Authentication
|
||||
/web/service/base.ts @iamjoel
|
||||
/web/service/base.ts @douxc @iamjoel
|
||||
|
||||
# Frontend - WebApp Authentication and Access Control
|
||||
/web/app/(shareLayout)/components/ @iamjoel
|
||||
/web/app/(shareLayout)/webapp-signin/ @iamjoel
|
||||
/web/app/(shareLayout)/webapp-reset-password/ @iamjoel
|
||||
/web/app/components/app/app-access-control/ @iamjoel
|
||||
/web/app/(shareLayout)/components/ @douxc @iamjoel
|
||||
/web/app/(shareLayout)/webapp-signin/ @douxc @iamjoel
|
||||
/web/app/(shareLayout)/webapp-reset-password/ @douxc @iamjoel
|
||||
/web/app/components/app/app-access-control/ @douxc @iamjoel
|
||||
|
||||
# Frontend - Explore Page
|
||||
/web/app/components/explore/ @CodingOnStar @iamjoel
|
||||
@@ -239,6 +265,7 @@
|
||||
/web/app/components/base/**/*.spec.tsx @hyoban @CodingOnStar
|
||||
|
||||
# Frontend - Utils and Hooks
|
||||
/web/utils/classnames.ts @iamjoel @zxhlyh
|
||||
/web/utils/time.ts @iamjoel @zxhlyh
|
||||
/web/utils/format.ts @iamjoel @zxhlyh
|
||||
/web/utils/clipboard.ts @iamjoel @zxhlyh
|
||||
|
||||
@@ -47,6 +47,9 @@ jobs:
|
||||
- name: Install dependencies
|
||||
run: uv sync --project api --dev
|
||||
|
||||
- name: Run dify config tests
|
||||
run: uv run --project api pytest api/tests/unit_tests/configs/test_env_consistency.py
|
||||
|
||||
- name: Run Unit Tests
|
||||
run: |
|
||||
uv run --project api pytest \
|
||||
|
||||
@@ -27,7 +27,7 @@ jobs:
|
||||
steps:
|
||||
- id: skip_check
|
||||
continue-on-error: true
|
||||
uses: fkirc/skip-duplicate-actions@b974a9395958c231af965b70070979a577efa578 # v5.3.2
|
||||
uses: fkirc/skip-duplicate-actions@f75f66ce1886f00957d99748a42c724f4330bdcf # v5.3.1
|
||||
with:
|
||||
cancel_others: 'true'
|
||||
concurrent_skipping: same_content_newer
|
||||
@@ -72,7 +72,6 @@ jobs:
|
||||
- 'docker/volumes/sandbox/conf/**'
|
||||
cli:
|
||||
- 'cli/**'
|
||||
- 'packages/contracts/**'
|
||||
- 'packages/tsconfig/**'
|
||||
- 'package.json'
|
||||
- 'pnpm-lock.yaml'
|
||||
@@ -81,6 +80,7 @@ jobs:
|
||||
- '.npmrc'
|
||||
- '.nvmrc'
|
||||
- '.github/workflows/cli-tests.yml'
|
||||
- '.github/workflows/cli-docker-build.yml'
|
||||
- '.github/actions/setup-web/**'
|
||||
web:
|
||||
- 'web/**'
|
||||
@@ -105,7 +105,6 @@ jobs:
|
||||
- 'docker/docker-compose.middleware.yaml'
|
||||
- 'docker/envs/middleware.env.example'
|
||||
- '.github/workflows/web-e2e.yml'
|
||||
- '.github/workflows/main-ci.yml'
|
||||
- '.github/actions/setup-web/**'
|
||||
vdb:
|
||||
- 'api/core/rag/datasource/**'
|
||||
@@ -335,8 +334,6 @@ jobs:
|
||||
- check-changes
|
||||
if: needs.pre_job.outputs.should_skip != 'true' && needs.check-changes.outputs.e2e-changed == 'true'
|
||||
uses: ./.github/workflows/web-e2e.yml
|
||||
with:
|
||||
run-external-runtime: false
|
||||
secrets: inherit
|
||||
|
||||
web-e2e-skip:
|
||||
|
||||
@@ -26,30 +26,19 @@ jobs:
|
||||
external_e2e:
|
||||
- 'e2e/features/agent-v2/**'
|
||||
- 'e2e/features/step-definitions/agent-v2/**'
|
||||
- 'e2e/features/step-definitions/common/**'
|
||||
- 'e2e/features/support/**'
|
||||
- 'e2e/fixtures/auth.ts'
|
||||
- 'e2e/fixtures/test-materials/**'
|
||||
- 'e2e/scripts/**'
|
||||
- 'e2e/support/**'
|
||||
- 'e2e/cucumber.config.ts'
|
||||
- 'e2e/package.json'
|
||||
- 'e2e/test-env.ts'
|
||||
- 'e2e/tsconfig.json'
|
||||
- 'e2e/tsx-register.js'
|
||||
- 'package.json'
|
||||
- 'pnpm-lock.yaml'
|
||||
- '.nvmrc'
|
||||
- '.github/workflows/post-merge.yml'
|
||||
- '.github/workflows/web-e2e.yml'
|
||||
- '.github/actions/setup-web/**'
|
||||
- 'docker/docker-compose.middleware.yaml'
|
||||
- 'docker/envs/middleware.env.example'
|
||||
- 'dify-agent/**'
|
||||
- 'dify-agent-runtime/**'
|
||||
- 'api/pyproject.toml'
|
||||
- 'api/uv.lock'
|
||||
- 'api/tests/integration_tests/.env.example'
|
||||
- 'api/clients/agent_backend/**'
|
||||
- 'api/core/app/apps/agent_app/**'
|
||||
- 'api/core/workflow/nodes/agent_v2/**'
|
||||
@@ -59,13 +48,8 @@ jobs:
|
||||
- 'api/services/plugin/**'
|
||||
- 'api/core/tools/**'
|
||||
- 'api/services/tools/**'
|
||||
- 'packages/contracts/package.json'
|
||||
- 'packages/contracts/generated/api/console/agent/**'
|
||||
- 'packages/contracts/generated/api/console/apps/**'
|
||||
- 'packages/contracts/generated/api/console/datasets/**'
|
||||
- 'packages/contracts/generated/api/console/orpc.gen.ts'
|
||||
- 'packages/contracts/generated/api/console/workspaces/**'
|
||||
- 'packages/contracts/generated/api/service/**'
|
||||
- 'web/features/agent-v2/**'
|
||||
- 'web/app/(commonLayout)/agents/**'
|
||||
- 'web/app/(commonLayout)/@detailSidebar/agents/**'
|
||||
|
||||
@@ -27,7 +27,7 @@ jobs:
|
||||
persist-credentials: false
|
||||
|
||||
- name: Setup Go
|
||||
uses: actions/setup-go@b7ad1dad31e06c5925ef5d2fc7ad053ef454303e # v7.0.0
|
||||
uses: actions/setup-go@d35c59abb061a4a6fb18e82ac0862c26744d6ab5 # v5.5.0
|
||||
with:
|
||||
go-version-file: dify-agent-runtime/go.mod
|
||||
cache-dependency-path: dify-agent-runtime/go.sum
|
||||
@@ -51,13 +51,13 @@ jobs:
|
||||
persist-credentials: false
|
||||
|
||||
- name: Setup Go
|
||||
uses: actions/setup-go@b7ad1dad31e06c5925ef5d2fc7ad053ef454303e # v7.0.0
|
||||
uses: actions/setup-go@d35c59abb061a4a6fb18e82ac0862c26744d6ab5 # v5.5.0
|
||||
with:
|
||||
go-version-file: dify-agent-runtime/go.mod
|
||||
cache-dependency-path: dify-agent-runtime/go.sum
|
||||
|
||||
- name: Run golangci-lint
|
||||
uses: golangci/golangci-lint-action@ba0d7d2ec06a0ea1cb5fa41b2e4a3ab91d21278a # v6.5.0
|
||||
uses: golangci/golangci-lint-action@4afd733a84b1f43292c63897423277bb7f4313a9 # v6.5.0
|
||||
with:
|
||||
working-directory: dify-agent-runtime
|
||||
version: latest
|
||||
@@ -78,7 +78,7 @@ jobs:
|
||||
persist-credentials: false
|
||||
|
||||
- name: Setup Go
|
||||
uses: actions/setup-go@b7ad1dad31e06c5925ef5d2fc7ad053ef454303e # v7.0.0
|
||||
uses: actions/setup-go@d35c59abb061a4a6fb18e82ac0862c26744d6ab5 # v5.5.0
|
||||
with:
|
||||
go-version-file: dify-agent-runtime/go.mod
|
||||
cache-dependency-path: dify-agent-runtime/go.sum
|
||||
|
||||
@@ -29,7 +29,7 @@ jobs:
|
||||
persist-credentials: false
|
||||
|
||||
- name: Use Node.js
|
||||
uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0
|
||||
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0
|
||||
with:
|
||||
node-version: 22
|
||||
cache: ''
|
||||
|
||||
@@ -158,7 +158,7 @@ jobs:
|
||||
|
||||
- name: Run Claude Code for Translation Sync
|
||||
if: steps.context.outputs.CHANGED_FILES != ''
|
||||
uses: anthropics/claude-code-action@af0559ee4f514d1ef21826982bed13f7edc3c35e # v1.0.178
|
||||
uses: anthropics/claude-code-action@e90deca47693f9457b72f2b53c17d7c445a87342 # v1.0.171
|
||||
with:
|
||||
anthropic_api_key: ${{ secrets.ANTHROPIC_API_KEY }}
|
||||
github_token: ${{ secrets.GITHUB_TOKEN }}
|
||||
|
||||
@@ -4,9 +4,9 @@ on:
|
||||
workflow_call:
|
||||
inputs:
|
||||
run-external-runtime:
|
||||
description: Run only the prepared and external runtime suite instead of the core suites.
|
||||
required: true
|
||||
required: false
|
||||
type: boolean
|
||||
default: false
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
@@ -19,7 +19,6 @@ jobs:
|
||||
test:
|
||||
name: Web Full-Stack E2E
|
||||
runs-on: depot-ubuntu-24.04-4
|
||||
timeout-minutes: 120
|
||||
defaults:
|
||||
run:
|
||||
shell: bash
|
||||
@@ -46,7 +45,6 @@ jobs:
|
||||
run: uv sync --project api --dev
|
||||
|
||||
- name: Run E2E support unit tests
|
||||
if: ${{ !inputs.run-external-runtime }}
|
||||
working-directory: ./e2e
|
||||
run: vp run test:unit
|
||||
|
||||
@@ -55,7 +53,6 @@ jobs:
|
||||
run: vp run e2e:install
|
||||
|
||||
- name: Run isolated source-api and built-web Cucumber E2E tests
|
||||
if: ${{ !inputs.run-external-runtime }}
|
||||
working-directory: ./e2e
|
||||
env:
|
||||
E2E_ADMIN_EMAIL: e2e-admin@example.com
|
||||
@@ -65,53 +62,7 @@ jobs:
|
||||
E2E_INIT_PASSWORD: E2eInit12345
|
||||
run: vp run e2e:full
|
||||
|
||||
- name: Preserve Chromium E2E report and logs
|
||||
if: ${{ !cancelled() && !inputs.run-external-runtime }}
|
||||
run: |
|
||||
if [[ -d e2e/cucumber-report ]]; then
|
||||
mv e2e/cucumber-report e2e/cucumber-report-non-external
|
||||
fi
|
||||
if [[ -d e2e/.logs ]]; then
|
||||
mv e2e/.logs e2e/.logs-non-external
|
||||
fi
|
||||
|
||||
- name: Run WebKit keyboard and browser smoke tests
|
||||
if: ${{ !inputs.run-external-runtime }}
|
||||
working-directory: ./e2e
|
||||
env:
|
||||
E2E_ADMIN_EMAIL: e2e-admin@example.com
|
||||
E2E_ADMIN_NAME: E2E Admin
|
||||
E2E_ADMIN_PASSWORD: E2eAdmin12345
|
||||
E2E_BROWSER: webkit
|
||||
E2E_INIT_PASSWORD: E2eInit12345
|
||||
run: |
|
||||
teardown_webkit_smoke() {
|
||||
local run_status=$?
|
||||
trap - EXIT
|
||||
if ! vp run e2e:middleware:down; then
|
||||
echo "::error title=E2E teardown failed::WebKit smoke middleware did not shut down cleanly."
|
||||
if [[ "$run_status" -eq 0 ]]; then
|
||||
run_status=1
|
||||
fi
|
||||
fi
|
||||
exit "$run_status"
|
||||
}
|
||||
|
||||
trap teardown_webkit_smoke EXIT
|
||||
vp run e2e:middleware:up
|
||||
vp run e2e -- --tags '@browser-smoke'
|
||||
|
||||
- name: Preserve WebKit E2E report and logs
|
||||
if: ${{ !cancelled() && !inputs.run-external-runtime }}
|
||||
run: |
|
||||
if [[ -d e2e/cucumber-report ]]; then
|
||||
mv e2e/cucumber-report e2e/cucumber-report-webkit
|
||||
fi
|
||||
if [[ -d e2e/.logs ]]; then
|
||||
mv e2e/.logs e2e/.logs-webkit
|
||||
fi
|
||||
|
||||
- name: Run prepared and external runtime E2E tests
|
||||
- name: Run external runtime E2E tests
|
||||
if: ${{ inputs.run-external-runtime }}
|
||||
working-directory: ./e2e
|
||||
env:
|
||||
@@ -121,10 +72,13 @@ jobs:
|
||||
E2E_AGENT_DECISION_MODEL_NAME: ${{ vars.E2E_AGENT_DECISION_MODEL_NAME || 'gpt-5.5' }}
|
||||
E2E_AGENT_DECISION_MODEL_PROVIDER: ${{ vars.E2E_AGENT_DECISION_MODEL_PROVIDER || 'openai' }}
|
||||
E2E_AGENT_DECISION_MODEL_TYPE: ${{ vars.E2E_AGENT_DECISION_MODEL_TYPE || 'llm' }}
|
||||
E2E_EXTERNAL_RUNTIME_SEED_SPECS: ${{ vars.E2E_EXTERNAL_RUNTIME_SEED_SPECS }}
|
||||
E2E_EXTERNAL_RUNTIME_TAGS: ${{ vars.E2E_EXTERNAL_RUNTIME_TAGS }}
|
||||
E2E_FORCE_WEB_BUILD: "1"
|
||||
E2E_INIT_PASSWORD: E2eInit12345
|
||||
E2E_MARKETPLACE_API_URL: ${{ vars.E2E_MARKETPLACE_API_URL }}
|
||||
E2E_MARKETPLACE_PLUGIN_IDS: ${{ vars.E2E_MARKETPLACE_PLUGIN_IDS }}
|
||||
E2E_MARKETPLACE_PLUGIN_UNIQUE_IDENTIFIERS: ${{ vars.E2E_MARKETPLACE_PLUGIN_UNIQUE_IDENTIFIERS }}
|
||||
E2E_MODEL_PROVIDER_CREDENTIALS_JSON: ${{ secrets.E2E_MODEL_PROVIDER_CREDENTIALS_JSON }}
|
||||
E2E_SPEECH_TO_TEXT_MODEL_NAME: ${{ vars.E2E_SPEECH_TO_TEXT_MODEL_NAME || 'gpt-4o-mini-transcribe' }}
|
||||
E2E_SPEECH_TO_TEXT_MODEL_PROVIDER: ${{ vars.E2E_SPEECH_TO_TEXT_MODEL_PROVIDER || 'openai' }}
|
||||
@@ -138,22 +92,20 @@ jobs:
|
||||
exit 1
|
||||
fi
|
||||
|
||||
teardown_external_runtime() {
|
||||
local run_status=$?
|
||||
trap - EXIT
|
||||
if ! vp run e2e:middleware:down; then
|
||||
echo "::error title=E2E teardown failed::External runtime middleware did not shut down cleanly."
|
||||
if [[ "$run_status" -eq 0 ]]; then
|
||||
run_status=1
|
||||
fi
|
||||
fi
|
||||
exit "$run_status"
|
||||
}
|
||||
if [[ -d cucumber-report ]]; then
|
||||
rm -rf cucumber-report-non-external
|
||||
mv cucumber-report cucumber-report-non-external
|
||||
fi
|
||||
|
||||
trap teardown_external_runtime EXIT
|
||||
if [[ -d .logs ]]; then
|
||||
rm -rf .logs-non-external
|
||||
mv .logs .logs-non-external
|
||||
fi
|
||||
|
||||
trap 'vp run e2e:middleware:down' EXIT
|
||||
vp run e2e:middleware:up
|
||||
vp run e2e:post-merge:prepare
|
||||
vp run e2e:post-merge
|
||||
vp run e2e:external:prepare
|
||||
vp run e2e:external
|
||||
|
||||
- name: Upload Cucumber report
|
||||
if: ${{ !cancelled() }}
|
||||
@@ -163,7 +115,6 @@ jobs:
|
||||
path: |
|
||||
e2e/cucumber-report
|
||||
e2e/cucumber-report-non-external
|
||||
e2e/cucumber-report-webkit
|
||||
retention-days: 7
|
||||
|
||||
- name: Upload E2E logs
|
||||
@@ -174,15 +125,5 @@ jobs:
|
||||
path: |
|
||||
e2e/.logs/*.log
|
||||
e2e/.logs-non-external/*.log
|
||||
e2e/.logs-webkit/*.log
|
||||
include-hidden-files: true
|
||||
retention-days: 7
|
||||
|
||||
- name: Upload E2E seed report
|
||||
if: ${{ !cancelled() && inputs.run-external-runtime }}
|
||||
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
|
||||
with:
|
||||
name: e2e-seed-report
|
||||
path: e2e/seed-report
|
||||
if-no-files-found: ignore
|
||||
retention-days: 7
|
||||
|
||||
@@ -17,6 +17,8 @@ jobs:
|
||||
test:
|
||||
name: Web Tests (${{ matrix.shardIndex }}/${{ matrix.shardTotal }})
|
||||
runs-on: depot-ubuntu-24.04-4
|
||||
env:
|
||||
VITEST_COVERAGE_SCOPE: app-components
|
||||
strategy:
|
||||
fail-fast: false
|
||||
matrix:
|
||||
|
||||
Vendored
+18
-4
@@ -5,9 +5,7 @@
|
||||
"name": "Python: API (gevent)",
|
||||
"type": "debugpy",
|
||||
"request": "launch",
|
||||
"module": "gevent.monkey",
|
||||
"args": ["--module", "app"],
|
||||
"gevent": true,
|
||||
"program": "${workspaceFolder}/api/app.py",
|
||||
"jinja": true,
|
||||
"justMyCode": true,
|
||||
"cwd": "${workspaceFolder}/api",
|
||||
@@ -35,6 +33,22 @@
|
||||
"justMyCode": false,
|
||||
"cwd": "${workspaceFolder}/api",
|
||||
"python": "${workspaceFolder}/api/.venv/bin/python"
|
||||
}
|
||||
},
|
||||
{
|
||||
"name": "Next.js: debug full stack",
|
||||
"type": "node",
|
||||
"request": "launch",
|
||||
"program": "${workspaceFolder}/web/node_modules/next/dist/bin/next",
|
||||
"runtimeArgs": ["--inspect"],
|
||||
"skipFiles": ["<node_internals>/**"],
|
||||
"serverReadyAction": {
|
||||
"action": "debugWithChrome",
|
||||
"killOnServerStop": true,
|
||||
"pattern": "- Local:.+(https?://.+)",
|
||||
"uriFormat": "%s",
|
||||
"webRoot": "${workspaceFolder}/web"
|
||||
},
|
||||
"cwd": "${workspaceFolder}/web"
|
||||
}
|
||||
]
|
||||
}
|
||||
|
||||
@@ -107,7 +107,6 @@ test:
|
||||
echo "Target: $(TARGET_TESTS)"; \
|
||||
uv run --project api --dev pytest $(TARGET_TESTS); \
|
||||
else \
|
||||
set -e; \
|
||||
echo "Running backend unit tests"; \
|
||||
uv run --project api --dev pytest -p no:benchmark --timeout "$${PYTEST_TIMEOUT:-20}" -n auto \
|
||||
api/tests/unit_tests \
|
||||
@@ -125,7 +124,6 @@ test-all:
|
||||
echo "Target: $(TARGET_TESTS)"; \
|
||||
uv run --project api --dev pytest $(TARGET_TESTS); \
|
||||
else \
|
||||
set -e; \
|
||||
echo "Running backend unit tests"; \
|
||||
uv run --project api --dev pytest -p no:benchmark --timeout "$${PYTEST_TIMEOUT:-20}" -n auto \
|
||||
api/tests/unit_tests \
|
||||
|
||||
@@ -15,7 +15,7 @@
|
||||
<a href="https://discord.gg/FngNHpbcY7" target="_blank">
|
||||
<img src="https://img.shields.io/discord/1082486657678311454?logo=discord&labelColor=%20%235462eb&logoColor=%20%23f5f5f5&color=%20%235462eb"
|
||||
alt="chat on Discord"></a>
|
||||
<a href="https://reddit.com/r/difyai" target="_blank">
|
||||
<a href="https://reddit.com/r/difyai" target="_blank">
|
||||
<img src="https://img.shields.io/reddit/subreddit-subscribers/difyai?style=plastic&logo=reddit&label=r%2Fdifyai&labelColor=white"
|
||||
alt="join Reddit"></a>
|
||||
<a href="https://twitter.com/intent/follow?screen_name=dify_ai" target="_blank">
|
||||
@@ -71,7 +71,7 @@ Dify is an open-source LLM app development platform. Its intuitive interface com
|
||||
|
||||
<br/>
|
||||
|
||||
The easiest way to start the Dify server is through [Docker Compose](docker/docker-compose.yaml). Before running Dify with the following commands, make sure that [Docker](https://docs.docker.com/get-docker/) and Docker Compose v2.24.0 or later are installed on your machine:
|
||||
The easiest way to start the Dify server is through [Docker Compose](docker/docker-compose.yaml). Before running Dify with the following commands, make sure that [Docker](https://docs.docker.com/get-docker/) and [Docker Compose](https://docs.docker.com/compose/install/) are installed on your machine:
|
||||
|
||||
```bash
|
||||
cd dify
|
||||
@@ -207,16 +207,9 @@ At the same time, please consider supporting Dify by sharing it on social media
|
||||
<img src="https://contrib.rocks/image?repo=langgenius/dify" />
|
||||
</a>
|
||||
|
||||
## Star History
|
||||
## Star history
|
||||
|
||||
<!-- GitHub token name: star-history -->
|
||||
<a href="https://www.star-history.com/?repos=langgenius%2Fdify&type=date&legend=top-left">
|
||||
<picture>
|
||||
<source media="(prefers-color-scheme: dark)" srcset="https://api.star-history.com/chart?repos=langgenius/dify&type=date&theme=dark&legend=top-left&sealed_token=p-SWD-UXZEDc5a2d0EMfgMmyCwVyRlSof0Qox68v7k4PvQPKRlCx0jDIlNztw7mbA6DEn96R50DojO9pCi5LQUlDBAoIhRswt8-GuC8K3rZQ3naJUXbuHqR_oItIW2F0NNM-7npevzw5SXp7L5mwixgqJncAvAGzuGq5zmhQfiDRSW_Jd6y8TQCZMJbt" />
|
||||
<source media="(prefers-color-scheme: light)" srcset="https://api.star-history.com/chart?repos=langgenius/dify&type=date&legend=top-left&sealed_token=p-SWD-UXZEDc5a2d0EMfgMmyCwVyRlSof0Qox68v7k4PvQPKRlCx0jDIlNztw7mbA6DEn96R50DojO9pCi5LQUlDBAoIhRswt8-GuC8K3rZQ3naJUXbuHqR_oItIW2F0NNM-7npevzw5SXp7L5mwixgqJncAvAGzuGq5zmhQfiDRSW_Jd6y8TQCZMJbt" />
|
||||
<img alt="Star History Chart" src="https://api.star-history.com/chart?repos=langgenius/dify&type=date&legend=top-left&sealed_token=p-SWD-UXZEDc5a2d0EMfgMmyCwVyRlSof0Qox68v7k4PvQPKRlCx0jDIlNztw7mbA6DEn96R50DojO9pCi5LQUlDBAoIhRswt8-GuC8K3rZQ3naJUXbuHqR_oItIW2F0NNM-7npevzw5SXp7L5mwixgqJncAvAGzuGq5zmhQfiDRSW_Jd6y8TQCZMJbt" />
|
||||
</picture>
|
||||
</a>
|
||||
[](https://star-history.com/#langgenius/dify&Date)
|
||||
|
||||
## Security disclosure
|
||||
|
||||
|
||||
@@ -561,8 +561,6 @@ WORKFLOW_MAX_EXECUTION_STEPS=500
|
||||
WORKFLOW_MAX_EXECUTION_TIME=1200
|
||||
WORKFLOW_CALL_MAX_DEPTH=5
|
||||
MAX_VARIABLE_SIZE=204800
|
||||
# Maximum concurrent node-builder LLM calls per workflow generation request
|
||||
WORKFLOW_GENERATOR_NODE_BUILDER_MAX_WORKERS=6
|
||||
|
||||
# GraphEngine Worker Pool Configuration
|
||||
# Minimum number of workers per GraphEngine instance (default: 1)
|
||||
@@ -667,27 +665,10 @@ PLUGIN_REMOTE_INSTALL_HOST=localhost
|
||||
PLUGIN_MAX_PACKAGE_SIZE=15728640
|
||||
PLUGIN_MODEL_SCHEMA_CACHE_TTL=3600
|
||||
PLUGIN_MODEL_PROVIDERS_CACHE_TTL=86400
|
||||
# Comma-separated marketplace plugin IDs whose latest versions are installed for newly registered users.
|
||||
# Example: langgenius/openai,langgenius/gemini
|
||||
NEW_USER_DEFAULT_PLUGIN_IDS=
|
||||
# Comma-separated model_type:provider:model entries assigned after default plugins finish installing.
|
||||
# Example: llm:langgenius/openai/openai:gpt-4o-mini,text-embedding:langgenius/openai/openai:text-embedding-3-small
|
||||
NEW_USER_DEFAULT_MODELS=
|
||||
INNER_API_KEY_FOR_PLUGIN=QaHbTe77CtuXmsfyhR7+vRjI/+XbV1AaFy691iy+kGDv2Jvy0/eAh8Y1
|
||||
|
||||
# Dify Agent backend
|
||||
AGENT_BACKEND_BASE_URL=http://localhost:5050
|
||||
AGENT_BACKEND_STREAM_READ_TIMEOUT_SECONDS=30
|
||||
AGENT_BACKEND_STREAM_MAX_RECONNECTS=3
|
||||
AGENT_BACKEND_RUN_TIMEOUT_SECONDS=1200
|
||||
|
||||
# KnowledgeFS (Dataset 2.0)
|
||||
KNOWLEDGE_FS_ENABLED=false
|
||||
KNOWLEDGE_FS_BASE_URL=
|
||||
# Shared with KnowledgeFS; use at least 32 random characters.
|
||||
KNOWLEDGE_FS_JWT_SECRET=
|
||||
KNOWLEDGE_FS_SSE_READ_TIMEOUT_SECONDS=300
|
||||
KNOWLEDGE_FS_TIMEOUT_SECONDS=10
|
||||
|
||||
# Marketplace configuration
|
||||
MARKETPLACE_ENABLED=true
|
||||
|
||||
@@ -109,18 +109,6 @@ class Example:
|
||||
- Reuse existing helpers in `core/`, `services/`, and `libs/` before creating new abstractions.
|
||||
- Optimise for observability: deterministic control flow, clear logging, actionable errors.
|
||||
|
||||
### Owner-Bound Resource References
|
||||
|
||||
- Resolve and validate the outer owner before binding a nested resource ID.
|
||||
- For stable single-parent chains, use immutable nested `NamedTuple` refs.
|
||||
- Root refs carry tenant plus root ID; child refs carry the parent ref.
|
||||
- In production, construct refs through the domain ref service.
|
||||
- Python allowing direct construction does not grant authorization.
|
||||
- Scope every consuming query with complete owner predicates; refs are not security tokens.
|
||||
- Keep polymorphic owners flat until explicit nominal owner types exist.
|
||||
- Do not add generic ref bases or compatibility fields only for uniformity.
|
||||
- Reconstruct internal refs from validated database state after payload or async boundaries.
|
||||
|
||||
### Logging & Errors
|
||||
|
||||
- Never use `print`; use a module-level logger:
|
||||
|
||||
-19
@@ -1,24 +1,5 @@
|
||||
from __future__ import annotations
|
||||
|
||||
# ``python -m app`` (docker DEBUG=true, or IDE debugging) serves through the
|
||||
# gevent pywsgi server at the bottom of this file, so the stdlib must be
|
||||
# monkey-patched BEFORE any other import pulls in sockets or locks. Without
|
||||
# this, every request runs as a greenlet on one OS thread while blocking
|
||||
# calls (LLM invokes, ``Future.result`` waits, DB I/O) pin that thread — the
|
||||
# whole process freezes until the call returns. Gunicorn and Celery apply
|
||||
# their own patching (see gunicorn.conf.py / celery_entrypoint.py), and
|
||||
# ``flask run`` uses real Werkzeug threads, so both skip this branch.
|
||||
if __name__ == "__main__":
|
||||
from gevent import monkey
|
||||
|
||||
monkey.patch_all()
|
||||
|
||||
import psycogreen.gevent as psycogreen_gevent
|
||||
from grpc.experimental import gevent as grpc_gevent
|
||||
|
||||
grpc_gevent.init_gevent()
|
||||
psycogreen_gevent.patch_psycopg()
|
||||
|
||||
import logging
|
||||
import sys
|
||||
from typing import TYPE_CHECKING, cast
|
||||
|
||||
@@ -8,7 +8,7 @@ creating another wire contract.
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from collections.abc import Callable, Iterator
|
||||
from collections.abc import Iterator
|
||||
from typing import Protocol
|
||||
|
||||
from dify_agent.client import (
|
||||
@@ -45,13 +45,7 @@ class AgentBackendRunClient(Protocol):
|
||||
def cancel_run(self, run_id: str, request: CancelRunRequest | None = None) -> CancelRunResponse:
|
||||
"""Request explicit cancellation for one Agent backend run."""
|
||||
|
||||
def stream_events(
|
||||
self,
|
||||
run_id: str,
|
||||
*,
|
||||
after: str | None = None,
|
||||
should_stop: Callable[[], bool] | None = None,
|
||||
) -> Iterator[RunEvent]:
|
||||
def stream_events(self, run_id: str, *, after: str | None = None) -> Iterator[RunEvent]:
|
||||
"""Yield public ``dify-agent`` run events in stream order."""
|
||||
|
||||
def wait_run(self, run_id: str, *, timeout_seconds: float | None = None) -> RunStatusResponse:
|
||||
@@ -67,15 +61,7 @@ class _DifyAgentSyncClient(Protocol):
|
||||
def cancel_run_sync(self, run_id: str, request: CancelRunRequest | None = None) -> CancelRunResponse:
|
||||
"""Cancel one run synchronously."""
|
||||
|
||||
def stream_events_sync(
|
||||
self,
|
||||
run_id: str,
|
||||
*,
|
||||
after: str | None = None,
|
||||
max_reconnects: int | None = None,
|
||||
timeout_seconds: float | None = None,
|
||||
should_stop: Callable[[], bool] | None = None,
|
||||
) -> Iterator[RunEvent]:
|
||||
def stream_events_sync(self, run_id: str, *, after: str | None = None) -> Iterator[RunEvent]:
|
||||
"""Stream run events synchronously."""
|
||||
|
||||
def wait_run_sync(self, run_id: str, *, timeout_seconds: float | None = None) -> RunStatusResponse:
|
||||
@@ -87,16 +73,8 @@ class DifyAgentBackendRunClient:
|
||||
|
||||
client: _DifyAgentSyncClient
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
client: _DifyAgentSyncClient,
|
||||
*,
|
||||
stream_max_reconnects: int = 3,
|
||||
stream_timeout_seconds: float = 1200,
|
||||
) -> None:
|
||||
def __init__(self, client: _DifyAgentSyncClient) -> None:
|
||||
self.client = client
|
||||
self._stream_max_reconnects = stream_max_reconnects
|
||||
self._stream_timeout_seconds = stream_timeout_seconds
|
||||
|
||||
def create_run(self, request: CreateRunRequest) -> CreateRunResponse:
|
||||
"""Create one run through ``POST /runs`` and normalize client exceptions."""
|
||||
@@ -112,22 +90,10 @@ class DifyAgentBackendRunClient:
|
||||
except Exception as exc:
|
||||
raise _normalize_dify_agent_error(exc) from exc
|
||||
|
||||
def stream_events(
|
||||
self,
|
||||
run_id: str,
|
||||
*,
|
||||
after: str | None = None,
|
||||
should_stop: Callable[[], bool] | None = None,
|
||||
) -> Iterator[RunEvent]:
|
||||
def stream_events(self, run_id: str, *, after: str | None = None) -> Iterator[RunEvent]:
|
||||
"""Stream run events from ``/events/sse`` with the wrapped client's reconnect policy."""
|
||||
try:
|
||||
yield from self.client.stream_events_sync(
|
||||
run_id,
|
||||
after=after,
|
||||
max_reconnects=self._stream_max_reconnects,
|
||||
timeout_seconds=self._stream_timeout_seconds,
|
||||
should_stop=should_stop,
|
||||
)
|
||||
yield from self.client.stream_events_sync(run_id, after=after)
|
||||
except Exception as exc:
|
||||
raise _normalize_dify_agent_error(exc) from exc
|
||||
|
||||
|
||||
@@ -13,17 +13,10 @@ def create_agent_backend_run_client(
|
||||
base_url: str | None = None,
|
||||
use_fake: bool = False,
|
||||
fake_scenario: str | FakeAgentBackendScenario = FakeAgentBackendScenario.SUCCESS,
|
||||
stream_read_timeout_seconds: float = 30,
|
||||
stream_max_reconnects: int = 3,
|
||||
stream_run_timeout_seconds: float = 1200,
|
||||
) -> AgentBackendRunClient:
|
||||
"""Create the API-side run client without hiding the ``dify-agent`` protocol."""
|
||||
if use_fake:
|
||||
return FakeAgentBackendRunClient(scenario=FakeAgentBackendScenario(fake_scenario))
|
||||
if base_url is None:
|
||||
raise ValueError("base_url is required when creating a real Agent backend client")
|
||||
return DifyAgentBackendRunClient(
|
||||
Client(base_url=base_url, stream_timeout=stream_read_timeout_seconds),
|
||||
stream_max_reconnects=stream_max_reconnects,
|
||||
stream_timeout_seconds=stream_run_timeout_seconds,
|
||||
)
|
||||
return DifyAgentBackendRunClient(Client(base_url=base_url))
|
||||
|
||||
@@ -7,7 +7,7 @@ separate ``agent-backend.v1`` event stream.
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from collections.abc import Callable, Iterator
|
||||
from collections.abc import Iterator
|
||||
from datetime import UTC, datetime
|
||||
from enum import StrEnum
|
||||
|
||||
@@ -69,17 +69,9 @@ class FakeAgentBackendRunClient:
|
||||
del request
|
||||
return CancelRunResponse(run_id=run_id, status="cancelled")
|
||||
|
||||
def stream_events(
|
||||
self,
|
||||
run_id: str,
|
||||
*,
|
||||
after: str | None = None,
|
||||
should_stop: Callable[[], bool] | None = None,
|
||||
) -> Iterator[RunEvent]:
|
||||
def stream_events(self, run_id: str, *, after: str | None = None) -> Iterator[RunEvent]:
|
||||
"""Yield the deterministic public ``RunEvent`` sequence for ``run_id``."""
|
||||
for event in self._events(run_id):
|
||||
if should_stop is not None and should_stop():
|
||||
return
|
||||
if after is not None and event.id is not None and event.id <= after:
|
||||
continue
|
||||
yield event
|
||||
|
||||
@@ -9,7 +9,6 @@ from sqlalchemy import delete, func, select
|
||||
from sqlalchemy.engine import CursorResult
|
||||
|
||||
from configs import dify_config
|
||||
from core.db.session_factory import session_factory
|
||||
from core.helper import encrypter
|
||||
from core.plugin.entities.plugin_daemon import CredentialType
|
||||
from core.plugin.impl.plugin import PluginInstaller
|
||||
@@ -579,6 +578,9 @@ def install_rag_pipeline_plugins(input_file, output_file, workers):
|
||||
"""
|
||||
click.echo(click.style("Installing rag pipeline plugins", fg="yellow"))
|
||||
plugin_migration = PluginMigration()
|
||||
with session_factory.create_session() as session:
|
||||
plugin_migration.install_rag_pipeline_plugins(input_file, output_file, workers, session=session)
|
||||
plugin_migration.install_rag_pipeline_plugins(
|
||||
input_file,
|
||||
output_file,
|
||||
workers,
|
||||
)
|
||||
click.echo(click.style("Installing rag pipeline plugins successfully", fg="green"))
|
||||
|
||||
+63
-209
@@ -1,8 +1,6 @@
|
||||
import datetime
|
||||
import logging
|
||||
import re
|
||||
import time
|
||||
import uuid
|
||||
from collections.abc import Callable
|
||||
from typing import TypedDict
|
||||
|
||||
@@ -23,7 +21,6 @@ from tasks.remove_app_and_related_data_task import delete_draft_variables_batch
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
_HEX_PREFIXES = tuple("0123456789abcdef")
|
||||
_TARGET_MONTH_PATTERN = re.compile(r"^\d{4}-(0[1-9]|1[0-2])$")
|
||||
|
||||
|
||||
class WorkflowRunArchivePlanRow(TypedDict):
|
||||
@@ -69,7 +66,6 @@ def _parse_tenant_prefixes(prefixes: str | None) -> list[str]:
|
||||
|
||||
|
||||
def _parse_comma_separated_ids(raw_ids: str | None, *, param_name: str) -> list[str] | None:
|
||||
"""Keep an omitted scope unset while rejecting an explicitly empty scope."""
|
||||
if raw_ids is None:
|
||||
return None
|
||||
parsed = sorted({raw_id.strip() for raw_id in raw_ids.split(",") if raw_id.strip()})
|
||||
@@ -78,27 +74,6 @@ def _parse_comma_separated_ids(raw_ids: str | None, *, param_name: str) -> list[
|
||||
return parsed
|
||||
|
||||
|
||||
def _parse_archive_target_month(target_month: str) -> tuple[int, int]:
|
||||
"""Validate the V2 catalog month selector and return its numeric components."""
|
||||
if not _TARGET_MONTH_PATTERN.fullmatch(target_month):
|
||||
raise click.BadParameter("target-month must use YYYY-MM format", param_hint="--target-month")
|
||||
year_text, month_text = target_month.split("-", maxsplit=1)
|
||||
return int(year_text), int(month_text)
|
||||
|
||||
|
||||
def _parse_archive_catalog_cursor(after_catalog_id: str | None) -> str | None:
|
||||
"""Normalize the exclusive V2 catalog keyset cursor when one is provided."""
|
||||
if after_catalog_id is None:
|
||||
return None
|
||||
try:
|
||||
return str(uuid.UUID(after_catalog_id))
|
||||
except ValueError as exc:
|
||||
raise click.BadParameter(
|
||||
"after-catalog-id must be a UUID returned by the same V2 operation and scope",
|
||||
param_hint="--after-catalog-id",
|
||||
) from exc
|
||||
|
||||
|
||||
def _get_archive_candidate_tenant_ids_by_prefix(
|
||||
session: Session,
|
||||
prefix: str,
|
||||
@@ -835,11 +810,9 @@ def backfill_workflow_run_archive_bundles(
|
||||
click.echo(click.style(f" ... and {len(summary.errors) - 10} more failures", fg="red"))
|
||||
|
||||
|
||||
def _echo_bundle_archive_operation_summary(summary, *, dry_run: bool) -> None:
|
||||
def _echo_bundle_archive_operation_summary(summary) -> None:
|
||||
status = "completed successfully" if summary.bundles_failed == 0 else "completed with failures"
|
||||
fg = "green" if summary.bundles_failed == 0 else "red"
|
||||
cursor_label = "preview_next_catalog_id" if dry_run else "next_catalog_id"
|
||||
cursor_value = summary.preview_next_catalog_id if dry_run else summary.next_catalog_id
|
||||
click.echo(
|
||||
click.style(
|
||||
f"{summary.operation} {status}. "
|
||||
@@ -848,12 +821,10 @@ def _echo_bundle_archive_operation_summary(summary, *, dry_run: bool) -> None:
|
||||
f"archive_bytes={summary.archive_bytes} duration={summary.elapsed_time:.2f}s "
|
||||
f"validation_time={summary.validation_time:.2f}s "
|
||||
f"runs_per_second={summary.runs_per_second:.2f} rows_per_second={summary.rows_per_second:.2f} "
|
||||
f"bytes_per_second={summary.bytes_per_second:.2f} {cursor_label}={cursor_value or 'none'}",
|
||||
f"bytes_per_second={summary.bytes_per_second:.2f}",
|
||||
fg=fg,
|
||||
)
|
||||
)
|
||||
if dry_run:
|
||||
click.echo(click.style("Dry-run cursor is preview-only; do not persist it for a destructive run.", fg="yellow"))
|
||||
click.echo(click.style("table,row_count", fg="white"))
|
||||
for table_name in [
|
||||
"workflow_runs",
|
||||
@@ -871,8 +842,7 @@ def _echo_bundle_archive_operation_summary(summary, *, dry_run: bool) -> None:
|
||||
click.style(
|
||||
f" bundle={result.bundle_id} tenant={result.tenant_id} runs={result.run_count} "
|
||||
f"rows={result.row_count} archive_bytes={result.archive_bytes} "
|
||||
f"catalog_id={result.catalog_id} time={result.elapsed_time:.2f}s "
|
||||
f"validation={result.validation_time:.2f}s",
|
||||
f"time={result.elapsed_time:.2f}s validation={result.validation_time:.2f}s",
|
||||
fg="white",
|
||||
)
|
||||
)
|
||||
@@ -880,7 +850,7 @@ def _echo_bundle_archive_operation_summary(summary, *, dry_run: bool) -> None:
|
||||
click.echo(
|
||||
click.style(
|
||||
f" failed bundle={result.bundle_id} tenant={result.tenant_id} "
|
||||
f"catalog_id={result.catalog_id} object_prefix={result.object_prefix} error={result.error}",
|
||||
f"object_prefix={result.object_prefix} error={result.error}",
|
||||
fg="red",
|
||||
)
|
||||
)
|
||||
@@ -897,24 +867,25 @@ def _echo_bundle_archive_operation_summary(summary, *, dry_run: bool) -> None:
|
||||
)
|
||||
@click.option("--run-id", required=False, help="Workflow run ID to restore.")
|
||||
@click.option(
|
||||
"--target-month",
|
||||
metavar="YYYY-MM",
|
||||
"--start-from",
|
||||
type=click.DateTime(formats=["%Y-%m-%d", "%Y-%m-%dT%H:%M:%S"]),
|
||||
default=None,
|
||||
help="V2 catalog month to restore; required unless --run-id is used.",
|
||||
help="Optional lower bound (inclusive) for created_at; must be paired with --end-before.",
|
||||
)
|
||||
@click.option(
|
||||
"--after-catalog-id",
|
||||
"--end-before",
|
||||
type=click.DateTime(formats=["%Y-%m-%d", "%Y-%m-%dT%H:%M:%S"]),
|
||||
default=None,
|
||||
help="Exclusive V2 cursor from the same restore month and tenant scope.",
|
||||
help="Optional upper bound (exclusive) for created_at; must be paired with --start-from.",
|
||||
)
|
||||
@click.option("--workers", default=1, show_default=True, type=int, help="V1 --run-id compatibility only.")
|
||||
@click.option("--limit", type=click.IntRange(min=1), default=100, show_default=True, help="Maximum V2 catalog rows.")
|
||||
@click.option("--limit", type=int, default=100, show_default=True, help="Maximum number of V2 bundles to restore.")
|
||||
@click.option("--dry-run", is_flag=True, help="Preview without restoring.")
|
||||
def restore_workflow_runs(
|
||||
tenant_ids: str | None,
|
||||
run_id: str | None,
|
||||
target_month: str | None,
|
||||
after_catalog_id: str | None,
|
||||
start_from: datetime.datetime | None,
|
||||
end_before: datetime.datetime | None,
|
||||
workers: int,
|
||||
limit: int,
|
||||
dry_run: bool,
|
||||
@@ -934,20 +905,23 @@ def restore_workflow_runs(
|
||||
from services.retention.workflow_run.bundle_archive_maintenance import WorkflowRunBundleArchiveMaintenance
|
||||
from services.retention.workflow_run.restore_archived_workflow_run import WorkflowRunRestore
|
||||
|
||||
parsed_tenant_ids = _parse_comma_separated_ids(tenant_ids, param_name="tenant-ids")
|
||||
parsed_tenant_ids = None
|
||||
if tenant_ids:
|
||||
parsed_tenant_ids = [tid.strip() for tid in tenant_ids.split(",") if tid.strip()]
|
||||
if not parsed_tenant_ids:
|
||||
raise click.BadParameter("tenant-ids must not be empty")
|
||||
|
||||
if (start_from is None) ^ (end_before is None):
|
||||
raise click.UsageError("--start-from and --end-before must be provided together.")
|
||||
if run_id is None and (start_from is None or end_before is None):
|
||||
raise click.UsageError("--start-from and --end-before are required for batch restore.")
|
||||
if workers < 1:
|
||||
raise click.BadParameter("workers must be at least 1")
|
||||
if run_id is not None and (target_month is not None or after_catalog_id is not None):
|
||||
raise click.UsageError("--target-month and --after-catalog-id are only valid for V2 batch restore.")
|
||||
if run_id is None and target_month is None:
|
||||
raise click.UsageError("--target-month is required for V2 batch restore.")
|
||||
|
||||
start_time = datetime.datetime.now(datetime.UTC)
|
||||
target_desc = f"workflow run {run_id}" if run_id else f"workflow archive catalog month {target_month}"
|
||||
click.echo(
|
||||
click.style(
|
||||
f"Starting restore of {target_desc} at {start_time.isoformat()}.",
|
||||
f"Starting restore of workflow run {run_id} at {start_time.isoformat()}.",
|
||||
fg="white",
|
||||
)
|
||||
)
|
||||
@@ -981,20 +955,17 @@ def restore_workflow_runs(
|
||||
click.echo(
|
||||
click.style("--workers is ignored for V2 bundle restore; bundles are processed serially.", fg="yellow")
|
||||
)
|
||||
assert target_month is not None
|
||||
target_year, target_month_number = _parse_archive_target_month(target_month)
|
||||
catalog_cursor = _parse_archive_catalog_cursor(after_catalog_id)
|
||||
assert start_from is not None
|
||||
assert end_before is not None
|
||||
bundle_restorer = WorkflowRunBundleArchiveMaintenance(dry_run=dry_run, strict_content_validation=True)
|
||||
summary = bundle_restorer.restore_batch(
|
||||
tenant_ids=parsed_tenant_ids,
|
||||
target_year=target_year,
|
||||
target_month=target_month_number,
|
||||
after_catalog_id=catalog_cursor,
|
||||
start_date=start_from,
|
||||
end_date=end_before,
|
||||
limit=limit,
|
||||
)
|
||||
_echo_bundle_archive_operation_summary(summary, dry_run=dry_run)
|
||||
if summary.bundles_failed:
|
||||
raise click.exceptions.Exit(1)
|
||||
_echo_bundle_archive_operation_summary(summary)
|
||||
return
|
||||
|
||||
|
||||
@click.command(
|
||||
@@ -1008,41 +979,23 @@ def restore_workflow_runs(
|
||||
)
|
||||
@click.option("--run-id", required=False, help="Workflow run ID to delete.")
|
||||
@click.option(
|
||||
"--target-month",
|
||||
metavar="YYYY-MM",
|
||||
"--start-from",
|
||||
type=click.DateTime(formats=["%Y-%m-%d", "%Y-%m-%dT%H:%M:%S"]),
|
||||
default=None,
|
||||
help="V2 catalog month to delete; required unless --run-id is used.",
|
||||
help="Optional lower bound (inclusive) for created_at; must be paired with --end-before.",
|
||||
)
|
||||
@click.option(
|
||||
"--after-catalog-id",
|
||||
"--end-before",
|
||||
type=click.DateTime(formats=["%Y-%m-%d", "%Y-%m-%dT%H:%M:%S"]),
|
||||
default=None,
|
||||
help="Exclusive V2 cursor from the same delete month and tenant scope.",
|
||||
)
|
||||
@click.option(
|
||||
"--run-shard-index",
|
||||
default=None,
|
||||
type=click.IntRange(min=0),
|
||||
help="Zero-based archive shard index. Must be paired with --run-shard-total.",
|
||||
)
|
||||
@click.option(
|
||||
"--run-shard-total",
|
||||
default=None,
|
||||
type=click.IntRange(min=1, max=16),
|
||||
help="Total archive shard count. Must be paired with --run-shard-index.",
|
||||
)
|
||||
@click.option("--all-pages", is_flag=True, help="Process catalog pages until an empty page is reached.")
|
||||
@click.option(
|
||||
"--limit",
|
||||
type=click.IntRange(min=1),
|
||||
default=100,
|
||||
show_default=True,
|
||||
help="Maximum V2 catalog rows per page.",
|
||||
help="Optional upper bound (exclusive) for created_at; must be paired with --start-from.",
|
||||
)
|
||||
@click.option("--limit", type=int, default=100, show_default=True, help="Maximum number of V2 bundles to delete.")
|
||||
@click.option("--dry-run", is_flag=True, help="Preview without deleting.")
|
||||
@click.option(
|
||||
"--skip-bad-archives",
|
||||
is_flag=True,
|
||||
help="V1 --run-id only: continue when one archive object fails validation.",
|
||||
help="Continue batch deletion when one archive object fails validation.",
|
||||
)
|
||||
@click.option(
|
||||
"--restore-sample-interval",
|
||||
@@ -1054,11 +1007,8 @@ def restore_workflow_runs(
|
||||
def delete_archived_workflow_runs(
|
||||
tenant_ids: str | None,
|
||||
run_id: str | None,
|
||||
target_month: str | None,
|
||||
after_catalog_id: str | None,
|
||||
run_shard_index: int | None,
|
||||
run_shard_total: int | None,
|
||||
all_pages: bool,
|
||||
start_from: datetime.datetime | None,
|
||||
end_before: datetime.datetime | None,
|
||||
limit: int,
|
||||
dry_run: bool,
|
||||
skip_bad_archives: bool,
|
||||
@@ -1068,38 +1018,26 @@ def delete_archived_workflow_runs(
|
||||
Delete archived workflow runs from the database.
|
||||
|
||||
Batch delete uses V2 bundle metadata and validates object existence, manifest schema, object size, checksum, row
|
||||
counts, and source/archive content checksums before deleting source rows. Parallel workers may select one exact
|
||||
archive shard; all-pages mode keeps only the current bounded page in memory. `--run-id` keeps the V1 per-run path.
|
||||
counts, and source/archive content checksums before deleting source rows. `--run-id` keeps the V1 per-run path.
|
||||
"""
|
||||
from services.retention.workflow_run.bundle_archive_maintenance import WorkflowRunBundleArchiveMaintenance
|
||||
from services.retention.workflow_run.delete_archived_workflow_run import ArchivedWorkflowRunDeletion
|
||||
|
||||
parsed_tenant_ids = _parse_comma_separated_ids(tenant_ids, param_name="tenant-ids")
|
||||
parsed_tenant_ids = None
|
||||
if tenant_ids:
|
||||
parsed_tenant_ids = [tid.strip() for tid in tenant_ids.split(",") if tid.strip()]
|
||||
if not parsed_tenant_ids:
|
||||
raise click.BadParameter("tenant-ids must not be empty")
|
||||
|
||||
if (start_from is None) ^ (end_before is None):
|
||||
raise click.UsageError("--start-from and --end-before must be provided together.")
|
||||
if run_id is None and (start_from is None or end_before is None):
|
||||
raise click.UsageError("--start-from and --end-before are required for batch delete.")
|
||||
if restore_sample_interval < 0:
|
||||
raise click.BadParameter("restore-sample-interval must be >= 0")
|
||||
if run_id is not None and (
|
||||
target_month is not None
|
||||
or after_catalog_id is not None
|
||||
or run_shard_index is not None
|
||||
or run_shard_total is not None
|
||||
or all_pages
|
||||
):
|
||||
raise click.UsageError(
|
||||
"--target-month, --after-catalog-id, --run-shard-index, --run-shard-total, and --all-pages "
|
||||
"are only valid for V2 batch delete."
|
||||
)
|
||||
if run_id is None and target_month is None:
|
||||
raise click.UsageError("--target-month is required for V2 batch delete.")
|
||||
if run_id is None and skip_bad_archives:
|
||||
raise click.UsageError("--skip-bad-archives is not supported for V2 catalog batches; they fail fast.")
|
||||
if (run_shard_index is None) ^ (run_shard_total is None):
|
||||
raise click.UsageError("--run-shard-index and --run-shard-total must be provided together.")
|
||||
if run_shard_index is not None and run_shard_total is not None and run_shard_index >= run_shard_total:
|
||||
raise click.UsageError("--run-shard-index must be less than --run-shard-total.")
|
||||
|
||||
start_time = datetime.datetime.now(datetime.UTC)
|
||||
target_desc = f"workflow run {run_id}" if run_id else f"workflow archive catalog month {target_month}"
|
||||
target_desc = f"workflow run {run_id}" if run_id else "workflow runs"
|
||||
click.echo(
|
||||
click.style(
|
||||
f"Starting delete of {target_desc} at {start_time.isoformat()}.",
|
||||
@@ -1172,104 +1110,20 @@ def delete_archived_workflow_runs(
|
||||
|
||||
if restore_sample_interval:
|
||||
click.echo(click.style("--restore-sample-interval is ignored for V2 bundle delete.", fg="yellow"))
|
||||
assert target_month is not None
|
||||
target_year, target_month_number = _parse_archive_target_month(target_month)
|
||||
catalog_cursor = _parse_archive_catalog_cursor(after_catalog_id)
|
||||
shard = (
|
||||
f"{run_shard_index:02d}-of-{run_shard_total:02d}"
|
||||
if run_shard_index is not None and run_shard_total is not None
|
||||
else None
|
||||
assert start_from is not None
|
||||
assert end_before is not None
|
||||
bundle_deleter = WorkflowRunBundleArchiveMaintenance(
|
||||
dry_run=dry_run,
|
||||
strict_content_validation=True,
|
||||
stop_on_error=not skip_bad_archives,
|
||||
)
|
||||
bundle_deleter = WorkflowRunBundleArchiveMaintenance(dry_run=dry_run, strict_content_validation=True)
|
||||
if run_shard_total is not None:
|
||||
try:
|
||||
bundle_deleter.validate_catalog_shards(
|
||||
target_year=target_year,
|
||||
target_month=target_month_number,
|
||||
shard_total=run_shard_total,
|
||||
tenant_ids=parsed_tenant_ids,
|
||||
)
|
||||
except ValueError as exc:
|
||||
logger.exception(
|
||||
"Archive catalog shard preflight failed: target_month=%s shard=%s",
|
||||
target_month,
|
||||
shard,
|
||||
)
|
||||
raise click.ClickException(
|
||||
f"Archive catalog shard preflight failed for target_month={target_month} shard={shard}: {exc}"
|
||||
) from exc
|
||||
|
||||
initial_catalog_cursor = catalog_cursor
|
||||
pages_processed = 0
|
||||
bundles_succeeded = 0
|
||||
runs_processed = 0
|
||||
rows_processed = 0
|
||||
archive_bytes = 0
|
||||
while True:
|
||||
summary = bundle_deleter.delete_batch(
|
||||
tenant_ids=parsed_tenant_ids,
|
||||
target_year=target_year,
|
||||
target_month=target_month_number,
|
||||
after_catalog_id=catalog_cursor,
|
||||
limit=limit,
|
||||
shard=shard,
|
||||
)
|
||||
_echo_bundle_archive_operation_summary(summary, dry_run=dry_run)
|
||||
if summary.bundles_failed:
|
||||
failed_result = next((result for result in summary.results if not result.success), None)
|
||||
failed_catalog_id = failed_result.catalog_id if failed_result is not None else "unknown"
|
||||
page_resume_cursor = summary.preview_next_catalog_id if dry_run else summary.next_catalog_id
|
||||
resume_cursor = page_resume_cursor or catalog_cursor
|
||||
if dry_run:
|
||||
cursor_details = (
|
||||
f"preview_after_catalog_id={resume_cursor or 'none'} "
|
||||
f"destructive_retry_after_catalog_id={initial_catalog_cursor or 'none'}"
|
||||
)
|
||||
else:
|
||||
cursor_details = f"resume_after_catalog_id={resume_cursor or 'none'}"
|
||||
click.echo(
|
||||
click.style(
|
||||
f"Delete stopped: target_month={target_month} shard={shard or 'all'} "
|
||||
f"failed_catalog_id={failed_catalog_id} "
|
||||
f"{cursor_details}",
|
||||
fg="red",
|
||||
)
|
||||
)
|
||||
raise click.exceptions.Exit(1)
|
||||
|
||||
if not all_pages:
|
||||
break
|
||||
if summary.bundles_processed == 0:
|
||||
break
|
||||
|
||||
pages_processed += 1
|
||||
bundles_succeeded += summary.bundles_succeeded
|
||||
runs_processed += summary.runs_processed
|
||||
rows_processed += summary.rows_processed
|
||||
archive_bytes += summary.archive_bytes
|
||||
next_catalog_id = summary.preview_next_catalog_id if dry_run else summary.next_catalog_id
|
||||
if next_catalog_id is None or (catalog_cursor is not None and next_catalog_id <= catalog_cursor):
|
||||
click.echo(
|
||||
click.style(
|
||||
f"Delete cursor did not advance: target_month={target_month} shard={shard or 'all'} "
|
||||
f"after_catalog_id={catalog_cursor or 'none'} next_catalog_id={next_catalog_id or 'none'}",
|
||||
fg="red",
|
||||
)
|
||||
)
|
||||
raise click.exceptions.Exit(1)
|
||||
catalog_cursor = next_catalog_id
|
||||
|
||||
if all_pages:
|
||||
final_cursor_label = "preview_final_catalog_id" if dry_run else "final_catalog_id"
|
||||
click.echo(
|
||||
click.style(
|
||||
f"Delete all-pages completed successfully. target_month={target_month} shard={shard or 'all'} "
|
||||
f"pages={pages_processed} bundles_success={bundles_succeeded} runs={runs_processed} "
|
||||
f"rows={rows_processed} archive_bytes={archive_bytes} "
|
||||
f"{final_cursor_label}={catalog_cursor or 'none'}",
|
||||
fg="green",
|
||||
)
|
||||
)
|
||||
summary = bundle_deleter.delete_batch(
|
||||
tenant_ids=parsed_tenant_ids,
|
||||
start_date=start_from,
|
||||
end_date=end_before,
|
||||
limit=limit,
|
||||
)
|
||||
_echo_bundle_archive_operation_summary(summary)
|
||||
|
||||
|
||||
def _find_orphaned_draft_variables(batch_size: int = 1000) -> list[str]:
|
||||
|
||||
@@ -188,26 +188,23 @@ where sites.id is null limit 1000"""
|
||||
if app_id in failed_app_ids:
|
||||
continue
|
||||
|
||||
session = db.session()
|
||||
try:
|
||||
app = session.scalar(select(App).where(App.id == app_id))
|
||||
app = db.session.scalar(select(App).where(App.id == app_id))
|
||||
if not app:
|
||||
logger.info("App %s not found", app_id)
|
||||
continue
|
||||
|
||||
tenant = session.get(Tenant, app.tenant_id)
|
||||
tenant = app.tenant
|
||||
if tenant:
|
||||
accounts = tenant.get_accounts(session=session)
|
||||
accounts = tenant.get_accounts()
|
||||
if not accounts:
|
||||
logger.info("Fix failed for app %s", app.id)
|
||||
continue
|
||||
|
||||
account = accounts[0]
|
||||
logger.info("Fixing missing site for app %s", app.id)
|
||||
app_was_created.send(app, account=account, session=session)
|
||||
session.commit()
|
||||
app_was_created.send(app, account=account)
|
||||
except Exception:
|
||||
session.rollback()
|
||||
failed_app_ids.append(app_id)
|
||||
click.echo(click.style(f"Failed to fix missing site for app {app_id}", fg="red"))
|
||||
logger.exception("Failed to fix app related site missing issue, app_id: %s", app_id)
|
||||
|
||||
+6
-12
@@ -1,11 +1,10 @@
|
||||
import json
|
||||
from typing import cast
|
||||
|
||||
import click
|
||||
from flask import current_app
|
||||
from sqlalchemy import select
|
||||
from sqlalchemy.exc import SQLAlchemyError
|
||||
from sqlalchemy.orm import Session, sessionmaker
|
||||
from sqlalchemy.orm import sessionmaker
|
||||
|
||||
from configs import dify_config
|
||||
from core.rag.datasource.vdb.vector_factory import Vector
|
||||
@@ -102,8 +101,7 @@ def migrate_annotation_vector_database():
|
||||
)
|
||||
documents.append(document)
|
||||
|
||||
with Session(db.engine) as session:
|
||||
vector = Vector(dataset, attributes=["doc_id", "annotation_id", "app_id"], session=session)
|
||||
vector = Vector(dataset, attributes=["doc_id", "annotation_id", "app_id"])
|
||||
click.echo(f"Migrating annotations for app: {app.id}.")
|
||||
|
||||
try:
|
||||
@@ -178,7 +176,6 @@ def migrate_knowledge_vector_database():
|
||||
VectorType.OCEANBASE,
|
||||
}
|
||||
page = 1
|
||||
db_session = db.session()
|
||||
while True:
|
||||
try:
|
||||
stmt = (
|
||||
@@ -187,7 +184,7 @@ def migrate_knowledge_vector_database():
|
||||
.order_by(Dataset.created_at.desc())
|
||||
)
|
||||
|
||||
datasets = paginate_query(stmt, page=page, per_page=50, max_per_page=50, session=db_session)
|
||||
datasets = paginate_query(stmt, page=page, per_page=50, max_per_page=50)
|
||||
if not datasets.items:
|
||||
break
|
||||
except SQLAlchemyError:
|
||||
@@ -230,8 +227,7 @@ def migrate_knowledge_vector_database():
|
||||
|
||||
index_struct_dict = {"type": vector_type, "vector_store": {"class_prefix": collection_name}}
|
||||
dataset.index_struct = json.dumps(index_struct_dict)
|
||||
with Session(db.engine) as session:
|
||||
vector = Vector(dataset, session=session)
|
||||
vector = Vector(dataset)
|
||||
click.echo(f"Migrating dataset {dataset.id}.")
|
||||
|
||||
try:
|
||||
@@ -278,7 +274,7 @@ def migrate_knowledge_vector_database():
|
||||
},
|
||||
)
|
||||
if dataset_document.doc_form == IndexStructureType.PARENT_CHILD_INDEX:
|
||||
child_chunks = segment.get_child_chunks(session=db_session)
|
||||
child_chunks = segment.get_child_chunks()
|
||||
if child_chunks:
|
||||
child_documents = []
|
||||
for child_chunk in child_chunks:
|
||||
@@ -414,9 +410,7 @@ def old_metadata_migration():
|
||||
.where(DatasetDocument.doc_metadata.is_not(None))
|
||||
.order_by(DatasetDocument.created_at.desc())
|
||||
)
|
||||
documents = paginate_query(
|
||||
stmt, page=page, per_page=50, max_per_page=50, session=cast(Session, db.session())
|
||||
)
|
||||
documents = paginate_query(stmt, page=page, per_page=50, max_per_page=50)
|
||||
except SQLAlchemyError:
|
||||
raise
|
||||
if not documents:
|
||||
|
||||
@@ -14,12 +14,6 @@ class EnterpriseFeatureConfig(BaseSettings):
|
||||
default=False,
|
||||
)
|
||||
|
||||
WEBAPP_PUBLIC_ACCESS_ENABLED: bool = Field(
|
||||
description="Whether admins are allowed to set a webapp's access mode to public (anyone with the link, "
|
||||
"no auth). Disable in security-sensitive on-prem deployments.",
|
||||
default=True,
|
||||
)
|
||||
|
||||
CAN_REPLACE_LOGO: bool = Field(
|
||||
description="Allow customization of the enterprise logo.",
|
||||
default=False,
|
||||
|
||||
@@ -1,6 +1,5 @@
|
||||
from configs.extra.agent_backend_config import AgentBackendConfig
|
||||
from configs.extra.archive_config import ArchiveStorageConfig
|
||||
from configs.extra.knowledge_fs_config import KnowledgeFSConfig
|
||||
from configs.extra.notion_config import NotionConfig
|
||||
from configs.extra.sentry_config import SentryConfig
|
||||
|
||||
@@ -9,7 +8,6 @@ class ExtraServiceConfig(
|
||||
# place the configs in alphabet order
|
||||
AgentBackendConfig,
|
||||
ArchiveStorageConfig,
|
||||
KnowledgeFSConfig,
|
||||
NotionConfig,
|
||||
SentryConfig,
|
||||
):
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
from pydantic import Field, NonNegativeFloat, NonNegativeInt, PositiveFloat
|
||||
from pydantic import Field, NonNegativeFloat
|
||||
from pydantic_settings import BaseSettings
|
||||
|
||||
|
||||
@@ -22,21 +22,6 @@ class AgentBackendConfig(BaseSettings):
|
||||
default="success",
|
||||
)
|
||||
|
||||
AGENT_BACKEND_STREAM_READ_TIMEOUT_SECONDS: PositiveFloat = Field(
|
||||
description="Read timeout for one Agent backend SSE connection.",
|
||||
default=30,
|
||||
)
|
||||
|
||||
AGENT_BACKEND_STREAM_MAX_RECONNECTS: NonNegativeInt = Field(
|
||||
description="Maximum Agent backend SSE reconnects before failing the run.",
|
||||
default=3,
|
||||
)
|
||||
|
||||
AGENT_BACKEND_RUN_TIMEOUT_SECONDS: PositiveFloat = Field(
|
||||
description="Total deadline for one Agent backend run event stream.",
|
||||
default=1200,
|
||||
)
|
||||
|
||||
AGENT_SHELL_ENABLED: bool = Field(
|
||||
description=(
|
||||
"Inject the dify.shell layer (sandboxed bash workspace) into Agent runs. "
|
||||
|
||||
@@ -1,64 +0,0 @@
|
||||
"""Configuration for the optional KnowledgeFS Console bridge."""
|
||||
|
||||
from urllib.parse import urlsplit
|
||||
|
||||
from pydantic import Field, PositiveFloat, SecretStr, field_validator, model_validator
|
||||
from pydantic_settings import BaseSettings
|
||||
|
||||
|
||||
class KnowledgeFSConfig(BaseSettings):
|
||||
"""Server-only settings for the KnowledgeFS production connection."""
|
||||
|
||||
KNOWLEDGE_FS_ENABLED: bool = Field(
|
||||
default=False,
|
||||
description="Enable the private KnowledgeFS Console bridge.",
|
||||
)
|
||||
KNOWLEDGE_FS_BASE_URL: str | None = Field(default=None, description="KnowledgeFS gateway base URL.")
|
||||
KNOWLEDGE_FS_JWT_SECRET: SecretStr | None = Field(
|
||||
default=None,
|
||||
min_length=32,
|
||||
description="Shared secret used to sign short-lived KnowledgeFS service JWTs.",
|
||||
)
|
||||
KNOWLEDGE_FS_SSE_READ_TIMEOUT_SECONDS: PositiveFloat = Field(default=300.0, le=3600.0, allow_inf_nan=False)
|
||||
KNOWLEDGE_FS_TIMEOUT_SECONDS: PositiveFloat = Field(default=10.0, le=60.0, allow_inf_nan=False)
|
||||
|
||||
@field_validator(
|
||||
"KNOWLEDGE_FS_BASE_URL",
|
||||
"KNOWLEDGE_FS_JWT_SECRET",
|
||||
mode="before",
|
||||
)
|
||||
@classmethod
|
||||
def normalize_optional_string(cls, value: object) -> object:
|
||||
if isinstance(value, SecretStr):
|
||||
normalized = value.get_secret_value().strip()
|
||||
return SecretStr(normalized) if normalized else None
|
||||
if isinstance(value, str):
|
||||
normalized = value.strip()
|
||||
return normalized or None
|
||||
return value
|
||||
|
||||
@field_validator("KNOWLEDGE_FS_BASE_URL")
|
||||
@classmethod
|
||||
def validate_base_url(cls, value: str | None) -> str | None:
|
||||
if value is None:
|
||||
return None
|
||||
parsed = urlsplit(value)
|
||||
if parsed.scheme not in {"http", "https"} or not parsed.netloc:
|
||||
raise ValueError("KNOWLEDGE_FS_BASE_URL must be an absolute HTTP(S) URL")
|
||||
try:
|
||||
_ = parsed.port
|
||||
except ValueError as exc:
|
||||
raise ValueError("KNOWLEDGE_FS_BASE_URL must include a valid port") from exc
|
||||
if parsed.username or parsed.password or parsed.query or parsed.fragment:
|
||||
raise ValueError("KNOWLEDGE_FS_BASE_URL must not include credentials, query, or fragment")
|
||||
return value.rstrip("/")
|
||||
|
||||
@model_validator(mode="after")
|
||||
def validate_enabled_connection(self) -> "KnowledgeFSConfig":
|
||||
if not self.KNOWLEDGE_FS_ENABLED:
|
||||
return self
|
||||
if bool(self.KNOWLEDGE_FS_BASE_URL) != bool(self.KNOWLEDGE_FS_JWT_SECRET):
|
||||
raise ValueError("KNOWLEDGE_FS_BASE_URL and KNOWLEDGE_FS_JWT_SECRET must be configured together")
|
||||
if not self.KNOWLEDGE_FS_BASE_URL:
|
||||
raise ValueError("KnowledgeFS connection settings are required when the integration is enabled")
|
||||
return self
|
||||
@@ -1,4 +1,4 @@
|
||||
from datetime import datetime, timedelta
|
||||
from datetime import timedelta
|
||||
from enum import StrEnum
|
||||
from typing import Literal
|
||||
|
||||
@@ -11,7 +11,6 @@ from pydantic import (
|
||||
PositiveFloat,
|
||||
PositiveInt,
|
||||
computed_field,
|
||||
field_validator,
|
||||
)
|
||||
from pydantic_settings import BaseSettings
|
||||
|
||||
@@ -276,63 +275,6 @@ class PluginConfig(BaseSettings):
|
||||
default=50 * 1024 * 1024,
|
||||
)
|
||||
|
||||
NEW_USER_DEFAULT_PLUGIN_IDS: str = Field(
|
||||
description="Comma-separated marketplace plugin IDs whose latest versions are installed for new users",
|
||||
default="",
|
||||
)
|
||||
|
||||
@field_validator("PLUGIN_REMOTE_INSTALL_PORT", mode="before")
|
||||
@classmethod
|
||||
def _reject_host_port_shaped_plugin_remote_install_port(cls, v):
|
||||
"""Reject ``host:port``-shaped values with an actionable hint.
|
||||
|
||||
``EXPOSE_PLUGIN_DEBUGGING_PORT`` is overloaded: it feeds both the
|
||||
plugin_daemon ``ports:`` mapping (where ``127.0.0.1:5003`` is valid
|
||||
compose syntax) and this integer app setting advertised in the console.
|
||||
Without this guard a loopback bind spec crashloops the api container
|
||||
with an opaque ``int_parsing`` traceback. See issue #39323.
|
||||
"""
|
||||
if isinstance(v, str) and ":" in v.strip():
|
||||
raise ValueError(
|
||||
"PLUGIN_REMOTE_INSTALL_PORT must be a bare port number, got "
|
||||
f"{v!r}. A 'host:port' value usually means "
|
||||
"EXPOSE_PLUGIN_DEBUGGING_PORT was set to a compose publish spec "
|
||||
"like '127.0.0.1:5003'; bind loopback via a "
|
||||
"docker-compose.override.yaml instead of overloading this var."
|
||||
)
|
||||
return v
|
||||
|
||||
@property
|
||||
def NEW_USER_DEFAULT_PLUGIN_ID_LIST(self) -> list[str]:
|
||||
return [item.strip() for item in self.NEW_USER_DEFAULT_PLUGIN_IDS.split(",") if item.strip()]
|
||||
|
||||
NEW_USER_DEFAULT_MODELS: str = Field(
|
||||
description=("Comma-separated default models for new users in 'model_type:provider:model' format"),
|
||||
default="",
|
||||
)
|
||||
|
||||
@property
|
||||
def NEW_USER_DEFAULT_MODEL_LIST(self) -> list[tuple[str, str, str]]:
|
||||
default_models: list[tuple[str, str, str]] = []
|
||||
configured_model_types: set[str] = set()
|
||||
|
||||
for item in self.NEW_USER_DEFAULT_MODELS.split(","):
|
||||
if not item.strip():
|
||||
continue
|
||||
|
||||
parts = tuple(part.strip() for part in item.split(":", 2))
|
||||
if len(parts) != 3 or not all(parts):
|
||||
raise ValueError("NEW_USER_DEFAULT_MODELS entries must use 'model_type:provider:model' format")
|
||||
|
||||
model_type, provider, model = parts
|
||||
if model_type in configured_model_types:
|
||||
raise ValueError(f"NEW_USER_DEFAULT_MODELS contains duplicate model type: {model_type}")
|
||||
|
||||
configured_model_types.add(model_type)
|
||||
default_models.append((model_type, provider, model))
|
||||
|
||||
return default_models
|
||||
|
||||
|
||||
class MarketplaceConfig(BaseSettings):
|
||||
"""
|
||||
@@ -842,11 +784,6 @@ class WorkflowConfig(BaseSettings):
|
||||
default=500,
|
||||
)
|
||||
|
||||
WORKFLOW_GENERATOR_NODE_BUILDER_MAX_WORKERS: PositiveInt = Field(
|
||||
description="Maximum concurrent node-builder LLM calls per workflow generation request",
|
||||
default=6,
|
||||
)
|
||||
|
||||
WORKFLOW_MAX_EXECUTION_TIME: PositiveInt = Field(
|
||||
description="Maximum execution time in seconds for a single workflow",
|
||||
default=1200,
|
||||
@@ -1160,16 +1097,6 @@ class HomepageConfig(BaseSettings):
|
||||
default=True,
|
||||
)
|
||||
|
||||
ENABLE_STEP_BY_STEP_TOUR: bool = Field(
|
||||
description="Enable account-level Step-by-step Tour eligibility checks",
|
||||
default=False,
|
||||
)
|
||||
|
||||
STEP_BY_STEP_TOUR_ROLLOUT_STARTED_AT: datetime | None = Field(
|
||||
description="UTC timestamp after which newly initialized accounts are eligible for Step-by-step Tour",
|
||||
default=None,
|
||||
)
|
||||
|
||||
|
||||
class RagEtlConfig(BaseSettings):
|
||||
"""
|
||||
@@ -1497,11 +1424,6 @@ class LoginConfig(BaseSettings):
|
||||
|
||||
|
||||
class AccountConfig(BaseSettings):
|
||||
ENABLE_CHANGE_EMAIL: bool = Field(
|
||||
description="whether users can change their email address",
|
||||
default=True,
|
||||
)
|
||||
|
||||
ACCOUNT_DELETION_TOKEN_EXPIRY_MINUTES: PositiveInt = Field(
|
||||
description="Duration in minutes for which a account deletion token remains valid",
|
||||
default=5,
|
||||
|
||||
@@ -1,29 +1,27 @@
|
||||
from typing import Any
|
||||
|
||||
from sqlalchemy import select
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from core.app.apps.agent_app.app_feature_projection import merge_agent_app_features
|
||||
from core.app.apps.agent_app.app_variable_projection import agent_app_variables_to_user_input_form
|
||||
from core.app.apps.agent_app.errors import AgentAppGeneratorError, AgentAppNotPublishedError
|
||||
from extensions.ext_database import db
|
||||
from models.agent import Agent, AgentConfigSnapshot, AgentStatus
|
||||
from models.agent_config_entities import AgentSoulConfig
|
||||
from models.model import App, load_annotation_reply_config
|
||||
from models.model import App
|
||||
|
||||
|
||||
def get_published_agent_app_feature_dict_and_user_input_form(
|
||||
app_model: App,
|
||||
*,
|
||||
session: Session,
|
||||
) -> tuple[dict[str, Any], list[dict[str, Any]]]:
|
||||
"""Return public Agent App parameters backed by the published Agent Soul."""
|
||||
app_model_config = app_model.app_model_config_with_session(session=session)
|
||||
app_model_config = app_model.app_model_config
|
||||
|
||||
agent_id = app_model.bound_agent_id
|
||||
if not agent_id:
|
||||
raise AgentAppGeneratorError("Agent App has no bound Agent")
|
||||
|
||||
agent = session.scalar(
|
||||
agent = db.session.scalar(
|
||||
select(Agent)
|
||||
.where(
|
||||
Agent.tenant_id == app_model.tenant_id,
|
||||
@@ -39,7 +37,7 @@ def get_published_agent_app_feature_dict_and_user_input_form(
|
||||
if not agent.active_config_snapshot_id:
|
||||
raise AgentAppNotPublishedError("Agent has not been published")
|
||||
|
||||
snapshot = session.scalar(
|
||||
snapshot = db.session.scalar(
|
||||
select(AgentConfigSnapshot)
|
||||
.where(
|
||||
AgentConfigSnapshot.tenant_id == app_model.tenant_id,
|
||||
@@ -52,10 +50,5 @@ def get_published_agent_app_feature_dict_and_user_input_form(
|
||||
raise AgentAppGeneratorError("Agent published version not found")
|
||||
|
||||
agent_soul = AgentSoulConfig.model_validate(snapshot.config_snapshot_dict)
|
||||
annotation_reply = load_annotation_reply_config(session, app_model.id) if app_model_config else None
|
||||
features_dict = merge_agent_app_features(
|
||||
agent_soul=agent_soul,
|
||||
app_model_config=app_model_config,
|
||||
annotation_reply=annotation_reply,
|
||||
)
|
||||
features_dict = merge_agent_app_features(agent_soul=agent_soul, app_model_config=app_model_config)
|
||||
return features_dict, agent_app_variables_to_user_input_form(agent_soul.app_variables)
|
||||
|
||||
@@ -4,8 +4,7 @@ from collections.abc import Sequence
|
||||
from dataclasses import dataclass
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from extensions.ext_database import db
|
||||
from services.enterprise import rbac_service as enterprise_rbac_service
|
||||
|
||||
if TYPE_CHECKING:
|
||||
@@ -69,7 +68,6 @@ def resolve_app_access_filter(
|
||||
tenant_id: str,
|
||||
account_id: str,
|
||||
*,
|
||||
session: Session,
|
||||
permissions: MyPermissionsResponse | None = None,
|
||||
) -> AppAccessFilter:
|
||||
"""Compute the RBAC app-access filter for ``account_id`` in ``tenant_id``.
|
||||
@@ -79,7 +77,7 @@ def resolve_app_access_filter(
|
||||
inner-API round trip; otherwise it is fetched here.
|
||||
"""
|
||||
if permissions is None:
|
||||
permissions = enterprise_rbac_service.RBACService.MyPermissions.get(tenant_id, account_id, session=session)
|
||||
permissions = enterprise_rbac_service.RBACService.MyPermissions.get(tenant_id, account_id, session=db.session())
|
||||
whitelist_scope = enterprise_rbac_service.RBACService.AppAccess.whitelist_resources(tenant_id, account_id)
|
||||
|
||||
can_manage_own_apps = _MANAGE_OWN_APPS_PERMISSION_KEY in permissions.workspace.permission_keys
|
||||
|
||||
@@ -2,10 +2,8 @@
|
||||
|
||||
`with_session` is an HTTP controller helper: it opens one SQLAlchemy session
|
||||
for a Resource handler and injects it as the first argument after `self`.
|
||||
Write handlers commit on success and roll back on failure. They use a regular
|
||||
Session context so existing services may commit an intermediate unit and keep
|
||||
using the same Session through SQLAlchemy's autobegin behavior. Pure read
|
||||
handlers may opt out with `write=False`.
|
||||
Handlers use a transaction by default so migrated write paths keep
|
||||
commit/rollback handling; pure read handlers may opt out with `write=False`.
|
||||
"""
|
||||
|
||||
from collections.abc import Callable
|
||||
@@ -40,20 +38,14 @@ def with_session[T, **P, R](
|
||||
) -> (
|
||||
Callable[Concatenate[T, P], R] | Callable[[Callable[Concatenate[T, Session, P], R]], Callable[Concatenate[T, P], R]]
|
||||
):
|
||||
"""Inject a request-scoped session and finalize write handlers."""
|
||||
"""Inject a request-scoped session, using a transaction only for write handlers."""
|
||||
|
||||
def decorator(view: Callable[Concatenate[T, Session, P], R]) -> Callable[Concatenate[T, P], R]:
|
||||
@wraps(view)
|
||||
def wrapper(self: T, *args: P.args, **kwargs: P.kwargs) -> R:
|
||||
if write:
|
||||
with session_factory.create_session() as session:
|
||||
try:
|
||||
result = view(self, session, *args, **kwargs)
|
||||
session.commit()
|
||||
return result
|
||||
except Exception:
|
||||
session.rollback() # noqa: no-new-controller-sqlalchemy decorator owns transaction rollback
|
||||
raise
|
||||
with session_factory.get_session_maker().begin() as session:
|
||||
return view(self, session, *args, **kwargs)
|
||||
|
||||
with session_factory.create_session() as session:
|
||||
return view(self, session, *args, **kwargs)
|
||||
|
||||
@@ -10,7 +10,6 @@ from extensions.ext_database import db
|
||||
from libs.login import current_account_with_tenant
|
||||
from models.dataset import Dataset
|
||||
from models.model import App
|
||||
from services.agent.roster_service import AgentRosterService
|
||||
from services.enterprise.rbac_service import RBACService
|
||||
|
||||
__all__ = ["RBACPermission", "RBACResourceScope", "enforce_rbac_access", "rbac_permission_required"]
|
||||
@@ -52,7 +51,7 @@ def enforce_rbac_access(
|
||||
check_resource_type = None if resource_type == RBACResourceScope.WORKSPACE else resource_type
|
||||
resource_id = None
|
||||
if resource_required and check_resource_type:
|
||||
resource_id = _extract_resource_id(resource_type, tenant_id, path_args)
|
||||
resource_id = _extract_resource_id(resource_type, path_args)
|
||||
if _is_resource_owned_by_current_user(tenant_id, account_id, resource_type, resource_id):
|
||||
return
|
||||
allowed = RBACService.CheckAccess.check(
|
||||
@@ -132,14 +131,11 @@ def _is_resource_owned_by_current_user(
|
||||
return False
|
||||
|
||||
|
||||
def _extract_resource_id(
|
||||
resource_type: RBACResourceScope, tenant_id: str, path_args: dict[str, object] | None = None
|
||||
) -> str:
|
||||
def _extract_resource_id(resource_type: RBACResourceScope, path_args: dict[str, object] | None = None) -> str:
|
||||
"""Extract the resource ID from matched path arguments.
|
||||
|
||||
Some legacy route classes use neutral names such as ``resource_id`` for
|
||||
app/dataset resources, and Agent routes carry ``agent_id``, which is
|
||||
resolved to the App backing that Agent.
|
||||
app/dataset resources, and Agent App routes use ``agent_id`` as the app id.
|
||||
Dataset endpoints behind a rag-pipeline route contain ``pipeline_id``
|
||||
instead of ``dataset_id``. In that case we look up the associated
|
||||
``Dataset`` row via ``Dataset.pipeline_id``.
|
||||
@@ -150,19 +146,10 @@ def _extract_resource_id(
|
||||
matched_args = {**view_args, **(path_args or {})}
|
||||
|
||||
if resource_type == RBACResourceScope.APP:
|
||||
app_id = matched_args.get("app_id")
|
||||
if app_id:
|
||||
return str(app_id)
|
||||
|
||||
agent_id = matched_args.get("agent_id")
|
||||
if agent_id:
|
||||
authz_app_id = AgentRosterService(db.session).peek_authz_app_id(tenant_id=tenant_id, agent_id=str(agent_id))
|
||||
return authz_app_id or str(agent_id)
|
||||
|
||||
resource_id = matched_args.get("resource_id")
|
||||
if resource_id:
|
||||
return str(resource_id)
|
||||
raise ValueError("Missing app_id in request path")
|
||||
app_id = matched_args.get("app_id") or matched_args.get("agent_id") or matched_args.get("resource_id")
|
||||
if not app_id:
|
||||
raise ValueError("Missing app_id in request path")
|
||||
return str(app_id)
|
||||
|
||||
if resource_type == RBACResourceScope.DATASET:
|
||||
dataset_id = matched_args.get("dataset_id") or matched_args.get("resource_id")
|
||||
|
||||
@@ -38,9 +38,7 @@ from . import (
|
||||
feature,
|
||||
human_input_form,
|
||||
init_validate,
|
||||
knowledge_fs_proxy,
|
||||
notification,
|
||||
onboarding,
|
||||
ping,
|
||||
setup,
|
||||
spec,
|
||||
@@ -197,7 +195,6 @@ __all__ = [
|
||||
"human_input_form",
|
||||
"init_validate",
|
||||
"installed_app",
|
||||
"knowledge_fs_proxy",
|
||||
"load_balancing_config",
|
||||
"login",
|
||||
"mcp_server",
|
||||
@@ -210,7 +207,6 @@ __all__ = [
|
||||
"notification",
|
||||
"oauth",
|
||||
"oauth_server",
|
||||
"onboarding",
|
||||
"ops_trace",
|
||||
"parameter",
|
||||
"ping",
|
||||
|
||||
@@ -1,21 +1,20 @@
|
||||
from uuid import UUID
|
||||
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from extensions.ext_database import db
|
||||
from models.model import App
|
||||
from services.agent.roster_service import AgentRosterService
|
||||
|
||||
|
||||
def resolve_agent_app_model(*, session: Session, tenant_id: str, agent_id: UUID) -> App:
|
||||
def resolve_agent_app_model(*, tenant_id: str, agent_id: UUID) -> App:
|
||||
"""Resolve a roster Agent's public Agent App."""
|
||||
return AgentRosterService(session).get_agent_app_model(tenant_id=tenant_id, agent_id=str(agent_id))
|
||||
return AgentRosterService(db.session).get_agent_app_model(tenant_id=tenant_id, agent_id=str(agent_id))
|
||||
|
||||
|
||||
def resolve_agent_runtime_app_model(*, session: Session, tenant_id: str, agent_id: UUID) -> App:
|
||||
def resolve_agent_runtime_app_model(*, tenant_id: str, agent_id: UUID) -> App:
|
||||
"""Resolve the App that backs an Agent runtime surface.
|
||||
|
||||
This accepts both roster Agent Apps and workflow-only inline Agents with a
|
||||
hidden backing App.
|
||||
"""
|
||||
|
||||
return AgentRosterService(session).get_agent_runtime_app_model(tenant_id=tenant_id, agent_id=str(agent_id))
|
||||
return AgentRosterService(db.session).get_agent_runtime_app_model(tenant_id=tenant_id, agent_id=str(agent_id))
|
||||
|
||||
@@ -2,11 +2,9 @@ from uuid import UUID
|
||||
|
||||
from flask import request
|
||||
from flask_restx import Resource
|
||||
from sqlalchemy.orm import Session
|
||||
from werkzeug.exceptions import NotFound
|
||||
|
||||
from controllers.common.schema import query_params_from_model, register_response_schema_models, register_schema_models
|
||||
from controllers.common.session import with_session
|
||||
from controllers.console import console_ns
|
||||
from controllers.console.app.wraps import get_app_model
|
||||
from controllers.console.wraps import (
|
||||
@@ -19,6 +17,7 @@ from controllers.console.wraps import (
|
||||
with_current_tenant_id,
|
||||
with_current_user_id,
|
||||
)
|
||||
from extensions.ext_database import db
|
||||
from fields.agent_fields import (
|
||||
AgentAppComposerResponse,
|
||||
AgentComposerCandidatesResponse,
|
||||
@@ -60,21 +59,20 @@ class WorkflowAgentComposerApi(Resource):
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@get_app_model(mode=[AppMode.WORKFLOW, AppMode.ADVANCED_CHAT])
|
||||
@with_current_user_id
|
||||
@with_current_tenant_id
|
||||
@with_session
|
||||
@get_app_model(mode=[AppMode.WORKFLOW, AppMode.ADVANCED_CHAT])
|
||||
def get(self, session: Session, tenant_id: str, account_id: str, app_model: App, node_id: str):
|
||||
def get(self, tenant_id: str, account_id: str, app_model: App, node_id: str):
|
||||
query = WorkflowAgentComposerQuery.model_validate(request.args.to_dict(flat=True))
|
||||
return dump_response(
|
||||
WorkflowAgentComposerResponse,
|
||||
AgentComposerService.load_workflow_composer(
|
||||
session=session,
|
||||
tenant_id=tenant_id,
|
||||
app_id=app_model.id,
|
||||
node_id=node_id,
|
||||
account_id=account_id,
|
||||
snapshot_id=query.snapshot_id,
|
||||
session=db.session(),
|
||||
),
|
||||
)
|
||||
|
||||
@@ -87,21 +85,20 @@ class WorkflowAgentComposerApi(Resource):
|
||||
@account_initialization_required
|
||||
@edit_permission_required
|
||||
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_EDIT)
|
||||
@get_app_model(mode=[AppMode.WORKFLOW, AppMode.ADVANCED_CHAT])
|
||||
@with_current_user_id
|
||||
@with_current_tenant_id
|
||||
@with_session
|
||||
@get_app_model(mode=[AppMode.WORKFLOW, AppMode.ADVANCED_CHAT])
|
||||
def put(self, session: Session, tenant_id: str, account_id: str, app_model: App, node_id: str):
|
||||
def put(self, tenant_id: str, account_id: str, app_model: App, node_id: str):
|
||||
payload = ComposerSavePayload.model_validate(console_ns.payload or {})
|
||||
return dump_response(
|
||||
WorkflowAgentComposerResponse,
|
||||
AgentComposerService.save_workflow_composer(
|
||||
session=session,
|
||||
tenant_id=tenant_id,
|
||||
app_id=app_model.id,
|
||||
node_id=node_id,
|
||||
account_id=account_id,
|
||||
payload=payload,
|
||||
session=db.session(),
|
||||
),
|
||||
)
|
||||
|
||||
@@ -119,16 +116,14 @@ class WorkflowAgentComposerCopyFromRosterApi(Resource):
|
||||
@account_initialization_required
|
||||
@edit_permission_required
|
||||
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_EDIT)
|
||||
@get_app_model(mode=[AppMode.WORKFLOW, AppMode.ADVANCED_CHAT])
|
||||
@with_current_user_id
|
||||
@with_current_tenant_id
|
||||
@with_session
|
||||
@get_app_model(mode=[AppMode.WORKFLOW, AppMode.ADVANCED_CHAT])
|
||||
def post(self, session: Session, tenant_id: str, account_id: str, app_model: App, node_id: str):
|
||||
def post(self, tenant_id: str, account_id: str, app_model: App, node_id: str):
|
||||
payload = WorkflowComposerCopyFromRosterPayload.model_validate(console_ns.payload or {})
|
||||
return dump_response(
|
||||
WorkflowAgentComposerResponse,
|
||||
AgentComposerService.copy_workflow_composer_from_roster(
|
||||
session=session,
|
||||
tenant_id=tenant_id,
|
||||
app_id=app_model.id,
|
||||
node_id=node_id,
|
||||
@@ -136,6 +131,7 @@ class WorkflowAgentComposerCopyFromRosterApi(Resource):
|
||||
source_agent_id=payload.source_agent_id,
|
||||
source_snapshot_id=payload.source_snapshot_id,
|
||||
idempotency_key=payload.idempotency_key,
|
||||
session=db.session(),
|
||||
),
|
||||
)
|
||||
|
||||
@@ -149,22 +145,19 @@ class WorkflowAgentComposerValidateApi(Resource):
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@with_current_tenant_id
|
||||
@with_session(write=False)
|
||||
@get_app_model(mode=[AppMode.WORKFLOW, AppMode.ADVANCED_CHAT])
|
||||
def post(self, session: Session, tenant_id: str, app_model: App, node_id: str):
|
||||
@with_current_tenant_id
|
||||
def post(self, tenant_id: str, app_model: App, node_id: str):
|
||||
payload = ComposerSavePayload.model_validate(console_ns.payload or {})
|
||||
ComposerConfigValidator.validate_publish_payload(payload)
|
||||
AgentComposerService.validate_knowledge_datasets(
|
||||
session=session, tenant_id=tenant_id, agent_soul=payload.agent_soul
|
||||
)
|
||||
AgentComposerService.validate_knowledge_datasets(tenant_id=tenant_id, agent_soul=payload.agent_soul)
|
||||
findings = AgentComposerService.collect_validation_findings(
|
||||
session=session,
|
||||
tenant_id=tenant_id,
|
||||
payload=payload,
|
||||
agent_id=AgentComposerService.resolve_workflow_node_agent_id(
|
||||
session=session, tenant_id=tenant_id, app_id=app_model.id, node_id=node_id
|
||||
tenant_id=tenant_id, app_id=app_model.id, node_id=node_id, session=db.session()
|
||||
),
|
||||
session=db.session(),
|
||||
)
|
||||
return dump_response(AgentComposerValidateResponse, {"result": "success", "errors": [], **findings})
|
||||
|
||||
@@ -177,19 +170,18 @@ class WorkflowAgentComposerCandidatesApi(Resource):
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@get_app_model(mode=[AppMode.WORKFLOW, AppMode.ADVANCED_CHAT])
|
||||
@with_current_user_id
|
||||
@with_current_tenant_id
|
||||
@with_session(write=False)
|
||||
@get_app_model(mode=[AppMode.WORKFLOW, AppMode.ADVANCED_CHAT])
|
||||
def get(self, session: Session, tenant_id: str, current_user_id: str, app_model: App, node_id: str):
|
||||
def get(self, tenant_id: str, current_user_id: str, app_model: App, node_id: str):
|
||||
return dump_response(
|
||||
AgentComposerCandidatesResponse,
|
||||
AgentComposerService.get_workflow_candidates(
|
||||
session=session,
|
||||
tenant_id=tenant_id,
|
||||
app_id=app_model.id,
|
||||
node_id=node_id,
|
||||
user_id=current_user_id,
|
||||
session=db.session(),
|
||||
),
|
||||
)
|
||||
|
||||
@@ -201,10 +193,9 @@ class WorkflowAgentComposerImpactApi(Resource):
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@with_current_tenant_id
|
||||
@with_session(write=False)
|
||||
@get_app_model(mode=[AppMode.WORKFLOW, AppMode.ADVANCED_CHAT])
|
||||
def post(self, session: Session, tenant_id: str, app_model: App, node_id: str):
|
||||
@with_current_tenant_id
|
||||
def post(self, tenant_id: str, app_model: App, node_id: str):
|
||||
payload = ComposerSavePayload.model_validate(console_ns.payload or {})
|
||||
current_snapshot_id = payload.binding.current_snapshot_id if payload.binding else None
|
||||
if not current_snapshot_id:
|
||||
@@ -214,7 +205,7 @@ class WorkflowAgentComposerImpactApi(Resource):
|
||||
return dump_response(
|
||||
AgentComposerImpactResponse,
|
||||
AgentComposerService.calculate_impact(
|
||||
session=session, tenant_id=tenant_id, current_snapshot_id=current_snapshot_id
|
||||
tenant_id=tenant_id, current_snapshot_id=current_snapshot_id, session=db.session()
|
||||
),
|
||||
)
|
||||
|
||||
@@ -230,28 +221,26 @@ class WorkflowAgentComposerSaveToRosterApi(Resource):
|
||||
@account_initialization_required
|
||||
@edit_permission_required
|
||||
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_EDIT)
|
||||
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.AGENT_MANAGE, resource_required=False)
|
||||
@get_app_model(mode=[AppMode.WORKFLOW, AppMode.ADVANCED_CHAT])
|
||||
@with_current_user_id
|
||||
@with_current_tenant_id
|
||||
@with_session
|
||||
@get_app_model(mode=[AppMode.WORKFLOW, AppMode.ADVANCED_CHAT])
|
||||
def post(self, session: Session, tenant_id: str, account_id: str, app_model: App, node_id: str):
|
||||
def post(self, tenant_id: str, account_id: str, app_model: App, node_id: str):
|
||||
payload = ComposerSavePayload.model_validate(console_ns.payload or {})
|
||||
return dump_response(
|
||||
WorkflowAgentComposerResponse,
|
||||
AgentComposerService.save_workflow_composer(
|
||||
session=session,
|
||||
tenant_id=tenant_id,
|
||||
app_id=app_model.id,
|
||||
node_id=node_id,
|
||||
account_id=account_id,
|
||||
payload=payload,
|
||||
session=db.session(),
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
def _require_snippet_app_id(*, session: Session, tenant_id: str, snippet_id: UUID) -> str:
|
||||
snippet = SnippetService(session=session).get_snippet_by_id(
|
||||
def _require_snippet_app_id(*, tenant_id: str, snippet_id: UUID) -> str:
|
||||
snippet = SnippetService(session=db.session()).get_snippet_by_id(
|
||||
snippet_id=str(snippet_id),
|
||||
tenant_id=tenant_id,
|
||||
)
|
||||
@@ -269,18 +258,17 @@ class SnippetAgentComposerApi(Resource):
|
||||
@account_initialization_required
|
||||
@with_current_user_id
|
||||
@with_current_tenant_id
|
||||
@with_session
|
||||
def get(self, session: Session, tenant_id: str, account_id: str, snippet_id: UUID, node_id: str):
|
||||
def get(self, tenant_id: str, account_id: str, snippet_id: UUID, node_id: str):
|
||||
query = WorkflowAgentComposerQuery.model_validate(request.args.to_dict(flat=True))
|
||||
return dump_response(
|
||||
WorkflowAgentComposerResponse,
|
||||
AgentComposerService.load_workflow_composer(
|
||||
session=session,
|
||||
tenant_id=tenant_id,
|
||||
app_id=_require_snippet_app_id(session=session, tenant_id=tenant_id, snippet_id=snippet_id),
|
||||
app_id=_require_snippet_app_id(tenant_id=tenant_id, snippet_id=snippet_id),
|
||||
node_id=node_id,
|
||||
account_id=account_id,
|
||||
snapshot_id=query.snapshot_id,
|
||||
session=db.session(),
|
||||
),
|
||||
)
|
||||
|
||||
@@ -295,18 +283,17 @@ class SnippetAgentComposerApi(Resource):
|
||||
)
|
||||
@with_current_user_id
|
||||
@with_current_tenant_id
|
||||
@with_session
|
||||
def put(self, session: Session, tenant_id: str, account_id: str, snippet_id: UUID, node_id: str):
|
||||
def put(self, tenant_id: str, account_id: str, snippet_id: UUID, node_id: str):
|
||||
payload = ComposerSavePayload.model_validate(console_ns.payload or {})
|
||||
return dump_response(
|
||||
WorkflowAgentComposerResponse,
|
||||
AgentComposerService.save_workflow_composer(
|
||||
session=session,
|
||||
tenant_id=tenant_id,
|
||||
app_id=_require_snippet_app_id(session=session, tenant_id=tenant_id, snippet_id=snippet_id),
|
||||
app_id=_require_snippet_app_id(tenant_id=tenant_id, snippet_id=snippet_id),
|
||||
node_id=node_id,
|
||||
account_id=account_id,
|
||||
payload=payload,
|
||||
session=db.session(),
|
||||
),
|
||||
)
|
||||
|
||||
@@ -326,20 +313,19 @@ class SnippetAgentComposerCopyFromRosterApi(Resource):
|
||||
)
|
||||
@with_current_user_id
|
||||
@with_current_tenant_id
|
||||
@with_session
|
||||
def post(self, session: Session, tenant_id: str, account_id: str, snippet_id: UUID, node_id: str):
|
||||
def post(self, tenant_id: str, account_id: str, snippet_id: UUID, node_id: str):
|
||||
payload = WorkflowComposerCopyFromRosterPayload.model_validate(console_ns.payload or {})
|
||||
return dump_response(
|
||||
WorkflowAgentComposerResponse,
|
||||
AgentComposerService.copy_workflow_composer_from_roster(
|
||||
session=session,
|
||||
tenant_id=tenant_id,
|
||||
app_id=_require_snippet_app_id(session=session, tenant_id=tenant_id, snippet_id=snippet_id),
|
||||
app_id=_require_snippet_app_id(tenant_id=tenant_id, snippet_id=snippet_id),
|
||||
node_id=node_id,
|
||||
account_id=account_id,
|
||||
source_agent_id=payload.source_agent_id,
|
||||
source_snapshot_id=payload.source_snapshot_id,
|
||||
idempotency_key=payload.idempotency_key,
|
||||
session=db.session(),
|
||||
),
|
||||
)
|
||||
|
||||
@@ -354,24 +340,21 @@ class SnippetAgentComposerValidateApi(Resource):
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@with_current_tenant_id
|
||||
@with_session(write=False)
|
||||
def post(self, session: Session, tenant_id: str, snippet_id: UUID, node_id: str):
|
||||
app_id = _require_snippet_app_id(session=session, tenant_id=tenant_id, snippet_id=snippet_id)
|
||||
def post(self, tenant_id: str, snippet_id: UUID, node_id: str):
|
||||
app_id = _require_snippet_app_id(tenant_id=tenant_id, snippet_id=snippet_id)
|
||||
payload = ComposerSavePayload.model_validate(console_ns.payload or {})
|
||||
ComposerConfigValidator.validate_publish_payload(payload)
|
||||
AgentComposerService.validate_knowledge_datasets(
|
||||
session=session, tenant_id=tenant_id, agent_soul=payload.agent_soul
|
||||
)
|
||||
AgentComposerService.validate_knowledge_datasets(tenant_id=tenant_id, agent_soul=payload.agent_soul)
|
||||
findings = AgentComposerService.collect_validation_findings(
|
||||
session=session,
|
||||
tenant_id=tenant_id,
|
||||
payload=payload,
|
||||
agent_id=AgentComposerService.resolve_workflow_node_agent_id(
|
||||
session=session,
|
||||
tenant_id=tenant_id,
|
||||
app_id=app_id,
|
||||
node_id=node_id,
|
||||
session=db.session(),
|
||||
),
|
||||
session=db.session(),
|
||||
)
|
||||
return dump_response(AgentComposerValidateResponse, {"result": "success", "errors": [], **findings})
|
||||
|
||||
@@ -386,16 +369,15 @@ class SnippetAgentComposerCandidatesApi(Resource):
|
||||
@account_initialization_required
|
||||
@with_current_user_id
|
||||
@with_current_tenant_id
|
||||
@with_session(write=False)
|
||||
def get(self, session: Session, tenant_id: str, current_user_id: str, snippet_id: UUID, node_id: str):
|
||||
def get(self, tenant_id: str, current_user_id: str, snippet_id: UUID, node_id: str):
|
||||
return dump_response(
|
||||
AgentComposerCandidatesResponse,
|
||||
AgentComposerService.get_workflow_candidates(
|
||||
session=session,
|
||||
tenant_id=tenant_id,
|
||||
app_id=_require_snippet_app_id(session=session, tenant_id=tenant_id, snippet_id=snippet_id),
|
||||
app_id=_require_snippet_app_id(tenant_id=tenant_id, snippet_id=snippet_id),
|
||||
node_id=node_id,
|
||||
user_id=current_user_id,
|
||||
session=db.session(),
|
||||
),
|
||||
)
|
||||
|
||||
@@ -408,9 +390,8 @@ class SnippetAgentComposerImpactApi(Resource):
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@with_current_tenant_id
|
||||
@with_session(write=False)
|
||||
def post(self, session: Session, tenant_id: str, snippet_id: UUID, node_id: str):
|
||||
_require_snippet_app_id(session=session, tenant_id=tenant_id, snippet_id=snippet_id)
|
||||
def post(self, tenant_id: str, snippet_id: UUID, node_id: str):
|
||||
_require_snippet_app_id(tenant_id=tenant_id, snippet_id=snippet_id)
|
||||
payload = ComposerSavePayload.model_validate(console_ns.payload or {})
|
||||
current_snapshot_id = payload.binding.current_snapshot_id if payload.binding else None
|
||||
if not current_snapshot_id:
|
||||
@@ -420,9 +401,9 @@ class SnippetAgentComposerImpactApi(Resource):
|
||||
return dump_response(
|
||||
AgentComposerImpactResponse,
|
||||
AgentComposerService.calculate_impact(
|
||||
session=session,
|
||||
tenant_id=tenant_id,
|
||||
current_snapshot_id=current_snapshot_id,
|
||||
session=db.session(),
|
||||
),
|
||||
)
|
||||
|
||||
@@ -440,21 +421,19 @@ class SnippetAgentComposerSaveToRosterApi(Resource):
|
||||
@rbac_permission_required(
|
||||
RBACResourceScope.WORKSPACE, RBACPermission.SNIPPETS_CREATE_AND_MODIFY, resource_required=False
|
||||
)
|
||||
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.AGENT_MANAGE, resource_required=False)
|
||||
@with_current_user_id
|
||||
@with_current_tenant_id
|
||||
@with_session
|
||||
def post(self, session: Session, tenant_id: str, account_id: str, snippet_id: UUID, node_id: str):
|
||||
def post(self, tenant_id: str, account_id: str, snippet_id: UUID, node_id: str):
|
||||
payload = ComposerSavePayload.model_validate(console_ns.payload or {})
|
||||
return dump_response(
|
||||
WorkflowAgentComposerResponse,
|
||||
AgentComposerService.save_workflow_composer(
|
||||
session=session,
|
||||
tenant_id=tenant_id,
|
||||
app_id=_require_snippet_app_id(session=session, tenant_id=tenant_id, snippet_id=snippet_id),
|
||||
app_id=_require_snippet_app_id(tenant_id=tenant_id, snippet_id=snippet_id),
|
||||
node_id=node_id,
|
||||
account_id=account_id,
|
||||
payload=payload,
|
||||
session=db.session(),
|
||||
),
|
||||
)
|
||||
|
||||
@@ -466,11 +445,10 @@ class AgentComposerApi(Resource):
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@with_current_tenant_id
|
||||
@with_session
|
||||
def get(self, session: Session, tenant_id: str, agent_id: UUID):
|
||||
def get(self, tenant_id: str, agent_id: UUID):
|
||||
return dump_response(
|
||||
AgentAppComposerResponse,
|
||||
AgentComposerService.load_agent_composer(session=session, tenant_id=tenant_id, agent_id=str(agent_id)),
|
||||
AgentComposerService.load_agent_composer(tenant_id=tenant_id, agent_id=str(agent_id), session=db.session()),
|
||||
)
|
||||
|
||||
@console_ns.expect(console_ns.models[ComposerSavePayload.__name__])
|
||||
@@ -480,20 +458,18 @@ class AgentComposerApi(Resource):
|
||||
@account_initialization_required
|
||||
@edit_permission_required
|
||||
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_EDIT)
|
||||
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.AGENT_MANAGE, resource_required=False)
|
||||
@with_current_user_id
|
||||
@with_current_tenant_id
|
||||
@with_session
|
||||
def put(self, session: Session, tenant_id: str, account_id: str, agent_id: UUID):
|
||||
def put(self, tenant_id: str, account_id: str, agent_id: UUID):
|
||||
payload = ComposerSavePayload.model_validate(console_ns.payload or {})
|
||||
return dump_response(
|
||||
AgentAppComposerResponse,
|
||||
AgentComposerService.save_agent_composer(
|
||||
session=session,
|
||||
tenant_id=tenant_id,
|
||||
agent_id=str(agent_id),
|
||||
account_id=account_id,
|
||||
payload=payload,
|
||||
session=db.session(),
|
||||
),
|
||||
)
|
||||
|
||||
@@ -508,19 +484,16 @@ class AgentComposerValidateApi(Resource):
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@with_current_tenant_id
|
||||
@with_session
|
||||
def post(self, session: Session, tenant_id: str, agent_id: UUID):
|
||||
AgentComposerService.load_agent_composer(session=session, tenant_id=tenant_id, agent_id=str(agent_id))
|
||||
def post(self, tenant_id: str, agent_id: UUID):
|
||||
AgentComposerService.load_agent_composer(tenant_id=tenant_id, agent_id=str(agent_id), session=db.session())
|
||||
payload = ComposerSavePayload.model_validate(console_ns.payload or {})
|
||||
ComposerConfigValidator.validate_publish_payload(payload)
|
||||
AgentComposerService.validate_knowledge_datasets(
|
||||
session=session, tenant_id=tenant_id, agent_soul=payload.agent_soul
|
||||
)
|
||||
AgentComposerService.validate_knowledge_datasets(tenant_id=tenant_id, agent_soul=payload.agent_soul)
|
||||
findings = AgentComposerService.collect_validation_findings(
|
||||
session=session,
|
||||
tenant_id=tenant_id,
|
||||
payload=payload,
|
||||
agent_id=str(agent_id),
|
||||
session=db.session(),
|
||||
)
|
||||
return dump_response(AgentComposerValidateResponse, {"result": "success", "errors": [], **findings})
|
||||
|
||||
@@ -535,14 +508,13 @@ class AgentComposerCandidatesApi(Resource):
|
||||
@account_initialization_required
|
||||
@with_current_user_id
|
||||
@with_current_tenant_id
|
||||
@with_session(write=False)
|
||||
def get(self, session: Session, tenant_id: str, current_user_id: str, agent_id: UUID):
|
||||
def get(self, tenant_id: str, current_user_id: str, agent_id: UUID):
|
||||
return dump_response(
|
||||
AgentComposerCandidatesResponse,
|
||||
AgentComposerService.get_agent_app_candidates(
|
||||
session=session,
|
||||
tenant_id=tenant_id,
|
||||
agent_id=str(agent_id),
|
||||
user_id=current_user_id,
|
||||
session=db.session(),
|
||||
),
|
||||
)
|
||||
|
||||
@@ -4,7 +4,6 @@ from flask import abort, request
|
||||
from flask_restx import Resource
|
||||
from pydantic import AliasChoices, BaseModel, Field, field_validator
|
||||
from sqlalchemy import func, select
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from controllers.common.schema import (
|
||||
query_params_from_model,
|
||||
@@ -12,8 +11,8 @@ from controllers.common.schema import (
|
||||
register_response_schema_models,
|
||||
register_schema_models,
|
||||
)
|
||||
from controllers.common.session import with_session
|
||||
from controllers.console import console_ns
|
||||
from controllers.console.agent.app_helpers import resolve_agent_app_model, resolve_agent_runtime_app_model
|
||||
from controllers.console.apikey import ApiKeyItem, ApiKeyList, BaseApiKeyListResource, BaseApiKeyResource
|
||||
from controllers.console.app.app import (
|
||||
APP_LIST_QUERY_ARRAY_FIELDS,
|
||||
@@ -43,6 +42,7 @@ from controllers.console.wraps import (
|
||||
with_current_tenant_id,
|
||||
with_current_user,
|
||||
)
|
||||
from extensions.ext_database import db
|
||||
from fields.agent_fields import (
|
||||
AgentConfigDraftSummaryResponse,
|
||||
AgentConfigSnapshotDetailResponse,
|
||||
@@ -62,7 +62,7 @@ from libs.datetime_utils import parse_time_range
|
||||
from libs.helper import dump_response
|
||||
from libs.login import login_required
|
||||
from models import Account
|
||||
from models.agent import Agent, AgentConfigDraftType, AgentStatus
|
||||
from models.agent import Agent, AgentStatus
|
||||
from models.agent_config_entities import AgentSoulConfig
|
||||
from models.enums import ApiTokenType
|
||||
from models.model import ApiToken, App, IconType
|
||||
@@ -177,7 +177,7 @@ class AgentLogsQuery(BaseModel):
|
||||
default_factory=list,
|
||||
description=(
|
||||
"Filter by one or more source IDs, e.g. webapp:<app_id> "
|
||||
"or workflow:<app_id>. Exact workflow:<app_id>:<workflow_id>:<version>:<node_id> IDs remain supported."
|
||||
"or workflow:<app_id>:<workflow_id>:<version>:<node_id>"
|
||||
),
|
||||
)
|
||||
sort_by: str = Field(default="updated_at", description="Sort by created_at or updated_at")
|
||||
@@ -221,10 +221,7 @@ class AgentLogsQuery(BaseModel):
|
||||
class AgentStatisticsQuery(BaseModel):
|
||||
source: str | None = Field(
|
||||
default=None,
|
||||
description=(
|
||||
"Filter by a structured webapp:<app_id> or workflow:<app_id> source ID. "
|
||||
"Legacy invoke sources and exact workflow version/node source IDs remain supported."
|
||||
),
|
||||
description="Filter by all, console/explore, api/service-api, web-app, debugger, openapi, or trigger",
|
||||
)
|
||||
start: str | None = Field(default=None, description="Start date (YYYY-MM-DD HH:MM)")
|
||||
end: str | None = Field(default=None, description="End date (YYYY-MM-DD HH:MM)")
|
||||
@@ -244,7 +241,6 @@ class AgentAppPartial(GenericAppPartial):
|
||||
debug_conversation_id: str | None = None
|
||||
role: str | None = None
|
||||
active_config_is_published: bool = False
|
||||
reference_count: int | None = None
|
||||
published_reference_count: int = 0
|
||||
published_references: list[AgentAppPublishedReferenceResponse] = Field(default_factory=list)
|
||||
|
||||
@@ -266,13 +262,6 @@ class AgentDebugConversationRefreshResponse(BaseModel):
|
||||
debug_conversation_message_count: int = 0
|
||||
|
||||
|
||||
class AgentDebugConversationRefreshPayload(BaseModel):
|
||||
draft_type: AgentConfigDraftType = Field(
|
||||
default=AgentConfigDraftType.DEBUG_BUILD,
|
||||
description="Agent draft surface whose conversation should be refreshed",
|
||||
)
|
||||
|
||||
|
||||
class AgentPublishPayload(BaseModel):
|
||||
version_note: str | None = Field(default=None, description="Optional note for this published Agent version")
|
||||
|
||||
@@ -316,7 +305,6 @@ register_schema_models(
|
||||
AgentAppCopyPayload,
|
||||
AgentPublishPayload,
|
||||
AgentBuildDraftCheckoutPayload,
|
||||
AgentDebugConversationRefreshPayload,
|
||||
ComposerSavePayload,
|
||||
AgentApiStatusPayload,
|
||||
AgentInviteOptionsQuery,
|
||||
@@ -351,13 +339,11 @@ register_response_schema_models(
|
||||
)
|
||||
|
||||
|
||||
def _agent_roster_service(session: Session) -> AgentRosterService:
|
||||
return AgentRosterService(session)
|
||||
def _agent_roster_service() -> AgentRosterService:
|
||||
return AgentRosterService(db.session)
|
||||
|
||||
|
||||
def _serialize_agent_app_detail(
|
||||
session: Session, app_model, *, current_user: Account, agent_id: str | None = None
|
||||
) -> dict:
|
||||
def _serialize_agent_app_detail(app_model, *, current_user: Account, agent_id: str | None = None) -> dict:
|
||||
"""Serialize an Agent App detail using roster-only DTOs.
|
||||
|
||||
`/agent` responses are roster-shaped rather than raw app-shaped: `id`
|
||||
@@ -367,19 +353,15 @@ def _serialize_agent_app_detail(
|
||||
roster persona fields without widening the shared /apps detail schema.
|
||||
"""
|
||||
|
||||
app_model = AppService().get_app(app_model, session=session)
|
||||
app_model = AppService().get_app(app_model)
|
||||
if FeatureService.get_system_features().webapp_auth.enabled:
|
||||
app_setting = EnterpriseService.WebAppAuth.get_app_access_mode_by_id(app_id=str(app_model.id))
|
||||
app_model.access_mode = app_setting.access_mode # type: ignore[attr-defined]
|
||||
|
||||
roster_service = _agent_roster_service(session)
|
||||
payload = AgentAppDetailWithSite.model_validate(
|
||||
app_model,
|
||||
from_attributes=True,
|
||||
context={"session": session},
|
||||
).model_dump(mode="json")
|
||||
roster_service = _agent_roster_service()
|
||||
payload = AgentAppDetailWithSite.model_validate(app_model, from_attributes=True).model_dump(mode="json")
|
||||
agent = (
|
||||
session.scalar(
|
||||
db.session.scalar(
|
||||
select(Agent).where(
|
||||
Agent.tenant_id == app_model.tenant_id,
|
||||
Agent.id == agent_id,
|
||||
@@ -400,8 +382,6 @@ def _serialize_agent_app_detail(
|
||||
tenant_id=app_model.tenant_id,
|
||||
agent_id=agent.id,
|
||||
account_id=current_user.id,
|
||||
draft_type=AgentConfigDraftType.DEBUG_BUILD,
|
||||
commit=False,
|
||||
)
|
||||
message_count = roster_service.count_agent_app_debug_conversation_messages(
|
||||
conversation_id=debug_conversation_id,
|
||||
@@ -417,7 +397,7 @@ def _serialize_agent_app_detail(
|
||||
return payload
|
||||
|
||||
|
||||
def _serialize_agent_app_pagination(session: Session, app_pagination, *, tenant_id: str, current_user: Account) -> dict:
|
||||
def _serialize_agent_app_pagination(app_pagination, *, tenant_id: str, current_user: Account) -> dict:
|
||||
"""Serialize Agent App lists with roster-shaped items.
|
||||
|
||||
Each item starts from the shared App list shape, then drops
|
||||
@@ -427,7 +407,7 @@ def _serialize_agent_app_pagination(session: Session, app_pagination, *, tenant_
|
||||
"""
|
||||
|
||||
app_ids = [str(app.id) for app in app_pagination.items]
|
||||
roster_service = _agent_roster_service(session)
|
||||
roster_service = _agent_roster_service()
|
||||
agents_by_app_id = roster_service.load_app_backing_agents_by_app_id(
|
||||
tenant_id=tenant_id,
|
||||
app_ids=app_ids,
|
||||
@@ -440,21 +420,12 @@ def _serialize_agent_app_pagination(session: Session, app_pagination, *, tenant_
|
||||
tenant_id=tenant_id,
|
||||
agent_ids=[agent.id for agent in agents_by_app_id.values()],
|
||||
)
|
||||
reference_counts_by_agent_id = roster_service.load_reference_counts_by_agent_id(
|
||||
tenant_id=tenant_id,
|
||||
agent_ids=[agent.id for agent in agents_by_app_id.values()],
|
||||
)
|
||||
debug_conversation_ids_by_agent_id = roster_service.load_or_create_agent_app_debug_conversation_ids_by_agent_id(
|
||||
tenant_id=tenant_id,
|
||||
agents=list(agents_by_app_id.values()),
|
||||
account_id=current_user.id,
|
||||
draft_type=AgentConfigDraftType.DEBUG_BUILD,
|
||||
)
|
||||
payload = AgentAppPagination.model_validate(
|
||||
app_pagination,
|
||||
from_attributes=True,
|
||||
context={"session": session},
|
||||
).model_dump(mode="json")
|
||||
payload = AgentAppPagination.model_validate(app_pagination, from_attributes=True).model_dump(mode="json")
|
||||
for item in payload["data"]:
|
||||
app_id = item["id"]
|
||||
item.pop("bound_agent_id", None)
|
||||
@@ -467,7 +438,6 @@ def _serialize_agent_app_pagination(session: Session, app_pagination, *, tenant_
|
||||
item["debug_conversation_id"] = debug_conversation_ids_by_agent_id.get(agent.id)
|
||||
item["role"] = agent.role or ""
|
||||
item["active_config_is_published"] = active_config_is_published_by_agent_id.get(agent.id, False)
|
||||
item["reference_count"] = reference_counts_by_agent_id.get(agent.id, 0)
|
||||
published_references = published_references_by_agent_id.get(agent.id, [])
|
||||
item["published_reference_count"] = len(published_references)
|
||||
item["published_references"] = [
|
||||
@@ -486,17 +456,13 @@ def _serialize_agent_app_pagination(session: Session, app_pagination, *, tenant_
|
||||
)
|
||||
|
||||
|
||||
def _resolve_agent_app_model(session: Session, *, tenant_id: str, agent_id: UUID) -> App:
|
||||
return _agent_roster_service(session).get_agent_app_model(tenant_id=tenant_id, agent_id=str(agent_id))
|
||||
def _resolve_agent_app_model(*, tenant_id: str, agent_id: UUID):
|
||||
return resolve_agent_app_model(tenant_id=tenant_id, agent_id=agent_id)
|
||||
|
||||
|
||||
def _resolve_agent_runtime_app_model(session: Session, *, tenant_id: str, agent_id: UUID) -> App:
|
||||
return _agent_roster_service(session).get_agent_runtime_app_model(tenant_id=tenant_id, agent_id=str(agent_id))
|
||||
|
||||
|
||||
def _agent_api_key_count(session: Session, app_id: str) -> int:
|
||||
def _agent_api_key_count(app_id: str) -> int:
|
||||
return (
|
||||
session.scalar(
|
||||
db.session.scalar(
|
||||
select(func.count(ApiToken.id)).where(
|
||||
ApiToken.type == ApiTokenType.APP,
|
||||
ApiToken.app_id == app_id,
|
||||
@@ -506,7 +472,7 @@ def _agent_api_key_count(session: Session, app_id: str) -> int:
|
||||
)
|
||||
|
||||
|
||||
def _serialize_agent_api_access(session: Session, app_model: App) -> dict:
|
||||
def _serialize_agent_api_access(app_model: App) -> dict:
|
||||
base_url = app_model.api_base_url
|
||||
response = AgentApiAccessResponse(
|
||||
enabled=bool(app_model.enable_api),
|
||||
@@ -521,13 +487,13 @@ def _serialize_agent_api_access(session: Session, app_model: App) -> dict:
|
||||
meta_endpoint=f"{base_url}/meta",
|
||||
api_rpm=app_model.api_rpm or 0,
|
||||
api_rph=app_model.api_rph or 0,
|
||||
api_key_count=_agent_api_key_count(session, str(app_model.id)),
|
||||
api_key_count=_agent_api_key_count(str(app_model.id)),
|
||||
)
|
||||
return response.model_dump(mode="json")
|
||||
|
||||
|
||||
def _agent_observability_service(session: Session) -> AgentObservabilityService:
|
||||
return AgentObservabilityService(session)
|
||||
def _agent_observability_service() -> AgentObservabilityService:
|
||||
return AgentObservabilityService(db.session)
|
||||
|
||||
|
||||
def _parse_observability_time_range(start: str | None, end: str | None, account: Account):
|
||||
@@ -552,11 +518,9 @@ class AgentAppListApi(Resource):
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.AGENT_MANAGE, resource_required=False)
|
||||
@with_current_user
|
||||
@with_current_tenant_id
|
||||
@with_session
|
||||
def get(self, session: Session, current_tenant_id: str, current_user: Account):
|
||||
def get(self, current_tenant_id: str, current_user: Account):
|
||||
args = query_params_from_request(AppListQuery, list_fields=APP_LIST_QUERY_ARRAY_FIELDS)
|
||||
params = AppListParams(
|
||||
page=args.page,
|
||||
@@ -570,13 +534,12 @@ class AgentAppListApi(Resource):
|
||||
status="normal",
|
||||
)
|
||||
|
||||
app_pagination = AppService().get_paginate_apps(current_user.id, current_tenant_id, params, session)
|
||||
app_pagination = AppService().get_paginate_apps(current_user.id, current_tenant_id, params, db.session())
|
||||
if app_pagination is None:
|
||||
empty = AgentAppPagination(page=args.page, limit=args.limit, total=0, has_more=False, data=[])
|
||||
return empty.model_dump(mode="json")
|
||||
|
||||
return _serialize_agent_app_pagination(
|
||||
session,
|
||||
app_pagination,
|
||||
tenant_id=current_tenant_id,
|
||||
current_user=current_user,
|
||||
@@ -590,11 +553,9 @@ class AgentAppListApi(Resource):
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@edit_permission_required
|
||||
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.AGENT_MANAGE, resource_required=False)
|
||||
@with_current_user
|
||||
@with_current_tenant_id
|
||||
@with_session
|
||||
def post(self, session: Session, current_tenant_id: str, current_user: Account):
|
||||
def post(self, current_tenant_id: str, current_user: Account):
|
||||
args = AgentAppCreatePayload.model_validate(console_ns.payload)
|
||||
params = CreateAppParams(
|
||||
name=args.name,
|
||||
@@ -606,8 +567,8 @@ class AgentAppListApi(Resource):
|
||||
icon_background=args.icon_background,
|
||||
)
|
||||
|
||||
app = AppService().create_app(current_tenant_id, params, current_user, session=session)
|
||||
return _serialize_agent_app_detail(session, app, current_user=current_user), 201
|
||||
app = AppService().create_app(current_tenant_id, params, current_user, session=db.session())
|
||||
return _serialize_agent_app_detail(app, current_user=current_user), 201
|
||||
|
||||
|
||||
@console_ns.route("/agent/<uuid:agent_id>")
|
||||
@@ -619,10 +580,9 @@ class AgentAppApi(Resource):
|
||||
@enterprise_license_required
|
||||
@with_current_user
|
||||
@with_current_tenant_id
|
||||
@with_session
|
||||
def get(self, session: Session, tenant_id: str, current_user: Account, agent_id: UUID):
|
||||
app_model = _resolve_agent_runtime_app_model(session, tenant_id=tenant_id, agent_id=agent_id)
|
||||
return _serialize_agent_app_detail(session, app_model, current_user=current_user, agent_id=str(agent_id))
|
||||
def get(self, tenant_id: str, current_user: Account, agent_id: UUID):
|
||||
app_model = resolve_agent_runtime_app_model(tenant_id=tenant_id, agent_id=agent_id)
|
||||
return _serialize_agent_app_detail(app_model, current_user=current_user, agent_id=str(agent_id))
|
||||
|
||||
@console_ns.expect(console_ns.models[AgentAppUpdatePayload.__name__])
|
||||
@console_ns.response(200, "Agent app updated successfully", console_ns.models[AgentAppDetailWithSite.__name__])
|
||||
@@ -632,12 +592,10 @@ class AgentAppApi(Resource):
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@edit_permission_required
|
||||
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.AGENT_MANAGE, resource_required=False)
|
||||
@with_current_user
|
||||
@with_current_tenant_id
|
||||
@with_session
|
||||
def put(self, session: Session, tenant_id: str, current_user: Account, agent_id: UUID):
|
||||
app_model = _resolve_agent_app_model(session, tenant_id=tenant_id, agent_id=agent_id)
|
||||
def put(self, tenant_id: str, current_user: Account, agent_id: UUID):
|
||||
app_model = _resolve_agent_app_model(tenant_id=tenant_id, agent_id=agent_id)
|
||||
args = AgentAppUpdatePayload.model_validate(console_ns.payload)
|
||||
args_dict: AppService.ArgsDict = {
|
||||
"name": args.name,
|
||||
@@ -649,8 +607,8 @@ class AgentAppApi(Resource):
|
||||
"max_active_requests": args.max_active_requests or 0,
|
||||
"role": args.role,
|
||||
}
|
||||
updated = AppService().update_app(app_model, args_dict, session=session)
|
||||
return _serialize_agent_app_detail(session, updated, current_user=current_user)
|
||||
updated = AppService().update_app(app_model, args_dict, session=db.session())
|
||||
return _serialize_agent_app_detail(updated, current_user=current_user)
|
||||
|
||||
@console_ns.response(204, "Agent app deleted successfully")
|
||||
@console_ns.response(403, "Insufficient permissions")
|
||||
@@ -658,27 +616,15 @@ class AgentAppApi(Resource):
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@edit_permission_required
|
||||
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.AGENT_MANAGE, resource_required=False)
|
||||
@with_current_tenant_id
|
||||
@with_session
|
||||
def delete(self, session: Session, tenant_id: str, agent_id: UUID):
|
||||
app_model = _resolve_agent_app_model(session, tenant_id=tenant_id, agent_id=agent_id)
|
||||
AppService().delete_app(app_model, session=session)
|
||||
def delete(self, tenant_id: str, agent_id: UUID):
|
||||
app_model = _resolve_agent_app_model(tenant_id=tenant_id, agent_id=agent_id)
|
||||
AppService().delete_app(app_model, session=db.session())
|
||||
return "", 204
|
||||
|
||||
|
||||
@console_ns.route("/agent/<uuid:agent_id>/debug-conversation/refresh")
|
||||
class AgentDebugConversationRefreshApi(Resource):
|
||||
@console_ns.expect(console_ns.models[AgentDebugConversationRefreshPayload.__name__])
|
||||
@console_ns.doc(
|
||||
params={
|
||||
"payload": {
|
||||
"in": "body",
|
||||
"required": False,
|
||||
"schema": {"$ref": f"#/components/schemas/{AgentDebugConversationRefreshPayload.__name__}"},
|
||||
}
|
||||
}
|
||||
)
|
||||
@console_ns.response(
|
||||
200,
|
||||
"Agent debug conversation refreshed",
|
||||
@@ -691,14 +637,11 @@ class AgentDebugConversationRefreshApi(Resource):
|
||||
@edit_permission_required
|
||||
@with_current_user
|
||||
@with_current_tenant_id
|
||||
@with_session
|
||||
def post(self, session: Session, tenant_id: str, current_user: Account, agent_id: UUID):
|
||||
args = AgentDebugConversationRefreshPayload.model_validate(request.get_json(silent=True) or {})
|
||||
debug_conversation_id = _agent_roster_service(session).refresh_agent_app_debug_conversation_id(
|
||||
def post(self, tenant_id: str, current_user: Account, agent_id: UUID):
|
||||
debug_conversation_id = _agent_roster_service().refresh_agent_app_debug_conversation_id(
|
||||
tenant_id=tenant_id,
|
||||
agent_id=str(agent_id),
|
||||
account_id=current_user.id,
|
||||
draft_type=args.draft_type,
|
||||
)
|
||||
return AgentDebugConversationRefreshResponse(
|
||||
debug_conversation_id=debug_conversation_id,
|
||||
@@ -716,18 +659,16 @@ class AgentPublishApi(Resource):
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@edit_permission_required
|
||||
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.AGENT_MANAGE, resource_required=False)
|
||||
@with_current_user
|
||||
@with_current_tenant_id
|
||||
@with_session
|
||||
def post(self, session: Session, tenant_id: str, current_user: Account, agent_id: UUID):
|
||||
def post(self, tenant_id: str, current_user: Account, agent_id: UUID):
|
||||
args = AgentPublishPayload.model_validate(console_ns.payload or {})
|
||||
return AgentComposerService.publish_agent_app_draft(
|
||||
session=session,
|
||||
tenant_id=tenant_id,
|
||||
agent_id=str(agent_id),
|
||||
account_id=current_user.id,
|
||||
version_note=args.version_note,
|
||||
session=db.session(),
|
||||
)
|
||||
|
||||
|
||||
@@ -739,38 +680,34 @@ class AgentBuildDraftCheckoutApi(Resource):
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@edit_permission_required
|
||||
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.AGENT_MANAGE, resource_required=False)
|
||||
@with_current_user
|
||||
@with_current_tenant_id
|
||||
@with_session
|
||||
def post(self, session: Session, tenant_id: str, current_user: Account, agent_id: UUID):
|
||||
def post(self, tenant_id: str, current_user: Account, agent_id: UUID):
|
||||
args = AgentBuildDraftCheckoutPayload.model_validate(console_ns.payload or {})
|
||||
return AgentComposerService.checkout_agent_app_build_draft(
|
||||
session=session,
|
||||
tenant_id=tenant_id,
|
||||
agent_id=str(agent_id),
|
||||
account_id=current_user.id,
|
||||
force=args.force,
|
||||
session=db.session(),
|
||||
)
|
||||
|
||||
|
||||
@console_ns.route("/agent/<uuid:agent_id>/build-draft")
|
||||
class AgentBuildDraftApi(Resource):
|
||||
@console_ns.response(200, "Agent build draft", console_ns.models[AgentBuildDraftResponse.__name__])
|
||||
@console_ns.response(404, "Agent build draft not found")
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@edit_permission_required
|
||||
@with_current_user
|
||||
@with_current_tenant_id
|
||||
@with_session(write=False)
|
||||
def get(self, session: Session, tenant_id: str, current_user: Account, agent_id: UUID):
|
||||
def get(self, tenant_id: str, current_user: Account, agent_id: UUID):
|
||||
return AgentComposerService.load_agent_app_build_draft(
|
||||
session=session,
|
||||
tenant_id=tenant_id,
|
||||
agent_id=str(agent_id),
|
||||
account_id=current_user.id,
|
||||
session=db.session(),
|
||||
)
|
||||
|
||||
@console_ns.expect(console_ns.models[ComposerSavePayload.__name__])
|
||||
@@ -781,15 +718,14 @@ class AgentBuildDraftApi(Resource):
|
||||
@edit_permission_required
|
||||
@with_current_user
|
||||
@with_current_tenant_id
|
||||
@with_session
|
||||
def put(self, session: Session, tenant_id: str, current_user: Account, agent_id: UUID):
|
||||
def put(self, tenant_id: str, current_user: Account, agent_id: UUID):
|
||||
payload = ComposerSavePayload.model_validate(console_ns.payload or {})
|
||||
return AgentComposerService.save_agent_app_build_draft(
|
||||
session=session,
|
||||
tenant_id=tenant_id,
|
||||
agent_id=str(agent_id),
|
||||
account_id=current_user.id,
|
||||
payload=payload,
|
||||
session=db.session(),
|
||||
)
|
||||
|
||||
@console_ns.response(200, "Agent build draft discarded", console_ns.models[AgentSimpleResultResponse.__name__])
|
||||
@@ -799,13 +735,12 @@ class AgentBuildDraftApi(Resource):
|
||||
@edit_permission_required
|
||||
@with_current_user
|
||||
@with_current_tenant_id
|
||||
@with_session
|
||||
def delete(self, session: Session, tenant_id: str, current_user: Account, agent_id: UUID):
|
||||
def delete(self, tenant_id: str, current_user: Account, agent_id: UUID):
|
||||
return AgentComposerService.discard_agent_app_build_draft(
|
||||
session=session,
|
||||
tenant_id=tenant_id,
|
||||
agent_id=str(agent_id),
|
||||
account_id=current_user.id,
|
||||
session=db.session(),
|
||||
)
|
||||
|
||||
|
||||
@@ -816,16 +751,14 @@ class AgentBuildDraftApplyApi(Resource):
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@edit_permission_required
|
||||
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.AGENT_MANAGE, resource_required=False)
|
||||
@with_current_user
|
||||
@with_current_tenant_id
|
||||
@with_session
|
||||
def post(self, session: Session, tenant_id: str, current_user: Account, agent_id: UUID):
|
||||
def post(self, tenant_id: str, current_user: Account, agent_id: UUID):
|
||||
return AgentComposerService.apply_agent_app_build_draft(
|
||||
session=session,
|
||||
tenant_id=tenant_id,
|
||||
agent_id=str(agent_id),
|
||||
account_id=current_user.id,
|
||||
session=db.session(),
|
||||
)
|
||||
|
||||
|
||||
@@ -839,13 +772,11 @@ class AgentAppCopyApi(Resource):
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@edit_permission_required
|
||||
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.AGENT_MANAGE, resource_required=False)
|
||||
@with_current_user
|
||||
@with_current_tenant_id
|
||||
@with_session
|
||||
def post(self, session: Session, tenant_id: str, current_user: Account, agent_id: UUID):
|
||||
def post(self, tenant_id: str, current_user: Account, agent_id: UUID):
|
||||
args = AgentAppCopyPayload.model_validate(console_ns.payload or {})
|
||||
copied_app = _agent_roster_service(session).duplicate_agent_app(
|
||||
copied_app = _agent_roster_service().duplicate_agent_app(
|
||||
tenant_id=tenant_id,
|
||||
agent_id=str(agent_id),
|
||||
account=current_user,
|
||||
@@ -856,7 +787,7 @@ class AgentAppCopyApi(Resource):
|
||||
icon=args.icon,
|
||||
icon_background=args.icon_background,
|
||||
)
|
||||
return _serialize_agent_app_detail(session, copied_app, current_user=current_user), 201
|
||||
return _serialize_agent_app_detail(copied_app, current_user=current_user), 201
|
||||
|
||||
|
||||
@console_ns.route("/agent/<uuid:agent_id>/api-access")
|
||||
@@ -865,12 +796,10 @@ class AgentApiAccessApi(Resource):
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.AGENT_MANAGE, resource_required=False)
|
||||
@with_current_tenant_id
|
||||
@with_session(write=False)
|
||||
def get(self, session: Session, tenant_id: str, agent_id: UUID):
|
||||
app_model = _resolve_agent_app_model(session, tenant_id=tenant_id, agent_id=agent_id)
|
||||
return _serialize_agent_api_access(session, app_model)
|
||||
def get(self, tenant_id: str, agent_id: UUID):
|
||||
app_model = _resolve_agent_app_model(tenant_id=tenant_id, agent_id=agent_id)
|
||||
return _serialize_agent_api_access(app_model)
|
||||
|
||||
|
||||
@console_ns.route("/agent/<uuid:agent_id>/api-enable")
|
||||
@@ -882,15 +811,13 @@ class AgentApiStatusApi(Resource):
|
||||
@login_required
|
||||
@is_admin_or_owner_required
|
||||
@account_initialization_required
|
||||
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.AGENT_MANAGE, resource_required=False)
|
||||
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_RELEASE_AND_VERSION)
|
||||
@with_current_tenant_id
|
||||
@with_session
|
||||
def post(self, session: Session, tenant_id: str, agent_id: UUID):
|
||||
app_model = _resolve_agent_app_model(session, tenant_id=tenant_id, agent_id=agent_id)
|
||||
def post(self, tenant_id: str, agent_id: UUID):
|
||||
app_model = _resolve_agent_app_model(tenant_id=tenant_id, agent_id=agent_id)
|
||||
args = AgentApiStatusPayload.model_validate(console_ns.payload)
|
||||
app_model = AppService().update_app_api_status(app_model, args.enable_api, session=session)
|
||||
return _serialize_agent_api_access(session, app_model)
|
||||
app_model = AppService().update_app_api_status(app_model, args.enable_api, session=db.session())
|
||||
return _serialize_agent_api_access(app_model)
|
||||
|
||||
|
||||
@console_ns.route("/agent/<uuid:agent_id>/api-keys")
|
||||
@@ -901,26 +828,19 @@ class AgentApiKeyListApi(BaseApiKeyListResource):
|
||||
token_prefix = "app-"
|
||||
|
||||
@console_ns.response(200, "Agent service API keys", console_ns.models[ApiKeyList.__name__])
|
||||
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.AGENT_MANAGE, resource_required=False)
|
||||
@with_current_tenant_id
|
||||
@with_session(write=False)
|
||||
def get(self, session: Session, tenant_id: str, agent_id: UUID) -> dict[str, object]:
|
||||
app_model = _resolve_agent_app_model(session, tenant_id=tenant_id, agent_id=agent_id)
|
||||
return dump_response(ApiKeyList, self._get_api_key_list(str(app_model.id), tenant_id, session=session))
|
||||
def get(self, tenant_id: str, agent_id: UUID) -> dict[str, object]:
|
||||
app_model = _resolve_agent_app_model(tenant_id=tenant_id, agent_id=agent_id)
|
||||
return dump_response(ApiKeyList, self._get_api_key_list(str(app_model.id), tenant_id))
|
||||
|
||||
@console_ns.response(201, "Agent service API key created", console_ns.models[ApiKeyItem.__name__])
|
||||
@console_ns.response(400, "Maximum keys exceeded")
|
||||
@with_current_tenant_id
|
||||
@edit_permission_required
|
||||
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.AGENT_MANAGE, resource_required=False)
|
||||
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_RELEASE_AND_VERSION)
|
||||
@with_session
|
||||
def post(self, session: Session, tenant_id: str, agent_id: UUID) -> tuple[dict[str, object], int]:
|
||||
app_model = _resolve_agent_app_model(session, tenant_id=tenant_id, agent_id=agent_id)
|
||||
return dump_response(
|
||||
ApiKeyItem,
|
||||
self._create_api_key(str(app_model.id), tenant_id, session=session),
|
||||
), 201
|
||||
def post(self, tenant_id: str, agent_id: UUID) -> tuple[dict[str, object], int]:
|
||||
app_model = _resolve_agent_app_model(tenant_id=tenant_id, agent_id=agent_id)
|
||||
return dump_response(ApiKeyItem, self._create_api_key(str(app_model.id), tenant_id)), 201
|
||||
|
||||
|
||||
@console_ns.route("/agent/<uuid:agent_id>/api-keys/<uuid:api_key_id>")
|
||||
@@ -932,19 +852,10 @@ class AgentApiKeyApi(BaseApiKeyResource):
|
||||
@console_ns.response(204, "Agent service API key deleted")
|
||||
@with_current_user
|
||||
@with_current_tenant_id
|
||||
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.AGENT_MANAGE, resource_required=False)
|
||||
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_RELEASE_AND_VERSION)
|
||||
@with_session
|
||||
def delete(
|
||||
self,
|
||||
session: Session,
|
||||
tenant_id: str,
|
||||
current_user: Account,
|
||||
agent_id: UUID,
|
||||
api_key_id: UUID,
|
||||
) -> tuple[str, int]:
|
||||
app_model = _resolve_agent_app_model(session, tenant_id=tenant_id, agent_id=agent_id)
|
||||
self._delete_api_key(str(app_model.id), str(api_key_id), tenant_id, current_user, session=session)
|
||||
def delete(self, tenant_id: str, current_user: Account, agent_id: UUID, api_key_id: UUID) -> tuple[str, int]:
|
||||
app_model = _resolve_agent_app_model(tenant_id=tenant_id, agent_id=agent_id)
|
||||
self._delete_api_key(str(app_model.id), str(api_key_id), tenant_id, current_user)
|
||||
return "", 204
|
||||
|
||||
|
||||
@@ -956,12 +867,11 @@ class AgentInviteOptionsApi(Resource):
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@with_current_tenant_id
|
||||
@with_session(write=False)
|
||||
def get(self, session: Session, tenant_id: str):
|
||||
def get(self, tenant_id: str):
|
||||
query = AgentInviteOptionsQuery.model_validate(request.args.to_dict(flat=True))
|
||||
return dump_response(
|
||||
AgentInviteOptionsResponse,
|
||||
_agent_roster_service(session).list_invite_options(
|
||||
_agent_roster_service().list_invite_options(
|
||||
tenant_id=tenant_id,
|
||||
page=query.page,
|
||||
limit=query.limit,
|
||||
@@ -978,19 +888,17 @@ class AgentLogsApi(Resource):
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.AGENT_MANAGE, resource_required=False)
|
||||
@with_current_user
|
||||
@with_current_tenant_id
|
||||
@with_session(write=False)
|
||||
def get(self, session: Session, tenant_id: str, current_user: Account, agent_id: UUID):
|
||||
app_model = _resolve_agent_runtime_app_model(session, tenant_id=tenant_id, agent_id=agent_id)
|
||||
def get(self, tenant_id: str, current_user: Account, agent_id: UUID):
|
||||
app_model = resolve_agent_runtime_app_model(tenant_id=tenant_id, agent_id=agent_id)
|
||||
query_data: dict[str, object] = dict(request.args.to_dict(flat=True))
|
||||
query_data["sources"] = _query_values("sources", "source")
|
||||
query_data["statuses"] = _query_values("statuses", "status")
|
||||
query = AgentLogsQuery.model_validate(query_data)
|
||||
start, end = _parse_observability_time_range(query.start, query.end, current_user)
|
||||
try:
|
||||
payload = _agent_observability_service(session).list_logs(
|
||||
payload = _agent_observability_service().list_logs(
|
||||
app=app_model,
|
||||
agent_id=str(agent_id),
|
||||
params=AgentLogQueryParams(
|
||||
@@ -1017,19 +925,17 @@ class AgentLogMessagesApi(Resource):
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.AGENT_MANAGE, resource_required=False)
|
||||
@with_current_user
|
||||
@with_current_tenant_id
|
||||
@with_session(write=False)
|
||||
def get(self, session: Session, tenant_id: str, current_user: Account, agent_id: UUID, conversation_id: UUID):
|
||||
app_model = _resolve_agent_runtime_app_model(session, tenant_id=tenant_id, agent_id=agent_id)
|
||||
def get(self, tenant_id: str, current_user: Account, agent_id: UUID, conversation_id: UUID):
|
||||
app_model = resolve_agent_runtime_app_model(tenant_id=tenant_id, agent_id=agent_id)
|
||||
query_data: dict[str, object] = dict(request.args.to_dict(flat=True))
|
||||
query_data["sources"] = _query_values("sources", "source")
|
||||
query_data["statuses"] = _query_values("statuses", "status")
|
||||
query = AgentLogsQuery.model_validate(query_data)
|
||||
start, end = _parse_observability_time_range(query.start, query.end, current_user)
|
||||
try:
|
||||
payload = _agent_observability_service(session).list_log_messages(
|
||||
payload = _agent_observability_service().list_log_messages(
|
||||
app=app_model,
|
||||
agent_id=str(agent_id),
|
||||
conversation_id=str(conversation_id),
|
||||
@@ -1056,13 +962,11 @@ class AgentLogSourcesApi(Resource):
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.AGENT_MANAGE, resource_required=False)
|
||||
@with_current_user
|
||||
@with_current_tenant_id
|
||||
@with_session(write=False)
|
||||
def get(self, session: Session, tenant_id: str, current_user: Account, agent_id: UUID):
|
||||
app_model = _resolve_agent_runtime_app_model(session, tenant_id=tenant_id, agent_id=agent_id)
|
||||
payload = _agent_observability_service(session).list_log_sources(app=app_model, agent_id=str(agent_id))
|
||||
def get(self, tenant_id: str, current_user: Account, agent_id: UUID):
|
||||
app_model = resolve_agent_runtime_app_model(tenant_id=tenant_id, agent_id=agent_id)
|
||||
payload = _agent_observability_service().list_log_sources(app=app_model, agent_id=str(agent_id))
|
||||
return dump_response(AgentLogSourceListResponse, payload)
|
||||
|
||||
|
||||
@@ -1077,17 +981,15 @@ class AgentStatisticsSummaryApi(Resource):
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.AGENT_MANAGE, resource_required=False)
|
||||
@with_current_user
|
||||
@with_current_tenant_id
|
||||
@with_session(write=False)
|
||||
def get(self, session: Session, tenant_id: str, current_user: Account, agent_id: UUID):
|
||||
app_model = _resolve_agent_runtime_app_model(session, tenant_id=tenant_id, agent_id=agent_id)
|
||||
def get(self, tenant_id: str, current_user: Account, agent_id: UUID):
|
||||
app_model = resolve_agent_runtime_app_model(tenant_id=tenant_id, agent_id=agent_id)
|
||||
query = AgentStatisticsQuery.model_validate(request.args.to_dict(flat=True))
|
||||
timezone = current_user.timezone or "UTC"
|
||||
start, end = _parse_observability_time_range(query.start, query.end, current_user)
|
||||
try:
|
||||
payload = _agent_observability_service(session).get_statistics_summary(
|
||||
payload = _agent_observability_service().get_statistics_summary(
|
||||
app=app_model,
|
||||
agent_id=str(agent_id),
|
||||
params=AgentStatisticsQueryParams(source=query.source, start=start, end=end, timezone=timezone),
|
||||
@@ -1103,13 +1005,11 @@ class AgentRosterVersionsApi(Resource):
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.AGENT_MANAGE, resource_required=False)
|
||||
@with_current_tenant_id
|
||||
@with_session(write=False)
|
||||
def get(self, session: Session, tenant_id: str, agent_id: UUID):
|
||||
def get(self, tenant_id: str, agent_id: UUID):
|
||||
return dump_response(
|
||||
AgentConfigSnapshotListResponse,
|
||||
{"data": _agent_roster_service(session).list_agent_versions(tenant_id=tenant_id, agent_id=str(agent_id))},
|
||||
{"data": _agent_roster_service().list_agent_versions(tenant_id=tenant_id, agent_id=str(agent_id))},
|
||||
)
|
||||
|
||||
|
||||
@@ -1119,13 +1019,11 @@ class AgentRosterVersionDetailApi(Resource):
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.AGENT_MANAGE, resource_required=False)
|
||||
@with_current_tenant_id
|
||||
@with_session(write=False)
|
||||
def get(self, session: Session, tenant_id: str, agent_id: UUID, version_id: UUID):
|
||||
def get(self, tenant_id: str, agent_id: UUID, version_id: UUID):
|
||||
return dump_response(
|
||||
AgentConfigSnapshotDetailResponse,
|
||||
_agent_roster_service(session).get_agent_version_detail(
|
||||
_agent_roster_service().get_agent_version_detail(
|
||||
tenant_id=tenant_id,
|
||||
agent_id=str(agent_id),
|
||||
version_id=str(version_id),
|
||||
@@ -1140,14 +1038,12 @@ class AgentRosterVersionRestoreApi(Resource):
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@edit_permission_required
|
||||
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.AGENT_MANAGE, resource_required=False)
|
||||
@with_current_user
|
||||
@with_current_tenant_id
|
||||
@with_session
|
||||
def post(self, session: Session, tenant_id: str, current_user: Account, agent_id: UUID, version_id: UUID):
|
||||
def post(self, tenant_id: str, current_user: Account, agent_id: UUID, version_id: UUID):
|
||||
return dump_response(
|
||||
AgentConfigSnapshotRestoreResponse,
|
||||
_agent_roster_service(session).restore_agent_version(
|
||||
_agent_roster_service().restore_agent_version(
|
||||
tenant_id=tenant_id,
|
||||
agent_id=str(agent_id),
|
||||
version_id=str(version_id),
|
||||
|
||||
@@ -6,13 +6,12 @@ from flask_restx import Resource
|
||||
from flask_restx._http import HTTPStatus
|
||||
from pydantic import field_validator
|
||||
from sqlalchemy import delete, func, select
|
||||
from sqlalchemy.orm import Session
|
||||
from sqlalchemy.orm import sessionmaker
|
||||
from werkzeug.exceptions import Forbidden
|
||||
|
||||
from configs import dify_config
|
||||
from controllers.common.schema import register_response_schema_models
|
||||
from controllers.common.session import with_session
|
||||
from controllers.console.app.wraps import agent_manage_required_for_agent_app
|
||||
from extensions.ext_database import db
|
||||
from fields.base import ResponseModel
|
||||
from libs.helper import dump_response, to_timestamp
|
||||
from libs.login import login_required
|
||||
@@ -55,10 +54,11 @@ class ApiKeyList(ResponseModel):
|
||||
register_response_schema_models(console_ns, ApiKeyItem, ApiKeyList)
|
||||
|
||||
|
||||
def _get_resource(resource_id, tenant_id, resource_model, *, session: Session):
|
||||
resource = session.execute(
|
||||
select(resource_model).filter_by(id=resource_id, tenant_id=tenant_id)
|
||||
).scalar_one_or_none()
|
||||
def _get_resource(resource_id, tenant_id, resource_model):
|
||||
with sessionmaker(db.engine).begin() as session:
|
||||
resource = session.execute(
|
||||
select(resource_model).filter_by(id=resource_id, tenant_id=tenant_id)
|
||||
).scalar_one_or_none()
|
||||
|
||||
if resource is None:
|
||||
flask_restx.abort(HTTPStatus.NOT_FOUND, message=f"{resource_model.__name__} not found.")
|
||||
@@ -75,18 +75,14 @@ class BaseApiKeyListResource(Resource):
|
||||
token_prefix: str | None = None
|
||||
max_keys = 10
|
||||
|
||||
@with_session(write=False)
|
||||
def get(self, session: Session, resource_id: str, current_tenant_id: str) -> dict[str, object]:
|
||||
return dump_response(
|
||||
ApiKeyList,
|
||||
self._get_api_key_list(resource_id, current_tenant_id, session=session),
|
||||
)
|
||||
def get(self, resource_id: str, current_tenant_id: str) -> dict[str, object]:
|
||||
return dump_response(ApiKeyList, self._get_api_key_list(resource_id, current_tenant_id))
|
||||
|
||||
def _get_api_key_list(self, resource_id: str, current_tenant_id: str, *, session: Session) -> ApiKeyList:
|
||||
def _get_api_key_list(self, resource_id: str, current_tenant_id: str) -> ApiKeyList:
|
||||
assert self.resource_id_field is not None, "resource_id_field must be set"
|
||||
|
||||
_get_resource(resource_id, current_tenant_id, self.resource_model, session=session)
|
||||
keys = session.scalars(
|
||||
_get_resource(resource_id, current_tenant_id, self.resource_model)
|
||||
keys = db.session.scalars(
|
||||
select(ApiToken).where(
|
||||
ApiToken.type == self.resource_type, getattr(ApiToken, self.resource_id_field) == resource_id
|
||||
)
|
||||
@@ -94,18 +90,14 @@ class BaseApiKeyListResource(Resource):
|
||||
return ApiKeyList.model_validate({"data": keys}, from_attributes=True)
|
||||
|
||||
@edit_permission_required
|
||||
@with_session
|
||||
def post(self, session: Session, resource_id: str, current_tenant_id: str) -> tuple[dict[str, object], int]:
|
||||
return dump_response(
|
||||
ApiKeyItem,
|
||||
self._create_api_key(resource_id, current_tenant_id, session=session),
|
||||
), 201
|
||||
def post(self, resource_id: str, current_tenant_id: str) -> tuple[dict[str, object], int]:
|
||||
return dump_response(ApiKeyItem, self._create_api_key(resource_id, current_tenant_id)), 201
|
||||
|
||||
def _create_api_key(self, resource_id: str, current_tenant_id: str, *, session: Session) -> ApiToken:
|
||||
def _create_api_key(self, resource_id: str, current_tenant_id: str) -> ApiToken:
|
||||
assert self.resource_id_field is not None, "resource_id_field must be set"
|
||||
_get_resource(resource_id, current_tenant_id, self.resource_model, session=session)
|
||||
_get_resource(resource_id, current_tenant_id, self.resource_model)
|
||||
current_key_count: int = (
|
||||
session.scalar(
|
||||
db.session.scalar(
|
||||
select(func.count(ApiToken.id)).where(
|
||||
ApiToken.type == self.resource_type, getattr(ApiToken, self.resource_id_field) == resource_id
|
||||
)
|
||||
@@ -120,15 +112,15 @@ class BaseApiKeyListResource(Resource):
|
||||
custom="max_keys_exceeded",
|
||||
)
|
||||
|
||||
key = ApiToken.generate_api_key(self.token_prefix or "", 24, session=session)
|
||||
key = ApiToken.generate_api_key(self.token_prefix or "", 24)
|
||||
assert self.resource_type is not None, "resource_type must be set"
|
||||
api_token = ApiToken()
|
||||
setattr(api_token, self.resource_id_field, resource_id)
|
||||
api_token.tenant_id = current_tenant_id
|
||||
api_token.token = key
|
||||
api_token.type = self.resource_type
|
||||
session.add(api_token)
|
||||
session.commit()
|
||||
db.session.add(api_token)
|
||||
db.session.commit()
|
||||
return api_token
|
||||
|
||||
|
||||
@@ -139,16 +131,10 @@ class BaseApiKeyResource(Resource):
|
||||
resource_model: type | None = None
|
||||
resource_id_field: str | None = None
|
||||
|
||||
@with_session
|
||||
def delete(
|
||||
self,
|
||||
session: Session,
|
||||
resource_id: str,
|
||||
api_key_id: str,
|
||||
current_tenant_id: str,
|
||||
current_user: Account,
|
||||
self, resource_id: str, api_key_id: str, current_tenant_id: str, current_user: Account
|
||||
) -> tuple[str, int]:
|
||||
self._delete_api_key(resource_id, api_key_id, current_tenant_id, current_user, session=session)
|
||||
self._delete_api_key(resource_id, api_key_id, current_tenant_id, current_user)
|
||||
return "", 204
|
||||
|
||||
def _delete_api_key(
|
||||
@@ -157,16 +143,14 @@ class BaseApiKeyResource(Resource):
|
||||
api_key_id: str,
|
||||
current_tenant_id: str,
|
||||
current_user: Account,
|
||||
*,
|
||||
session: Session,
|
||||
) -> None:
|
||||
assert self.resource_id_field is not None, "resource_id_field must be set"
|
||||
_get_resource(resource_id, current_tenant_id, self.resource_model, session=session)
|
||||
_get_resource(resource_id, current_tenant_id, self.resource_model)
|
||||
|
||||
if not dify_config.RBAC_ENABLED and not current_user.is_admin_or_owner:
|
||||
raise Forbidden()
|
||||
|
||||
key = session.scalar(
|
||||
key = db.session.scalar(
|
||||
select(ApiToken)
|
||||
.where(
|
||||
getattr(ApiToken, self.resource_id_field) == resource_id,
|
||||
@@ -184,8 +168,8 @@ class BaseApiKeyResource(Resource):
|
||||
assert key is not None # nosec - for type checker only
|
||||
ApiTokenCache.delete(key.token, key.type)
|
||||
|
||||
session.execute(delete(ApiToken).where(ApiToken.id == api_key_id))
|
||||
session.commit()
|
||||
db.session.execute(delete(ApiToken).where(ApiToken.id == api_key_id))
|
||||
db.session.commit()
|
||||
|
||||
|
||||
@console_ns.route("/apps/<uuid:resource_id>/api-keys")
|
||||
@@ -195,14 +179,9 @@ class AppApiKeyListResource(BaseApiKeyListResource):
|
||||
@console_ns.doc(params={"resource_id": "App ID"})
|
||||
@console_ns.response(200, "API keys retrieved successfully", console_ns.models[ApiKeyList.__name__])
|
||||
@with_current_tenant_id
|
||||
@agent_manage_required_for_agent_app
|
||||
@with_session(write=False)
|
||||
def get(self, session: Session, current_tenant_id: str, resource_id: UUID) -> dict[str, object]:
|
||||
def get(self, current_tenant_id: str, resource_id: UUID) -> dict[str, object]:
|
||||
"""Get all API keys for an app"""
|
||||
return dump_response(
|
||||
ApiKeyList,
|
||||
self._get_api_key_list(str(resource_id), current_tenant_id, session=session),
|
||||
)
|
||||
return dump_response(ApiKeyList, self._get_api_key_list(str(resource_id), current_tenant_id))
|
||||
|
||||
@console_ns.doc("create_app_api_key")
|
||||
@console_ns.doc(description="Create a new API key for an app")
|
||||
@@ -212,14 +191,9 @@ class AppApiKeyListResource(BaseApiKeyListResource):
|
||||
@with_current_tenant_id
|
||||
@edit_permission_required
|
||||
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_RELEASE_AND_VERSION)
|
||||
@agent_manage_required_for_agent_app
|
||||
@with_session
|
||||
def post(self, session: Session, current_tenant_id: str, resource_id: UUID) -> tuple[dict[str, object], int]:
|
||||
def post(self, current_tenant_id: str, resource_id: UUID) -> tuple[dict[str, object], int]:
|
||||
"""Create a new API key for an app"""
|
||||
return dump_response(
|
||||
ApiKeyItem,
|
||||
self._create_api_key(str(resource_id), current_tenant_id, session=session),
|
||||
), 201
|
||||
return dump_response(ApiKeyItem, self._create_api_key(str(resource_id), current_tenant_id)), 201
|
||||
|
||||
resource_type = ApiTokenType.APP
|
||||
resource_model = App
|
||||
@@ -236,24 +210,11 @@ class AppApiKeyResource(BaseApiKeyResource):
|
||||
@with_current_user
|
||||
@with_current_tenant_id
|
||||
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_RELEASE_AND_VERSION)
|
||||
@agent_manage_required_for_agent_app
|
||||
@with_session
|
||||
def delete(
|
||||
self,
|
||||
session: Session,
|
||||
current_tenant_id: str,
|
||||
current_user: Account,
|
||||
resource_id: UUID,
|
||||
api_key_id: UUID,
|
||||
self, current_tenant_id: str, current_user: Account, resource_id: UUID, api_key_id: UUID
|
||||
) -> tuple[str, int]:
|
||||
"""Delete an API key for an app"""
|
||||
self._delete_api_key(
|
||||
str(resource_id),
|
||||
str(api_key_id),
|
||||
current_tenant_id,
|
||||
current_user,
|
||||
session=session,
|
||||
)
|
||||
self._delete_api_key(str(resource_id), str(api_key_id), current_tenant_id, current_user)
|
||||
return "", 204
|
||||
|
||||
resource_type = ApiTokenType.APP
|
||||
@@ -268,13 +229,9 @@ class DatasetApiKeyListResource(BaseApiKeyListResource):
|
||||
@console_ns.doc(params={"resource_id": "Dataset ID"})
|
||||
@console_ns.response(200, "API keys retrieved successfully", console_ns.models[ApiKeyList.__name__])
|
||||
@with_current_tenant_id
|
||||
@with_session(write=False)
|
||||
def get(self, session: Session, current_tenant_id: str, resource_id: UUID) -> dict[str, object]:
|
||||
def get(self, current_tenant_id: str, resource_id: UUID) -> dict[str, object]:
|
||||
"""Get all API keys for a dataset"""
|
||||
return dump_response(
|
||||
ApiKeyList,
|
||||
self._get_api_key_list(str(resource_id), current_tenant_id, session=session),
|
||||
)
|
||||
return dump_response(ApiKeyList, self._get_api_key_list(str(resource_id), current_tenant_id))
|
||||
|
||||
@console_ns.doc("create_dataset_api_key")
|
||||
@console_ns.doc(description="Create a new API key for a dataset")
|
||||
@@ -284,13 +241,9 @@ class DatasetApiKeyListResource(BaseApiKeyListResource):
|
||||
@with_current_tenant_id
|
||||
@edit_permission_required
|
||||
@rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_API_KEY_MANAGE)
|
||||
@with_session
|
||||
def post(self, session: Session, current_tenant_id: str, resource_id: UUID) -> tuple[dict[str, object], int]:
|
||||
def post(self, current_tenant_id: str, resource_id: UUID) -> tuple[dict[str, object], int]:
|
||||
"""Create a new API key for a dataset"""
|
||||
return dump_response(
|
||||
ApiKeyItem,
|
||||
self._create_api_key(str(resource_id), current_tenant_id, session=session),
|
||||
), 201
|
||||
return dump_response(ApiKeyItem, self._create_api_key(str(resource_id), current_tenant_id)), 201
|
||||
|
||||
resource_type = ApiTokenType.DATASET
|
||||
resource_model = Dataset
|
||||
@@ -307,23 +260,11 @@ class DatasetApiKeyResource(BaseApiKeyResource):
|
||||
@with_current_user
|
||||
@with_current_tenant_id
|
||||
@rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_API_KEY_MANAGE)
|
||||
@with_session
|
||||
def delete(
|
||||
self,
|
||||
session: Session,
|
||||
current_tenant_id: str,
|
||||
current_user: Account,
|
||||
resource_id: UUID,
|
||||
api_key_id: UUID,
|
||||
self, current_tenant_id: str, current_user: Account, resource_id: UUID, api_key_id: UUID
|
||||
) -> tuple[str, int]:
|
||||
"""Delete an API key for a dataset"""
|
||||
self._delete_api_key(
|
||||
str(resource_id),
|
||||
str(api_key_id),
|
||||
current_tenant_id,
|
||||
current_user,
|
||||
session=session,
|
||||
)
|
||||
self._delete_api_key(str(resource_id), str(api_key_id), current_tenant_id, current_user)
|
||||
return "", 204
|
||||
|
||||
resource_type = ApiTokenType.DATASET
|
||||
|
||||
@@ -5,7 +5,6 @@ from flask import request
|
||||
from flask_restx import Resource
|
||||
from pydantic import BaseModel, Field, field_validator
|
||||
from sqlalchemy import select
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from controllers.common.schema import (
|
||||
query_params_from_model,
|
||||
@@ -13,7 +12,6 @@ from controllers.common.schema import (
|
||||
register_response_schema_models,
|
||||
register_schema_models,
|
||||
)
|
||||
from controllers.common.session import with_session
|
||||
from controllers.console import console_ns
|
||||
from controllers.console.agent.app_helpers import resolve_agent_runtime_app_model
|
||||
from controllers.console.app.wraps import get_app_model
|
||||
@@ -26,6 +24,7 @@ from controllers.console.wraps import (
|
||||
with_current_tenant_id,
|
||||
with_current_user,
|
||||
)
|
||||
from extensions.ext_database import db
|
||||
from fields.base import ResponseModel
|
||||
from libs.helper import uuid_value
|
||||
from libs.login import login_required
|
||||
@@ -170,23 +169,23 @@ register_response_schema_models(
|
||||
)
|
||||
|
||||
|
||||
def _resolve_agent_id(session: Session, app_model: App, node_id: str | None) -> str | None:
|
||||
def _resolve_agent_id(app_model: App, node_id: str | None) -> str | None:
|
||||
if node_id and app_model.mode != AppMode.AGENT:
|
||||
return AgentComposerService.resolve_workflow_node_agent_id(
|
||||
session=session, tenant_id=app_model.tenant_id, app_id=app_model.id, node_id=node_id
|
||||
tenant_id=app_model.tenant_id, app_id=app_model.id, node_id=node_id, session=db.session()
|
||||
)
|
||||
return app_model.bound_agent_id_with_session(session=session)
|
||||
return app_model.bound_agent_id
|
||||
|
||||
|
||||
def _agent_not_bound() -> tuple[dict[str, str], int]:
|
||||
return {"code": "agent_not_bound", "message": "no agent is bound for this app/node"}, 400
|
||||
|
||||
|
||||
def _upload_skill_for_app(*, session: Session, current_user: Account, app_model: App):
|
||||
def _upload_skill_for_app(*, current_user: Account, app_model: App):
|
||||
"""Upload one skill package and commit its normalized files into the agent drive."""
|
||||
|
||||
query = query_params_from_request(AgentDriveMutationQuery)
|
||||
agent_id = _resolve_agent_id(session, app_model, query.node_id)
|
||||
agent_id = _resolve_agent_id(app_model, query.node_id)
|
||||
if not agent_id:
|
||||
return _agent_not_bound()
|
||||
if "file" not in request.files:
|
||||
@@ -203,22 +202,22 @@ def _upload_skill_for_app(*, session: Session, current_user: Account, app_model:
|
||||
tenant_id=app_model.tenant_id,
|
||||
user_id=current_user.id,
|
||||
agent_id=agent_id,
|
||||
session=session,
|
||||
session=db.session(),
|
||||
)
|
||||
except (SkillPackageError, AgentDriveError) as exc:
|
||||
return {"code": exc.code, "message": exc.message}, exc.status_code
|
||||
return result, 201
|
||||
|
||||
|
||||
def _commit_drive_file_for_app(*, session: Session, current_user: Account, app_model: App, allow_node_id: bool = True):
|
||||
def _commit_drive_file_for_app(*, current_user: Account, app_model: App, allow_node_id: bool = True):
|
||||
query = query_params_from_request(AgentDriveMutationQuery)
|
||||
node_id = query.node_id if allow_node_id else None
|
||||
agent_id = _resolve_agent_id(session, app_model, node_id)
|
||||
agent_id = _resolve_agent_id(app_model, node_id)
|
||||
if not agent_id:
|
||||
return _agent_not_bound()
|
||||
payload = AgentDriveFilePayload.model_validate(console_ns.payload or {})
|
||||
|
||||
upload_file = session.scalar(
|
||||
upload_file = db.session.scalar(
|
||||
select(UploadFile).where(
|
||||
UploadFile.id == payload.upload_file_id,
|
||||
UploadFile.tenant_id == app_model.tenant_id,
|
||||
@@ -242,7 +241,7 @@ def _commit_drive_file_for_app(*, session: Session, current_user: Account, app_m
|
||||
value_owned_by_drive=True,
|
||||
)
|
||||
],
|
||||
session=session,
|
||||
session=db.session(),
|
||||
)
|
||||
except AgentDriveError as exc:
|
||||
return {"code": exc.code, "message": exc.message}, exc.status_code
|
||||
@@ -259,10 +258,10 @@ def _commit_drive_file_for_app(*, session: Session, current_user: Account, app_m
|
||||
}, 201
|
||||
|
||||
|
||||
def _delete_drive_file_for_app(*, session: Session, current_user: Account, app_model: App, allow_node_id: bool = True):
|
||||
def _delete_drive_file_for_app(*, current_user: Account, app_model: App, allow_node_id: bool = True):
|
||||
query = query_params_from_request(AgentDriveDeleteFileQuery)
|
||||
node_id = query.node_id if allow_node_id else None
|
||||
agent_id = _resolve_agent_id(session, app_model, node_id)
|
||||
agent_id = _resolve_agent_id(app_model, node_id)
|
||||
if not agent_id:
|
||||
return _agent_not_bound()
|
||||
try:
|
||||
@@ -276,7 +275,7 @@ def _delete_drive_file_for_app(*, session: Session, current_user: Account, app_m
|
||||
user_id=current_user.id,
|
||||
agent_id=agent_id,
|
||||
items=[DriveCommitItem(key=key, file_ref=None)],
|
||||
session=session,
|
||||
session=db.session(),
|
||||
)
|
||||
except AgentDriveError as exc:
|
||||
return {"code": exc.code, "message": exc.message}, exc.status_code
|
||||
@@ -284,12 +283,10 @@ def _delete_drive_file_for_app(*, session: Session, current_user: Account, app_m
|
||||
return {"result": "success", "removed_keys": removed_keys}
|
||||
|
||||
|
||||
def _delete_skill_for_app(
|
||||
*, session: Session, current_user: Account, app_model: App, slug: str, allow_node_id: bool = True
|
||||
):
|
||||
def _delete_skill_for_app(*, current_user: Account, app_model: App, slug: str, allow_node_id: bool = True):
|
||||
query = query_params_from_request(AgentDriveMutationQuery)
|
||||
node_id = query.node_id if allow_node_id else None
|
||||
agent_id = _resolve_agent_id(session, app_model, node_id)
|
||||
agent_id = _resolve_agent_id(app_model, node_id)
|
||||
if not agent_id:
|
||||
return _agent_not_bound()
|
||||
if "/" in slug or not slug.strip():
|
||||
@@ -304,7 +301,7 @@ def _delete_skill_for_app(
|
||||
DriveCommitItem(key=f"{slug}/SKILL.md", file_ref=None),
|
||||
DriveCommitItem(key=f"{slug}/.DIFY-SKILL-FULL.zip", file_ref=None),
|
||||
],
|
||||
session=session,
|
||||
session=db.session(),
|
||||
)
|
||||
except AgentDriveError as exc:
|
||||
return {"code": exc.code, "message": exc.message}, exc.status_code
|
||||
@@ -312,16 +309,16 @@ def _delete_skill_for_app(
|
||||
return {"result": "success", "removed_keys": removed_keys}
|
||||
|
||||
|
||||
def _infer_skill_tools_for_app(*, session: Session, app_model: App, slug: str):
|
||||
def _infer_skill_tools_for_app(*, app_model: App, slug: str):
|
||||
query = query_params_from_request(AgentDriveMutationQuery)
|
||||
agent_id = _resolve_agent_id(session, app_model, query.node_id)
|
||||
agent_id = _resolve_agent_id(app_model, query.node_id)
|
||||
if not agent_id:
|
||||
return _agent_not_bound()
|
||||
if "/" in slug or not slug.strip():
|
||||
return {"code": "drive_key_invalid", "message": "skill slug must be a single path segment"}, 400
|
||||
try:
|
||||
return SkillToolInferenceService().infer(
|
||||
tenant_id=app_model.tenant_id, agent_id=agent_id, slug=slug, session=session
|
||||
tenant_id=app_model.tenant_id, agent_id=agent_id, slug=slug, session=db.session()
|
||||
)
|
||||
except SkillToolInferenceError as exc:
|
||||
return {"code": exc.code, "message": exc.message}, exc.status_code
|
||||
@@ -339,13 +336,12 @@ class AgentLogApi(Resource):
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_VIEW_LAYOUT)
|
||||
@with_session(write=False)
|
||||
@get_app_model(mode=[AppMode.AGENT_CHAT])
|
||||
def get(self, session: Session, app_model: App):
|
||||
def get(self, app_model: App):
|
||||
"""Get agent logs"""
|
||||
args = AgentLogQuery.model_validate(request.args.to_dict(flat=True))
|
||||
|
||||
return AgentService.get_agent_logs(app_model, args.conversation_id, args.message_id, session)
|
||||
return AgentService.get_agent_logs(app_model, args.conversation_id, args.message_id, db.session())
|
||||
|
||||
|
||||
@console_ns.route("/agent/<uuid:agent_id>/skills/upload")
|
||||
@@ -360,10 +356,9 @@ class AgentSkillUploadByAgentApi(Resource):
|
||||
@account_initialization_required
|
||||
@with_current_user
|
||||
@with_current_tenant_id
|
||||
@with_session
|
||||
def post(self, session: Session, tenant_id: str, current_user: Account, agent_id: UUID):
|
||||
app_model = resolve_agent_runtime_app_model(session=session, tenant_id=tenant_id, agent_id=agent_id)
|
||||
return _upload_skill_for_app(session=session, current_user=current_user, app_model=app_model)
|
||||
def post(self, tenant_id: str, current_user: Account, agent_id: UUID):
|
||||
app_model = resolve_agent_runtime_app_model(tenant_id=tenant_id, agent_id=agent_id)
|
||||
return _upload_skill_for_app(current_user=current_user, app_model=app_model)
|
||||
|
||||
|
||||
@console_ns.route("/apps/<uuid:app_id>/agent/skills/upload")
|
||||
@@ -383,12 +378,11 @@ class AgentSkillUploadApi(Resource):
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@with_current_user
|
||||
@with_session
|
||||
@get_app_model(mode=_WORKFLOW_AGENT_DRIVE_APP_MODES)
|
||||
def post(self, session: Session, current_user: Account, app_model: App):
|
||||
@with_current_user
|
||||
def post(self, current_user: Account, app_model: App):
|
||||
"""Upload a Skill, validate it, and commit drive-backed skill files."""
|
||||
return _upload_skill_for_app(session=session, current_user=current_user, app_model=app_model)
|
||||
return _upload_skill_for_app(current_user=current_user, app_model=app_model)
|
||||
|
||||
|
||||
@console_ns.route("/agent/<uuid:agent_id>/files")
|
||||
@@ -405,12 +399,9 @@ class AgentDriveFilesByAgentApi(Resource):
|
||||
@account_initialization_required
|
||||
@with_current_user
|
||||
@with_current_tenant_id
|
||||
@with_session
|
||||
def post(self, session: Session, tenant_id: str, current_user: Account, agent_id: UUID):
|
||||
app_model = resolve_agent_runtime_app_model(session=session, tenant_id=tenant_id, agent_id=agent_id)
|
||||
return _commit_drive_file_for_app(
|
||||
session=session, current_user=current_user, app_model=app_model, allow_node_id=False
|
||||
)
|
||||
def post(self, tenant_id: str, current_user: Account, agent_id: UUID):
|
||||
app_model = resolve_agent_runtime_app_model(tenant_id=tenant_id, agent_id=agent_id)
|
||||
return _commit_drive_file_for_app(current_user=current_user, app_model=app_model, allow_node_id=False)
|
||||
|
||||
@console_ns.doc("delete_agent_drive_file_by_agent")
|
||||
@console_ns.doc(description="Delete one Agent App drive file by key")
|
||||
@@ -421,12 +412,9 @@ class AgentDriveFilesByAgentApi(Resource):
|
||||
@account_initialization_required
|
||||
@with_current_user
|
||||
@with_current_tenant_id
|
||||
@with_session
|
||||
def delete(self, session: Session, tenant_id: str, current_user: Account, agent_id: UUID):
|
||||
app_model = resolve_agent_runtime_app_model(session=session, tenant_id=tenant_id, agent_id=agent_id)
|
||||
return _delete_drive_file_for_app(
|
||||
session=session, current_user=current_user, app_model=app_model, allow_node_id=False
|
||||
)
|
||||
def delete(self, tenant_id: str, current_user: Account, agent_id: UUID):
|
||||
app_model = resolve_agent_runtime_app_model(tenant_id=tenant_id, agent_id=agent_id)
|
||||
return _delete_drive_file_for_app(current_user=current_user, app_model=app_model, allow_node_id=False)
|
||||
|
||||
|
||||
@console_ns.route("/apps/<uuid:app_id>/agent/files")
|
||||
@@ -441,12 +429,11 @@ class AgentDriveFilesApi(Resource):
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@with_current_user
|
||||
@with_session
|
||||
@get_app_model(mode=_WORKFLOW_AGENT_DRIVE_APP_MODES)
|
||||
def post(self, session: Session, current_user: Account, app_model: App):
|
||||
@with_current_user
|
||||
def post(self, current_user: Account, app_model: App):
|
||||
"""ADD FILE: commit one uploaded file into the bound agent's drive."""
|
||||
return _commit_drive_file_for_app(session=session, current_user=current_user, app_model=app_model)
|
||||
return _commit_drive_file_for_app(current_user=current_user, app_model=app_model)
|
||||
|
||||
@console_ns.doc("delete_agent_drive_file")
|
||||
@console_ns.doc(description="Delete one drive file by key via drive commit-null semantics")
|
||||
@@ -455,11 +442,10 @@ class AgentDriveFilesApi(Resource):
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@with_current_user
|
||||
@with_session
|
||||
@get_app_model(mode=_WORKFLOW_AGENT_DRIVE_APP_MODES)
|
||||
def delete(self, session: Session, current_user: Account, app_model: App):
|
||||
return _delete_drive_file_for_app(session=session, current_user=current_user, app_model=app_model)
|
||||
@with_current_user
|
||||
def delete(self, current_user: Account, app_model: App):
|
||||
return _delete_drive_file_for_app(current_user=current_user, app_model=app_model)
|
||||
|
||||
|
||||
@console_ns.route("/agent/<uuid:agent_id>/skills/<string:slug>")
|
||||
@@ -473,12 +459,9 @@ class AgentSkillByAgentApi(Resource):
|
||||
@account_initialization_required
|
||||
@with_current_user
|
||||
@with_current_tenant_id
|
||||
@with_session
|
||||
def delete(self, session: Session, tenant_id: str, current_user: Account, agent_id: UUID, slug: str):
|
||||
app_model = resolve_agent_runtime_app_model(session=session, tenant_id=tenant_id, agent_id=agent_id)
|
||||
return _delete_skill_for_app(
|
||||
session=session, current_user=current_user, app_model=app_model, slug=slug, allow_node_id=False
|
||||
)
|
||||
def delete(self, tenant_id: str, current_user: Account, agent_id: UUID, slug: str):
|
||||
app_model = resolve_agent_runtime_app_model(tenant_id=tenant_id, agent_id=agent_id)
|
||||
return _delete_skill_for_app(current_user=current_user, app_model=app_model, slug=slug, allow_node_id=False)
|
||||
|
||||
|
||||
@console_ns.route("/apps/<uuid:app_id>/agent/skills/<string:slug>")
|
||||
@@ -496,11 +479,10 @@ class AgentSkillApi(Resource):
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@with_current_user
|
||||
@with_session
|
||||
@get_app_model(mode=_WORKFLOW_AGENT_DRIVE_APP_MODES)
|
||||
def delete(self, session: Session, current_user: Account, app_model: App, slug: str):
|
||||
return _delete_skill_for_app(session=session, current_user=current_user, app_model=app_model, slug=slug)
|
||||
@with_current_user
|
||||
def delete(self, current_user: Account, app_model: App, slug: str):
|
||||
return _delete_skill_for_app(current_user=current_user, app_model=app_model, slug=slug)
|
||||
|
||||
|
||||
@console_ns.route("/agent/<uuid:agent_id>/skills/<string:slug>/infer-tools")
|
||||
@@ -517,10 +499,9 @@ class AgentSkillInferToolsByAgentApi(Resource):
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@with_current_tenant_id
|
||||
@with_session(write=False)
|
||||
def post(self, session: Session, tenant_id: str, agent_id: UUID, slug: str):
|
||||
app_model = resolve_agent_runtime_app_model(session=session, tenant_id=tenant_id, agent_id=agent_id)
|
||||
return _infer_skill_tools_for_app(session=session, app_model=app_model, slug=slug)
|
||||
def post(self, tenant_id: str, agent_id: UUID, slug: str):
|
||||
app_model = resolve_agent_runtime_app_model(tenant_id=tenant_id, agent_id=agent_id)
|
||||
return _infer_skill_tools_for_app(app_model=app_model, slug=slug)
|
||||
|
||||
|
||||
@console_ns.route("/apps/<uuid:app_id>/agent/skills/<string:slug>/infer-tools")
|
||||
@@ -544,8 +525,7 @@ class AgentSkillInferToolsApi(Resource):
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@with_session(write=False)
|
||||
@get_app_model(mode=_WORKFLOW_AGENT_DRIVE_APP_MODES)
|
||||
def post(self, session: Session, app_model: App, slug: str):
|
||||
def post(self, app_model: App, slug: str):
|
||||
"""Suggest CLI tools/env for a skill. Saving still goes through composer validation."""
|
||||
return _infer_skill_tools_for_app(session=session, app_model=app_model, slug=slug)
|
||||
return _infer_skill_tools_for_app(app_model=app_model, slug=slug)
|
||||
|
||||
@@ -9,13 +9,12 @@ from uuid import UUID
|
||||
|
||||
from flask_restx import Resource
|
||||
from pydantic import Field
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from controllers.common.schema import register_response_schema_models
|
||||
from controllers.common.session import with_session
|
||||
from controllers.console import console_ns
|
||||
from controllers.console.agent.app_helpers import resolve_agent_app_model
|
||||
from controllers.console.wraps import account_initialization_required, setup_required, with_current_tenant_id
|
||||
from extensions.ext_database import db
|
||||
from fields.base import ResponseModel
|
||||
from libs.login import login_required
|
||||
from services.agent.roster_service import AgentRosterService
|
||||
@@ -56,10 +55,9 @@ class AgentAppReferencingWorkflowsResource(Resource):
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@with_current_tenant_id
|
||||
@with_session(write=False)
|
||||
def get(self, session: Session, tenant_id: str, agent_id: UUID):
|
||||
app_model = resolve_agent_app_model(session=session, tenant_id=tenant_id, agent_id=agent_id)
|
||||
workflows = AgentRosterService(session).list_workflows_referencing_app_agent(
|
||||
def get(self, tenant_id: str, agent_id: UUID):
|
||||
app_model = resolve_agent_app_model(tenant_id=tenant_id, agent_id=agent_id)
|
||||
workflows = AgentRosterService(db.session).list_workflows_referencing_app_agent(
|
||||
tenant_id=tenant_id, app_id=app_model.id
|
||||
)
|
||||
return AgentReferencingWorkflowsResponse(
|
||||
|
||||
@@ -13,11 +13,9 @@ from uuid import UUID
|
||||
|
||||
from flask_restx import Resource
|
||||
from pydantic import BaseModel, Field
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from controllers.common.fields import SimpleResultResponse
|
||||
from controllers.common.schema import register_response_schema_models, register_schema_models
|
||||
from controllers.common.session import with_session
|
||||
from controllers.console import console_ns
|
||||
from controllers.console.agent.app_helpers import resolve_agent_runtime_app_model
|
||||
from controllers.console.wraps import (
|
||||
@@ -31,6 +29,7 @@ from controllers.console.wraps import (
|
||||
with_current_user,
|
||||
)
|
||||
from events.app_event import app_model_config_was_updated
|
||||
from extensions.ext_database import db
|
||||
from libs.login import login_required
|
||||
from models import Account
|
||||
from models.agent_config_entities import (
|
||||
@@ -86,23 +85,17 @@ class AgentAppFeatureConfigResource(Resource):
|
||||
@account_initialization_required
|
||||
@with_current_user
|
||||
@with_current_tenant_id
|
||||
@with_session
|
||||
def post(self, session: Session, tenant_id: str, current_user: Account, agent_id: UUID):
|
||||
app_model = resolve_agent_runtime_app_model(session=session, tenant_id=tenant_id, agent_id=agent_id)
|
||||
def post(self, tenant_id: str, current_user: Account, agent_id: UUID):
|
||||
app_model = resolve_agent_runtime_app_model(tenant_id=tenant_id, agent_id=agent_id)
|
||||
args = AgentAppFeaturesPayload.model_validate(console_ns.payload or {})
|
||||
|
||||
new_app_model_config = AgentAppFeatureConfigService.update_features(
|
||||
app_model=app_model,
|
||||
account=current_user,
|
||||
config=args.model_dump(exclude_none=True),
|
||||
session=session,
|
||||
session=db.session(),
|
||||
)
|
||||
|
||||
app_model_config_was_updated.send(
|
||||
app_model,
|
||||
app_model_config=new_app_model_config,
|
||||
session=session,
|
||||
)
|
||||
session.commit()
|
||||
app_model_config_was_updated.send(app_model, app_model_config=new_app_model_config)
|
||||
|
||||
return SimpleResultResponse(result="success").model_dump(mode="json")
|
||||
|
||||
@@ -14,7 +14,6 @@ from dify_agent.client import DifyAgentClientError, DifyAgentHTTPError, DifyAgen
|
||||
from flask import request
|
||||
from flask_restx import Resource
|
||||
from pydantic import BaseModel, Field
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from controllers.common.schema import (
|
||||
query_params_from_model,
|
||||
@@ -22,7 +21,6 @@ from controllers.common.schema import (
|
||||
register_response_schema_models,
|
||||
register_schema_models,
|
||||
)
|
||||
from controllers.common.session import with_session
|
||||
from controllers.console import console_ns
|
||||
from controllers.console.agent.app_helpers import resolve_agent_runtime_app_model
|
||||
from controllers.console.app.wraps import get_app_model
|
||||
@@ -155,9 +153,8 @@ class AgentAppSandboxInfoResource(Resource):
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@with_current_tenant_id
|
||||
@with_session(write=False)
|
||||
def get(self, session: Session, tenant_id: str, agent_id: UUID):
|
||||
app_model = resolve_agent_runtime_app_model(session=session, tenant_id=tenant_id, agent_id=agent_id)
|
||||
def get(self, tenant_id: str, agent_id: UUID):
|
||||
app_model = resolve_agent_runtime_app_model(tenant_id=tenant_id, agent_id=agent_id)
|
||||
query = query_params_from_request(AgentSandboxInfoQuery)
|
||||
try:
|
||||
result = AgentAppSandboxService().get_info(
|
||||
@@ -180,9 +177,8 @@ class AgentAppSandboxListResource(Resource):
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@with_current_tenant_id
|
||||
@with_session(write=False)
|
||||
def get(self, session: Session, tenant_id: str, agent_id: UUID):
|
||||
app_model = resolve_agent_runtime_app_model(session=session, tenant_id=tenant_id, agent_id=agent_id)
|
||||
def get(self, tenant_id: str, agent_id: UUID):
|
||||
app_model = resolve_agent_runtime_app_model(tenant_id=tenant_id, agent_id=agent_id)
|
||||
query = query_params_from_request(AgentSandboxListQuery)
|
||||
try:
|
||||
result = AgentAppSandboxService().list_files(
|
||||
@@ -206,9 +202,8 @@ class AgentAppSandboxReadResource(Resource):
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@with_current_tenant_id
|
||||
@with_session(write=False)
|
||||
def get(self, session: Session, tenant_id: str, agent_id: UUID):
|
||||
app_model = resolve_agent_runtime_app_model(session=session, tenant_id=tenant_id, agent_id=agent_id)
|
||||
def get(self, tenant_id: str, agent_id: UUID):
|
||||
app_model = resolve_agent_runtime_app_model(tenant_id=tenant_id, agent_id=agent_id)
|
||||
query = query_params_from_request(AgentSandboxFileQuery)
|
||||
try:
|
||||
result = AgentAppSandboxService().read_file(
|
||||
@@ -232,9 +227,8 @@ class AgentAppSandboxUploadResource(Resource):
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@with_current_tenant_id
|
||||
@with_session(write=False)
|
||||
def post(self, session: Session, tenant_id: str, agent_id: UUID):
|
||||
app_model = resolve_agent_runtime_app_model(session=session, tenant_id=tenant_id, agent_id=agent_id)
|
||||
def post(self, tenant_id: str, agent_id: UUID):
|
||||
app_model = resolve_agent_runtime_app_model(tenant_id=tenant_id, agent_id=agent_id)
|
||||
payload = AgentSandboxUploadPayload.model_validate(request.get_json(silent=True) or {})
|
||||
try:
|
||||
result = AgentAppSandboxService().upload_file(
|
||||
|
||||
@@ -13,7 +13,6 @@ from uuid import UUID
|
||||
from flask import Response, request, send_file, url_for
|
||||
from flask_restx import Resource
|
||||
from pydantic import BaseModel, Field
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from controllers.common.schema import (
|
||||
query_params_from_model,
|
||||
@@ -21,7 +20,6 @@ from controllers.common.schema import (
|
||||
register_response_schema_models,
|
||||
register_schema_models,
|
||||
)
|
||||
from controllers.common.session import with_session
|
||||
from controllers.console import console_ns
|
||||
from controllers.console.agent.app_helpers import resolve_agent_runtime_app_model
|
||||
from controllers.console.app.wraps import get_app_model
|
||||
@@ -35,6 +33,7 @@ from controllers.console.wraps import (
|
||||
with_current_tenant_id,
|
||||
with_current_user,
|
||||
)
|
||||
from extensions.ext_database import db
|
||||
from fields.base import ResponseModel
|
||||
from libs.login import login_required
|
||||
from models.account import Account
|
||||
@@ -100,7 +99,6 @@ class AgentConfigSkillItemResponse(ResponseModel):
|
||||
id: str
|
||||
name: str
|
||||
file_id: str | None = None
|
||||
is_missing: bool = False
|
||||
description: str = ""
|
||||
size: int | None = None
|
||||
mime_type: str | None = None
|
||||
@@ -111,7 +109,6 @@ class AgentConfigFileItemResponse(ResponseModel):
|
||||
id: str
|
||||
name: str
|
||||
file_id: str | None = None
|
||||
is_missing: bool = False
|
||||
size: int | None = None
|
||||
mime_type: str | None = None
|
||||
hash: str | None = None
|
||||
@@ -250,15 +247,15 @@ def _service() -> AgentConfigService:
|
||||
return AgentConfigService()
|
||||
|
||||
|
||||
def _resolve_agent_id(session: Session, app_model: App, node_id: str | None) -> str | None:
|
||||
def _resolve_agent_id(app_model: App, node_id: str | None) -> str | None:
|
||||
if node_id:
|
||||
return AgentComposerService.resolve_workflow_node_agent_id(
|
||||
session=session,
|
||||
tenant_id=app_model.tenant_id,
|
||||
app_id=app_model.id,
|
||||
node_id=node_id,
|
||||
session=db.session(),
|
||||
)
|
||||
return app_model.bound_agent_id_with_session(session=session)
|
||||
return app_model.bound_agent_id
|
||||
|
||||
|
||||
def _agent_not_bound() -> tuple[dict[str, object], int]:
|
||||
@@ -278,7 +275,6 @@ def _json_response(data: Mapping[str, Any]) -> Response:
|
||||
|
||||
def _resolve_console_version(
|
||||
*,
|
||||
session: Session,
|
||||
tenant_id: str,
|
||||
agent_id: str,
|
||||
account_id: str,
|
||||
@@ -290,24 +286,26 @@ def _resolve_console_version(
|
||||
try:
|
||||
if draft_type == "debug_build":
|
||||
state = AgentComposerService.load_agent_app_build_draft(
|
||||
session=session,
|
||||
tenant_id=tenant_id,
|
||||
agent_id=agent_id,
|
||||
account_id=account_id,
|
||||
session=db.session(),
|
||||
)
|
||||
draft = state.get("draft") or {}
|
||||
draft_id = draft.get("id")
|
||||
if isinstance(draft_id, str) and draft_id:
|
||||
return draft_id, AgentConfigVersionKind.BUILD_DRAFT
|
||||
else:
|
||||
state = AgentComposerService.load_agent_composer(session=session, tenant_id=tenant_id, agent_id=agent_id)
|
||||
state = AgentComposerService.load_agent_composer(
|
||||
tenant_id=tenant_id, agent_id=agent_id, session=db.session()
|
||||
)
|
||||
draft = state.get("draft") or {}
|
||||
draft_id = draft.get("id")
|
||||
if isinstance(draft_id, str) and draft_id:
|
||||
# load_agent_composer creates the normal draft on first access.
|
||||
# Config asset services use their own SQLAlchemy session, so the
|
||||
# draft must be visible before we hand its id across that boundary.
|
||||
session.commit()
|
||||
db.session.commit()
|
||||
return draft_id, AgentConfigVersionKind.DRAFT
|
||||
except AgentVersionNotFoundError as exc:
|
||||
raise AgentConfigServiceError(
|
||||
@@ -324,7 +322,6 @@ def _resolve_console_version(
|
||||
|
||||
def _resolve_target(
|
||||
*,
|
||||
session: Session,
|
||||
tenant_id: str,
|
||||
agent_id: str,
|
||||
account_id: str,
|
||||
@@ -332,7 +329,6 @@ def _resolve_target(
|
||||
draft_type: str | None,
|
||||
) -> _ResolvedConsoleTarget:
|
||||
resolved_version_id, version_kind = _resolve_console_version(
|
||||
session=session,
|
||||
tenant_id=tenant_id,
|
||||
agent_id=agent_id,
|
||||
account_id=account_id,
|
||||
@@ -350,15 +346,13 @@ def _resolve_target(
|
||||
|
||||
def _resolve_agent_route_target(
|
||||
*,
|
||||
session: Session,
|
||||
tenant_id: str,
|
||||
agent_id: UUID,
|
||||
current_user: Account,
|
||||
query: AgentConfigByAgentQuery,
|
||||
) -> _ResolvedConsoleTarget:
|
||||
resolve_agent_runtime_app_model(session=session, tenant_id=tenant_id, agent_id=agent_id)
|
||||
resolve_agent_runtime_app_model(tenant_id=tenant_id, agent_id=agent_id)
|
||||
return _resolve_target(
|
||||
session=session,
|
||||
tenant_id=tenant_id,
|
||||
agent_id=str(agent_id),
|
||||
account_id=current_user.id,
|
||||
@@ -369,16 +363,14 @@ def _resolve_agent_route_target(
|
||||
|
||||
def _resolve_app_route_target(
|
||||
*,
|
||||
session: Session,
|
||||
app_model: App,
|
||||
current_user: Account,
|
||||
query: AgentConfigQuery,
|
||||
) -> _ResolvedConsoleTarget | tuple[dict[str, object], int]:
|
||||
agent_id = _resolve_agent_id(session, app_model, query.node_id)
|
||||
agent_id = _resolve_agent_id(app_model, query.node_id)
|
||||
if not agent_id:
|
||||
return _agent_not_bound()
|
||||
return _resolve_target(
|
||||
session=session,
|
||||
tenant_id=app_model.tenant_id,
|
||||
agent_id=agent_id,
|
||||
account_id=current_user.id,
|
||||
@@ -389,7 +381,6 @@ def _resolve_app_route_target(
|
||||
|
||||
def _with_agent_route_target(
|
||||
*,
|
||||
session: Session,
|
||||
tenant_id: str,
|
||||
agent_id: UUID,
|
||||
current_user: Account,
|
||||
@@ -398,7 +389,6 @@ def _with_agent_route_target(
|
||||
query = query_params_from_request(AgentConfigByAgentQuery)
|
||||
try:
|
||||
target = _resolve_agent_route_target(
|
||||
session=session,
|
||||
tenant_id=tenant_id,
|
||||
agent_id=agent_id,
|
||||
current_user=current_user,
|
||||
@@ -411,14 +401,13 @@ def _with_agent_route_target(
|
||||
|
||||
def _with_app_route_target(
|
||||
*,
|
||||
session: Session,
|
||||
app_model: App,
|
||||
current_user: Account,
|
||||
action: Callable[[_ResolvedConsoleTarget], Any],
|
||||
) -> Any:
|
||||
query = query_params_from_request(AgentConfigQuery)
|
||||
try:
|
||||
target = _resolve_app_route_target(session=session, app_model=app_model, current_user=current_user, query=query)
|
||||
target = _resolve_app_route_target(app_model=app_model, current_user=current_user, query=query)
|
||||
if isinstance(target, tuple):
|
||||
return target
|
||||
return action(target)
|
||||
@@ -660,10 +649,8 @@ class AgentConfigManifestByAgentApi(Resource):
|
||||
@account_initialization_required
|
||||
@with_current_user
|
||||
@with_current_tenant_id
|
||||
@with_session
|
||||
def get(self, session: Session, tenant_id: str, current_user: Account, agent_id: UUID):
|
||||
def get(self, tenant_id: str, current_user: Account, agent_id: UUID):
|
||||
return _with_agent_route_target(
|
||||
session=session,
|
||||
tenant_id=tenant_id,
|
||||
agent_id=agent_id,
|
||||
current_user=current_user,
|
||||
@@ -679,13 +666,10 @@ class AgentConfigManifestApi(Resource):
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@with_current_user
|
||||
@with_session
|
||||
@get_app_model(mode=_WORKFLOW_APP_MODES)
|
||||
def get(self, session: Session, current_user: Account, app_model: App):
|
||||
return _with_app_route_target(
|
||||
session=session, app_model=app_model, current_user=current_user, action=_manifest_response
|
||||
)
|
||||
@with_current_user
|
||||
def get(self, current_user: Account, app_model: App):
|
||||
return _with_app_route_target(app_model=app_model, current_user=current_user, action=_manifest_response)
|
||||
|
||||
|
||||
@console_ns.route("/agent/<uuid:agent_id>/config/skills/upload")
|
||||
@@ -705,10 +689,8 @@ class AgentConfigSkillUploadByAgentApi(Resource):
|
||||
@account_initialization_required
|
||||
@with_current_user
|
||||
@with_current_tenant_id
|
||||
@with_session
|
||||
def post(self, session: Session, tenant_id: str, current_user: Account, agent_id: UUID):
|
||||
def post(self, tenant_id: str, current_user: Account, agent_id: UUID):
|
||||
return _with_agent_route_target(
|
||||
session=session,
|
||||
tenant_id=tenant_id,
|
||||
agent_id=agent_id,
|
||||
current_user=current_user,
|
||||
@@ -733,13 +715,10 @@ class AgentConfigSkillUploadApi(Resource):
|
||||
@account_initialization_required
|
||||
@edit_permission_required
|
||||
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_EDIT)
|
||||
@with_current_user
|
||||
@with_session
|
||||
@get_app_model(mode=_WORKFLOW_APP_MODES)
|
||||
def post(self, session: Session, current_user: Account, app_model: App):
|
||||
return _with_app_route_target(
|
||||
session=session, app_model=app_model, current_user=current_user, action=_skill_upload_response
|
||||
)
|
||||
@with_current_user
|
||||
def post(self, current_user: Account, app_model: App):
|
||||
return _with_app_route_target(app_model=app_model, current_user=current_user, action=_skill_upload_response)
|
||||
|
||||
|
||||
@console_ns.route("/agent/<uuid:agent_id>/config/skills")
|
||||
@@ -752,10 +731,8 @@ class AgentConfigSkillsByAgentApi(Resource):
|
||||
@account_initialization_required
|
||||
@with_current_user
|
||||
@with_current_tenant_id
|
||||
@with_session
|
||||
def get(self, session: Session, tenant_id: str, current_user: Account, agent_id: UUID):
|
||||
def get(self, tenant_id: str, current_user: Account, agent_id: UUID):
|
||||
return _with_agent_route_target(
|
||||
session=session,
|
||||
tenant_id=tenant_id,
|
||||
agent_id=agent_id,
|
||||
current_user=current_user,
|
||||
@@ -771,13 +748,10 @@ class AgentConfigSkillsApi(Resource):
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@with_current_user
|
||||
@with_session
|
||||
@get_app_model(mode=_WORKFLOW_APP_MODES)
|
||||
def get(self, session: Session, current_user: Account, app_model: App):
|
||||
return _with_app_route_target(
|
||||
session=session, app_model=app_model, current_user=current_user, action=_skill_list_response
|
||||
)
|
||||
@with_current_user
|
||||
def get(self, current_user: Account, app_model: App):
|
||||
return _with_app_route_target(app_model=app_model, current_user=current_user, action=_skill_list_response)
|
||||
|
||||
|
||||
@console_ns.route("/agent/<uuid:agent_id>/config/files")
|
||||
@@ -790,10 +764,8 @@ class AgentConfigFilesByAgentApi(Resource):
|
||||
@account_initialization_required
|
||||
@with_current_user
|
||||
@with_current_tenant_id
|
||||
@with_session
|
||||
def get(self, session: Session, tenant_id: str, current_user: Account, agent_id: UUID):
|
||||
def get(self, tenant_id: str, current_user: Account, agent_id: UUID):
|
||||
return _with_agent_route_target(
|
||||
session=session,
|
||||
tenant_id=tenant_id,
|
||||
agent_id=agent_id,
|
||||
current_user=current_user,
|
||||
@@ -809,11 +781,9 @@ class AgentConfigFilesByAgentApi(Resource):
|
||||
@account_initialization_required
|
||||
@with_current_user
|
||||
@with_current_tenant_id
|
||||
@with_session
|
||||
def post(self, session: Session, tenant_id: str, current_user: Account, agent_id: UUID):
|
||||
def post(self, tenant_id: str, current_user: Account, agent_id: UUID):
|
||||
payload = AgentConfigFileUploadPayload.model_validate(console_ns.payload or {})
|
||||
return _with_agent_route_target(
|
||||
session=session,
|
||||
tenant_id=tenant_id,
|
||||
agent_id=agent_id,
|
||||
current_user=current_user,
|
||||
@@ -829,13 +799,10 @@ class AgentConfigFilesApi(Resource):
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@with_current_user
|
||||
@with_session
|
||||
@get_app_model(mode=_WORKFLOW_APP_MODES)
|
||||
def get(self, session: Session, current_user: Account, app_model: App):
|
||||
return _with_app_route_target(
|
||||
session=session, app_model=app_model, current_user=current_user, action=_file_list_response
|
||||
)
|
||||
@with_current_user
|
||||
def get(self, current_user: Account, app_model: App):
|
||||
return _with_app_route_target(app_model=app_model, current_user=current_user, action=_file_list_response)
|
||||
|
||||
@console_ns.doc("upload_agent_config_file")
|
||||
@console_ns.doc(params={"app_id": "Application ID", **query_params_from_model(AgentConfigQuery)})
|
||||
@@ -846,13 +813,11 @@ class AgentConfigFilesApi(Resource):
|
||||
@account_initialization_required
|
||||
@edit_permission_required
|
||||
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_EDIT)
|
||||
@with_current_user
|
||||
@with_session
|
||||
@get_app_model(mode=_WORKFLOW_APP_MODES)
|
||||
def post(self, session: Session, current_user: Account, app_model: App):
|
||||
@with_current_user
|
||||
def post(self, current_user: Account, app_model: App):
|
||||
payload = AgentConfigFileUploadPayload.model_validate(console_ns.payload or {})
|
||||
return _with_app_route_target(
|
||||
session=session,
|
||||
app_model=app_model,
|
||||
current_user=current_user,
|
||||
action=lambda target: _file_upload_response(target, payload),
|
||||
@@ -871,10 +836,8 @@ class AgentConfigSkillInspectByAgentApi(Resource):
|
||||
@account_initialization_required
|
||||
@with_current_user
|
||||
@with_current_tenant_id
|
||||
@with_session
|
||||
def get(self, session: Session, tenant_id: str, current_user: Account, agent_id: UUID, name: str):
|
||||
def get(self, tenant_id: str, current_user: Account, agent_id: UUID, name: str):
|
||||
return _with_agent_route_target(
|
||||
session=session,
|
||||
tenant_id=tenant_id,
|
||||
agent_id=agent_id,
|
||||
current_user=current_user,
|
||||
@@ -892,12 +855,10 @@ class AgentConfigSkillInspectApi(Resource):
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@with_current_user
|
||||
@with_session
|
||||
@get_app_model(mode=_WORKFLOW_APP_MODES)
|
||||
def get(self, session: Session, current_user: Account, app_model: App, name: str):
|
||||
@with_current_user
|
||||
def get(self, current_user: Account, app_model: App, name: str):
|
||||
return _with_app_route_target(
|
||||
session=session,
|
||||
app_model=app_model,
|
||||
current_user=current_user,
|
||||
action=lambda target: _skill_inspect_response(target, name),
|
||||
@@ -922,12 +883,10 @@ class AgentConfigSkillFilePreviewByAgentApi(Resource):
|
||||
@account_initialization_required
|
||||
@with_current_user
|
||||
@with_current_tenant_id
|
||||
@with_session
|
||||
def get(self, session: Session, tenant_id: str, current_user: Account, agent_id: UUID, name: str):
|
||||
def get(self, tenant_id: str, current_user: Account, agent_id: UUID, name: str):
|
||||
query = query_params_from_request(AgentConfigSkillFileByAgentQuery)
|
||||
try:
|
||||
target = _resolve_agent_route_target(
|
||||
session=session,
|
||||
tenant_id=tenant_id,
|
||||
agent_id=agent_id,
|
||||
current_user=current_user,
|
||||
@@ -954,15 +913,12 @@ class AgentConfigSkillFilePreviewApi(Resource):
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@with_current_user
|
||||
@with_session
|
||||
@get_app_model(mode=_WORKFLOW_APP_MODES)
|
||||
def get(self, session: Session, current_user: Account, app_model: App, name: str):
|
||||
@with_current_user
|
||||
def get(self, current_user: Account, app_model: App, name: str):
|
||||
query = query_params_from_request(AgentConfigSkillFileQuery)
|
||||
try:
|
||||
target = _resolve_app_route_target(
|
||||
session=session, app_model=app_model, current_user=current_user, query=query
|
||||
)
|
||||
target = _resolve_app_route_target(app_model=app_model, current_user=current_user, query=query)
|
||||
if isinstance(target, tuple):
|
||||
return target
|
||||
return _skill_file_preview_response(target, name, query.path)
|
||||
@@ -982,10 +938,8 @@ class AgentConfigSkillDownloadByAgentApi(Resource):
|
||||
@account_initialization_required
|
||||
@with_current_user
|
||||
@with_current_tenant_id
|
||||
@with_session
|
||||
def get(self, session: Session, tenant_id: str, current_user: Account, agent_id: UUID, name: str):
|
||||
def get(self, tenant_id: str, current_user: Account, agent_id: UUID, name: str):
|
||||
return _with_agent_route_target(
|
||||
session=session,
|
||||
tenant_id=tenant_id,
|
||||
agent_id=agent_id,
|
||||
current_user=current_user,
|
||||
@@ -1003,12 +957,10 @@ class AgentConfigSkillDownloadApi(Resource):
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@with_current_user
|
||||
@with_session
|
||||
@get_app_model(mode=_WORKFLOW_APP_MODES)
|
||||
def get(self, session: Session, current_user: Account, app_model: App, name: str):
|
||||
@with_current_user
|
||||
def get(self, current_user: Account, app_model: App, name: str):
|
||||
return _with_app_route_target(
|
||||
session=session,
|
||||
app_model=app_model,
|
||||
current_user=current_user,
|
||||
action=lambda target: _skill_download_response(target, name),
|
||||
@@ -1031,12 +983,10 @@ class AgentConfigSkillFileDownloadByAgentApi(Resource):
|
||||
@account_initialization_required
|
||||
@with_current_user
|
||||
@with_current_tenant_id
|
||||
@with_session
|
||||
def get(self, session: Session, tenant_id: str, current_user: Account, agent_id: UUID, name: str):
|
||||
def get(self, tenant_id: str, current_user: Account, agent_id: UUID, name: str):
|
||||
query = query_params_from_request(AgentConfigSkillFileByAgentQuery)
|
||||
try:
|
||||
target = _resolve_agent_route_target(
|
||||
session=session,
|
||||
tenant_id=tenant_id,
|
||||
agent_id=agent_id,
|
||||
current_user=current_user,
|
||||
@@ -1067,15 +1017,12 @@ class AgentConfigSkillFileDownloadApi(Resource):
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@with_current_user
|
||||
@with_session
|
||||
@get_app_model(mode=_WORKFLOW_APP_MODES)
|
||||
def get(self, session: Session, current_user: Account, app_model: App, name: str):
|
||||
@with_current_user
|
||||
def get(self, current_user: Account, app_model: App, name: str):
|
||||
query = query_params_from_request(AgentConfigSkillFileQuery)
|
||||
try:
|
||||
target = _resolve_app_route_target(
|
||||
session=session, app_model=app_model, current_user=current_user, query=query
|
||||
)
|
||||
target = _resolve_app_route_target(app_model=app_model, current_user=current_user, query=query)
|
||||
if isinstance(target, tuple):
|
||||
return target
|
||||
return _skill_file_download_response(
|
||||
@@ -1100,12 +1047,10 @@ class AgentConfigSkillFileDownloadContentByAgentApi(Resource):
|
||||
@account_initialization_required
|
||||
@with_current_user
|
||||
@with_current_tenant_id
|
||||
@with_session
|
||||
def get(self, session: Session, tenant_id: str, current_user: Account, agent_id: UUID, name: str):
|
||||
def get(self, tenant_id: str, current_user: Account, agent_id: UUID, name: str):
|
||||
query = query_params_from_request(AgentConfigSkillFileByAgentQuery)
|
||||
try:
|
||||
target = _resolve_agent_route_target(
|
||||
session=session,
|
||||
tenant_id=tenant_id,
|
||||
agent_id=agent_id,
|
||||
current_user=current_user,
|
||||
@@ -1124,15 +1069,12 @@ class AgentConfigSkillFileDownloadContentApi(Resource):
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@with_current_user
|
||||
@with_session
|
||||
@get_app_model(mode=_WORKFLOW_APP_MODES)
|
||||
def get(self, session: Session, current_user: Account, app_model: App, name: str):
|
||||
@with_current_user
|
||||
def get(self, current_user: Account, app_model: App, name: str):
|
||||
query = query_params_from_request(AgentConfigSkillFileQuery)
|
||||
try:
|
||||
target = _resolve_app_route_target(
|
||||
session=session, app_model=app_model, current_user=current_user, query=query
|
||||
)
|
||||
target = _resolve_app_route_target(app_model=app_model, current_user=current_user, query=query)
|
||||
if isinstance(target, tuple):
|
||||
return target
|
||||
return _skill_file_raw_download_response(target, name, query.path)
|
||||
@@ -1152,10 +1094,8 @@ class AgentConfigSkillByAgentApi(Resource):
|
||||
@account_initialization_required
|
||||
@with_current_user
|
||||
@with_current_tenant_id
|
||||
@with_session
|
||||
def delete(self, session: Session, tenant_id: str, current_user: Account, agent_id: UUID, name: str):
|
||||
def delete(self, tenant_id: str, current_user: Account, agent_id: UUID, name: str):
|
||||
return _with_agent_route_target(
|
||||
session=session,
|
||||
tenant_id=tenant_id,
|
||||
agent_id=agent_id,
|
||||
current_user=current_user,
|
||||
@@ -1175,12 +1115,10 @@ class AgentConfigSkillApi(Resource):
|
||||
@account_initialization_required
|
||||
@edit_permission_required
|
||||
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_EDIT)
|
||||
@with_current_user
|
||||
@with_session
|
||||
@get_app_model(mode=_WORKFLOW_APP_MODES)
|
||||
def delete(self, session: Session, current_user: Account, app_model: App, name: str):
|
||||
@with_current_user
|
||||
def delete(self, current_user: Account, app_model: App, name: str):
|
||||
return _with_app_route_target(
|
||||
session=session,
|
||||
app_model=app_model,
|
||||
current_user=current_user,
|
||||
action=lambda target: _skill_delete_response(target, name),
|
||||
@@ -1199,10 +1137,8 @@ class AgentConfigFilePreviewByAgentApi(Resource):
|
||||
@account_initialization_required
|
||||
@with_current_user
|
||||
@with_current_tenant_id
|
||||
@with_session
|
||||
def get(self, session: Session, tenant_id: str, current_user: Account, agent_id: UUID, name: str):
|
||||
def get(self, tenant_id: str, current_user: Account, agent_id: UUID, name: str):
|
||||
return _with_agent_route_target(
|
||||
session=session,
|
||||
tenant_id=tenant_id,
|
||||
agent_id=agent_id,
|
||||
current_user=current_user,
|
||||
@@ -1220,12 +1156,10 @@ class AgentConfigFilePreviewApi(Resource):
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@with_current_user
|
||||
@with_session
|
||||
@get_app_model(mode=_WORKFLOW_APP_MODES)
|
||||
def get(self, session: Session, current_user: Account, app_model: App, name: str):
|
||||
@with_current_user
|
||||
def get(self, current_user: Account, app_model: App, name: str):
|
||||
return _with_app_route_target(
|
||||
session=session,
|
||||
app_model=app_model,
|
||||
current_user=current_user,
|
||||
action=lambda target: _file_preview_response(target, name),
|
||||
@@ -1244,10 +1178,8 @@ class AgentConfigFileDownloadByAgentApi(Resource):
|
||||
@account_initialization_required
|
||||
@with_current_user
|
||||
@with_current_tenant_id
|
||||
@with_session
|
||||
def get(self, session: Session, tenant_id: str, current_user: Account, agent_id: UUID, name: str):
|
||||
def get(self, tenant_id: str, current_user: Account, agent_id: UUID, name: str):
|
||||
return _with_agent_route_target(
|
||||
session=session,
|
||||
tenant_id=tenant_id,
|
||||
agent_id=agent_id,
|
||||
current_user=current_user,
|
||||
@@ -1265,12 +1197,10 @@ class AgentConfigFileDownloadApi(Resource):
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@with_current_user
|
||||
@with_session
|
||||
@get_app_model(mode=_WORKFLOW_APP_MODES)
|
||||
def get(self, session: Session, current_user: Account, app_model: App, name: str):
|
||||
@with_current_user
|
||||
def get(self, current_user: Account, app_model: App, name: str):
|
||||
return _with_app_route_target(
|
||||
session=session,
|
||||
app_model=app_model,
|
||||
current_user=current_user,
|
||||
action=lambda target: _file_download_response(target, name),
|
||||
@@ -1289,10 +1219,8 @@ class AgentConfigFileByAgentApi(Resource):
|
||||
@account_initialization_required
|
||||
@with_current_user
|
||||
@with_current_tenant_id
|
||||
@with_session
|
||||
def delete(self, session: Session, tenant_id: str, current_user: Account, agent_id: UUID, name: str):
|
||||
def delete(self, tenant_id: str, current_user: Account, agent_id: UUID, name: str):
|
||||
return _with_agent_route_target(
|
||||
session=session,
|
||||
tenant_id=tenant_id,
|
||||
agent_id=agent_id,
|
||||
current_user=current_user,
|
||||
@@ -1312,12 +1240,10 @@ class AgentConfigFileApi(Resource):
|
||||
@account_initialization_required
|
||||
@edit_permission_required
|
||||
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_EDIT)
|
||||
@with_current_user
|
||||
@with_session
|
||||
@get_app_model(mode=_WORKFLOW_APP_MODES)
|
||||
def delete(self, session: Session, current_user: Account, app_model: App, name: str):
|
||||
@with_current_user
|
||||
def delete(self, current_user: Account, app_model: App, name: str):
|
||||
return _with_app_route_target(
|
||||
session=session,
|
||||
app_model=app_model,
|
||||
current_user=current_user,
|
||||
action=lambda target: _file_delete_response(target, name),
|
||||
|
||||
@@ -18,18 +18,17 @@ from uuid import UUID
|
||||
from flask import Response
|
||||
from flask_restx import Resource
|
||||
from pydantic import BaseModel, Field
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from controllers.common.schema import (
|
||||
query_params_from_model,
|
||||
query_params_from_request,
|
||||
register_response_schema_models,
|
||||
)
|
||||
from controllers.common.session import with_session
|
||||
from controllers.console import console_ns
|
||||
from controllers.console.agent.app_helpers import resolve_agent_runtime_app_model
|
||||
from controllers.console.app.wraps import get_app_model
|
||||
from controllers.console.wraps import account_initialization_required, setup_required, with_current_tenant_id
|
||||
from extensions.ext_database import db
|
||||
from fields.base import ResponseModel
|
||||
from libs.login import login_required
|
||||
from models.model import App, AppMode
|
||||
@@ -145,13 +144,13 @@ register_response_schema_models(
|
||||
)
|
||||
|
||||
|
||||
def _resolve_agent_id(session: Session, app_model: App, node_id: str | None) -> str | None:
|
||||
def _resolve_agent_id(app_model: App, node_id: str | None) -> str | None:
|
||||
"""Agent identity for the drive: app-bound agent, or the workflow node binding."""
|
||||
if node_id:
|
||||
return AgentComposerService.resolve_workflow_node_agent_id(
|
||||
session=session, tenant_id=app_model.tenant_id, app_id=app_model.id, node_id=node_id
|
||||
tenant_id=app_model.tenant_id, app_id=app_model.id, node_id=node_id, session=db.session()
|
||||
)
|
||||
return app_model.bound_agent_id_with_session(session=session)
|
||||
return app_model.bound_agent_id
|
||||
|
||||
|
||||
def _agent_not_bound() -> tuple[dict[str, object], int]:
|
||||
@@ -182,13 +181,12 @@ class AgentDriveListByAgentApi(Resource):
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@with_current_tenant_id
|
||||
@with_session(write=False)
|
||||
def get(self, session: Session, tenant_id: str, agent_id: UUID):
|
||||
def get(self, tenant_id: str, agent_id: UUID):
|
||||
query = query_params_from_request(AgentDriveListByAgentQuery)
|
||||
resolve_agent_runtime_app_model(session=session, tenant_id=tenant_id, agent_id=agent_id)
|
||||
resolve_agent_runtime_app_model(tenant_id=tenant_id, agent_id=agent_id)
|
||||
try:
|
||||
items = AgentDriveService().manifest(
|
||||
tenant_id=tenant_id, agent_id=str(agent_id), prefix=query.prefix, session=session
|
||||
tenant_id=tenant_id, agent_id=str(agent_id), prefix=query.prefix, session=db.session()
|
||||
)
|
||||
except AgentDriveError as exc:
|
||||
return _handle(exc)
|
||||
@@ -205,11 +203,10 @@ class AgentDriveSkillListByAgentApi(Resource):
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@with_current_tenant_id
|
||||
@with_session(write=False)
|
||||
def get(self, session: Session, tenant_id: str, agent_id: UUID):
|
||||
resolve_agent_runtime_app_model(session=session, tenant_id=tenant_id, agent_id=agent_id)
|
||||
def get(self, tenant_id: str, agent_id: UUID):
|
||||
resolve_agent_runtime_app_model(tenant_id=tenant_id, agent_id=agent_id)
|
||||
try:
|
||||
items = AgentDriveService().list_skills(tenant_id=tenant_id, agent_id=str(agent_id), session=session)
|
||||
items = AgentDriveService().list_skills(tenant_id=tenant_id, agent_id=str(agent_id), session=db.session())
|
||||
except AgentDriveError as exc:
|
||||
return _handle(exc)
|
||||
return {"items": items}
|
||||
@@ -225,16 +222,15 @@ class AgentDriveSkillInspectByAgentApi(Resource):
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@with_current_tenant_id
|
||||
@with_session(write=False)
|
||||
def get(self, session: Session, tenant_id: str, agent_id: UUID, skill_path: str):
|
||||
resolve_agent_runtime_app_model(session=session, tenant_id=tenant_id, agent_id=agent_id)
|
||||
def get(self, tenant_id: str, agent_id: UUID, skill_path: str):
|
||||
resolve_agent_runtime_app_model(tenant_id=tenant_id, agent_id=agent_id)
|
||||
try:
|
||||
return _json_response(
|
||||
AgentDriveService().inspect_skill(
|
||||
tenant_id=tenant_id,
|
||||
agent_id=str(agent_id),
|
||||
skill_path=skill_path,
|
||||
session=session,
|
||||
session=db.session(),
|
||||
)
|
||||
)
|
||||
except AgentDriveError as exc:
|
||||
@@ -251,13 +247,12 @@ class AgentDrivePreviewByAgentApi(Resource):
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@with_current_tenant_id
|
||||
@with_session(write=False)
|
||||
def get(self, session: Session, tenant_id: str, agent_id: UUID):
|
||||
def get(self, tenant_id: str, agent_id: UUID):
|
||||
query = query_params_from_request(AgentDriveFileByAgentQuery)
|
||||
resolve_agent_runtime_app_model(session=session, tenant_id=tenant_id, agent_id=agent_id)
|
||||
resolve_agent_runtime_app_model(tenant_id=tenant_id, agent_id=agent_id)
|
||||
try:
|
||||
return AgentDriveService().preview(
|
||||
tenant_id=tenant_id, agent_id=str(agent_id), key=query.key, session=session
|
||||
tenant_id=tenant_id, agent_id=str(agent_id), key=query.key, session=db.session()
|
||||
)
|
||||
except AgentDriveError as exc:
|
||||
return _handle(exc)
|
||||
@@ -273,13 +268,12 @@ class AgentDriveDownloadByAgentApi(Resource):
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@with_current_tenant_id
|
||||
@with_session(write=False)
|
||||
def get(self, session: Session, tenant_id: str, agent_id: UUID):
|
||||
def get(self, tenant_id: str, agent_id: UUID):
|
||||
query = query_params_from_request(AgentDriveFileByAgentQuery)
|
||||
resolve_agent_runtime_app_model(session=session, tenant_id=tenant_id, agent_id=agent_id)
|
||||
resolve_agent_runtime_app_model(tenant_id=tenant_id, agent_id=agent_id)
|
||||
try:
|
||||
url = AgentDriveService().download_url(
|
||||
tenant_id=tenant_id, agent_id=str(agent_id), key=query.key, session=session
|
||||
tenant_id=tenant_id, agent_id=str(agent_id), key=query.key, session=db.session()
|
||||
)
|
||||
except AgentDriveError as exc:
|
||||
return _handle(exc)
|
||||
@@ -295,16 +289,15 @@ class AgentDriveListApi(Resource):
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@with_session(write=False)
|
||||
@get_app_model(mode=_WORKFLOW_APP_MODES)
|
||||
def get(self, session: Session, app_model: App):
|
||||
def get(self, app_model: App):
|
||||
query = query_params_from_request(AgentDriveListQuery)
|
||||
agent_id = _resolve_agent_id(session, app_model, query.node_id)
|
||||
agent_id = _resolve_agent_id(app_model, query.node_id)
|
||||
if not agent_id:
|
||||
return _agent_not_bound()
|
||||
try:
|
||||
items = AgentDriveService().manifest(
|
||||
tenant_id=app_model.tenant_id, agent_id=agent_id, prefix=query.prefix, session=session
|
||||
tenant_id=app_model.tenant_id, agent_id=agent_id, prefix=query.prefix, session=db.session()
|
||||
)
|
||||
except AgentDriveError as exc:
|
||||
return _handle(exc)
|
||||
@@ -322,15 +315,16 @@ class AgentDriveSkillListApi(Resource):
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@with_session(write=False)
|
||||
@get_app_model(mode=_WORKFLOW_APP_MODES)
|
||||
def get(self, session: Session, app_model: App):
|
||||
def get(self, app_model: App):
|
||||
query = query_params_from_request(AgentDriveListQuery)
|
||||
agent_id = _resolve_agent_id(session, app_model, query.node_id)
|
||||
agent_id = _resolve_agent_id(app_model, query.node_id)
|
||||
if not agent_id:
|
||||
return _agent_not_bound()
|
||||
try:
|
||||
items = AgentDriveService().list_skills(tenant_id=app_model.tenant_id, agent_id=agent_id, session=session)
|
||||
items = AgentDriveService().list_skills(
|
||||
tenant_id=app_model.tenant_id, agent_id=agent_id, session=db.session()
|
||||
)
|
||||
except AgentDriveError as exc:
|
||||
return _handle(exc)
|
||||
return {"items": items}
|
||||
@@ -351,11 +345,10 @@ class AgentDriveSkillInspectApi(Resource):
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@with_session(write=False)
|
||||
@get_app_model(mode=_WORKFLOW_APP_MODES)
|
||||
def get(self, session: Session, app_model: App, skill_path: str):
|
||||
def get(self, app_model: App, skill_path: str):
|
||||
query = query_params_from_request(AgentDriveSkillInspectQuery)
|
||||
agent_id = _resolve_agent_id(session, app_model, query.node_id)
|
||||
agent_id = _resolve_agent_id(app_model, query.node_id)
|
||||
if not agent_id:
|
||||
return _agent_not_bound()
|
||||
try:
|
||||
@@ -364,7 +357,7 @@ class AgentDriveSkillInspectApi(Resource):
|
||||
tenant_id=app_model.tenant_id,
|
||||
agent_id=agent_id,
|
||||
skill_path=skill_path,
|
||||
session=session,
|
||||
session=db.session(),
|
||||
)
|
||||
)
|
||||
except AgentDriveError as exc:
|
||||
@@ -380,16 +373,15 @@ class AgentDrivePreviewApi(Resource):
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@with_session(write=False)
|
||||
@get_app_model(mode=_WORKFLOW_APP_MODES)
|
||||
def get(self, session: Session, app_model: App):
|
||||
def get(self, app_model: App):
|
||||
query = query_params_from_request(AgentDriveFileQuery)
|
||||
agent_id = _resolve_agent_id(session, app_model, query.node_id)
|
||||
agent_id = _resolve_agent_id(app_model, query.node_id)
|
||||
if not agent_id:
|
||||
return _agent_not_bound()
|
||||
try:
|
||||
return AgentDriveService().preview(
|
||||
tenant_id=app_model.tenant_id, agent_id=agent_id, key=query.key, session=session
|
||||
tenant_id=app_model.tenant_id, agent_id=agent_id, key=query.key, session=db.session()
|
||||
)
|
||||
except AgentDriveError as exc:
|
||||
return _handle(exc)
|
||||
@@ -404,16 +396,15 @@ class AgentDriveDownloadApi(Resource):
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@with_session(write=False)
|
||||
@get_app_model(mode=_WORKFLOW_APP_MODES)
|
||||
def get(self, session: Session, app_model: App):
|
||||
def get(self, app_model: App):
|
||||
query = query_params_from_request(AgentDriveFileQuery)
|
||||
agent_id = _resolve_agent_id(session, app_model, query.node_id)
|
||||
agent_id = _resolve_agent_id(app_model, query.node_id)
|
||||
if not agent_id:
|
||||
return _agent_not_bound()
|
||||
try:
|
||||
url = AgentDriveService().download_url(
|
||||
tenant_id=app_model.tenant_id, agent_id=agent_id, key=query.key, session=session
|
||||
tenant_id=app_model.tenant_id, agent_id=agent_id, key=query.key, session=db.session()
|
||||
)
|
||||
except AgentDriveError as exc:
|
||||
return _handle(exc)
|
||||
|
||||
@@ -5,12 +5,10 @@ from flask import abort, request
|
||||
from flask_restx import Resource
|
||||
from pydantic import BaseModel, Field, TypeAdapter, field_validator
|
||||
from sqlalchemy import select
|
||||
from sqlalchemy.orm import Session
|
||||
from werkzeug.exceptions import NotFound
|
||||
|
||||
from controllers.common.errors import NoFileUploadedError, TooManyFilesError
|
||||
from controllers.common.schema import query_params_from_model, register_response_schema_models, register_schema_models
|
||||
from controllers.common.session import with_session
|
||||
from controllers.console import console_ns
|
||||
from controllers.console.wraps import (
|
||||
RBACPermission,
|
||||
@@ -23,6 +21,7 @@ from controllers.console.wraps import (
|
||||
rbac_permission_required,
|
||||
setup_required,
|
||||
)
|
||||
from extensions.ext_database import db
|
||||
from extensions.ext_redis import redis_client
|
||||
from fields.annotation_fields import (
|
||||
Annotation,
|
||||
@@ -47,9 +46,9 @@ from services.annotation_service import (
|
||||
from services.app_ref_service import AppRef, AppRefService
|
||||
|
||||
|
||||
def _get_app_ref(session: Session, app_id: str) -> AppRef:
|
||||
def _get_app_ref(app_id: str) -> AppRef:
|
||||
_, current_tenant_id = current_account_with_tenant()
|
||||
app = session.scalar(
|
||||
app = db.session.scalar(
|
||||
select(App).where(App.id == app_id, App.tenant_id == current_tenant_id, App.status == "normal").limit(1)
|
||||
)
|
||||
if app is None:
|
||||
@@ -211,9 +210,8 @@ class AppAnnotationSettingDetailApi(Resource):
|
||||
@account_initialization_required
|
||||
@edit_permission_required
|
||||
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_VIEW_LAYOUT)
|
||||
@with_session(write=False)
|
||||
def get(self, session: Session, app_id: UUID):
|
||||
result = AppAnnotationService.get_app_annotation_setting_by_app_id(str(app_id), session)
|
||||
def get(self, app_id: UUID):
|
||||
result = AppAnnotationService.get_app_annotation_setting_by_app_id(str(app_id), session=db.session())
|
||||
return dump_response(AnnotationSettingResponse, result), 200
|
||||
|
||||
|
||||
@@ -230,15 +228,14 @@ class AppAnnotationSettingUpdateApi(Resource):
|
||||
@account_initialization_required
|
||||
@edit_permission_required
|
||||
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_EDIT)
|
||||
@with_session
|
||||
def post(self, session: Session, app_id: UUID, annotation_setting_id: UUID):
|
||||
def post(self, app_id: UUID, annotation_setting_id: UUID):
|
||||
annotation_setting_id_str = str(annotation_setting_id)
|
||||
|
||||
args = AnnotationSettingUpdatePayload.model_validate(console_ns.payload)
|
||||
|
||||
setting_args: UpdateAnnotationSettingArgs = {"score_threshold": args.score_threshold}
|
||||
result = AppAnnotationService.update_app_annotation_setting(
|
||||
str(app_id), annotation_setting_id_str, setting_args, session
|
||||
str(app_id), annotation_setting_id_str, setting_args, session=db.session()
|
||||
)
|
||||
return dump_response(AnnotationSettingResponse, result), 200
|
||||
|
||||
@@ -289,15 +286,14 @@ class AnnotationApi(Resource):
|
||||
@account_initialization_required
|
||||
@edit_permission_required
|
||||
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_VIEW_LAYOUT)
|
||||
@with_session(write=False)
|
||||
def get(self, session: Session, app_id: UUID):
|
||||
def get(self, app_id: UUID):
|
||||
args = AnnotationListQuery.model_validate(request.args.to_dict(flat=True))
|
||||
page = args.page
|
||||
limit = args.limit
|
||||
keyword = args.keyword
|
||||
|
||||
annotation_list, total = AppAnnotationService.get_annotation_list_by_app_id(
|
||||
str(app_id), page, limit, keyword, session
|
||||
str(app_id), page, limit, keyword, session=db.session()
|
||||
)
|
||||
annotation_models = TypeAdapter(list[Annotation]).validate_python(annotation_list, from_attributes=True)
|
||||
return AnnotationList(
|
||||
@@ -316,8 +312,7 @@ class AnnotationApi(Resource):
|
||||
@cloud_edition_billing_resource_check("annotation")
|
||||
@edit_permission_required
|
||||
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_EDIT)
|
||||
@with_session
|
||||
def post(self, session: Session, app_id: UUID):
|
||||
def post(self, app_id: UUID):
|
||||
args = CreateAnnotationPayload.model_validate(console_ns.payload)
|
||||
upsert_args: UpsertAnnotationArgs = {}
|
||||
if args.answer is not None:
|
||||
@@ -328,7 +323,9 @@ class AnnotationApi(Resource):
|
||||
upsert_args["message_id"] = args.message_id
|
||||
if args.question is not None:
|
||||
upsert_args["question"] = args.question
|
||||
annotation = AppAnnotationService.up_insert_app_annotation_from_message(upsert_args, str(app_id), session)
|
||||
annotation = AppAnnotationService.up_insert_app_annotation_from_message(
|
||||
upsert_args, str(app_id), session=db.session()
|
||||
)
|
||||
return dump_response(Annotation, annotation), 201
|
||||
|
||||
@setup_required
|
||||
@@ -337,8 +334,7 @@ class AnnotationApi(Resource):
|
||||
@edit_permission_required
|
||||
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_CREATE_AND_MANAGEMENT)
|
||||
@console_ns.response(204, "Annotations deleted successfully")
|
||||
@with_session
|
||||
def delete(self, session: Session, app_id: UUID):
|
||||
def delete(self, app_id: UUID):
|
||||
|
||||
# Use request.args.getlist to get annotation_ids array directly
|
||||
annotation_ids = request.args.getlist("annotation_id")
|
||||
@@ -352,12 +348,12 @@ class AnnotationApi(Resource):
|
||||
"message": "annotation_ids are required if the parameter is provided.",
|
||||
}, 400
|
||||
|
||||
app_ref = _get_app_ref(session, str(app_id))
|
||||
AppAnnotationService.delete_app_annotations_in_batch(app_ref, annotation_ids, session)
|
||||
app_ref = _get_app_ref(str(app_id))
|
||||
AppAnnotationService.delete_app_annotations_in_batch(app_ref, annotation_ids, session=db.session())
|
||||
return "", 204
|
||||
# If no annotation_ids are provided, handle clearing all annotations
|
||||
else:
|
||||
AppAnnotationService.clear_all_annotations(str(app_id), session)
|
||||
AppAnnotationService.clear_all_annotations(str(app_id), session=db.session())
|
||||
return "", 204
|
||||
|
||||
|
||||
@@ -377,9 +373,8 @@ class AnnotationExportApi(Resource):
|
||||
@account_initialization_required
|
||||
@edit_permission_required
|
||||
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_VIEW_LAYOUT)
|
||||
@with_session(write=False)
|
||||
def get(self, session: Session, app_id: UUID):
|
||||
annotation_list = AppAnnotationService.export_annotation_list_by_app_id(str(app_id), session)
|
||||
def get(self, app_id: UUID):
|
||||
annotation_list = AppAnnotationService.export_annotation_list_by_app_id(str(app_id), session=db.session())
|
||||
annotation_models = TypeAdapter(list[Annotation]).validate_python(annotation_list, from_attributes=True)
|
||||
return (
|
||||
AnnotationExportList(data=annotation_models).model_dump(mode="json"),
|
||||
@@ -406,17 +401,16 @@ class AnnotationUpdateDeleteApi(Resource):
|
||||
@cloud_edition_billing_resource_check("annotation")
|
||||
@edit_permission_required
|
||||
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_EDIT)
|
||||
@with_session
|
||||
def post(self, session: Session, app_id: UUID, annotation_id: UUID):
|
||||
def post(self, app_id: UUID, annotation_id: UUID):
|
||||
args = UpdateAnnotationPayload.model_validate(console_ns.payload)
|
||||
update_args: UpdateAnnotationArgs = {}
|
||||
if args.answer is not None:
|
||||
update_args["answer"] = args.answer
|
||||
if args.question is not None:
|
||||
update_args["question"] = args.question
|
||||
app_ref = _get_app_ref(session, str(app_id))
|
||||
app_ref = _get_app_ref(str(app_id))
|
||||
annotation_ref = AppRefService.create_annotation_ref(app_ref, str(annotation_id))
|
||||
annotation = AppAnnotationService.update_app_annotation_directly(update_args, annotation_ref, session)
|
||||
annotation = AppAnnotationService.update_app_annotation_directly(update_args, annotation_ref, db.session())
|
||||
return Annotation.model_validate(annotation, from_attributes=True).model_dump(mode="json")
|
||||
|
||||
@setup_required
|
||||
@@ -425,11 +419,10 @@ class AnnotationUpdateDeleteApi(Resource):
|
||||
@edit_permission_required
|
||||
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_EDIT)
|
||||
@console_ns.response(204, "Annotation deleted successfully")
|
||||
@with_session
|
||||
def delete(self, session: Session, app_id: UUID, annotation_id: UUID):
|
||||
app_ref = _get_app_ref(session, str(app_id))
|
||||
def delete(self, app_id: UUID, annotation_id: UUID):
|
||||
app_ref = _get_app_ref(str(app_id))
|
||||
annotation_ref = AppRefService.create_annotation_ref(app_ref, str(annotation_id))
|
||||
AppAnnotationService.delete_app_annotation(annotation_ref, session)
|
||||
AppAnnotationService.delete_app_annotation(annotation_ref, db.session())
|
||||
return "", 204
|
||||
|
||||
|
||||
@@ -453,8 +446,7 @@ class AnnotationBatchImportApi(Resource):
|
||||
@annotation_import_concurrency_limit
|
||||
@edit_permission_required
|
||||
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_EDIT)
|
||||
@with_session
|
||||
def post(self, session: Session, app_id: UUID):
|
||||
def post(self, app_id: UUID):
|
||||
from configs import dify_config
|
||||
|
||||
# check file
|
||||
@@ -489,7 +481,7 @@ class AnnotationBatchImportApi(Resource):
|
||||
|
||||
return dump_response(
|
||||
AnnotationBatchImportResponse,
|
||||
AppAnnotationService.batch_import_app_annotations(str(app_id), file, session),
|
||||
AppAnnotationService.batch_import_app_annotations(str(app_id), file, session=db.session()),
|
||||
)
|
||||
|
||||
|
||||
@@ -541,17 +533,16 @@ class AnnotationHitHistoryListApi(Resource):
|
||||
@account_initialization_required
|
||||
@edit_permission_required
|
||||
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_VIEW_LAYOUT)
|
||||
@with_session(write=False)
|
||||
def get(self, session: Session, app_id: UUID, annotation_id: UUID):
|
||||
def get(self, app_id: UUID, annotation_id: UUID):
|
||||
page = request.args.get("page", default=1, type=int)
|
||||
limit = request.args.get("limit", default=20, type=int)
|
||||
app_ref = _get_app_ref(session, str(app_id))
|
||||
app_ref = _get_app_ref(str(app_id))
|
||||
annotation_ref = AppRefService.create_annotation_ref(app_ref, str(annotation_id))
|
||||
annotation_hit_history_list, total = AppAnnotationService.get_annotation_hit_histories(
|
||||
annotation_ref,
|
||||
page,
|
||||
limit,
|
||||
session,
|
||||
session=db.session(),
|
||||
)
|
||||
history_models = TypeAdapter(list[AnnotationHitHistory]).validate_python(
|
||||
annotation_hit_history_list, from_attributes=True
|
||||
|
||||
@@ -6,10 +6,10 @@ from typing import Any, Literal
|
||||
|
||||
from flask import request
|
||||
from flask_restx import Resource
|
||||
from pydantic import AliasChoices, BaseModel, Field, ValidationInfo, computed_field, field_validator, model_validator
|
||||
from pydantic import AliasChoices, BaseModel, Field, computed_field, field_validator
|
||||
from sqlalchemy import select
|
||||
from sqlalchemy.orm import Session
|
||||
from werkzeug.exceptions import BadRequest, Forbidden, NotFound
|
||||
from werkzeug.exceptions import BadRequest, NotFound
|
||||
|
||||
from configs import dify_config
|
||||
from controllers.common.app_access import resolve_app_access_filter
|
||||
@@ -23,7 +23,7 @@ from controllers.common.schema import (
|
||||
register_schema_models,
|
||||
)
|
||||
from controllers.console import console_ns
|
||||
from controllers.console.app.wraps import agent_manage_required_for_agent_app, get_app_model, with_session
|
||||
from controllers.console.app.wraps import get_app_model, with_session
|
||||
from controllers.console.workspace.models import LoadBalancingPayload
|
||||
from controllers.console.wraps import (
|
||||
RBACPermission,
|
||||
@@ -52,14 +52,7 @@ from libs.login import login_required
|
||||
from models import Account, App, DatasetPermissionEnum, Workflow
|
||||
from models.model import IconType
|
||||
from services.app_dsl_service import AppDslService
|
||||
from services.app_service import (
|
||||
AppListParams,
|
||||
AppListSortBy,
|
||||
AppResponseView,
|
||||
AppService,
|
||||
CreateAppParams,
|
||||
StarredAppListParams,
|
||||
)
|
||||
from services.app_service import AppListParams, AppListSortBy, AppService, CreateAppParams, StarredAppListParams
|
||||
from services.enterprise import rbac_service as enterprise_rbac_service
|
||||
from services.enterprise.enterprise_service import EnterpriseService
|
||||
from services.entities.dsl_entities import DslImportWarning, ImportMode, ImportStatus
|
||||
@@ -75,7 +68,6 @@ from services.entities.knowledge_entities.knowledge_entities import (
|
||||
WeightModel,
|
||||
WeightVectorSetting,
|
||||
)
|
||||
from services.errors.account import NoPermissionError
|
||||
from services.feature_service import FeatureService
|
||||
from tasks.initialize_created_app_rbac_access_task import initialize_created_app_rbac_access_task
|
||||
|
||||
@@ -247,7 +239,7 @@ class ModelConfigPartial(ResponseModel):
|
||||
return to_timestamp(value)
|
||||
|
||||
|
||||
class AppModelConfigResponse(ResponseModel):
|
||||
class ModelConfig(ResponseModel):
|
||||
opening_statement: str | None = None
|
||||
suggested_questions: Any | None = Field(
|
||||
default=None, validation_alias=AliasChoices("suggested_questions_list", "suggested_questions")
|
||||
@@ -356,18 +348,7 @@ class DeletedTool(ResponseModel):
|
||||
provider_id: str
|
||||
|
||||
|
||||
class AppResponseModel(ResponseModel):
|
||||
@model_validator(mode="before")
|
||||
@classmethod
|
||||
def _use_request_session(cls, value: Any, info: ValidationInfo) -> Any:
|
||||
if not isinstance(value, App):
|
||||
return value
|
||||
if info.context is None or "session" not in info.context:
|
||||
raise ValueError("session context is required to serialize an App")
|
||||
return AppResponseView(value, session=info.context["session"])
|
||||
|
||||
|
||||
class AppPartial(AppResponseModel):
|
||||
class AppPartial(ResponseModel):
|
||||
id: str
|
||||
name: str
|
||||
max_active_requests: int | None = None
|
||||
@@ -411,7 +392,7 @@ class AppPartial(AppResponseModel):
|
||||
return to_timestamp(value)
|
||||
|
||||
|
||||
class AppDetail(AppResponseModel):
|
||||
class AppDetail(ResponseModel):
|
||||
id: str
|
||||
name: str
|
||||
description: str | None = None
|
||||
@@ -420,7 +401,7 @@ class AppDetail(AppResponseModel):
|
||||
icon_background: str | None = None
|
||||
enable_site: bool
|
||||
enable_api: bool
|
||||
model_config_: AppModelConfigResponse | None = Field(
|
||||
model_config_: ModelConfig | None = Field(
|
||||
default=None,
|
||||
validation_alias=AliasChoices("app_model_config", "model_config"),
|
||||
alias="model_config",
|
||||
@@ -526,13 +507,7 @@ def _enrich_app_list_items(session: Session, *, apps: Sequence[App], tenant_id:
|
||||
|
||||
register_enum_models(console_ns, RetrievalMethod, WorkflowExecutionStatus, DatasetPermissionEnum)
|
||||
register_response_schema_models(
|
||||
console_ns,
|
||||
RedirectUrlResponse,
|
||||
SimpleResultResponse,
|
||||
AppImportResponse,
|
||||
AppTraceResponse,
|
||||
AppModelConfigResponse,
|
||||
AppDetail,
|
||||
console_ns, RedirectUrlResponse, SimpleResultResponse, AppImportResponse, AppTraceResponse
|
||||
)
|
||||
|
||||
register_schema_models(
|
||||
@@ -551,8 +526,10 @@ register_schema_models(
|
||||
Tag,
|
||||
WorkflowPartial,
|
||||
ModelConfigPartial,
|
||||
ModelConfig,
|
||||
AppDetailSiteResponse,
|
||||
DeletedTool,
|
||||
AppDetail,
|
||||
AppExportResponse,
|
||||
Segmentation,
|
||||
PreProcessingRule,
|
||||
@@ -610,13 +587,12 @@ class AppListApi(Resource):
|
||||
permissions = enterprise_rbac_service.RBACService.MyPermissions.get(
|
||||
str(current_tenant_id),
|
||||
current_user_id,
|
||||
session=session,
|
||||
session=db.session(),
|
||||
)
|
||||
if dify_config.RBAC_ENABLED:
|
||||
access_filter = resolve_app_access_filter(
|
||||
str(current_tenant_id),
|
||||
current_user_id,
|
||||
session=session,
|
||||
permissions=permissions,
|
||||
)
|
||||
access_filter.apply_to_params(params)
|
||||
@@ -632,11 +608,7 @@ class AppListApi(Resource):
|
||||
permission_keys_map = permissions.app.permission_keys_by_resource_ids(app_ids)
|
||||
_enrich_app_list_items(session, apps=app_pagination.items, tenant_id=current_tenant_id)
|
||||
|
||||
pagination_model = AppPagination.model_validate(
|
||||
app_pagination,
|
||||
from_attributes=True,
|
||||
context={"session": session},
|
||||
)
|
||||
pagination_model = AppPagination.model_validate(app_pagination, from_attributes=True)
|
||||
if app_pagination.items:
|
||||
pagination_model = pagination_model.model_copy(
|
||||
update={
|
||||
@@ -662,8 +634,7 @@ class AppListApi(Resource):
|
||||
@edit_permission_required
|
||||
@with_current_user
|
||||
@with_current_tenant_id
|
||||
@with_session
|
||||
def post(self, session: Session, current_tenant_id: str, current_user: Account):
|
||||
def post(self, current_tenant_id: str, current_user: Account):
|
||||
"""Create app"""
|
||||
args = CreateAppPayload.model_validate(console_ns.payload)
|
||||
params = CreateAppParams(
|
||||
@@ -676,7 +647,7 @@ class AppListApi(Resource):
|
||||
)
|
||||
|
||||
app_service = AppService()
|
||||
app = app_service.create_app(current_tenant_id, params, current_user, session=session)
|
||||
app = app_service.create_app(current_tenant_id, params, current_user, session=db.session())
|
||||
if dify_config.RBAC_ENABLED:
|
||||
enterprise_rbac_service.RBACService.AppAccess.replace_whitelist(
|
||||
tenant_id=str(current_tenant_id),
|
||||
@@ -689,13 +660,11 @@ class AppListApi(Resource):
|
||||
str(current_tenant_id),
|
||||
current_user.id,
|
||||
[str(app.id)],
|
||||
session=session,
|
||||
session=db.session(),
|
||||
)
|
||||
app_detail = AppDetailWithSite.model_validate(app, from_attributes=True).model_copy(
|
||||
update={"permission_keys": permission_keys_map.get(str(app.id), [])}
|
||||
)
|
||||
app_detail = AppDetailWithSite.model_validate(
|
||||
app,
|
||||
from_attributes=True,
|
||||
context={"session": session},
|
||||
).model_copy(update={"permission_keys": permission_keys_map.get(str(app.id), [])})
|
||||
return app_detail.model_dump(mode="json"), 201
|
||||
|
||||
|
||||
@@ -731,14 +700,7 @@ class StarredAppListApi(Resource):
|
||||
return empty.model_dump(mode="json"), 200
|
||||
|
||||
_enrich_app_list_items(session, apps=app_pagination.items, tenant_id=current_tenant_id)
|
||||
return (
|
||||
AppPagination.model_validate(
|
||||
app_pagination,
|
||||
from_attributes=True,
|
||||
context={"session": session},
|
||||
).model_dump(mode="json"),
|
||||
200,
|
||||
)
|
||||
return AppPagination.model_validate(app_pagination, from_attributes=True).model_dump(mode="json"), 200
|
||||
|
||||
|
||||
@console_ns.route("/apps/<uuid:app_id>/star")
|
||||
@@ -789,13 +751,12 @@ class AppApi(Resource):
|
||||
@with_current_user
|
||||
@with_current_tenant_id
|
||||
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_VIEW_LAYOUT)
|
||||
@with_session(write=False)
|
||||
@get_app_model(mode=None)
|
||||
def get(self, session: Session, current_tenant_id: str, current_user: Account, app_model: App):
|
||||
def get(self, current_tenant_id: str, current_user: Account, app_model: App):
|
||||
"""Get app detail"""
|
||||
app_service = AppService()
|
||||
|
||||
app_model = app_service.get_app(app_model, session=session)
|
||||
app_model = app_service.get_app(app_model)
|
||||
|
||||
if FeatureService.get_system_features().webapp_auth.enabled:
|
||||
app_setting = EnterpriseService.WebAppAuth.get_app_access_mode_by_id(app_id=str(app_model.id))
|
||||
@@ -805,15 +766,13 @@ class AppApi(Resource):
|
||||
str(current_tenant_id),
|
||||
current_user.id,
|
||||
app_id=str(app_model.id),
|
||||
session=session,
|
||||
session=db.session(),
|
||||
)
|
||||
permission_keys_map = permissions.app.permission_keys_by_resource_ids([str(app_model.id)])
|
||||
|
||||
response_model = AppDetailWithSite.model_validate(
|
||||
app_model,
|
||||
from_attributes=True,
|
||||
context={"session": session},
|
||||
).model_copy(update={"permission_keys": permission_keys_map.get(str(app_model.id), [])})
|
||||
response_model = AppDetailWithSite.model_validate(app_model, from_attributes=True).model_copy(
|
||||
update={"permission_keys": permission_keys_map.get(str(app_model.id), [])}
|
||||
)
|
||||
return response_model.model_dump(mode="json")
|
||||
|
||||
@console_ns.doc("update_app")
|
||||
@@ -828,10 +787,8 @@ class AppApi(Resource):
|
||||
@account_initialization_required
|
||||
@edit_permission_required
|
||||
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_EDIT)
|
||||
@agent_manage_required_for_agent_app
|
||||
@with_session
|
||||
@get_app_model(mode=None)
|
||||
def put(self, session: Session, app_model: App):
|
||||
def put(self, app_model: App):
|
||||
"""Update app"""
|
||||
args = UpdateAppPayload.model_validate(console_ns.payload)
|
||||
|
||||
@@ -846,12 +803,8 @@ class AppApi(Resource):
|
||||
"use_icon_as_answer_icon": args.use_icon_as_answer_icon or False,
|
||||
"max_active_requests": args.max_active_requests or 0,
|
||||
}
|
||||
app_model = app_service.update_app(app_model, args_dict, session=session)
|
||||
return AppDetailWithSite.model_validate(
|
||||
app_model,
|
||||
from_attributes=True,
|
||||
context={"session": session},
|
||||
).model_dump(mode="json")
|
||||
app_model = app_service.update_app(app_model, args_dict, session=db.session())
|
||||
return dump_response(AppDetailWithSite, app_model)
|
||||
|
||||
@console_ns.doc("delete_app")
|
||||
@console_ns.doc(description="Delete application")
|
||||
@@ -863,13 +816,11 @@ class AppApi(Resource):
|
||||
@account_initialization_required
|
||||
@edit_permission_required
|
||||
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_DELETE)
|
||||
@agent_manage_required_for_agent_app
|
||||
@with_session
|
||||
@get_app_model
|
||||
def delete(self, session: Session, app_model: App):
|
||||
def delete(self, app_model: App):
|
||||
"""Delete app"""
|
||||
app_service = AppService()
|
||||
app_service.delete_app(app_model, session=session)
|
||||
app_service.delete_app(app_model, session=db.session())
|
||||
|
||||
return "", 204
|
||||
|
||||
@@ -888,7 +839,6 @@ class AppCopyApi(Resource):
|
||||
@account_initialization_required
|
||||
@edit_permission_required
|
||||
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_CREATE_AND_MANAGEMENT)
|
||||
@agent_manage_required_for_agent_app
|
||||
@with_current_user
|
||||
@with_current_tenant_id
|
||||
@get_app_model(mode=None)
|
||||
@@ -900,19 +850,16 @@ class AppCopyApi(Resource):
|
||||
with Session(db.engine, expire_on_commit=False) as session:
|
||||
import_service = AppDslService(session)
|
||||
yaml_content = import_service.export_dsl(app_model=app_model, session=session, include_secret=True)
|
||||
try:
|
||||
result = import_service.import_app(
|
||||
account=current_user,
|
||||
import_mode=ImportMode.YAML_CONTENT,
|
||||
yaml_content=yaml_content,
|
||||
name=args.name,
|
||||
description=args.description,
|
||||
icon_type=args.icon_type,
|
||||
icon=args.icon,
|
||||
icon_background=args.icon_background,
|
||||
)
|
||||
except NoPermissionError as e:
|
||||
raise Forbidden(str(e))
|
||||
result = import_service.import_app(
|
||||
account=current_user,
|
||||
import_mode=ImportMode.YAML_CONTENT,
|
||||
yaml_content=yaml_content,
|
||||
name=args.name,
|
||||
description=args.description,
|
||||
icon_type=args.icon_type,
|
||||
icon=args.icon,
|
||||
icon_background=args.icon_background,
|
||||
)
|
||||
if result.status == ImportStatus.FAILED:
|
||||
session.rollback()
|
||||
return dump_response(AppImportResponse, result), 400
|
||||
@@ -936,21 +883,20 @@ class AppCopyApi(Resource):
|
||||
|
||||
stmt = select(App).where(App.id == result.app_id)
|
||||
app = session.scalar(stmt)
|
||||
if not app:
|
||||
raise NotFound("App not found")
|
||||
|
||||
permission_keys_map = enterprise_rbac_service.RBACService.AppPermissions.batch_get(
|
||||
str(current_tenant_id),
|
||||
current_user.id,
|
||||
[str(app.id)],
|
||||
session=session,
|
||||
)
|
||||
response_model = AppDetailWithSite.model_validate(
|
||||
app,
|
||||
from_attributes=True,
|
||||
context={"session": session},
|
||||
).model_copy(update={"permission_keys": permission_keys_map.get(str(app.id), [])})
|
||||
return response_model.model_dump(mode="json"), 201
|
||||
if not app:
|
||||
raise NotFound("App not found")
|
||||
|
||||
permission_keys_map = enterprise_rbac_service.RBACService.AppPermissions.batch_get(
|
||||
str(current_tenant_id),
|
||||
current_user.id,
|
||||
[str(app.id)],
|
||||
session=db.session(),
|
||||
)
|
||||
response_model = AppDetailWithSite.model_validate(app, from_attributes=True).model_copy(
|
||||
update={"permission_keys": permission_keys_map.get(str(app.id), [])}
|
||||
)
|
||||
return response_model.model_dump(mode="json"), 201
|
||||
|
||||
|
||||
@console_ns.route("/apps/<uuid:app_id>/export")
|
||||
@@ -966,7 +912,6 @@ class AppExportApi(Resource):
|
||||
@account_initialization_required
|
||||
@edit_permission_required
|
||||
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_IMPORT_EXPORT_DSL)
|
||||
@agent_manage_required_for_agent_app
|
||||
@get_app_model
|
||||
def get(self, app_model: App):
|
||||
"""Export app"""
|
||||
@@ -991,7 +936,6 @@ class AppPublishToCreatorsPlatformApi(Resource):
|
||||
@account_initialization_required
|
||||
@edit_permission_required
|
||||
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_IMPORT_EXPORT_DSL)
|
||||
@agent_manage_required_for_agent_app
|
||||
@with_current_user_id
|
||||
@get_app_model(mode=None)
|
||||
def post(self, current_user_id: str, app_model: App):
|
||||
@@ -1022,19 +966,13 @@ class AppNameApi(Resource):
|
||||
@account_initialization_required
|
||||
@edit_permission_required
|
||||
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_EDIT)
|
||||
@agent_manage_required_for_agent_app
|
||||
@with_session
|
||||
@get_app_model(mode=None)
|
||||
def post(self, session: Session, app_model: App):
|
||||
def post(self, app_model: App):
|
||||
args = AppNamePayload.model_validate(console_ns.payload)
|
||||
|
||||
app_service = AppService()
|
||||
app_model = app_service.update_app_name(app_model, args.name, session=session)
|
||||
return AppDetail.model_validate(
|
||||
app_model,
|
||||
from_attributes=True,
|
||||
context={"session": session},
|
||||
).model_dump(mode="json")
|
||||
app_model = app_service.update_app_name(app_model, args.name, session=db.session())
|
||||
return dump_response(AppDetail, app_model)
|
||||
|
||||
|
||||
@console_ns.route("/apps/<uuid:app_id>/icon")
|
||||
@@ -1050,10 +988,8 @@ class AppIconApi(Resource):
|
||||
@account_initialization_required
|
||||
@edit_permission_required
|
||||
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_EDIT)
|
||||
@agent_manage_required_for_agent_app
|
||||
@with_session
|
||||
@get_app_model(mode=None)
|
||||
def post(self, session: Session, app_model: App):
|
||||
def post(self, app_model: App):
|
||||
args = AppIconPayload.model_validate(console_ns.payload or {})
|
||||
|
||||
app_service = AppService()
|
||||
@@ -1062,13 +998,9 @@ class AppIconApi(Resource):
|
||||
args.icon or "",
|
||||
args.icon_background or "",
|
||||
args.icon_type,
|
||||
session=session,
|
||||
session=db.session(),
|
||||
)
|
||||
return AppDetail.model_validate(
|
||||
app_model,
|
||||
from_attributes=True,
|
||||
context={"session": session},
|
||||
).model_dump(mode="json")
|
||||
return dump_response(AppDetail, app_model)
|
||||
|
||||
|
||||
@console_ns.route("/apps/<uuid:app_id>/site-enable")
|
||||
@@ -1084,19 +1016,13 @@ class AppSiteStatus(Resource):
|
||||
@account_initialization_required
|
||||
@edit_permission_required
|
||||
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_RELEASE_AND_VERSION)
|
||||
@agent_manage_required_for_agent_app
|
||||
@with_session
|
||||
@get_app_model(mode=None)
|
||||
def post(self, session: Session, app_model: App):
|
||||
def post(self, app_model: App):
|
||||
args = AppSiteStatusPayload.model_validate(console_ns.payload)
|
||||
|
||||
app_service = AppService()
|
||||
app_model = app_service.update_app_site_status(app_model, args.enable_site, session=session)
|
||||
return AppDetail.model_validate(
|
||||
app_model,
|
||||
from_attributes=True,
|
||||
context={"session": session},
|
||||
).model_dump(mode="json")
|
||||
app_model = app_service.update_app_site_status(app_model, args.enable_site, session=db.session())
|
||||
return dump_response(AppDetail, app_model)
|
||||
|
||||
|
||||
@console_ns.route("/apps/<uuid:app_id>/api-enable")
|
||||
@@ -1112,19 +1038,13 @@ class AppApiStatus(Resource):
|
||||
@is_admin_or_owner_required
|
||||
@account_initialization_required
|
||||
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_RELEASE_AND_VERSION)
|
||||
@agent_manage_required_for_agent_app
|
||||
@with_session
|
||||
@get_app_model(mode=None)
|
||||
def post(self, session: Session, app_model: App):
|
||||
def post(self, app_model: App):
|
||||
args = AppApiStatusPayload.model_validate(console_ns.payload)
|
||||
|
||||
app_service = AppService()
|
||||
app_model = app_service.update_app_api_status(app_model, args.enable_api, session=session)
|
||||
return AppDetail.model_validate(
|
||||
app_model,
|
||||
from_attributes=True,
|
||||
context={"session": session},
|
||||
).model_dump(mode="json")
|
||||
app_model = app_service.update_app_api_status(app_model, args.enable_api, session=db.session())
|
||||
return dump_response(AppDetail, app_model)
|
||||
|
||||
|
||||
@console_ns.route("/apps/<uuid:app_id>/trace")
|
||||
|
||||
@@ -1,7 +1,6 @@
|
||||
from flask_restx import Resource
|
||||
from pydantic import BaseModel, Field
|
||||
from sqlalchemy.orm import Session
|
||||
from werkzeug.exceptions import Forbidden
|
||||
|
||||
from configs import dify_config
|
||||
from controllers.common.schema import register_enum_models, register_schema_models
|
||||
@@ -29,7 +28,6 @@ from services.app_dsl_service import (
|
||||
)
|
||||
from services.enterprise.enterprise_service import EnterpriseService
|
||||
from services.entities.dsl_entities import CheckDependenciesResult, ImportStatus
|
||||
from services.errors.account import NoPermissionError
|
||||
from services.feature_service import FeatureService
|
||||
|
||||
from .. import console_ns
|
||||
@@ -93,21 +91,18 @@ class AppImportApi(Resource):
|
||||
import_service = AppDslService(session)
|
||||
# Import app
|
||||
account = current_user
|
||||
try:
|
||||
result = import_service.import_app(
|
||||
account=account,
|
||||
import_mode=args.mode,
|
||||
yaml_content=args.yaml_content,
|
||||
yaml_url=args.yaml_url,
|
||||
name=args.name,
|
||||
description=args.description,
|
||||
icon_type=args.icon_type,
|
||||
icon=args.icon,
|
||||
icon_background=args.icon_background,
|
||||
app_id=args.app_id,
|
||||
)
|
||||
except NoPermissionError as e:
|
||||
raise Forbidden(str(e))
|
||||
result = import_service.import_app(
|
||||
account=account,
|
||||
import_mode=args.mode,
|
||||
yaml_content=args.yaml_content,
|
||||
yaml_url=args.yaml_url,
|
||||
name=args.name,
|
||||
description=args.description,
|
||||
icon_type=args.icon_type,
|
||||
icon=args.icon,
|
||||
icon_background=args.icon_background,
|
||||
app_id=args.app_id,
|
||||
)
|
||||
if result.status == ImportStatus.FAILED:
|
||||
session.rollback()
|
||||
else:
|
||||
@@ -162,10 +157,7 @@ class AppImportConfirmApi(Resource):
|
||||
import_service = AppDslService(session)
|
||||
# Confirm import
|
||||
account = current_user
|
||||
try:
|
||||
result = import_service.confirm_import(import_id=import_id, account=account)
|
||||
except NoPermissionError as e:
|
||||
raise Forbidden(str(e))
|
||||
result = import_service.confirm_import(import_id=import_id, account=account)
|
||||
if result.status == ImportStatus.FAILED:
|
||||
session.rollback()
|
||||
else:
|
||||
|
||||
@@ -127,7 +127,6 @@ def _transcribe_audio_to_text(
|
||||
*,
|
||||
app_model: App,
|
||||
file: FileStorage | None,
|
||||
session: Session,
|
||||
agent_soul: AgentSoulConfig | None = None,
|
||||
) -> dict[str, str]:
|
||||
try:
|
||||
@@ -135,7 +134,6 @@ def _transcribe_audio_to_text(
|
||||
response = AudioService.transcript_asr(
|
||||
app_model=app_model,
|
||||
file=file,
|
||||
session=session,
|
||||
end_user=None,
|
||||
)
|
||||
else:
|
||||
@@ -143,7 +141,6 @@ def _transcribe_audio_to_text(
|
||||
app_model=app_model,
|
||||
agent_soul=agent_soul,
|
||||
file=file,
|
||||
session=session,
|
||||
end_user=None,
|
||||
)
|
||||
return dump_response(AudioTranscriptResponse, response)
|
||||
@@ -197,11 +194,7 @@ class ChatMessageAudioApi(Resource):
|
||||
@account_initialization_required
|
||||
@get_app_model(mode=_CONSOLE_AUDIO_TRANSCRIPT_APP_MODES)
|
||||
def post(self, app_model: App):
|
||||
return _transcribe_audio_to_text(
|
||||
app_model=app_model,
|
||||
file=request.files.get("file"),
|
||||
session=db.session(),
|
||||
)
|
||||
return _transcribe_audio_to_text(app_model=app_model, file=request.files.get("file"))
|
||||
|
||||
|
||||
@console_ns.route("/agent/<uuid:agent_id>/audio-to-text")
|
||||
@@ -235,11 +228,7 @@ class AgentChatMessageAudioApi(Resource):
|
||||
agent_id: UUID,
|
||||
):
|
||||
payload = AgentAudioTranscriptFormPayload.model_validate(request.form.to_dict(flat=True))
|
||||
app_model = resolve_agent_runtime_app_model(
|
||||
session=session,
|
||||
tenant_id=current_tenant_id,
|
||||
agent_id=agent_id,
|
||||
)
|
||||
app_model = resolve_agent_runtime_app_model(tenant_id=current_tenant_id, agent_id=agent_id)
|
||||
# Agent routes expose Agent ids, while APP RBAC is keyed by the resolved runtime App id.
|
||||
enforce_rbac_access(
|
||||
tenant_id=current_tenant_id,
|
||||
@@ -259,7 +248,6 @@ class AgentChatMessageAudioApi(Resource):
|
||||
app_model=app_model,
|
||||
agent_soul=agent_soul,
|
||||
file=request.files.get("file"),
|
||||
session=session,
|
||||
)
|
||||
|
||||
|
||||
|
||||
@@ -44,12 +44,12 @@ from core.errors.error import (
|
||||
QuotaExceededError,
|
||||
)
|
||||
from core.helper.trace_id_helper import get_external_trace_id
|
||||
from extensions.ext_database import db
|
||||
from graphon.model_runtime.errors.invoke import InvokeError
|
||||
from libs import helper
|
||||
from libs.helper import uuid_value
|
||||
from libs.login import login_required
|
||||
from models import Account
|
||||
from models.agent import AgentConfigDraftType
|
||||
from models.model import App, AppMode
|
||||
from services.agent.errors import AgentNotFoundError
|
||||
from services.agent.roster_service import AgentRosterService
|
||||
@@ -158,8 +158,8 @@ class CompletionMessageApi(Resource):
|
||||
@account_initialization_required
|
||||
@with_current_user
|
||||
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_TEST_AND_RUN)
|
||||
@with_session
|
||||
@get_app_model(mode=AppMode.COMPLETION)
|
||||
@with_session
|
||||
def post(self, session: Session, current_user: Account, app_model: App):
|
||||
args_model = CompletionMessagePayload.model_validate(console_ns.payload)
|
||||
args = args_model.model_dump(exclude_none=True, by_alias=True)
|
||||
@@ -240,8 +240,8 @@ class ChatMessageApi(Resource):
|
||||
@with_current_user
|
||||
@with_current_tenant_id
|
||||
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_TEST_AND_RUN)
|
||||
@with_session
|
||||
@get_app_model(mode=[AppMode.CHAT, AppMode.AGENT_CHAT, AppMode.AGENT])
|
||||
@with_session
|
||||
def post(self, session: Session, current_tenant_id: str, current_user: Account, app_model: App):
|
||||
return _create_chat_message(
|
||||
session=session, current_tenant_id=current_tenant_id, current_user=current_user, app_model=app_model
|
||||
@@ -266,9 +266,7 @@ class AgentChatMessageApi(Resource):
|
||||
@with_current_tenant_id
|
||||
@with_session
|
||||
def post(self, session: Session, current_tenant_id: str, current_user: Account, agent_id: UUID):
|
||||
app_model = AgentRosterService(session).get_agent_runtime_app_model(
|
||||
tenant_id=current_tenant_id, agent_id=str(agent_id)
|
||||
)
|
||||
app_model = resolve_agent_runtime_app_model(tenant_id=current_tenant_id, agent_id=agent_id)
|
||||
return _create_chat_message(
|
||||
session=session,
|
||||
current_tenant_id=current_tenant_id,
|
||||
@@ -295,9 +293,7 @@ class AgentBuildChatFinalizeApi(Resource):
|
||||
@with_current_tenant_id
|
||||
@with_session
|
||||
def post(self, session: Session, current_tenant_id: str, current_user: Account, agent_id: UUID):
|
||||
app_model = AgentRosterService(session).get_agent_runtime_app_model(
|
||||
tenant_id=current_tenant_id, agent_id=str(agent_id)
|
||||
)
|
||||
app_model = resolve_agent_runtime_app_model(tenant_id=current_tenant_id, agent_id=agent_id)
|
||||
return _create_build_chat_finalization_message(
|
||||
session=session,
|
||||
current_tenant_id=current_tenant_id,
|
||||
@@ -333,34 +329,20 @@ class AgentChatMessageStopApi(Resource):
|
||||
@account_initialization_required
|
||||
@with_current_user_id
|
||||
@with_current_tenant_id
|
||||
@with_session(write=False)
|
||||
def post(self, session: Session, current_tenant_id: str, current_user_id: str, agent_id: UUID, task_id: str):
|
||||
app_model = resolve_agent_runtime_app_model(
|
||||
session=session,
|
||||
tenant_id=current_tenant_id,
|
||||
agent_id=agent_id,
|
||||
)
|
||||
def post(self, current_tenant_id: str, current_user_id: str, agent_id: UUID, task_id: str):
|
||||
app_model = resolve_agent_runtime_app_model(tenant_id=current_tenant_id, agent_id=agent_id)
|
||||
return _stop_chat_message(current_user_id=current_user_id, app_model=app_model, task_id=task_id)
|
||||
|
||||
|
||||
def _resolve_current_user_agent_debug_conversation_id(
|
||||
*,
|
||||
session: Session,
|
||||
current_tenant_id: str,
|
||||
current_user: Account,
|
||||
app_model: App,
|
||||
agent_id: str | None,
|
||||
draft_type: AgentConfigDraftType,
|
||||
*, current_tenant_id: str, current_user: Account, app_model: App, agent_id: str | None
|
||||
) -> str:
|
||||
"""Resolve the current editor's conversation without crossing draft surfaces."""
|
||||
|
||||
roster_service = AgentRosterService(session)
|
||||
roster_service = AgentRosterService(db.session)
|
||||
if agent_id:
|
||||
return roster_service.get_or_create_agent_app_debug_conversation_id(
|
||||
tenant_id=current_tenant_id,
|
||||
agent_id=agent_id,
|
||||
account_id=current_user.id,
|
||||
draft_type=draft_type,
|
||||
)
|
||||
|
||||
agent = roster_service.get_app_backing_agent(tenant_id=current_tenant_id, app_id=str(app_model.id))
|
||||
@@ -370,7 +352,6 @@ def _resolve_current_user_agent_debug_conversation_id(
|
||||
tenant_id=current_tenant_id,
|
||||
agent_id=agent.id,
|
||||
account_id=current_user.id,
|
||||
draft_type=draft_type,
|
||||
)
|
||||
|
||||
|
||||
@@ -388,12 +369,10 @@ def _create_chat_message(
|
||||
|
||||
if AppMode.value_of(app_model.mode) == AppMode.AGENT:
|
||||
debug_conversation_id = _resolve_current_user_agent_debug_conversation_id(
|
||||
session=session,
|
||||
current_tenant_id=current_tenant_id or app_model.tenant_id,
|
||||
current_user=current_user,
|
||||
app_model=app_model,
|
||||
agent_id=agent_id,
|
||||
draft_type=AgentConfigDraftType(args_model.draft_type),
|
||||
)
|
||||
if args_model.conversation_id and args_model.conversation_id != debug_conversation_id:
|
||||
raise NotFound("Conversation Not Exists.")
|
||||
@@ -425,12 +404,10 @@ def _create_build_chat_finalization_message(
|
||||
*, session: Session, current_user: Account, app_model: App, current_tenant_id: str, agent_id: str
|
||||
):
|
||||
debug_conversation_id = _resolve_current_user_agent_debug_conversation_id(
|
||||
session=session,
|
||||
current_tenant_id=current_tenant_id,
|
||||
current_user=current_user,
|
||||
app_model=app_model,
|
||||
agent_id=agent_id,
|
||||
draft_type=AgentConfigDraftType.DEBUG_BUILD,
|
||||
)
|
||||
args: dict[str, Any] = {
|
||||
"query": _BUILD_CHAT_FINALIZATION_QUERY,
|
||||
|
||||
@@ -6,11 +6,10 @@ from flask import abort, request
|
||||
from flask_restx import Resource
|
||||
from pydantic import BaseModel, Field, field_validator
|
||||
from sqlalchemy import func, or_
|
||||
from sqlalchemy.orm import Session, selectinload
|
||||
from sqlalchemy.orm import selectinload
|
||||
from werkzeug.exceptions import NotFound
|
||||
|
||||
from controllers.common.schema import query_params_from_model, register_response_schema_models, register_schema_models
|
||||
from controllers.common.session import with_session
|
||||
from controllers.console import console_ns
|
||||
from controllers.console.app.wraps import get_app_model
|
||||
from controllers.console.wraps import (
|
||||
@@ -23,6 +22,7 @@ from controllers.console.wraps import (
|
||||
with_current_user,
|
||||
)
|
||||
from core.app.entities.app_invoke_entities import InvokeFrom
|
||||
from extensions.ext_database import db
|
||||
from fields.conversation_fields import (
|
||||
Conversation as ConversationResponse,
|
||||
)
|
||||
@@ -35,7 +35,6 @@ from fields.conversation_fields import (
|
||||
from fields.conversation_fields import (
|
||||
ConversationPagination as ConversationPaginationResponse,
|
||||
)
|
||||
from fields.conversation_fields import ConversationResponseSource
|
||||
from fields.conversation_fields import (
|
||||
ConversationWithSummaryPagination as ConversationWithSummaryPaginationResponse,
|
||||
)
|
||||
@@ -106,9 +105,8 @@ class CompletionConversationApi(Resource):
|
||||
@edit_permission_required
|
||||
@with_current_user
|
||||
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_VIEW_LAYOUT)
|
||||
@with_session(write=False)
|
||||
@get_app_model(mode=AppMode.COMPLETION)
|
||||
def get(self, session: Session, current_user: Account, app_model: App):
|
||||
def get(self, current_user: Account, app_model: App):
|
||||
args = CompletionConversationQuery.model_validate(request.args.to_dict(flat=True))
|
||||
|
||||
query = sa.select(Conversation).where(
|
||||
@@ -159,18 +157,9 @@ class CompletionConversationApi(Resource):
|
||||
|
||||
query = query.order_by(Conversation.created_at.desc())
|
||||
|
||||
conversations = paginate_query(query, session=session, page=args.page, per_page=args.limit)
|
||||
conversations = paginate_query(query, page=args.page, per_page=args.limit)
|
||||
|
||||
return dump_response(
|
||||
ConversationPaginationResponse,
|
||||
{
|
||||
"page": conversations.page,
|
||||
"per_page": conversations.per_page,
|
||||
"total": conversations.total,
|
||||
"has_next": conversations.has_next,
|
||||
"items": [ConversationResponseSource(item, session=session) for item in conversations.items],
|
||||
},
|
||||
)
|
||||
return dump_response(ConversationPaginationResponse, conversations)
|
||||
|
||||
|
||||
@console_ns.route("/apps/<uuid:app_id>/completion-conversations/<uuid:conversation_id>")
|
||||
@@ -187,15 +176,11 @@ class CompletionConversationDetailApi(Resource):
|
||||
@edit_permission_required
|
||||
@with_current_user
|
||||
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_VIEW_LAYOUT)
|
||||
@with_session
|
||||
@get_app_model(mode=AppMode.COMPLETION)
|
||||
def get(self, session: Session, current_user: Account, app_model: App, conversation_id: UUID):
|
||||
def get(self, current_user: Account, app_model: App, conversation_id: UUID):
|
||||
conversation_id_str = str(conversation_id)
|
||||
return dump_response(
|
||||
ConversationMessageDetailResponse,
|
||||
ConversationResponseSource(
|
||||
_get_conversation(session, current_user, app_model, conversation_id_str), session=session
|
||||
),
|
||||
ConversationMessageDetailResponse, _get_conversation(current_user, app_model, conversation_id_str)
|
||||
)
|
||||
|
||||
@console_ns.doc("delete_completion_conversation")
|
||||
@@ -210,13 +195,12 @@ class CompletionConversationDetailApi(Resource):
|
||||
@edit_permission_required
|
||||
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_EDIT)
|
||||
@with_current_user
|
||||
@with_session
|
||||
@get_app_model(mode=AppMode.COMPLETION)
|
||||
def delete(self, session: Session, current_user: Account, app_model: App, conversation_id: UUID):
|
||||
def delete(self, current_user: Account, app_model: App, conversation_id: UUID):
|
||||
conversation_id_str = str(conversation_id)
|
||||
|
||||
try:
|
||||
ConversationService.delete(app_model, conversation_id_str, current_user, session=session)
|
||||
ConversationService.delete(app_model, conversation_id_str, current_user, session=db.session())
|
||||
except ConversationNotExistsError:
|
||||
raise NotFound("Conversation Not Exists.")
|
||||
|
||||
@@ -236,9 +220,8 @@ class ChatConversationApi(Resource):
|
||||
@edit_permission_required
|
||||
@with_current_user
|
||||
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_VIEW_LAYOUT)
|
||||
@with_session(write=False)
|
||||
@get_app_model(mode=[AppMode.CHAT, AppMode.AGENT_CHAT, AppMode.ADVANCED_CHAT, AppMode.AGENT])
|
||||
def get(self, session: Session, current_user: Account, app_model: App):
|
||||
def get(self, current_user: Account, app_model: App):
|
||||
args = ChatConversationQuery.model_validate(request.args.to_dict(flat=True))
|
||||
|
||||
subquery = (
|
||||
@@ -328,18 +311,9 @@ class ChatConversationApi(Resource):
|
||||
case _:
|
||||
query = query.order_by(Conversation.created_at.desc())
|
||||
|
||||
conversations = paginate_query(query, session=session, page=args.page, per_page=args.limit)
|
||||
conversations = paginate_query(query, page=args.page, per_page=args.limit)
|
||||
|
||||
return dump_response(
|
||||
ConversationWithSummaryPaginationResponse,
|
||||
{
|
||||
"page": conversations.page,
|
||||
"per_page": conversations.per_page,
|
||||
"total": conversations.total,
|
||||
"has_next": conversations.has_next,
|
||||
"items": [ConversationResponseSource(item, session=session) for item in conversations.items],
|
||||
},
|
||||
)
|
||||
return dump_response(ConversationWithSummaryPaginationResponse, conversations)
|
||||
|
||||
|
||||
@console_ns.route("/apps/<uuid:app_id>/chat-conversations/<uuid:conversation_id>")
|
||||
@@ -356,15 +330,11 @@ class ChatConversationDetailApi(Resource):
|
||||
@edit_permission_required
|
||||
@with_current_user
|
||||
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_VIEW_LAYOUT)
|
||||
@with_session
|
||||
@get_app_model(mode=[AppMode.CHAT, AppMode.AGENT_CHAT, AppMode.ADVANCED_CHAT, AppMode.AGENT])
|
||||
def get(self, session: Session, current_user: Account, app_model: App, conversation_id: UUID):
|
||||
def get(self, current_user: Account, app_model: App, conversation_id: UUID):
|
||||
conversation_id_str = str(conversation_id)
|
||||
return dump_response(
|
||||
ConversationDetailResponse,
|
||||
ConversationResponseSource(
|
||||
_get_conversation(session, current_user, app_model, conversation_id_str), session=session
|
||||
),
|
||||
ConversationDetailResponse, _get_conversation(current_user, app_model, conversation_id_str)
|
||||
)
|
||||
|
||||
@console_ns.doc("delete_chat_conversation")
|
||||
@@ -379,28 +349,27 @@ class ChatConversationDetailApi(Resource):
|
||||
@edit_permission_required
|
||||
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_EDIT)
|
||||
@with_current_user
|
||||
@with_session
|
||||
@get_app_model(mode=[AppMode.CHAT, AppMode.AGENT_CHAT, AppMode.ADVANCED_CHAT, AppMode.AGENT])
|
||||
def delete(self, session: Session, current_user: Account, app_model: App, conversation_id: UUID):
|
||||
def delete(self, current_user: Account, app_model: App, conversation_id: UUID):
|
||||
conversation_id_str = str(conversation_id)
|
||||
|
||||
try:
|
||||
ConversationService.delete(app_model, conversation_id_str, current_user, session=session)
|
||||
ConversationService.delete(app_model, conversation_id_str, current_user, session=db.session())
|
||||
except ConversationNotExistsError:
|
||||
raise NotFound("Conversation Not Exists.")
|
||||
|
||||
return "", 204
|
||||
|
||||
|
||||
def _get_conversation(session: Session, current_user: Account, app_model, conversation_id):
|
||||
conversation = session.scalar(
|
||||
def _get_conversation(current_user: Account, app_model, conversation_id):
|
||||
conversation = db.session.scalar(
|
||||
sa.select(Conversation).where(Conversation.id == conversation_id, Conversation.app_id == app_model.id).limit(1)
|
||||
)
|
||||
|
||||
if not conversation:
|
||||
raise NotFound("Conversation Not Exists.")
|
||||
|
||||
session.execute(
|
||||
db.session.execute(
|
||||
sa.update(Conversation)
|
||||
.where(Conversation.id == conversation_id, Conversation.read_at.is_(None))
|
||||
# Keep updated_at unchanged when only marking a conversation as read.
|
||||
@@ -410,7 +379,7 @@ def _get_conversation(session: Session, current_user: Account, app_model, conver
|
||||
updated_at=Conversation.updated_at,
|
||||
)
|
||||
)
|
||||
session.flush()
|
||||
session.refresh(conversation)
|
||||
db.session.commit()
|
||||
db.session.refresh(conversation)
|
||||
|
||||
return conversation
|
||||
|
||||
@@ -153,8 +153,8 @@ class AppMCPServerController(Resource):
|
||||
select(AppMCPServer)
|
||||
.where(
|
||||
AppMCPServer.id == server_ref.server_id,
|
||||
AppMCPServer.tenant_id == server_ref.app.tenant_id,
|
||||
AppMCPServer.app_id == server_ref.app.app_id,
|
||||
AppMCPServer.tenant_id == server_ref.tenant_id,
|
||||
AppMCPServer.app_id == server_ref.app_id,
|
||||
)
|
||||
.limit(1)
|
||||
)
|
||||
|
||||
@@ -6,13 +6,11 @@ from flask import request
|
||||
from flask_restx import Resource
|
||||
from pydantic import BaseModel, Field, field_validator
|
||||
from sqlalchemy import exists, func, select
|
||||
from sqlalchemy.orm import Session
|
||||
from werkzeug.exceptions import InternalServerError, NotFound
|
||||
|
||||
from controllers.common.controller_schemas import MessageFeedbackPayload as _MessageFeedbackPayloadBase
|
||||
from controllers.common.fields import SimpleResultResponse, TextFileResponse
|
||||
from controllers.common.schema import query_params_from_model, register_response_schema_models, register_schema_models
|
||||
from controllers.common.session import with_session
|
||||
from controllers.console import console_ns
|
||||
from controllers.console.agent.app_helpers import resolve_agent_runtime_app_model
|
||||
from controllers.console.app.error import (
|
||||
@@ -41,7 +39,6 @@ from fields.base import ResponseModel
|
||||
from fields.conversation_fields import (
|
||||
MessageDetail as BaseMessageDetailResponse,
|
||||
)
|
||||
from fields.conversation_fields import MessageResponseSource
|
||||
from graphon.model_runtime.errors.invoke import InvokeError
|
||||
from libs.helper import dump_response, uuid_value
|
||||
from libs.infinite_scroll_pagination import InfiniteScrollPagination
|
||||
@@ -155,10 +152,9 @@ class ChatMessageListApi(Resource):
|
||||
@edit_permission_required
|
||||
@with_current_user
|
||||
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_VIEW_LAYOUT)
|
||||
@with_session(write=False)
|
||||
@get_app_model(mode=[AppMode.CHAT, AppMode.AGENT_CHAT, AppMode.ADVANCED_CHAT, AppMode.AGENT])
|
||||
def get(self, session: Session, current_user: Account, app_model: App):
|
||||
return _list_chat_messages(session=session, app_model=app_model, current_user=current_user)
|
||||
def get(self, current_user: Account, app_model: App):
|
||||
return _list_chat_messages(app_model=app_model, current_user=current_user)
|
||||
|
||||
|
||||
@console_ns.route("/agent/<uuid:agent_id>/chat-messages")
|
||||
@@ -176,14 +172,9 @@ class AgentChatMessageListApi(Resource):
|
||||
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_VIEW_LAYOUT)
|
||||
@with_current_user
|
||||
@with_current_tenant_id
|
||||
@with_session(write=False)
|
||||
def get(self, session: Session, current_tenant_id: str, current_user: Account, agent_id: UUID):
|
||||
app_model = resolve_agent_runtime_app_model(
|
||||
session=session,
|
||||
tenant_id=current_tenant_id,
|
||||
agent_id=agent_id,
|
||||
)
|
||||
return _list_chat_messages(session=session, app_model=app_model, current_user=current_user)
|
||||
def get(self, current_tenant_id: str, current_user: Account, agent_id: UUID):
|
||||
app_model = resolve_agent_runtime_app_model(tenant_id=current_tenant_id, agent_id=agent_id)
|
||||
return _list_chat_messages(app_model=app_model, current_user=current_user)
|
||||
|
||||
|
||||
@console_ns.route("/apps/<uuid:app_id>/feedbacks")
|
||||
@@ -199,10 +190,9 @@ class MessageFeedbackApi(Resource):
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@with_current_user
|
||||
@with_session
|
||||
@get_app_model
|
||||
def post(self, session: Session, current_user: Account, app_model: App):
|
||||
return _update_message_feedback(session=session, current_user=current_user, app_model=app_model)
|
||||
def post(self, current_user: Account, app_model: App):
|
||||
return _update_message_feedback(current_user=current_user, app_model=app_model)
|
||||
|
||||
|
||||
@console_ns.route("/agent/<uuid:agent_id>/feedbacks")
|
||||
@@ -218,14 +208,9 @@ class AgentMessageFeedbackApi(Resource):
|
||||
@account_initialization_required
|
||||
@with_current_user
|
||||
@with_current_tenant_id
|
||||
@with_session
|
||||
def post(self, session: Session, current_tenant_id: str, current_user: Account, agent_id: UUID):
|
||||
app_model = resolve_agent_runtime_app_model(
|
||||
session=session,
|
||||
tenant_id=current_tenant_id,
|
||||
agent_id=agent_id,
|
||||
)
|
||||
return _update_message_feedback(session=session, current_user=current_user, app_model=app_model)
|
||||
def post(self, current_tenant_id: str, current_user: Account, agent_id: UUID):
|
||||
app_model = resolve_agent_runtime_app_model(tenant_id=current_tenant_id, agent_id=agent_id)
|
||||
return _update_message_feedback(current_user=current_user, app_model=app_model)
|
||||
|
||||
|
||||
@console_ns.route("/apps/<uuid:app_id>/annotations/count")
|
||||
@@ -267,12 +252,9 @@ class MessageSuggestedQuestionApi(Resource):
|
||||
@account_initialization_required
|
||||
@with_current_user
|
||||
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_VIEW_LAYOUT)
|
||||
@with_session(write=False)
|
||||
@get_app_model(mode=[AppMode.CHAT, AppMode.AGENT_CHAT, AppMode.ADVANCED_CHAT, AppMode.AGENT])
|
||||
def get(self, session: Session, current_user: Account, app_model: App, message_id: UUID):
|
||||
return _get_message_suggested_questions(
|
||||
session=session, current_user=current_user, app_model=app_model, message_id=message_id
|
||||
)
|
||||
def get(self, current_user: Account, app_model: App, message_id: UUID):
|
||||
return _get_message_suggested_questions(current_user=current_user, app_model=app_model, message_id=message_id)
|
||||
|
||||
|
||||
@console_ns.route("/agent/<uuid:agent_id>/chat-messages/<uuid:message_id>/suggested-questions")
|
||||
@@ -291,16 +273,9 @@ class AgentMessageSuggestedQuestionApi(Resource):
|
||||
@account_initialization_required
|
||||
@with_current_user
|
||||
@with_current_tenant_id
|
||||
@with_session(write=False)
|
||||
def get(self, session: Session, current_tenant_id: str, current_user: Account, agent_id: UUID, message_id: UUID):
|
||||
app_model = resolve_agent_runtime_app_model(
|
||||
session=session,
|
||||
tenant_id=current_tenant_id,
|
||||
agent_id=agent_id,
|
||||
)
|
||||
return _get_message_suggested_questions(
|
||||
session=session, current_user=current_user, app_model=app_model, message_id=message_id
|
||||
)
|
||||
def get(self, current_tenant_id: str, current_user: Account, agent_id: UUID, message_id: UUID):
|
||||
app_model = resolve_agent_runtime_app_model(tenant_id=current_tenant_id, agent_id=agent_id)
|
||||
return _get_message_suggested_questions(current_user=current_user, app_model=app_model, message_id=message_id)
|
||||
|
||||
|
||||
@console_ns.route("/apps/<uuid:app_id>/feedbacks/export")
|
||||
@@ -358,10 +333,9 @@ class MessageApi(Resource):
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_VIEW_LAYOUT)
|
||||
@with_session(write=False)
|
||||
@get_app_model
|
||||
def get(self, session: Session, app_model: App, message_id: UUID):
|
||||
return _get_message_detail(session=session, app_model=app_model, message_id=message_id)
|
||||
def get(self, app_model: App, message_id: UUID):
|
||||
return _get_message_detail(app_model=app_model, message_id=message_id)
|
||||
|
||||
|
||||
@console_ns.route("/agent/<uuid:agent_id>/messages/<uuid:message_id>")
|
||||
@@ -375,17 +349,12 @@ class AgentMessageApi(Resource):
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@with_current_tenant_id
|
||||
@with_session(write=False)
|
||||
def get(self, session: Session, current_tenant_id: str, agent_id: UUID, message_id: UUID):
|
||||
app_model = resolve_agent_runtime_app_model(
|
||||
session=session,
|
||||
tenant_id=current_tenant_id,
|
||||
agent_id=agent_id,
|
||||
)
|
||||
return _get_message_detail(session=session, app_model=app_model, message_id=message_id)
|
||||
def get(self, current_tenant_id: str, agent_id: UUID, message_id: UUID):
|
||||
app_model = resolve_agent_runtime_app_model(tenant_id=current_tenant_id, agent_id=agent_id)
|
||||
return _get_message_detail(app_model=app_model, message_id=message_id)
|
||||
|
||||
|
||||
def _list_chat_messages(*, session: Session, app_model: App, current_user: Account | None = None):
|
||||
def _list_chat_messages(*, app_model: App, current_user: Account | None = None):
|
||||
args = ChatMessagesQuery.model_validate(request.args.to_dict())
|
||||
|
||||
if AppMode.value_of(app_model.mode) == AppMode.AGENT and current_user is not None:
|
||||
@@ -394,12 +363,12 @@ def _list_chat_messages(*, session: Session, app_model: App, current_user: Accou
|
||||
app_model=app_model,
|
||||
conversation_id=args.conversation_id,
|
||||
user=current_user,
|
||||
session=session,
|
||||
session=db.session(),
|
||||
)
|
||||
except ConversationNotExistsError:
|
||||
raise NotFound("Conversation Not Exists.")
|
||||
else:
|
||||
conversation = session.scalar(
|
||||
conversation = db.session.scalar(
|
||||
select(Conversation)
|
||||
.where(Conversation.id == args.conversation_id, Conversation.app_id == app_model.id)
|
||||
.limit(1)
|
||||
@@ -409,14 +378,14 @@ def _list_chat_messages(*, session: Session, app_model: App, current_user: Accou
|
||||
raise NotFound("Conversation Not Exists.")
|
||||
|
||||
if args.first_id:
|
||||
first_message = session.scalar(
|
||||
first_message = db.session.scalar(
|
||||
select(Message).where(Message.conversation_id == conversation.id, Message.id == args.first_id).limit(1)
|
||||
)
|
||||
|
||||
if not first_message:
|
||||
raise NotFound("First message not found")
|
||||
|
||||
history_messages = session.scalars(
|
||||
history_messages = db.session.scalars(
|
||||
select(Message)
|
||||
.where(
|
||||
Message.conversation_id == conversation.id,
|
||||
@@ -427,7 +396,7 @@ def _list_chat_messages(*, session: Session, app_model: App, current_user: Accou
|
||||
.limit(args.limit)
|
||||
).all()
|
||||
else:
|
||||
history_messages = session.scalars(
|
||||
history_messages = db.session.scalars(
|
||||
select(Message)
|
||||
.where(Message.conversation_id == conversation.id)
|
||||
.order_by(Message.created_at.desc())
|
||||
@@ -438,7 +407,7 @@ def _list_chat_messages(*, session: Session, app_model: App, current_user: Accou
|
||||
if len(history_messages) == args.limit:
|
||||
current_page_first_message = history_messages[-1]
|
||||
# Check if there are more messages before the current page
|
||||
has_more = session.scalar(
|
||||
has_more = db.session.scalar(
|
||||
select(
|
||||
exists().where(
|
||||
Message.conversation_id == conversation.id,
|
||||
@@ -456,28 +425,26 @@ def _list_chat_messages(*, session: Session, app_model: App, current_user: Accou
|
||||
|
||||
return dump_response(
|
||||
MessageInfiniteScrollPaginationResponse,
|
||||
InfiniteScrollPagination(
|
||||
data=[MessageResponseSource(message, session=session) for message in history_messages],
|
||||
limit=args.limit,
|
||||
has_more=has_more,
|
||||
),
|
||||
InfiniteScrollPagination(data=history_messages, limit=args.limit, has_more=has_more),
|
||||
)
|
||||
|
||||
|
||||
def _update_message_feedback(*, session: Session, current_user: Account, app_model: App):
|
||||
def _update_message_feedback(*, current_user: Account, app_model: App):
|
||||
args = MessageFeedbackPayload.model_validate(console_ns.payload)
|
||||
|
||||
message_id = args.message_id
|
||||
|
||||
message = session.scalar(select(Message).where(Message.id == message_id, Message.app_id == app_model.id).limit(1))
|
||||
message = db.session.scalar(
|
||||
select(Message).where(Message.id == message_id, Message.app_id == app_model.id).limit(1)
|
||||
)
|
||||
|
||||
if not message:
|
||||
raise NotFound("Message Not Exists.")
|
||||
|
||||
feedback = message.admin_feedback_with_session(session=session)
|
||||
feedback = message.admin_feedback
|
||||
|
||||
if not args.rating and feedback:
|
||||
session.delete(feedback)
|
||||
db.session.delete(feedback)
|
||||
elif args.rating and feedback:
|
||||
feedback.rating = FeedbackRating(args.rating)
|
||||
feedback.content = args.content
|
||||
@@ -496,14 +463,14 @@ def _update_message_feedback(*, session: Session, current_user: Account, app_mod
|
||||
from_source=FeedbackFromSource.ADMIN,
|
||||
from_account_id=current_user.id,
|
||||
)
|
||||
session.add(feedback)
|
||||
db.session.add(feedback)
|
||||
|
||||
session.commit()
|
||||
db.session.commit()
|
||||
|
||||
return SimpleResultResponse(result="success").model_dump(mode="json")
|
||||
|
||||
|
||||
def _get_message_suggested_questions(*, session: Session, current_user: Account, app_model: App, message_id: UUID):
|
||||
def _get_message_suggested_questions(*, current_user: Account, app_model: App, message_id: UUID):
|
||||
message_id_str = str(message_id)
|
||||
|
||||
try:
|
||||
@@ -512,7 +479,7 @@ def _get_message_suggested_questions(*, session: Session, current_user: Account,
|
||||
message_id=message_id_str,
|
||||
user=current_user,
|
||||
invoke_from=InvokeFrom.DEBUGGER,
|
||||
session=session,
|
||||
session=db.session(),
|
||||
)
|
||||
except MessageNotExistsError:
|
||||
raise NotFound("Message not found")
|
||||
@@ -535,10 +502,10 @@ def _get_message_suggested_questions(*, session: Session, current_user: Account,
|
||||
return dump_response(SuggestedQuestionsResponse, {"data": questions})
|
||||
|
||||
|
||||
def _get_message_detail(*, session: Session, app_model: App, message_id: UUID):
|
||||
def _get_message_detail(*, app_model: App, message_id: UUID):
|
||||
message_id_str = str(message_id)
|
||||
|
||||
message = session.scalar(
|
||||
message = db.session.scalar(
|
||||
select(Message).where(Message.id == message_id_str, Message.app_id == app_model.id).limit(1)
|
||||
)
|
||||
|
||||
@@ -546,4 +513,4 @@ def _get_message_detail(*, session: Session, app_model: App, message_id: UUID):
|
||||
raise NotFound("Message Not Exists.")
|
||||
|
||||
attach_message_extra_contents([message])
|
||||
return dump_response(MessageDetailResponse, MessageResponseSource(message, session=session))
|
||||
return dump_response(MessageDetailResponse, message)
|
||||
|
||||
@@ -4,11 +4,9 @@ from typing import Any, cast
|
||||
from flask import request
|
||||
from flask_restx import Resource
|
||||
from pydantic import BaseModel, Field
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from controllers.common.fields import SimpleResultResponse
|
||||
from controllers.common.schema import register_response_schema_models, register_schema_models
|
||||
from controllers.common.session import with_session
|
||||
from controllers.console import console_ns
|
||||
from controllers.console.app.wraps import get_app_model
|
||||
from controllers.console.wraps import (
|
||||
@@ -25,6 +23,7 @@ from core.agent.entities import AgentToolEntity
|
||||
from core.tools.tool_manager import ToolManager
|
||||
from core.tools.utils.configuration import ToolParameterConfigurationManager
|
||||
from events.app_event import app_model_config_was_updated
|
||||
from extensions.ext_database import db
|
||||
from libs.datetime_utils import naive_utc_now
|
||||
from libs.login import login_required
|
||||
from models.model import App, AppMode, AppModelConfig
|
||||
@@ -94,16 +93,14 @@ class ModelConfigResource(Resource):
|
||||
@account_initialization_required
|
||||
@with_current_user_id
|
||||
@with_current_tenant_id
|
||||
@with_session
|
||||
@get_app_model(mode=[AppMode.AGENT_CHAT, AppMode.CHAT, AppMode.COMPLETION])
|
||||
def post(self, session: Session, current_tenant_id: str, current_user_id: str, app_model: App):
|
||||
"""Modify the app model config and dataset joins in one request transaction."""
|
||||
def post(self, current_tenant_id: str, current_user_id: str, app_model: App):
|
||||
"""Modify app model config"""
|
||||
# validate config
|
||||
model_configuration = AppModelConfigService.validate_configuration(
|
||||
tenant_id=current_tenant_id,
|
||||
config=cast(dict, request.json),
|
||||
app_mode=AppMode.value_of(app_model.mode),
|
||||
session=session,
|
||||
)
|
||||
|
||||
new_app_model_config = AppModelConfig(
|
||||
@@ -113,8 +110,9 @@ class ModelConfigResource(Resource):
|
||||
)
|
||||
new_app_model_config = new_app_model_config.from_model_config_dict(model_configuration)
|
||||
|
||||
if app_model.mode == AppMode.AGENT_CHAT or app_model.is_agent_with_session(session=session):
|
||||
original_app_model_config = app_model.app_model_config_with_session(session=session)
|
||||
if app_model.mode == AppMode.AGENT_CHAT or app_model.is_agent:
|
||||
# get original app model config
|
||||
original_app_model_config = db.session.get(AppModelConfig, app_model.app_model_config_id)
|
||||
if original_app_model_config is None:
|
||||
raise ValueError("Original app model config not found")
|
||||
agent_mode = original_app_model_config.agent_mode_dict
|
||||
@@ -206,17 +204,14 @@ class ModelConfigResource(Resource):
|
||||
# update app model config
|
||||
new_app_model_config.agent_mode = json.dumps(agent_mode)
|
||||
|
||||
session.add(new_app_model_config)
|
||||
session.flush()
|
||||
db.session.add(new_app_model_config)
|
||||
db.session.flush()
|
||||
|
||||
app_model.app_model_config_id = new_app_model_config.id
|
||||
app_model.updated_by = current_user_id
|
||||
app_model.updated_at = naive_utc_now()
|
||||
db.session.commit()
|
||||
|
||||
app_model_config_was_updated.send(
|
||||
app_model,
|
||||
app_model_config=new_app_model_config,
|
||||
session=session,
|
||||
)
|
||||
app_model_config_was_updated.send(app_model, app_model_config=new_app_model_config)
|
||||
|
||||
return {"result": "success"}
|
||||
|
||||
@@ -3,14 +3,12 @@ from typing import Literal
|
||||
from flask_restx import Resource
|
||||
from pydantic import BaseModel, Field, field_validator
|
||||
from sqlalchemy import select
|
||||
from sqlalchemy.orm import Session
|
||||
from werkzeug.exceptions import NotFound
|
||||
|
||||
from constants.languages import supported_language
|
||||
from controllers.common.schema import register_schema_models
|
||||
from controllers.common.session import with_session
|
||||
from controllers.console import console_ns
|
||||
from controllers.console.app.wraps import agent_manage_required_for_agent_app, get_app_model
|
||||
from controllers.console.app.wraps import get_app_model
|
||||
from controllers.console.wraps import (
|
||||
RBACPermission,
|
||||
RBACResourceScope,
|
||||
@@ -21,6 +19,7 @@ from controllers.console.wraps import (
|
||||
setup_required,
|
||||
with_current_user,
|
||||
)
|
||||
from extensions.ext_database import db
|
||||
from fields.base import ResponseModel
|
||||
from libs.datetime_utils import naive_utc_now
|
||||
from libs.helper import dump_response
|
||||
@@ -93,14 +92,12 @@ class AppSite(Resource):
|
||||
@login_required
|
||||
@edit_permission_required
|
||||
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_RELEASE_AND_VERSION)
|
||||
@agent_manage_required_for_agent_app
|
||||
@account_initialization_required
|
||||
@with_current_user
|
||||
@with_session
|
||||
@get_app_model
|
||||
def post(self, session: Session, current_user: Account, app_model: App):
|
||||
def post(self, current_user: Account, app_model: App):
|
||||
args = AppSiteUpdatePayload.model_validate(console_ns.payload or {})
|
||||
site = session.scalar(select(Site).where(Site.app_id == app_model.id).limit(1))
|
||||
site = db.session.scalar(select(Site).where(Site.app_id == app_model.id).limit(1))
|
||||
if not site:
|
||||
raise NotFound
|
||||
|
||||
@@ -129,7 +126,7 @@ class AppSite(Resource):
|
||||
|
||||
site.updated_by = current_user.id
|
||||
site.updated_at = naive_utc_now()
|
||||
session.flush()
|
||||
db.session.commit()
|
||||
|
||||
return dump_response(AppSiteResponse, site)
|
||||
|
||||
@@ -146,20 +143,18 @@ class AppSiteAccessTokenReset(Resource):
|
||||
@login_required
|
||||
@is_admin_or_owner_required
|
||||
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_RELEASE_AND_VERSION)
|
||||
@agent_manage_required_for_agent_app
|
||||
@account_initialization_required
|
||||
@with_current_user
|
||||
@with_session
|
||||
@get_app_model
|
||||
def post(self, session: Session, current_user: Account, app_model: App):
|
||||
site = session.scalar(select(Site).where(Site.app_id == app_model.id).limit(1))
|
||||
def post(self, current_user: Account, app_model: App):
|
||||
site = db.session.scalar(select(Site).where(Site.app_id == app_model.id).limit(1))
|
||||
|
||||
if not site:
|
||||
raise NotFound
|
||||
|
||||
site.code = Site.generate_code(16, session=session)
|
||||
site.code = Site.generate_code(16)
|
||||
site.updated_by = current_user.id
|
||||
site.updated_at = naive_utc_now()
|
||||
session.flush()
|
||||
db.session.commit()
|
||||
|
||||
return dump_response(AppSiteResponse, site)
|
||||
|
||||
@@ -58,7 +58,6 @@ from extensions.ext_database import db
|
||||
from extensions.ext_redis import redis_client
|
||||
from factories import file_factory, variable_factory
|
||||
from fields.base import ResponseModel
|
||||
from fields.conversation_variable_fields import WorkflowConversationVariableResponse
|
||||
from fields.member_fields import SimpleAccount
|
||||
from fields.workflow_run_fields import WorkflowRunNodeExecutionResponse
|
||||
from graphon.enums import NodeType
|
||||
@@ -105,7 +104,6 @@ class SyncDraftWorkflowPayload(BaseModel):
|
||||
graph: dict[str, Any]
|
||||
features: dict[str, Any]
|
||||
hash: str | None = None
|
||||
is_collaborative: bool = Field(default=False, alias="_is_collaborative")
|
||||
environment_variables: list[dict[str, Any]] = Field(
|
||||
default_factory=list,
|
||||
)
|
||||
@@ -240,6 +238,21 @@ class WorkflowOnlineUsersPayload(BaseModel):
|
||||
return list(dict.fromkeys(app_id.strip() for app_id in app_ids if app_id.strip()))
|
||||
|
||||
|
||||
class WorkflowConversationVariableResponse(ResponseModel):
|
||||
id: str
|
||||
name: str
|
||||
value_type: str
|
||||
value: Any
|
||||
description: str
|
||||
|
||||
@field_validator("value_type", mode="before")
|
||||
@classmethod
|
||||
def _serialize_value_type(cls, value: Any) -> str:
|
||||
if hasattr(value, "exposed_type"):
|
||||
return str(value.exposed_type())
|
||||
return str(value)
|
||||
|
||||
|
||||
class PipelineVariableResponse(ResponseModel):
|
||||
label: str
|
||||
variable: str
|
||||
@@ -311,27 +324,6 @@ class WorkflowResponse(ResponseModel):
|
||||
return [_serialize_environment_variable(item) for item in value]
|
||||
|
||||
|
||||
class _WorkflowResponseSource:
|
||||
def __init__(self, workflow: Workflow, *, session: Session) -> None:
|
||||
self._workflow = workflow
|
||||
self._session = session
|
||||
|
||||
def __getattr__(self, name: str) -> object:
|
||||
return getattr(self._workflow, name) # noqa: no-new-getattr response adapter delegates model fields
|
||||
|
||||
@property
|
||||
def created_by_account(self) -> Account | None:
|
||||
return self._workflow.get_created_by_account(session=self._session)
|
||||
|
||||
@property
|
||||
def updated_by_account(self) -> Account | None:
|
||||
return self._workflow.get_updated_by_account(session=self._session)
|
||||
|
||||
@property
|
||||
def tool_published(self) -> bool:
|
||||
return self._workflow.get_tool_published(session=self._session)
|
||||
|
||||
|
||||
class WorkflowPaginationResponse(ResponseModel):
|
||||
items: list[WorkflowResponse]
|
||||
page: int
|
||||
@@ -359,12 +351,6 @@ class WorkflowPublishResponse(ResponseModel):
|
||||
created_at: int
|
||||
|
||||
|
||||
class SyncDraftWorkflowResponse(ResponseModel):
|
||||
result: str
|
||||
hash: str
|
||||
updated_at: int
|
||||
|
||||
|
||||
class WorkflowRestoreResponse(ResponseModel):
|
||||
result: str
|
||||
hash: str
|
||||
@@ -447,7 +433,6 @@ register_response_schema_models(
|
||||
WorkflowOnlineUsersByApp,
|
||||
WorkflowOnlineUsersResponse,
|
||||
WorkflowPublishResponse,
|
||||
SyncDraftWorkflowResponse,
|
||||
WorkflowRestoreResponse,
|
||||
DefaultBlockConfigsResponse,
|
||||
DefaultBlockConfigResponse,
|
||||
@@ -563,7 +548,14 @@ class DraftWorkflowApi(Resource):
|
||||
@console_ns.response(
|
||||
200,
|
||||
"Draft workflow synced successfully",
|
||||
console_ns.models[SyncDraftWorkflowResponse.__name__],
|
||||
console_ns.model(
|
||||
"SyncDraftWorkflowResponse",
|
||||
{
|
||||
"result": fields.String,
|
||||
"hash": fields.String,
|
||||
"updated_at": fields.String,
|
||||
},
|
||||
),
|
||||
)
|
||||
@console_ns.response(400, "Invalid workflow configuration")
|
||||
@console_ns.response(403, "Permission denied")
|
||||
@@ -611,21 +603,17 @@ class DraftWorkflowApi(Resource):
|
||||
environment_variables=environment_variables,
|
||||
conversation_variables=conversation_variables,
|
||||
session=db.session(),
|
||||
graph_only=args["is_collaborative"],
|
||||
)
|
||||
except WorkflowHashNotEqualError:
|
||||
raise DraftWorkflowNotSync()
|
||||
except VariableError as e:
|
||||
raise InvalidArgumentError(description=str(e))
|
||||
|
||||
return dump_response(
|
||||
SyncDraftWorkflowResponse,
|
||||
{
|
||||
"result": "success",
|
||||
"hash": workflow.unique_hash,
|
||||
"updated_at": TimestampField().format(workflow.updated_at or workflow.created_at),
|
||||
},
|
||||
)
|
||||
return {
|
||||
"result": "success",
|
||||
"hash": workflow.unique_hash,
|
||||
"updated_at": TimestampField().format(workflow.updated_at or workflow.created_at),
|
||||
}
|
||||
|
||||
|
||||
@console_ns.route("/apps/<uuid:app_id>/advanced-chat/workflows/draft/run")
|
||||
@@ -641,10 +629,10 @@ class AdvancedChatDraftWorkflowRunApi(Resource):
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_TEST_AND_RUN)
|
||||
@get_app_model(mode=[AppMode.ADVANCED_CHAT])
|
||||
@with_current_user
|
||||
@edit_permission_required
|
||||
@with_session
|
||||
@get_app_model(mode=[AppMode.ADVANCED_CHAT])
|
||||
def post(self, session: Session, current_user: Account, app_model: App):
|
||||
"""
|
||||
Run draft workflow
|
||||
@@ -1086,10 +1074,10 @@ class DraftWorkflowRunApi(Resource):
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_TEST_AND_RUN)
|
||||
@get_app_model(mode=[AppMode.WORKFLOW])
|
||||
@with_current_user
|
||||
@edit_permission_required
|
||||
@with_session
|
||||
@get_app_model(mode=[AppMode.WORKFLOW])
|
||||
def post(self, session: Session, current_user: Account, app_model: App):
|
||||
"""
|
||||
Run draft workflow
|
||||
@@ -1450,7 +1438,7 @@ class PublishedAllWorkflowApi(Resource):
|
||||
)
|
||||
return WorkflowPaginationResponse.model_validate(
|
||||
{
|
||||
"items": [_WorkflowResponseSource(workflow, session=session) for workflow in workflows],
|
||||
"items": workflows,
|
||||
"page": page,
|
||||
"limit": limit,
|
||||
"has_more": has_more,
|
||||
@@ -1544,9 +1532,7 @@ class WorkflowByIdApi(Resource):
|
||||
if not workflow:
|
||||
raise NotFound("Workflow not found")
|
||||
|
||||
response = dump_response(WorkflowResponse, _WorkflowResponseSource(workflow, session=session))
|
||||
|
||||
return response
|
||||
return dump_response(WorkflowResponse, workflow)
|
||||
|
||||
@setup_required
|
||||
@login_required
|
||||
@@ -1640,10 +1626,10 @@ class DraftWorkflowTriggerRunApi(Resource):
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_TEST_AND_RUN)
|
||||
@get_app_model(mode=[AppMode.WORKFLOW])
|
||||
@with_current_user
|
||||
@edit_permission_required
|
||||
@with_session
|
||||
@get_app_model(mode=[AppMode.WORKFLOW])
|
||||
def post(self, session: Session, current_user: Account, app_model: App):
|
||||
"""
|
||||
Poll for trigger events and execute full workflow when event arrives
|
||||
@@ -1651,7 +1637,7 @@ class DraftWorkflowTriggerRunApi(Resource):
|
||||
args = DraftWorkflowTriggerRunPayload.model_validate(console_ns.payload or {})
|
||||
node_id = args.node_id
|
||||
workflow_service = WorkflowService()
|
||||
draft_workflow = workflow_service.get_draft_workflow(app_model, session=session)
|
||||
draft_workflow = workflow_service.get_draft_workflow(app_model, session=db.session())
|
||||
if not draft_workflow:
|
||||
raise ValueError("Workflow not found")
|
||||
|
||||
@@ -1791,11 +1777,11 @@ class DraftWorkflowTriggerRunAllApi(Resource):
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@get_app_model(mode=[AppMode.WORKFLOW])
|
||||
@with_current_user
|
||||
@edit_permission_required
|
||||
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_TEST_AND_RUN)
|
||||
@with_session
|
||||
@get_app_model(mode=[AppMode.WORKFLOW])
|
||||
def post(self, session: Session, current_user: Account, app_model: App):
|
||||
"""
|
||||
Full workflow debug when the start node is a trigger
|
||||
@@ -1804,7 +1790,7 @@ class DraftWorkflowTriggerRunAllApi(Resource):
|
||||
args = DraftWorkflowTriggerRunAllPayload.model_validate(console_ns.payload or {})
|
||||
node_ids = args.node_ids
|
||||
workflow_service = WorkflowService()
|
||||
draft_workflow = workflow_service.get_draft_workflow(app_model, session=session)
|
||||
draft_workflow = workflow_service.get_draft_workflow(app_model, session=db.session())
|
||||
if not draft_workflow:
|
||||
raise ValueError("Workflow not found")
|
||||
|
||||
|
||||
@@ -1,8 +1,9 @@
|
||||
"""Controller decorators for console app resources.
|
||||
|
||||
`get_app_model` still supports legacy handlers backed by Flask-SQLAlchemy's
|
||||
scoped session. Trial app handlers compose `get_app_model_with_trial` under
|
||||
`controllers.common.session.with_session` and always reuse that request session.
|
||||
App-loading decorators prefer a session injected by
|
||||
`controllers.common.session.with_session` when present, while still supporting
|
||||
existing handlers that have not been migrated yet and still rely on
|
||||
Flask-SQLAlchemy's scoped `db.session`.
|
||||
"""
|
||||
|
||||
from collections.abc import Callable
|
||||
@@ -12,22 +13,14 @@ from typing import cast, overload
|
||||
from sqlalchemy import select
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from configs import dify_config
|
||||
from controllers.common.session import with_session
|
||||
from controllers.common.wraps import RBACPermission, RBACResourceScope, enforce_rbac_access
|
||||
from controllers.console.app.error import AppNotFoundError
|
||||
from extensions.ext_database import db
|
||||
from libs.login import current_account_with_tenant
|
||||
from models import App, AppMode, TrialApp
|
||||
from models.agent import AgentScope
|
||||
from services.recommended_app_service import RecommendedAppService
|
||||
|
||||
__all__ = [
|
||||
"agent_manage_required_for_agent_app",
|
||||
"get_app_model",
|
||||
"get_app_model_with_trial",
|
||||
"with_session",
|
||||
]
|
||||
__all__ = ["get_app_model", "get_app_model_with_trial", "with_session"]
|
||||
|
||||
|
||||
def _load_app_model(session: Session, app_id: str) -> App | None:
|
||||
@@ -48,53 +41,14 @@ def _load_app_model_from_scoped_session(app_id: str) -> App | None:
|
||||
return app_model
|
||||
|
||||
|
||||
def _load_app_model_with_trial(session: Session, app_id: str) -> App | None:
|
||||
def _load_app_model_with_trial(app_id: str) -> App | None:
|
||||
"""Load a normal app through its trial registration without applying current-tenant scope."""
|
||||
app_model = session.scalar(
|
||||
app_model = db.session.scalar(
|
||||
select(App).join(TrialApp, TrialApp.app_id == App.id).where(App.id == app_id, App.status == "normal").limit(1)
|
||||
)
|
||||
return app_model
|
||||
|
||||
|
||||
def agent_manage_required_for_agent_app[**P, R](view: Callable[P, R]) -> Callable[P, R]:
|
||||
"""Gate generic app management routes that target an Agent App.
|
||||
|
||||
A hidden workflow-only backing App only reuses the App runtime and is not
|
||||
part of the general app management plane, so generic routes reject it
|
||||
outright. Managing a roster Agent App mutates the roster Agent behind it
|
||||
(rename/icon sync, archive, API enablement), so it additionally requires
|
||||
workspace ``agent.manage`` on top of the route's existing App permission
|
||||
checks when RBAC is enabled. A no-op for non-agent Apps. Must be placed
|
||||
above ``get_app_model`` so the ``app_id`` path parameter is still present.
|
||||
"""
|
||||
|
||||
@wraps(view)
|
||||
def decorated(*args: P.args, **kwargs: P.kwargs) -> R:
|
||||
raw_app_id = kwargs.get("app_id") or kwargs.get("resource_id")
|
||||
if raw_app_id is not None:
|
||||
app_model = _load_app_model_from_scoped_session(str(raw_app_id))
|
||||
binding = (
|
||||
app_model.agent_app_binding_with_session(session=db.session(), include_archived=True)
|
||||
if app_model is not None
|
||||
else None
|
||||
)
|
||||
if binding is not None:
|
||||
if binding.scope == AgentScope.WORKFLOW_ONLY:
|
||||
raise AppNotFoundError()
|
||||
if dify_config.RBAC_ENABLED:
|
||||
current_user, current_tenant_id = current_account_with_tenant()
|
||||
enforce_rbac_access(
|
||||
tenant_id=current_tenant_id,
|
||||
account_id=current_user.id,
|
||||
resource_type=RBACResourceScope.WORKSPACE,
|
||||
scene=RBACPermission.AGENT_MANAGE,
|
||||
resource_required=False,
|
||||
)
|
||||
return view(*args, **kwargs)
|
||||
|
||||
return decorated
|
||||
|
||||
|
||||
def _get_injected_session(args: tuple[object, ...]) -> Session | None:
|
||||
"""Return the request session inserted by `with_session`, if this handler has been migrated."""
|
||||
if len(args) < 2:
|
||||
@@ -203,7 +157,7 @@ def get_app_model_with_trial[**P, R](
|
||||
*,
|
||||
mode: AppMode | list[AppMode] | None = None,
|
||||
) -> Callable[P, R] | Callable[[Callable[P, R]], Callable[P, R]]:
|
||||
"""Inject a trial-registered or recommended App using the Session supplied by `with_session`."""
|
||||
"""Inject an app registered for trial or available from the recommended catalog."""
|
||||
|
||||
def decorator(view_func: Callable[P, R]) -> Callable[P, R]:
|
||||
@wraps(view_func)
|
||||
@@ -216,12 +170,9 @@ def get_app_model_with_trial[**P, R](
|
||||
|
||||
del kwargs["app_id"]
|
||||
|
||||
session = _get_injected_session(args)
|
||||
if session is None:
|
||||
raise RuntimeError("get_app_model_with_trial requires @with_session")
|
||||
app_model = _load_app_model_with_trial(session, app_id)
|
||||
app_model = _load_app_model_with_trial(app_id)
|
||||
if app_model is None:
|
||||
app_model = RecommendedAppService.get_app(app_id, session=session)
|
||||
app_model = RecommendedAppService.get_app(app_id, session=db.session())
|
||||
|
||||
if not app_model:
|
||||
raise AppNotFoundError()
|
||||
|
||||
@@ -7,13 +7,10 @@ from configs import dify_config
|
||||
from constants.languages import supported_language
|
||||
from controllers.common.schema import query_params_from_model, register_schema_models
|
||||
from controllers.console import console_ns
|
||||
from controllers.console.auth.error import InvitationAccountMismatchError
|
||||
from controllers.console.error import AccountInFreezeError, AlreadyActivateError
|
||||
from extensions.ext_database import db
|
||||
from libs.datetime_utils import naive_utc_now
|
||||
from libs.helper import EmailStr, timezone
|
||||
from libs.login import current_account_with_tenant
|
||||
from libs.token import extract_access_token
|
||||
from models import AccountStatus
|
||||
from models.account import TenantAccountJoin, TenantAccountRole
|
||||
from services.account_service import RegisterService, TenantService
|
||||
@@ -139,12 +136,6 @@ class ActivateApi(Resource):
|
||||
)
|
||||
@console_ns.response(400, "Already activated or invalid token")
|
||||
def post(self):
|
||||
"""Accept an invitation without letting an existing session act for another account.
|
||||
|
||||
Token-only activation remains available for legacy clients. When the request already
|
||||
carries a console session, that session must belong to the account encoded in the
|
||||
invitation before the token is consumed or tenant membership is changed.
|
||||
"""
|
||||
args = ActivatePayload.model_validate(console_ns.payload)
|
||||
|
||||
normalized_request_email = args.email.lower() if args.email else None
|
||||
@@ -155,11 +146,6 @@ class ActivateApi(Resource):
|
||||
raise AlreadyActivateError()
|
||||
|
||||
account = invitation["account"]
|
||||
if extract_access_token(request):
|
||||
current_account, _ = current_account_with_tenant()
|
||||
if current_account.id != account.id:
|
||||
raise InvitationAccountMismatchError()
|
||||
|
||||
if dify_config.BILLING_ENABLED and BillingService.is_email_in_freeze(account.email):
|
||||
raise AccountInFreezeError()
|
||||
|
||||
|
||||
@@ -13,12 +13,6 @@ class InvalidEmailError(BaseHTTPException):
|
||||
code = 400
|
||||
|
||||
|
||||
class InvitationAccountMismatchError(BaseHTTPException):
|
||||
error_code = "invitation_account_mismatch"
|
||||
description = "This invitation was sent to another account. Please sign in with the invited account."
|
||||
code = 403
|
||||
|
||||
|
||||
class PasswordMismatchError(BaseHTTPException):
|
||||
error_code = "password_mismatch"
|
||||
description = "The passwords do not match."
|
||||
|
||||
@@ -16,6 +16,7 @@ from controllers.console.auth.error import (
|
||||
)
|
||||
from controllers.console.error import AccountNotFound, EmailSendIpLimitError
|
||||
from controllers.console.wraps import email_password_login_enabled, setup_required
|
||||
from events.tenant_event import tenant_was_created
|
||||
from extensions.ext_database import db
|
||||
from libs.helper import EmailStr, extract_remote_ip
|
||||
from libs.password import hash_password
|
||||
@@ -200,4 +201,7 @@ class ForgotPasswordResetApi(Resource):
|
||||
not TenantService.get_join_tenants(account, session=db.session())
|
||||
and FeatureService.get_system_features().is_allow_create_workspace
|
||||
):
|
||||
TenantService.create_owner_tenant(account, session=db.session())
|
||||
tenant = TenantService.create_tenant(f"{account.name}'s Workspace", session=db.session())
|
||||
TenantService.create_tenant_member(tenant, account, db.session(), role="owner")
|
||||
account.current_tenant = tenant
|
||||
tenant_was_created.send(tenant)
|
||||
|
||||
@@ -4,7 +4,6 @@ import flask_login
|
||||
from flask import make_response, request
|
||||
from flask_restx import Resource
|
||||
from pydantic import BaseModel, Field, field_validator
|
||||
from sqlalchemy.orm import Session
|
||||
from werkzeug.exceptions import Unauthorized
|
||||
|
||||
import services
|
||||
@@ -17,7 +16,6 @@ from controllers.common.fields import (
|
||||
SimpleResultResponse,
|
||||
)
|
||||
from controllers.common.schema import register_response_schema_models, register_schema_models
|
||||
from controllers.common.session import with_session
|
||||
from controllers.console import console_ns
|
||||
from controllers.console.auth.error import (
|
||||
AuthenticationFailedError,
|
||||
@@ -42,6 +40,7 @@ from controllers.console.wraps import (
|
||||
setup_required,
|
||||
with_current_user,
|
||||
)
|
||||
from events.tenant_event import tenant_was_created
|
||||
from extensions.ext_database import db
|
||||
from libs.helper import EmailStr, extract_remote_ip
|
||||
from libs.helper import timezone as validate_timezone_string
|
||||
@@ -316,7 +315,10 @@ class EmailCodeLoginApi(Resource):
|
||||
if not FeatureService.get_system_features().is_allow_create_workspace:
|
||||
raise NotAllowedCreateWorkspace()
|
||||
else:
|
||||
TenantService.create_owner_tenant(account, session=db.session())
|
||||
new_tenant = TenantService.create_tenant(f"{account.name}'s Workspace", session=db.session())
|
||||
TenantService.create_tenant_member(new_tenant, account, db.session(), role="owner")
|
||||
account.current_tenant = new_tenant
|
||||
tenant_was_created.send(new_tenant)
|
||||
|
||||
if account is None:
|
||||
try:
|
||||
@@ -354,8 +356,7 @@ class EmailCodeLoginApi(Resource):
|
||||
class RefreshTokenApi(Resource):
|
||||
@console_ns.response(200, "Success", console_ns.models[SimpleResultResponse.__name__])
|
||||
@console_ns.response(401, "Unauthorized", console_ns.models[SimpleResultMessageResponse.__name__])
|
||||
@with_session(write=False)
|
||||
def post(self, session: Session):
|
||||
def post(self):
|
||||
# Get refresh token from cookie instead of request body
|
||||
refresh_token = extract_refresh_token(request)
|
||||
|
||||
@@ -365,7 +366,7 @@ class RefreshTokenApi(Resource):
|
||||
), 401
|
||||
|
||||
try:
|
||||
new_token_pair = AccountService.refresh_token(refresh_token, session=session)
|
||||
new_token_pair = AccountService.refresh_token(refresh_token, session=db.session())
|
||||
except Unauthorized as exc:
|
||||
return SimpleResultMessageResponse(result="fail", message=exc.description or "Unauthorized.").model_dump(
|
||||
mode="json"
|
||||
|
||||
@@ -6,12 +6,12 @@ from flask import current_app, redirect, request
|
||||
from flask_restx import Resource
|
||||
from pydantic import BaseModel, Field
|
||||
from werkzeug.exceptions import Unauthorized
|
||||
from werkzeug.wrappers import Response
|
||||
|
||||
from configs import dify_config
|
||||
from constants.languages import languages
|
||||
from controllers.common.fields import RedirectResponse
|
||||
from controllers.common.schema import query_params_from_model, register_response_schema_model, register_schema_models
|
||||
from events.tenant_event import tenant_was_created
|
||||
from extensions.ext_database import db
|
||||
from libs.datetime_utils import naive_utc_now
|
||||
from libs.helper import extract_remote_ip
|
||||
@@ -128,20 +128,6 @@ def _preferred_interface_language(language: str | None = None) -> str:
|
||||
return languages[0]
|
||||
|
||||
|
||||
def _redirect_with_console_session(account: Account, target_url: str) -> Response:
|
||||
"""Create a console session and attach its cookies to a redirect response."""
|
||||
token_pair = AccountService.login(
|
||||
account=account,
|
||||
session=db.session(),
|
||||
ip_address=extract_remote_ip(request),
|
||||
)
|
||||
response = redirect(target_url)
|
||||
set_access_token_to_cookie(request, response, token_pair.access_token)
|
||||
set_refresh_token_to_cookie(request, response, token_pair.refresh_token)
|
||||
set_csrf_token_to_cookie(request, response, token_pair.csrf_token)
|
||||
return response
|
||||
|
||||
|
||||
@console_ns.route("/oauth/login/<provider>")
|
||||
class OAuthLogin(Resource):
|
||||
@console_ns.doc("oauth_login")
|
||||
@@ -210,26 +196,16 @@ class OAuthCallback(Resource):
|
||||
return redirect(f"{dify_config.CONSOLE_WEB_URL}/signin?message={urllib.parse.quote(str(e))}")
|
||||
|
||||
if invite_token and RegisterService.is_valid_invite_token(invite_token):
|
||||
invitation = RegisterService.get_invitation_if_token_valid(
|
||||
None,
|
||||
None,
|
||||
invite_token,
|
||||
session=db.session(),
|
||||
)
|
||||
if not invitation:
|
||||
return redirect(f"{dify_config.CONSOLE_WEB_URL}/signin?message=Invalid invitation token.")
|
||||
if invitation["data"]["email"].lower() != user_info.email.lower():
|
||||
message = "This invitation was sent to another account. Please sign in with the invited account."
|
||||
query = urllib.parse.urlencode({"message": message, "invite_token": invite_token})
|
||||
return redirect(f"{dify_config.CONSOLE_WEB_URL}/signin?{query}")
|
||||
invitation = RegisterService.get_invitation_by_token(token=invite_token)
|
||||
if invitation:
|
||||
invitation_email = invitation.get("email", None)
|
||||
invitation_email_normalized = (
|
||||
invitation_email.lower() if isinstance(invitation_email, str) else invitation_email
|
||||
)
|
||||
if invitation_email_normalized != user_info.email.lower():
|
||||
return redirect(f"{dify_config.CONSOLE_WEB_URL}/signin?message=Invalid invitation token.")
|
||||
|
||||
account = invitation["account"]
|
||||
if account.status == AccountStatus.BANNED:
|
||||
return redirect(f"{dify_config.CONSOLE_WEB_URL}/signin?message=Account is banned.")
|
||||
|
||||
AccountService.link_account_integrate(provider, user_info.id, account, session=db.session())
|
||||
target_url = f"{dify_config.CONSOLE_WEB_URL}/signin/invite-settings?invite_token={invite_token}"
|
||||
return _redirect_with_console_session(account, target_url)
|
||||
return redirect(f"{dify_config.CONSOLE_WEB_URL}/signin/invite-settings?invite_token={invite_token}")
|
||||
|
||||
try:
|
||||
account, oauth_new_user = _generate_account(provider, user_info, timezone=timezone, language=language)
|
||||
@@ -264,10 +240,21 @@ class OAuthCallback(Resource):
|
||||
"?message=Workspace not found, please contact system admin to invite you to join in a workspace."
|
||||
)
|
||||
|
||||
token_pair = AccountService.login(
|
||||
account=account,
|
||||
session=db.session(),
|
||||
ip_address=extract_remote_ip(request),
|
||||
)
|
||||
|
||||
target_url = _get_redirect_target(redirect_url)
|
||||
query_char = "&" if "?" in target_url else "?"
|
||||
target_url = f"{target_url}{query_char}oauth_new_user={str(oauth_new_user).lower()}"
|
||||
return _redirect_with_console_session(account, target_url)
|
||||
response = redirect(target_url)
|
||||
|
||||
set_access_token_to_cookie(request, response, token_pair.access_token)
|
||||
set_refresh_token_to_cookie(request, response, token_pair.refresh_token)
|
||||
set_csrf_token_to_cookie(request, response, token_pair.csrf_token)
|
||||
return response
|
||||
|
||||
|
||||
def _get_account_by_openid_or_email(provider: str, user_info: OAuthUserInfo) -> Account | None:
|
||||
@@ -295,7 +282,10 @@ def _generate_account(
|
||||
if not FeatureService.get_system_features().is_allow_create_workspace:
|
||||
raise WorkSpaceNotAllowedCreateError()
|
||||
else:
|
||||
TenantService.create_owner_tenant(account, session=db.session())
|
||||
new_tenant = TenantService.create_tenant(f"{account.name}'s Workspace", session=db.session())
|
||||
TenantService.create_tenant_member(new_tenant, account, db.session(), role="owner")
|
||||
account.current_tenant = new_tenant
|
||||
tenant_was_created.send(new_tenant)
|
||||
|
||||
if not account:
|
||||
normalized_email = user_info.email.lower()
|
||||
|
||||
@@ -10,7 +10,7 @@ from werkzeug.exceptions import BadRequest, NotFound
|
||||
|
||||
from controllers.common.schema import register_response_schema_models, register_schema_models
|
||||
from controllers.console.wraps import account_initialization_required, setup_required, with_current_user
|
||||
from core.db.session_factory import session_factory
|
||||
from extensions.ext_database import db
|
||||
from graphon.model_runtime.utils.encoders import jsonable_encoder
|
||||
from libs.login import login_required
|
||||
from models import Account
|
||||
@@ -132,10 +132,9 @@ def oauth_server_access_token_required[T, **P, R](
|
||||
response.headers["WWW-Authenticate"] = "Bearer"
|
||||
return response
|
||||
|
||||
with session_factory.create_session() as session:
|
||||
account = OAuthServerService.validate_oauth_access_token(
|
||||
oauth_provider_app.client_id, access_token, session
|
||||
)
|
||||
account = OAuthServerService.validate_oauth_access_token(
|
||||
oauth_provider_app.client_id, access_token, db.session()
|
||||
)
|
||||
if not account:
|
||||
response = jsonify({"error": "access_token or client_id is invalid"})
|
||||
response.status_code = 401
|
||||
|
||||
@@ -8,12 +8,11 @@ from flask import request
|
||||
from flask_restx import Resource
|
||||
from pydantic import BaseModel, Field, field_serializer
|
||||
from sqlalchemy import select
|
||||
from sqlalchemy.orm import Session
|
||||
from sqlalchemy.orm import sessionmaker
|
||||
from werkzeug.exceptions import NotFound
|
||||
|
||||
from controllers.common.fields import SimpleResultResponse, TextContentResponse
|
||||
from controllers.common.schema import query_params_from_model, register_response_schema_models, register_schema_models
|
||||
from controllers.common.session import with_session
|
||||
from core.datasource.entities.datasource_entities import DatasourceProviderType, OnlineDocumentPagesMessage
|
||||
from core.datasource.online_document.online_document_plugin import OnlineDocumentDatasourcePlugin
|
||||
from core.entities.knowledge_entities import IndexingEstimate
|
||||
@@ -189,16 +188,16 @@ class DataSourceApi(Resource):
|
||||
@account_initialization_required
|
||||
@console_ns.response(200, "Success", console_ns.models[SimpleResultResponse.__name__])
|
||||
@with_current_tenant_id
|
||||
@with_session
|
||||
def patch(
|
||||
self, session: Session, current_tenant_id: str, binding_id: UUID, action: Literal["enable", "disable"]
|
||||
self, current_tenant_id: str, binding_id: UUID, action: Literal["enable", "disable"]
|
||||
) -> tuple[dict[str, str], int]:
|
||||
binding_id_str = str(binding_id)
|
||||
data_source_binding = session.scalar(
|
||||
select(DataSourceOauthBinding).where(
|
||||
DataSourceOauthBinding.id == binding_id_str, DataSourceOauthBinding.tenant_id == current_tenant_id
|
||||
)
|
||||
)
|
||||
with sessionmaker(db.engine, expire_on_commit=False).begin() as session:
|
||||
data_source_binding = session.execute(
|
||||
select(DataSourceOauthBinding).where(
|
||||
DataSourceOauthBinding.id == binding_id_str, DataSourceOauthBinding.tenant_id == current_tenant_id
|
||||
)
|
||||
).scalar_one_or_none()
|
||||
if data_source_binding is None:
|
||||
raise NotFound("Data source binding not found.")
|
||||
# enable binding
|
||||
@@ -207,6 +206,8 @@ class DataSourceApi(Resource):
|
||||
if data_source_binding.disabled:
|
||||
data_source_binding.disabled = False
|
||||
data_source_binding.updated_at = naive_utc_now()
|
||||
db.session.add(data_source_binding)
|
||||
db.session.commit()
|
||||
else:
|
||||
raise ValueError("Data source is not disabled.")
|
||||
# disable binding
|
||||
@@ -214,6 +215,8 @@ class DataSourceApi(Resource):
|
||||
if not data_source_binding.disabled:
|
||||
data_source_binding.disabled = True
|
||||
data_source_binding.updated_at = naive_utc_now()
|
||||
db.session.add(data_source_binding)
|
||||
db.session.commit()
|
||||
else:
|
||||
raise ValueError("Data source is disabled.")
|
||||
return {"result": "success"}, 200
|
||||
@@ -228,8 +231,7 @@ class DataSourceNotionListApi(Resource):
|
||||
@console_ns.response(200, "Success", console_ns.models[NotionIntegrateInfoListResponse.__name__])
|
||||
@with_current_user
|
||||
@with_current_tenant_id
|
||||
@with_session(write=False)
|
||||
def get(self, session: Session, current_tenant_id: str, current_user: Account) -> tuple[dict[str, Any], int]:
|
||||
def get(self, current_tenant_id: str, current_user: Account) -> tuple[dict[str, Any], int]:
|
||||
query = DataSourceNotionListQuery.model_validate(request.args.to_dict(flat=True))
|
||||
datasource_provider_service = DatasourceProviderService()
|
||||
credential = datasource_provider_service.get_datasource_credentials(
|
||||
@@ -243,13 +245,13 @@ class DataSourceNotionListApi(Resource):
|
||||
exist_page_ids = []
|
||||
# import notion in the exist dataset
|
||||
if query.dataset_id:
|
||||
dataset = DatasetService.get_dataset(query.dataset_id, session)
|
||||
dataset = DatasetService.get_dataset(query.dataset_id, db.session())
|
||||
if not dataset:
|
||||
raise NotFound("Dataset not found.")
|
||||
if dataset.data_source_type != "notion_import":
|
||||
raise ValueError("Dataset is not notion type.")
|
||||
|
||||
documents = session.scalars(
|
||||
documents = db.session.scalars(
|
||||
select(Document).where(
|
||||
Document.dataset_id == query.dataset_id,
|
||||
Document.tenant_id == current_tenant_id,
|
||||
@@ -353,8 +355,7 @@ class DataSourceNotionIndexingEstimateApi(Resource):
|
||||
@console_ns.expect(console_ns.models[NotionEstimatePayload.__name__])
|
||||
@console_ns.response(200, "Success", console_ns.models[IndexingEstimate.__name__])
|
||||
@with_current_tenant_id
|
||||
@with_session
|
||||
def post(self, session: Session, current_tenant_id: str) -> tuple[dict[str, Any], int]:
|
||||
def post(self, current_tenant_id: str) -> tuple[dict[str, Any], int]:
|
||||
payload = NotionEstimatePayload.model_validate(console_ns.payload or {})
|
||||
args = payload.model_dump()
|
||||
# validate args
|
||||
@@ -381,12 +382,11 @@ class DataSourceNotionIndexingEstimateApi(Resource):
|
||||
extract_settings.append(extract_setting)
|
||||
indexing_runner = IndexingRunner()
|
||||
response = indexing_runner.indexing_estimate(
|
||||
tenant_id=current_tenant_id,
|
||||
extract_settings=extract_settings,
|
||||
tmp_processing_rule=args["process_rule"],
|
||||
doc_form=args["doc_form"],
|
||||
doc_language=args["doc_language"],
|
||||
session=session,
|
||||
current_tenant_id,
|
||||
extract_settings,
|
||||
args["process_rule"],
|
||||
args["doc_form"],
|
||||
args["doc_language"],
|
||||
)
|
||||
return dump_response(IndexingEstimate, response), 200
|
||||
|
||||
@@ -398,14 +398,13 @@ class DataSourceNotionDatasetSyncApi(Resource):
|
||||
@account_initialization_required
|
||||
@console_ns.response(200, "Success", console_ns.models[SimpleResultResponse.__name__])
|
||||
@rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_CREATE_AND_MANAGEMENT)
|
||||
@with_session(write=False)
|
||||
def get(self, session: Session, dataset_id: UUID) -> tuple[dict[str, str], int]:
|
||||
def get(self, dataset_id: UUID) -> tuple[dict[str, str], int]:
|
||||
dataset_id_str = str(dataset_id)
|
||||
dataset = DatasetService.get_dataset(dataset_id_str, session)
|
||||
dataset = DatasetService.get_dataset(dataset_id_str, db.session())
|
||||
if dataset is None:
|
||||
raise NotFound("Dataset not found.")
|
||||
|
||||
documents = DocumentService.get_document_by_dataset_id(dataset_id_str, session)
|
||||
documents = DocumentService.get_document_by_dataset_id(dataset_id_str, db.session())
|
||||
for document in documents:
|
||||
document_indexing_sync_task.delay(dataset_id_str, document.id)
|
||||
return {"result": "success"}, 200
|
||||
@@ -418,15 +417,14 @@ class DataSourceNotionDocumentSyncApi(Resource):
|
||||
@account_initialization_required
|
||||
@console_ns.response(200, "Success", console_ns.models[SimpleResultResponse.__name__])
|
||||
@rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_CREATE_AND_MANAGEMENT)
|
||||
@with_session(write=False)
|
||||
def get(self, session: Session, dataset_id: UUID, document_id: UUID) -> tuple[dict[str, str], int]:
|
||||
def get(self, dataset_id: UUID, document_id: UUID) -> tuple[dict[str, str], int]:
|
||||
dataset_id_str = str(dataset_id)
|
||||
document_id_str = str(document_id)
|
||||
dataset = DatasetService.get_dataset(dataset_id_str, session)
|
||||
dataset = DatasetService.get_dataset(dataset_id_str, db.session())
|
||||
if dataset is None:
|
||||
raise NotFound("Dataset not found.")
|
||||
|
||||
document = DocumentService.get_document(dataset_id_str, document_id_str, session=session)
|
||||
document = DocumentService.get_document(dataset_id_str, document_id_str, session=db.session())
|
||||
if document is None:
|
||||
raise NotFound("Document not found.")
|
||||
document_indexing_sync_task.delay(dataset_id_str, document_id_str)
|
||||
|
||||
@@ -1,4 +1,3 @@
|
||||
from dataclasses import dataclass
|
||||
from datetime import datetime
|
||||
from typing import Any
|
||||
from uuid import UUID
|
||||
@@ -14,10 +13,10 @@ import services
|
||||
from configs import dify_config
|
||||
from controllers.common.fields import ApiBaseUrlResponse, SimpleResultResponse, UsageCheckResponse
|
||||
from controllers.common.schema import query_params_from_model, register_response_schema_models, register_schema_models
|
||||
from controllers.common.session import with_session
|
||||
from controllers.console import console_ns
|
||||
from controllers.console.apikey import ApiKeyItem, ApiKeyList
|
||||
from controllers.console.app.error import ProviderNotInitializeError
|
||||
from controllers.console.app.wraps import with_session
|
||||
from controllers.console.datasets.error import DatasetInUseError, DatasetNameDuplicateError, IndexingEstimateError
|
||||
from controllers.console.wraps import (
|
||||
RBACPermission,
|
||||
@@ -40,18 +39,18 @@ from core.rag.extractor.entity.datasource_type import DatasourceType
|
||||
from core.rag.extractor.entity.extract_setting import ExtractSetting, NotionInfo, WebsiteInfo
|
||||
from core.rag.index_processor.constant.index_type import IndexTechniqueType
|
||||
from core.rag.retrieval.retrieval_methods import RetrievalMethod
|
||||
from extensions.ext_database import db
|
||||
from fields.base import ResponseModel
|
||||
from fields.dataset_fields import DatasetDetailResponse, dataset_detail_response_source
|
||||
from fields.dataset_fields import DatasetDetailResponse
|
||||
from graphon.model_runtime.entities.model_entities import ModelType
|
||||
from libs.helper import build_icon_url, dump_response, to_timestamp
|
||||
from libs.login import login_required
|
||||
from libs.url_utils import normalize_api_base_url
|
||||
from models import Account, ApiToken, App, Dataset, Document, DocumentSegment, UploadFile
|
||||
from models.dataset import DatasetPermission, DatasetPermissionEnum, DatasetQuery
|
||||
from models import Account, ApiToken, Dataset, Document, DocumentSegment, UploadFile
|
||||
from models.dataset import DatasetPermission, DatasetPermissionEnum
|
||||
from models.enums import ApiTokenType, SegmentStatus
|
||||
from models.provider_ids import ModelProviderID
|
||||
from services.api_token_service import ApiTokenCache
|
||||
from services.app_service import AppService
|
||||
from services.dataset_service import DatasetPermissionService, DatasetService, DocumentService
|
||||
from services.enterprise import rbac_service as enterprise_rbac_service
|
||||
from services.enterprise.rbac_service import RBACResourceWhitelistScope, ReplaceMemberBindings
|
||||
@@ -206,21 +205,6 @@ class DatasetQueryDetailResponse(ResponseModel):
|
||||
return to_timestamp(value)
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class _DatasetQueryResponseSource:
|
||||
"""Expose query content through the request's database session."""
|
||||
|
||||
query: DatasetQuery
|
||||
session: Session
|
||||
|
||||
@property
|
||||
def queries(self) -> list[dict[str, Any]]:
|
||||
return self.query.get_queries(session=self.session)
|
||||
|
||||
def __getattr__(self, name: str) -> Any:
|
||||
return getattr(self.query, name) # noqa: no-new-getattr response adapter delegates model fields
|
||||
|
||||
|
||||
class DatasetQueryListResponse(ResponseModel):
|
||||
data: list[DatasetQueryDetailResponse]
|
||||
has_more: bool
|
||||
@@ -245,21 +229,6 @@ class RelatedAppResponse(ResponseModel):
|
||||
return self
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class _RelatedAppResponseSource:
|
||||
"""Expose the compatible app mode through the request's database session."""
|
||||
|
||||
app: App
|
||||
session: Session
|
||||
|
||||
@property
|
||||
def mode_compatible_with_agent(self) -> str:
|
||||
return self.app.mode_compatible_with_agent_with_session(session=self.session)
|
||||
|
||||
def __getattr__(self, name: str) -> Any:
|
||||
return getattr(self.app, name) # noqa: no-new-getattr response adapter delegates model fields
|
||||
|
||||
|
||||
class RelatedAppListResponse(ResponseModel):
|
||||
data: list[RelatedAppResponse]
|
||||
total: int
|
||||
@@ -428,8 +397,7 @@ class DatasetListApi(Resource):
|
||||
@enterprise_license_required
|
||||
@with_current_user
|
||||
@with_current_tenant_id
|
||||
@with_session(write=False)
|
||||
def get(self, session: Session, current_tenant_id: str, current_user: Account):
|
||||
def get(self, current_tenant_id: str, current_user: Account):
|
||||
# Convert query parameters to dict, handling list parameters correctly
|
||||
query_params: dict[str, str | list[str]] = dict(request.args.to_dict())
|
||||
# Handle ids and tag_ids as lists (Flask request.args.getlist returns list even for single value)
|
||||
@@ -442,7 +410,7 @@ class DatasetListApi(Resource):
|
||||
permissions = enterprise_rbac_service.RBACService.MyPermissions.get(
|
||||
str(current_tenant_id),
|
||||
current_user.id,
|
||||
session=session,
|
||||
session=db.session(),
|
||||
)
|
||||
|
||||
accessible_dataset_ids: list[str] | None = None
|
||||
@@ -481,13 +449,12 @@ class DatasetListApi(Resource):
|
||||
user=current_user,
|
||||
accessible_dataset_ids=accessible_dataset_ids,
|
||||
include_own_datasets=include_own_datasets,
|
||||
session=session,
|
||||
)
|
||||
else:
|
||||
datasets, total = DatasetService.get_datasets(
|
||||
query.page,
|
||||
query.limit,
|
||||
session,
|
||||
db.session(),
|
||||
current_tenant_id,
|
||||
current_user,
|
||||
query.keyword,
|
||||
@@ -512,14 +479,11 @@ class DatasetListApi(Resource):
|
||||
for embedding_model in embedding_models:
|
||||
model_names.append(f"{embedding_model.model}:{embedding_model.provider.provider}")
|
||||
|
||||
data = [
|
||||
dump_response(DatasetDetailResponse, dataset_detail_response_source(dataset, session=session))
|
||||
for dataset in datasets
|
||||
]
|
||||
data = [dump_response(DatasetDetailResponse, dataset) for dataset in datasets]
|
||||
dataset_ids = [item["id"] for item in data if item.get("permission") == "partial_members"]
|
||||
partial_members_map: dict[str, list[str]] = {}
|
||||
if dataset_ids:
|
||||
partial_member_rows = session.execute(
|
||||
partial_member_rows = db.session.execute(
|
||||
select(DatasetPermission.dataset_id, DatasetPermission.account_id).where(
|
||||
DatasetPermission.dataset_id.in_(dataset_ids)
|
||||
)
|
||||
@@ -607,13 +571,6 @@ class DatasetListApi(Resource):
|
||||
ReplaceMemberBindings(scope=RBACResourceWhitelistScope.ALL),
|
||||
)
|
||||
initialize_created_app_rbac_access_task.delay(current_tenant_id, current_user.id, dataset_id=dataset.id)
|
||||
else:
|
||||
enterprise_rbac_service.RBACService.DatasetAccess.replace_whitelist(
|
||||
current_tenant_id,
|
||||
current_user.id,
|
||||
dataset.id,
|
||||
ReplaceMemberBindings(scope=RBACResourceWhitelistScope.SPECIFIC),
|
||||
)
|
||||
|
||||
permission_keys_map = enterprise_rbac_service.RBACService.DatasetPermissions.batch_get(
|
||||
current_tenant_id,
|
||||
@@ -622,9 +579,9 @@ class DatasetListApi(Resource):
|
||||
session=session,
|
||||
)
|
||||
|
||||
item = DatasetDetailWithPartialMembersResponse.model_validate(
|
||||
dataset_detail_response_source(dataset, session=session), from_attributes=True
|
||||
).model_dump(mode="json")
|
||||
item = DatasetDetailWithPartialMembersResponse.model_validate(dataset, from_attributes=True).model_dump(
|
||||
mode="json"
|
||||
)
|
||||
item["permission_keys"] = permission_keys_map.get(dataset.id, [])
|
||||
return item, 201
|
||||
|
||||
@@ -647,31 +604,30 @@ class DatasetApi(Resource):
|
||||
@rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_READONLY)
|
||||
@with_current_user
|
||||
@with_current_tenant_id
|
||||
@with_session(write=False)
|
||||
def get(self, session: Session, current_tenant_id: str, current_user: Account, dataset_id: UUID):
|
||||
def get(self, current_tenant_id: str, current_user: Account, dataset_id: UUID):
|
||||
dataset_id_str = str(dataset_id)
|
||||
dataset = DatasetService.get_dataset(dataset_id_str, session)
|
||||
dataset = DatasetService.get_dataset(dataset_id_str, db.session())
|
||||
if dataset is None:
|
||||
raise NotFound("Dataset not found.")
|
||||
try:
|
||||
DatasetService.check_dataset_permission(dataset, current_user, session)
|
||||
DatasetService.check_dataset_permission(dataset, current_user, db.session())
|
||||
except services.errors.account.NoPermissionError as e:
|
||||
raise Forbidden(str(e))
|
||||
permissions = enterprise_rbac_service.RBACService.MyPermissions.get(
|
||||
current_tenant_id,
|
||||
current_user.id,
|
||||
dataset_id=dataset_id_str,
|
||||
session=session,
|
||||
session=db.session(),
|
||||
)
|
||||
permission_keys_map = permissions.dataset.permission_keys_by_resource_ids([dataset_id_str])
|
||||
data = dump_response(DatasetDetailResponse, dataset_detail_response_source(dataset, session=session))
|
||||
data = dump_response(DatasetDetailResponse, dataset)
|
||||
data["permission_keys"] = permission_keys_map.get(dataset_id_str, [])
|
||||
if dataset.indexing_technique == IndexTechniqueType.HIGH_QUALITY:
|
||||
if dataset.embedding_model_provider:
|
||||
provider_id = ModelProviderID(dataset.embedding_model_provider)
|
||||
data["embedding_model_provider"] = str(provider_id)
|
||||
if data.get("permission") == "partial_members":
|
||||
part_users_list = DatasetPermissionService.get_dataset_partial_member_list(dataset_id_str, session)
|
||||
part_users_list = DatasetPermissionService.get_dataset_partial_member_list(dataset_id_str, db.session())
|
||||
data.update({"partial_member_list": part_users_list})
|
||||
|
||||
# check embedding setting
|
||||
@@ -715,7 +671,7 @@ class DatasetApi(Resource):
|
||||
@with_session
|
||||
def patch(self, session: Session, current_tenant_id: str, current_user: Account, dataset_id: UUID):
|
||||
dataset_id_str = str(dataset_id)
|
||||
dataset = DatasetService.get_dataset(dataset_id_str, session)
|
||||
dataset = DatasetService.get_dataset(dataset_id_str, db.session())
|
||||
if dataset is None:
|
||||
raise NotFound("Dataset not found.")
|
||||
|
||||
@@ -748,19 +704,19 @@ class DatasetApi(Resource):
|
||||
[dataset_id_str],
|
||||
session=session,
|
||||
)
|
||||
result_data = dump_response(DatasetDetailResponse, dataset_detail_response_source(dataset, session=session))
|
||||
result_data = dump_response(DatasetDetailResponse, dataset)
|
||||
result_data["permission_keys"] = permission_keys_map.get(dataset_id_str, [])
|
||||
tenant_id = current_tenant_id
|
||||
|
||||
if payload.partial_member_list is not None and payload.permission == DatasetPermissionEnum.PARTIAL_TEAM:
|
||||
DatasetPermissionService.update_partial_member_list(
|
||||
tenant_id, dataset_id_str, payload.partial_member_list, session
|
||||
tenant_id, dataset_id_str, payload.partial_member_list, db.session()
|
||||
)
|
||||
# clear partial member list when permission is only_me or all_team_members
|
||||
elif payload.permission in {DatasetPermissionEnum.ONLY_ME, DatasetPermissionEnum.ALL_TEAM}:
|
||||
DatasetPermissionService.clear_partial_member_list(dataset_id_str, session)
|
||||
DatasetPermissionService.clear_partial_member_list(dataset_id_str, db.session())
|
||||
|
||||
partial_member_list = DatasetPermissionService.get_dataset_partial_member_list(dataset_id_str, session)
|
||||
partial_member_list = DatasetPermissionService.get_dataset_partial_member_list(dataset_id_str, db.session())
|
||||
result_data.update({"partial_member_list": partial_member_list})
|
||||
|
||||
return dump_response(DatasetDetailWithPartialMembersResponse, result_data), 200
|
||||
@@ -772,16 +728,15 @@ class DatasetApi(Resource):
|
||||
@console_ns.response(204, "Dataset deleted successfully")
|
||||
@with_current_user
|
||||
@rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_EDIT)
|
||||
@with_session
|
||||
def delete(self, session: Session, current_user: Account, dataset_id: UUID):
|
||||
def delete(self, current_user: Account, dataset_id: UUID):
|
||||
dataset_id_str = str(dataset_id)
|
||||
|
||||
if not (current_user.has_edit_permission or current_user.is_dataset_operator):
|
||||
raise Forbidden()
|
||||
|
||||
try:
|
||||
if DatasetService.delete_dataset(dataset_id_str, current_user, session):
|
||||
DatasetPermissionService.clear_partial_member_list(dataset_id_str, session)
|
||||
if DatasetService.delete_dataset(dataset_id_str, current_user, db.session()):
|
||||
DatasetPermissionService.clear_partial_member_list(dataset_id_str, db.session())
|
||||
return "", 204
|
||||
else:
|
||||
raise NotFound("Dataset not found.")
|
||||
@@ -803,11 +758,10 @@ class DatasetUseCheckApi(Resource):
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_READONLY)
|
||||
@with_session(write=False)
|
||||
def get(self, session: Session, dataset_id: UUID):
|
||||
def get(self, dataset_id: UUID):
|
||||
dataset_id_str = str(dataset_id)
|
||||
|
||||
dataset_is_using = DatasetService.dataset_use_check(dataset_id_str, session)
|
||||
dataset_is_using = DatasetService.dataset_use_check(dataset_id_str, db.session())
|
||||
return UsageCheckResponse(is_using=dataset_is_using).model_dump(mode="json"), 200
|
||||
|
||||
|
||||
@@ -826,27 +780,24 @@ class DatasetQueryApi(Resource):
|
||||
@account_initialization_required
|
||||
@with_current_user
|
||||
@rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_READONLY)
|
||||
@with_session(write=False)
|
||||
def get(self, session: Session, current_user: Account, dataset_id: UUID):
|
||||
def get(self, current_user: Account, dataset_id: UUID):
|
||||
dataset_id_str = str(dataset_id)
|
||||
dataset = DatasetService.get_dataset(dataset_id_str, session)
|
||||
dataset = DatasetService.get_dataset(dataset_id_str, db.session())
|
||||
if dataset is None:
|
||||
raise NotFound("Dataset not found.")
|
||||
|
||||
try:
|
||||
DatasetService.check_dataset_permission(dataset, current_user, session)
|
||||
DatasetService.check_dataset_permission(dataset, current_user, db.session())
|
||||
except services.errors.account.NoPermissionError as e:
|
||||
raise Forbidden(str(e))
|
||||
|
||||
page = request.args.get("page", default=1, type=int)
|
||||
limit = request.args.get("limit", default=20, type=int)
|
||||
|
||||
dataset_queries, total = DatasetService.get_dataset_queries(
|
||||
dataset_id=dataset.id, page=page, per_page=limit, session=session
|
||||
)
|
||||
dataset_queries, total = DatasetService.get_dataset_queries(dataset_id=dataset.id, page=page, per_page=limit)
|
||||
|
||||
response = {
|
||||
"data": [_DatasetQueryResponseSource(query=query, session=session) for query in dataset_queries],
|
||||
"data": dataset_queries,
|
||||
"has_more": len(dataset_queries) == limit,
|
||||
"limit": limit,
|
||||
"total": total,
|
||||
@@ -869,8 +820,7 @@ class DatasetIndexingEstimateApi(Resource):
|
||||
@account_initialization_required
|
||||
@console_ns.expect(console_ns.models[IndexingEstimatePayload.__name__])
|
||||
@with_current_tenant_id
|
||||
@with_session
|
||||
def post(self, session: Session, current_tenant_id: str):
|
||||
def post(self, current_tenant_id: str):
|
||||
payload = IndexingEstimatePayload.model_validate(console_ns.payload or {})
|
||||
args = payload.model_dump()
|
||||
# validate args
|
||||
@@ -879,10 +829,10 @@ class DatasetIndexingEstimateApi(Resource):
|
||||
match args["info_list"]["data_source_type"]:
|
||||
case "upload_file":
|
||||
file_ids = args["info_list"]["file_info_list"]["file_ids"]
|
||||
file_details = session.scalars(
|
||||
file_details = db.session.scalars(
|
||||
select(UploadFile).where(UploadFile.tenant_id == current_tenant_id, UploadFile.id.in_(file_ids))
|
||||
).all()
|
||||
if not file_details:
|
||||
if file_details is None:
|
||||
raise NotFound("File not found.")
|
||||
|
||||
if file_details:
|
||||
@@ -936,14 +886,13 @@ class DatasetIndexingEstimateApi(Resource):
|
||||
indexing_runner = IndexingRunner()
|
||||
try:
|
||||
response = indexing_runner.indexing_estimate(
|
||||
tenant_id=current_tenant_id,
|
||||
extract_settings=extract_settings,
|
||||
tmp_processing_rule=args["process_rule"],
|
||||
doc_form=args["doc_form"],
|
||||
doc_language=args["doc_language"],
|
||||
dataset_id=args["dataset_id"],
|
||||
indexing_technique=args["indexing_technique"],
|
||||
session=session,
|
||||
current_tenant_id,
|
||||
extract_settings,
|
||||
args["process_rule"],
|
||||
args["doc_form"],
|
||||
args["doc_language"],
|
||||
args["dataset_id"],
|
||||
args["indexing_technique"],
|
||||
)
|
||||
except LLMBadRequestError:
|
||||
raise ProviderNotInitializeError(
|
||||
@@ -982,25 +931,24 @@ class DatasetRelatedAppListApi(Resource):
|
||||
@account_initialization_required
|
||||
@with_current_user
|
||||
@rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_READONLY)
|
||||
@with_session(write=False)
|
||||
def get(self, session: Session, current_user: Account, dataset_id: UUID):
|
||||
def get(self, current_user: Account, dataset_id: UUID):
|
||||
dataset_id_str = str(dataset_id)
|
||||
dataset = DatasetService.get_dataset(dataset_id_str, session)
|
||||
dataset = DatasetService.get_dataset(dataset_id_str, db.session())
|
||||
if dataset is None:
|
||||
raise NotFound("Dataset not found.")
|
||||
|
||||
try:
|
||||
DatasetService.check_dataset_permission(dataset, current_user, session)
|
||||
DatasetService.check_dataset_permission(dataset, current_user, db.session())
|
||||
except services.errors.account.NoPermissionError as e:
|
||||
raise Forbidden(str(e))
|
||||
|
||||
app_dataset_joins = DatasetService.get_related_apps(dataset.id, session)
|
||||
app_dataset_joins = DatasetService.get_related_apps(dataset.id, db.session())
|
||||
|
||||
related_apps = []
|
||||
for app_dataset_join in app_dataset_joins:
|
||||
app_model = AppService.get_app_by_id(app_dataset_join.app_id, session)
|
||||
app_model = app_dataset_join.app
|
||||
if app_model:
|
||||
related_apps.append(_RelatedAppResponseSource(app=app_model, session=session))
|
||||
related_apps.append(app_model)
|
||||
|
||||
return dump_response(RelatedAppListResponse, {"data": related_apps, "total": len(related_apps)}), 200
|
||||
|
||||
@@ -1020,16 +968,15 @@ class DatasetIndexingStatusApi(Resource):
|
||||
@account_initialization_required
|
||||
@with_current_tenant_id
|
||||
@rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_READONLY)
|
||||
@with_session(write=False)
|
||||
def get(self, session: Session, current_tenant_id: str, dataset_id: UUID):
|
||||
def get(self, current_tenant_id: str, dataset_id: UUID):
|
||||
dataset_id_str = str(dataset_id)
|
||||
documents = session.scalars(
|
||||
documents = db.session.scalars(
|
||||
select(Document).where(Document.dataset_id == dataset_id_str, Document.tenant_id == current_tenant_id)
|
||||
).all()
|
||||
documents_status = []
|
||||
for document in documents:
|
||||
completed_segments = (
|
||||
session.scalar(
|
||||
db.session.scalar(
|
||||
select(func.count(DocumentSegment.id)).where(
|
||||
DocumentSegment.completed_at.isnot(None),
|
||||
DocumentSegment.document_id == str(document.id),
|
||||
@@ -1039,7 +986,7 @@ class DatasetIndexingStatusApi(Resource):
|
||||
or 0
|
||||
)
|
||||
total_segments = (
|
||||
session.scalar(
|
||||
db.session.scalar(
|
||||
select(func.count(DocumentSegment.id)).where(
|
||||
DocumentSegment.document_id == str(document.id),
|
||||
DocumentSegment.status != SegmentStatus.RE_SEGMENT,
|
||||
@@ -1079,9 +1026,8 @@ class DatasetApiKeyApi(Resource):
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@with_current_tenant_id
|
||||
@with_session(write=False)
|
||||
def get(self, session: Session, current_tenant_id: str):
|
||||
keys = session.scalars(
|
||||
def get(self, current_tenant_id: str):
|
||||
keys = db.session.scalars(
|
||||
select(ApiToken).where(ApiToken.type == self.resource_type, ApiToken.tenant_id == current_tenant_id)
|
||||
).all()
|
||||
return dump_response(ApiKeyList, {"data": keys})
|
||||
@@ -1094,10 +1040,9 @@ class DatasetApiKeyApi(Resource):
|
||||
@rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_API_KEY_MANAGE, resource_required=False)
|
||||
@account_initialization_required
|
||||
@with_current_tenant_id
|
||||
@with_session
|
||||
def post(self, session: Session, current_tenant_id: str):
|
||||
def post(self, current_tenant_id: str):
|
||||
current_key_count = (
|
||||
session.scalar(
|
||||
db.session.scalar(
|
||||
select(func.count(ApiToken.id)).where(
|
||||
ApiToken.type == self.resource_type, ApiToken.tenant_id == current_tenant_id
|
||||
)
|
||||
@@ -1112,13 +1057,13 @@ class DatasetApiKeyApi(Resource):
|
||||
custom="max_keys_exceeded",
|
||||
)
|
||||
|
||||
key = ApiToken.generate_api_key(self.token_prefix, 24, session=session)
|
||||
key = ApiToken.generate_api_key(self.token_prefix, 24)
|
||||
api_token = ApiToken()
|
||||
api_token.tenant_id = current_tenant_id
|
||||
api_token.token = key
|
||||
api_token.type = self.resource_type
|
||||
session.add(api_token)
|
||||
session.flush()
|
||||
db.session.add(api_token)
|
||||
db.session.commit()
|
||||
return dump_response(ApiKeyItem, api_token), 200
|
||||
|
||||
|
||||
@@ -1136,10 +1081,9 @@ class DatasetApiDeleteApi(Resource):
|
||||
@rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_API_KEY_MANAGE, resource_required=False)
|
||||
@account_initialization_required
|
||||
@with_current_tenant_id
|
||||
@with_session
|
||||
def delete(self, session: Session, current_tenant_id: str, api_key_id: UUID):
|
||||
def delete(self, current_tenant_id: str, api_key_id: UUID):
|
||||
api_key_id_str = str(api_key_id)
|
||||
key = session.scalar(
|
||||
key = db.session.scalar(
|
||||
select(ApiToken)
|
||||
.where(
|
||||
ApiToken.tenant_id == current_tenant_id,
|
||||
@@ -1157,7 +1101,8 @@ class DatasetApiDeleteApi(Resource):
|
||||
assert key is not None # nosec - for type checker only
|
||||
ApiTokenCache.delete(key.token, key.type)
|
||||
|
||||
session.delete(key)
|
||||
db.session.delete(key)
|
||||
db.session.commit()
|
||||
|
||||
return "", 204
|
||||
|
||||
@@ -1169,11 +1114,10 @@ class DatasetEnableApiApi(Resource):
|
||||
@account_initialization_required
|
||||
@console_ns.response(200, "Success", console_ns.models[SimpleResultResponse.__name__])
|
||||
@rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_EDIT)
|
||||
@with_session
|
||||
def post(self, session: Session, dataset_id: UUID, status: str):
|
||||
def post(self, dataset_id: UUID, status: str):
|
||||
dataset_id_str = str(dataset_id)
|
||||
|
||||
DatasetService.update_dataset_api_status(dataset_id_str, status == "enable", session)
|
||||
DatasetService.update_dataset_api_status(dataset_id_str, status == "enable", db.session())
|
||||
|
||||
return SimpleResultResponse(result="success").model_dump(mode="json"), 200
|
||||
|
||||
@@ -1240,13 +1184,12 @@ class DatasetErrorDocs(Resource):
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_READONLY)
|
||||
@with_session(write=False)
|
||||
def get(self, session: Session, dataset_id: UUID):
|
||||
def get(self, dataset_id: UUID):
|
||||
dataset_id_str = str(dataset_id)
|
||||
dataset = DatasetService.get_dataset(dataset_id_str, session)
|
||||
dataset = DatasetService.get_dataset(dataset_id_str, db.session())
|
||||
if dataset is None:
|
||||
raise NotFound("Dataset not found.")
|
||||
results = DocumentService.get_error_documents_by_dataset_id(dataset_id_str, session)
|
||||
results = DocumentService.get_error_documents_by_dataset_id(dataset_id_str, db.session())
|
||||
|
||||
return dump_response(ErrorDocsResponse, {"data": results, "total": len(results)}), 200
|
||||
|
||||
@@ -1268,18 +1211,17 @@ class DatasetPermissionUserListApi(Resource):
|
||||
@account_initialization_required
|
||||
@with_current_user
|
||||
@rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_READONLY)
|
||||
@with_session(write=False)
|
||||
def get(self, session: Session, current_user: Account, dataset_id: UUID):
|
||||
def get(self, current_user: Account, dataset_id: UUID):
|
||||
dataset_id_str = str(dataset_id)
|
||||
dataset = DatasetService.get_dataset(dataset_id_str, session)
|
||||
dataset = DatasetService.get_dataset(dataset_id_str, db.session())
|
||||
if dataset is None:
|
||||
raise NotFound("Dataset not found.")
|
||||
try:
|
||||
DatasetService.check_dataset_permission(dataset, current_user, session)
|
||||
DatasetService.check_dataset_permission(dataset, current_user, db.session())
|
||||
except services.errors.account.NoPermissionError as e:
|
||||
raise Forbidden(str(e))
|
||||
|
||||
partial_members_list = DatasetPermissionService.get_dataset_partial_member_list(dataset_id_str, session)
|
||||
partial_members_list = DatasetPermissionService.get_dataset_partial_member_list(dataset_id_str, db.session())
|
||||
|
||||
return dump_response(PartialMemberListResponse, {"data": partial_members_list}), 200
|
||||
|
||||
@@ -1299,11 +1241,10 @@ class DatasetAutoDisableLogApi(Resource):
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_READONLY)
|
||||
@with_session(write=False)
|
||||
def get(self, session: Session, dataset_id: UUID):
|
||||
def get(self, dataset_id: UUID):
|
||||
dataset_id_str = str(dataset_id)
|
||||
dataset = DatasetService.get_dataset(dataset_id_str, session)
|
||||
dataset = DatasetService.get_dataset(dataset_id_str, db.session())
|
||||
if dataset is None:
|
||||
raise NotFound("Dataset not found.")
|
||||
auto_disable_logs = DatasetService.get_dataset_auto_disable_logs(dataset_id_str, session)
|
||||
auto_disable_logs = DatasetService.get_dataset_auto_disable_logs(dataset_id_str, db.session())
|
||||
return dump_response(AutoDisableLogsResponse, auto_disable_logs), 200
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
import json
|
||||
import logging
|
||||
from argparse import ArgumentTypeError
|
||||
from collections.abc import Mapping, Sequence
|
||||
from collections.abc import Sequence
|
||||
from contextlib import ExitStack
|
||||
from datetime import datetime
|
||||
from typing import Any, Literal, cast
|
||||
@@ -12,14 +12,12 @@ from flask import request, send_file
|
||||
from flask_restx import Resource
|
||||
from pydantic import BaseModel, Field, JsonValue, field_validator
|
||||
from sqlalchemy import asc, desc, func, select
|
||||
from sqlalchemy.orm import Session
|
||||
from werkzeug.exceptions import Forbidden, NotFound
|
||||
|
||||
import services
|
||||
from controllers.common.controller_schemas import DocumentBatchDownloadZipPayload
|
||||
from controllers.common.fields import SimpleResultMessageResponse, SimpleResultResponse, UrlResponse
|
||||
from controllers.common.schema import register_response_schema_models, register_schema_models
|
||||
from controllers.common.session import with_session
|
||||
from controllers.console import console_ns
|
||||
from controllers.console.wraps import RBACPermission, RBACResourceScope, rbac_permission_required
|
||||
from core.entities.knowledge_entities import IndexingEstimate
|
||||
@@ -36,15 +34,13 @@ from core.rag.entities import Rule
|
||||
from core.rag.extractor.entity.datasource_type import DatasourceType
|
||||
from core.rag.extractor.entity.extract_setting import ExtractSetting, NotionInfo, WebsiteInfo
|
||||
from core.rag.index_processor.constant.index_type import IndexTechniqueType
|
||||
from extensions.ext_database import db
|
||||
from fields.base import ResponseModel
|
||||
from fields.document_fields import (
|
||||
DocumentMetadataResponse,
|
||||
DocumentResponse,
|
||||
DocumentStatusListResponse,
|
||||
DocumentStatusResponse,
|
||||
DocumentWithSession,
|
||||
document_response,
|
||||
document_responses,
|
||||
normalize_enum,
|
||||
)
|
||||
from graphon.model_runtime.entities.model_entities import ModelType
|
||||
@@ -53,7 +49,7 @@ from libs.datetime_utils import naive_utc_now
|
||||
from libs.helper import dump_response, to_timestamp
|
||||
from libs.login import login_required
|
||||
from libs.pagination import paginate_query
|
||||
from models import Account, Document, DocumentSegment, UploadFile
|
||||
from models import Account, DatasetProcessRule, Document, DocumentSegment, UploadFile
|
||||
from models.dataset import DocumentPipelineExecutionLog
|
||||
from models.enums import IndexingStatus, ProcessRuleMode, SegmentStatus
|
||||
from services.dataset_ref_service import DatasetRefService
|
||||
@@ -114,24 +110,6 @@ class DocumentWithSegmentsResponse(DocumentResponse):
|
||||
total_segments: int | None = Field(default=None, exclude_if=lambda value: value is None)
|
||||
|
||||
|
||||
class DocumentWithSegmentsSession(DocumentWithSession):
|
||||
@property
|
||||
def process_rule_dict(self) -> Any:
|
||||
process_rule = self.document.get_dataset_process_rule(session=self.session)
|
||||
return process_rule.to_dict() if process_rule else None
|
||||
|
||||
|
||||
def document_with_segments_responses(
|
||||
documents: Sequence[Document], *, session: Session
|
||||
) -> list[DocumentWithSegmentsResponse]:
|
||||
return [
|
||||
DocumentWithSegmentsResponse.model_validate(
|
||||
DocumentWithSegmentsSession(document=document, session=session), from_attributes=True
|
||||
)
|
||||
for document in documents
|
||||
]
|
||||
|
||||
|
||||
class DatasetAndDocumentResponse(ResponseModel):
|
||||
dataset: DatasetResponse
|
||||
documents: list[DocumentResponse]
|
||||
@@ -291,18 +269,18 @@ register_response_schema_models(
|
||||
|
||||
class DocumentResource(Resource):
|
||||
def get_document(
|
||||
self, session: Session, dataset_id: str, document_id: str, current_user: Account, current_tenant_id: str
|
||||
self, dataset_id: str, document_id: str, current_user: Account, current_tenant_id: str
|
||||
) -> Document:
|
||||
dataset = DatasetService.get_dataset(dataset_id, session)
|
||||
dataset = DatasetService.get_dataset(dataset_id, db.session())
|
||||
if not dataset:
|
||||
raise NotFound("Dataset not found.")
|
||||
|
||||
try:
|
||||
DatasetService.check_dataset_permission(dataset, current_user, session)
|
||||
DatasetService.check_dataset_permission(dataset, current_user, db.session())
|
||||
except services.errors.account.NoPermissionError as e:
|
||||
raise Forbidden(str(e))
|
||||
|
||||
document = DocumentService.get_document(dataset_id, document_id, session=session)
|
||||
document = DocumentService.get_document(dataset_id, document_id, session=db.session())
|
||||
|
||||
if not document:
|
||||
raise NotFound("Document not found.")
|
||||
@@ -312,19 +290,17 @@ class DocumentResource(Resource):
|
||||
|
||||
return document
|
||||
|
||||
def get_batch_documents(
|
||||
self, session: Session, dataset_id: str, batch: str, current_user: Account
|
||||
) -> Sequence[Document]:
|
||||
dataset = DatasetService.get_dataset(dataset_id, session)
|
||||
def get_batch_documents(self, dataset_id: str, batch: str, current_user: Account) -> Sequence[Document]:
|
||||
dataset = DatasetService.get_dataset(dataset_id, db.session())
|
||||
if not dataset:
|
||||
raise NotFound("Dataset not found.")
|
||||
|
||||
try:
|
||||
DatasetService.check_dataset_permission(dataset, current_user, session)
|
||||
DatasetService.check_dataset_permission(dataset, current_user, db.session())
|
||||
except services.errors.account.NoPermissionError as e:
|
||||
raise Forbidden(str(e))
|
||||
|
||||
documents = DocumentService.get_batch_documents(dataset_id, batch, session)
|
||||
documents = DocumentService.get_batch_documents(dataset_id, batch, db.session())
|
||||
|
||||
if not documents:
|
||||
raise NotFound("Documents not found.")
|
||||
@@ -342,8 +318,7 @@ class GetProcessRuleApi(Resource):
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@with_current_user
|
||||
@with_session(write=False)
|
||||
def get(self, session: Session, current_user: Account):
|
||||
def get(self, current_user: Account):
|
||||
req_data = request.args
|
||||
|
||||
document_id = req_data.get("document_id")
|
||||
@@ -353,21 +328,26 @@ class GetProcessRuleApi(Resource):
|
||||
rules = DocumentService.DEFAULT_RULES["rules"]
|
||||
limits = DocumentService.DEFAULT_RULES["limits"]
|
||||
if document_id:
|
||||
document = DocumentService.get_document_by_id(document_id, session)
|
||||
if document is None:
|
||||
raise NotFound("Document not found.")
|
||||
# get the latest process rule
|
||||
document = db.get_or_404(Document, document_id)
|
||||
|
||||
dataset = DatasetService.get_dataset(document.dataset_id, session)
|
||||
dataset = DatasetService.get_dataset(document.dataset_id, db.session())
|
||||
|
||||
if not dataset:
|
||||
raise NotFound("Dataset not found.")
|
||||
|
||||
try:
|
||||
DatasetService.check_dataset_permission(dataset, current_user, session)
|
||||
DatasetService.check_dataset_permission(dataset, current_user, db.session())
|
||||
except services.errors.account.NoPermissionError as e:
|
||||
raise Forbidden(str(e))
|
||||
|
||||
dataset_process_rule = dataset.get_latest_process_rule(session=session)
|
||||
# get the latest process rule
|
||||
dataset_process_rule = db.session.scalar(
|
||||
select(DatasetProcessRule)
|
||||
.where(DatasetProcessRule.dataset_id == document.dataset_id)
|
||||
.order_by(DatasetProcessRule.created_at.desc())
|
||||
.limit(1)
|
||||
)
|
||||
if dataset_process_rule:
|
||||
mode = dataset_process_rule.mode
|
||||
rules = dataset_process_rule.rules_dict
|
||||
@@ -401,8 +381,7 @@ class DatasetDocumentListApi(Resource):
|
||||
@with_current_user
|
||||
@with_current_tenant_id
|
||||
@rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_CREATE_AND_MANAGEMENT)
|
||||
@with_session(write=False)
|
||||
def get(self, session: Session, current_tenant_id: str, current_user: Account, dataset_id: UUID):
|
||||
def get(self, current_tenant_id: str, current_user: Account, dataset_id: UUID):
|
||||
dataset_id_str = str(dataset_id)
|
||||
raw_args = request.args.to_dict()
|
||||
param = DocumentDatasetListParam.model_validate(raw_args)
|
||||
@@ -428,12 +407,12 @@ class DatasetDocumentListApi(Resource):
|
||||
)
|
||||
except (ArgumentTypeError, ValueError, Exception):
|
||||
fetch = False
|
||||
dataset = DatasetService.get_dataset(dataset_id_str, session)
|
||||
dataset = DatasetService.get_dataset(dataset_id_str, db.session())
|
||||
if not dataset:
|
||||
raise NotFound("Dataset not found.")
|
||||
|
||||
try:
|
||||
DatasetService.check_dataset_permission(dataset, current_user, session)
|
||||
DatasetService.check_dataset_permission(dataset, current_user, db.session())
|
||||
except services.errors.account.NoPermissionError as e:
|
||||
raise Forbidden(str(e))
|
||||
|
||||
@@ -478,20 +457,20 @@ class DatasetDocumentListApi(Resource):
|
||||
desc(Document.position),
|
||||
)
|
||||
|
||||
paginated_documents = paginate_query(query, session=session, page=page, per_page=limit, max_per_page=100)
|
||||
paginated_documents = paginate_query(query, page=page, per_page=limit, max_per_page=100)
|
||||
documents = paginated_documents.items
|
||||
|
||||
DocumentService.enrich_documents_with_summary_index_status(
|
||||
documents=documents,
|
||||
dataset=dataset,
|
||||
tenant_id=current_tenant_id,
|
||||
session=session,
|
||||
session=db.session(),
|
||||
)
|
||||
|
||||
if fetch:
|
||||
for document in documents:
|
||||
completed_segments = (
|
||||
session.scalar(
|
||||
db.session.scalar(
|
||||
select(func.count(DocumentSegment.id)).where(
|
||||
DocumentSegment.completed_at.isnot(None),
|
||||
DocumentSegment.document_id == str(document.id),
|
||||
@@ -501,7 +480,7 @@ class DatasetDocumentListApi(Resource):
|
||||
or 0
|
||||
)
|
||||
total_segments = (
|
||||
session.scalar(
|
||||
db.session.scalar(
|
||||
select(func.count(DocumentSegment.id)).where(
|
||||
DocumentSegment.document_id == str(document.id),
|
||||
DocumentSegment.status != SegmentStatus.RE_SEGMENT,
|
||||
@@ -512,7 +491,7 @@ class DatasetDocumentListApi(Resource):
|
||||
document.completed_segments = completed_segments
|
||||
document.total_segments = total_segments
|
||||
response = {
|
||||
"data": document_with_segments_responses(documents, session=session),
|
||||
"data": documents,
|
||||
"has_more": len(documents) == limit,
|
||||
"limit": limit,
|
||||
"total": paginated_documents.total,
|
||||
@@ -530,11 +509,10 @@ class DatasetDocumentListApi(Resource):
|
||||
@console_ns.response(200, "Documents created successfully", console_ns.models[DatasetAndDocumentResponse.__name__])
|
||||
@with_current_user
|
||||
@rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_EDIT)
|
||||
@with_session
|
||||
def post(self, session: Session, current_user: Account, dataset_id: UUID):
|
||||
def post(self, current_user: Account, dataset_id: UUID):
|
||||
dataset_id_str = str(dataset_id)
|
||||
|
||||
dataset = DatasetService.get_dataset(dataset_id_str, session)
|
||||
dataset = DatasetService.get_dataset(dataset_id_str, db.session())
|
||||
|
||||
if not dataset:
|
||||
raise NotFound("Dataset not found.")
|
||||
@@ -544,7 +522,7 @@ class DatasetDocumentListApi(Resource):
|
||||
raise Forbidden()
|
||||
|
||||
try:
|
||||
DatasetService.check_dataset_permission(dataset, current_user, session)
|
||||
DatasetService.check_dataset_permission(dataset, current_user, db.session())
|
||||
except services.errors.account.NoPermissionError as e:
|
||||
raise Forbidden(str(e))
|
||||
|
||||
@@ -558,9 +536,9 @@ class DatasetDocumentListApi(Resource):
|
||||
|
||||
try:
|
||||
documents, batch = DocumentService.save_document_with_dataset_id(
|
||||
dataset, knowledge_config, current_user, session=session
|
||||
dataset, knowledge_config, current_user, session=db.session()
|
||||
)
|
||||
dataset = DatasetService.get_dataset(dataset_id_str, session)
|
||||
dataset = DatasetService.get_dataset(dataset_id_str, db.session())
|
||||
|
||||
except ProviderTokenNotInitError as ex:
|
||||
raise ProviderNotInitializeError(ex.description)
|
||||
@@ -569,10 +547,7 @@ class DatasetDocumentListApi(Resource):
|
||||
except ModelCurrentlyNotSupportError:
|
||||
raise ProviderModelCurrentlyNotSupportError()
|
||||
|
||||
return dump_response(
|
||||
DatasetAndDocumentResponse,
|
||||
{"dataset": dataset, "documents": document_responses(documents, session=session), "batch": batch},
|
||||
)
|
||||
return dump_response(DatasetAndDocumentResponse, {"dataset": dataset, "documents": documents, "batch": batch})
|
||||
|
||||
@setup_required
|
||||
@login_required
|
||||
@@ -580,10 +555,9 @@ class DatasetDocumentListApi(Resource):
|
||||
@cloud_edition_billing_rate_limit_check("knowledge")
|
||||
@console_ns.response(204, "Documents deleted successfully")
|
||||
@rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_EDIT)
|
||||
@with_session
|
||||
def delete(self, session: Session, dataset_id: UUID):
|
||||
def delete(self, dataset_id: UUID):
|
||||
dataset_id_str = str(dataset_id)
|
||||
dataset = DatasetService.get_dataset(dataset_id_str, session)
|
||||
dataset = DatasetService.get_dataset(dataset_id_str, db.session())
|
||||
if dataset is None:
|
||||
raise NotFound("Dataset not found.")
|
||||
# check user's model setting
|
||||
@@ -592,7 +566,7 @@ class DatasetDocumentListApi(Resource):
|
||||
try:
|
||||
document_ids = request.args.getlist("document_id")
|
||||
dataset_ref = DatasetRefService.create_dataset_ref(dataset)
|
||||
DocumentService.delete_documents(dataset_ref, document_ids, dataset.get_doc_form(session=session), session)
|
||||
DocumentService.delete_documents(dataset_ref, document_ids, dataset.doc_form, db.session())
|
||||
except services.errors.document.DocumentIndexingError:
|
||||
raise DocumentIndexingError("Cannot delete document during indexing.")
|
||||
|
||||
@@ -615,8 +589,7 @@ class DatasetInitApi(Resource):
|
||||
@cloud_edition_billing_rate_limit_check("knowledge")
|
||||
@with_current_user
|
||||
@with_current_tenant_id
|
||||
@with_session
|
||||
def post(self, session: Session, current_tenant_id: str, current_user: Account):
|
||||
def post(self, current_tenant_id: str, current_user: Account):
|
||||
# The role of the current user in the ta table must be admin, owner, dataset_operator, or editor
|
||||
if not current_user.is_dataset_editor:
|
||||
raise Forbidden()
|
||||
@@ -652,7 +625,7 @@ class DatasetInitApi(Resource):
|
||||
tenant_id=current_tenant_id,
|
||||
knowledge_config=knowledge_config,
|
||||
account=current_user,
|
||||
session=session,
|
||||
session=db.session(),
|
||||
)
|
||||
except ProviderTokenNotInitError as ex:
|
||||
raise ProviderNotInitializeError(ex.description)
|
||||
@@ -661,10 +634,7 @@ class DatasetInitApi(Resource):
|
||||
except ModelCurrentlyNotSupportError:
|
||||
raise ProviderModelCurrentlyNotSupportError()
|
||||
|
||||
return dump_response(
|
||||
DatasetAndDocumentResponse,
|
||||
{"dataset": dataset, "documents": document_responses(documents, session=session), "batch": batch},
|
||||
)
|
||||
return dump_response(DatasetAndDocumentResponse, {"dataset": dataset, "documents": documents, "batch": batch})
|
||||
|
||||
|
||||
@console_ns.route("/datasets/<uuid:dataset_id>/documents/<uuid:document_id>/indexing-estimate")
|
||||
@@ -685,24 +655,23 @@ class DocumentIndexingEstimateApi(DocumentResource):
|
||||
@with_current_user
|
||||
@with_current_tenant_id
|
||||
@rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_CREATE_AND_MANAGEMENT)
|
||||
@with_session
|
||||
def get(self, session: Session, current_tenant_id: str, current_user: Account, dataset_id: UUID, document_id: UUID):
|
||||
def get(self, current_tenant_id: str, current_user: Account, dataset_id: UUID, document_id: UUID):
|
||||
dataset_id_str = str(dataset_id)
|
||||
document_id_str = str(document_id)
|
||||
document = self.get_document(session, dataset_id_str, document_id_str, current_user, current_tenant_id)
|
||||
document = self.get_document(dataset_id_str, document_id_str, current_user, current_tenant_id)
|
||||
|
||||
if document.indexing_status in {IndexingStatus.COMPLETED, IndexingStatus.ERROR}:
|
||||
raise DocumentAlreadyFinishedError()
|
||||
|
||||
data_process_rule = document.get_dataset_process_rule(session=session)
|
||||
data_process_rule_dict: Mapping[str, Any] = data_process_rule.to_dict() if data_process_rule else {}
|
||||
data_process_rule = document.dataset_process_rule
|
||||
data_process_rule_dict = data_process_rule.to_dict() if data_process_rule else {}
|
||||
|
||||
if document.data_source_type == "upload_file":
|
||||
data_source_info = document.data_source_info_dict
|
||||
if data_source_info and "upload_file_id" in data_source_info:
|
||||
file_id = data_source_info["upload_file_id"]
|
||||
|
||||
file = session.scalar(
|
||||
file = db.session.scalar(
|
||||
select(UploadFile)
|
||||
.where(UploadFile.tenant_id == document.tenant_id, UploadFile.id == file_id)
|
||||
.limit(1)
|
||||
@@ -720,13 +689,12 @@ class DocumentIndexingEstimateApi(DocumentResource):
|
||||
|
||||
try:
|
||||
estimate_response = indexing_runner.indexing_estimate(
|
||||
tenant_id=current_tenant_id,
|
||||
extract_settings=[extract_setting],
|
||||
tmp_processing_rule=data_process_rule_dict,
|
||||
doc_form=document.doc_form,
|
||||
doc_language="English",
|
||||
dataset_id=dataset_id_str,
|
||||
session=session,
|
||||
current_tenant_id,
|
||||
[extract_setting],
|
||||
data_process_rule_dict,
|
||||
document.doc_form,
|
||||
"English",
|
||||
dataset_id_str,
|
||||
)
|
||||
return (
|
||||
# TODO: why using zero here? the same for the below endpoint
|
||||
@@ -777,10 +745,9 @@ class DocumentBatchIndexingEstimateApi(DocumentResource):
|
||||
@with_current_user
|
||||
@with_current_tenant_id
|
||||
@rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_CREATE_AND_MANAGEMENT)
|
||||
@with_session
|
||||
def get(self, session: Session, current_tenant_id: str, current_user: Account, dataset_id: UUID, batch: str):
|
||||
def get(self, current_tenant_id: str, current_user: Account, dataset_id: UUID, batch: str):
|
||||
dataset_id_str = str(dataset_id)
|
||||
documents = self.get_batch_documents(session, dataset_id_str, batch, current_user)
|
||||
documents = self.get_batch_documents(dataset_id_str, batch, current_user)
|
||||
if not documents:
|
||||
return (
|
||||
IndexingEstimateResponse(
|
||||
@@ -792,8 +759,8 @@ class DocumentBatchIndexingEstimateApi(DocumentResource):
|
||||
).model_dump(mode="json", exclude_none=True),
|
||||
200,
|
||||
)
|
||||
data_process_rule = documents[0].get_dataset_process_rule(session=session)
|
||||
data_process_rule_dict: Mapping[str, Any] = data_process_rule.to_dict() if data_process_rule else {}
|
||||
data_process_rule = documents[0].dataset_process_rule
|
||||
data_process_rule_dict = data_process_rule.to_dict() if data_process_rule else {}
|
||||
extract_settings = []
|
||||
for document in documents:
|
||||
if document.indexing_status in {IndexingStatus.COMPLETED, IndexingStatus.ERROR}:
|
||||
@@ -804,7 +771,7 @@ class DocumentBatchIndexingEstimateApi(DocumentResource):
|
||||
if not data_source_info:
|
||||
continue
|
||||
file_id = data_source_info["upload_file_id"]
|
||||
file_detail = session.scalar(
|
||||
file_detail = db.session.scalar(
|
||||
select(UploadFile)
|
||||
.where(UploadFile.tenant_id == current_tenant_id, UploadFile.id == file_id)
|
||||
.limit(1)
|
||||
@@ -858,13 +825,12 @@ class DocumentBatchIndexingEstimateApi(DocumentResource):
|
||||
indexing_runner = IndexingRunner()
|
||||
try:
|
||||
response = indexing_runner.indexing_estimate(
|
||||
tenant_id=current_tenant_id,
|
||||
extract_settings=extract_settings,
|
||||
tmp_processing_rule=data_process_rule_dict,
|
||||
doc_form=document.doc_form,
|
||||
doc_language="English",
|
||||
dataset_id=dataset_id_str,
|
||||
session=session,
|
||||
current_tenant_id,
|
||||
extract_settings,
|
||||
data_process_rule_dict,
|
||||
document.doc_form,
|
||||
"English",
|
||||
dataset_id_str,
|
||||
)
|
||||
return (
|
||||
IndexingEstimateResponse(
|
||||
@@ -899,14 +865,13 @@ class DocumentBatchIndexingStatusApi(DocumentResource):
|
||||
@account_initialization_required
|
||||
@with_current_user
|
||||
@rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_CREATE_AND_MANAGEMENT)
|
||||
@with_session(write=False)
|
||||
def get(self, session: Session, current_user: Account, dataset_id: UUID, batch: str):
|
||||
def get(self, current_user: Account, dataset_id: UUID, batch: str):
|
||||
dataset_id_str = str(dataset_id)
|
||||
documents = self.get_batch_documents(session, dataset_id_str, batch, current_user)
|
||||
documents = self.get_batch_documents(dataset_id_str, batch, current_user)
|
||||
documents_status = []
|
||||
for document in documents:
|
||||
completed_segments = (
|
||||
session.scalar(
|
||||
db.session.scalar(
|
||||
select(func.count(DocumentSegment.id)).where(
|
||||
DocumentSegment.completed_at.isnot(None),
|
||||
DocumentSegment.document_id == str(document.id),
|
||||
@@ -916,7 +881,7 @@ class DocumentBatchIndexingStatusApi(DocumentResource):
|
||||
or 0
|
||||
)
|
||||
total_segments = (
|
||||
session.scalar(
|
||||
db.session.scalar(
|
||||
select(func.count(DocumentSegment.id)).where(
|
||||
DocumentSegment.document_id == str(document.id),
|
||||
DocumentSegment.status != SegmentStatus.RE_SEGMENT,
|
||||
@@ -958,14 +923,13 @@ class DocumentIndexingStatusApi(DocumentResource):
|
||||
@with_current_user
|
||||
@with_current_tenant_id
|
||||
@rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_CREATE_AND_MANAGEMENT)
|
||||
@with_session(write=False)
|
||||
def get(self, session: Session, current_tenant_id: str, current_user: Account, dataset_id: UUID, document_id: UUID):
|
||||
def get(self, current_tenant_id: str, current_user: Account, dataset_id: UUID, document_id: UUID):
|
||||
dataset_id_str = str(dataset_id)
|
||||
document_id_str = str(document_id)
|
||||
document = self.get_document(session, dataset_id_str, document_id_str, current_user, current_tenant_id)
|
||||
document = self.get_document(dataset_id_str, document_id_str, current_user, current_tenant_id)
|
||||
|
||||
completed_segments = (
|
||||
session.scalar(
|
||||
db.session.scalar(
|
||||
select(func.count(DocumentSegment.id)).where(
|
||||
DocumentSegment.completed_at.isnot(None),
|
||||
DocumentSegment.document_id == document_id_str,
|
||||
@@ -975,7 +939,7 @@ class DocumentIndexingStatusApi(DocumentResource):
|
||||
or 0
|
||||
)
|
||||
total_segments = (
|
||||
session.scalar(
|
||||
db.session.scalar(
|
||||
select(func.count(DocumentSegment.id)).where(
|
||||
DocumentSegment.document_id == document_id_str,
|
||||
DocumentSegment.status != SegmentStatus.RE_SEGMENT,
|
||||
@@ -1023,11 +987,10 @@ class DocumentApi(DocumentResource):
|
||||
@with_current_user
|
||||
@with_current_tenant_id
|
||||
@rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_CREATE_AND_MANAGEMENT)
|
||||
@with_session(write=False)
|
||||
def get(self, session: Session, current_tenant_id: str, current_user: Account, dataset_id: UUID, document_id: UUID):
|
||||
def get(self, current_tenant_id: str, current_user: Account, dataset_id: UUID, document_id: UUID):
|
||||
dataset_id_str = str(dataset_id)
|
||||
document_id_str = str(document_id)
|
||||
document = self.get_document(session, dataset_id_str, document_id_str, current_user, current_tenant_id)
|
||||
document = self.get_document(dataset_id_str, document_id_str, current_user, current_tenant_id)
|
||||
|
||||
metadata = request.args.get("metadata", "all")
|
||||
if metadata not in self.METADATA_CHOICES:
|
||||
@@ -1039,22 +1002,20 @@ class DocumentApi(DocumentResource):
|
||||
{
|
||||
"id": document.id,
|
||||
"doc_type": document.doc_type,
|
||||
"doc_metadata": document.get_doc_metadata_details(session=session),
|
||||
"doc_metadata": document.doc_metadata_details,
|
||||
}
|
||||
)
|
||||
return response.model_dump(mode="json", include={"id", *metadata_fields}, exclude_unset=True), 200
|
||||
|
||||
dataset_process_rules = DatasetService.get_process_rules(dataset_id_str, session)
|
||||
document_process_rule = document.get_dataset_process_rule(session=session)
|
||||
document_process_rules: Mapping[str, Any] = document_process_rule.to_dict() if document_process_rule else {}
|
||||
segment_count = document.get_segment_count(session=session)
|
||||
dataset_process_rules = DatasetService.get_process_rules(dataset_id_str, db.session())
|
||||
document_process_rules = document.dataset_process_rule.to_dict() if document.dataset_process_rule else {}
|
||||
response = DocumentDetailResponse.model_validate(
|
||||
{
|
||||
"id": document.id,
|
||||
"position": document.position,
|
||||
"data_source_type": document.data_source_type,
|
||||
"data_source_info": document.data_source_info_dict,
|
||||
"data_source_detail_dict": document.get_data_source_detail_dict(session=session),
|
||||
"data_source_detail_dict": document.data_source_detail_dict,
|
||||
"dataset_process_rule_id": document.dataset_process_rule_id,
|
||||
"dataset_process_rule": dataset_process_rules,
|
||||
"document_process_rule": document_process_rules,
|
||||
@@ -1073,10 +1034,10 @@ class DocumentApi(DocumentResource):
|
||||
"disabled_by": document.disabled_by,
|
||||
"archived": document.archived,
|
||||
"doc_type": document.doc_type,
|
||||
"doc_metadata": document.get_doc_metadata_details(session=session),
|
||||
"segment_count": segment_count,
|
||||
"average_segment_length": (document.word_count or 0) // segment_count if segment_count else 0,
|
||||
"hit_count": document.get_hit_count(session=session),
|
||||
"doc_metadata": document.doc_metadata_details,
|
||||
"segment_count": document.segment_count,
|
||||
"average_segment_length": document.average_segment_length,
|
||||
"hit_count": document.hit_count,
|
||||
"display_status": document.display_status,
|
||||
"doc_form": document.doc_form,
|
||||
"doc_language": document.doc_language,
|
||||
@@ -1094,22 +1055,19 @@ class DocumentApi(DocumentResource):
|
||||
@with_current_user
|
||||
@with_current_tenant_id
|
||||
@rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_EDIT)
|
||||
@with_session
|
||||
def delete(
|
||||
self, session: Session, current_tenant_id: str, current_user: Account, dataset_id: UUID, document_id: UUID
|
||||
):
|
||||
def delete(self, current_tenant_id: str, current_user: Account, dataset_id: UUID, document_id: UUID):
|
||||
dataset_id_str = str(dataset_id)
|
||||
document_id_str = str(document_id)
|
||||
dataset = DatasetService.get_dataset(dataset_id_str, session)
|
||||
dataset = DatasetService.get_dataset(dataset_id_str, db.session())
|
||||
if dataset is None:
|
||||
raise NotFound("Dataset not found.")
|
||||
# check user's model setting
|
||||
DatasetService.check_dataset_model_setting(dataset)
|
||||
|
||||
document = self.get_document(session, dataset_id_str, document_id_str, current_user, current_tenant_id)
|
||||
document = self.get_document(dataset_id_str, document_id_str, current_user, current_tenant_id)
|
||||
|
||||
try:
|
||||
DocumentService.delete_document(document, session)
|
||||
DocumentService.delete_document(document, db.session())
|
||||
except services.errors.document.DocumentIndexingError:
|
||||
raise DocumentIndexingError("Cannot delete document during indexing.")
|
||||
|
||||
@@ -1130,13 +1088,12 @@ class DocumentDownloadApi(DocumentResource):
|
||||
@with_current_user
|
||||
@with_current_tenant_id
|
||||
@rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_DOCUMENT_DOWNLOAD)
|
||||
@with_session(write=False)
|
||||
def get(
|
||||
self, session: Session, current_tenant_id: str, current_user: Account, dataset_id: UUID, document_id: UUID
|
||||
) -> dict[str, Any]:
|
||||
def get(self, current_tenant_id: str, current_user: Account, dataset_id: UUID, document_id: UUID) -> dict[str, Any]:
|
||||
# Reuse the shared permission/tenant checks implemented in DocumentResource.
|
||||
document = self.get_document(session, str(dataset_id), str(document_id), current_user, current_tenant_id)
|
||||
return UrlResponse(url=DocumentService.get_document_download_url(document, session)).model_dump(mode="json")
|
||||
document = self.get_document(str(dataset_id), str(document_id), current_user, current_tenant_id)
|
||||
return UrlResponse(url=DocumentService.get_document_download_url(document, db.session())).model_dump(
|
||||
mode="json"
|
||||
)
|
||||
|
||||
|
||||
@console_ns.route("/datasets/<uuid:dataset_id>/documents/download-zip")
|
||||
@@ -1154,8 +1111,7 @@ class DocumentBatchDownloadZipApi(DocumentResource):
|
||||
@with_current_user
|
||||
@with_current_tenant_id
|
||||
@rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_EDIT)
|
||||
@with_session(write=False)
|
||||
def post(self, session: Session, current_tenant_id: str, current_user: Account, dataset_id: UUID):
|
||||
def post(self, current_tenant_id: str, current_user: Account, dataset_id: UUID):
|
||||
"""Stream a ZIP archive containing the requested uploaded documents."""
|
||||
# Parse and validate request payload.
|
||||
payload = DocumentBatchDownloadZipPayload.model_validate(console_ns.payload or {})
|
||||
@@ -1167,7 +1123,7 @@ class DocumentBatchDownloadZipApi(DocumentResource):
|
||||
document_ids=document_ids,
|
||||
tenant_id=current_tenant_id,
|
||||
current_user=current_user,
|
||||
session=session,
|
||||
session=db.session(),
|
||||
)
|
||||
|
||||
# Delegate ZIP packing to FileService, but keep Flask response+cleanup in the route.
|
||||
@@ -1206,10 +1162,8 @@ class DocumentProcessingApi(DocumentResource):
|
||||
@with_current_user
|
||||
@with_current_tenant_id
|
||||
@rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_EDIT)
|
||||
@with_session
|
||||
def patch(
|
||||
self,
|
||||
session: Session,
|
||||
current_tenant_id: str,
|
||||
current_user: Account,
|
||||
dataset_id: UUID,
|
||||
@@ -1218,7 +1172,7 @@ class DocumentProcessingApi(DocumentResource):
|
||||
):
|
||||
dataset_id_str = str(dataset_id)
|
||||
document_id_str = str(document_id)
|
||||
document = self.get_document(session, dataset_id_str, document_id_str, current_user, current_tenant_id)
|
||||
document = self.get_document(dataset_id_str, document_id_str, current_user, current_tenant_id)
|
||||
|
||||
# The role of the current user in the ta table must be admin, owner, dataset_operator, or editor
|
||||
if not current_user.is_dataset_editor:
|
||||
@@ -1232,6 +1186,7 @@ class DocumentProcessingApi(DocumentResource):
|
||||
document.paused_by = current_user.id
|
||||
document.paused_at = naive_utc_now()
|
||||
document.is_paused = True
|
||||
db.session.commit()
|
||||
|
||||
case "resume":
|
||||
if document.indexing_status not in {IndexingStatus.PAUSED, IndexingStatus.ERROR}:
|
||||
@@ -1240,6 +1195,7 @@ class DocumentProcessingApi(DocumentResource):
|
||||
document.paused_by = None
|
||||
document.paused_at = None
|
||||
document.is_paused = False
|
||||
db.session.commit()
|
||||
|
||||
return SimpleResultResponse(result="success").model_dump(mode="json"), 200
|
||||
|
||||
@@ -1263,11 +1219,10 @@ class DocumentMetadataApi(DocumentResource):
|
||||
@with_current_user
|
||||
@with_current_tenant_id
|
||||
@rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_EDIT)
|
||||
@with_session
|
||||
def put(self, session: Session, current_tenant_id: str, current_user: Account, dataset_id: UUID, document_id: UUID):
|
||||
def put(self, current_tenant_id: str, current_user: Account, dataset_id: UUID, document_id: UUID):
|
||||
dataset_id_str = str(dataset_id)
|
||||
document_id_str = str(document_id)
|
||||
document = self.get_document(session, dataset_id_str, document_id_str, current_user, current_tenant_id)
|
||||
document = self.get_document(dataset_id_str, document_id_str, current_user, current_tenant_id)
|
||||
|
||||
req_data = DocumentMetadataUpdatePayload.model_validate(request.get_json() or {})
|
||||
|
||||
@@ -1299,6 +1254,7 @@ class DocumentMetadataApi(DocumentResource):
|
||||
|
||||
document.doc_type = doc_type
|
||||
document.updated_at = naive_utc_now()
|
||||
db.session.commit()
|
||||
|
||||
return SimpleResultMessageResponse(result="success", message="Document metadata updated.").model_dump(
|
||||
mode="json"
|
||||
@@ -1315,16 +1271,11 @@ class DocumentStatusApi(DocumentResource):
|
||||
@console_ns.response(200, "Success", console_ns.models[SimpleResultResponse.__name__])
|
||||
@with_current_user
|
||||
@rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_EDIT)
|
||||
@with_session
|
||||
def patch(
|
||||
self,
|
||||
session: Session,
|
||||
current_user: Account,
|
||||
dataset_id: UUID,
|
||||
action: Literal["enable", "disable", "archive", "un_archive"],
|
||||
self, current_user: Account, dataset_id: UUID, action: Literal["enable", "disable", "archive", "un_archive"]
|
||||
):
|
||||
dataset_id_str = str(dataset_id)
|
||||
dataset = DatasetService.get_dataset(dataset_id_str, session)
|
||||
dataset = DatasetService.get_dataset(dataset_id_str, db.session())
|
||||
if dataset is None:
|
||||
raise NotFound("Dataset not found.")
|
||||
|
||||
@@ -1336,12 +1287,12 @@ class DocumentStatusApi(DocumentResource):
|
||||
DatasetService.check_dataset_model_setting(dataset)
|
||||
|
||||
# check user's permission
|
||||
DatasetService.check_dataset_permission(dataset, current_user, session)
|
||||
DatasetService.check_dataset_permission(dataset, current_user, db.session())
|
||||
|
||||
document_ids = request.args.getlist("document_id")
|
||||
|
||||
try:
|
||||
DocumentService.batch_update_document_status(dataset, document_ids, action, current_user, session)
|
||||
DocumentService.batch_update_document_status(dataset, document_ids, action, current_user, db.session())
|
||||
except services.errors.document.DocumentIndexingError as e:
|
||||
raise InvalidActionError(str(e))
|
||||
except ValueError as e:
|
||||
@@ -1360,17 +1311,16 @@ class DocumentPauseApi(DocumentResource):
|
||||
@cloud_edition_billing_rate_limit_check("knowledge")
|
||||
@console_ns.response(204, "Document paused successfully")
|
||||
@rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_EDIT)
|
||||
@with_session
|
||||
def patch(self, session: Session, dataset_id: UUID, document_id: UUID):
|
||||
def patch(self, dataset_id: UUID, document_id: UUID):
|
||||
"""pause document."""
|
||||
dataset_id_str = str(dataset_id)
|
||||
document_id_str = str(document_id)
|
||||
|
||||
dataset = DatasetService.get_dataset(dataset_id_str, session)
|
||||
dataset = DatasetService.get_dataset(dataset_id_str, db.session())
|
||||
if not dataset:
|
||||
raise NotFound("Dataset not found.")
|
||||
|
||||
document = DocumentService.get_document(dataset.id, document_id_str, session=session)
|
||||
document = DocumentService.get_document(dataset.id, document_id_str, session=db.session())
|
||||
|
||||
# 404 if document not found
|
||||
if document is None:
|
||||
@@ -1382,7 +1332,7 @@ class DocumentPauseApi(DocumentResource):
|
||||
|
||||
try:
|
||||
# pause document
|
||||
DocumentService.pause_document(document, session)
|
||||
DocumentService.pause_document(document, db.session())
|
||||
except services.errors.document.DocumentIndexingError:
|
||||
raise DocumentIndexingError("Cannot pause completed document.")
|
||||
|
||||
@@ -1397,15 +1347,14 @@ class DocumentRecoverApi(DocumentResource):
|
||||
@cloud_edition_billing_rate_limit_check("knowledge")
|
||||
@console_ns.response(204, "Document resumed successfully")
|
||||
@rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_EDIT)
|
||||
@with_session
|
||||
def patch(self, session: Session, dataset_id: UUID, document_id: UUID):
|
||||
def patch(self, dataset_id: UUID, document_id: UUID):
|
||||
"""recover document."""
|
||||
dataset_id_str = str(dataset_id)
|
||||
document_id_str = str(document_id)
|
||||
dataset = DatasetService.get_dataset(dataset_id_str, session)
|
||||
dataset = DatasetService.get_dataset(dataset_id_str, db.session())
|
||||
if not dataset:
|
||||
raise NotFound("Dataset not found.")
|
||||
document = DocumentService.get_document(dataset.id, document_id_str, session=session)
|
||||
document = DocumentService.get_document(dataset.id, document_id_str, session=db.session())
|
||||
|
||||
# 404 if document not found
|
||||
if document is None:
|
||||
@@ -1416,7 +1365,7 @@ class DocumentRecoverApi(DocumentResource):
|
||||
raise ArchivedDocumentImmutableError()
|
||||
try:
|
||||
# pause document
|
||||
DocumentService.recover_document(document, session)
|
||||
DocumentService.recover_document(document, db.session())
|
||||
except services.errors.document.DocumentIndexingError:
|
||||
raise DocumentIndexingError("Document is not in paused status.")
|
||||
|
||||
@@ -1432,18 +1381,17 @@ class DocumentRetryApi(DocumentResource):
|
||||
@console_ns.expect(console_ns.models[DocumentRetryPayload.__name__])
|
||||
@console_ns.response(204, "Documents retry started successfully")
|
||||
@rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_EDIT)
|
||||
@with_session
|
||||
def post(self, session: Session, dataset_id: UUID):
|
||||
def post(self, dataset_id: UUID):
|
||||
"""retry document."""
|
||||
payload = DocumentRetryPayload.model_validate(console_ns.payload or {})
|
||||
dataset_id_str = str(dataset_id)
|
||||
dataset = DatasetService.get_dataset(dataset_id_str, session)
|
||||
dataset = DatasetService.get_dataset(dataset_id_str, db.session())
|
||||
retry_documents = []
|
||||
if not dataset:
|
||||
raise NotFound("Dataset not found.")
|
||||
for document_id in payload.document_ids:
|
||||
try:
|
||||
document = DocumentService.get_document(dataset.id, document_id, session=session)
|
||||
document = DocumentService.get_document(dataset.id, document_id, session=db.session())
|
||||
|
||||
# 404 if document not found
|
||||
if document is None:
|
||||
@@ -1461,7 +1409,7 @@ class DocumentRetryApi(DocumentResource):
|
||||
logger.exception("Failed to retry document, document id: %s", document_id)
|
||||
continue
|
||||
# retry document
|
||||
DocumentService.retry_document(dataset_id_str, retry_documents, session)
|
||||
DocumentService.retry_document(dataset_id_str, retry_documents, db.session())
|
||||
|
||||
return "", 204
|
||||
|
||||
@@ -1475,23 +1423,22 @@ class DocumentRenameApi(DocumentResource):
|
||||
@console_ns.expect(console_ns.models[DocumentRenamePayload.__name__])
|
||||
@with_current_user
|
||||
@rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_EDIT)
|
||||
@with_session
|
||||
def post(self, session: Session, current_user: Account, dataset_id: UUID, document_id: UUID):
|
||||
def post(self, current_user: Account, dataset_id: UUID, document_id: UUID):
|
||||
# The role of the current user in the ta table must be admin, owner, editor, or dataset_operator
|
||||
if not current_user.is_dataset_editor:
|
||||
raise Forbidden()
|
||||
dataset = DatasetService.get_dataset(dataset_id, session)
|
||||
dataset = DatasetService.get_dataset(str(dataset_id), db.session())
|
||||
if not dataset:
|
||||
raise NotFound("Dataset not found.")
|
||||
DatasetService.check_dataset_operator_permission(current_user, dataset, session=session)
|
||||
DatasetService.check_dataset_operator_permission(current_user, dataset, session=db.session())
|
||||
payload = DocumentRenamePayload.model_validate(console_ns.payload or {})
|
||||
|
||||
try:
|
||||
document = DocumentService.rename_document(str(dataset_id), str(document_id), payload.name, session)
|
||||
document = DocumentService.rename_document(str(dataset_id), str(document_id), payload.name, db.session())
|
||||
except services.errors.document.DocumentIndexingError:
|
||||
raise DocumentIndexingError("Cannot delete document during indexing.")
|
||||
|
||||
return dump_response(DocumentResponse, document_response(document, session=session))
|
||||
return dump_response(DocumentResponse, document)
|
||||
|
||||
|
||||
@console_ns.route("/datasets/<uuid:dataset_id>/documents/<uuid:document_id>/website-sync")
|
||||
@@ -1502,15 +1449,14 @@ class WebsiteDocumentSyncApi(DocumentResource):
|
||||
@console_ns.response(200, "Success", console_ns.models[SimpleResultResponse.__name__])
|
||||
@with_current_tenant_id
|
||||
@rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_CREATE_AND_MANAGEMENT)
|
||||
@with_session
|
||||
def get(self, session: Session, current_tenant_id: str, dataset_id: UUID, document_id: UUID):
|
||||
def get(self, current_tenant_id: str, dataset_id: UUID, document_id: UUID):
|
||||
"""sync website document."""
|
||||
dataset_id_str = str(dataset_id)
|
||||
dataset = DatasetService.get_dataset(dataset_id_str, session)
|
||||
dataset = DatasetService.get_dataset(dataset_id_str, db.session())
|
||||
if not dataset:
|
||||
raise NotFound("Dataset not found.")
|
||||
document_id_str = str(document_id)
|
||||
document = DocumentService.get_document(dataset.id, document_id_str, session=session)
|
||||
document = DocumentService.get_document(dataset.id, document_id_str, session=db.session())
|
||||
if not document:
|
||||
raise NotFound("Document not found.")
|
||||
if document.tenant_id != current_tenant_id:
|
||||
@@ -1521,7 +1467,7 @@ class WebsiteDocumentSyncApi(DocumentResource):
|
||||
if DocumentService.check_archived(document):
|
||||
raise ArchivedDocumentImmutableError()
|
||||
# sync document
|
||||
DocumentService.sync_website_document(dataset_id_str, document, session)
|
||||
DocumentService.sync_website_document(dataset_id_str, document, db.session())
|
||||
|
||||
return SimpleResultResponse(result="success").model_dump(mode="json"), 200
|
||||
|
||||
@@ -1537,18 +1483,17 @@ class DocumentPipelineExecutionLogApi(DocumentResource):
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_CREATE_AND_MANAGEMENT)
|
||||
@with_session(write=False)
|
||||
def get(self, session: Session, dataset_id: UUID, document_id: UUID):
|
||||
def get(self, dataset_id: UUID, document_id: UUID):
|
||||
dataset_id_str = str(dataset_id)
|
||||
document_id_str = str(document_id)
|
||||
|
||||
dataset = DatasetService.get_dataset(dataset_id_str, session)
|
||||
dataset = DatasetService.get_dataset(dataset_id_str, db.session())
|
||||
if not dataset:
|
||||
raise NotFound("Dataset not found.")
|
||||
document = DocumentService.get_document(dataset.id, document_id_str, session=session)
|
||||
document = DocumentService.get_document(dataset.id, document_id_str, session=db.session())
|
||||
if not document:
|
||||
raise NotFound("Document not found.")
|
||||
log = session.scalar(
|
||||
log = db.session.scalar(
|
||||
select(DocumentPipelineExecutionLog)
|
||||
.where(DocumentPipelineExecutionLog.document_id == document_id_str)
|
||||
.order_by(DocumentPipelineExecutionLog.created_at.desc())
|
||||
@@ -1587,8 +1532,7 @@ class DocumentGenerateSummaryApi(Resource):
|
||||
@cloud_edition_billing_rate_limit_check("knowledge")
|
||||
@with_current_user
|
||||
@rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_EDIT)
|
||||
@with_session
|
||||
def post(self, session: Session, current_user: Account, dataset_id: UUID):
|
||||
def post(self, current_user: Account, dataset_id: UUID):
|
||||
"""
|
||||
Generate summary index for specified documents.
|
||||
|
||||
@@ -1599,7 +1543,7 @@ class DocumentGenerateSummaryApi(Resource):
|
||||
dataset_id_str = str(dataset_id)
|
||||
|
||||
# Get dataset
|
||||
dataset = DatasetService.get_dataset(dataset_id_str, session)
|
||||
dataset = DatasetService.get_dataset(dataset_id_str, db.session())
|
||||
if not dataset:
|
||||
raise NotFound("Dataset not found.")
|
||||
|
||||
@@ -1608,7 +1552,7 @@ class DocumentGenerateSummaryApi(Resource):
|
||||
raise Forbidden()
|
||||
|
||||
try:
|
||||
DatasetService.check_dataset_permission(dataset, current_user, session)
|
||||
DatasetService.check_dataset_permission(dataset, current_user, db.session())
|
||||
except services.errors.account.NoPermissionError as e:
|
||||
raise Forbidden(str(e))
|
||||
|
||||
@@ -1633,7 +1577,7 @@ class DocumentGenerateSummaryApi(Resource):
|
||||
raise ValueError("Summary index is not enabled for this dataset. Please enable it in the dataset settings.")
|
||||
|
||||
# Verify all documents exist and belong to the dataset
|
||||
documents = DocumentService.get_documents_by_ids(dataset_id_str, document_list, session)
|
||||
documents = DocumentService.get_documents_by_ids(dataset_id_str, document_list, db.session())
|
||||
|
||||
if len(documents) != len(document_list):
|
||||
found_ids = {doc.id for doc in documents}
|
||||
@@ -1649,7 +1593,7 @@ class DocumentGenerateSummaryApi(Resource):
|
||||
DocumentService.update_documents_need_summary(
|
||||
dataset_id=dataset_id_str,
|
||||
document_ids=document_ids_to_update,
|
||||
session=session,
|
||||
session=db.session(),
|
||||
need_summary=True,
|
||||
)
|
||||
|
||||
@@ -1687,8 +1631,7 @@ class DocumentSummaryStatusApi(DocumentResource):
|
||||
@account_initialization_required
|
||||
@with_current_user
|
||||
@rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_CREATE_AND_MANAGEMENT)
|
||||
@with_session(write=False)
|
||||
def get(self, session: Session, current_user: Account, dataset_id: UUID, document_id: UUID):
|
||||
def get(self, current_user: Account, dataset_id: UUID, document_id: UUID):
|
||||
"""
|
||||
Get summary index generation status for a document.
|
||||
|
||||
@@ -1706,13 +1649,13 @@ class DocumentSummaryStatusApi(DocumentResource):
|
||||
document_id_str = str(document_id)
|
||||
|
||||
# Get dataset
|
||||
dataset = DatasetService.get_dataset(dataset_id_str, session)
|
||||
dataset = DatasetService.get_dataset(dataset_id_str, db.session())
|
||||
if not dataset:
|
||||
raise NotFound("Dataset not found.")
|
||||
|
||||
# Check permissions
|
||||
try:
|
||||
DatasetService.check_dataset_permission(dataset, current_user, session)
|
||||
DatasetService.check_dataset_permission(dataset, current_user, db.session())
|
||||
except services.errors.account.NoPermissionError as e:
|
||||
raise Forbidden(str(e))
|
||||
|
||||
@@ -1722,7 +1665,7 @@ class DocumentSummaryStatusApi(DocumentResource):
|
||||
result = SummaryIndexService.get_document_summary_status_detail(
|
||||
document_id=document_id_str,
|
||||
dataset_id=dataset_id_str,
|
||||
session=session,
|
||||
session=db.session(),
|
||||
)
|
||||
|
||||
return dump_response(DocumentSummaryStatusResponse, result), 200
|
||||
|
||||
@@ -8,7 +8,6 @@ from flask_restx import Resource
|
||||
from pydantic import BaseModel, Field
|
||||
from sqlalchemy import String, case, cast, func, literal, or_, select
|
||||
from sqlalchemy.dialects.postgresql import JSONB
|
||||
from sqlalchemy.orm import Session
|
||||
from werkzeug.exceptions import Forbidden, NotFound
|
||||
|
||||
import services
|
||||
@@ -21,7 +20,6 @@ from controllers.common.schema import (
|
||||
register_response_schema_models,
|
||||
register_schema_models,
|
||||
)
|
||||
from controllers.common.session import with_session
|
||||
from controllers.console import console_ns
|
||||
from controllers.console.app.error import ProviderNotInitializeError
|
||||
from controllers.console.datasets.error import (
|
||||
@@ -44,6 +42,7 @@ from controllers.console.wraps import (
|
||||
from core.errors.error import LLMBadRequestError, ProviderTokenNotInitError
|
||||
from core.model_manager import ModelManager
|
||||
from core.rag.index_processor.constant.index_type import IndexTechniqueType
|
||||
from extensions.ext_database import db
|
||||
from extensions.ext_redis import redis_client
|
||||
from fields.base import ResponseModel
|
||||
from fields.segment_fields import (
|
||||
@@ -166,7 +165,7 @@ register_response_schema_models(
|
||||
|
||||
|
||||
def _get_segment_for_document(
|
||||
session: Session, dataset: Dataset, document: Document, segment_id: str
|
||||
dataset: Dataset, document: Document, segment_id: str
|
||||
) -> tuple[SegmentRef, DocumentSegment]:
|
||||
dataset_ref = DatasetRefService.create_dataset_ref(dataset)
|
||||
document_ref = DatasetRefService.create_document_ref(dataset_ref, document)
|
||||
@@ -174,7 +173,7 @@ def _get_segment_for_document(
|
||||
raise NotFound("Document not found.")
|
||||
|
||||
segment_ref = DatasetRefService.create_segment_ref(document_ref, segment_id)
|
||||
segment = SegmentService.get_segment_by_ref(segment_ref, session=session)
|
||||
segment = SegmentService.get_segment_by_ref(segment_ref, db.session())
|
||||
if not segment:
|
||||
raise NotFound("Segment not found.")
|
||||
return segment_ref, segment
|
||||
@@ -191,20 +190,19 @@ class DatasetDocumentSegmentListApi(Resource):
|
||||
@with_current_user
|
||||
@with_current_tenant_id
|
||||
@rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_READONLY)
|
||||
@with_session(write=False)
|
||||
def get(self, session: Session, current_tenant_id: str, current_user: Account, dataset_id: UUID, document_id: UUID):
|
||||
def get(self, current_tenant_id: str, current_user: Account, dataset_id: UUID, document_id: UUID):
|
||||
dataset_id_str = str(dataset_id)
|
||||
document_id_str = str(document_id)
|
||||
dataset = DatasetService.get_dataset(dataset_id_str, session)
|
||||
dataset = DatasetService.get_dataset(dataset_id_str, db.session())
|
||||
if not dataset:
|
||||
raise NotFound("Dataset not found.")
|
||||
|
||||
try:
|
||||
DatasetService.check_dataset_permission(dataset, current_user, session)
|
||||
DatasetService.check_dataset_permission(dataset, current_user, db.session())
|
||||
except services.errors.account.NoPermissionError as e:
|
||||
raise Forbidden(str(e))
|
||||
|
||||
document = DocumentService.get_document(dataset_id_str, document_id_str, session=session)
|
||||
document = DocumentService.get_document(dataset_id_str, document_id_str, session=db.session())
|
||||
|
||||
if not document:
|
||||
raise NotFound("Document not found.")
|
||||
@@ -273,19 +271,19 @@ class DatasetDocumentSegmentListApi(Resource):
|
||||
elif args.enabled.lower() == "false":
|
||||
query = query.where(DocumentSegment.enabled == False)
|
||||
|
||||
segments = paginate_query(query, session=session, page=page, per_page=limit, max_per_page=100)
|
||||
segments = paginate_query(query, page=page, per_page=limit, max_per_page=100)
|
||||
|
||||
segment_list = list(segments.items)
|
||||
segment_ids = [segment.id for segment in segment_list]
|
||||
summaries: dict[str, str | None] = {}
|
||||
if segment_ids:
|
||||
summary_records = SummaryIndexService.get_segments_summaries(
|
||||
segment_ids=segment_ids, dataset_id=dataset_id_str, session=session
|
||||
segment_ids=segment_ids, dataset_id=dataset_id_str, session=db.session()
|
||||
)
|
||||
summaries = {chunk_id: summary.summary_content for chunk_id, summary in summary_records.items()}
|
||||
|
||||
response = {
|
||||
"data": segment_responses_with_summaries(segment_list, summaries, session=session),
|
||||
"data": segment_responses_with_summaries(segment_list, summaries),
|
||||
"limit": limit,
|
||||
"total": segments.total,
|
||||
"total_pages": segments.pages,
|
||||
@@ -302,18 +300,17 @@ class DatasetDocumentSegmentListApi(Resource):
|
||||
@console_ns.response(204, "Segments deleted successfully")
|
||||
@with_current_user
|
||||
@rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_EDIT)
|
||||
@with_session
|
||||
def delete(self, session: Session, current_user: Account, dataset_id: UUID, document_id: UUID):
|
||||
def delete(self, current_user: Account, dataset_id: UUID, document_id: UUID):
|
||||
# check dataset
|
||||
dataset_id_str = str(dataset_id)
|
||||
dataset = DatasetService.get_dataset(dataset_id_str, session)
|
||||
dataset = DatasetService.get_dataset(dataset_id_str, db.session())
|
||||
if not dataset:
|
||||
raise NotFound("Dataset not found.")
|
||||
# check user's model setting
|
||||
DatasetService.check_dataset_model_setting(dataset)
|
||||
# check document
|
||||
document_id_str = str(document_id)
|
||||
document = DocumentService.get_document(dataset_id_str, document_id_str, session=session)
|
||||
document = DocumentService.get_document(dataset_id_str, document_id_str, session=db.session())
|
||||
if not document:
|
||||
raise NotFound("Document not found.")
|
||||
segment_ids = request.args.getlist("segment_id")
|
||||
@@ -322,10 +319,10 @@ class DatasetDocumentSegmentListApi(Resource):
|
||||
if not current_user.is_dataset_editor:
|
||||
raise Forbidden()
|
||||
try:
|
||||
DatasetService.check_dataset_permission(dataset, current_user, session)
|
||||
DatasetService.check_dataset_permission(dataset, current_user, db.session())
|
||||
except services.errors.account.NoPermissionError as e:
|
||||
raise Forbidden(str(e))
|
||||
SegmentService.delete_segments(segment_ids, document, dataset, session)
|
||||
SegmentService.delete_segments(segment_ids, document, dataset, db.session())
|
||||
return "", 204
|
||||
|
||||
|
||||
@@ -342,10 +339,8 @@ class DatasetDocumentSegmentApi(Resource):
|
||||
@with_current_user
|
||||
@with_current_tenant_id
|
||||
@rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_EDIT)
|
||||
@with_session
|
||||
def patch(
|
||||
self,
|
||||
session: Session,
|
||||
current_tenant_id: str,
|
||||
current_user: Account,
|
||||
dataset_id: UUID,
|
||||
@@ -353,11 +348,11 @@ class DatasetDocumentSegmentApi(Resource):
|
||||
action: Literal["enable", "disable"],
|
||||
):
|
||||
dataset_id_str = str(dataset_id)
|
||||
dataset = DatasetService.get_dataset(dataset_id_str, session)
|
||||
dataset = DatasetService.get_dataset(dataset_id_str, db.session())
|
||||
if not dataset:
|
||||
raise NotFound("Dataset not found.")
|
||||
document_id_str = str(document_id)
|
||||
document = DocumentService.get_document(dataset_id_str, document_id_str, session=session)
|
||||
document = DocumentService.get_document(dataset_id_str, document_id_str, session=db.session())
|
||||
if not document:
|
||||
raise NotFound("Document not found.")
|
||||
# check user's model setting
|
||||
@@ -367,7 +362,7 @@ class DatasetDocumentSegmentApi(Resource):
|
||||
raise Forbidden()
|
||||
|
||||
try:
|
||||
DatasetService.check_dataset_permission(dataset, current_user, session)
|
||||
DatasetService.check_dataset_permission(dataset, current_user, db.session())
|
||||
except services.errors.account.NoPermissionError as e:
|
||||
raise Forbidden(str(e))
|
||||
if dataset.indexing_technique == IndexTechniqueType.HIGH_QUALITY:
|
||||
@@ -393,7 +388,7 @@ class DatasetDocumentSegmentApi(Resource):
|
||||
if cache_result is not None:
|
||||
raise InvalidActionError("Document is being indexed, please try again later")
|
||||
try:
|
||||
SegmentService.update_segments_status(segment_ids, action, dataset, document, session)
|
||||
SegmentService.update_segments_status(segment_ids, action, dataset, document, db.session())
|
||||
except Exception as e:
|
||||
raise InvalidActionError(str(e))
|
||||
return SimpleResultResponse(result="success").model_dump(mode="json"), 200
|
||||
@@ -413,23 +408,15 @@ class DatasetDocumentSegmentAddApi(Resource):
|
||||
@with_current_user
|
||||
@with_current_tenant_id
|
||||
@rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_EDIT)
|
||||
@with_session
|
||||
def post(
|
||||
self,
|
||||
session: Session,
|
||||
current_tenant_id: str,
|
||||
current_user: Account,
|
||||
dataset_id: UUID,
|
||||
document_id: UUID,
|
||||
):
|
||||
def post(self, current_tenant_id: str, current_user: Account, dataset_id: UUID, document_id: UUID):
|
||||
# check dataset
|
||||
dataset_id_str = str(dataset_id)
|
||||
dataset = DatasetService.get_dataset(dataset_id_str, session)
|
||||
dataset = DatasetService.get_dataset(dataset_id_str, db.session())
|
||||
if not dataset:
|
||||
raise NotFound("Dataset not found.")
|
||||
# check document
|
||||
document_id_str = str(document_id)
|
||||
document = DocumentService.get_document(dataset_id_str, document_id_str, session=session)
|
||||
document = DocumentService.get_document(dataset_id_str, document_id_str, session=db.session())
|
||||
if not document:
|
||||
raise NotFound("Document not found.")
|
||||
if not current_user.is_dataset_editor:
|
||||
@@ -451,21 +438,22 @@ class DatasetDocumentSegmentAddApi(Resource):
|
||||
except ProviderTokenNotInitError as ex:
|
||||
raise ProviderNotInitializeError(ex.description)
|
||||
try:
|
||||
DatasetService.check_dataset_permission(dataset, current_user, session)
|
||||
DatasetService.check_dataset_permission(dataset, current_user, db.session())
|
||||
except services.errors.account.NoPermissionError as e:
|
||||
raise Forbidden(str(e))
|
||||
# validate args
|
||||
payload = SegmentCreatePayload.model_validate(console_ns.payload or {})
|
||||
payload_dict = payload.model_dump(exclude_none=True)
|
||||
SegmentService.segment_create_args_validate(payload_dict, document)
|
||||
segment = type_cast(DocumentSegment, SegmentService.create_segment(payload_dict, document, dataset, session))
|
||||
segment = type_cast(
|
||||
DocumentSegment,
|
||||
SegmentService.create_segment(payload_dict, document, dataset, db.session()),
|
||||
)
|
||||
summary = SummaryIndexService.get_segment_summary(
|
||||
segment_id=segment.id, dataset_id=dataset_id_str, session=session
|
||||
segment_id=segment.id, dataset_id=dataset_id_str, session=db.session()
|
||||
)
|
||||
response = {
|
||||
"data": segment_response_with_summary(
|
||||
segment, summary.summary_content if summary else None, session=session
|
||||
),
|
||||
"data": segment_response_with_summary(segment, summary.summary_content if summary else None),
|
||||
"doc_form": document.doc_form,
|
||||
}
|
||||
return dump_response(SegmentDetailResponse, response), 200
|
||||
@@ -484,33 +472,26 @@ class DatasetDocumentSegmentUpdateApi(Resource):
|
||||
@with_current_user
|
||||
@with_current_tenant_id
|
||||
@rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_EDIT)
|
||||
@with_session
|
||||
def patch(
|
||||
self,
|
||||
session: Session,
|
||||
current_tenant_id: str,
|
||||
current_user: Account,
|
||||
dataset_id: UUID,
|
||||
document_id: UUID,
|
||||
segment_id: UUID,
|
||||
self, current_tenant_id: str, current_user: Account, dataset_id: UUID, document_id: UUID, segment_id: UUID
|
||||
):
|
||||
# check dataset
|
||||
dataset_id_str = str(dataset_id)
|
||||
dataset = DatasetService.get_dataset(dataset_id_str, session)
|
||||
dataset = DatasetService.get_dataset(dataset_id_str, db.session())
|
||||
if not dataset:
|
||||
raise NotFound("Dataset not found.")
|
||||
# check user's model setting
|
||||
DatasetService.check_dataset_model_setting(dataset)
|
||||
# check document
|
||||
document_id_str = str(document_id)
|
||||
document = DocumentService.get_document(dataset_id_str, document_id_str, session=session)
|
||||
document = DocumentService.get_document(dataset_id_str, document_id_str, session=db.session())
|
||||
if not document:
|
||||
raise NotFound("Document not found.")
|
||||
# The role of the current user in the ta table must be admin, owner, dataset_operator, or editor
|
||||
if not current_user.is_dataset_editor:
|
||||
raise Forbidden()
|
||||
try:
|
||||
DatasetService.check_dataset_permission(dataset, current_user, session)
|
||||
DatasetService.check_dataset_permission(dataset, current_user, db.session())
|
||||
except services.errors.account.NoPermissionError as e:
|
||||
raise Forbidden(str(e))
|
||||
if dataset.indexing_technique == IndexTechniqueType.HIGH_QUALITY:
|
||||
@@ -530,7 +511,7 @@ class DatasetDocumentSegmentUpdateApi(Resource):
|
||||
except ProviderTokenNotInitError as ex:
|
||||
raise ProviderNotInitializeError(ex.description)
|
||||
segment_id_str = str(segment_id)
|
||||
_, segment = _get_segment_for_document(session, dataset, document, segment_id_str)
|
||||
_, segment = _get_segment_for_document(dataset, document, segment_id_str)
|
||||
# validate args
|
||||
payload = SegmentUpdatePayload.model_validate(console_ns.payload or {})
|
||||
payload_dict = payload.model_dump(exclude_none=True)
|
||||
@@ -542,15 +523,13 @@ class DatasetDocumentSegmentUpdateApi(Resource):
|
||||
segment,
|
||||
document,
|
||||
dataset,
|
||||
session,
|
||||
db.session(),
|
||||
)
|
||||
summary = SummaryIndexService.get_segment_summary(
|
||||
segment_id=segment.id, dataset_id=dataset_id_str, session=session
|
||||
segment_id=segment.id, dataset_id=dataset_id_str, session=db.session()
|
||||
)
|
||||
response = {
|
||||
"data": segment_response_with_summary(
|
||||
segment, summary.summary_content if summary else None, session=session
|
||||
),
|
||||
"data": segment_response_with_summary(segment, summary.summary_content if summary else None),
|
||||
"doc_form": document.doc_form,
|
||||
}
|
||||
return dump_response(SegmentDetailResponse, response), 200
|
||||
@@ -564,38 +543,31 @@ class DatasetDocumentSegmentUpdateApi(Resource):
|
||||
@with_current_user
|
||||
@with_current_tenant_id
|
||||
@rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_EDIT)
|
||||
@with_session
|
||||
def delete(
|
||||
self,
|
||||
session: Session,
|
||||
current_tenant_id: str,
|
||||
current_user: Account,
|
||||
dataset_id: UUID,
|
||||
document_id: UUID,
|
||||
segment_id: UUID,
|
||||
self, current_tenant_id: str, current_user: Account, dataset_id: UUID, document_id: UUID, segment_id: UUID
|
||||
):
|
||||
# check dataset
|
||||
dataset_id_str = str(dataset_id)
|
||||
dataset = DatasetService.get_dataset(dataset_id_str, session)
|
||||
dataset = DatasetService.get_dataset(dataset_id_str, db.session())
|
||||
if not dataset:
|
||||
raise NotFound("Dataset not found.")
|
||||
# check user's model setting
|
||||
DatasetService.check_dataset_model_setting(dataset)
|
||||
# check document
|
||||
document_id_str = str(document_id)
|
||||
document = DocumentService.get_document(dataset_id_str, document_id_str, session=session)
|
||||
document = DocumentService.get_document(dataset_id_str, document_id_str, session=db.session())
|
||||
if not document:
|
||||
raise NotFound("Document not found.")
|
||||
# The role of the current user in the ta table must be admin, owner, dataset_operator, or editor
|
||||
if not current_user.is_dataset_editor:
|
||||
raise Forbidden()
|
||||
try:
|
||||
DatasetService.check_dataset_permission(dataset, current_user, session)
|
||||
DatasetService.check_dataset_permission(dataset, current_user, db.session())
|
||||
except services.errors.account.NoPermissionError as e:
|
||||
raise Forbidden(str(e))
|
||||
segment_id_str = str(segment_id)
|
||||
_, segment = _get_segment_for_document(session, dataset, document, segment_id_str)
|
||||
SegmentService.delete_segment(segment, document, dataset, session)
|
||||
_, segment = _get_segment_for_document(dataset, document, segment_id_str)
|
||||
SegmentService.delete_segment(segment, document, dataset, db.session())
|
||||
return "", 204
|
||||
|
||||
|
||||
@@ -615,30 +587,22 @@ class DatasetDocumentSegmentBatchImportApi(Resource):
|
||||
@with_current_user
|
||||
@with_current_tenant_id
|
||||
@rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_EDIT)
|
||||
@with_session
|
||||
def post(
|
||||
self,
|
||||
session: Session,
|
||||
current_tenant_id: str,
|
||||
current_user: Account,
|
||||
dataset_id: UUID,
|
||||
document_id: UUID,
|
||||
):
|
||||
def post(self, current_tenant_id: str, current_user: Account, dataset_id: UUID, document_id: UUID):
|
||||
# check dataset
|
||||
dataset_id_str = str(dataset_id)
|
||||
dataset = DatasetService.get_dataset(dataset_id_str, session)
|
||||
dataset = DatasetService.get_dataset(dataset_id_str, db.session())
|
||||
if not dataset:
|
||||
raise NotFound("Dataset not found.")
|
||||
# check document
|
||||
document_id_str = str(document_id)
|
||||
document = DocumentService.get_document(dataset_id_str, document_id_str, session=session)
|
||||
document = DocumentService.get_document(dataset_id_str, document_id_str, session=db.session())
|
||||
if not document:
|
||||
raise NotFound("Document not found.")
|
||||
|
||||
payload = BatchImportPayload.model_validate(console_ns.payload or {})
|
||||
upload_file_id = payload.upload_file_id
|
||||
|
||||
upload_file = session.scalar(select(UploadFile).where(UploadFile.id == upload_file_id).limit(1))
|
||||
upload_file = db.session.scalar(select(UploadFile).where(UploadFile.id == upload_file_id).limit(1))
|
||||
if not upload_file:
|
||||
raise NotFound("UploadFile not found.")
|
||||
|
||||
@@ -696,30 +660,23 @@ class ChildChunkAddApi(Resource):
|
||||
@with_current_user
|
||||
@with_current_tenant_id
|
||||
@rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_EDIT)
|
||||
@with_session
|
||||
def post(
|
||||
self,
|
||||
session: Session,
|
||||
current_tenant_id: str,
|
||||
current_user: Account,
|
||||
dataset_id: UUID,
|
||||
document_id: UUID,
|
||||
segment_id: UUID,
|
||||
self, current_tenant_id: str, current_user: Account, dataset_id: UUID, document_id: UUID, segment_id: UUID
|
||||
):
|
||||
# check dataset
|
||||
dataset_id_str = str(dataset_id)
|
||||
dataset = DatasetService.get_dataset(dataset_id_str, session)
|
||||
dataset = DatasetService.get_dataset(dataset_id_str, db.session())
|
||||
if not dataset:
|
||||
raise NotFound("Dataset not found.")
|
||||
# check document
|
||||
document_id_str = str(document_id)
|
||||
document = DocumentService.get_document(dataset_id_str, document_id_str, session=session)
|
||||
document = DocumentService.get_document(dataset_id_str, document_id_str, session=db.session())
|
||||
if not document:
|
||||
raise NotFound("Document not found.")
|
||||
if not current_user.is_dataset_editor:
|
||||
raise Forbidden()
|
||||
try:
|
||||
DatasetService.check_dataset_permission(dataset, current_user, session)
|
||||
DatasetService.check_dataset_permission(dataset, current_user, db.session())
|
||||
except services.errors.account.NoPermissionError as e:
|
||||
raise Forbidden(str(e))
|
||||
# check embedding model setting
|
||||
@@ -739,11 +696,11 @@ class ChildChunkAddApi(Resource):
|
||||
except ProviderTokenNotInitError as ex:
|
||||
raise ProviderNotInitializeError(ex.description)
|
||||
segment_id_str = str(segment_id)
|
||||
_, segment = _get_segment_for_document(session, dataset, document, segment_id_str)
|
||||
_, segment = _get_segment_for_document(dataset, document, segment_id_str)
|
||||
# validate args
|
||||
try:
|
||||
payload = ChildChunkCreatePayload.model_validate(console_ns.payload or {})
|
||||
child_chunk = SegmentService.create_child_chunk(payload.content, segment, document, dataset, session)
|
||||
child_chunk = SegmentService.create_child_chunk(payload.content, segment, document, dataset, db.session())
|
||||
except ChildChunkIndexingServiceError as e:
|
||||
raise ChildChunkIndexingError(str(e))
|
||||
return dump_response(ChildChunkDetailResponse, {"data": child_chunk}), 200
|
||||
@@ -756,22 +713,21 @@ class ChildChunkAddApi(Resource):
|
||||
@account_initialization_required
|
||||
@with_current_tenant_id
|
||||
@rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_READONLY)
|
||||
@with_session(write=False)
|
||||
def get(self, session: Session, current_tenant_id: str, dataset_id: UUID, document_id: UUID, segment_id: UUID):
|
||||
def get(self, current_tenant_id: str, dataset_id: UUID, document_id: UUID, segment_id: UUID):
|
||||
# check dataset
|
||||
dataset_id_str = str(dataset_id)
|
||||
dataset = DatasetService.get_dataset(dataset_id_str, session)
|
||||
dataset = DatasetService.get_dataset(dataset_id_str, db.session())
|
||||
if not dataset:
|
||||
raise NotFound("Dataset not found.")
|
||||
# check user's model setting
|
||||
DatasetService.check_dataset_model_setting(dataset)
|
||||
# check document
|
||||
document_id_str = str(document_id)
|
||||
document = DocumentService.get_document(dataset_id_str, document_id_str, session=session)
|
||||
document = DocumentService.get_document(dataset_id_str, document_id_str, session=db.session())
|
||||
if not document:
|
||||
raise NotFound("Document not found.")
|
||||
segment_id_str = str(segment_id)
|
||||
_get_segment_for_document(session, dataset, document, segment_id_str)
|
||||
_get_segment_for_document(dataset, document, segment_id_str)
|
||||
args = query_params_from_request(ChildChunkListQuery, use_defaults_for_malformed_ints=True)
|
||||
|
||||
page = args.page
|
||||
@@ -779,13 +735,7 @@ class ChildChunkAddApi(Resource):
|
||||
keyword = args.keyword
|
||||
|
||||
child_chunks = SegmentService.get_child_chunks(
|
||||
segment_id_str,
|
||||
document_id_str,
|
||||
dataset_id_str,
|
||||
page,
|
||||
limit,
|
||||
keyword,
|
||||
session=session,
|
||||
segment_id_str, document_id_str, dataset_id_str, page, limit, keyword
|
||||
)
|
||||
response = {
|
||||
"data": child_chunks.items,
|
||||
@@ -811,41 +761,34 @@ class ChildChunkAddApi(Resource):
|
||||
@with_current_user
|
||||
@with_current_tenant_id
|
||||
@rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_EDIT)
|
||||
@with_session
|
||||
def patch(
|
||||
self,
|
||||
session: Session,
|
||||
current_tenant_id: str,
|
||||
current_user: Account,
|
||||
dataset_id: UUID,
|
||||
document_id: UUID,
|
||||
segment_id: UUID,
|
||||
self, current_tenant_id: str, current_user: Account, dataset_id: UUID, document_id: UUID, segment_id: UUID
|
||||
):
|
||||
# check dataset
|
||||
dataset_id_str = str(dataset_id)
|
||||
dataset = DatasetService.get_dataset(dataset_id_str, session)
|
||||
dataset = DatasetService.get_dataset(dataset_id_str, db.session())
|
||||
if not dataset:
|
||||
raise NotFound("Dataset not found.")
|
||||
# check user's model setting
|
||||
DatasetService.check_dataset_model_setting(dataset)
|
||||
# check document
|
||||
document_id_str = str(document_id)
|
||||
document = DocumentService.get_document(dataset_id_str, document_id_str, session=session)
|
||||
document = DocumentService.get_document(dataset_id_str, document_id_str, session=db.session())
|
||||
if not document:
|
||||
raise NotFound("Document not found.")
|
||||
# The role of the current user in the ta table must be admin, owner, dataset_operator, or editor
|
||||
if not current_user.is_dataset_editor:
|
||||
raise Forbidden()
|
||||
try:
|
||||
DatasetService.check_dataset_permission(dataset, current_user, session)
|
||||
DatasetService.check_dataset_permission(dataset, current_user, db.session())
|
||||
except services.errors.account.NoPermissionError as e:
|
||||
raise Forbidden(str(e))
|
||||
segment_id_str = str(segment_id)
|
||||
_, segment = _get_segment_for_document(session, dataset, document, segment_id_str)
|
||||
_, segment = _get_segment_for_document(dataset, document, segment_id_str)
|
||||
# validate args
|
||||
payload = ChildChunkBatchUpdatePayload.model_validate(console_ns.payload or {})
|
||||
try:
|
||||
child_chunks = SegmentService.update_child_chunks(payload.chunks, segment, document, dataset, session)
|
||||
child_chunks = SegmentService.update_child_chunks(payload.chunks, segment, document, dataset, db.session())
|
||||
except ChildChunkIndexingServiceError as e:
|
||||
raise ChildChunkIndexingError(str(e))
|
||||
return dump_response(ChildChunkBatchUpdateResponse, {"data": child_chunks}), 200
|
||||
@@ -864,10 +807,8 @@ class ChildChunkUpdateApi(Resource):
|
||||
@with_current_user
|
||||
@with_current_tenant_id
|
||||
@rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_EDIT)
|
||||
@with_session
|
||||
def delete(
|
||||
self,
|
||||
session: Session,
|
||||
current_tenant_id: str,
|
||||
current_user: Account,
|
||||
dataset_id: UUID,
|
||||
@@ -877,31 +818,31 @@ class ChildChunkUpdateApi(Resource):
|
||||
):
|
||||
# check dataset
|
||||
dataset_id_str = str(dataset_id)
|
||||
dataset = DatasetService.get_dataset(dataset_id_str, session)
|
||||
dataset = DatasetService.get_dataset(dataset_id_str, db.session())
|
||||
if not dataset:
|
||||
raise NotFound("Dataset not found.")
|
||||
# check user's model setting
|
||||
DatasetService.check_dataset_model_setting(dataset)
|
||||
# check document
|
||||
document_id_str = str(document_id)
|
||||
document = DocumentService.get_document(dataset_id_str, document_id_str, session=session)
|
||||
document = DocumentService.get_document(dataset_id_str, document_id_str, session=db.session())
|
||||
if not document:
|
||||
raise NotFound("Document not found.")
|
||||
# The role of the current user in the ta table must be admin, owner, dataset_operator, or editor
|
||||
if not current_user.is_dataset_editor:
|
||||
raise Forbidden()
|
||||
try:
|
||||
DatasetService.check_dataset_permission(dataset, current_user, session)
|
||||
DatasetService.check_dataset_permission(dataset, current_user, db.session())
|
||||
except services.errors.account.NoPermissionError as e:
|
||||
raise Forbidden(str(e))
|
||||
segment_id_str = str(segment_id)
|
||||
segment_ref, _ = _get_segment_for_document(session, dataset, document, segment_id_str)
|
||||
segment_ref, _ = _get_segment_for_document(dataset, document, segment_id_str)
|
||||
child_chunk_id_str = str(child_chunk_id)
|
||||
child_chunk = SegmentService.get_child_chunk_by_segment_ref(child_chunk_id_str, segment_ref, session=session)
|
||||
child_chunk = SegmentService.get_child_chunk_by_segment_ref(child_chunk_id_str, segment_ref, db.session())
|
||||
if not child_chunk:
|
||||
raise NotFound("Child chunk not found.")
|
||||
try:
|
||||
SegmentService.delete_child_chunk(child_chunk, dataset, session)
|
||||
SegmentService.delete_child_chunk(child_chunk, dataset, db.session())
|
||||
except ChildChunkDeleteIndexServiceError as e:
|
||||
raise ChildChunkDeleteIndexError(str(e))
|
||||
return "", 204
|
||||
@@ -917,10 +858,8 @@ class ChildChunkUpdateApi(Resource):
|
||||
@with_current_user
|
||||
@with_current_tenant_id
|
||||
@rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_EDIT)
|
||||
@with_session
|
||||
def patch(
|
||||
self,
|
||||
session: Session,
|
||||
current_tenant_id: str,
|
||||
current_user: Account,
|
||||
dataset_id: UUID,
|
||||
@@ -930,34 +869,34 @@ class ChildChunkUpdateApi(Resource):
|
||||
):
|
||||
# check dataset
|
||||
dataset_id_str = str(dataset_id)
|
||||
dataset = DatasetService.get_dataset(dataset_id_str, session)
|
||||
dataset = DatasetService.get_dataset(dataset_id_str, db.session())
|
||||
if not dataset:
|
||||
raise NotFound("Dataset not found.")
|
||||
# check user's model setting
|
||||
DatasetService.check_dataset_model_setting(dataset)
|
||||
# check document
|
||||
document_id_str = str(document_id)
|
||||
document = DocumentService.get_document(dataset_id_str, document_id_str, session=session)
|
||||
document = DocumentService.get_document(dataset_id_str, document_id_str, session=db.session())
|
||||
if not document:
|
||||
raise NotFound("Document not found.")
|
||||
# The role of the current user in the ta table must be admin, owner, dataset_operator, or editor
|
||||
if not current_user.is_dataset_editor:
|
||||
raise Forbidden()
|
||||
try:
|
||||
DatasetService.check_dataset_permission(dataset, current_user, session)
|
||||
DatasetService.check_dataset_permission(dataset, current_user, db.session())
|
||||
except services.errors.account.NoPermissionError as e:
|
||||
raise Forbidden(str(e))
|
||||
segment_id_str = str(segment_id)
|
||||
segment_ref, segment = _get_segment_for_document(session, dataset, document, segment_id_str)
|
||||
segment_ref, segment = _get_segment_for_document(dataset, document, segment_id_str)
|
||||
child_chunk_id_str = str(child_chunk_id)
|
||||
child_chunk = SegmentService.get_child_chunk_by_segment_ref(child_chunk_id_str, segment_ref, session=session)
|
||||
child_chunk = SegmentService.get_child_chunk_by_segment_ref(child_chunk_id_str, segment_ref, db.session())
|
||||
if not child_chunk:
|
||||
raise NotFound("Child chunk not found.")
|
||||
# validate args
|
||||
try:
|
||||
payload = ChildChunkUpdatePayload.model_validate(console_ns.payload or {})
|
||||
child_chunk = SegmentService.update_child_chunk(
|
||||
payload.content, child_chunk, segment, document, dataset, session
|
||||
payload.content, child_chunk, segment, document, dataset, db.session()
|
||||
)
|
||||
except ChildChunkIndexingServiceError as e:
|
||||
raise ChildChunkIndexingError(str(e))
|
||||
|
||||
@@ -1,4 +1,3 @@
|
||||
from dataclasses import dataclass
|
||||
from datetime import datetime
|
||||
from typing import Any
|
||||
from uuid import UUID
|
||||
@@ -11,13 +10,9 @@ from werkzeug.exceptions import Forbidden, InternalServerError, NotFound
|
||||
|
||||
import services
|
||||
from controllers.common.fields import UsageCountResponse
|
||||
from controllers.common.schema import (
|
||||
query_params_from_model,
|
||||
register_response_schema_models,
|
||||
register_schema_models,
|
||||
)
|
||||
from controllers.common.session import with_session
|
||||
from controllers.common.schema import query_params_from_model, register_response_schema_models, register_schema_models
|
||||
from controllers.console import console_ns
|
||||
from controllers.console.app.wraps import with_session
|
||||
from controllers.console.datasets.error import DatasetNameDuplicateError
|
||||
from controllers.console.wraps import (
|
||||
RBACPermission,
|
||||
@@ -30,11 +25,10 @@ from controllers.console.wraps import (
|
||||
with_current_user,
|
||||
)
|
||||
from fields.base import ResponseModel
|
||||
from fields.dataset_fields import DatasetDetailResponse, dataset_detail_response_source
|
||||
from fields.dataset_fields import DatasetDetailResponse
|
||||
from libs.helper import dump_response
|
||||
from libs.login import login_required
|
||||
from models import Account
|
||||
from models.dataset import ExternalKnowledgeApis
|
||||
from services.dataset_service import DatasetService
|
||||
from services.enterprise import rbac_service as enterprise_rbac_service
|
||||
from services.external_knowledge_service import ExternalDatasetService
|
||||
@@ -96,28 +90,6 @@ class ExternalKnowledgeApiResponse(ResponseModel):
|
||||
return value
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class ExternalKnowledgeApiResponseSource:
|
||||
external_knowledge_api: ExternalKnowledgeApis
|
||||
session: Session
|
||||
|
||||
@property
|
||||
def dataset_bindings(self) -> Any:
|
||||
return self.external_knowledge_api.get_dataset_bindings(session=self.session)
|
||||
|
||||
def __getattr__(self, name: str) -> Any:
|
||||
return getattr(self.external_knowledge_api, name) # noqa: no-new-getattr response adapter delegates model fields
|
||||
|
||||
|
||||
def external_knowledge_api_response(
|
||||
external_knowledge_api: ExternalKnowledgeApis, *, session: Session
|
||||
) -> ExternalKnowledgeApiResponse:
|
||||
return ExternalKnowledgeApiResponse.model_validate(
|
||||
ExternalKnowledgeApiResponseSource(external_knowledge_api=external_knowledge_api, session=session),
|
||||
from_attributes=True,
|
||||
)
|
||||
|
||||
|
||||
class ExternalKnowledgeApiListResponse(ResponseModel):
|
||||
data: list[ExternalKnowledgeApiResponse]
|
||||
has_more: bool
|
||||
@@ -190,15 +162,14 @@ class ExternalApiTemplateListApi(Resource):
|
||||
@login_required
|
||||
@with_current_tenant_id
|
||||
@account_initialization_required
|
||||
@with_session(write=False)
|
||||
def get(self, session: Session, current_tenant_id: str):
|
||||
def get(self, current_tenant_id: str):
|
||||
query = ExternalApiTemplateListQuery.model_validate(request.args.to_dict())
|
||||
|
||||
external_knowledge_apis, total = ExternalDatasetService.get_external_knowledge_apis(
|
||||
query.page, query.limit, current_tenant_id, query.keyword, session=session
|
||||
query.page, query.limit, current_tenant_id, query.keyword
|
||||
)
|
||||
return ExternalKnowledgeApiListResponse(
|
||||
data=[external_knowledge_api_response(item, session=session) for item in external_knowledge_apis],
|
||||
data=[ExternalKnowledgeApiResponse.model_validate(item) for item in external_knowledge_apis],
|
||||
has_more=len(external_knowledge_apis) == query.limit,
|
||||
limit=query.limit,
|
||||
total=total,
|
||||
@@ -239,7 +210,7 @@ class ExternalApiTemplateListApi(Resource):
|
||||
except services.errors.dataset.DatasetNameDuplicateError:
|
||||
raise DatasetNameDuplicateError()
|
||||
|
||||
return external_knowledge_api_response(external_knowledge_api, session=session).model_dump(mode="json"), 201
|
||||
return dump_response(ExternalKnowledgeApiResponse, external_knowledge_api), 201
|
||||
|
||||
|
||||
@console_ns.route("/datasets/external-knowledge-api/<uuid:external_knowledge_api_id>")
|
||||
@@ -266,7 +237,7 @@ class ExternalApiTemplateApi(Resource):
|
||||
if external_knowledge_api is None:
|
||||
raise NotFound("API template not found.")
|
||||
|
||||
return external_knowledge_api_response(external_knowledge_api, session=session).model_dump(mode="json"), 200
|
||||
return dump_response(ExternalKnowledgeApiResponse, external_knowledge_api), 200
|
||||
|
||||
@console_ns.doc("update_external_api_template")
|
||||
@console_ns.doc(description="Update external knowledge API template")
|
||||
@@ -298,7 +269,7 @@ class ExternalApiTemplateApi(Resource):
|
||||
session=session,
|
||||
)
|
||||
|
||||
return external_knowledge_api_response(external_knowledge_api, session=session).model_dump(mode="json"), 200
|
||||
return dump_response(ExternalKnowledgeApiResponse, external_knowledge_api), 200
|
||||
|
||||
@setup_required
|
||||
@login_required
|
||||
@@ -383,9 +354,7 @@ class ExternalDatasetCreateApi(Resource):
|
||||
[dataset_id_str],
|
||||
session=session,
|
||||
)
|
||||
data = DatasetDetailResponse.model_validate(
|
||||
dataset_detail_response_source(dataset, session=session)
|
||||
).model_dump(mode="json")
|
||||
data = DatasetDetailResponse.model_validate(dataset).model_dump(mode="json")
|
||||
data["permission_keys"] = permission_keys_map.get(dataset_id_str, [])
|
||||
return data, 201
|
||||
|
||||
|
||||
@@ -53,7 +53,7 @@ class HitTestingApi(Resource, DatasetsHitTestingBase):
|
||||
) -> dict[str, object]:
|
||||
dataset_id_str = str(dataset_id)
|
||||
|
||||
dataset = self.get_and_validate_dataset(session, dataset_id_str, current_user, current_tenant_id)
|
||||
dataset = self.get_and_validate_dataset(dataset_id_str, current_user, current_tenant_id)
|
||||
args = self.parse_args(console_ns.payload)
|
||||
self.hit_testing_args_check(args)
|
||||
|
||||
|
||||
@@ -19,6 +19,7 @@ from core.errors.error import (
|
||||
ProviderTokenNotInitError,
|
||||
QuotaExceededError,
|
||||
)
|
||||
from extensions.ext_database import db
|
||||
from graphon.model_runtime.errors.invoke import InvokeError
|
||||
from libs.login import resolve_account_fallback
|
||||
from models.account import Account
|
||||
@@ -82,18 +83,15 @@ class DatasetsHitTestingBase:
|
||||
|
||||
@staticmethod
|
||||
def get_and_validate_dataset(
|
||||
session: Session,
|
||||
dataset_id: str,
|
||||
current_user: Account | None = None,
|
||||
current_tenant_id: str | None = None,
|
||||
dataset_id: str, current_user: Account | None = None, current_tenant_id: str | None = None
|
||||
) -> Dataset:
|
||||
current_user, _ = resolve_account_fallback(current_user, current_tenant_id)
|
||||
dataset = DatasetService.get_dataset(dataset_id, session)
|
||||
dataset = DatasetService.get_dataset(dataset_id, db.session())
|
||||
if dataset is None:
|
||||
raise NotFound("Dataset not found.")
|
||||
|
||||
try:
|
||||
DatasetService.check_dataset_permission(dataset, current_user, session)
|
||||
DatasetService.check_dataset_permission(dataset, current_user, db.session())
|
||||
except services.errors.account.NoPermissionError as e:
|
||||
raise Forbidden(str(e))
|
||||
|
||||
|
||||
@@ -2,12 +2,10 @@ from typing import Literal
|
||||
from uuid import UUID
|
||||
|
||||
from flask_restx import Resource
|
||||
from sqlalchemy.orm import Session
|
||||
from werkzeug.exceptions import NotFound
|
||||
|
||||
from controllers.common.controller_schemas import MetadataUpdatePayload
|
||||
from controllers.common.schema import register_response_schema_models, register_schema_models
|
||||
from controllers.common.session import with_session
|
||||
from controllers.console import console_ns
|
||||
from controllers.console.wraps import (
|
||||
RBACPermission,
|
||||
@@ -19,6 +17,7 @@ from controllers.console.wraps import (
|
||||
with_current_tenant_id,
|
||||
with_current_user,
|
||||
)
|
||||
from extensions.ext_database import db
|
||||
from fields.dataset_fields import (
|
||||
DatasetMetadataBuiltInFieldsResponse,
|
||||
DatasetMetadataListResponse,
|
||||
@@ -58,18 +57,17 @@ class DatasetMetadataCreateApi(Resource):
|
||||
@with_current_user
|
||||
@with_current_tenant_id
|
||||
@rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_EDIT)
|
||||
@with_session
|
||||
def post(self, session: Session, current_tenant_id: str, current_user: Account, dataset_id: UUID):
|
||||
def post(self, current_tenant_id: str, current_user: Account, dataset_id: UUID):
|
||||
metadata_args = MetadataArgs.model_validate(console_ns.payload or {})
|
||||
|
||||
dataset_id_str = str(dataset_id)
|
||||
dataset = DatasetService.get_dataset(dataset_id_str, session)
|
||||
dataset = DatasetService.get_dataset(dataset_id_str, db.session())
|
||||
if dataset is None:
|
||||
raise NotFound("Dataset not found.")
|
||||
DatasetService.check_dataset_permission(dataset, current_user, session)
|
||||
DatasetService.check_dataset_permission(dataset, current_user, db.session())
|
||||
|
||||
metadata = MetadataService.create_metadata(
|
||||
dataset_id_str, metadata_args, current_user, current_tenant_id, session=session
|
||||
dataset_id_str, metadata_args, current_user, current_tenant_id, session=db.session()
|
||||
)
|
||||
return dump_response(DatasetMetadataResponse, metadata), 201
|
||||
|
||||
@@ -81,13 +79,12 @@ class DatasetMetadataCreateApi(Resource):
|
||||
200, "Metadata retrieved successfully", console_ns.models[DatasetMetadataListResponse.__name__]
|
||||
)
|
||||
@rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_CREATE_AND_MANAGEMENT)
|
||||
@with_session(write=False)
|
||||
def get(self, session: Session, dataset_id: UUID):
|
||||
def get(self, dataset_id: UUID):
|
||||
dataset_id_str = str(dataset_id)
|
||||
dataset = DatasetService.get_dataset(dataset_id_str, session)
|
||||
dataset = DatasetService.get_dataset(dataset_id_str, db.session())
|
||||
if dataset is None:
|
||||
raise NotFound("Dataset not found.")
|
||||
metadata = MetadataService.get_dataset_metadatas(dataset, session)
|
||||
metadata = MetadataService.get_dataset_metadatas(dataset, session=db.session())
|
||||
return dump_response(DatasetMetadataListResponse, metadata), 200
|
||||
|
||||
|
||||
@@ -102,27 +99,19 @@ class DatasetMetadataApi(Resource):
|
||||
@with_current_user
|
||||
@with_current_tenant_id
|
||||
@rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_EDIT)
|
||||
@with_session
|
||||
def patch(
|
||||
self,
|
||||
session: Session,
|
||||
current_tenant_id: str,
|
||||
current_user: Account,
|
||||
dataset_id: UUID,
|
||||
metadata_id: UUID,
|
||||
):
|
||||
def patch(self, current_tenant_id: str, current_user: Account, dataset_id: UUID, metadata_id: UUID):
|
||||
payload = MetadataUpdatePayload.model_validate(console_ns.payload or {})
|
||||
name = payload.name
|
||||
|
||||
dataset_id_str = str(dataset_id)
|
||||
metadata_id_str = str(metadata_id)
|
||||
dataset = DatasetService.get_dataset(dataset_id_str, session)
|
||||
dataset = DatasetService.get_dataset(dataset_id_str, db.session())
|
||||
if dataset is None:
|
||||
raise NotFound("Dataset not found.")
|
||||
DatasetService.check_dataset_permission(dataset, current_user, session)
|
||||
DatasetService.check_dataset_permission(dataset, current_user, db.session())
|
||||
|
||||
metadata = MetadataService.update_metadata_name(
|
||||
dataset_id_str, metadata_id_str, name, current_user, current_tenant_id, session=session
|
||||
dataset_id_str, metadata_id_str, name, current_user, current_tenant_id, session=db.session()
|
||||
)
|
||||
return dump_response(DatasetMetadataResponse, metadata), 200
|
||||
|
||||
@@ -133,16 +122,15 @@ class DatasetMetadataApi(Resource):
|
||||
@console_ns.response(204, "Metadata deleted successfully")
|
||||
@with_current_user
|
||||
@rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_EDIT)
|
||||
@with_session
|
||||
def delete(self, session: Session, current_user: Account, dataset_id: UUID, metadata_id: UUID):
|
||||
def delete(self, current_user: Account, dataset_id: UUID, metadata_id: UUID):
|
||||
dataset_id_str = str(dataset_id)
|
||||
metadata_id_str = str(metadata_id)
|
||||
dataset = DatasetService.get_dataset(dataset_id_str, session)
|
||||
dataset = DatasetService.get_dataset(dataset_id_str, db.session())
|
||||
if dataset is None:
|
||||
raise NotFound("Dataset not found.")
|
||||
DatasetService.check_dataset_permission(dataset, current_user, session)
|
||||
DatasetService.check_dataset_permission(dataset, current_user, db.session())
|
||||
|
||||
MetadataService.delete_metadata(dataset_id_str, metadata_id_str, session)
|
||||
MetadataService.delete_metadata(dataset_id_str, metadata_id_str, session=db.session())
|
||||
# Frontend callers only await success and invalidate metadata caches; no response body is consumed.
|
||||
return "", 204
|
||||
|
||||
@@ -172,19 +160,18 @@ class DatasetMetadataBuiltInFieldActionApi(Resource):
|
||||
@console_ns.response(204, "Action completed successfully")
|
||||
@with_current_user
|
||||
@rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_EDIT)
|
||||
@with_session
|
||||
def post(self, session: Session, current_user: Account, dataset_id: UUID, action: Literal["enable", "disable"]):
|
||||
def post(self, current_user: Account, dataset_id: UUID, action: Literal["enable", "disable"]):
|
||||
dataset_id_str = str(dataset_id)
|
||||
dataset = DatasetService.get_dataset(dataset_id_str, session)
|
||||
dataset = DatasetService.get_dataset(dataset_id_str, db.session())
|
||||
if dataset is None:
|
||||
raise NotFound("Dataset not found.")
|
||||
DatasetService.check_dataset_permission(dataset, current_user, session)
|
||||
DatasetService.check_dataset_permission(dataset, current_user, db.session())
|
||||
|
||||
match action:
|
||||
case "enable":
|
||||
MetadataService.enable_built_in_field(dataset, session)
|
||||
MetadataService.enable_built_in_field(dataset, session=db.session())
|
||||
case "disable":
|
||||
MetadataService.disable_built_in_field(dataset, session)
|
||||
MetadataService.disable_built_in_field(dataset, session=db.session())
|
||||
# Frontend callers only await success and invalidate metadata caches; no response body is consumed.
|
||||
return "", 204
|
||||
|
||||
@@ -202,17 +189,16 @@ class DocumentMetadataEditApi(Resource):
|
||||
)
|
||||
@with_current_user
|
||||
@rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_EDIT)
|
||||
@with_session
|
||||
def post(self, session: Session, current_user: Account, dataset_id: UUID):
|
||||
def post(self, current_user: Account, dataset_id: UUID):
|
||||
dataset_id_str = str(dataset_id)
|
||||
dataset = DatasetService.get_dataset(dataset_id_str, session)
|
||||
dataset = DatasetService.get_dataset(dataset_id_str, db.session())
|
||||
if dataset is None:
|
||||
raise NotFound("Dataset not found.")
|
||||
DatasetService.check_dataset_permission(dataset, current_user, session)
|
||||
DatasetService.check_dataset_permission(dataset, current_user, db.session())
|
||||
|
||||
metadata_args = MetadataOperationData.model_validate(console_ns.payload or {})
|
||||
|
||||
MetadataService.update_documents_metadata(dataset, metadata_args, current_user, session=session)
|
||||
MetadataService.update_documents_metadata(dataset, metadata_args, current_user, session=db.session())
|
||||
|
||||
# Frontend callers only await success and invalidate caches; no response body is consumed.
|
||||
return "", 204
|
||||
|
||||
@@ -27,7 +27,7 @@ from controllers.console.app.workflow import (
|
||||
WorkflowResponse,
|
||||
)
|
||||
from controllers.console.app.wraps import with_session
|
||||
from controllers.console.datasets.wraps import get_rag_pipeline, load_rag_pipeline
|
||||
from controllers.console.datasets.wraps import get_rag_pipeline
|
||||
from controllers.console.wraps import (
|
||||
RBACPermission,
|
||||
RBACResourceScope,
|
||||
@@ -344,11 +344,11 @@ class DraftRagPipelineRunApi(Resource):
|
||||
@rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_EDIT)
|
||||
@with_current_user
|
||||
@with_session
|
||||
def post(self, session: Session, current_user: Account, pipeline_id: UUID):
|
||||
@get_rag_pipeline
|
||||
def post(self, session: Session, current_user: Account, pipeline: Pipeline):
|
||||
"""
|
||||
Run draft workflow
|
||||
"""
|
||||
pipeline = load_rag_pipeline(session, str(pipeline_id))
|
||||
payload = DraftWorkflowRunPayload.model_validate(console_ns.payload or {})
|
||||
args = payload.model_dump()
|
||||
|
||||
@@ -378,11 +378,11 @@ class PublishedRagPipelineRunApi(Resource):
|
||||
@rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_EDIT)
|
||||
@with_current_user
|
||||
@with_session
|
||||
def post(self, session: Session, current_user: Account, pipeline_id: UUID):
|
||||
@get_rag_pipeline
|
||||
def post(self, session: Session, current_user: Account, pipeline: Pipeline):
|
||||
"""
|
||||
Run published workflow
|
||||
"""
|
||||
pipeline = load_rag_pipeline(session, str(pipeline_id))
|
||||
payload = PublishedWorkflowRunPayload.model_validate(console_ns.payload or {})
|
||||
args = payload.model_dump(exclude_none=True)
|
||||
streaming = payload.response_mode == "streaming"
|
||||
|
||||
@@ -1,21 +1,13 @@
|
||||
from collections.abc import Callable
|
||||
from functools import wraps
|
||||
|
||||
from sqlalchemy import select
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from controllers.console.datasets.error import PipelineNotFoundError
|
||||
from extensions.ext_database import db
|
||||
from libs.login import current_account_with_tenant
|
||||
from models.dataset import Pipeline
|
||||
from services.rag_pipeline.rag_pipeline import RagPipelineService
|
||||
|
||||
|
||||
def load_rag_pipeline(session: Session, pipeline_id: str) -> Pipeline:
|
||||
_, current_tenant_id = current_account_with_tenant()
|
||||
pipeline = RagPipelineService.get_pipeline_by_id(pipeline_id, current_tenant_id, session=session)
|
||||
if not pipeline:
|
||||
raise PipelineNotFoundError()
|
||||
return pipeline
|
||||
|
||||
|
||||
def get_rag_pipeline[**P, R](view_func: Callable[P, R]) -> Callable[P, R]:
|
||||
@@ -24,11 +16,22 @@ def get_rag_pipeline[**P, R](view_func: Callable[P, R]) -> Callable[P, R]:
|
||||
if not kwargs.get("pipeline_id"):
|
||||
raise ValueError("missing pipeline_id in path parameters")
|
||||
|
||||
_, current_tenant_id = current_account_with_tenant()
|
||||
|
||||
pipeline_id = kwargs.get("pipeline_id")
|
||||
pipeline_id = str(pipeline_id)
|
||||
|
||||
del kwargs["pipeline_id"]
|
||||
kwargs["pipeline"] = load_rag_pipeline(db.session(), pipeline_id)
|
||||
|
||||
stmt = select(Pipeline).where(Pipeline.id == pipeline_id, Pipeline.tenant_id == current_tenant_id).limit(1)
|
||||
# Migrated handlers pass the request Session as args[1]; legacy handlers still use db.session.
|
||||
session = args[1] if len(args) > 1 and isinstance(args[1], Session) else db.session
|
||||
pipeline = session.scalar(stmt)
|
||||
|
||||
if not pipeline:
|
||||
raise PipelineNotFoundError()
|
||||
|
||||
kwargs["pipeline"] = pipeline
|
||||
|
||||
return view_func(*args, **kwargs)
|
||||
|
||||
|
||||
@@ -50,19 +50,14 @@ register_response_schema_models(console_ns, AudioBinaryResponse, AudioTranscript
|
||||
class ChatAudioApi(InstalledAppResource):
|
||||
@console_ns.response(200, "Success", console_ns.models[AudioTranscriptResponse.__name__])
|
||||
def post(self, installed_app: InstalledApp):
|
||||
app_model = installed_app.app_with_session(session=db.session())
|
||||
app_model = installed_app.app
|
||||
if app_model is None:
|
||||
raise AppUnavailableError()
|
||||
|
||||
file = request.files["file"]
|
||||
|
||||
try:
|
||||
response = AudioService.transcript_asr(
|
||||
app_model=app_model,
|
||||
file=file,
|
||||
session=db.session(),
|
||||
end_user=None,
|
||||
)
|
||||
response = AudioService.transcript_asr(app_model=app_model, file=file, end_user=None)
|
||||
|
||||
return response
|
||||
except services.errors.app_model_config.AppModelConfigBrokenError:
|
||||
@@ -101,7 +96,7 @@ class ChatTextApi(InstalledAppResource):
|
||||
@console_ns.expect(console_ns.models[TextToAudioPayload.__name__])
|
||||
@console_ns.response(200, "Success", console_ns.models[AudioBinaryResponse.__name__])
|
||||
def post(self, installed_app: InstalledApp):
|
||||
app_model = installed_app.app_with_session(session=db.session())
|
||||
app_model = installed_app.app
|
||||
if app_model is None:
|
||||
raise AppUnavailableError()
|
||||
try:
|
||||
|
||||
@@ -90,7 +90,7 @@ class CompletionApi(InstalledAppResource):
|
||||
@with_current_user
|
||||
@with_session
|
||||
def post(self, session: Session, current_user: Account, installed_app: InstalledApp):
|
||||
app_model = installed_app.app_with_session(session=session)
|
||||
app_model = installed_app.app
|
||||
if app_model is None:
|
||||
raise AppUnavailableError()
|
||||
if app_model.mode != AppMode.COMPLETION:
|
||||
@@ -146,9 +146,8 @@ class CompletionApi(InstalledAppResource):
|
||||
class CompletionStopApi(InstalledAppResource):
|
||||
@console_ns.response(200, "Success", console_ns.models[SimpleResultResponse.__name__])
|
||||
@with_current_user_id
|
||||
@with_session(write=False)
|
||||
def post(self, session: Session, current_user_id: str, installed_app: InstalledApp, task_id: str):
|
||||
app_model = installed_app.app_with_session(session=session)
|
||||
def post(self, current_user_id: str, installed_app: InstalledApp, task_id: str):
|
||||
app_model = installed_app.app
|
||||
if app_model is None:
|
||||
raise AppUnavailableError()
|
||||
if app_model.mode != AppMode.COMPLETION:
|
||||
@@ -174,7 +173,7 @@ class ChatApi(InstalledAppResource):
|
||||
@with_current_user
|
||||
@with_session
|
||||
def post(self, session: Session, current_user: Account, installed_app: InstalledApp):
|
||||
app_model = installed_app.app_with_session(session=session)
|
||||
app_model = installed_app.app
|
||||
if app_model is None:
|
||||
raise AppUnavailableError()
|
||||
app_mode = AppMode.value_of(app_model.mode)
|
||||
@@ -196,7 +195,7 @@ class ChatApi(InstalledAppResource):
|
||||
app_model=app_model,
|
||||
conversation_id=payload.conversation_id,
|
||||
user=current_user,
|
||||
session=session,
|
||||
session=db.session(),
|
||||
)
|
||||
|
||||
response = AppGenerateService.generate(
|
||||
@@ -241,9 +240,8 @@ class ChatApi(InstalledAppResource):
|
||||
class ChatStopApi(InstalledAppResource):
|
||||
@console_ns.response(200, "Success", console_ns.models[SimpleResultResponse.__name__])
|
||||
@with_current_user_id
|
||||
@with_session(write=False)
|
||||
def post(self, session: Session, current_user_id: str, installed_app: InstalledApp, task_id: str):
|
||||
app_model = installed_app.app_with_session(session=session)
|
||||
def post(self, current_user_id: str, installed_app: InstalledApp, task_id: str):
|
||||
app_model = installed_app.app
|
||||
if app_model is None:
|
||||
raise AppUnavailableError()
|
||||
app_mode = AppMode.value_of(app_model.mode)
|
||||
|
||||
@@ -16,7 +16,6 @@ from core.app.entities.app_invoke_entities import InvokeFrom
|
||||
from extensions.ext_database import db
|
||||
from fields.conversation_fields import (
|
||||
ConversationInfiniteScrollPagination,
|
||||
ConversationResponseSource,
|
||||
ResultResponse,
|
||||
SimpleConversation,
|
||||
)
|
||||
@@ -54,7 +53,7 @@ class ConversationListApi(InstalledAppResource):
|
||||
@console_ns.response(200, "Success", console_ns.models[ConversationInfiniteScrollPagination.__name__])
|
||||
@with_current_user
|
||||
def get(self, current_user: Account, installed_app: InstalledApp):
|
||||
app_model = installed_app.app_with_session(session=db.session())
|
||||
app_model = installed_app.app
|
||||
if app_model is None:
|
||||
raise AppUnavailableError()
|
||||
app_mode = AppMode.value_of(app_model.mode)
|
||||
@@ -85,13 +84,7 @@ class ConversationListApi(InstalledAppResource):
|
||||
pinned=args.pinned,
|
||||
)
|
||||
adapter = TypeAdapter(SimpleConversation)
|
||||
conversations = [
|
||||
adapter.validate_python(
|
||||
ConversationResponseSource(item, session=session),
|
||||
from_attributes=True,
|
||||
)
|
||||
for item in pagination.data
|
||||
]
|
||||
conversations = [adapter.validate_python(item, from_attributes=True) for item in pagination.data]
|
||||
return ConversationInfiniteScrollPagination(
|
||||
limit=pagination.limit,
|
||||
has_more=pagination.has_more,
|
||||
@@ -109,7 +102,7 @@ class ConversationApi(InstalledAppResource):
|
||||
@console_ns.response(204, "Conversation deleted successfully")
|
||||
@with_current_user
|
||||
def delete(self, current_user: Account, installed_app: InstalledApp, c_id: UUID):
|
||||
app_model = installed_app.app_with_session(session=db.session())
|
||||
app_model = installed_app.app
|
||||
if app_model is None:
|
||||
raise AppUnavailableError()
|
||||
app_mode = AppMode.value_of(app_model.mode)
|
||||
@@ -134,7 +127,7 @@ class ConversationRenameApi(InstalledAppResource):
|
||||
@console_ns.response(200, "Conversation renamed successfully", console_ns.models[SimpleConversation.__name__])
|
||||
@with_current_user
|
||||
def post(self, current_user: Account, installed_app: InstalledApp, c_id: UUID):
|
||||
app_model = installed_app.app_with_session(session=db.session())
|
||||
app_model = installed_app.app
|
||||
if app_model is None:
|
||||
raise AppUnavailableError()
|
||||
app_mode = AppMode.value_of(app_model.mode)
|
||||
@@ -146,13 +139,12 @@ class ConversationRenameApi(InstalledAppResource):
|
||||
payload = ConversationRenamePayload.model_validate(console_ns.payload or {})
|
||||
|
||||
try:
|
||||
session = db.session()
|
||||
conversation = ConversationService.rename(
|
||||
app_model, conversation_id, current_user, payload.name, payload.auto_generate, session=session
|
||||
app_model, conversation_id, current_user, payload.name, payload.auto_generate, session=db.session()
|
||||
)
|
||||
return (
|
||||
TypeAdapter(SimpleConversation)
|
||||
.validate_python(ConversationResponseSource(conversation, session=session), from_attributes=True)
|
||||
.validate_python(conversation, from_attributes=True)
|
||||
.model_dump(mode="json")
|
||||
)
|
||||
except ConversationNotExistsError:
|
||||
@@ -167,7 +159,7 @@ class ConversationPinApi(InstalledAppResource):
|
||||
@console_ns.response(200, "Success", console_ns.models[ResultResponse.__name__])
|
||||
@with_current_user
|
||||
def patch(self, current_user: Account, installed_app: InstalledApp, c_id: UUID):
|
||||
app_model = installed_app.app_with_session(session=db.session())
|
||||
app_model = installed_app.app
|
||||
if app_model is None:
|
||||
raise AppUnavailableError()
|
||||
app_mode = AppMode.value_of(app_model.mode)
|
||||
@@ -192,7 +184,7 @@ class ConversationUnPinApi(InstalledAppResource):
|
||||
@console_ns.response(200, "Success", console_ns.models[ResultResponse.__name__])
|
||||
@with_current_user
|
||||
def patch(self, current_user: Account, installed_app: InstalledApp, c_id: UUID):
|
||||
app_model = installed_app.app_with_session(session=db.session())
|
||||
app_model = installed_app.app
|
||||
if app_model is None:
|
||||
raise AppUnavailableError()
|
||||
app_mode = AppMode.value_of(app_model.mode)
|
||||
|
||||
@@ -28,7 +28,7 @@ from controllers.console.wraps import with_current_user
|
||||
from core.app.entities.app_invoke_entities import InvokeFrom
|
||||
from core.errors.error import ModelCurrentlyNotSupportError, ProviderTokenNotInitError, QuotaExceededError
|
||||
from extensions.ext_database import db
|
||||
from fields.conversation_fields import MessageResponseSource, ResultResponse
|
||||
from fields.conversation_fields import ResultResponse
|
||||
from fields.message_fields import (
|
||||
ExploreMessageInfiniteScrollPagination,
|
||||
ExploreMessageListItem,
|
||||
@@ -76,8 +76,7 @@ class MessageListApi(InstalledAppResource):
|
||||
@console_ns.response(200, "Success", console_ns.models[ExploreMessageInfiniteScrollPagination.__name__])
|
||||
@with_current_user
|
||||
def get(self, current_user: Account, installed_app: InstalledApp):
|
||||
session = db.session()
|
||||
app_model = installed_app.app_with_session(session=session)
|
||||
app_model = installed_app.app
|
||||
if app_model is None:
|
||||
raise AppUnavailableError()
|
||||
|
||||
@@ -93,13 +92,10 @@ class MessageListApi(InstalledAppResource):
|
||||
args.conversation_id,
|
||||
args.first_id or None,
|
||||
args.limit,
|
||||
session=session,
|
||||
session=db.session(),
|
||||
)
|
||||
adapter = TypeAdapter(ExploreMessageListItem)
|
||||
items = [
|
||||
adapter.validate_python(MessageResponseSource(message, session=session), from_attributes=True)
|
||||
for message in pagination.data
|
||||
]
|
||||
items = [adapter.validate_python(message, from_attributes=True) for message in pagination.data]
|
||||
return ExploreMessageInfiniteScrollPagination(
|
||||
limit=pagination.limit,
|
||||
has_more=pagination.has_more,
|
||||
@@ -120,7 +116,7 @@ class MessageFeedbackApi(InstalledAppResource):
|
||||
@console_ns.response(200, "Feedback submitted successfully", console_ns.models[ResultResponse.__name__])
|
||||
@with_current_user
|
||||
def post(self, current_user: Account, installed_app: InstalledApp, message_id: UUID):
|
||||
app_model = installed_app.app_with_session(session=db.session())
|
||||
app_model = installed_app.app
|
||||
if app_model is None:
|
||||
raise AppUnavailableError()
|
||||
|
||||
@@ -153,7 +149,7 @@ class MessageMoreLikeThisApi(InstalledAppResource):
|
||||
@with_current_user
|
||||
@with_session
|
||||
def get(self, session: Session, current_user: Account, installed_app: InstalledApp, message_id: UUID):
|
||||
app_model = installed_app.app_with_session(session=session)
|
||||
app_model = installed_app.app
|
||||
if app_model is None:
|
||||
raise AppUnavailableError()
|
||||
if app_model.mode != "completion":
|
||||
@@ -203,7 +199,7 @@ class MessageSuggestedQuestionApi(InstalledAppResource):
|
||||
@console_ns.response(200, "Success", console_ns.models[SuggestedQuestionsResponse.__name__])
|
||||
@with_current_user
|
||||
def get(self, current_user: Account, installed_app: InstalledApp, message_id: UUID):
|
||||
app_model = installed_app.app_with_session(session=db.session())
|
||||
app_model = installed_app.app
|
||||
if app_model is None:
|
||||
raise AppUnavailableError()
|
||||
app_mode = AppMode.value_of(app_model.mode)
|
||||
|
||||
@@ -9,7 +9,7 @@ from controllers.console.app.error import AppUnavailableError
|
||||
from controllers.console.explore.wraps import InstalledAppResource
|
||||
from core.app.app_config.common.parameters_mapping import get_parameters_from_feature_dict
|
||||
from extensions.ext_database import db
|
||||
from models.model import AppMode, InstalledApp, load_annotation_reply_config
|
||||
from models.model import AppMode, InstalledApp
|
||||
from services.app_service import AppService
|
||||
|
||||
|
||||
@@ -32,29 +32,24 @@ class AppParameterApi(InstalledAppResource):
|
||||
@console_ns.response(200, "Success", console_ns.models[fields.Parameters.__name__])
|
||||
def get(self, installed_app: InstalledApp):
|
||||
"""Retrieve app parameters."""
|
||||
session = db.session()
|
||||
app_model = installed_app.app_with_session(session=session)
|
||||
app_model = installed_app.app
|
||||
|
||||
if app_model is None:
|
||||
raise AppUnavailableError()
|
||||
|
||||
if app_model.mode in {AppMode.ADVANCED_CHAT, AppMode.WORKFLOW}:
|
||||
workflow = app_model.workflow_with_session(session=session)
|
||||
workflow = app_model.workflow
|
||||
if workflow is None:
|
||||
raise AppUnavailableError()
|
||||
|
||||
features_dict: dict[str, Any] = workflow.features_dict
|
||||
user_input_form = workflow.user_input_form(to_old_structure=True)
|
||||
else:
|
||||
app_model_config = app_model.app_model_config_with_session(session=session)
|
||||
app_model_config = app_model.app_model_config
|
||||
if app_model_config is None:
|
||||
raise AppUnavailableError()
|
||||
|
||||
annotation_reply = load_annotation_reply_config(session, app_model.id)
|
||||
features_dict = cast(
|
||||
dict[str, Any],
|
||||
app_model_config.to_dict(annotation_reply=annotation_reply),
|
||||
)
|
||||
features_dict = cast(dict[str, Any], app_model_config.to_dict())
|
||||
|
||||
user_input_form = features_dict.get("user_input_form", [])
|
||||
|
||||
@@ -67,7 +62,7 @@ class ExploreAppMetaApi(InstalledAppResource):
|
||||
@console_ns.response(200, "Success", console_ns.models[ExploreAppMetaResponse.__name__])
|
||||
def get(self, installed_app: InstalledApp):
|
||||
"""Get app meta"""
|
||||
app_model = installed_app.app_with_session(session=db.session())
|
||||
app_model = installed_app.app
|
||||
if not app_model:
|
||||
raise ValueError("App not found")
|
||||
return AppService().get_app_meta(app_model, session=db.session())
|
||||
|
||||
@@ -12,7 +12,7 @@ from controllers.console.explore.error import NotCompletionAppError
|
||||
from controllers.console.explore.wraps import InstalledAppResource
|
||||
from controllers.console.wraps import with_current_user
|
||||
from extensions.ext_database import db
|
||||
from fields.conversation_fields import MessageResponseSource, ResultResponse
|
||||
from fields.conversation_fields import ResultResponse
|
||||
from fields.message_fields import SavedMessageInfiniteScrollPagination, SavedMessageItem
|
||||
from models import Account
|
||||
from models.model import InstalledApp
|
||||
@@ -29,8 +29,7 @@ class SavedMessageListApi(InstalledAppResource):
|
||||
@console_ns.response(200, "Success", console_ns.models[SavedMessageInfiniteScrollPagination.__name__])
|
||||
@with_current_user
|
||||
def get(self, current_user: Account, installed_app: InstalledApp):
|
||||
session = db.session()
|
||||
app_model = installed_app.app_with_session(session=session)
|
||||
app_model = installed_app.app
|
||||
if app_model is None:
|
||||
raise AppUnavailableError()
|
||||
if app_model.mode != "completion":
|
||||
@@ -39,13 +38,10 @@ class SavedMessageListApi(InstalledAppResource):
|
||||
args = SavedMessageListQuery.model_validate(request.args.to_dict())
|
||||
|
||||
pagination = SavedMessageService.pagination_by_last_id(
|
||||
app_model, current_user, str(args.last_id) if args.last_id else None, args.limit, session=session
|
||||
app_model, current_user, str(args.last_id) if args.last_id else None, args.limit, session=db.session()
|
||||
)
|
||||
adapter = TypeAdapter(SavedMessageItem)
|
||||
items = [
|
||||
adapter.validate_python(MessageResponseSource(message, session=session), from_attributes=True)
|
||||
for message in pagination.data
|
||||
]
|
||||
items = [adapter.validate_python(message, from_attributes=True) for message in pagination.data]
|
||||
return SavedMessageInfiniteScrollPagination(
|
||||
limit=pagination.limit,
|
||||
has_more=pagination.has_more,
|
||||
@@ -56,7 +52,7 @@ class SavedMessageListApi(InstalledAppResource):
|
||||
@console_ns.response(200, "Success", console_ns.models[ResultResponse.__name__])
|
||||
@with_current_user
|
||||
def post(self, current_user: Account, installed_app: InstalledApp):
|
||||
app_model = installed_app.app_with_session(session=db.session())
|
||||
app_model = installed_app.app
|
||||
if app_model is None:
|
||||
raise AppUnavailableError()
|
||||
if app_model.mode != "completion":
|
||||
@@ -79,7 +75,7 @@ class SavedMessageApi(InstalledAppResource):
|
||||
@console_ns.response(204, "Saved message deleted successfully")
|
||||
@with_current_user
|
||||
def delete(self, current_user: Account, installed_app: InstalledApp, message_id: UUID):
|
||||
app_model = installed_app.app_with_session(session=db.session())
|
||||
app_model = installed_app.app
|
||||
if app_model is None:
|
||||
raise AppUnavailableError()
|
||||
|
||||
|
||||
@@ -1,6 +1,4 @@
|
||||
import logging
|
||||
from collections.abc import Mapping
|
||||
from dataclasses import dataclass
|
||||
from datetime import datetime
|
||||
from typing import Any, Literal
|
||||
|
||||
@@ -47,9 +45,7 @@ from controllers.console.explore.error import (
|
||||
NotWorkflowAppError,
|
||||
)
|
||||
from controllers.console.explore.wraps import TrialAppResource, trial_feature_enable
|
||||
from controllers.console.files import FILE_UPLOAD_PARAMS, upload_file_from_request
|
||||
from controllers.console.remote_files import RemoteFileUploadPayload, upload_remote_file_from_request
|
||||
from controllers.console.wraps import cloud_edition_billing_resource_check, with_current_user
|
||||
from controllers.console.wraps import with_current_user
|
||||
from controllers.web.error import InvokeRateLimitError as InvokeRateLimitHttpError
|
||||
from core.app.app_config.common.parameters_mapping import get_parameters_from_feature_dict
|
||||
from core.app.apps.base_app_queue_manager import AppQueueManager
|
||||
@@ -62,21 +58,18 @@ from core.errors.error import (
|
||||
from extensions.ext_database import db
|
||||
from extensions.ext_redis import redis_client
|
||||
from fields.base import ResponseModel
|
||||
from fields.conversation_variable_fields import WorkflowConversationVariableResponse
|
||||
from fields.file_fields import FileResponse, FileWithSignedUrl
|
||||
from fields.message_fields import SuggestedQuestionsResponse
|
||||
from graphon.graph_engine.manager import GraphEngineManager
|
||||
from graphon.model_runtime.errors.invoke import InvokeError
|
||||
from libs import helper
|
||||
from libs.helper import dump_response, to_timestamp, uuid_value
|
||||
from models import Account, App
|
||||
from models import Account
|
||||
from models.account import TenantStatus
|
||||
from models.model import AppMode, Site, load_annotation_reply_config
|
||||
from models.model import AppMode, Site
|
||||
from models.workflow import Workflow
|
||||
from services.account_service import TenantService
|
||||
from services.app_generate_service import AppGenerateService
|
||||
from services.app_ref_service import AppRefService
|
||||
from services.app_service import AppResponseView, AppService
|
||||
from services.app_service import AppService
|
||||
from services.audio_service import AudioService
|
||||
from services.dataset_service import DatasetService
|
||||
from services.errors.audio import (
|
||||
@@ -377,7 +370,7 @@ class TrialWorkflowResponse(ResponseModel):
|
||||
updated_at: int | None = None
|
||||
tool_published: bool | None = None
|
||||
environment_variables: list[JsonObject] = Field(default_factory=list)
|
||||
conversation_variables: list[WorkflowConversationVariableResponse] = Field(default_factory=list)
|
||||
conversation_variables: list[JsonObject] = Field(default_factory=list)
|
||||
rag_pipeline_variables: list[JsonObject] = Field(default_factory=list)
|
||||
|
||||
@field_validator("created_at", "updated_at", mode="before")
|
||||
@@ -386,27 +379,6 @@ class TrialWorkflowResponse(ResponseModel):
|
||||
return to_timestamp(value)
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class TrialWorkflowResponseSource:
|
||||
workflow: Workflow
|
||||
session: Session
|
||||
|
||||
@property
|
||||
def created_by_account(self) -> Account | None:
|
||||
return self.workflow.get_created_by_account(session=self.session)
|
||||
|
||||
@property
|
||||
def updated_by_account(self) -> Account | None:
|
||||
return self.workflow.get_updated_by_account(session=self.session)
|
||||
|
||||
@property
|
||||
def tool_published(self) -> bool:
|
||||
return self.workflow.get_tool_published(session=self.session)
|
||||
|
||||
def __getattr__(self, name: str) -> Any:
|
||||
return getattr(self.workflow, name) # noqa: no-new-getattr response adapter delegates model fields
|
||||
|
||||
|
||||
register_schema_models(
|
||||
console_ns,
|
||||
WorkflowRunRequest,
|
||||
@@ -431,36 +403,6 @@ register_response_schema_models(
|
||||
simple_account_model = console_ns.models[TrialSimpleAccount.__name__]
|
||||
|
||||
|
||||
class TrialAppFileUploadApi(TrialAppResource):
|
||||
@trial_feature_enable
|
||||
@cloud_edition_billing_resource_check("documents")
|
||||
@console_ns.doc(consumes=["multipart/form-data"], params=FILE_UPLOAD_PARAMS)
|
||||
@console_ns.response(201, "File uploaded successfully", console_ns.models[FileResponse.__name__])
|
||||
@with_current_user
|
||||
def post(self, current_user: Account, app_model: App):
|
||||
"""Upload a file into the tenant that owns the trial app."""
|
||||
upload_file = upload_file_from_request(
|
||||
current_user=current_user,
|
||||
resource_tenant_id=app_model.tenant_id,
|
||||
)
|
||||
return dump_response(FileResponse, upload_file), 201
|
||||
|
||||
|
||||
class TrialAppRemoteFileUploadApi(TrialAppResource):
|
||||
@trial_feature_enable
|
||||
@cloud_edition_billing_resource_check("documents")
|
||||
@console_ns.expect(console_ns.models[RemoteFileUploadPayload.__name__])
|
||||
@console_ns.response(201, "File uploaded successfully", console_ns.models[FileWithSignedUrl.__name__])
|
||||
@with_current_user
|
||||
def post(self, current_user: Account, app_model: App):
|
||||
"""Upload a remote file into the tenant that owns the trial app."""
|
||||
remote_file = upload_remote_file_from_request(
|
||||
current_user=current_user,
|
||||
resource_tenant_id=app_model.tenant_id,
|
||||
)
|
||||
return remote_file.model_dump(mode="json"), 201
|
||||
|
||||
|
||||
class TrialAppWorkflowRunApi(TrialAppResource):
|
||||
@trial_feature_enable
|
||||
@console_ns.expect(console_ns.models[WorkflowRunRequest.__name__])
|
||||
@@ -645,19 +587,14 @@ class TrialChatAudioApi(TrialAppResource):
|
||||
def post(self, current_user: Account, trial_app):
|
||||
app_model = trial_app
|
||||
|
||||
file = request.files.get("file")
|
||||
file = request.files["file"]
|
||||
|
||||
try:
|
||||
# Get IDs before they might be detached from session
|
||||
app_id = app_model.id
|
||||
user_id = current_user.id
|
||||
|
||||
response = AudioService.transcript_asr(
|
||||
app_model=app_model,
|
||||
file=file,
|
||||
session=db.session(),
|
||||
end_user=None,
|
||||
)
|
||||
response = AudioService.transcript_asr(app_model=app_model, file=file, end_user=None)
|
||||
RecommendedAppService.add_trial_app_record(app_id, user_id, session=db.session())
|
||||
return response
|
||||
except services.errors.app_model_config.AppModelConfigBrokenError:
|
||||
@@ -809,21 +746,19 @@ class TrialSitApi(Resource):
|
||||
"""Resource for trial app sites."""
|
||||
|
||||
@console_ns.response(200, "Success", console_ns.models[SiteResponse.__name__])
|
||||
@with_session(write=False)
|
||||
@get_app_model_with_trial(None)
|
||||
def get(self, session: Session, app_model):
|
||||
def get(self, app_model):
|
||||
"""Retrieve app site info.
|
||||
|
||||
Returns the site configuration for the application including theme, icons, and text.
|
||||
"""
|
||||
site = session.scalar(select(Site).where(Site.app_id == app_model.id).limit(1))
|
||||
site = db.session.scalar(select(Site).where(Site.app_id == app_model.id).limit(1))
|
||||
|
||||
if not site:
|
||||
raise Forbidden()
|
||||
|
||||
tenant = TenantService.get_tenant_by_id(app_model.tenant_id, session=session)
|
||||
assert tenant
|
||||
if tenant.status == TenantStatus.ARCHIVE:
|
||||
assert app_model.tenant
|
||||
if app_model.tenant.status == TenantStatus.ARCHIVE:
|
||||
raise Forbidden()
|
||||
|
||||
return SiteResponse.model_validate(site).model_dump(mode="json")
|
||||
@@ -833,29 +768,26 @@ class TrialAppParameterApi(Resource):
|
||||
"""Resource for app variables."""
|
||||
|
||||
@console_ns.response(200, "Success", console_ns.models[ParametersResponse.__name__])
|
||||
@with_session(write=False)
|
||||
@get_app_model_with_trial(None)
|
||||
def get(self, session: Session, app_model):
|
||||
def get(self, app_model):
|
||||
"""Retrieve app parameters."""
|
||||
|
||||
if app_model is None:
|
||||
raise AppUnavailableError()
|
||||
|
||||
features_dict: Mapping[str, Any]
|
||||
if app_model.mode in {AppMode.ADVANCED_CHAT, AppMode.WORKFLOW}:
|
||||
workflow = app_model.workflow_with_session(session=session)
|
||||
workflow = app_model.workflow
|
||||
if workflow is None:
|
||||
raise AppUnavailableError()
|
||||
|
||||
features_dict = workflow.features_dict
|
||||
user_input_form = workflow.user_input_form(to_old_structure=True)
|
||||
else:
|
||||
app_model_config = app_model.app_model_config_with_session(session=session)
|
||||
app_model_config = app_model.app_model_config
|
||||
if app_model_config is None:
|
||||
raise AppUnavailableError()
|
||||
|
||||
annotation_reply = load_annotation_reply_config(session, app_model_config.app_id)
|
||||
features_dict = app_model_config.to_dict(annotation_reply=annotation_reply)
|
||||
features_dict = app_model_config.to_dict()
|
||||
|
||||
user_input_form = features_dict.get("user_input_form", [])
|
||||
|
||||
@@ -865,52 +797,43 @@ class TrialAppParameterApi(Resource):
|
||||
|
||||
class AppApi(Resource):
|
||||
@console_ns.response(200, "Success", console_ns.models[TrialAppDetailResponse.__name__])
|
||||
@with_session(write=False)
|
||||
@get_app_model_with_trial(None)
|
||||
def get(self, session: Session, app_model):
|
||||
def get(self, app_model):
|
||||
"""Get app detail"""
|
||||
|
||||
app_service = AppService()
|
||||
app_model = app_service.get_app(app_model, session=session)
|
||||
app_model = app_service.get_app(app_model)
|
||||
|
||||
return TrialAppDetailResponse.model_validate(
|
||||
AppResponseView(app_model, session=session),
|
||||
from_attributes=True,
|
||||
).model_dump(mode="json")
|
||||
return dump_response(TrialAppDetailResponse, app_model)
|
||||
|
||||
|
||||
class AppWorkflowApi(Resource):
|
||||
@console_ns.response(200, "Success", console_ns.models[TrialWorkflowResponse.__name__])
|
||||
@with_session(write=False)
|
||||
@get_app_model_with_trial(None)
|
||||
def get(self, session: Session, app_model):
|
||||
def get(self, app_model):
|
||||
"""Get workflow detail"""
|
||||
if not app_model.workflow_id:
|
||||
raise AppUnavailableError()
|
||||
|
||||
workflow = app_model.workflow_with_session(session=session)
|
||||
workflow = db.session.get(Workflow, app_model.workflow_id)
|
||||
if workflow is None:
|
||||
raise AppUnavailableError()
|
||||
|
||||
return TrialWorkflowResponse.model_validate(
|
||||
TrialWorkflowResponseSource(workflow=workflow, session=session),
|
||||
from_attributes=True,
|
||||
).model_dump(mode="json")
|
||||
return dump_response(TrialWorkflowResponse, workflow)
|
||||
|
||||
|
||||
class DatasetListApi(Resource):
|
||||
@console_ns.doc(params=query_params_from_model(TrialDatasetListQuery))
|
||||
@console_ns.response(200, "Success", console_ns.models[TrialDatasetListResponse.__name__])
|
||||
@with_session(write=False)
|
||||
@get_app_model_with_trial(None)
|
||||
def get(self, session: Session, app_model):
|
||||
def get(self, app_model):
|
||||
page = request.args.get("page", default=1, type=int)
|
||||
limit = request.args.get("limit", default=20, type=int)
|
||||
ids = request.args.getlist("ids")
|
||||
|
||||
tenant_id = app_model.tenant_id
|
||||
if ids:
|
||||
datasets, total = DatasetService.get_datasets_by_ids(ids, tenant_id, session=session)
|
||||
datasets, total = DatasetService.get_datasets_by_ids(ids, tenant_id)
|
||||
else:
|
||||
raise NeedAddIdsError()
|
||||
|
||||
@@ -920,18 +843,6 @@ class DatasetListApi(Resource):
|
||||
|
||||
console_ns.add_resource(TrialChatApi, "/trial-apps/<uuid:app_id>/chat-messages", endpoint="trial_app_chat_completion")
|
||||
|
||||
console_ns.add_resource(
|
||||
TrialAppFileUploadApi,
|
||||
"/trial-apps/<uuid:app_id>/files/upload",
|
||||
endpoint="trial_app_file_upload",
|
||||
)
|
||||
|
||||
console_ns.add_resource(
|
||||
TrialAppRemoteFileUploadApi,
|
||||
"/trial-apps/<uuid:app_id>/remote-files/upload",
|
||||
endpoint="trial_app_remote_file_upload",
|
||||
)
|
||||
|
||||
console_ns.add_resource(
|
||||
TrialMessageSuggestedQuestionApi,
|
||||
"/trial-apps/<uuid:app_id>/messages/<uuid:message_id>/suggested-questions",
|
||||
|
||||
@@ -51,7 +51,7 @@ class InstalledAppWorkflowRunApi(InstalledAppResource):
|
||||
"""
|
||||
Run workflow
|
||||
"""
|
||||
app_model = installed_app.app_with_session(session=session)
|
||||
app_model = installed_app.app
|
||||
if not app_model:
|
||||
raise NotWorkflowAppError()
|
||||
app_mode = AppMode.value_of(app_model.mode)
|
||||
@@ -92,12 +92,11 @@ class InstalledAppWorkflowRunApi(InstalledAppResource):
|
||||
@console_ns.route("/installed-apps/<uuid:installed_app_id>/workflows/tasks/<string:task_id>/stop")
|
||||
class InstalledAppWorkflowTaskStopApi(InstalledAppResource):
|
||||
@console_ns.response(200, "Success", console_ns.models[SimpleResultResponse.__name__])
|
||||
@with_session(write=False)
|
||||
def post(self, session: Session, installed_app: InstalledApp, task_id: str):
|
||||
def post(self, installed_app: InstalledApp, task_id: str):
|
||||
"""
|
||||
Stop workflow task
|
||||
"""
|
||||
app_model = installed_app.app_with_session(session=session)
|
||||
app_model = installed_app.app
|
||||
if not app_model:
|
||||
raise NotWorkflowAppError()
|
||||
app_mode = AppMode.value_of(app_model.mode)
|
||||
|
||||
@@ -30,7 +30,7 @@ def installed_app_required[**P, R](view: Callable[Concatenate[InstalledApp, P],
|
||||
if installed_app is None:
|
||||
raise NotFound("Installed app not found")
|
||||
|
||||
if not installed_app.app_with_session(session=db.session()):
|
||||
if not installed_app.app:
|
||||
db.session.delete(installed_app)
|
||||
db.session.commit()
|
||||
|
||||
@@ -74,18 +74,17 @@ def trial_app_required[**P, R](view: Callable[Concatenate[App, P], R] | None = N
|
||||
@wraps(view)
|
||||
def decorated(app_id: str, *args: P.args, **kwargs: P.kwargs):
|
||||
current_user, _ = current_account_with_tenant()
|
||||
session = db.session()
|
||||
|
||||
trial_app = session.scalar(select(TrialApp).where(TrialApp.app_id == str(app_id)).limit(1))
|
||||
trial_app = db.session.scalar(select(TrialApp).where(TrialApp.app_id == str(app_id)).limit(1))
|
||||
|
||||
if trial_app is None:
|
||||
raise TrialAppNotAllowed()
|
||||
app = trial_app.app_with_session(session=session)
|
||||
app = trial_app.app
|
||||
|
||||
if app is None:
|
||||
raise TrialAppNotAllowed()
|
||||
|
||||
account_trial_app_record = session.scalar(
|
||||
account_trial_app_record = db.session.scalar(
|
||||
select(AccountTrialAppRecord)
|
||||
.where(AccountTrialAppRecord.account_id == current_user.id, AccountTrialAppRecord.app_id == app_id)
|
||||
.limit(1)
|
||||
|
||||
@@ -29,7 +29,7 @@ from extensions.ext_database import db
|
||||
from fields.file_fields import FileResponse, UploadConfig
|
||||
from libs.helper import dump_response
|
||||
from libs.login import login_required
|
||||
from models import Account, UploadFile
|
||||
from models import Account
|
||||
from services.file_service import FileService
|
||||
|
||||
from . import console_ns
|
||||
@@ -39,7 +39,7 @@ register_response_schema_models(console_ns, AllowedExtensionsResponse, TextConte
|
||||
|
||||
PREVIEW_WORDS_LIMIT = 3000
|
||||
|
||||
FILE_UPLOAD_PARAMS = {
|
||||
_FILE_UPLOAD_PARAMS = {
|
||||
"file": {
|
||||
"description": "File to upload",
|
||||
"in": "formData",
|
||||
@@ -56,43 +56,6 @@ FILE_UPLOAD_PARAMS = {
|
||||
}
|
||||
|
||||
|
||||
def upload_file_from_request(*, current_user: Account, resource_tenant_id: str | None = None) -> UploadFile:
|
||||
"""Validate the multipart request and persist the file under the requested resource tenant."""
|
||||
source_str = request.form.get("source")
|
||||
source: Literal["datasets"] | None = "datasets" if source_str == "datasets" else None
|
||||
|
||||
if "file" not in request.files:
|
||||
raise NoFileUploadedError()
|
||||
|
||||
if len(request.files) > 1:
|
||||
raise TooManyFilesError()
|
||||
file = request.files["file"]
|
||||
|
||||
if not file.filename:
|
||||
raise FilenameNotExistsError
|
||||
if source == "datasets" and not current_user.is_dataset_editor:
|
||||
raise Forbidden()
|
||||
|
||||
if source not in ("datasets", None):
|
||||
source = None
|
||||
|
||||
try:
|
||||
return FileService(db.engine).upload_file(
|
||||
filename=file.filename,
|
||||
content=file.stream.read(),
|
||||
mimetype=file.mimetype,
|
||||
user=current_user,
|
||||
tenant_id=resource_tenant_id,
|
||||
source=source,
|
||||
)
|
||||
except services.errors.file.FileTooLargeError as file_too_large_error:
|
||||
raise FileTooLargeError(file_too_large_error.description)
|
||||
except services.errors.file.UnsupportedFileTypeError:
|
||||
raise UnsupportedFileTypeError()
|
||||
except services.errors.file.BlockedFileExtensionError as blocked_extension_error:
|
||||
raise BlockedFileExtensionError(blocked_extension_error.description)
|
||||
|
||||
|
||||
@console_ns.route("/files/upload")
|
||||
class FileApi(Resource):
|
||||
@setup_required
|
||||
@@ -118,11 +81,42 @@ class FileApi(Resource):
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@cloud_edition_billing_resource_check("documents")
|
||||
@console_ns.doc(consumes=["multipart/form-data"], params=FILE_UPLOAD_PARAMS)
|
||||
@console_ns.doc(consumes=["multipart/form-data"], params=_FILE_UPLOAD_PARAMS)
|
||||
@console_ns.response(201, "File uploaded successfully", console_ns.models[FileResponse.__name__])
|
||||
@with_current_user
|
||||
def post(self, current_user: Account):
|
||||
upload_file = upload_file_from_request(current_user=current_user)
|
||||
source_str = request.form.get("source")
|
||||
source: Literal["datasets"] | None = "datasets" if source_str == "datasets" else None
|
||||
|
||||
if "file" not in request.files:
|
||||
raise NoFileUploadedError()
|
||||
|
||||
if len(request.files) > 1:
|
||||
raise TooManyFilesError()
|
||||
file = request.files["file"]
|
||||
|
||||
if not file.filename:
|
||||
raise FilenameNotExistsError
|
||||
if source == "datasets" and not current_user.is_dataset_editor:
|
||||
raise Forbidden()
|
||||
|
||||
if source not in ("datasets", None):
|
||||
source = None
|
||||
|
||||
try:
|
||||
upload_file = FileService(db.engine).upload_file(
|
||||
filename=file.filename,
|
||||
content=file.stream.read(),
|
||||
mimetype=file.mimetype,
|
||||
user=current_user,
|
||||
source=source,
|
||||
)
|
||||
except services.errors.file.FileTooLargeError as file_too_large_error:
|
||||
raise FileTooLargeError(file_too_large_error.description)
|
||||
except services.errors.file.UnsupportedFileTypeError:
|
||||
raise UnsupportedFileTypeError()
|
||||
except services.errors.file.BlockedFileExtensionError as blocked_extension_error:
|
||||
raise BlockedFileExtensionError(blocked_extension_error.description)
|
||||
|
||||
return dump_response(FileResponse, upload_file), 201
|
||||
|
||||
|
||||
@@ -1,419 +0,0 @@
|
||||
"""Authenticated transport adapter for the Console-to-KnowledgeFS proxy.
|
||||
|
||||
These raw Blueprint routes deliberately stay outside Dify's OpenAPI surface:
|
||||
KnowledgeFS owns the wire contract consumed by the frontend. The catch-all path
|
||||
avoids resource-specific Dify controllers, while the forwarding module consumes
|
||||
only the operations explicitly enabled by Dify's product registry. The registry
|
||||
can be validated explicitly against the pinned KnowledgeFS contract during development.
|
||||
Console auth and contract-specific dataset RBAC run before forwarding. Request
|
||||
bodies are capped at 64 MiB, JSON and binary responses have separate bounds,
|
||||
SSE responses remain streaming with a bounded idle read timeout, and only safe
|
||||
response headers are exposed. Operation-specific upstream error mappings are
|
||||
applied before Console JSON error handling; the default maps 401 to 502 so it
|
||||
cannot trigger browser-session recovery and preserves resource-level 403.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
from collections.abc import Callable, Iterator
|
||||
from functools import wraps
|
||||
from http import HTTPStatus
|
||||
from typing import NoReturn, cast
|
||||
|
||||
import httpx
|
||||
from flask import Response, request, stream_with_context
|
||||
from flask.typing import ResponseReturnValue
|
||||
from werkzeug.exceptions import (
|
||||
BadGateway,
|
||||
Forbidden,
|
||||
GatewayTimeout,
|
||||
HTTPException,
|
||||
NotFound,
|
||||
RequestEntityTooLarge,
|
||||
ServiceUnavailable,
|
||||
default_exceptions,
|
||||
)
|
||||
|
||||
from configs import dify_config
|
||||
from controllers.console import api, bp
|
||||
from controllers.console.wraps import (
|
||||
account_initialization_required,
|
||||
cloud_edition_billing_rate_limit_check,
|
||||
setup_required,
|
||||
)
|
||||
from core.helper import ssrf_proxy
|
||||
from libs.login import current_account_with_tenant, login_required
|
||||
from services.knowledge_fs_operations import KnowledgeFSMethod
|
||||
from services.knowledge_fs_proxy import (
|
||||
KnowledgeFSAccessDeniedError,
|
||||
KnowledgeFSAuthorization,
|
||||
KnowledgeFSConfigurationError,
|
||||
KnowledgeFSRouteNotAllowedError,
|
||||
KnowledgeFSTimeoutError,
|
||||
KnowledgeFSTransportError,
|
||||
KnowledgeFSUpstreamResponse,
|
||||
authorize_knowledge_fs_request,
|
||||
get_knowledge_fs_operation,
|
||||
proxy_authorized_knowledge_fs_request,
|
||||
proxy_knowledge_fs_request,
|
||||
)
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
type _KnowledgeFSRequestForwarder = Callable[
|
||||
[str | None, str | None, bytes | None, bytes | None],
|
||||
KnowledgeFSUpstreamResponse,
|
||||
]
|
||||
|
||||
_MAX_PROXY_BODY_BYTES = 64 * 1024 * 1024
|
||||
_RESPONSE_HEADER_ALLOWLIST = (
|
||||
"Cache-Control",
|
||||
"Content-Disposition",
|
||||
"Content-Type",
|
||||
"Retry-After",
|
||||
"X-Trace-Id",
|
||||
)
|
||||
_RESPONSE_HEADER_DENYLIST = frozenset(
|
||||
{
|
||||
"authorization",
|
||||
"connection",
|
||||
"cookie",
|
||||
"keep-alive",
|
||||
"proxy-authenticate",
|
||||
"proxy-authorization",
|
||||
"set-cookie",
|
||||
"te",
|
||||
"trailer",
|
||||
"transfer-encoding",
|
||||
"upgrade",
|
||||
}
|
||||
)
|
||||
|
||||
|
||||
def _console_api_errors[**P](
|
||||
view: Callable[P, ResponseReturnValue],
|
||||
) -> Callable[P, ResponseReturnValue]:
|
||||
"""Route raw Blueprint exceptions through the Console API JSON handlers."""
|
||||
|
||||
@wraps(view)
|
||||
def decorated(*args: P.args, **kwargs: P.kwargs) -> ResponseReturnValue:
|
||||
try:
|
||||
return view(*args, **kwargs)
|
||||
except Exception as exc:
|
||||
return api.handle_error(exc)
|
||||
|
||||
return decorated
|
||||
|
||||
|
||||
def _knowledge_fs_enabled[**P](
|
||||
view: Callable[P, ResponseReturnValue],
|
||||
) -> Callable[P, ResponseReturnValue]:
|
||||
"""Hide the complete KnowledgeFS route surface while the bridge is disabled."""
|
||||
|
||||
@wraps(view)
|
||||
def decorated(*args: P.args, **kwargs: P.kwargs) -> ResponseReturnValue:
|
||||
if not dify_config.KNOWLEDGE_FS_ENABLED:
|
||||
raise NotFound()
|
||||
return view(*args, **kwargs)
|
||||
|
||||
return decorated
|
||||
|
||||
|
||||
def _translate_proxy_error(exc: Exception, *, tenant_id: str) -> NoReturn:
|
||||
"""Map forwarding failures to the stable Console HTTP error surface."""
|
||||
if isinstance(exc, KnowledgeFSRouteNotAllowedError):
|
||||
raise NotFound() from exc
|
||||
if isinstance(exc, KnowledgeFSAccessDeniedError):
|
||||
raise Forbidden() from exc
|
||||
if isinstance(exc, KnowledgeFSConfigurationError):
|
||||
logger.error("KnowledgeFS request was blocked by invalid configuration for tenant_id=%s", tenant_id)
|
||||
raise ServiceUnavailable("KnowledgeFS integration is misconfigured") from exc
|
||||
if isinstance(exc, KnowledgeFSTimeoutError):
|
||||
raise GatewayTimeout("KnowledgeFS request timed out") from exc
|
||||
if isinstance(exc, KnowledgeFSTransportError):
|
||||
logger.warning("KnowledgeFS transport request failed for tenant_id=%s", tenant_id)
|
||||
raise BadGateway("KnowledgeFS is unavailable") from exc
|
||||
raise exc
|
||||
|
||||
|
||||
def _knowledge_fs_operation_access_required(
|
||||
view: Callable[[KnowledgeFSAuthorization], ResponseReturnValue],
|
||||
) -> Callable[[KnowledgeFSMethod, str], ResponseReturnValue]:
|
||||
"""Authorize one declared operation before billing and request-body work."""
|
||||
|
||||
@wraps(view)
|
||||
def decorated(method: KnowledgeFSMethod, upstream_path: str) -> ResponseReturnValue:
|
||||
current_user, tenant_id = current_account_with_tenant()
|
||||
try:
|
||||
authorization = authorize_knowledge_fs_request(
|
||||
account=current_user,
|
||||
tenant_id=tenant_id,
|
||||
method=method,
|
||||
path=upstream_path,
|
||||
)
|
||||
except KnowledgeFSRouteNotAllowedError as exc:
|
||||
raise NotFound() from exc
|
||||
except KnowledgeFSAccessDeniedError as exc:
|
||||
_translate_proxy_error(exc, tenant_id=tenant_id)
|
||||
return view(authorization)
|
||||
|
||||
return decorated
|
||||
|
||||
|
||||
def _request_body() -> bytes:
|
||||
"""Read the raw body up to the proxy limit or raise RequestEntityTooLarge."""
|
||||
body = request.stream.read(_MAX_PROXY_BODY_BYTES + 1)
|
||||
if len(body) > _MAX_PROXY_BODY_BYTES:
|
||||
raise RequestEntityTooLarge("KnowledgeFS proxy request body is too large")
|
||||
return body
|
||||
|
||||
|
||||
def _stream_response_body(
|
||||
upstream: httpx.Response,
|
||||
*,
|
||||
tenant_id: str,
|
||||
max_response_bytes: int,
|
||||
) -> Iterator[bytes]:
|
||||
"""Yield one bounded SSE response and always release its pooled connection."""
|
||||
total_bytes = 0
|
||||
try:
|
||||
for chunk in upstream.iter_bytes():
|
||||
total_bytes += len(chunk)
|
||||
if total_bytes > max_response_bytes:
|
||||
logger.warning("KnowledgeFS stream exceeded the proxy limit for tenant_id=%s", tenant_id)
|
||||
raise ssrf_proxy.ResponseTooLargeError(f"response exceeded {max_response_bytes} bytes")
|
||||
yield chunk
|
||||
finally:
|
||||
upstream.close()
|
||||
|
||||
|
||||
def _proxy_response(
|
||||
upstream_result: KnowledgeFSUpstreamResponse,
|
||||
*,
|
||||
tenant_id: str,
|
||||
contract_response_headers: tuple[str, ...],
|
||||
max_response_bytes: int,
|
||||
) -> Response:
|
||||
"""Expose raw content, status, and allowlisted headers from KnowledgeFS.
|
||||
|
||||
Raises:
|
||||
HTTPException: KnowledgeFS returns a status normalized by the operation contract.
|
||||
"""
|
||||
upstream = upstream_result.response
|
||||
mapped_status = dict(upstream_result.operation.error_status_map).get(upstream.status_code)
|
||||
if mapped_status is not None:
|
||||
upstream.close()
|
||||
description = "KnowledgeFS upstream request failed"
|
||||
if upstream.status_code == HTTPStatus.UNAUTHORIZED:
|
||||
description = "KnowledgeFS authentication failed"
|
||||
logger.error(
|
||||
"KnowledgeFS rejected the Dify server credential with HTTP %s for tenant_id=%s",
|
||||
upstream.status_code,
|
||||
tenant_id,
|
||||
)
|
||||
exception_type = default_exceptions.get(mapped_status)
|
||||
if exception_type is None:
|
||||
exception = HTTPException(description)
|
||||
exception.code = mapped_status
|
||||
raise exception
|
||||
raise exception_type(description)
|
||||
|
||||
allowed_header_names = dict.fromkeys(
|
||||
name.lower() for name in (*_RESPONSE_HEADER_ALLOWLIST, *contract_response_headers)
|
||||
)
|
||||
headers = {
|
||||
name: value
|
||||
for name in allowed_header_names
|
||||
if name not in _RESPONSE_HEADER_DENYLIST
|
||||
if (value := upstream.headers.get(name)) is not None
|
||||
}
|
||||
if upstream_result.response_kind == "stream":
|
||||
response = Response(
|
||||
stream_with_context( # pyrefly: ignore[no-matching-overload]
|
||||
_stream_response_body(
|
||||
upstream,
|
||||
tenant_id=tenant_id,
|
||||
max_response_bytes=max_response_bytes,
|
||||
)
|
||||
),
|
||||
status=upstream.status_code,
|
||||
headers=headers,
|
||||
)
|
||||
response.call_on_close(upstream.close)
|
||||
return response
|
||||
|
||||
try:
|
||||
content = upstream.content
|
||||
finally:
|
||||
upstream.close()
|
||||
return Response(content, status=upstream.status_code, headers=headers)
|
||||
|
||||
|
||||
def _proxy_current_request(
|
||||
*,
|
||||
method: KnowledgeFSMethod,
|
||||
tenant_id: str,
|
||||
forward: _KnowledgeFSRequestForwarder,
|
||||
) -> Response:
|
||||
"""Forward the current raw request through one preconfigured service entry."""
|
||||
if not dify_config.KNOWLEDGE_FS_ENABLED:
|
||||
raise NotFound()
|
||||
try:
|
||||
proxy_result = forward(
|
||||
request.headers.get("Accept"),
|
||||
request.content_type,
|
||||
request.query_string or None,
|
||||
_request_body() if method != "GET" else None,
|
||||
)
|
||||
except (
|
||||
KnowledgeFSConfigurationError,
|
||||
KnowledgeFSAccessDeniedError,
|
||||
KnowledgeFSRouteNotAllowedError,
|
||||
KnowledgeFSTimeoutError,
|
||||
KnowledgeFSTransportError,
|
||||
) as exc:
|
||||
_translate_proxy_error(exc, tenant_id=tenant_id)
|
||||
return _proxy_response(
|
||||
proxy_result,
|
||||
tenant_id=tenant_id,
|
||||
contract_response_headers=proxy_result.operation.response_headers,
|
||||
max_response_bytes=proxy_result.operation.max_response_bytes,
|
||||
)
|
||||
|
||||
|
||||
def _proxy_request(
|
||||
method: KnowledgeFSMethod,
|
||||
upstream_path: str,
|
||||
) -> Response:
|
||||
"""Authorize and forward the current request through the combined service use case."""
|
||||
if not dify_config.KNOWLEDGE_FS_ENABLED:
|
||||
raise NotFound()
|
||||
current_user, tenant_id = current_account_with_tenant()
|
||||
|
||||
def forward(
|
||||
accept: str | None,
|
||||
content_type: str | None,
|
||||
query: bytes | None,
|
||||
body: bytes | None,
|
||||
) -> KnowledgeFSUpstreamResponse:
|
||||
return proxy_knowledge_fs_request(
|
||||
account=current_user,
|
||||
method=method,
|
||||
path=upstream_path,
|
||||
tenant_id=tenant_id,
|
||||
accept=accept,
|
||||
content_type=content_type,
|
||||
query=query,
|
||||
body=body,
|
||||
request_headers=request.headers,
|
||||
)
|
||||
|
||||
return _proxy_current_request(method=method, tenant_id=tenant_id, forward=forward)
|
||||
|
||||
|
||||
def _proxy_authorized_request(authorization: KnowledgeFSAuthorization) -> Response:
|
||||
"""Forward the current request using one previously authorized operation capability.
|
||||
|
||||
Args:
|
||||
authorization: Request-scoped capability produced before billing and body parsing.
|
||||
|
||||
Returns:
|
||||
The filtered response returned by KnowledgeFS.
|
||||
|
||||
Raises:
|
||||
HTTPException: The integration is disabled or forwarding fails.
|
||||
"""
|
||||
operation = authorization.operation
|
||||
tenant_id = authorization.tenant_id
|
||||
|
||||
def forward(
|
||||
accept: str | None,
|
||||
content_type: str | None,
|
||||
query: bytes | None,
|
||||
body: bytes | None,
|
||||
) -> KnowledgeFSUpstreamResponse:
|
||||
return proxy_authorized_knowledge_fs_request(
|
||||
authorization=authorization,
|
||||
accept=accept,
|
||||
content_type=content_type,
|
||||
query=query,
|
||||
body=body,
|
||||
request_headers=request.headers,
|
||||
)
|
||||
|
||||
return _proxy_current_request(method=operation.method, tenant_id=tenant_id, forward=forward)
|
||||
|
||||
|
||||
@_knowledge_fs_enabled
|
||||
@_knowledge_fs_operation_access_required
|
||||
@cloud_edition_billing_rate_limit_check("knowledge")
|
||||
def _proxy_knowledge_fs_non_get(
|
||||
authorization: KnowledgeFSAuthorization,
|
||||
) -> ResponseReturnValue:
|
||||
"""Apply knowledge billing checks to one allowlisted non-GET operation."""
|
||||
return _proxy_authorized_request(authorization)
|
||||
|
||||
|
||||
@bp.route(
|
||||
"/knowledge-fs/<path:upstream_path>",
|
||||
methods=["OPTIONS"],
|
||||
provide_automatic_options=False,
|
||||
)
|
||||
@_console_api_errors
|
||||
@_knowledge_fs_enabled
|
||||
def proxy_knowledge_fs_options(upstream_path: str) -> ResponseReturnValue:
|
||||
"""Complete a CORS preflight only for an enabled Console operation."""
|
||||
requested_method = cast(KnowledgeFSMethod, request.headers.get("Access-Control-Request-Method", "").upper())
|
||||
try:
|
||||
get_knowledge_fs_operation(requested_method, upstream_path)
|
||||
except KnowledgeFSRouteNotAllowedError as exc:
|
||||
raise NotFound() from exc
|
||||
return Response(status=HTTPStatus.NO_CONTENT)
|
||||
|
||||
|
||||
@bp.route(
|
||||
"/knowledge-fs/<path:upstream_path>",
|
||||
methods=["GET"],
|
||||
provide_automatic_options=False,
|
||||
)
|
||||
@_console_api_errors
|
||||
@_knowledge_fs_enabled
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
def proxy_knowledge_fs_get(upstream_path: str) -> ResponseReturnValue:
|
||||
"""Forward one authenticated, dataset-readable GET request.
|
||||
|
||||
Args:
|
||||
upstream_path: Relative KFS path captured after the Console proxy prefix.
|
||||
|
||||
Returns:
|
||||
The filtered raw KnowledgeFS response or a Console JSON error response.
|
||||
"""
|
||||
if request.method != "GET":
|
||||
raise NotFound()
|
||||
return _proxy_request("GET", upstream_path)
|
||||
|
||||
|
||||
@bp.route(
|
||||
"/knowledge-fs/<path:upstream_path>",
|
||||
methods=["DELETE", "PATCH", "POST", "PUT"],
|
||||
provide_automatic_options=False,
|
||||
)
|
||||
@_console_api_errors
|
||||
@_knowledge_fs_enabled
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
def proxy_knowledge_fs_write(upstream_path: str) -> ResponseReturnValue:
|
||||
"""Forward one authenticated non-GET request under its contract access policy.
|
||||
|
||||
Args:
|
||||
upstream_path: Relative KFS path captured after the Console proxy prefix.
|
||||
|
||||
Returns:
|
||||
The filtered raw KnowledgeFS response or a Console JSON error response.
|
||||
"""
|
||||
method = cast(KnowledgeFSMethod, request.method)
|
||||
return _proxy_knowledge_fs_non_get(method, upstream_path)
|
||||
@@ -1,106 +0,0 @@
|
||||
"""Console onboarding APIs.
|
||||
|
||||
This module keeps Step-by-step Tour persistence account-scoped. Workspace IDs
|
||||
are accepted only as presentation overrides; UI-only state such as minimized
|
||||
panels or the currently active task stays on the frontend. PATCH requests are
|
||||
action-based so callers do not replace server-side arrays with stale snapshots.
|
||||
"""
|
||||
|
||||
from datetime import datetime
|
||||
from typing import Literal, cast
|
||||
|
||||
from flask_restx import Resource
|
||||
from pydantic import BaseModel, ConfigDict, Field, model_validator
|
||||
|
||||
from controllers.common.schema import register_response_schema_models, register_schema_models
|
||||
from extensions.ext_database import db
|
||||
from fields.base import ResponseModel
|
||||
from libs.helper import dump_response
|
||||
from libs.login import login_required
|
||||
from models import Account
|
||||
from services.step_by_step_tour_service import StepByStepTourPatch, StepByStepTourService
|
||||
|
||||
from . import console_ns
|
||||
from .wraps import account_initialization_required, setup_required, with_current_tenant_id, with_current_user
|
||||
|
||||
StepByStepTourAction = Literal[
|
||||
"skip",
|
||||
"complete_task",
|
||||
"uncomplete_task",
|
||||
"enable_current_workspace",
|
||||
"disable_current_workspace",
|
||||
]
|
||||
StepByStepTourTaskId = Literal["home", "studio", "knowledge", "integration"]
|
||||
|
||||
|
||||
class StepByStepTourStatePatchPayload(BaseModel):
|
||||
action: StepByStepTourAction = Field(description="State update action")
|
||||
task_id: StepByStepTourTaskId | None = Field(default=None, description="Task ID for task actions")
|
||||
|
||||
model_config = ConfigDict(extra="forbid")
|
||||
|
||||
@model_validator(mode="after")
|
||||
def validate_patch_shape(self) -> "StepByStepTourStatePatchPayload":
|
||||
task_actions = {"complete_task", "uncomplete_task"}
|
||||
if self.action in task_actions and self.task_id is None:
|
||||
raise ValueError("task_id is required for task actions")
|
||||
if self.action not in task_actions and self.task_id is not None:
|
||||
raise ValueError("task_id is only supported for task actions")
|
||||
|
||||
return self
|
||||
|
||||
|
||||
class StepByStepTourStateResponse(ResponseModel):
|
||||
first_workspace_id: str | None = None
|
||||
skipped: bool = False
|
||||
completed_task_ids: list[StepByStepTourTaskId] = Field(default_factory=list)
|
||||
manually_enabled_workspace_ids: list[str] = Field(default_factory=list)
|
||||
manually_disabled_workspace_ids: list[str] = Field(default_factory=list)
|
||||
updated_at: datetime | None = None
|
||||
|
||||
|
||||
register_schema_models(console_ns, StepByStepTourStatePatchPayload)
|
||||
register_response_schema_models(console_ns, StepByStepTourStateResponse)
|
||||
|
||||
|
||||
@console_ns.route("/onboarding/step-by-step-tour/state")
|
||||
class StepByStepTourStateApi(Resource):
|
||||
@console_ns.doc("get_step_by_step_tour_state")
|
||||
@console_ns.doc(description="Get account-level Step-by-step Tour state")
|
||||
@console_ns.response(200, "Success", console_ns.models[StepByStepTourStateResponse.__name__])
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@with_current_user
|
||||
@with_current_tenant_id
|
||||
def get(self, current_tenant_id: str, current_user: Account):
|
||||
return dump_response(
|
||||
StepByStepTourStateResponse,
|
||||
StepByStepTourService.get_state(
|
||||
account=current_user,
|
||||
current_tenant_id=current_tenant_id,
|
||||
session=db.session,
|
||||
),
|
||||
)
|
||||
|
||||
@console_ns.doc("patch_step_by_step_tour_state")
|
||||
@console_ns.doc(description="Update account-level Step-by-step Tour state")
|
||||
@console_ns.expect(console_ns.models[StepByStepTourStatePatchPayload.__name__])
|
||||
@console_ns.response(200, "Success", console_ns.models[StepByStepTourStateResponse.__name__])
|
||||
@setup_required
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@with_current_user
|
||||
@with_current_tenant_id
|
||||
def patch(self, current_tenant_id: str, current_user: Account):
|
||||
payload = StepByStepTourStatePatchPayload.model_validate(console_ns.payload or {})
|
||||
patch = cast(StepByStepTourPatch, payload.model_dump(exclude_unset=True, exclude_none=True))
|
||||
return dump_response(
|
||||
StepByStepTourStateResponse,
|
||||
StepByStepTourService.patch_state(
|
||||
account=current_user,
|
||||
current_tenant_id=current_tenant_id,
|
||||
patch=patch,
|
||||
session=db.session,
|
||||
),
|
||||
)
|
||||
@@ -46,61 +46,6 @@ class GetRemoteFileInfo(Resource):
|
||||
).model_dump(mode="json")
|
||||
|
||||
|
||||
def upload_remote_file_from_request(
|
||||
*,
|
||||
current_user: Account,
|
||||
resource_tenant_id: str | None = None,
|
||||
) -> FileWithSignedUrl:
|
||||
"""Validate the JSON request, fetch its remote file, and persist it under the requested tenant."""
|
||||
payload = RemoteFileUploadPayload.model_validate(console_ns.payload)
|
||||
url = payload.url
|
||||
|
||||
# Try to fetch remote file metadata/content first
|
||||
try:
|
||||
resp = remote_fetcher.make_request("HEAD", url=url)
|
||||
if resp.status_code != httpx.codes.OK:
|
||||
resp = remote_fetcher.make_request("GET", url=url, timeout=3, follow_redirects=True)
|
||||
if resp.status_code != httpx.codes.OK:
|
||||
# Normalize into a user-friendly error message expected by tests
|
||||
raise RemoteFileUploadError(f"Failed to fetch file from {url}: {resp.text}")
|
||||
except httpx.RequestError as e:
|
||||
raise RemoteFileUploadError(f"Failed to fetch file from {url}: {str(e)}")
|
||||
|
||||
file_info = helpers.guess_file_info_from_response(resp)
|
||||
|
||||
# Enforce file size limit with 400 (Bad Request) per tests' expectation
|
||||
if not FileService.is_file_size_within_limit(extension=file_info.extension, file_size=file_info.size):
|
||||
raise FileTooLargeError()
|
||||
|
||||
# Load content if needed
|
||||
content = resp.content if resp.request.method == "GET" else remote_fetcher.make_request("GET", url).content
|
||||
|
||||
try:
|
||||
upload_file = FileService(db.engine).upload_file(
|
||||
filename=file_info.filename,
|
||||
content=content,
|
||||
mimetype=file_info.mimetype,
|
||||
user=current_user,
|
||||
tenant_id=resource_tenant_id,
|
||||
source_url=url,
|
||||
)
|
||||
except services.errors.file.FileTooLargeError as file_too_large_error:
|
||||
raise FileTooLargeError(file_too_large_error.description)
|
||||
except services.errors.file.UnsupportedFileTypeError:
|
||||
raise UnsupportedFileTypeError()
|
||||
|
||||
return FileWithSignedUrl(
|
||||
id=upload_file.id,
|
||||
name=upload_file.name,
|
||||
size=upload_file.size,
|
||||
extension=upload_file.extension,
|
||||
url=file_helpers.get_signed_file_url(upload_file_id=upload_file.id),
|
||||
mime_type=upload_file.mime_type,
|
||||
created_by=upload_file.created_by,
|
||||
created_at=int(upload_file.created_at.timestamp()),
|
||||
)
|
||||
|
||||
|
||||
@console_ns.route("/remote-files/upload")
|
||||
class RemoteFileUpload(Resource):
|
||||
@console_ns.expect(console_ns.models[RemoteFileUploadPayload.__name__])
|
||||
@@ -108,8 +53,53 @@ class RemoteFileUpload(Resource):
|
||||
@login_required
|
||||
@with_current_user
|
||||
def post(self, current_user: Account):
|
||||
remote_file = upload_remote_file_from_request(current_user=current_user)
|
||||
payload = RemoteFileUploadPayload.model_validate(console_ns.payload)
|
||||
url = payload.url
|
||||
|
||||
# Try to fetch remote file metadata/content first
|
||||
try:
|
||||
resp = remote_fetcher.make_request("HEAD", url=url)
|
||||
if resp.status_code != httpx.codes.OK:
|
||||
resp = remote_fetcher.make_request("GET", url=url, timeout=3, follow_redirects=True)
|
||||
if resp.status_code != httpx.codes.OK:
|
||||
# Normalize into a user-friendly error message expected by tests
|
||||
raise RemoteFileUploadError(f"Failed to fetch file from {url}: {resp.text}")
|
||||
except httpx.RequestError as e:
|
||||
raise RemoteFileUploadError(f"Failed to fetch file from {url}: {str(e)}")
|
||||
|
||||
file_info = helpers.guess_file_info_from_response(resp)
|
||||
|
||||
# Enforce file size limit with 400 (Bad Request) per tests' expectation
|
||||
if not FileService.is_file_size_within_limit(extension=file_info.extension, file_size=file_info.size):
|
||||
raise FileTooLargeError()
|
||||
|
||||
# Load content if needed
|
||||
content = resp.content if resp.request.method == "GET" else remote_fetcher.make_request("GET", url).content
|
||||
|
||||
try:
|
||||
upload_file = FileService(db.engine).upload_file(
|
||||
filename=file_info.filename,
|
||||
content=content,
|
||||
mimetype=file_info.mimetype,
|
||||
user=current_user,
|
||||
source_url=url,
|
||||
)
|
||||
except services.errors.file.FileTooLargeError as file_too_large_error:
|
||||
raise FileTooLargeError(file_too_large_error.description)
|
||||
except services.errors.file.UnsupportedFileTypeError:
|
||||
raise UnsupportedFileTypeError()
|
||||
|
||||
# Success: return created resource with 201 status
|
||||
return (
|
||||
remote_file.model_dump(mode="json"),
|
||||
FileWithSignedUrl(
|
||||
id=upload_file.id,
|
||||
name=upload_file.name,
|
||||
size=upload_file.size,
|
||||
extension=upload_file.extension,
|
||||
url=file_helpers.get_signed_file_url(upload_file_id=upload_file.id),
|
||||
mime_type=upload_file.mime_type,
|
||||
created_by=upload_file.created_by,
|
||||
created_at=int(upload_file.created_at.timestamp()),
|
||||
).model_dump(mode="json"),
|
||||
201,
|
||||
)
|
||||
|
||||
@@ -4,7 +4,7 @@ from typing import cast
|
||||
|
||||
from flask import Request as FlaskRequest
|
||||
|
||||
from core.db.session_factory import session_factory
|
||||
from extensions.ext_database import db
|
||||
from extensions.ext_socketio import sio
|
||||
from libs.passport import PassportService
|
||||
from libs.token import extract_access_token
|
||||
@@ -43,8 +43,8 @@ def socket_connect(sid, environ, auth):
|
||||
logging.warning("Socket connect rejected: missing user_id (sid=%s)", sid)
|
||||
return False
|
||||
|
||||
with sio.app.app_context(), session_factory.create_session() as session:
|
||||
user = AccountService.load_logged_in_account(account_id=user_id, session=session)
|
||||
with sio.app.app_context():
|
||||
user = AccountService.load_logged_in_account(account_id=user_id, session=db.session())
|
||||
if not user:
|
||||
logging.warning("Socket connect rejected: user not found (user_id=%s, sid=%s)", user_id, sid)
|
||||
return False
|
||||
@@ -69,8 +69,8 @@ def handle_user_connect(sid, data):
|
||||
if not workflow_id:
|
||||
return {"msg": "workflow_id is required"}, 400
|
||||
|
||||
with sio.app.app_context(), session_factory.create_session() as session:
|
||||
result = collaboration_service.authorize_and_join_workflow_room(workflow_id, sid, session=session)
|
||||
with sio.app.app_context():
|
||||
result = collaboration_service.authorize_and_join_workflow_room(workflow_id, sid, session=db.session())
|
||||
if not result:
|
||||
return {"msg": "unauthorized"}, 401
|
||||
|
||||
@@ -98,7 +98,6 @@ def handle_collaboration_event(sid, data):
|
||||
6. workflow_update
|
||||
7. comments_update
|
||||
8. node_panel_presence
|
||||
9. graph_view_state (session reports tab visibility; drives leader election)
|
||||
"""
|
||||
return collaboration_service.relay_collaboration_event(sid, data)
|
||||
|
||||
|
||||
@@ -4,16 +4,20 @@ from http import HTTPStatus
|
||||
from flask import redirect
|
||||
from flask_restx import Resource
|
||||
from pydantic import BaseModel, Field
|
||||
from werkzeug.exceptions import Conflict, Forbidden, NotFound
|
||||
from werkzeug.exceptions import Conflict, NotFound
|
||||
|
||||
from controllers.common.fields import RedirectResponse
|
||||
from controllers.common.schema import register_response_schema_models, register_schema_models
|
||||
from controllers.console import console_ns
|
||||
from controllers.console.wraps import (
|
||||
RBACPermission,
|
||||
RBACResourceScope,
|
||||
account_initialization_required,
|
||||
cloud_edition_billing_enabled,
|
||||
cloud_edition_billing_paid_plan_required,
|
||||
is_admin_or_owner_required,
|
||||
only_edition_cloud,
|
||||
rbac_permission_required,
|
||||
setup_required,
|
||||
)
|
||||
from extensions.ext_database import db
|
||||
@@ -21,7 +25,6 @@ from fields.base import ResponseModel
|
||||
from libs.archive_storage import get_export_storage
|
||||
from libs.helper import dump_response
|
||||
from libs.login import current_account_with_tenant, login_required
|
||||
from models import TenantAccountRole
|
||||
from services.retention.workflow_run.archive_download_preparation import ARCHIVE_DOWNLOAD_MIME_TYPE
|
||||
from services.retention.workflow_run.archive_download_task_cache import (
|
||||
WorkflowRunArchiveDownloadStatus,
|
||||
@@ -95,13 +98,11 @@ register_response_schema_models(
|
||||
)
|
||||
|
||||
|
||||
def _current_owner_or_admin_ids() -> tuple[str, str]:
|
||||
"""Return current Cloud workspace IDs for an owner or admin, independently of enterprise RBAC."""
|
||||
def _current_ids() -> tuple[str, str]:
|
||||
"""Return current `(tenant_id, account_id)` or raise when no workspace is selected."""
|
||||
current_user, current_tenant_id = current_account_with_tenant()
|
||||
if not current_tenant_id:
|
||||
raise NotFound("Current workspace not found")
|
||||
if not TenantAccountRole.is_privileged_role(current_user.current_role):
|
||||
raise Forbidden()
|
||||
return current_tenant_id, current_user.id
|
||||
|
||||
|
||||
@@ -123,8 +124,12 @@ class WorkflowRunArchivesApi(Resource):
|
||||
@only_edition_cloud
|
||||
@cloud_edition_billing_enabled
|
||||
@cloud_edition_billing_paid_plan_required
|
||||
@is_admin_or_owner_required
|
||||
@rbac_permission_required(
|
||||
RBACResourceScope.WORKSPACE, RBACPermission.WORKSPACE_ROLE_MANAGE, resource_required=False
|
||||
)
|
||||
def get(self):
|
||||
tenant_id, _ = _current_owner_or_admin_ids()
|
||||
tenant_id, _ = _current_ids()
|
||||
return dump_response(WorkflowRunArchiveListResponse, list_workflow_run_archives(db.session(), tenant_id))
|
||||
|
||||
|
||||
@@ -144,8 +149,12 @@ class WorkflowRunArchiveDownloadsApi(Resource):
|
||||
@only_edition_cloud
|
||||
@cloud_edition_billing_enabled
|
||||
@cloud_edition_billing_paid_plan_required
|
||||
@is_admin_or_owner_required
|
||||
@rbac_permission_required(
|
||||
RBACResourceScope.WORKSPACE, RBACPermission.WORKSPACE_ROLE_MANAGE, resource_required=False
|
||||
)
|
||||
def post(self):
|
||||
tenant_id, account_id = _current_owner_or_admin_ids()
|
||||
tenant_id, account_id = _current_ids()
|
||||
payload = WorkflowRunArchiveDownloadPayload.model_validate(console_ns.payload or {})
|
||||
try:
|
||||
task = create_workflow_run_archive_download_task(
|
||||
@@ -171,8 +180,12 @@ class WorkflowRunArchiveDownloadApi(Resource):
|
||||
@only_edition_cloud
|
||||
@cloud_edition_billing_enabled
|
||||
@cloud_edition_billing_paid_plan_required
|
||||
@is_admin_or_owner_required
|
||||
@rbac_permission_required(
|
||||
RBACResourceScope.WORKSPACE, RBACPermission.WORKSPACE_ROLE_MANAGE, resource_required=False
|
||||
)
|
||||
def get(self, download_id: str):
|
||||
tenant_id, _ = _current_owner_or_admin_ids()
|
||||
tenant_id, _ = _current_ids()
|
||||
try:
|
||||
task = get_workflow_run_archive_download_task(tenant_id=tenant_id, download_id=download_id)
|
||||
except WorkflowRunArchiveDownloadTaskNotFoundError as exc:
|
||||
@@ -196,8 +209,12 @@ class WorkflowRunArchiveDownloadFileApi(Resource):
|
||||
@only_edition_cloud
|
||||
@cloud_edition_billing_enabled
|
||||
@cloud_edition_billing_paid_plan_required
|
||||
@is_admin_or_owner_required
|
||||
@rbac_permission_required(
|
||||
RBACResourceScope.WORKSPACE, RBACPermission.WORKSPACE_ROLE_MANAGE, resource_required=False
|
||||
)
|
||||
def get(self, download_id: str):
|
||||
tenant_id, _ = _current_owner_or_admin_ids()
|
||||
tenant_id, _ = _current_ids()
|
||||
try:
|
||||
task = get_ready_workflow_run_archive_download_task(tenant_id=tenant_id, download_id=download_id)
|
||||
except WorkflowRunArchiveDownloadTaskNotFoundError as exc:
|
||||
|
||||
@@ -67,15 +67,6 @@ from services.plugin.plugin_parameter_service import PluginParameterService
|
||||
from services.plugin.plugin_permission_service import PluginPermissionService
|
||||
from services.tools.tools_transform_service import ToolTransformService
|
||||
|
||||
_PLUGIN_PACKAGE_UPLOAD_PARAMS = {
|
||||
"pkg": {
|
||||
"description": "Plugin package to upload",
|
||||
"in": "formData",
|
||||
"type": "file",
|
||||
"required": True,
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
class AutoUpgradeSettingsResponse(TypedDict):
|
||||
strategy_setting: TenantPluginAutoUpgradeStrategySetting
|
||||
@@ -654,7 +645,6 @@ class PluginAssetApi(Resource):
|
||||
|
||||
@console_ns.route("/workspaces/current/plugin/upload/pkg")
|
||||
class PluginUploadFromPkgApi(Resource):
|
||||
@console_ns.doc(consumes=["multipart/form-data"], params=_PLUGIN_PACKAGE_UPLOAD_PARAMS)
|
||||
@console_ns.response(200, "Success", console_ns.models[PluginDecodeResponse.__name__])
|
||||
@setup_required
|
||||
@login_required
|
||||
|
||||
@@ -6,8 +6,7 @@ from flask import request
|
||||
from flask_restx import Resource
|
||||
from pydantic import BaseModel, Field, field_validator
|
||||
from sqlalchemy import select
|
||||
from sqlalchemy.orm import Session
|
||||
from werkzeug.exceptions import NotFound, Unauthorized
|
||||
from werkzeug.exceptions import Unauthorized
|
||||
|
||||
import services
|
||||
from configs import dify_config
|
||||
@@ -24,7 +23,6 @@ from controllers.common.schema import (
|
||||
register_response_schema_models,
|
||||
register_schema_models,
|
||||
)
|
||||
from controllers.common.session import with_session
|
||||
from controllers.console import console_ns
|
||||
from controllers.console.admin import admin_required
|
||||
from controllers.console.error import AccountNotLinkTenantError
|
||||
@@ -88,7 +86,6 @@ class TenantInfoResponse(ResponseModel):
|
||||
custom_config: WorkspaceCustomConfigResponse | None = None
|
||||
trial_credits: int | None = None
|
||||
trial_credits_used: int | None = None
|
||||
trial_credits_exhausted_at: int | None = None
|
||||
next_credit_reset_date: int | None = None
|
||||
|
||||
@field_validator("plan", "status", "trial_end_reason", mode="before")
|
||||
@@ -223,11 +220,10 @@ class TenantListApi(Resource):
|
||||
@account_initialization_required
|
||||
@with_current_user
|
||||
@with_current_tenant_id
|
||||
@with_session(write=False)
|
||||
def get(self, session: Session, current_tenant_id: str, current_user: Account):
|
||||
def get(self, current_tenant_id: str, current_user: Account):
|
||||
tenant_rows: list[tuple[Tenant, TenantAccountJoin]] = [
|
||||
(tenant, membership)
|
||||
for tenant, membership in TenantService.get_workspaces_for_account(current_user.id, session=session)
|
||||
for tenant, membership in TenantService.get_workspaces_for_account(current_user.id, session=db.session())
|
||||
if tenant.status == TenantStatus.NORMAL
|
||||
]
|
||||
tenants = [tenant for tenant, _ in tenant_rows]
|
||||
@@ -278,12 +274,11 @@ class WorkspaceListApi(Resource):
|
||||
@console_ns.response(HTTPStatus.OK, "Success", console_ns.models[WorkspacePaginationResponse.__name__])
|
||||
@setup_required
|
||||
@admin_required
|
||||
@with_session(write=False)
|
||||
def get(self, session: Session):
|
||||
def get(self):
|
||||
args = query_params_from_request(WorkspaceListQuery)
|
||||
|
||||
stmt = select(Tenant).order_by(Tenant.created_at.desc())
|
||||
tenants = paginate_query(stmt, session=session, page=args.page, per_page=args.limit)
|
||||
tenants = paginate_query(stmt, page=args.page, per_page=args.limit)
|
||||
has_more = False
|
||||
|
||||
if tenants.has_next:
|
||||
@@ -302,8 +297,7 @@ class TenantApi(Resource):
|
||||
@account_initialization_required
|
||||
@console_ns.response(HTTPStatus.OK, "Success", console_ns.models[TenantInfoResponse.__name__])
|
||||
@with_current_user
|
||||
@with_session
|
||||
def post(self, session: Session, current_user: Account):
|
||||
def post(self, current_user: Account):
|
||||
if request.path == "/info":
|
||||
logger.warning("Deprecated URL /info was used.")
|
||||
|
||||
@@ -312,17 +306,17 @@ class TenantApi(Resource):
|
||||
raise ValueError("No current tenant")
|
||||
|
||||
if tenant.status == TenantStatus.ARCHIVE:
|
||||
tenants = TenantService.get_join_tenants(current_user, session=session)
|
||||
tenants = TenantService.get_join_tenants(current_user, session=db.session())
|
||||
# if there is any tenant, switch to the first one
|
||||
if len(tenants) > 0:
|
||||
TenantService.switch_tenant(current_user, tenants[0].id, session=session)
|
||||
TenantService.switch_tenant(current_user, tenants[0].id, session=db.session())
|
||||
tenant = tenants[0]
|
||||
# else, raise Unauthorized
|
||||
else:
|
||||
raise Unauthorized("workspace is archived")
|
||||
|
||||
return (
|
||||
dump_response(TenantInfoResponse, WorkspaceService.get_tenant_info(tenant, session=session)),
|
||||
dump_response(TenantInfoResponse, WorkspaceService.get_tenant_info(tenant, session=db.session())),
|
||||
HTTPStatus.OK,
|
||||
)
|
||||
|
||||
@@ -335,23 +329,22 @@ class SwitchWorkspaceApi(Resource):
|
||||
@login_required
|
||||
@account_initialization_required
|
||||
@with_current_user
|
||||
@with_session
|
||||
def post(self, session: Session, current_user: Account):
|
||||
def post(self, current_user: Account):
|
||||
payload = console_ns.payload or {}
|
||||
args = SwitchWorkspacePayload.model_validate(payload)
|
||||
|
||||
# Check whether the tenant_id belongs to the current account.
|
||||
try:
|
||||
TenantService.switch_tenant(current_user, args.tenant_id, session=session)
|
||||
TenantService.switch_tenant(current_user, args.tenant_id, session=db.session())
|
||||
except Exception:
|
||||
raise AccountNotLinkTenantError("Account not link tenant")
|
||||
|
||||
new_tenant = TenantService.get_tenant_by_id(args.tenant_id, session=session)
|
||||
new_tenant = db.session.get(Tenant, args.tenant_id) # Get new tenant
|
||||
if new_tenant is None:
|
||||
raise ValueError("Tenant not found")
|
||||
|
||||
return SwitchWorkspaceResponse(
|
||||
result="success", new_tenant=WorkspaceService.get_tenant_info(new_tenant, session=session)
|
||||
result="success", new_tenant=WorkspaceService.get_tenant_info(new_tenant, session=db.session())
|
||||
).model_dump(mode="json")
|
||||
|
||||
|
||||
@@ -364,13 +357,10 @@ class CustomConfigWorkspaceApi(Resource):
|
||||
@account_initialization_required
|
||||
@cloud_edition_billing_resource_check("workspace_custom")
|
||||
@with_current_tenant_id
|
||||
@with_session
|
||||
def post(self, session: Session, current_tenant_id: str):
|
||||
def post(self, current_tenant_id: str):
|
||||
payload = console_ns.payload or {}
|
||||
args = WorkspaceCustomConfigPayload.model_validate(payload)
|
||||
tenant = TenantService.get_tenant_by_id(current_tenant_id, session=session)
|
||||
if tenant is None:
|
||||
raise NotFound()
|
||||
tenant = db.get_or_404(Tenant, current_tenant_id)
|
||||
|
||||
custom_config_dict: TenantCustomConfigDict = {
|
||||
"remove_webapp_brand": args.remove_webapp_brand
|
||||
@@ -382,10 +372,10 @@ class CustomConfigWorkspaceApi(Resource):
|
||||
}
|
||||
|
||||
tenant.custom_config_dict = custom_config_dict
|
||||
session.commit()
|
||||
db.session.commit()
|
||||
|
||||
return WorkspaceTenantResultResponse(
|
||||
result="success", tenant=WorkspaceService.get_tenant_info(tenant, session=session)
|
||||
result="success", tenant=WorkspaceService.get_tenant_info(tenant, session=db.session())
|
||||
).model_dump(mode="json")
|
||||
|
||||
|
||||
@@ -440,21 +430,18 @@ class WorkspaceInfoApi(Resource):
|
||||
@account_initialization_required
|
||||
# Change workspace name
|
||||
@with_current_tenant_id
|
||||
@with_session
|
||||
def post(self, session: Session, current_tenant_id: str):
|
||||
def post(self, current_tenant_id: str):
|
||||
payload = console_ns.payload or {}
|
||||
args = WorkspaceInfoPayload.model_validate(payload)
|
||||
|
||||
if not current_tenant_id:
|
||||
raise ValueError("No current tenant")
|
||||
tenant = TenantService.get_tenant_by_id(current_tenant_id, session=session)
|
||||
if tenant is None:
|
||||
raise NotFound()
|
||||
tenant = db.get_or_404(Tenant, current_tenant_id)
|
||||
tenant.name = args.name
|
||||
session.commit()
|
||||
db.session.commit()
|
||||
|
||||
return WorkspaceTenantResultResponse(
|
||||
result="success", tenant=WorkspaceService.get_tenant_info(tenant, session=session)
|
||||
result="success", tenant=WorkspaceService.get_tenant_info(tenant, session=db.session())
|
||||
).model_dump(mode="json")
|
||||
|
||||
|
||||
|
||||
@@ -54,8 +54,9 @@ class EnterpriseAppDSLImport(Resource):
|
||||
if account is None:
|
||||
return {"message": f"account '{args.creator_email}' not found or inactive"}, 404
|
||||
|
||||
account.set_tenant_id(workspace_id)
|
||||
|
||||
with Session(db.engine, expire_on_commit=False) as session:
|
||||
account.set_tenant_id_with_session(workspace_id, session=session)
|
||||
dsl_service = AppDslService(session)
|
||||
result = dsl_service.import_app(
|
||||
account=account,
|
||||
|
||||
@@ -47,12 +47,10 @@ class EnterpriseWorkspace(Resource):
|
||||
if account is None:
|
||||
return {"message": "owner account not found."}, 404
|
||||
|
||||
tenant = TenantService.create_owner_tenant(
|
||||
account,
|
||||
name=args.name,
|
||||
is_from_dashboard=True,
|
||||
session=db.session(),
|
||||
)
|
||||
tenant = TenantService.create_tenant(args.name, is_from_dashboard=True, session=db.session())
|
||||
TenantService.create_tenant_member(tenant, account, db.session(), role="owner")
|
||||
|
||||
tenant_was_created.send(tenant)
|
||||
|
||||
resp = {
|
||||
"id": tenant.id,
|
||||
|
||||
@@ -4,11 +4,9 @@ from __future__ import annotations
|
||||
|
||||
from typing import Any, cast
|
||||
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from controllers.service_api.app.error import AppUnavailableError
|
||||
from models import App
|
||||
from models.model import AppMode, load_annotation_reply_config
|
||||
from models.model import AppMode
|
||||
|
||||
JSON_SCHEMA_DRAFT = "https://json-schema.org/draft/2020-12/schema"
|
||||
|
||||
@@ -91,13 +89,13 @@ def _form_to_jsonschema(form: list[dict[str, Any]]) -> tuple[dict[str, Any], lis
|
||||
return properties, required
|
||||
|
||||
|
||||
def resolve_app_config(app: App, *, session: Session) -> tuple[dict[str, Any], list[dict[str, Any]]]:
|
||||
def resolve_app_config(app: App) -> tuple[dict[str, Any], list[dict[str, Any]]]:
|
||||
"""Resolve `(features_dict, user_input_form)` for parameters / schema derivation.
|
||||
|
||||
Raises `AppUnavailableError` on misconfigured apps.
|
||||
"""
|
||||
if app.mode in {AppMode.ADVANCED_CHAT, AppMode.WORKFLOW}:
|
||||
workflow = app.workflow_with_session(session=session)
|
||||
workflow = app.workflow
|
||||
if workflow is None:
|
||||
raise AppUnavailableError()
|
||||
return (
|
||||
@@ -105,22 +103,21 @@ def resolve_app_config(app: App, *, session: Session) -> tuple[dict[str, Any], l
|
||||
cast(list[dict[str, Any]], workflow.user_input_form(to_old_structure=True)),
|
||||
)
|
||||
|
||||
app_model_config = app.app_model_config_with_session(session=session)
|
||||
app_model_config = app.app_model_config
|
||||
if app_model_config is None:
|
||||
raise AppUnavailableError()
|
||||
annotation_reply = load_annotation_reply_config(session, app_model_config.app_id)
|
||||
features_dict = cast(dict[str, Any], app_model_config.to_dict(annotation_reply=annotation_reply))
|
||||
features_dict = cast(dict[str, Any], app_model_config.to_dict())
|
||||
return features_dict, cast(list[dict[str, Any]], features_dict.get("user_input_form", []))
|
||||
|
||||
|
||||
def build_input_schema(app: App, *, session: Session) -> dict[str, Any]:
|
||||
def build_input_schema(app: App) -> dict[str, Any]:
|
||||
"""Derive Draft 2020-12 JSON Schema from `user_input_form` + app mode.
|
||||
|
||||
chat / agent-chat / advanced-chat: top-level `query` (required, minLength=1) + `inputs` object.
|
||||
completion / workflow: `inputs` object only.
|
||||
Raises `AppUnavailableError` on misconfigured apps.
|
||||
"""
|
||||
_, user_input_form = resolve_app_config(app, session=session)
|
||||
_, user_input_form = resolve_app_config(app)
|
||||
inputs_props, inputs_required = _form_to_jsonschema(user_input_form)
|
||||
|
||||
properties: dict[str, Any] = {}
|
||||
|
||||
@@ -3,10 +3,8 @@ from __future__ import annotations
|
||||
from datetime import UTC, datetime
|
||||
|
||||
from flask_restx import Resource
|
||||
from sqlalchemy.orm import Session
|
||||
from werkzeug.exceptions import NotFound
|
||||
|
||||
from controllers.common.session import with_session
|
||||
from controllers.openapi import openapi_ns
|
||||
from controllers.openapi._contract import accepts, returns
|
||||
from controllers.openapi._models import (
|
||||
@@ -20,6 +18,7 @@ from controllers.openapi._models import (
|
||||
)
|
||||
from controllers.openapi.auth.composition import auth_router
|
||||
from controllers.openapi.auth.data import AuthData
|
||||
from extensions.ext_database import db
|
||||
from extensions.ext_redis import redis_client
|
||||
from libs.oauth_bearer import (
|
||||
Scope,
|
||||
@@ -42,13 +41,14 @@ from services.oauth_device_flow import (
|
||||
class AccountApi(Resource):
|
||||
@auth_router.guard(scope=Scope.FULL, allowed_token_types=frozenset({TokenType.OAUTH_ACCOUNT}))
|
||||
@returns(200, AccountResponse, description="Account info")
|
||||
@with_session(write=False)
|
||||
def get(self, session: Session, *, auth_data: AuthData):
|
||||
def get(self, *, auth_data: AuthData):
|
||||
enforce(LIMIT_ME_PER_ACCOUNT, key=f"account:{auth_data.account_id}")
|
||||
|
||||
account_id_str = str(auth_data.account_id) if auth_data.account_id else None
|
||||
account = AccountService.get_account_by_id(account_id_str, session=session) if account_id_str else None
|
||||
memberships = TenantService.get_account_memberships(account_id_str, session=session) if account_id_str else []
|
||||
account = AccountService.get_account_by_id(account_id_str, session=db.session()) if account_id_str else None
|
||||
memberships = (
|
||||
TenantService.get_account_memberships(account_id_str, session=db.session()) if account_id_str else []
|
||||
)
|
||||
default_ws_id = _pick_default_workspace(memberships)
|
||||
|
||||
return AccountResponse(
|
||||
@@ -64,9 +64,8 @@ class AccountApi(Resource):
|
||||
class AccountSessionsSelfApi(Resource):
|
||||
@auth_router.guard(scope=Scope.FULL, allowed_token_types=frozenset({TokenType.OAUTH_ACCOUNT}))
|
||||
@returns(200, RevokeResponse, description="Session revoked")
|
||||
@with_session
|
||||
def delete(self, session: Session, *, auth_data: AuthData):
|
||||
revoke_oauth_token(redis_client, str(auth_data.token_id), session=session)
|
||||
def delete(self, *, auth_data: AuthData):
|
||||
revoke_oauth_token(redis_client, str(auth_data.token_id), session=db.session())
|
||||
return RevokeResponse(status="revoked")
|
||||
|
||||
|
||||
@@ -75,8 +74,7 @@ class AccountSessionsApi(Resource):
|
||||
@auth_router.guard(scope=Scope.FULL, allowed_token_types=frozenset({TokenType.OAUTH_ACCOUNT}))
|
||||
@returns(200, SessionListResponse, description="Session list")
|
||||
@accepts(query=SessionListQuery)
|
||||
@with_session(write=False)
|
||||
def get(self, session: Session, *, auth_data: AuthData, query: SessionListQuery):
|
||||
def get(self, *, auth_data: AuthData, query: SessionListQuery):
|
||||
# SessionListQuery enforces the advertised bounds (extra='forbid', page>=1,
|
||||
# 1<=limit<=MAX_PAGE_LIMIT) so the server rejects out-of-range paging rather
|
||||
# than silently coercing (e.g. page=0 -> empty slice).
|
||||
@@ -85,7 +83,7 @@ class AccountSessionsApi(Resource):
|
||||
page = query.page
|
||||
limit = query.limit
|
||||
|
||||
all_rows = list_active_sessions(ctx, now, session=session)
|
||||
all_rows = list_active_sessions(ctx, now, session=db.session())
|
||||
|
||||
total = len(all_rows)
|
||||
sliced = all_rows[(page - 1) * limit : page * limit]
|
||||
@@ -116,16 +114,15 @@ class AccountSessionsApi(Resource):
|
||||
class AccountSessionByIdApi(Resource):
|
||||
@auth_router.guard(scope=Scope.FULL, allowed_token_types=frozenset({TokenType.OAUTH_ACCOUNT}))
|
||||
@returns(200, RevokeResponse, description="Session revoked")
|
||||
@with_session
|
||||
def delete(self, session: Session, session_id: str, *, auth_data: AuthData):
|
||||
def delete(self, session_id: str, *, auth_data: AuthData):
|
||||
ctx = get_auth_ctx()
|
||||
|
||||
# 404 (not 403) on cross-subject so the endpoint doesn't leak
|
||||
# token IDs that belong to other subjects.
|
||||
if not token_belongs_to_subject(session_id, ctx, session=session):
|
||||
if not token_belongs_to_subject(session_id, ctx, session=db.session()):
|
||||
raise NotFound("session not found")
|
||||
|
||||
revoke_oauth_token(redis_client, session_id, session=session)
|
||||
revoke_oauth_token(redis_client, session_id, session=db.session())
|
||||
return RevokeResponse(status="revoked")
|
||||
|
||||
|
||||
|
||||
@@ -6,13 +6,11 @@ import uuid as _uuid
|
||||
from typing import Any, cast
|
||||
|
||||
from flask_restx import Resource
|
||||
from sqlalchemy.orm import Session
|
||||
from werkzeug.exceptions import Conflict, NotFound, UnprocessableEntity
|
||||
|
||||
from configs import dify_config
|
||||
from controllers.common.app_access import AppAccessFilter, resolve_app_access_filter
|
||||
from controllers.common.fields import Parameters
|
||||
from controllers.common.session import with_session
|
||||
from controllers.common.wraps import RBACPermission, RBACResourceScope
|
||||
from controllers.openapi import openapi_ns
|
||||
from controllers.openapi._contract import accepts, returns
|
||||
@@ -30,6 +28,7 @@ from controllers.openapi.auth.composition import auth_router
|
||||
from controllers.openapi.auth.data import AuthData, CallerKind, RBACRequirement
|
||||
from controllers.service_api.app.error import AppUnavailableError
|
||||
from core.app.app_config.common.parameters_mapping import get_parameters_from_feature_dict
|
||||
from extensions.ext_database import db
|
||||
from libs.oauth_bearer import Scope, TokenType
|
||||
from models import App
|
||||
from models.enums import AppStatus
|
||||
@@ -57,7 +56,7 @@ _EMPTY_PARAMETERS: dict[str, Any] = {
|
||||
class AppReadResource(Resource):
|
||||
"""Base for per-app read endpoints; subclasses call `_load()` for membership/exists checks."""
|
||||
|
||||
def _load(self, session: Session, app_id: str, workspace_id: str | None = None) -> App:
|
||||
def _load(self, app_id: str, workspace_id: str | None = None) -> App:
|
||||
try:
|
||||
parsed_uuid = _uuid.UUID(app_id)
|
||||
is_uuid = True
|
||||
@@ -67,13 +66,13 @@ class AppReadResource(Resource):
|
||||
|
||||
if is_uuid:
|
||||
# ``str(parsed_uuid)`` normalises to the canonical dashed form.
|
||||
app = AppService.get_visible_app_by_id(str(parsed_uuid), session)
|
||||
app = AppService.get_visible_app_by_id(str(parsed_uuid), session=db.session())
|
||||
if app is None:
|
||||
raise NotFound("app not found")
|
||||
else:
|
||||
if not workspace_id:
|
||||
raise UnprocessableEntity("workspace_id is required for name-based lookup")
|
||||
matches = AppService.find_visible_apps_by_name(session, name=app_id, tenant_id=workspace_id)
|
||||
matches = AppService.find_visible_apps_by_name(name=app_id, tenant_id=workspace_id, session=db.session())
|
||||
if len(matches) == 0:
|
||||
raise NotFound("app not found")
|
||||
if len(matches) > 1:
|
||||
@@ -87,14 +86,14 @@ class AppReadResource(Resource):
|
||||
return app
|
||||
|
||||
|
||||
def parameters_payload(app: App, *, session: Session) -> dict:
|
||||
def parameters_payload(app: App) -> dict:
|
||||
"""Mirrors service_api/app/app.py::AppParameterApi response body."""
|
||||
features_dict, user_input_form = resolve_app_config(app, session=session)
|
||||
features_dict, user_input_form = resolve_app_config(app)
|
||||
parameters = get_parameters_from_feature_dict(features_dict=features_dict, user_input_form=user_input_form)
|
||||
return Parameters.model_validate(parameters).model_dump(mode="json")
|
||||
|
||||
|
||||
def build_app_describe_response(app: App, fields: set[str] | None, *, session: Session) -> AppDescribeResponse:
|
||||
def build_app_describe_response(app: App, fields: set[str] | None) -> AppDescribeResponse:
|
||||
"""Public projection of an app (name / params / input schema) — never internal config."""
|
||||
want_info = fields is None or "info" in fields
|
||||
want_params = fields is None or "parameters" in fields
|
||||
@@ -118,12 +117,12 @@ def build_app_describe_response(app: App, fields: set[str] | None, *, session: S
|
||||
input_schema: dict[str, Any] | None = None
|
||||
if want_params:
|
||||
try:
|
||||
parameters = parameters_payload(app, session=session)
|
||||
parameters = parameters_payload(app)
|
||||
except AppUnavailableError:
|
||||
parameters = dict(_EMPTY_PARAMETERS)
|
||||
if want_schema:
|
||||
try:
|
||||
input_schema = build_input_schema(app, session=session)
|
||||
input_schema = build_input_schema(app)
|
||||
except AppUnavailableError:
|
||||
input_schema = dict(EMPTY_INPUT_SCHEMA)
|
||||
|
||||
@@ -139,11 +138,10 @@ class AppDescribeApi(AppReadResource):
|
||||
)
|
||||
@returns(200, AppDescribeResponse, description="App description")
|
||||
@accepts(query=AppDescribeQuery)
|
||||
@with_session(write=False)
|
||||
def get(self, session: Session, app_id: str, *, auth_data: AuthData, query: AppDescribeQuery):
|
||||
def get(self, app_id: str, *, auth_data: AuthData, query: AppDescribeQuery):
|
||||
# describe is UUID-only (workspace_id query param dropped in #37212).
|
||||
app = self._load(session, app_id)
|
||||
return build_app_describe_response(app, query.fields, session=session)
|
||||
app = self._load(app_id)
|
||||
return build_app_describe_response(app, query.fields)
|
||||
|
||||
|
||||
@openapi_ns.route("/apps")
|
||||
@@ -151,8 +149,7 @@ class AppListApi(Resource):
|
||||
@auth_router.guard_workspace(scope=Scope.APPS_READ, allowed_token_types=frozenset({TokenType.OAUTH_ACCOUNT}))
|
||||
@returns(200, AppListResponse, description="App list")
|
||||
@accepts(query=AppListQuery)
|
||||
@with_session(write=False)
|
||||
def get(self, session: Session, *, auth_data: AuthData, query: AppListQuery):
|
||||
def get(self, *, auth_data: AuthData, query: AppListQuery):
|
||||
workspace_id = query.workspace_id
|
||||
|
||||
empty = AppListResponse(page=query.page, limit=query.limit, total=0, has_more=False, data=[])
|
||||
@@ -176,15 +173,11 @@ class AppListApi(Resource):
|
||||
)
|
||||
access_filter = AppAccessFilter.unrestricted()
|
||||
if apply_rbac_filter:
|
||||
access_filter = resolve_app_access_filter(
|
||||
workspace_id,
|
||||
str(auth_data.account_id),
|
||||
session=session,
|
||||
)
|
||||
access_filter = resolve_app_access_filter(workspace_id, str(auth_data.account_id))
|
||||
|
||||
tenant_name: str | None = None
|
||||
if parsed_uuid is not None:
|
||||
app: App | None = AppService.get_visible_app_by_id(str(parsed_uuid), session)
|
||||
app: App | None = AppService.get_visible_app_by_id(str(parsed_uuid), session=db.session())
|
||||
if app is None or str(app.tenant_id) != workspace_id:
|
||||
return empty
|
||||
if not _is_listable(app):
|
||||
@@ -195,7 +188,7 @@ class AppListApi(Resource):
|
||||
str(app.id), str(app.maintainer) if app.maintainer else None, str(auth_data.account_id)
|
||||
):
|
||||
return empty
|
||||
tenant_name = TenantService.get_tenant_name(workspace_id, session=session)
|
||||
tenant_name = TenantService.get_tenant_name(workspace_id, session=db.session())
|
||||
item = AppListRow(
|
||||
id=str(app.id),
|
||||
name=app.name,
|
||||
@@ -222,13 +215,13 @@ class AppListApi(Resource):
|
||||
if apply_rbac_filter:
|
||||
access_filter.apply_to_params(params)
|
||||
|
||||
pagination = AppService().get_paginate_apps(str(auth_data.account_id), workspace_id, params, session)
|
||||
pagination = AppService().get_paginate_apps(str(auth_data.account_id), workspace_id, params, db.session())
|
||||
if pagination is None:
|
||||
return empty
|
||||
|
||||
tenant_name = None
|
||||
if pagination.items:
|
||||
tenant_name = TenantService.get_tenant_name(workspace_id, session=session)
|
||||
tenant_name = TenantService.get_tenant_name(workspace_id, session=db.session())
|
||||
|
||||
items = [
|
||||
AppListRow(
|
||||
|
||||
@@ -8,10 +8,8 @@ EE blueprint chain so this module is unreachable there.
|
||||
from __future__ import annotations
|
||||
|
||||
from flask_restx import Resource
|
||||
from sqlalchemy.orm import Session
|
||||
from werkzeug.exceptions import NotFound
|
||||
|
||||
from controllers.common.session import with_session
|
||||
from controllers.openapi import openapi_ns
|
||||
from controllers.openapi._contract import accepts, returns
|
||||
from controllers.openapi._models import (
|
||||
@@ -24,6 +22,7 @@ from controllers.openapi._models import (
|
||||
from controllers.openapi.apps import build_app_describe_response
|
||||
from controllers.openapi.auth.composition import auth_router
|
||||
from controllers.openapi.auth.data import AuthData, Edition
|
||||
from extensions.ext_database import db
|
||||
from libs.oauth_bearer import Scope, TokenType
|
||||
from models import App
|
||||
from models.enums import AppStatus
|
||||
@@ -41,8 +40,7 @@ class PermittedExternalAppsListApi(Resource):
|
||||
)
|
||||
@returns(200, PermittedExternalAppsListResponse, description="Permitted external apps list")
|
||||
@accepts(query=PermittedExternalAppsListQuery)
|
||||
@with_session(write=False)
|
||||
def get(self, session: Session, *, auth_data: AuthData, query: PermittedExternalAppsListQuery):
|
||||
def get(self, *, auth_data: AuthData, query: PermittedExternalAppsListQuery):
|
||||
page_result = list_permitted_apps(
|
||||
page=query.page,
|
||||
limit=query.limit,
|
||||
@@ -57,10 +55,10 @@ class PermittedExternalAppsListApi(Resource):
|
||||
return env
|
||||
|
||||
apps_by_id: dict[str, App] = {
|
||||
str(a.id): a for a in AppService.find_visible_apps_by_ids(page_result.app_ids, session)
|
||||
str(a.id): a for a in AppService.find_visible_apps_by_ids(page_result.app_ids, session=db.session())
|
||||
}
|
||||
tenant_ids = list({str(a.tenant_id) for a in apps_by_id.values()})
|
||||
tenants_by_id = {str(t.id): t for t in TenantService.get_tenants_by_ids(tenant_ids, session=session)}
|
||||
tenants_by_id = {str(t.id): t for t in TenantService.get_tenants_by_ids(tenant_ids, session=db.session())}
|
||||
|
||||
items: list[AppListRow] = []
|
||||
for app_id in page_result.app_ids:
|
||||
@@ -98,10 +96,9 @@ class PermittedExternalAppDescribeApi(Resource):
|
||||
)
|
||||
@returns(200, AppDescribeResponse, description="Permitted external app description")
|
||||
@accepts(query=AppDescribeQuery)
|
||||
@with_session(write=False)
|
||||
def get(self, session: Session, app_id: str, *, auth_data: AuthData, query: AppDescribeQuery):
|
||||
def get(self, app_id: str, *, auth_data: AuthData, query: AppDescribeQuery):
|
||||
# App already loaded and ACL-checked by the external_sso pipeline; project it.
|
||||
app = auth_data.app
|
||||
if app is None:
|
||||
raise NotFound("app not found")
|
||||
return build_app_describe_response(app, query.fields, session=session)
|
||||
return build_app_describe_response(app, query.fields)
|
||||
|
||||
@@ -6,7 +6,7 @@ from flask import request
|
||||
from werkzeug.exceptions import Forbidden, InternalServerError, NotFound, Unauthorized
|
||||
|
||||
from controllers.openapi.auth.data import AuthData, CallerKind
|
||||
from core.db.session_factory import session_factory
|
||||
from extensions.ext_database import db
|
||||
from models.account import AccountStatus, TenantStatus
|
||||
from models.enums import AppStatus, EndUserType
|
||||
from services.account_service import AccountService, TenantService
|
||||
@@ -23,8 +23,7 @@ def load_app(data: AuthData) -> None:
|
||||
uuid.UUID(app_id)
|
||||
except ValueError:
|
||||
raise NotFound("app not found")
|
||||
with session_factory.create_session() as session:
|
||||
app = AppService.get_app_by_id(app_id, session)
|
||||
app = AppService.get_app_by_id(app_id, session=db.session())
|
||||
if not app or app.status != AppStatus.NORMAL:
|
||||
raise NotFound("app not found")
|
||||
data.app = app
|
||||
@@ -35,8 +34,7 @@ def load_tenant(data: AuthData) -> None:
|
||||
return
|
||||
if data.app is None:
|
||||
raise InternalServerError("pipeline_invariant_violated: app not loaded before load_tenant")
|
||||
with session_factory.create_session() as session:
|
||||
tenant = TenantService.get_tenant_by_id(str(data.app.tenant_id), session=session)
|
||||
tenant = TenantService.get_tenant_by_id(str(data.app.tenant_id), session=db.session())
|
||||
if tenant is None or tenant.status == TenantStatus.ARCHIVE:
|
||||
raise Forbidden("workspace unavailable")
|
||||
data.tenant = tenant
|
||||
@@ -52,8 +50,7 @@ def load_tenant_from_request(data: AuthData) -> None:
|
||||
uuid.UUID(workspace_id)
|
||||
except ValueError:
|
||||
raise NotFound("workspace not found")
|
||||
with session_factory.create_session() as session:
|
||||
tenant = TenantService.get_tenant_by_id(workspace_id, session=session)
|
||||
tenant = TenantService.get_tenant_by_id(workspace_id, session=db.session())
|
||||
if tenant is None or tenant.status == TenantStatus.ARCHIVE:
|
||||
raise NotFound("workspace not found")
|
||||
data.tenant = tenant
|
||||
@@ -62,12 +59,11 @@ def load_tenant_from_request(data: AuthData) -> None:
|
||||
def load_account(data: AuthData) -> None:
|
||||
if data.caller is not None:
|
||||
return
|
||||
with session_factory.create_session() as session:
|
||||
account = AccountService.get_account_by_id(str(data.account_id), session=session)
|
||||
if account is None:
|
||||
raise Unauthorized("account not found")
|
||||
if data.tenant:
|
||||
account.set_current_tenant_with_session(data.tenant, session=session)
|
||||
account = AccountService.get_account_by_id(str(data.account_id), session=db.session())
|
||||
if account is None:
|
||||
raise Unauthorized("account not found")
|
||||
if data.tenant:
|
||||
account.current_tenant = data.tenant
|
||||
data.caller = account
|
||||
data.caller_kind = CallerKind.ACCOUNT
|
||||
|
||||
@@ -79,8 +75,7 @@ def load_workspace_role(data: AuthData) -> None:
|
||||
return
|
||||
if data.caller is not None and getattr(data.caller, "status", None) != AccountStatus.ACTIVE:
|
||||
return
|
||||
with session_factory.create_session() as session:
|
||||
role = TenantService.get_account_role_in_tenant(str(data.account_id), str(data.tenant.id), session=session)
|
||||
role = TenantService.get_account_role_in_tenant(str(data.account_id), str(data.tenant.id), session=db.session())
|
||||
if role is None:
|
||||
return
|
||||
data.tenant_role = role
|
||||
|
||||
@@ -15,11 +15,9 @@ from itertools import starmap
|
||||
from urllib import parse
|
||||
|
||||
from flask_restx import Resource
|
||||
from sqlalchemy.orm import Session
|
||||
from werkzeug.exceptions import BadRequest, NotFound
|
||||
|
||||
from configs import dify_config
|
||||
from controllers.common.session import with_session
|
||||
from controllers.openapi import openapi_ns
|
||||
from controllers.openapi._contract import accepts, returns
|
||||
from controllers.openapi._errors import MemberLicenseExceeded, MemberLimitExceeded
|
||||
@@ -37,6 +35,7 @@ from controllers.openapi._models import (
|
||||
)
|
||||
from controllers.openapi.auth.composition import auth_router
|
||||
from controllers.openapi.auth.data import AuthData
|
||||
from extensions.ext_database import db
|
||||
from libs.oauth_bearer import Scope, TokenType
|
||||
from models import Account, Tenant, TenantAccountJoin
|
||||
from models.account import TenantAccountRole, TenantStatus
|
||||
@@ -65,15 +64,15 @@ def _member_response(account: Account) -> MemberResponse:
|
||||
)
|
||||
|
||||
|
||||
def _load_tenant(session: Session, workspace_id: str) -> Tenant:
|
||||
tenant = TenantService.get_tenant_by_id(workspace_id, session=session)
|
||||
def _load_tenant(workspace_id: str) -> Tenant:
|
||||
tenant = TenantService.get_tenant_by_id(workspace_id, session=db.session())
|
||||
if tenant is None or tenant.status != TenantStatus.NORMAL:
|
||||
raise NotFound("workspace not found")
|
||||
return tenant
|
||||
|
||||
|
||||
def _load_account(session: Session, account_id: object) -> Account:
|
||||
account = AccountService.get_account_by_id(str(account_id), session=session) if account_id else None
|
||||
def _load_account(account_id: object) -> Account:
|
||||
account = AccountService.get_account_by_id(str(account_id), session=db.session()) if account_id else None
|
||||
if account is None:
|
||||
raise RuntimeError("authenticated account_id has no Account row")
|
||||
return account
|
||||
@@ -95,9 +94,8 @@ def _check_member_invite_quota(tenant_id: str) -> None:
|
||||
class WorkspacesApi(Resource):
|
||||
@auth_router.guard(scope=Scope.WORKSPACE_READ, allowed_token_types=frozenset({TokenType.OAUTH_ACCOUNT}))
|
||||
@returns(200, WorkspaceListResponse, description="Workspace list")
|
||||
@with_session(write=False)
|
||||
def get(self, session: Session, *, auth_data: AuthData):
|
||||
rows = TenantService.get_workspaces_for_account(str(auth_data.account_id), session=session)
|
||||
def get(self, *, auth_data: AuthData):
|
||||
rows = TenantService.get_workspaces_for_account(str(auth_data.account_id), session=db.session())
|
||||
|
||||
return WorkspaceListResponse(workspaces=list(starmap(_workspace_summary, rows)))
|
||||
|
||||
@@ -106,9 +104,8 @@ class WorkspacesApi(Resource):
|
||||
class WorkspaceByIdApi(Resource):
|
||||
@auth_router.guard(scope=Scope.WORKSPACE_READ, allowed_token_types=frozenset({TokenType.OAUTH_ACCOUNT}))
|
||||
@returns(200, WorkspaceDetailResponse, description="Workspace detail")
|
||||
@with_session(write=False)
|
||||
def get(self, session: Session, workspace_id: str, *, auth_data: AuthData):
|
||||
row = TenantService.find_workspace_for_account(str(auth_data.account_id), workspace_id, session=session)
|
||||
def get(self, workspace_id: str, *, auth_data: AuthData):
|
||||
row = TenantService.find_workspace_for_account(str(auth_data.account_id), workspace_id, session=db.session())
|
||||
# 404 (not 403) on non-member so workspace IDs don't leak across tenants.
|
||||
if row is None:
|
||||
raise NotFound("workspace not found")
|
||||
@@ -128,16 +125,15 @@ class WorkspaceSwitchApi(Resource):
|
||||
|
||||
@auth_router.guard_workspace(scope=Scope.WORKSPACE_READ, allowed_token_types=frozenset({TokenType.OAUTH_ACCOUNT}))
|
||||
@returns(200, WorkspaceDetailResponse, description="Workspace detail")
|
||||
@with_session
|
||||
def post(self, session: Session, workspace_id: str, *, auth_data: AuthData):
|
||||
account = _load_account(session, auth_data.account_id)
|
||||
def post(self, workspace_id: str, *, auth_data: AuthData):
|
||||
account = _load_account(auth_data.account_id)
|
||||
|
||||
try:
|
||||
TenantService.switch_tenant(account, workspace_id, session=session)
|
||||
TenantService.switch_tenant(account, workspace_id, session=db.session())
|
||||
except AccountNotLinkTenantError:
|
||||
raise NotFound("workspace not found")
|
||||
|
||||
row = TenantService.find_workspace_for_account(str(auth_data.account_id), workspace_id, session=session)
|
||||
row = TenantService.find_workspace_for_account(str(auth_data.account_id), workspace_id, session=db.session())
|
||||
if row is None:
|
||||
raise NotFound("workspace not found")
|
||||
tenant, membership = row
|
||||
@@ -155,10 +151,9 @@ class WorkspaceMembersApi(Resource):
|
||||
@auth_router.guard_workspace(scope=Scope.WORKSPACE_READ, allowed_token_types=frozenset({TokenType.OAUTH_ACCOUNT}))
|
||||
@returns(200, MemberListResponse, description="Member list")
|
||||
@accepts(query=MemberListQuery)
|
||||
@with_session(write=False)
|
||||
def get(self, session: Session, workspace_id: str, *, auth_data: AuthData, query: MemberListQuery):
|
||||
tenant = _load_tenant(session, workspace_id)
|
||||
members = TenantService.get_tenant_members(tenant, session=session)
|
||||
def get(self, workspace_id: str, *, auth_data: AuthData, query: MemberListQuery):
|
||||
tenant = _load_tenant(workspace_id)
|
||||
members = TenantService.get_tenant_members(tenant, session=db.session())
|
||||
total = len(members)
|
||||
start = (query.page - 1) * query.limit
|
||||
page_items = members[start : start + query.limit]
|
||||
@@ -177,10 +172,9 @@ class WorkspaceMembersApi(Resource):
|
||||
)
|
||||
@returns(201, MemberInviteResponse, description="Member invited")
|
||||
@accepts(body=MemberInvitePayload)
|
||||
@with_session
|
||||
def post(self, session: Session, workspace_id: str, *, auth_data: AuthData, body: MemberInvitePayload):
|
||||
inviter = _load_account(session, auth_data.account_id)
|
||||
tenant = _load_tenant(session, workspace_id)
|
||||
def post(self, workspace_id: str, *, auth_data: AuthData, body: MemberInvitePayload):
|
||||
inviter = _load_account(auth_data.account_id)
|
||||
tenant = _load_tenant(workspace_id)
|
||||
|
||||
_check_member_invite_quota(str(tenant.id))
|
||||
|
||||
@@ -191,7 +185,7 @@ class WorkspaceMembersApi(Resource):
|
||||
language=None,
|
||||
role=body.role,
|
||||
inviter=inviter,
|
||||
session=session,
|
||||
session=db.session(),
|
||||
)
|
||||
except AccountAlreadyInTenantError as exc:
|
||||
raise BadRequest(str(exc))
|
||||
@@ -203,7 +197,7 @@ class WorkspaceMembersApi(Resource):
|
||||
raise BadRequest(str(exc))
|
||||
|
||||
normalized_email = body.email.lower()
|
||||
member = AccountService.get_account_by_email_with_case_fallback(normalized_email, session=session)
|
||||
member = AccountService.get_account_by_email_with_case_fallback(normalized_email, session=db.session())
|
||||
if member is None:
|
||||
# invite_new_member just created or fetched this account.
|
||||
raise RuntimeError("invited member missing from DB after invite")
|
||||
@@ -235,16 +229,15 @@ class WorkspaceMemberApi(Resource):
|
||||
allowed_roles=frozenset({TenantAccountRole.OWNER, TenantAccountRole.ADMIN}),
|
||||
)
|
||||
@returns(200, MemberActionResponse, description="Member removed")
|
||||
@with_session
|
||||
def delete(self, session: Session, workspace_id: str, member_id: str, *, auth_data: AuthData):
|
||||
operator = _load_account(session, auth_data.account_id)
|
||||
tenant = _load_tenant(session, workspace_id)
|
||||
member = AccountService.get_account_by_id(member_id, session=session)
|
||||
def delete(self, workspace_id: str, member_id: str, *, auth_data: AuthData):
|
||||
operator = _load_account(auth_data.account_id)
|
||||
tenant = _load_tenant(workspace_id)
|
||||
member = AccountService.get_account_by_id(member_id, session=db.session())
|
||||
if member is None:
|
||||
raise NotFound("member not found")
|
||||
|
||||
try:
|
||||
TenantService.remove_member_from_tenant(tenant, member, operator, session=session)
|
||||
TenantService.remove_member_from_tenant(tenant, member, operator, session=db.session())
|
||||
except CannotOperateSelfError as exc:
|
||||
raise BadRequest(str(exc))
|
||||
except NoPermissionError as exc:
|
||||
@@ -261,24 +254,15 @@ class WorkspaceMemberApi(Resource):
|
||||
)
|
||||
@returns(200, MemberActionResponse, description="Role updated")
|
||||
@accepts(body=MemberRoleUpdatePayload)
|
||||
@with_session
|
||||
def patch(
|
||||
self,
|
||||
session: Session,
|
||||
workspace_id: str,
|
||||
member_id: str,
|
||||
*,
|
||||
auth_data: AuthData,
|
||||
body: MemberRoleUpdatePayload,
|
||||
):
|
||||
operator = _load_account(session, auth_data.account_id)
|
||||
tenant = _load_tenant(session, workspace_id)
|
||||
member = AccountService.get_account_by_id(member_id, session=session)
|
||||
def patch(self, workspace_id: str, member_id: str, *, auth_data: AuthData, body: MemberRoleUpdatePayload):
|
||||
operator = _load_account(auth_data.account_id)
|
||||
tenant = _load_tenant(workspace_id)
|
||||
member = AccountService.get_account_by_id(member_id, session=db.session())
|
||||
if member is None:
|
||||
raise NotFound("member not found")
|
||||
|
||||
try:
|
||||
TenantService.update_member_role(tenant, member, body.role, operator, session=session)
|
||||
TenantService.update_member_role(tenant, member, body.role, operator, session=db.session())
|
||||
except CannotOperateSelfError as exc:
|
||||
raise BadRequest(str(exc))
|
||||
except NoPermissionError as exc:
|
||||
|
||||
@@ -5,13 +5,12 @@ from flask import request
|
||||
from flask_restx import Resource
|
||||
from flask_restx.api import HTTPStatus
|
||||
from pydantic import BaseModel, Field, TypeAdapter
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from controllers.common.schema import query_params_from_model, register_response_schema_models, register_schema_models
|
||||
from controllers.common.session import with_session
|
||||
from controllers.console.wraps import edit_permission_required
|
||||
from controllers.service_api import service_api_ns
|
||||
from controllers.service_api.wraps import validate_app_token
|
||||
from extensions.ext_database import db
|
||||
from extensions.ext_redis import redis_client
|
||||
from fields.annotation_fields import (
|
||||
Annotation,
|
||||
@@ -206,13 +205,12 @@ class AnnotationListApi(Resource):
|
||||
service_api_ns.models[AnnotationList.__name__],
|
||||
)
|
||||
@validate_app_token
|
||||
@with_session(write=False)
|
||||
def get(self, session: Session, app_model: App):
|
||||
def get(self, app_model: App):
|
||||
"""List annotations for the application."""
|
||||
query = AnnotationListQuery.model_validate(request.args.to_dict(flat=True))
|
||||
|
||||
annotation_list, total = AppAnnotationService.get_annotation_list_by_app_id(
|
||||
app_model.id, query.page, query.limit, query.keyword, session
|
||||
app_model.id, query.page, query.limit, query.keyword, session=db.session()
|
||||
)
|
||||
annotation_models = TypeAdapter(list[Annotation]).validate_python(annotation_list, from_attributes=True)
|
||||
return AnnotationList(
|
||||
@@ -249,12 +247,13 @@ class AnnotationListApi(Resource):
|
||||
service_api_ns.models[Annotation.__name__],
|
||||
)
|
||||
@validate_app_token
|
||||
@with_session
|
||||
def post(self, session: Session, app_model: App):
|
||||
def post(self, app_model: App):
|
||||
"""Create a new annotation."""
|
||||
payload = AnnotationCreatePayload.model_validate(service_api_ns.payload or {})
|
||||
insert_args: InsertAnnotationArgs = {"question": payload.question, "answer": payload.answer}
|
||||
annotation = AppAnnotationService.insert_app_annotation_directly(insert_args, app_model.id, session)
|
||||
annotation = AppAnnotationService.insert_app_annotation_directly(
|
||||
insert_args, app_model.id, session=db.session()
|
||||
)
|
||||
return dump_response(Annotation, annotation), HTTPStatus.CREATED
|
||||
|
||||
|
||||
@@ -288,15 +287,14 @@ class AnnotationUpdateDeleteApi(Resource):
|
||||
service_api_ns.models[Annotation.__name__],
|
||||
)
|
||||
@validate_app_token
|
||||
@with_session
|
||||
@edit_permission_required
|
||||
def put(self, session: Session, app_model: App, annotation_id: UUID):
|
||||
def put(self, app_model: App, annotation_id: UUID):
|
||||
"""Update an existing annotation."""
|
||||
payload = AnnotationCreatePayload.model_validate(service_api_ns.payload or {})
|
||||
update_args: UpdateAnnotationArgs = {"question": payload.question, "answer": payload.answer}
|
||||
app_ref = AppRefService.create_app_ref(app_model)
|
||||
annotation_ref = AppRefService.create_annotation_ref(app_ref, str(annotation_id))
|
||||
annotation = AppAnnotationService.update_app_annotation_directly(update_args, annotation_ref, session)
|
||||
annotation = AppAnnotationService.update_app_annotation_directly(update_args, annotation_ref, db.session())
|
||||
return dump_response(Annotation, annotation)
|
||||
|
||||
@service_api_ns.doc(
|
||||
@@ -321,11 +319,10 @@ class AnnotationUpdateDeleteApi(Resource):
|
||||
}
|
||||
)
|
||||
@validate_app_token
|
||||
@with_session
|
||||
@edit_permission_required
|
||||
def delete(self, session: Session, app_model: App, annotation_id: UUID):
|
||||
def delete(self, app_model: App, annotation_id: UUID):
|
||||
"""Delete an annotation."""
|
||||
app_ref = AppRefService.create_app_ref(app_model)
|
||||
annotation_ref = AppRefService.create_annotation_ref(app_ref, str(annotation_id))
|
||||
AppAnnotationService.delete_app_annotation(annotation_ref, session)
|
||||
AppAnnotationService.delete_app_annotation(annotation_ref, db.session())
|
||||
return "", 204
|
||||
|
||||
@@ -2,7 +2,6 @@ from typing import Any, cast
|
||||
|
||||
from flask_restx import Resource
|
||||
from pydantic import Field
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from controllers.common.agent_app_parameters import get_published_agent_app_feature_dict_and_user_input_form
|
||||
from controllers.common.fields import Parameters
|
||||
@@ -14,7 +13,7 @@ from core.app.app_config.common.parameters_mapping import get_parameters_from_fe
|
||||
from core.app.apps.agent_app.errors import AgentAppGeneratorError, AgentAppNotPublishedError
|
||||
from extensions.ext_database import db
|
||||
from fields.base import ResponseModel
|
||||
from models.model import App, AppMode, load_annotation_reply_config
|
||||
from models.model import App, AppMode
|
||||
from services.app_service import AppService
|
||||
|
||||
|
||||
@@ -33,13 +32,9 @@ class AppMetaResponse(ResponseModel):
|
||||
register_response_schema_models(service_api_ns, Parameters, AppMetaResponse, AppInfoResponse)
|
||||
|
||||
|
||||
def _get_agent_app_feature_dict_and_user_input_form(
|
||||
app_model: App,
|
||||
*,
|
||||
session: Session,
|
||||
) -> tuple[dict[str, Any], list[dict[str, Any]]]:
|
||||
def _get_agent_app_feature_dict_and_user_input_form(app_model: App) -> tuple[dict[str, Any], list[dict[str, Any]]]:
|
||||
try:
|
||||
return get_published_agent_app_feature_dict_and_user_input_form(app_model, session=session)
|
||||
return get_published_agent_app_feature_dict_and_user_input_form(app_model)
|
||||
except AgentAppNotPublishedError:
|
||||
raise AgentNotPublishedError()
|
||||
except AgentAppGeneratorError:
|
||||
@@ -78,31 +73,23 @@ class AppParameterApi(Resource):
|
||||
|
||||
Returns the input form parameters and configuration for the application.
|
||||
"""
|
||||
session = db.session()
|
||||
features_dict: dict[str, Any]
|
||||
user_input_form: list[dict[str, Any]]
|
||||
if app_model.mode == AppMode.AGENT:
|
||||
features_dict, user_input_form = _get_agent_app_feature_dict_and_user_input_form(
|
||||
app_model,
|
||||
session=session,
|
||||
)
|
||||
features_dict, user_input_form = _get_agent_app_feature_dict_and_user_input_form(app_model)
|
||||
elif app_model.mode in {AppMode.ADVANCED_CHAT, AppMode.WORKFLOW}:
|
||||
workflow = app_model.workflow_with_session(session=session)
|
||||
workflow = app_model.workflow
|
||||
if workflow is None:
|
||||
raise AppUnavailableError()
|
||||
|
||||
features_dict = workflow.features_dict
|
||||
user_input_form = workflow.user_input_form(to_old_structure=True)
|
||||
else:
|
||||
app_model_config = app_model.app_model_config_with_session(session=session)
|
||||
app_model_config = app_model.app_model_config
|
||||
if app_model_config is None:
|
||||
raise AppUnavailableError()
|
||||
|
||||
annotation_reply = load_annotation_reply_config(session, app_model.id)
|
||||
features_dict = cast(
|
||||
dict[str, Any],
|
||||
app_model_config.to_dict(annotation_reply=annotation_reply),
|
||||
)
|
||||
features_dict = cast(dict[str, Any], app_model_config.to_dict())
|
||||
|
||||
user_input_form = features_dict.get("user_input_form", [])
|
||||
|
||||
|
||||
@@ -101,15 +101,10 @@ class AudioApi(Resource):
|
||||
|
||||
Accepts an audio file upload and returns the transcribed text.
|
||||
"""
|
||||
file = request.files.get("file")
|
||||
file = request.files["file"]
|
||||
|
||||
try:
|
||||
response = AudioService.transcript_asr(
|
||||
app_model=app_model,
|
||||
file=file,
|
||||
session=db.session(),
|
||||
end_user=end_user.id,
|
||||
)
|
||||
response = AudioService.transcript_asr(app_model=app_model, file=file, end_user=end_user.id)
|
||||
|
||||
return dump_response(AudioTranscriptResponse, response)
|
||||
except services.errors.app_model_config.AppModelConfigBrokenError:
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user