Compare commits

..
Author SHA1 Message Date
yyh 0b9e03c2c1 fix(agent-v2): preserve interrupted build input 2026-06-28 03:49:25 +08:00
autofix-ci[bot]andGitHub b34eb8f24b [autofix.ci] apply automated fixes 2026-06-27 19:42:43 +00:00
yyh 625cfad888 fix(agent-v2): fence stale build draft refresh 2026-06-28 03:36:34 +08:00
yyh 3859367e3a fix(agent-v2): cancel inline build refresh timer 2026-06-28 03:21:09 +08:00
yyh 92c0974e19 fix(agent-v2): fail explicit draft save commands 2026-06-28 03:14:51 +08:00
yyh f7d859afac fix(agent-v2): guard build draft refresh lifecycle 2026-06-28 03:12:34 +08:00
yyh b522f7df40 fix(agent-v2): align chat feature draft source 2026-06-28 03:07:16 +08:00
autofix-ci[bot]andGitHub 1787145f2a [autofix.ci] apply automated fixes 2026-06-27 19:00:10 +00:00
yyh f581dfc016 docs(agent-v2): align configure module boundary 2026-06-28 02:54:09 +08:00
yyh 26291a0f42 test(agent-v2): tighten state graph review coverage 2026-06-28 02:49:03 +08:00
yyh 80b9f5e083 fix(agent-v2): align inline build draft source 2026-06-28 01:55:05 +08:00
yyh bbebeec3dc fix(chat): settle stream read failures 2026-06-28 01:43:30 +08:00
yyh 6a69ec2a8a fix(agent-v2): honor debug refresh contract 2026-06-28 01:33:46 +08:00
autofix-ci[bot]andGitHub f874a6a019 [autofix.ci] apply automated fixes 2026-06-27 17:31:32 +00:00
yyh bcdd52a27b fix(agent-v2): reconcile publish state owners 2026-06-28 01:24:56 +08:00
autofix-ci[bot]andGitHub 32e9de8ae3 [autofix.ci] apply automated fixes 2026-06-27 17:17:33 +00:00
yyh 35c9e3e3d3 fix(agent-v2): close build draft state gaps 2026-06-28 01:12:04 +08:00
autofix-ci[bot]andGitHub d37783c895 [autofix.ci] apply automated fixes 2026-06-27 16:55:18 +00:00
yyh cceeabcb46 fix(agent-v2): align build draft lifecycle owners 2026-06-28 00:50:01 +08:00
yyh 8536c232ba fix(agent-v2): stabilize build chat run state 2026-06-28 00:35:16 +08:00
yyh 8383571c53 fix(agent-v2): preserve publish state for reverted drafts 2026-06-28 00:35:00 +08:00
yyh 953b42ac67 fix(agent-v2): clear publish action after reverted draft 2026-06-28 00:10:39 +08:00
yyh 4e069c1f23 fix(agent-v2): save effective model before build run 2026-06-28 00:07:08 +08:00
yyh 09b5d63c39 refactor: merge inline agent common translations 2026-06-28 00:05:05 +08:00
yyh 8f2cd4e5b0 fix(agent-v2): disable restart when chat reset is unavailable 2026-06-28 00:01:51 +08:00
yyh b57e5da995 fix(agent-v2): stabilize chat reset command 2026-06-27 23:52:51 +08:00
yyh daf13834a6 fix(agent-v2): stabilize build chat state 2026-06-27 23:47:19 +08:00
yyh 309777f8fc fix(agent-v2): skip unchanged draft save before build 2026-06-27 23:19:49 +08:00
yyh 184670f49d fix(agent-v2): clarify build draft action states 2026-06-27 23:10:58 +08:00
yyh 63aa8af5c1 refactor(agent-v2): scope configure conversation state 2026-06-27 23:09:31 +08:00
yyh ee30e59871 fix(agent-v2): keep build chat visible after conversation starts 2026-06-27 23:01:06 +08:00
yyh 0199589c3b refactor(agent-v2): clarify configure state graph 2026-06-27 22:57:36 +08:00
yyhandGitHub 7bb94cb6fe fix: gate deployments in route layout (#38078) 2026-06-27 13:35:37 +00:00
yyhandGitHub 484633d261 fix: prevent exiting toasts from blocking page clicks (#38063) 2026-06-27 07:35:25 +00:00
Stephen ZhouGitHubcopilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
a14310fc62 refactor(web): organize deployment feature state (#38065)
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
2026-06-27 07:34:46 +00:00
Byron.wangandGitHub 8218694691 fix: improve workflow run archive idempotency and batching (#38027) 2026-06-27 11:09:57 +08:00
yyhandGitHub 17bee5fb32 fix: improve members role chip accessibility (#38037) 2026-06-26 18:40:03 +00:00
yyhandGitHub 446b3962c1 fix: remove aria-busy from loading button (#38036) 2026-06-26 18:39:42 +00:00
github-actions[bot]GitHubclaude[bot] <41898282+claude[bot]@users.noreply.github.com>
00b4cdc68e chore(i18n): sync translations with en-US (#38035)
Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
2026-06-26 13:54:30 +00:00
Dx.andGitHub 449b46b863 fix(web): download markdown file-preview links as attachments (#38030) 2026-06-26 13:46:44 +00:00
8d09b32cd5 chore: inject session (#37941)
Co-authored-by: WH-2099 <wh2099@pm.me>
2026-06-26 13:42:47 +00:00
盐粒 YanliGitHubJoelyyhYansong Zhangautofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
52c106b532 feat(agent-v2): sync nightly updates to main (2026-06-25) (#37915)
Co-authored-by: Joel <iamjoel007@gmail.com>
Co-authored-by: yyh <yuanyouhuilyz@gmail.com>
Co-authored-by: Yansong Zhang <916125788@qq.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-06-26 13:13:35 +00:00
5b8679468d docs(web): fix testing guide link (#38006)
Co-authored-by: Harsh Kashyap <Harsh23Kashyap@users.noreply.github.com>
2026-06-26 12:50:12 +00:00
EvanandGitHub 4fbfedd287 refactor: replace patch logger with caplog in test_version.py (#38029) 2026-06-26 12:07:44 +00:00
CyberGitHubClaude Opus 4.7yyhyyhhjlarryautofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
051f4b32e3 feat(web): add customizable input placeholder for Agent/Chatflow/Chatbot web app (#37790)
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Co-authored-by: yyh <92089059+lyzno1@users.noreply.github.com>
Co-authored-by: yyh <yuanyouhuilyz@gmail.com>
Co-authored-by: hjlarry <hjlarry@163.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-06-26 09:57:31 +00:00
非法操作andGitHub e22fd9efd6 fix: plugin installation task popover layout when some failed too long (#38000) 2026-06-26 09:36:36 +00:00
35eeb743d1 fix(web): capture blog UTM/slug attribution reliably under CSP (#38022)
Co-authored-by: CodingOnStar <hanxujiang@dify.com>
2026-06-26 08:35:43 +00:00
林玮 (Jade Lin)andGitHub 1dbda1463e feat(api): cache workflow provider configurations (#37980) 2026-06-26 08:04:37 +00:00
Mr_xieGitHubautofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
3f2ef24755 refactor: pass db session into service calls (#37403) (#38016)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-06-26 07:53:27 +00:00
Escape0707andGitHub eb4ec93cea chore: migrate account role test to testcontainers (#38010) 2026-06-26 05:51:14 +00:00
Xiyuan ChenandGitHub fa6f4b0ea5 refactor(api): enforce enums over string literals on openapi surface (#38009) 2026-06-26 05:48:21 +00:00
267b34caaf fix(web): keep HITL input-field save button visible in the edit dialog (#38007)
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-26 05:40:51 +00:00
yyhandGitHub 6ab1dd06ac fix: isolate node selector keyboard events (#37998) 2026-06-26 05:38:10 +00:00
Stephen ZhouandGitHub 113d6d7e00 refactor: align deployment detail state ownership (#38008) 2026-06-26 05:37:21 +00:00
Mohammed AqibGitHubautofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
a66b1de477 refactor: Use pytest caplog instead of logger patching in test_spec.py (#37997)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-06-26 04:48:22 +00:00
Stephen ZhouandGitHub 16b698b54d refactor: simplify deployment access ownership (#37994) 2026-06-26 04:10:56 +00:00
yyhandGitHub 4cd8b8c733 fix(web): make marketplace URL builder SSR-safe (#37944) 2026-06-26 03:35:22 +00:00
677ab01806 refactor(tests): use caplog in workflow collaboration service tests (#37468) (#37991)
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-26 03:14:06 +00:00
Stephen ZhouandGitHub f4e832f35c fix: decouple deployment access control dialog (#37936) 2026-06-26 02:16:29 +00:00
kurokoboGitHubautofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
1c5e1280cb feat(webapp): display app description on chat and text-generation app screens (#37345)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-06-26 02:03:16 +00:00
-LAN-GitHubautofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
0035d90e36 fix(api): require edit access for trace config changes (#37973)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-06-26 00:51:39 +00:00
Xiyuan ChenandGitHub 2047e0dc12 fix(cli): use a(n) article in difyctl prerelease warning (#37976) 2026-06-26 00:49:42 +00:00
chaririandGitHub 2e9c0a3c7a refactor(api): migrate response contract tooling to BaseModel (#37950) 2026-06-26 00:11:04 +00:00
ojasarora.ethandGitHub a246dc8b17 ci: emit pyrefly diagnostics as GitHub workflow commands (#37974) 2026-06-25 18:27:46 +00:00
ojasarora.ethandGitHub bb921bcc45 test: replace logger mock with caplog in workflow collaboration test (#37971) 2026-06-25 16:45:59 +00:00
QuantumGhostandGitHub 4f4ac27de2 chore(api): surface pyrefly output on type-check failures (#37934) 2026-06-25 13:47:10 +00:00
1034 changed files with 35976 additions and 22134 deletions
+12 -2
View File
@@ -12,6 +12,7 @@ Use this as the component decision guide for Dify web. Existing code is referenc
| Question | Default | Promote or extract only when |
| --- | --- | --- |
| Where should code live? | Keep it local to the feature workflow, route, or owner. | Multiple verticals need the same stable primitive. |
| How should route/tab folders be named? | Match the current route segment, tab name, or user-visible surface. | Keep a historical or broader parent only when it still owns multiple surfaces. |
| Who owns state, data, and handlers? | The lowest component that uses them. | A parent coordinates shared loading, errors, empty UI, selection, submission, navigation, or one consistent snapshot. |
| Should this become Jotai state? | Keep synchronous UI/form state in component or DOM state. | Siblings need one source of truth, the value drives atoms, or scoped workflow state must survive hidden/unmounted steps. |
| Should URL state enter Jotai? | Let Next.js route params and `nuqs` own URL state and updates. | Query atoms or shared derived atoms need a read-only bridge hydrated at the route/surface boundary. |
@@ -23,7 +24,9 @@ Use this as the component decision guide for Dify web. Existing code is referenc
- Search before adding UI, hooks, helpers, query utilities, or styling patterns. Reuse existing base components, feature components, hooks, utilities, and design styles when they fit.
- Follow Dify's CSS-first Tailwind v4 contract from `packages/dify-ui/README.md` and `packages/dify-ui/AGENTS.md`. Prefer design-system tokens, utilities, and radius mappings over generic Tailwind choices.
- Group feature code by workflow, route, or ownership area: components, hooks, local types, query helpers, atoms, constants, and small utilities should live near the code that changes with them.
- Group feature code by workflow, route, or ownership area with route-aligned names: components, hooks, local types, query helpers, atoms, constants, tests, and small utilities should live near the code that changes with them.
- For each feature module, keep a module-local `README.md` as a boundary note. Start with the module name, a brief one-sentence description, then split dependencies into `Internal Modules` and `External Modules` sections; keep both sections and write `None.` when one category is empty. `Internal Modules` lists modules inside the same overall feature using paths from that feature root, such as `shared/domain/runtime-status`; `External Modules` lists project modules outside the feature using paths from the web root without a `web/` prefix, such as `app/components/base/skeleton`. Omit npm packages, workspace package dependencies, and whitelisted plumbing modules. Do not copy caller-relative import paths into the README.
- Module README whitelist: `@/service/client`, `@/next/*`.
- Keep source/default selection, validation, dirty checks, and payload shaping close to the workflow that owns submit behavior. Do not hide flow-specific priority order, fallback behavior, or submit semantics in generic utilities.
- Prefer direct conditionals for small branch-specific decisions, especially form source selection and request payload assembly.
- Loading states for page sections, cards, lists, tables, forms, and drawers should be skeletons scoped to the content being loaded. Use spinners only for small inline busy indicators.
@@ -32,6 +35,8 @@ Use this as the component decision guide for Dify web. Existing code is referenc
- State-heavy wizards, drawers, modals, and secondary workflows can be a small feature surface: an entry file, one feature-local state file when Jotai is actually needed, and shallow `ui/` owners that match real visual regions.
- The entry file handles route integration, provider wiring, close behavior, and surface mounting. The composition owner handles high-level workflow branching. The closest visual owner handles section branching.
- When a page or tab maps to a route segment, name its feature folder after that route/tab surface instead of a stale parent grouping. Remove misleading intermediate folders when only one surface remains.
- When a tab folder grows into several independent sections or action areas, split the first level by product/visual owners. Keep the root for the entry component and cross-owner state, colocate tests with the owner folder, and put truly shared local UI under a specifically named `components/` file.
- Repeated TanStack query calls in sibling components are acceptable when each component independently consumes the data; TanStack Query deduplicates and shares cache.
- Pass stable domain identity across boundaries. Do not forward derived presentation state when the receiver can derive it from its own data source.
- A component that owns a visual surface should also own data access, loading, empty, and error states for content rendered inside it unless a parent truly coordinates that state.
@@ -46,6 +51,7 @@ Use this as the component decision guide for Dify web. Existing code is referenc
- Use uncontrolled `@langgenius/dify-ui/form` and `@langgenius/dify-ui/field` controls for edit/create forms whose fields are read only at submit time. Initialize query-backed defaults with `defaultValue` and keyed remounts.
- Promote form state to atoms only when another component must react to in-progress values, a draft must survive unmount/remount in the scoped workflow, or multiple steps share the same editable draft before submit.
- Treat `useParams`, route args, and `nuqs` query state as framework-owned state. When atom logic needs those values, hydrate primitive atoms at the route or surface boundary, such as with `useHydrateAtoms(..., { dangerouslyForceHydrate: true })`; keep URL updates in the route/query-state APIs instead of write atoms.
- Within a route-owned feature, choose one source for route identity. If route params are bridged into feature atoms, use that bridge consistently for route-derived queries and actions instead of also threading the same route id through page, tab, and section props.
- For async work tied to atom state, use `atomWithQuery` or `atomWithMutation`; write atoms should update only the inputs that drive those atoms. This applies to pure frontend async work as well as network requests, so do not hand-roll loading/error/in-flight state with `useState` or `useRef` for atom-orchestrated async behavior. For component-owned remote work, use `useQuery` or `useMutation` directly.
- Row-local async state belongs to the row owner unless it participates in a shared Jotai workflow or needs atom-scoped reset semantics.
- Leave query and mutation atoms unscoped so they keep shared QueryClient cache and invalidation behavior. Scope resettable primitives and explicit hydration tuples; scope a derived atom only when every dependency should be private to that surface.
@@ -60,8 +66,10 @@ Use this as the component decision guide for Dify web. Existing code is referenc
- Type component signatures directly; do not use `FC` or `React.FC`.
- Prefer `function` for top-level components and module helpers. Use arrow functions for local callbacks, handlers, and lambda-style APIs.
- Prefer named exports. Use default exports only where the framework requires them, such as Next.js route files.
- Avoid barrel files that only re-export secondary owners. `index.tsx` is acceptable for a route/tab entry component; import header controls, switches, sections, and row owners from their concrete owner files.
- Type simple one-off props inline. Use a named `Props` type only when reused, exported, complex, or clearer.
- Use API-generated or API-returned types at component boundaries. Keep small UI conversion helpers and one-off UI extensions beside the component that needs them.
- Avoid `common.tsx` buckets for shared UI. Use a feature-local `components/` folder with concrete filenames that describe the shared role.
- Do not create type aliases that only rename another type. Use aliases only for real UI concepts, refinements, or reusable local contracts.
- Name values by their domain role and backend API contract, especially persistent IDs and route params. Normalize framework or route params at the boundary.
- Put fallback and invariant checks in the lowest component that already handles that state. Do not extract helpers whose only behavior is hiding missing display data.
@@ -83,11 +91,13 @@ Use this as the component decision guide for Dify web. Existing code is referenc
- Keep `web/contract/*` as the API shape source of truth and follow the `{ params, query?, body? }` input shape.
- Consume generated queries with `useQuery(consoleQuery.xxx.queryOptions(...))` or `useQuery(marketplaceQuery.xxx.queryOptions(...))`.
- If a generated query input comes from an atom, including a route-identity bridge atom, keep the query in `atomWithQuery`; do not unwrap the atom in a component just to call `useQuery`.
- Consume owner-local mutations with `useMutation(consoleQuery.xxx.mutationOptions(...))` or `useMutation(marketplaceQuery.xxx.mutationOptions(...))` when pending/error state is not consumed by feature atoms.
- In `atomWithQuery`, `atomWithInfiniteQuery`, and `atomWithMutation`, return generated `queryOptions()`, `infiniteOptions()`, or `mutationOptions()` directly. Pass `enabled`, `retry`, `placeholderData`, `select`, and pagination options into the generated call instead of spreading options into a hand-built object.
- For generated oRPC options with missing required input, branch the whole input with `input: condition ? validInput : skipToken` and `enabled: Boolean(condition)`. Never place `skipToken` inside a nested placeholder payload or coerce required IDs to `''`.
- When prefetch and render use the same request, extract local query options or a query-options atom so `prefetchQuery` and `useQuery`/`atomWithQuery` share the exact options.
- For custom query or mutation functions, wrap options with TanStack `queryOptions(...)` or `mutationOptions(...)`.
- Do not extract generated `queryOptions(...)` into a helper solely to share input construction; extract only when prefetch/render must share exact options or the helper owns real domain behavior.
- Avoid pass-through hooks and thin `web/service/use-*` wrappers that only rename generated options. Keep feature hooks for real orchestration, workflow state, or shared domain behavior.
- Put shared cache behavior in `createTanstackQueryUtils(...experimental_defaults...)`. Component or atom callbacks may handle local toasts, closing dialogs, and navigation, but should not replace shared invalidation or patch shared server state locally.
- For overlays that may open heavier secondary content, prefetch from the trigger/menu open event with `queryClient.prefetchQuery(queryOptions)` when `onOpenChange` is available. Do not mount hidden subscribers just to warm cache.
@@ -96,7 +106,7 @@ Use this as the component decision guide for Dify web. Existing code is referenc
## Boundaries And Overlays
- Use the first level below a page or tab to organize independent page sections when it adds structure. This layer is layout/semantic first, not automatically the data owner.
- Use the first level below a page or tab to organize independent page sections when it adds structure or the root folder becomes noisy. This layer is layout/semantic first, not automatically the data owner.
- Treat component names, semantic roles, and user- or design-marked visual regions as boundary constraints. Keep adjacent UI as a sibling owner or introduce a correctly named broader owner.
- Keep cohesive forms, menu bodies, and one-off helpers local unless they need their own state, reuse, or semantic boundary.
- Separate hidden secondary surfaces from the trigger's main flow. For dialogs, dropdowns, popovers, and similar branches, extract a small local component when hidden content would obscure the parent.
+3 -3
View File
@@ -16,7 +16,7 @@ concurrency:
jobs:
api-unit:
name: API Unit Tests
runs-on: depot-ubuntu-24.04-4
runs-on: depot-ubuntu-24.04
env:
COVERAGE_FILE: coverage-unit
defaults:
@@ -75,7 +75,7 @@ jobs:
api-integration:
name: API Integration Tests
runs-on: depot-ubuntu-24.04-4
runs-on: depot-ubuntu-24.04
env:
COVERAGE_FILE: coverage-integration
STORAGE_TYPE: opendal
@@ -129,7 +129,7 @@ jobs:
api-coverage:
name: API Coverage
runs-on: depot-ubuntu-24.04-4
runs-on: depot-ubuntu-24.04
needs:
- api-unit
- api-integration
+1 -1
View File
@@ -171,7 +171,7 @@ jobs:
create-manifest:
needs: build
runs-on: depot-ubuntu-24.04-4
runs-on: depot-ubuntu-24.04
if: github.repository == 'langgenius/dify'
strategy:
matrix:
+2 -2
View File
@@ -23,7 +23,7 @@ concurrency:
jobs:
validate:
name: validate manifest + resolve target Dify release
runs-on: depot-ubuntu-24.04-4
runs-on: depot-ubuntu-24.04
if: github.repository == 'langgenius/dify'
permissions:
contents: read
@@ -87,7 +87,7 @@ jobs:
release:
name: build + attach standalone binaries (all targets)
needs: validate
runs-on: depot-ubuntu-24.04-4
runs-on: depot-ubuntu-24.04
permissions:
contents: write
defaults:
+2 -2
View File
@@ -9,7 +9,7 @@ concurrency:
jobs:
db-migration-test-postgres:
runs-on: depot-ubuntu-24.04-4
runs-on: depot-ubuntu-24.04
steps:
- name: Checkout code
@@ -59,7 +59,7 @@ jobs:
run: uv run --directory api flask upgrade-db
db-migration-test-mysql:
runs-on: depot-ubuntu-24.04-4
runs-on: depot-ubuntu-24.04
steps:
- name: Checkout code
+1 -1
View File
@@ -13,7 +13,7 @@ on:
jobs:
deploy:
runs-on: depot-ubuntu-24.04-4
runs-on: depot-ubuntu-24.04
if: |
github.event.workflow_run.conclusion == 'success' &&
github.event.workflow_run.head_branch == 'deploy/agent'
+1 -1
View File
@@ -10,7 +10,7 @@ on:
jobs:
deploy:
runs-on: depot-ubuntu-24.04-4
runs-on: depot-ubuntu-24.04
if: |
github.event.workflow_run.conclusion == 'success' &&
github.event.workflow_run.head_branch == 'deploy/dev'
+1 -1
View File
@@ -13,7 +13,7 @@ on:
jobs:
deploy:
runs-on: depot-ubuntu-24.04-4
runs-on: depot-ubuntu-24.04
if: |
github.event.workflow_run.conclusion == 'success' &&
github.event.workflow_run.head_branch == 'deploy/enterprise'
+1 -1
View File
@@ -13,7 +13,7 @@ on:
jobs:
deploy:
runs-on: depot-ubuntu-24.04-4
runs-on: depot-ubuntu-24.04
if: |
github.event.workflow_run.conclusion == 'success' &&
github.event.workflow_run.head_branch == 'deploy/saas'
+1 -1
View File
@@ -22,7 +22,7 @@ concurrency:
jobs:
check-cherry-pick-provenance:
name: Require cherry-pick provenance
runs-on: depot-ubuntu-24.04-4
runs-on: depot-ubuntu-24.04
steps:
- uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
with:
+1 -1
View File
@@ -7,7 +7,7 @@ jobs:
permissions:
contents: read
pull-requests: write
runs-on: depot-ubuntu-24.04-4
runs-on: depot-ubuntu-24.04
steps:
- uses: actions/labeler@f27b608878404679385c85cfa523b85ccb86e213 # v6.1.0
with:
+14 -14
View File
@@ -23,7 +23,7 @@ concurrency:
jobs:
pre_job:
name: Skip Duplicate Checks
runs-on: depot-ubuntu-24.04-4
runs-on: depot-ubuntu-24.04
outputs:
should_skip: ${{ steps.skip_check.outputs.should_skip || 'false' }}
steps:
@@ -39,7 +39,7 @@ jobs:
name: Check Changed Files
needs: pre_job
if: needs.pre_job.outputs.should_skip != 'true'
runs-on: depot-ubuntu-24.04-4
runs-on: depot-ubuntu-24.04
outputs:
api-changed: ${{ steps.changes.outputs.api }}
cli-changed: ${{ steps.changes.outputs.cli }}
@@ -152,7 +152,7 @@ jobs:
- pre_job
- check-changes
if: needs.pre_job.outputs.should_skip != 'true' && needs.check-changes.outputs.api-changed != 'true'
runs-on: depot-ubuntu-24.04-4
runs-on: depot-ubuntu-24.04
steps:
- name: Report skipped API tests
run: echo "No API-related changes detected; skipping API tests."
@@ -165,7 +165,7 @@ jobs:
- check-changes
- api-tests-run
- api-tests-skip
runs-on: depot-ubuntu-24.04-4
runs-on: depot-ubuntu-24.04
steps:
- name: Finalize API Tests status
env:
@@ -212,7 +212,7 @@ jobs:
- pre_job
- check-changes
if: needs.pre_job.outputs.should_skip != 'true' && needs.check-changes.outputs.cli-changed != 'true'
runs-on: depot-ubuntu-24.04-4
runs-on: depot-ubuntu-24.04
steps:
- name: Report skipped CLI tests
run: echo "No CLI-related changes detected; skipping CLI tests."
@@ -225,7 +225,7 @@ jobs:
- check-changes
- cli-tests-run
- cli-tests-skip
runs-on: depot-ubuntu-24.04-4
runs-on: depot-ubuntu-24.04
steps:
- name: Finalize CLI Tests status
env:
@@ -272,7 +272,7 @@ jobs:
- pre_job
- check-changes
if: needs.pre_job.outputs.should_skip != 'true' && needs.check-changes.outputs.web-changed != 'true'
runs-on: depot-ubuntu-24.04-4
runs-on: depot-ubuntu-24.04
steps:
- name: Report skipped web tests
run: echo "No web-related changes detected; skipping web tests."
@@ -285,7 +285,7 @@ jobs:
- check-changes
- web-tests-run
- web-tests-skip
runs-on: depot-ubuntu-24.04-4
runs-on: depot-ubuntu-24.04
steps:
- name: Finalize Web Tests status
env:
@@ -331,7 +331,7 @@ jobs:
- pre_job
- check-changes
if: needs.pre_job.outputs.should_skip != 'true' && needs.check-changes.outputs.e2e-changed != 'true'
runs-on: depot-ubuntu-24.04-4
runs-on: depot-ubuntu-24.04
steps:
- name: Report skipped web full-stack e2e
run: echo "No E2E-related changes detected; skipping web full-stack E2E."
@@ -344,7 +344,7 @@ jobs:
- check-changes
- web-e2e-run
- web-e2e-skip
runs-on: depot-ubuntu-24.04-4
runs-on: depot-ubuntu-24.04
steps:
- name: Finalize Web Full-Stack E2E status
env:
@@ -396,7 +396,7 @@ jobs:
- pre_job
- check-changes
if: needs.pre_job.outputs.should_skip != 'true' && needs.check-changes.outputs.vdb-changed != 'true'
runs-on: depot-ubuntu-24.04-4
runs-on: depot-ubuntu-24.04
steps:
- name: Report skipped VDB tests
run: echo "No VDB-related changes detected; skipping VDB tests."
@@ -409,7 +409,7 @@ jobs:
- check-changes
- vdb-tests-run
- vdb-tests-skip
runs-on: depot-ubuntu-24.04-4
runs-on: depot-ubuntu-24.04
steps:
- name: Finalize VDB Tests status
env:
@@ -455,7 +455,7 @@ jobs:
- pre_job
- check-changes
if: needs.pre_job.outputs.should_skip != 'true' && needs.check-changes.outputs.migration-changed != 'true'
runs-on: depot-ubuntu-24.04-4
runs-on: depot-ubuntu-24.04
steps:
- name: Report skipped DB migration tests
run: echo "No migration-related changes detected; skipping DB migration tests."
@@ -468,7 +468,7 @@ jobs:
- check-changes
- db-migration-test-run
- db-migration-test-skip
runs-on: depot-ubuntu-24.04-4
runs-on: depot-ubuntu-24.04
steps:
- name: Finalize DB Migration Test status
env:
+1 -1
View File
@@ -12,7 +12,7 @@ permissions: {}
jobs:
comment:
name: Comment PR with pyrefly diff
runs-on: depot-ubuntu-24.04-4
runs-on: depot-ubuntu-24.04
permissions:
actions: read
contents: read
+1 -1
View File
@@ -10,7 +10,7 @@ permissions:
jobs:
pyrefly-diff:
runs-on: depot-ubuntu-24.04-4
runs-on: depot-ubuntu-24.04
permissions:
contents: read
issues: write
@@ -12,7 +12,7 @@ permissions: {}
jobs:
comment:
name: Comment PR with type coverage
runs-on: depot-ubuntu-24.04-4
runs-on: depot-ubuntu-24.04
permissions:
actions: read
contents: read
+1 -1
View File
@@ -10,7 +10,7 @@ permissions:
jobs:
pyrefly-type-coverage:
runs-on: depot-ubuntu-24.04-4
runs-on: depot-ubuntu-24.04
permissions:
contents: read
issues: write
+1 -1
View File
@@ -16,7 +16,7 @@ jobs:
name: Validate PR title
permissions:
pull-requests: read
runs-on: depot-ubuntu-24.04-4
runs-on: depot-ubuntu-24.04
steps:
- name: Complete merge group check
if: github.event_name == 'merge_group'
+1 -1
View File
@@ -12,7 +12,7 @@ on:
jobs:
stale:
runs-on: depot-ubuntu-24.04-4
runs-on: depot-ubuntu-24.04
permissions:
issues: write
pull-requests: write
+5 -3
View File
@@ -15,7 +15,7 @@ permissions:
jobs:
python-style:
name: Python Style
runs-on: depot-ubuntu-24.04-4
runs-on: depot-ubuntu-24.04
steps:
- name: Checkout code
@@ -53,6 +53,8 @@ jobs:
- name: Run Type Checks
if: steps.changed-files.outputs.any_changed == 'true'
env:
PYREFLY_OUTPUT_FORMAT: github
run: make type-check-core
- name: Dotenv check
@@ -61,7 +63,7 @@ jobs:
web-style:
name: Web Style
runs-on: depot-ubuntu-24.04-4
runs-on: depot-ubuntu-24.04
defaults:
run:
working-directory: ./web
@@ -175,7 +177,7 @@ jobs:
superlinter:
name: SuperLinter
runs-on: depot-ubuntu-24.04-4
runs-on: depot-ubuntu-24.04
steps:
- name: Checkout code
+1 -1
View File
@@ -17,7 +17,7 @@ concurrency:
jobs:
build:
name: unit test for Node.js SDK
runs-on: depot-ubuntu-24.04-4
runs-on: depot-ubuntu-24.04
defaults:
run:
+1 -1
View File
@@ -35,7 +35,7 @@ concurrency:
jobs:
translate:
if: github.repository == 'langgenius/dify'
runs-on: depot-ubuntu-24.04-4
runs-on: depot-ubuntu-24.04
timeout-minutes: 120
steps:
+1 -1
View File
@@ -16,7 +16,7 @@ concurrency:
jobs:
trigger:
if: github.repository == 'langgenius/dify'
runs-on: depot-ubuntu-24.04-4
runs-on: depot-ubuntu-24.04
timeout-minutes: 5
steps:
+1 -1
View File
@@ -16,7 +16,7 @@ jobs:
test:
name: Full VDB Tests
if: github.repository == 'langgenius/dify'
runs-on: depot-ubuntu-24.04-4
runs-on: depot-ubuntu-24.04
strategy:
matrix:
python-version:
+1 -1
View File
@@ -13,7 +13,7 @@ concurrency:
jobs:
test:
name: VDB Smoke Tests
runs-on: depot-ubuntu-24.04-4
runs-on: depot-ubuntu-24.04
strategy:
matrix:
python-version:
+2 -20
View File
@@ -2,14 +2,14 @@ import logging
import time
import socketio
from flask import g, request
from flask import request
from opentelemetry.trace import get_current_span
from opentelemetry.trace.span import INVALID_SPAN_ID, INVALID_TRACE_ID
from configs import dify_config
from contexts.wrapper import RecyclableContextVar
from controllers.console.error import UnauthorizedAndForceLogout
from core.logging.context import clear_request_context, init_request_context
from core.logging.context import init_request_context
from dify_app import DifyApp
from extensions.ext_socketio import sio
from services.enterprise.enterprise_service import EnterpriseService
@@ -60,7 +60,6 @@ def create_flask_app_with_configs() -> DifyApp:
def before_request():
# Initialize logging context for this request
init_request_context()
g._logging_context_cleanup_scheduled = False
RecyclableContextVar.increment_thread_recycles()
# Enterprise license validation for API endpoints (both console and webapp)
@@ -118,26 +117,9 @@ def create_flask_app_with_configs() -> DifyApp:
logger.warning("Failed to add trace headers to response", exc_info=True)
return response
@dify_app.after_request
def schedule_logging_context_cleanup(response):
"""Keep logging context through streaming, then clear it when WSGI closes the response."""
if response.direct_passthrough:
return response
g._logging_context_cleanup_scheduled = True
response.call_on_close(clear_request_context)
return response
@dify_app.teardown_request
def clear_unscheduled_logging_context(_error: BaseException | None) -> None:
"""Clear when no response-close callback can own cleanup."""
if not g.get("_logging_context_cleanup_scheduled", False):
clear_request_context()
# Capture the decorator return values so static checkers do not treat the hooks as unused.
_ = before_request
_ = add_trace_headers
_ = schedule_logging_context_cleanup
_ = clear_unscheduled_logging_context
return dify_app
+55 -44
View File
@@ -78,11 +78,22 @@ def _filter_snapshot_to_specs(
return CompositorSessionSnapshot(schema_version=snapshot.schema_version, layers=filtered_layers)
def _shell_layer_deps(*, include_drive: bool) -> dict[str, str]:
deps = {"execution_context": DIFY_EXECUTION_CONTEXT_LAYER_ID}
if include_drive:
deps["drive"] = DIFY_DRIVE_LAYER_ID
return deps
def _shell_layer_deps() -> dict[str, str]:
return {"execution_context": DIFY_EXECUTION_CONTEXT_LAYER_ID}
def _drive_layer_deps() -> dict[str, str]:
return {"shell": DIFY_SHELL_LAYER_ID}
def _shell_config_with_drive_ref(
shell_config: DifyShellLayerConfig | None,
drive_config: DifyDriveLayerConfig | None,
) -> DifyShellLayerConfig:
config = shell_config or DifyShellLayerConfig()
if drive_config is None:
return config
return config.model_copy(update={"agent_stub_drive_ref": drive_config.drive_ref})
class AgentBackendModelConfig(BaseModel):
@@ -263,14 +274,29 @@ class AgentBackendRunRequestBuilder:
]
)
include_shell = run_input.include_shell or run_input.drive_config is not None
if include_shell:
# Sandboxed bash workspace (dify.shell). It enters before drive so
# drive can materialize mentioned targets with `dify-agent drive pull`
# in the same shell-visible filesystem used by model commands.
layers.append(
RunLayerSpec(
name=DIFY_SHELL_LAYER_ID,
type=DIFY_SHELL_LAYER_TYPE_ID,
deps=_shell_layer_deps(),
metadata=run_input.metadata,
config=_shell_config_with_drive_ref(run_input.shell_config, run_input.drive_config),
)
)
if run_input.drive_config is not None:
# Drive Skills & Files declaration (dify.drive): a config-only index;
# the agent pulls listed entries through the back proxy by drive_ref.
# Drive Skills & Files declaration (dify.drive): the catalog plus
# prompt-mentioned entries eagerly pulled through the shell layer.
layers.append(
RunLayerSpec(
name=DIFY_DRIVE_LAYER_ID,
type=DIFY_DRIVE_LAYER_TYPE_ID,
deps={"execution_context": DIFY_EXECUTION_CONTEXT_LAYER_ID},
deps=_drive_layer_deps(),
metadata=run_input.metadata,
config=run_input.drive_config,
)
@@ -312,7 +338,7 @@ class AgentBackendRunRequestBuilder:
)
)
if run_input.knowledge is not None and run_input.knowledge.dataset_ids:
if run_input.knowledge is not None and run_input.knowledge.sets:
layers.append(
RunLayerSpec(
name=DIFY_KNOWLEDGE_BASE_LAYER_ID,
@@ -336,21 +362,6 @@ class AgentBackendRunRequestBuilder:
)
)
if run_input.include_shell:
# Sandboxed bash workspace (dify.shell). Depends on execution_context
# so the agent server can mint per-command Agent Stub env, and on
# drive when present so that env points at /mnt/drive/<drive_ref>.
# shellctl connection itself is server-injected.
layers.append(
RunLayerSpec(
name=DIFY_SHELL_LAYER_ID,
type=DIFY_SHELL_LAYER_TYPE_ID,
deps=_shell_layer_deps(include_drive=run_input.drive_config is not None),
metadata=run_input.metadata,
config=run_input.shell_config or DifyShellLayerConfig(),
)
)
if run_input.output is not None:
layers.append(
RunLayerSpec(
@@ -445,7 +456,7 @@ class AgentBackendRunRequestBuilder:
name=WORKFLOW_NODE_JOB_PROMPT_LAYER_ID,
type=PLAIN_PROMPT_LAYER_TYPE_ID,
metadata={**run_input.metadata, "origin": "workflow_node_job"},
config=PromptLayerConfig(prefix=run_input.workflow_node_job_prompt),
config=PromptLayerConfig(user=run_input.workflow_node_job_prompt),
),
RunLayerSpec(
name=WORKFLOW_USER_PROMPT_LAYER_ID,
@@ -462,14 +473,29 @@ class AgentBackendRunRequestBuilder:
]
)
include_shell = run_input.include_shell or run_input.drive_config is not None
if include_shell:
# Sandboxed bash workspace (dify.shell). It enters before drive so
# drive can materialize mentioned targets with `dify-agent drive pull`
# in the same shell-visible filesystem used by model commands.
layers.append(
RunLayerSpec(
name=DIFY_SHELL_LAYER_ID,
type=DIFY_SHELL_LAYER_TYPE_ID,
deps=_shell_layer_deps(),
metadata=run_input.metadata,
config=_shell_config_with_drive_ref(run_input.shell_config, run_input.drive_config),
)
)
if run_input.drive_config is not None:
# Drive Skills & Files declaration (dify.drive): a config-only index;
# the agent pulls listed entries through the back proxy by drive_ref.
# Drive Skills & Files declaration (dify.drive): the catalog plus
# prompt-mentioned entries eagerly pulled through the shell layer.
layers.append(
RunLayerSpec(
name=DIFY_DRIVE_LAYER_ID,
type=DIFY_DRIVE_LAYER_TYPE_ID,
deps={"execution_context": DIFY_EXECUTION_CONTEXT_LAYER_ID},
deps=_drive_layer_deps(),
metadata=run_input.metadata,
config=run_input.drive_config,
)
@@ -513,7 +539,7 @@ class AgentBackendRunRequestBuilder:
)
)
if run_input.knowledge is not None and run_input.knowledge.dataset_ids:
if run_input.knowledge is not None and run_input.knowledge.sets:
layers.append(
RunLayerSpec(
name=DIFY_KNOWLEDGE_BASE_LAYER_ID,
@@ -537,21 +563,6 @@ class AgentBackendRunRequestBuilder:
)
)
if run_input.include_shell:
# Sandboxed bash workspace (dify.shell). Depends on execution_context
# so the agent server can mint per-command Agent Stub env, and on
# drive when present so that env points at /mnt/drive/<drive_ref>.
# shellctl connection itself is server-injected.
layers.append(
RunLayerSpec(
name=DIFY_SHELL_LAYER_ID,
type=DIFY_SHELL_LAYER_TYPE_ID,
deps=_shell_layer_deps(include_drive=run_input.drive_config is not None),
metadata=run_input.metadata,
config=run_input.shell_config or DifyShellLayerConfig(),
)
)
if run_input.output is not None:
layers.append(
RunLayerSpec(
+1 -2
View File
@@ -22,7 +22,7 @@ from .plugin import (
setup_system_trigger_oauth_client,
transform_datasource_credentials,
)
from .rbac import migrate_dataset_permissions_to_rbac, migrate_member_roles_to_rbac
from .rbac import migrate_member_roles_to_rbac
from .retention import (
archive_workflow_runs,
archive_workflow_runs_plan,
@@ -76,7 +76,6 @@ __all__ = [
"legacy_model_types",
"migrate_annotation_vector_database",
"migrate_data_for_plugin",
"migrate_dataset_permissions_to_rbac",
"migrate_knowledge_vector_database",
"migrate_member_roles_to_rbac",
"migrate_oss",
-2
View File
@@ -7,7 +7,6 @@ from typing import cast
import click
from commands.rbac import migrate_dataset_permissions_to_rbac
from extensions.ext_database import db
from graphon.model_runtime.entities.model_entities import ModelType
from services.legacy_model_type_migration import (
@@ -178,4 +177,3 @@ def legacy_model_types(
data_migrate.add_command(legacy_model_types)
data_migrate.add_command(migrate_dataset_permissions_to_rbac)
+65 -437
View File
@@ -1,55 +1,11 @@
from __future__ import annotations
import json
from collections.abc import Iterator
from concurrent.futures import ThreadPoolExecutor, as_completed
import click
from sqlalchemy import select
from configs import dify_config
from core.db.session_factory import session_factory
from core.rbac import RBACResourceWhitelistScope
from models import Dataset, DatasetPermission, DatasetPermissionEnum, TenantAccountJoin, TenantAccountRole
from services.enterprise.rbac_service import ListOption, RBACService, ReplaceMemberBindings, ReplaceUserAccessPolicies
_RBAC_DEFAULT_ACCESS_POLICY_ID = "default"
_LEGACY_ROLE_TO_BUILTIN_TAG = {
TenantAccountRole.OWNER.value: "owner",
TenantAccountRole.ADMIN.value: "admin",
TenantAccountRole.EDITOR.value: "editor",
TenantAccountRole.NORMAL.value: "normal",
TenantAccountRole.DATASET_OPERATOR.value: "dataset_operator",
}
def _resolve_builtin_role_ids(tenant_id: str, operator_account_id: str) -> dict[str, str]:
"""Resolve every legacy workspace role to the current tenant's builtin RBAC role id.
The migration replays the old `TenantAccountJoin.role` values onto the
RBAC member-role binding API. Builtin RBAC roles are tenant-scoped and
identified by runtime ids, so the command must look them up per tenant.
"""
roles = RBACService.Roles.list(
tenant_id=tenant_id,
account_id=operator_account_id,
options=ListOption(page_number=1, results_per_page=100),
).data
role_id_by_tag = {
role.role_tag: role.id
for role in roles
if role.is_builtin and role.category == "global_system_default" and role.role_tag
}
resolved: dict[str, str] = {}
for legacy_role, expected_builtin_tag in _LEGACY_ROLE_TO_BUILTIN_TAG.items():
role_id = role_id_by_tag.get(expected_builtin_tag)
if expected_builtin_tag == "dataset_operator" and not dify_config.DATASET_OPERATOR_ENABLED:
continue
if not role_id:
raise ValueError(f"Builtin RBAC role not found for tenant={tenant_id}, legacy_role={legacy_role}")
resolved[legacy_role] = role_id
return resolved
from models import TenantAccountJoin, TenantAccountRole
from services.enterprise.rbac_service import ListOption, RBACService
def _resolve_builtin_role_id(tenant_id: str, operator_account_id: str, legacy_role: str) -> str:
@@ -59,86 +15,26 @@ def _resolve_builtin_role_id(tenant_id: str, operator_account_id: str, legacy_ro
RBAC member-role binding API. Builtin RBAC roles are tenant-scoped and
identified by runtime ids, so the command must look them up per tenant.
"""
if legacy_role not in _LEGACY_ROLE_TO_BUILTIN_TAG:
expected_builtin_tag = {
TenantAccountRole.OWNER.value: "owner",
TenantAccountRole.ADMIN.value: "admin",
TenantAccountRole.EDITOR.value: "editor",
TenantAccountRole.NORMAL.value: "normal",
TenantAccountRole.DATASET_OPERATOR.value: "dataset_operator",
}.get(legacy_role)
if not expected_builtin_tag:
raise ValueError(f"Unsupported legacy workspace role: {legacy_role}")
return _resolve_builtin_role_ids(tenant_id, operator_account_id)[legacy_role]
def _iter_tenant_member_batches(
tenant_id: str | None,
*,
db_batch_size: int,
api_batch_size: int,
) -> Iterator[tuple[str, str, list[tuple[str, str]]]]:
"""Yield legacy member roles in tenant-scoped API-sized batches.
Rows are projected to primitive values and streamed from the database, so
the command never materializes every TenantAccountJoin ORM object. The
iterator only keeps one tenant's API-sized batches in memory while it
finds that tenant's owner account.
"""
with session_factory.create_session() as session:
stmt = (
select(TenantAccountJoin.tenant_id, TenantAccountJoin.account_id, TenantAccountJoin.role)
.order_by(TenantAccountJoin.tenant_id.asc(), TenantAccountJoin.id.asc())
.execution_options(yield_per=db_batch_size)
)
if tenant_id:
stmt = stmt.where(TenantAccountJoin.tenant_id == tenant_id)
current_tenant_id: str | None = None
owner_account_id: str | None = None
batches: list[list[tuple[str, str]]] = []
batch: list[tuple[str, str]] = []
def flush_current_tenant() -> Iterator[tuple[str, str, list[tuple[str, str]]]]:
if current_tenant_id is None:
return
if batch:
batches.append(batch.copy())
if not owner_account_id:
raise ValueError(f"Workspace owner not found for tenant={current_tenant_id}")
for item in batches:
yield current_tenant_id, owner_account_id, item
for row in session.execute(stmt):
workspace_id = str(row.tenant_id)
if current_tenant_id is not None and workspace_id != current_tenant_id:
yield from flush_current_tenant()
owner_account_id = None
batches = []
batch = []
current_tenant_id = workspace_id
account_id = str(row.account_id)
role = str(row.role)
if role == TenantAccountRole.OWNER.value:
owner_account_id = account_id
batch.append((account_id, role))
if len(batch) >= api_batch_size:
batches.append(batch)
batch = []
yield from flush_current_tenant()
def _member_already_has_role(current_roles_by_account_id: dict[str, set[str]], account_id: str, role_id: str) -> bool:
return current_roles_by_account_id.get(account_id) == {role_id}
def _replace_member_role(
tenant_id: str,
operator_account_id: str,
member_account_id: str,
role_id: str,
) -> str:
RBACService.MemberRoles.replace(
roles = RBACService.Roles.list(
tenant_id=tenant_id,
account_id=operator_account_id,
member_account_id=member_account_id,
role_ids=[role_id],
)
return member_account_id
options=ListOption(page_number=1, results_per_page=100),
).data
for role in roles:
if role.is_builtin and role.category == "global_system_default" and role.role_tag == expected_builtin_tag:
return str(role.id)
raise ValueError(f"Builtin RBAC role not found for tenant={tenant_id}, legacy_role={legacy_role}")
@click.command(
@@ -146,16 +42,7 @@ def _replace_member_role(
)
@click.option("--tenant-id", help="Only migrate a single workspace.")
@click.option("--dry-run", is_flag=True, default=False, help="Preview the migration without writing RBAC bindings.")
@click.option("--db-batch-size", default=5000, show_default=True, help="Rows fetched per database batch.")
@click.option("--api-batch-size", default=200, show_default=True, help="Members checked per RBAC batch_get call.")
@click.option("--workers", default=1, show_default=True, help="Concurrent member role replace calls per tenant batch.")
def migrate_member_roles_to_rbac(
tenant_id: str | None,
dry_run: bool,
db_batch_size: int,
api_batch_size: int,
workers: int,
) -> None:
def migrate_member_roles_to_rbac(tenant_id: str | None, dry_run: bool) -> None:
"""Backfill RBAC member-role bindings from legacy `TenantAccountJoin.role` data.
This is an offline migration command for workspaces that already have
@@ -163,322 +50,63 @@ def migrate_member_roles_to_rbac(
member-role binding store.
"""
click.echo(click.style("Starting RBAC member-role migration.", fg="green"))
if workers < 1:
raise click.BadParameter("workers must be >= 1", param_hint="--workers")
tenant_count = 0
scanned_count = 0
skipped_count = 0
migrated_count = 0
current_tenant_id: str | None = None
role_ids_by_legacy_role: dict[str, str] = {}
with session_factory.create_session() as session:
stmt = select(TenantAccountJoin).order_by(TenantAccountJoin.tenant_id.asc(), TenantAccountJoin.id.asc())
if tenant_id:
stmt = stmt.where(TenantAccountJoin.tenant_id == tenant_id)
for workspace_id, owner_account_id, batch in _iter_tenant_member_batches(
tenant_id,
db_batch_size=db_batch_size,
api_batch_size=api_batch_size,
):
scanned_count += len(batch)
if workspace_id != current_tenant_id:
tenant_count += 1
current_tenant_id = workspace_id
role_ids_by_legacy_role = _resolve_builtin_role_ids(workspace_id, owner_account_id)
click.echo(f"tenant={workspace_id}")
joins = list(session.scalars(stmt).all())
current_roles_by_account_id: dict[str, set[str]] = {}
if not dry_run:
current_roles = RBACService.MemberRoles.batch_get(
tenant_id=workspace_id,
account_id=owner_account_id,
member_account_ids=[account_id for account_id, _ in batch],
)
current_roles_by_account_id = {item.account_id: {role.id for role in item.roles} for item in current_roles}
replace_jobs: list[tuple[str, str]] = []
for member_account_id, legacy_role in batch:
resolved_role_id = role_ids_by_legacy_role.get(legacy_role)
if not resolved_role_id:
raise ValueError(f"Unsupported legacy workspace role: {legacy_role}")
if dry_run:
click.echo(
f"tenant={workspace_id} member={member_account_id} "
f"legacy_role={legacy_role} -> rbac_role_id={resolved_role_id}"
)
continue
if _member_already_has_role(current_roles_by_account_id, member_account_id, resolved_role_id):
skipped_count += 1
continue
replace_jobs.append((member_account_id, resolved_role_id))
if replace_jobs:
if workers == 1:
for member_account_id, resolved_role_id in replace_jobs:
_replace_member_role(workspace_id, owner_account_id, member_account_id, resolved_role_id)
migrated_count += 1
else:
with ThreadPoolExecutor(max_workers=workers) as executor:
futures = [
executor.submit(
_replace_member_role,
workspace_id,
owner_account_id,
member_account_id,
resolved_role_id,
)
for member_account_id, resolved_role_id in replace_jobs
]
for future in as_completed(futures):
future.result()
migrated_count += 1
if scanned_count % 10000 == 0:
click.echo(
f"progress scanned={scanned_count} migrated={migrated_count} skipped={skipped_count}",
err=True,
)
if scanned_count == 0:
if not joins:
click.echo(click.style("No workspace members found for migration.", fg="yellow"))
return
if dry_run:
click.echo(
click.style(
f"Dry run completed. Scanned {scanned_count} members across {tenant_count} tenants. "
"No RBAC bindings were written.",
fg="yellow",
owner_account_by_tenant: dict[str, str] = {}
resolved_role_ids: dict[tuple[str, str], str] = {}
migrated_count = 0
for join in joins:
workspace_id = str(join.tenant_id)
member_account_id = str(join.account_id)
legacy_role = str(join.role)
if workspace_id not in owner_account_by_tenant:
owner_join = next(
(
item
for item in joins
if str(item.tenant_id) == workspace_id and str(item.role) == TenantAccountRole.OWNER.value
),
None,
)
)
else:
if not owner_join:
raise ValueError(f"Workspace owner not found for tenant={workspace_id}")
owner_account_by_tenant[workspace_id] = str(owner_join.account_id)
operator_account_id = owner_account_by_tenant[workspace_id]
cache_key = (workspace_id, legacy_role)
if cache_key not in resolved_role_ids:
resolved_role_ids[cache_key] = _resolve_builtin_role_id(workspace_id, operator_account_id, legacy_role)
resolved_role_id = resolved_role_ids[cache_key]
click.echo(
click.style(
f"RBAC member-role migration completed. Scanned {scanned_count} members across {tenant_count} tenants, "
f"migrated {migrated_count}, skipped {skipped_count} already up-to-date.",
fg="green",
)
f"tenant={workspace_id} member={member_account_id} "
f"legacy_role={legacy_role} -> rbac_role_id={resolved_role_id}"
)
if dry_run:
continue
def _dataset_permission_enum(permission: DatasetPermissionEnum | str | None) -> DatasetPermissionEnum:
if permission is None:
return DatasetPermissionEnum.ONLY_ME
try:
return DatasetPermissionEnum(permission)
except ValueError as exc:
raise ValueError(f"Unsupported legacy dataset permission: {permission}") from exc
def _rbac_dataset_scope_for_legacy_permission(permission: DatasetPermissionEnum) -> RBACResourceWhitelistScope:
if permission is DatasetPermissionEnum.ALL_TEAM:
return RBACResourceWhitelistScope.ALL
if permission in {DatasetPermissionEnum.ONLY_ME, DatasetPermissionEnum.PARTIAL_TEAM}:
return RBACResourceWhitelistScope.SPECIFIC
raise ValueError(f"Unsupported legacy dataset permission: {permission}")
def _emit_dataset_permission_migration_event(payload: dict[str, object]) -> None:
click.echo(json.dumps(payload, sort_keys=True))
@click.command(
"rbac-migrate-dataset-permissions",
help=(
"Migrate legacy dataset permission scopes and partial members into RBAC dataset access bindings. "
"Side effect: replacing each dataset whitelist clears existing per-user policy bindings; "
"the command then recreates legacy partial-member default bindings."
),
)
@click.option("--tenant-id", help="Only migrate datasets in a single workspace.")
@click.option("--dataset-id", help="Only migrate a single dataset.")
@click.option("--batch-size", default=500, show_default=True, type=click.IntRange(min=1))
@click.option(
"--dry-run/--apply",
default=True,
show_default=True,
help="Preview the migration without writing RBAC bindings. Use --apply to write changes.",
)
def migrate_dataset_permissions_to_rbac(
tenant_id: str | None,
dataset_id: str | None,
batch_size: int,
dry_run: bool,
) -> None:
"""Backfill RBAC dataset access config from legacy `Dataset.permission`.
Legacy mapping:
- all_team_members -> RBAC dataset whitelist scope "all"
- partial_members -> RBAC dataset whitelist scope "specific" plus each partial member gets the
virtual default policy
- only_me -> RBAC dataset whitelist scope "specific" with no member policy bindings
The command replaces each dataset's RBAC whitelist scope first. RBAC clears
existing per-user policy bindings during that replace, then this command
recreates the legacy partial-member default bindings. Re-running it is
therefore idempotent for a dataset's current legacy configuration.
"""
click.echo(click.style("Starting RBAC dataset permission migration.", fg="green"))
scanned_count = 0
scope_migrated_count = 0
user_policy_migrated_count = 0
partial_dataset_count = 0
last_dataset_id: str | None = None
while True:
with session_factory.create_session() as session:
stmt = (
select(Dataset.id, Dataset.tenant_id, Dataset.permission, Dataset.created_by)
.order_by(Dataset.id.asc())
.limit(batch_size)
)
if tenant_id:
stmt = stmt.where(Dataset.tenant_id == tenant_id)
if dataset_id:
stmt = stmt.where(Dataset.id == dataset_id)
if last_dataset_id:
stmt = stmt.where(Dataset.id > last_dataset_id)
dataset_rows = list(session.execute(stmt).all())
if not dataset_rows:
break
dataset_ids = [str(row.id) for row in dataset_rows]
partial_members_by_dataset_id: dict[str, list[str]] = {item: [] for item in dataset_ids}
permission_rows = session.execute(
select(DatasetPermission.dataset_id, DatasetPermission.account_id).where(
DatasetPermission.dataset_id.in_(dataset_ids)
)
).all()
for row in permission_rows:
partial_members_by_dataset_id[str(row.dataset_id)].append(str(row.account_id))
for dataset in dataset_rows:
workspace_id = str(dataset.tenant_id)
current_dataset_id = str(dataset.id)
operator_account_id = str(dataset.created_by)
permission_value = _dataset_permission_enum(dataset.permission)
scope = _rbac_dataset_scope_for_legacy_permission(permission_value)
partial_member_ids = sorted(set(partial_members_by_dataset_id[current_dataset_id]))
should_bind_partial_members = permission_value is DatasetPermissionEnum.PARTIAL_TEAM
click.echo(
f"tenant={workspace_id} dataset={current_dataset_id} "
f"operator={operator_account_id} "
f"legacy_permission={permission_value} -> rbac_scope={scope} "
f"partial_members={len(partial_member_ids) if should_bind_partial_members else 0}"
)
scanned_count += 1
replace_whitelist_payload = ReplaceMemberBindings(scope=scope)
if dry_run:
_emit_dataset_permission_migration_event(
{
"event": "dataset_permission_migration_proposed_change",
"action": "replace_whitelist",
"dry_run": True,
"tenant_id": workspace_id,
"dataset_id": current_dataset_id,
"operator_account_id": operator_account_id,
"before": {
"legacy_dataset_permission": permission_value.value,
"legacy_partial_member_ids": partial_member_ids if should_bind_partial_members else [],
},
"after": {
"rbac_whitelist_scope": scope.value,
},
"call": {
"method": "RBACService.DatasetAccess.replace_whitelist",
"kwargs": {
"tenant_id": workspace_id,
"account_id": operator_account_id,
"dataset_id": current_dataset_id,
"payload": replace_whitelist_payload.model_dump(mode="json"),
},
},
}
)
if not dry_run:
RBACService.DatasetAccess.replace_whitelist(
tenant_id=workspace_id,
account_id=operator_account_id,
dataset_id=current_dataset_id,
payload=replace_whitelist_payload,
)
scope_migrated_count += 1
if should_bind_partial_members:
partial_dataset_count += 1
for member_account_id in partial_member_ids:
replace_user_access_policies_payload = ReplaceUserAccessPolicies(
access_policy_ids=[_RBAC_DEFAULT_ACCESS_POLICY_ID],
)
if dry_run:
_emit_dataset_permission_migration_event(
{
"event": "dataset_permission_migration_proposed_change",
"action": "replace_user_access_policies",
"dry_run": True,
"tenant_id": workspace_id,
"dataset_id": current_dataset_id,
"operator_account_id": operator_account_id,
"target_account_id": member_account_id,
"before": {
"legacy_dataset_permission": permission_value.value,
"legacy_partial_member_id": member_account_id,
},
"after": {
"rbac_user_access_policy_ids": [_RBAC_DEFAULT_ACCESS_POLICY_ID],
},
"call": {
"method": "RBACService.DatasetAccess.replace_user_access_policies",
"kwargs": {
"tenant_id": workspace_id,
"account_id": operator_account_id,
"dataset_id": current_dataset_id,
"target_account_id": member_account_id,
"payload": replace_user_access_policies_payload.model_dump(
mode="json", exclude_unset=True
),
},
},
}
)
continue
RBACService.DatasetAccess.replace_user_access_policies(
tenant_id=workspace_id,
account_id=operator_account_id,
dataset_id=current_dataset_id,
target_account_id=member_account_id,
payload=replace_user_access_policies_payload,
)
user_policy_migrated_count += 1
last_dataset_id = dataset_ids[-1]
if dataset_id:
break
if scanned_count == 0:
click.echo(click.style("No datasets found for migration.", fg="yellow"))
return
RBACService.MemberRoles.replace(
tenant_id=workspace_id,
account_id=operator_account_id,
member_account_id=member_account_id,
role_ids=[resolved_role_id],
)
migrated_count += 1
if dry_run:
click.echo(
click.style(
f"Dry run completed. Scanned {scanned_count} datasets; "
f"{partial_dataset_count} partial-member datasets would be migrated.",
fg="yellow",
)
)
click.echo(click.style("Dry run completed. No RBAC bindings were written.", fg="yellow"))
else:
click.echo(
click.style(
"RBAC dataset permission migration completed. "
f"Scanned {scanned_count} datasets, migrated {scope_migrated_count} scopes, "
f"wrote {user_policy_migrated_count} user default-policy bindings.",
fg="green",
)
)
click.echo(click.style(f"RBAC member-role migration completed. Migrated {migrated_count} members.", fg="green"))
+29 -2
View File
@@ -35,6 +35,12 @@ class WorkflowRunArchiveTenantPlan(TypedDict):
unpaid_tenant_ids: list[str]
def _normalize_utc_datetime(value: datetime.datetime) -> datetime.datetime:
if value.tzinfo is None:
return value.replace(tzinfo=datetime.UTC)
return value.astimezone(datetime.UTC)
def _parse_tenant_prefixes(prefixes: str | None) -> list[str]:
if not prefixes:
return []
@@ -156,11 +162,16 @@ def _resolve_archive_time_range(
raise click.UsageError("Choose either day offsets or explicit dates, not both.")
if from_days_ago <= to_days_ago:
raise click.UsageError("--from-days-ago must be greater than --to-days-ago.")
now = datetime.datetime.now()
now = datetime.datetime.now(datetime.UTC)
start_from = now - datetime.timedelta(days=from_days_ago)
end_before = now - datetime.timedelta(days=to_days_ago)
before_days = 0
if start_from is not None:
start_from = _normalize_utc_datetime(start_from)
if end_before is not None:
end_before = _normalize_utc_datetime(end_before)
if start_from and end_before and start_from >= end_before:
raise click.UsageError("--start-from must be earlier than --end-before.")
@@ -402,6 +413,13 @@ def archive_workflow_runs_plan(
fg="white",
)
)
click.echo(
click.style(
"fixed_archive_window="
f"{start_from.isoformat() if start_from else 'unbounded'},{plan_end_before.isoformat()}",
fg="white",
)
)
click.echo("tenant_prefix,total_tenants,workflow_runs,workflow_node_executions,paid_tenants,unpaid_tenants")
for row in rows:
click.echo(
@@ -451,7 +469,7 @@ def archive_workflow_runs_plan(
default=None,
help="Archive runs created before this timestamp (UTC if no timezone).",
)
@click.option("--batch-size", default=100, show_default=True, help="Maximum workflow runs per archive bundle.")
@click.option("--batch-size", default=10000, show_default=True, help="Maximum workflow runs per archive bundle.")
@click.option(
"--workers",
default=1,
@@ -521,6 +539,7 @@ def archive_workflow_runs(
)
)
uses_relative_window = start_from is None and end_before is None
try:
before_days, start_from, end_before = _resolve_archive_time_range(
before_days=before_days,
@@ -546,6 +565,14 @@ def archive_workflow_runs(
if delete_after_archive:
click.echo(click.style("delete-after-archive is not supported by bundle archive.", fg="red"))
return
if uses_relative_window:
click.echo(
click.style(
"Relative archive windows are evaluated at command start. For multi-day prefix/shard rollout, "
"reuse absolute --start-from/--end-before values from archive-workflow-runs-plan.",
fg="yellow",
)
)
try:
tenant_plan = _resolve_archive_tenant_ids_from_plan(
-6
View File
@@ -34,12 +34,6 @@ class EnterpriseFeatureConfig(BaseSettings):
default=False,
)
ENTERPRISE_RBAC_REQUEST_TIMEOUT: int = Field(
ge=1,
description="Maximum timeout in seconds for inner RBAC requests.",
default=30,
)
class EnterpriseTelemetryConfig(BaseSettings):
"""
+3 -3
View File
@@ -36,8 +36,8 @@ class AgentBackendConfig(BaseSettings):
description=(
"Inject the dify.drive layer (Skills & Files drive manifest declaration) "
"into Agent runs. The declaration is an index only — the agent backend "
"pulls the actual SKILL.md / files through the back proxy. Keep it off "
"until the agent backend registers the dify.drive layer type."
"pulls the actual SKILL.md / files through the back proxy. Set this to "
"false only when temporarily rolling back the drive integration."
),
default=False,
default=True,
)
+1
View File
@@ -183,6 +183,7 @@ class Site(BaseModel):
description: str | None = None
copyright: str | None = None
privacy_policy: str | None = None
input_placeholder: str | None = None
custom_disclaimer: str | None = None
default_language: str
show_workflow_steps: bool
+11 -1
View File
@@ -6,5 +6,15 @@ from services.agent.roster_service import AgentRosterService
def resolve_agent_app_model(*, tenant_id: str, agent_id: UUID) -> App:
"""Resolve the hidden Agent App backing an Agent Console resource."""
"""Resolve a roster Agent's public Agent App."""
return AgentRosterService(db.session).get_agent_app_model(tenant_id=tenant_id, agent_id=str(agent_id))
def resolve_agent_runtime_app_model(*, tenant_id: str, agent_id: UUID) -> App:
"""Resolve the App that backs an Agent runtime surface.
This accepts both roster Agent Apps and workflow-only inline Agents with a
hidden backing App.
"""
return AgentRosterService(db.session).get_agent_runtime_app_model(tenant_id=tenant_id, agent_id=str(agent_id))
+20 -16
View File
@@ -1,10 +1,10 @@
from uuid import UUID
from flask import request
from flask_restx import Resource
from controllers.common.schema import register_response_schema_models, register_schema_models
from controllers.common.schema import query_params_from_model, register_response_schema_models, register_schema_models
from controllers.console import console_ns
from controllers.console.agent.app_helpers import resolve_agent_app_model
from controllers.console.app.wraps import get_app_model
from controllers.console.wraps import (
RBACPermission,
@@ -28,9 +28,15 @@ from libs.login import login_required
from models.model import App, AppMode
from services.agent.composer_service import AgentComposerService
from services.agent.composer_validator import ComposerConfigValidator
from services.entities.agent_entities import ComposerSavePayload, WorkflowComposerCopyFromRosterPayload
from services.entities.agent_entities import (
ComposerSavePayload,
WorkflowAgentComposerQuery,
WorkflowComposerCopyFromRosterPayload,
)
register_schema_models(console_ns, ComposerSavePayload, WorkflowComposerCopyFromRosterPayload)
register_schema_models(
console_ns, ComposerSavePayload, WorkflowAgentComposerQuery, WorkflowComposerCopyFromRosterPayload
)
register_response_schema_models(
console_ns,
AgentAppComposerResponse,
@@ -41,27 +47,26 @@ register_response_schema_models(
)
def _resolve_agent_app_id(*, tenant_id: str, agent_id: UUID) -> str:
return resolve_agent_app_model(tenant_id=tenant_id, agent_id=agent_id).id
@console_ns.route("/apps/<uuid:app_id>/workflows/draft/nodes/<string:node_id>/agent-composer")
class WorkflowAgentComposerApi(Resource):
@console_ns.response(
200, "Workflow agent composer state", console_ns.models[WorkflowAgentComposerResponse.__name__]
)
@console_ns.doc(params=query_params_from_model(WorkflowAgentComposerQuery))
@setup_required
@login_required
@account_initialization_required
@get_app_model(mode=[AppMode.WORKFLOW, AppMode.ADVANCED_CHAT])
@with_current_tenant_id
def get(self, tenant_id: str, app_model: App, node_id: str):
query = WorkflowAgentComposerQuery.model_validate(request.args.to_dict(flat=True))
return dump_response(
WorkflowAgentComposerResponse,
AgentComposerService.load_workflow_composer(
tenant_id=tenant_id,
app_id=app_model.id,
node_id=node_id,
snapshot_id=query.snapshot_id,
),
)
@@ -137,6 +142,7 @@ class WorkflowAgentComposerValidateApi(Resource):
def post(self, tenant_id: str, app_model: App, node_id: str):
payload = ComposerSavePayload.model_validate(console_ns.payload or {})
ComposerConfigValidator.validate_publish_payload(payload)
AgentComposerService.validate_knowledge_datasets(tenant_id=tenant_id, agent_soul=payload.agent_soul)
findings = AgentComposerService.collect_validation_findings(
tenant_id=tenant_id,
payload=payload,
@@ -228,10 +234,9 @@ class AgentComposerApi(Resource):
@account_initialization_required
@with_current_tenant_id
def get(self, tenant_id: str, agent_id: UUID):
app_id = _resolve_agent_app_id(tenant_id=tenant_id, agent_id=agent_id)
return dump_response(
AgentAppComposerResponse,
AgentComposerService.load_agent_app_composer(tenant_id=tenant_id, app_id=app_id),
AgentComposerService.load_agent_composer(tenant_id=tenant_id, agent_id=str(agent_id)),
)
@console_ns.expect(console_ns.models[ComposerSavePayload.__name__])
@@ -244,13 +249,12 @@ class AgentComposerApi(Resource):
@with_current_user_id
@with_current_tenant_id
def put(self, tenant_id: str, account_id: str, agent_id: UUID):
app_id = _resolve_agent_app_id(tenant_id=tenant_id, agent_id=agent_id)
payload = ComposerSavePayload.model_validate(console_ns.payload or {})
return dump_response(
AgentAppComposerResponse,
AgentComposerService.save_agent_app_composer(
AgentComposerService.save_agent_composer(
tenant_id=tenant_id,
app_id=app_id,
agent_id=str(agent_id),
account_id=account_id,
payload=payload,
),
@@ -268,9 +272,10 @@ class AgentComposerValidateApi(Resource):
@account_initialization_required
@with_current_tenant_id
def post(self, tenant_id: str, agent_id: UUID):
_resolve_agent_app_id(tenant_id=tenant_id, agent_id=agent_id)
AgentComposerService.load_agent_composer(tenant_id=tenant_id, agent_id=str(agent_id))
payload = ComposerSavePayload.model_validate(console_ns.payload or {})
ComposerConfigValidator.validate_publish_payload(payload)
AgentComposerService.validate_knowledge_datasets(tenant_id=tenant_id, agent_soul=payload.agent_soul)
findings = AgentComposerService.collect_validation_findings(
tenant_id=tenant_id,
payload=payload,
@@ -290,12 +295,11 @@ class AgentComposerCandidatesApi(Resource):
@with_current_user_id
@with_current_tenant_id
def get(self, tenant_id: str, current_user_id: str, agent_id: UUID):
app_id = _resolve_agent_app_id(tenant_id=tenant_id, agent_id=agent_id)
return dump_response(
AgentComposerCandidatesResponse,
AgentComposerService.get_agent_app_candidates(
tenant_id=tenant_id,
app_id=app_id,
agent_id=str(agent_id),
user_id=current_user_id,
),
)
+174 -11
View File
@@ -7,7 +7,7 @@ from sqlalchemy import func, select
from controllers.common.schema import query_params_from_model, register_response_schema_models, register_schema_models
from controllers.console import console_ns
from controllers.console.agent.app_helpers import resolve_agent_app_model
from controllers.console.agent.app_helpers import resolve_agent_app_model, resolve_agent_runtime_app_model
from controllers.console.apikey import ApiKeyItem, ApiKeyList, BaseApiKeyListResource, BaseApiKeyResource
from controllers.console.app.app import (
AppDetailWithSite as GenericAppDetailWithSite,
@@ -54,8 +54,10 @@ from libs.datetime_utils import parse_time_range
from libs.helper import dump_response
from libs.login import login_required
from models import Account
from models.agent import Agent, AgentStatus
from models.enums import ApiTokenType
from models.model import ApiToken, App, IconType
from services.agent.composer_service import AgentComposerService
from services.agent.errors import AgentNotFoundError
from services.agent.observability_service import (
AgentLogQueryParams,
@@ -65,7 +67,7 @@ from services.agent.observability_service import (
from services.agent.roster_service import AgentRosterService
from services.app_service import AppListParams, AppService, CreateAppParams
from services.enterprise.enterprise_service import EnterpriseService
from services.entities.agent_entities import RosterListQuery
from services.entities.agent_entities import ComposerSavePayload, RosterListQuery
from services.feature_service import FeatureService
@@ -232,6 +234,8 @@ class AgentStatisticsQuery(BaseModel):
class AgentAppPartial(GenericAppPartial):
app_id: str | None = None
backing_app_id: str | None = None
hidden_app_backed: bool = False
debug_conversation_id: str | None = None
role: str | None = None
active_config_is_published: bool = False
@@ -241,6 +245,8 @@ class AgentAppPartial(GenericAppPartial):
class AgentAppDetailWithSite(GenericAppDetailWithSite):
app_id: str | None = None
backing_app_id: str | None = None
hidden_app_backed: bool = False
debug_conversation_id: str | None = None
role: str | None = None
active_config_is_published: bool = False
@@ -250,6 +256,36 @@ class AgentDebugConversationRefreshResponse(BaseModel):
debug_conversation_id: str
class AgentPublishPayload(BaseModel):
version_note: str | None = Field(default=None, description="Optional note for this published Agent version")
class AgentPublishResponse(BaseModel):
result: str
active_config_snapshot_id: str
active_config_snapshot: dict[str, object] | None = None
draft: dict[str, object] | None = None
class AgentBuildDraftCheckoutPayload(BaseModel):
force: bool = Field(default=False, description="Overwrite the existing current-user build draft")
class AgentBuildDraftResponse(BaseModel):
variant: str
draft: dict[str, object]
agent_soul: dict[str, object]
class AgentBuildDraftApplyResponse(BaseModel):
result: str
draft: dict[str, object]
class AgentSimpleResultResponse(BaseModel):
result: str
class AgentAppPagination(GenericAppPagination):
data: list[AgentAppPartial] = Field( # type: ignore[assignment] # pyrefly: ignore[bad-override-mutable-attribute]
validation_alias=AliasChoices("items", "data")
@@ -261,6 +297,9 @@ register_schema_models(
AgentAppCreatePayload,
AgentAppUpdatePayload,
AgentAppCopyPayload,
AgentPublishPayload,
AgentBuildDraftCheckoutPayload,
ComposerSavePayload,
AgentApiStatusPayload,
AgentInviteOptionsQuery,
AgentLogsQuery,
@@ -277,6 +316,10 @@ register_response_schema_models(
AgentAppDetailWithSite,
AgentAppPartial,
AgentDebugConversationRefreshResponse,
AgentPublishResponse,
AgentBuildDraftResponse,
AgentBuildDraftApplyResponse,
AgentSimpleResultResponse,
AgentConfigSnapshotDetailResponse,
AgentConfigSnapshotListResponse,
AgentConfigSnapshotRestoreResponse,
@@ -294,7 +337,7 @@ def _agent_roster_service() -> AgentRosterService:
return AgentRosterService(db.session)
def _serialize_agent_app_detail(app_model, *, current_user: Account) -> dict:
def _serialize_agent_app_detail(app_model, *, current_user: Account, agent_id: str | None = None) -> dict:
"""Serialize an Agent App detail using roster-only DTOs.
`/agent` responses are roster-shaped rather than raw app-shaped: `id`
@@ -311,11 +354,23 @@ def _serialize_agent_app_detail(app_model, *, current_user: Account) -> dict:
roster_service = _agent_roster_service()
payload = AgentAppDetailWithSite.model_validate(app_model, from_attributes=True).model_dump(mode="json")
agent = roster_service.get_app_backing_agent(tenant_id=app_model.tenant_id, app_id=str(app_model.id))
agent = (
db.session.scalar(
select(Agent).where(
Agent.tenant_id == app_model.tenant_id,
Agent.id == agent_id,
Agent.status == AgentStatus.ACTIVE,
)
)
if agent_id
else roster_service.get_app_backing_agent(tenant_id=app_model.tenant_id, app_id=str(app_model.id))
)
if not agent:
raise AgentNotFoundError()
payload.pop("bound_agent_id", None)
payload["app_id"] = str(app_model.id)
payload["app_id"] = agent.app_id
payload["backing_app_id"] = roster_service.runtime_backing_app_id(agent)
payload["hidden_app_backed"] = bool(agent.backing_app_id and agent.backing_app_id != agent.app_id)
payload["id"] = agent.id
payload["debug_conversation_id"] = roster_service.get_or_create_agent_app_debug_conversation_id(
tenant_id=app_model.tenant_id,
@@ -365,6 +420,8 @@ def _serialize_agent_app_pagination(app_pagination, *, tenant_id: str, current_u
agent = agents_by_app_id.get(app_id)
if agent:
item["app_id"] = app_id
item["backing_app_id"] = agent.backing_app_id or app_id
item["hidden_app_backed"] = False
item["id"] = agent.id
item["debug_conversation_id"] = debug_conversation_ids_by_agent_id.get(agent.id)
item["role"] = agent.role or ""
@@ -516,8 +573,8 @@ class AgentAppApi(Resource):
@with_current_user
@with_current_tenant_id
def get(self, tenant_id: str, current_user: Account, agent_id: UUID):
app_model = _resolve_agent_app_model(tenant_id=tenant_id, agent_id=agent_id)
return _serialize_agent_app_detail(app_model, current_user=current_user)
app_model = resolve_agent_runtime_app_model(tenant_id=tenant_id, agent_id=agent_id)
return _serialize_agent_app_detail(app_model, current_user=current_user, agent_id=str(agent_id))
@console_ns.expect(console_ns.models[AgentAppUpdatePayload.__name__])
@console_ns.response(200, "Agent app updated successfully", console_ns.models[AgentAppDetailWithSite.__name__])
@@ -583,6 +640,112 @@ class AgentDebugConversationRefreshApi(Resource):
)
@console_ns.route("/agent/<uuid:agent_id>/publish")
class AgentPublishApi(Resource):
@console_ns.expect(console_ns.models[AgentPublishPayload.__name__])
@console_ns.response(200, "Agent draft published", console_ns.models[AgentPublishResponse.__name__])
@console_ns.response(403, "Insufficient permissions")
@setup_required
@login_required
@account_initialization_required
@edit_permission_required
@with_current_user
@with_current_tenant_id
def post(self, tenant_id: str, current_user: Account, agent_id: UUID):
args = AgentPublishPayload.model_validate(console_ns.payload or {})
return AgentComposerService.publish_agent_app_draft(
tenant_id=tenant_id,
agent_id=str(agent_id),
account_id=current_user.id,
version_note=args.version_note,
)
@console_ns.route("/agent/<uuid:agent_id>/build-draft/checkout")
class AgentBuildDraftCheckoutApi(Resource):
@console_ns.expect(console_ns.models[AgentBuildDraftCheckoutPayload.__name__])
@console_ns.response(200, "Agent build draft checked out", console_ns.models[AgentBuildDraftResponse.__name__])
@setup_required
@login_required
@account_initialization_required
@edit_permission_required
@with_current_user
@with_current_tenant_id
def post(self, tenant_id: str, current_user: Account, agent_id: UUID):
args = AgentBuildDraftCheckoutPayload.model_validate(console_ns.payload or {})
return AgentComposerService.checkout_agent_app_build_draft(
tenant_id=tenant_id,
agent_id=str(agent_id),
account_id=current_user.id,
force=args.force,
)
@console_ns.route("/agent/<uuid:agent_id>/build-draft")
class AgentBuildDraftApi(Resource):
@console_ns.response(200, "Agent build draft", console_ns.models[AgentBuildDraftResponse.__name__])
@setup_required
@login_required
@account_initialization_required
@edit_permission_required
@with_current_user
@with_current_tenant_id
def get(self, tenant_id: str, current_user: Account, agent_id: UUID):
return AgentComposerService.load_agent_app_build_draft(
tenant_id=tenant_id,
agent_id=str(agent_id),
account_id=current_user.id,
)
@console_ns.expect(console_ns.models[ComposerSavePayload.__name__])
@console_ns.response(200, "Agent build draft saved", console_ns.models[AgentBuildDraftResponse.__name__])
@setup_required
@login_required
@account_initialization_required
@edit_permission_required
@with_current_user
@with_current_tenant_id
def put(self, tenant_id: str, current_user: Account, agent_id: UUID):
payload = ComposerSavePayload.model_validate(console_ns.payload or {})
return AgentComposerService.save_agent_app_build_draft(
tenant_id=tenant_id,
agent_id=str(agent_id),
account_id=current_user.id,
payload=payload,
)
@console_ns.response(200, "Agent build draft discarded", console_ns.models[AgentSimpleResultResponse.__name__])
@setup_required
@login_required
@account_initialization_required
@edit_permission_required
@with_current_user
@with_current_tenant_id
def delete(self, tenant_id: str, current_user: Account, agent_id: UUID):
return AgentComposerService.discard_agent_app_build_draft(
tenant_id=tenant_id,
agent_id=str(agent_id),
account_id=current_user.id,
)
@console_ns.route("/agent/<uuid:agent_id>/build-draft/apply")
class AgentBuildDraftApplyApi(Resource):
@console_ns.response(200, "Agent build draft applied", console_ns.models[AgentBuildDraftApplyResponse.__name__])
@setup_required
@login_required
@account_initialization_required
@edit_permission_required
@with_current_user
@with_current_tenant_id
def post(self, tenant_id: str, current_user: Account, agent_id: UUID):
return AgentComposerService.apply_agent_app_build_draft(
tenant_id=tenant_id,
agent_id=str(agent_id),
account_id=current_user.id,
)
@console_ns.route("/agent/<uuid:agent_id>/copy")
class AgentAppCopyApi(Resource):
@console_ns.expect(console_ns.models[AgentAppCopyPayload.__name__])
@@ -712,7 +875,7 @@ class AgentLogsApi(Resource):
@with_current_user
@with_current_tenant_id
def get(self, tenant_id: str, current_user: Account, agent_id: UUID):
app_model = _resolve_agent_app_model(tenant_id=tenant_id, agent_id=agent_id)
app_model = resolve_agent_runtime_app_model(tenant_id=tenant_id, agent_id=agent_id)
query_data: dict[str, object] = dict(request.args.to_dict(flat=True))
query_data["sources"] = _multi_query_values("sources", "source")
query_data["statuses"] = _multi_query_values("statuses", "status")
@@ -749,7 +912,7 @@ class AgentLogMessagesApi(Resource):
@with_current_user
@with_current_tenant_id
def get(self, tenant_id: str, current_user: Account, agent_id: UUID, conversation_id: UUID):
app_model = _resolve_agent_app_model(tenant_id=tenant_id, agent_id=agent_id)
app_model = resolve_agent_runtime_app_model(tenant_id=tenant_id, agent_id=agent_id)
query_data: dict[str, object] = dict(request.args.to_dict(flat=True))
query_data["sources"] = _multi_query_values("sources", "source")
query_data["statuses"] = _multi_query_values("statuses", "status")
@@ -786,7 +949,7 @@ class AgentLogSourcesApi(Resource):
@with_current_user
@with_current_tenant_id
def get(self, tenant_id: str, current_user: Account, agent_id: UUID):
app_model = _resolve_agent_app_model(tenant_id=tenant_id, agent_id=agent_id)
app_model = resolve_agent_runtime_app_model(tenant_id=tenant_id, agent_id=agent_id)
payload = _agent_observability_service().list_log_sources(app=app_model, agent_id=str(agent_id))
return dump_response(AgentLogSourceListResponse, payload)
@@ -805,7 +968,7 @@ class AgentStatisticsSummaryApi(Resource):
@with_current_user
@with_current_tenant_id
def get(self, tenant_id: str, current_user: Account, agent_id: UUID):
app_model = _resolve_agent_app_model(tenant_id=tenant_id, agent_id=agent_id)
app_model = resolve_agent_runtime_app_model(tenant_id=tenant_id, agent_id=agent_id)
query = AgentStatisticsQuery.model_validate(request.args.to_dict(flat=True))
timezone = current_user.timezone or "UTC"
start, end = _parse_observability_time_range(query.start, query.end, current_user)
+6 -6
View File
@@ -13,7 +13,7 @@ from controllers.common.schema import (
register_schema_models,
)
from controllers.console import console_ns
from controllers.console.agent.app_helpers import resolve_agent_app_model
from controllers.console.agent.app_helpers import resolve_agent_runtime_app_model
from controllers.console.app.wraps import get_app_model
from controllers.console.wraps import (
RBACPermission,
@@ -351,7 +351,7 @@ class AgentSkillUploadByAgentApi(Resource):
@with_current_user
@with_current_tenant_id
def post(self, tenant_id: str, current_user: Account, agent_id: UUID):
app_model = resolve_agent_app_model(tenant_id=tenant_id, agent_id=agent_id)
app_model = resolve_agent_runtime_app_model(tenant_id=tenant_id, agent_id=agent_id)
return _upload_skill_for_app(current_user=current_user, app_model=app_model)
@@ -394,7 +394,7 @@ class AgentDriveFilesByAgentApi(Resource):
@with_current_user
@with_current_tenant_id
def post(self, tenant_id: str, current_user: Account, agent_id: UUID):
app_model = resolve_agent_app_model(tenant_id=tenant_id, agent_id=agent_id)
app_model = resolve_agent_runtime_app_model(tenant_id=tenant_id, agent_id=agent_id)
return _commit_drive_file_for_app(current_user=current_user, app_model=app_model, allow_node_id=False)
@console_ns.doc("delete_agent_drive_file_by_agent")
@@ -407,7 +407,7 @@ class AgentDriveFilesByAgentApi(Resource):
@with_current_user
@with_current_tenant_id
def delete(self, tenant_id: str, current_user: Account, agent_id: UUID):
app_model = resolve_agent_app_model(tenant_id=tenant_id, agent_id=agent_id)
app_model = resolve_agent_runtime_app_model(tenant_id=tenant_id, agent_id=agent_id)
return _delete_drive_file_for_app(current_user=current_user, app_model=app_model, allow_node_id=False)
@@ -454,7 +454,7 @@ class AgentSkillByAgentApi(Resource):
@with_current_user
@with_current_tenant_id
def delete(self, tenant_id: str, current_user: Account, agent_id: UUID, slug: str):
app_model = resolve_agent_app_model(tenant_id=tenant_id, agent_id=agent_id)
app_model = resolve_agent_runtime_app_model(tenant_id=tenant_id, agent_id=agent_id)
return _delete_skill_for_app(current_user=current_user, app_model=app_model, slug=slug, allow_node_id=False)
@@ -494,7 +494,7 @@ class AgentSkillInferToolsByAgentApi(Resource):
@account_initialization_required
@with_current_tenant_id
def post(self, tenant_id: str, agent_id: UUID, slug: str):
app_model = resolve_agent_app_model(tenant_id=tenant_id, agent_id=agent_id)
app_model = resolve_agent_runtime_app_model(tenant_id=tenant_id, agent_id=agent_id)
return _infer_skill_tools_for_app(app_model=app_model, slug=slug)
@@ -17,7 +17,7 @@ from pydantic import BaseModel, Field
from controllers.common.fields import SimpleResultResponse
from controllers.common.schema import register_response_schema_models, register_schema_models
from controllers.console import console_ns
from controllers.console.agent.app_helpers import resolve_agent_app_model
from controllers.console.agent.app_helpers import resolve_agent_runtime_app_model
from controllers.console.wraps import (
RBACPermission,
RBACResourceScope,
@@ -87,7 +87,7 @@ class AgentAppFeatureConfigResource(Resource):
@with_current_user
@with_current_tenant_id
def post(self, tenant_id: str, current_user: Account, agent_id: UUID):
app_model = resolve_agent_app_model(tenant_id=tenant_id, agent_id=agent_id)
app_model = resolve_agent_runtime_app_model(tenant_id=tenant_id, agent_id=agent_id)
args = AgentAppFeaturesPayload.model_validate(console_ns.payload or {})
new_app_model_config = AgentAppFeatureConfigService.update_features(
@@ -22,7 +22,7 @@ from controllers.common.schema import (
register_schema_models,
)
from controllers.console import console_ns
from controllers.console.agent.app_helpers import resolve_agent_app_model
from controllers.console.agent.app_helpers import resolve_agent_runtime_app_model
from controllers.console.app.wraps import get_app_model
from controllers.console.wraps import account_initialization_required, setup_required, with_current_tenant_id
from fields.base import ResponseModel
@@ -144,7 +144,7 @@ class AgentAppSandboxListResource(Resource):
@account_initialization_required
@with_current_tenant_id
def get(self, tenant_id: str, agent_id: UUID):
app_model = resolve_agent_app_model(tenant_id=tenant_id, agent_id=agent_id)
app_model = resolve_agent_runtime_app_model(tenant_id=tenant_id, agent_id=agent_id)
query = query_params_from_request(AgentSandboxListQuery)
try:
result = AgentAppSandboxService().list_files(
@@ -169,7 +169,7 @@ class AgentAppSandboxReadResource(Resource):
@account_initialization_required
@with_current_tenant_id
def get(self, tenant_id: str, agent_id: UUID):
app_model = resolve_agent_app_model(tenant_id=tenant_id, agent_id=agent_id)
app_model = resolve_agent_runtime_app_model(tenant_id=tenant_id, agent_id=agent_id)
query = query_params_from_request(AgentSandboxFileQuery)
try:
result = AgentAppSandboxService().read_file(
@@ -194,7 +194,7 @@ class AgentAppSandboxUploadResource(Resource):
@account_initialization_required
@with_current_tenant_id
def post(self, tenant_id: str, agent_id: UUID):
app_model = resolve_agent_app_model(tenant_id=tenant_id, agent_id=agent_id)
app_model = resolve_agent_runtime_app_model(tenant_id=tenant_id, agent_id=agent_id)
payload = AgentSandboxUploadPayload.model_validate(request.get_json(silent=True) or {})
try:
result = AgentAppSandboxService().upload_file(
@@ -25,7 +25,7 @@ from controllers.common.schema import (
register_response_schema_models,
)
from controllers.console import console_ns
from controllers.console.agent.app_helpers import resolve_agent_app_model
from controllers.console.agent.app_helpers import resolve_agent_runtime_app_model
from controllers.console.app.wraps import get_app_model
from controllers.console.wraps import account_initialization_required, setup_required, with_current_tenant_id
from fields.base import ResponseModel
@@ -182,7 +182,7 @@ class AgentDriveListByAgentApi(Resource):
@with_current_tenant_id
def get(self, tenant_id: str, agent_id: UUID):
query = query_params_from_request(AgentDriveListByAgentQuery)
resolve_agent_app_model(tenant_id=tenant_id, agent_id=agent_id)
resolve_agent_runtime_app_model(tenant_id=tenant_id, agent_id=agent_id)
try:
items = AgentDriveService().manifest(tenant_id=tenant_id, agent_id=str(agent_id), prefix=query.prefix)
except AgentDriveError as exc:
@@ -201,7 +201,7 @@ class AgentDriveSkillListByAgentApi(Resource):
@account_initialization_required
@with_current_tenant_id
def get(self, tenant_id: str, agent_id: UUID):
resolve_agent_app_model(tenant_id=tenant_id, agent_id=agent_id)
resolve_agent_runtime_app_model(tenant_id=tenant_id, agent_id=agent_id)
try:
items = AgentDriveService().list_skills(tenant_id=tenant_id, agent_id=str(agent_id))
except AgentDriveError as exc:
@@ -220,7 +220,7 @@ class AgentDriveSkillInspectByAgentApi(Resource):
@account_initialization_required
@with_current_tenant_id
def get(self, tenant_id: str, agent_id: UUID, skill_path: str):
resolve_agent_app_model(tenant_id=tenant_id, agent_id=agent_id)
resolve_agent_runtime_app_model(tenant_id=tenant_id, agent_id=agent_id)
try:
return _json_response(
AgentDriveService().inspect_skill(
@@ -245,7 +245,7 @@ class AgentDrivePreviewByAgentApi(Resource):
@with_current_tenant_id
def get(self, tenant_id: str, agent_id: UUID):
query = query_params_from_request(AgentDriveFileByAgentQuery)
resolve_agent_app_model(tenant_id=tenant_id, agent_id=agent_id)
resolve_agent_runtime_app_model(tenant_id=tenant_id, agent_id=agent_id)
try:
return AgentDriveService().preview(tenant_id=tenant_id, agent_id=str(agent_id), key=query.key)
except AgentDriveError as exc:
@@ -264,7 +264,7 @@ class AgentDriveDownloadByAgentApi(Resource):
@with_current_tenant_id
def get(self, tenant_id: str, agent_id: UUID):
query = query_params_from_request(AgentDriveFileByAgentQuery)
resolve_agent_app_model(tenant_id=tenant_id, agent_id=agent_id)
resolve_agent_runtime_app_model(tenant_id=tenant_id, agent_id=agent_id)
try:
url = AgentDriveService().download_url(tenant_id=tenant_id, agent_id=str(agent_id), key=query.key)
except AgentDriveError as exc:
+4 -13
View File
@@ -43,7 +43,6 @@ from controllers.console.wraps import (
from core.ops.ops_trace_manager import OpsTraceManager
from core.rag.entities import PreProcessingRule, Rule, Segmentation
from core.rag.retrieval.retrieval_methods import RetrievalMethod
from core.rbac import RBACResourceWhitelistScope
from core.trigger.constants import TRIGGER_NODE_TYPES
from extensions.ext_database import db
from fields.base import ResponseModel
@@ -70,7 +69,6 @@ from services.entities.knowledge_entities.knowledge_entities import (
WeightVectorSetting,
)
from services.feature_service import FeatureService
from tasks.initialize_created_app_rbac_access_task import initialize_created_app_rbac_access_task
ALLOW_CREATE_APP_MODES = ["chat", "agent-chat", "advanced-chat", "workflow", "completion"]
@@ -333,7 +331,7 @@ class ModelConfig(ResponseModel):
return to_timestamp(value)
class Site(ResponseModel):
class AppDetailSiteResponse(ResponseModel):
access_token: str | None = Field(default=None, validation_alias="code")
code: str | None = None
title: str | None = None
@@ -347,6 +345,7 @@ class Site(ResponseModel):
customize_domain: str | None = None
copyright: str | None = None
privacy_policy: str | None = None
input_placeholder: str | None = None
custom_disclaimer: str | None = None
customize_token_strategy: str | None = None
prompt_public: bool | None = None
@@ -463,7 +462,7 @@ class AppDetailWithSite(AppDetail):
api_base_url: str | None = None
max_active_requests: int | None = None
deleted_tools: list[DeletedTool] = Field(default_factory=list)
site: Site | None = None
site: AppDetailSiteResponse | None = None
# For Agent App type: the roster Agent backing this app (None otherwise).
bound_agent_id: str | None = None
# For Agent App responses exposed through /agent.
@@ -548,7 +547,7 @@ register_schema_models(
WorkflowPartial,
ModelConfigPartial,
ModelConfig,
Site,
AppDetailSiteResponse,
DeletedTool,
AppDetail,
AppExportResponse,
@@ -668,14 +667,6 @@ class AppListApi(Resource):
app_service = AppService()
app = app_service.create_app(current_tenant_id, params, current_user)
if dify_config.RBAC_ENABLED:
enterprise_rbac_service.RBACService.AppAccess.replace_whitelist(
tenant_id=str(current_tenant_id),
account_id=current_user.id,
app_id=str(app.id),
payload=enterprise_rbac_service.ReplaceMemberBindings(scope=RBACResourceWhitelistScope.ALL),
)
initialize_created_app_rbac_access_task.delay(current_tenant_id, current_user.id, app_id=app.id)
permission_keys_map = enterprise_rbac_service.RBACService.AppPermissions.batch_get(
str(current_tenant_id),
current_user.id,
+7 -3
View File
@@ -11,7 +11,7 @@ import services
from controllers.common.fields import GeneratedAppResponse, SimpleResultResponse
from controllers.common.schema import register_response_schema_models, register_schema_models
from controllers.console import console_ns
from controllers.console.agent.app_helpers import resolve_agent_app_model
from controllers.console.agent.app_helpers import resolve_agent_runtime_app_model
from controllers.console.app.error import (
AppUnavailableError,
CompletionRequestError,
@@ -93,6 +93,10 @@ class ChatMessagePayload(BaseMessagePayload):
query: str = Field(..., description="User query")
conversation_id: str | None = Field(default=None, description="Conversation ID")
parent_message_id: str | None = Field(default=None, description="Parent message ID")
draft_type: Literal["draft", "debug_build"] = Field(
default="draft",
description="Agent App debug config source. Use debug_build while the Agent is in build mode.",
)
@field_validator("conversation_id", "parent_message_id")
@classmethod
@@ -218,7 +222,7 @@ class AgentChatMessageApi(Resource):
@with_current_user
@with_current_tenant_id
def post(self, current_tenant_id: str, current_user: Account, agent_id: UUID):
app_model = resolve_agent_app_model(tenant_id=current_tenant_id, agent_id=agent_id)
app_model = resolve_agent_runtime_app_model(tenant_id=current_tenant_id, agent_id=agent_id)
return _create_chat_message(
current_tenant_id=current_tenant_id,
current_user=current_user,
@@ -254,7 +258,7 @@ class AgentChatMessageStopApi(Resource):
@with_current_user_id
@with_current_tenant_id
def post(self, current_tenant_id: str, current_user_id: str, agent_id: UUID, task_id: str):
app_model = resolve_agent_app_model(tenant_id=current_tenant_id, agent_id=agent_id)
app_model = resolve_agent_runtime_app_model(tenant_id=current_tenant_id, agent_id=agent_id)
return _stop_chat_message(current_user_id=current_user_id, app_model=app_model, task_id=task_id)
+10 -6
View File
@@ -13,7 +13,7 @@ from controllers.common.controller_schemas import MessageFeedbackPayload as _Mes
from controllers.common.fields import SimpleResultResponse, TextFileResponse
from controllers.common.schema import query_params_from_model, register_response_schema_models, register_schema_models
from controllers.console import console_ns
from controllers.console.agent.app_helpers import resolve_agent_app_model
from controllers.console.agent.app_helpers import resolve_agent_runtime_app_model
from controllers.console.app.error import (
CompletionRequestError,
ProviderModelCurrentlyNotSupportError,
@@ -167,12 +167,16 @@ register_schema_models(
ChatMessagesQuery,
MessageFeedbackPayload,
FeedbackExportQuery,
)
register_response_schema_models(
console_ns,
AnnotationCountResponse,
SuggestedQuestionsResponse,
MessageDetailResponse,
MessageInfiniteScrollPaginationResponse,
SimpleResultResponse,
TextFileResponse,
)
register_response_schema_models(console_ns, SimpleResultResponse, TextFileResponse)
@console_ns.route("/apps/<uuid:app_id>/chat-messages")
@@ -210,7 +214,7 @@ class AgentChatMessageListApi(Resource):
@with_current_user
@with_current_tenant_id
def get(self, current_tenant_id: str, current_user: Account, agent_id: UUID):
app_model = resolve_agent_app_model(tenant_id=current_tenant_id, agent_id=agent_id)
app_model = resolve_agent_runtime_app_model(tenant_id=current_tenant_id, agent_id=agent_id)
return _list_chat_messages(app_model=app_model, current_user=current_user)
@@ -246,7 +250,7 @@ class AgentMessageFeedbackApi(Resource):
@with_current_user
@with_current_tenant_id
def post(self, current_tenant_id: str, current_user: Account, agent_id: UUID):
app_model = resolve_agent_app_model(tenant_id=current_tenant_id, agent_id=agent_id)
app_model = resolve_agent_runtime_app_model(tenant_id=current_tenant_id, agent_id=agent_id)
return _update_message_feedback(current_user=current_user, app_model=app_model)
@@ -311,7 +315,7 @@ class AgentMessageSuggestedQuestionApi(Resource):
@with_current_user
@with_current_tenant_id
def get(self, current_tenant_id: str, current_user: Account, agent_id: UUID, message_id: UUID):
app_model = resolve_agent_app_model(tenant_id=current_tenant_id, agent_id=agent_id)
app_model = resolve_agent_runtime_app_model(tenant_id=current_tenant_id, agent_id=agent_id)
return _get_message_suggested_questions(current_user=current_user, app_model=app_model, message_id=message_id)
@@ -389,7 +393,7 @@ class AgentMessageApi(Resource):
@account_initialization_required
@with_current_tenant_id
def get(self, current_tenant_id: str, agent_id: UUID, message_id: UUID):
app_model = resolve_agent_app_model(tenant_id=current_tenant_id, agent_id=agent_id)
app_model = resolve_agent_runtime_app_model(tenant_id=current_tenant_id, agent_id=agent_id)
return _get_message_detail(app_model=app_model, message_id=message_id)
+3
View File
@@ -40,6 +40,7 @@ class AppSiteUpdatePayload(BaseModel):
customize_domain: str | None = Field(default=None)
copyright: str | None = Field(default=None)
privacy_policy: str | None = Field(default=None)
input_placeholder: str | None = Field(default=None)
custom_disclaimer: str | None = Field(default=None)
customize_token_strategy: Literal["must", "allow", "not_allow"] | None = Field(default=None)
prompt_public: bool | None = Field(default=None)
@@ -66,6 +67,7 @@ class AppSiteResponse(ResponseModel):
customize_domain: str | None = None
copyright: str | None = None
privacy_policy: str | None = None
input_placeholder: str | None = None
custom_disclaimer: str | None = None
customize_token_strategy: str
prompt_public: bool
@@ -110,6 +112,7 @@ class AppSite(Resource):
"customize_domain",
"copyright",
"privacy_policy",
"input_placeholder",
"custom_disclaimer",
"customize_token_strategy",
"prompt_public",
+2 -10
View File
@@ -19,8 +19,7 @@ from controllers.console.app.error import AppNotFoundError
from core.db.session_factory import session_factory
from extensions.ext_database import db
from libs.login import current_account_with_tenant
from models import App, AppMode, TrialApp
from services.recommended_app_service import RecommendedAppService
from models import App, AppMode
def _load_app_model(session: Session, app_id: str) -> App | None:
@@ -42,10 +41,7 @@ def _load_app_model_from_scoped_session(app_id: str) -> App | None:
def _load_app_model_with_trial(app_id: str) -> App | None:
"""Load a normal app through its trial registration without applying current-tenant scope."""
app_model = db.session.scalar(
select(App).join(TrialApp, TrialApp.app_id == App.id).where(App.id == app_id, App.status == "normal").limit(1)
)
app_model = db.session.scalar(select(App).where(App.id == app_id, App.status == "normal").limit(1))
return app_model
@@ -199,8 +195,6 @@ def get_app_model_with_trial[**P, R](
*,
mode: AppMode | list[AppMode] | None = None,
) -> Callable[P, R] | Callable[[Callable[P, R]], Callable[P, R]]:
"""Inject an app registered for trial or available from the recommended catalog."""
def decorator(view_func: Callable[P, R]) -> Callable[P, R]:
@wraps(view_func)
def decorated_view(*args: P.args, **kwargs: P.kwargs) -> R:
@@ -213,8 +207,6 @@ def get_app_model_with_trial[**P, R](
del kwargs["app_id"]
app_model = _load_app_model_with_trial(app_id)
if app_model is None:
app_model = RecommendedAppService.get_app(app_id, session=db.session())
if not app_model:
raise AppNotFoundError()
@@ -23,9 +23,9 @@ from libs.password import valid_password
from models import Account
from services.account_service import AccountService
from services.billing_service import BillingService
from services.errors.account import AccountRegisterError, SeatsLimitExceededError
from services.errors.account import AccountRegisterError
from ..error import AccountInFreezeError, EmailSendIpLimitError, SeatsLimitExceeded
from ..error import AccountInFreezeError, EmailSendIpLimitError
from ..wraps import email_password_login_enabled, email_register_enabled, setup_required
@@ -208,7 +208,5 @@ class EmailRegisterResetApi(Resource):
timezone=timezone,
session=db.session,
)
except SeatsLimitExceededError:
raise SeatsLimitExceeded()
except AccountRegisterError:
raise AccountInFreezeError()
+1 -4
View File
@@ -25,7 +25,6 @@ from controllers.console.error import (
AccountNotFound,
EmailSendIpLimitError,
NotAllowedCreateWorkspace,
SeatsLimitExceeded,
WorkspacesLimitExceeded,
)
from controllers.console.wraps import (
@@ -52,7 +51,7 @@ from models.account import Account
from services.account_service import AccountService, InvitationDetailDict, RegisterService, TenantService
from services.billing_service import BillingService
from services.entities.auth_entities import LoginFailureReason, LoginPayloadBase
from services.errors.account import AccountRegisterError, SeatsLimitExceededError
from services.errors.account import AccountRegisterError
from services.errors.workspace import WorkSpaceNotAllowedCreateError, WorkspacesLimitExceededError
from services.feature_service import FeatureService
@@ -318,8 +317,6 @@ class EmailCodeLoginApi(Resource):
)
except WorkSpaceNotAllowedCreateError:
raise NotAllowedCreateWorkspace()
except SeatsLimitExceededError:
raise SeatsLimitExceeded()
except AccountRegisterError:
_log_console_login_failure(email=user_email, reason=LoginFailureReason.ACCOUNT_IN_FREEZE)
raise AccountInFreezeError()
+4 -40
View File
@@ -25,7 +25,7 @@ from libs.token import (
from models import Account, AccountStatus
from services.account_service import AccountService, RegisterService, TenantService
from services.billing_service import BillingService
from services.errors.account import AccountNotFoundError, AccountRegisterError, SeatsLimitExceededError
from services.errors.account import AccountNotFoundError, AccountRegisterError
from services.errors.workspace import WorkSpaceNotAllowedCreateError, WorkSpaceNotFoundError
from services.feature_service import FeatureService
@@ -38,7 +38,6 @@ class OAuthLoginQuery(BaseModel):
invite_token: str | None = Field(default=None, description="Optional invitation token")
timezone: str | None = Field(default=None, description="Preferred timezone")
language: str | None = Field(default=None, description="Preferred interface language")
redirect_url: str | None = Field(default=None, description="Relative page to resume after login")
class OAuthCallbackQuery(BaseModel):
@@ -88,36 +87,6 @@ def _validated_language(value: str | None) -> str | None:
return None
def _url_origin(url: str) -> tuple[str, str, int] | None:
parsed_url = urllib.parse.urlsplit(url)
if parsed_url.scheme not in {"http", "https"} or parsed_url.hostname is None:
return None
try:
port = parsed_url.port
except ValueError:
return None
if port is None:
port = 443 if parsed_url.scheme == "https" else 80
return parsed_url.scheme, parsed_url.hostname, port
def _get_redirect_target(redirect_url: str | None) -> str:
if not redirect_url:
return dify_config.CONSOLE_WEB_URL
parsed_url = urllib.parse.urlsplit(redirect_url)
normalized_path = redirect_url.lstrip().replace("\\", "/")
if not parsed_url.scheme and not parsed_url.netloc and not normalized_path.startswith("//"):
return redirect_url
redirect_origin = _url_origin(redirect_url)
if redirect_origin is not None and redirect_origin == _url_origin(dify_config.CONSOLE_WEB_URL):
return redirect_url
return dify_config.CONSOLE_WEB_URL
def _preferred_interface_language(language: str | None = None) -> str:
if language:
return language
@@ -140,7 +109,6 @@ class OAuthLogin(Resource):
invite_token = request.args.get("invite_token") or None
timezone = _validated_timezone(request.args.get("timezone") or None)
language = _validated_language(request.args.get("language") or None)
redirect_url = request.args.get("redirect_url") or None
OAUTH_PROVIDERS = get_oauth_providers()
with current_app.app_context():
oauth_provider = OAUTH_PROVIDERS.get(provider)
@@ -151,7 +119,6 @@ class OAuthLogin(Resource):
invite_token=invite_token,
timezone=timezone,
language=language,
redirect_url=redirect_url,
)
return redirect(auth_url)
@@ -177,7 +144,6 @@ class OAuthCallback(Resource):
invite_token = oauth_state.get("invite_token")
timezone = _validated_timezone(oauth_state.get("timezone"))
language = _validated_language(oauth_state.get("language"))
redirect_url = oauth_state.get("redirect_url")
if not code:
return {"error": "Authorization code is required"}, 400
@@ -216,8 +182,6 @@ class OAuthCallback(Resource):
f"{dify_config.CONSOLE_WEB_URL}/signin"
"?message=Workspace not found, please contact system admin to invite you to join in a workspace."
)
except SeatsLimitExceededError:
return redirect(f"{dify_config.CONSOLE_WEB_URL}/signin?message=Licensed seats limit exceeded.")
except AccountRegisterError as e:
return redirect(f"{dify_config.CONSOLE_WEB_URL}/signin?message={e.description}")
@@ -246,9 +210,9 @@ class OAuthCallback(Resource):
ip_address=extract_remote_ip(request),
)
target_url = _get_redirect_target(redirect_url)
query_char = "&" if "?" in target_url else "?"
target_url = f"{target_url}{query_char}oauth_new_user={str(oauth_new_user).lower()}"
base_url = dify_config.CONSOLE_WEB_URL
query_char = "&" if "?" in base_url else "?"
target_url = f"{base_url}{query_char}oauth_new_user={str(oauth_new_user).lower()}"
response = redirect(target_url)
set_access_token_to_cookie(request, response, token_pair.access_token)
+3 -2
View File
@@ -16,6 +16,7 @@ from controllers.console.wraps import (
with_current_user,
)
from enums.cloud_plan import CloudPlan
from extensions.ext_database import db
from libs.login import login_required
from models import Account
from services.billing_service import BillingService
@@ -50,7 +51,7 @@ class Subscription(Resource):
@with_current_tenant_id
def get(self, current_tenant_id: str, current_user: Account):
args = SubscriptionQuery.model_validate(request.args.to_dict(flat=True))
BillingService.is_tenant_owner_or_admin(current_user)
BillingService.is_tenant_owner_or_admin(db.session, current_user)
return BillingService.get_subscription(args.plan, args.interval, current_user.email, current_tenant_id)
@@ -64,7 +65,7 @@ class Invoices(Resource):
@with_current_user
@with_current_tenant_id
def get(self, current_tenant_id: str, current_user: Account):
BillingService.is_tenant_owner_or_admin(current_user)
BillingService.is_tenant_owner_or_admin(db.session, current_user)
return BillingService.get_invoices(current_user.email, current_tenant_id)
+4 -16
View File
@@ -50,8 +50,6 @@ from models.provider_ids import ModelProviderID
from services.api_token_service import ApiTokenCache
from services.dataset_service import DatasetPermissionService, DatasetService, DocumentService
from services.enterprise import rbac_service as enterprise_rbac_service
from services.enterprise.rbac_service import RBACResourceWhitelistScope, ReplaceMemberBindings
from tasks.initialize_created_app_rbac_access_task import initialize_created_app_rbac_access_task
register_response_schema_models(console_ns, ApiBaseUrlResponse, SimpleResultResponse, UsageCheckResponse)
@@ -567,16 +565,6 @@ class DatasetListApi(Resource):
except services.errors.dataset.DatasetNameDuplicateError:
raise DatasetNameDuplicateError()
if dify_config.RBAC_ENABLED:
if permission == DatasetPermissionEnum.ALL_TEAM:
enterprise_rbac_service.RBACService.DatasetAccess.replace_whitelist(
current_tenant_id,
current_user.id,
dataset.id,
ReplaceMemberBindings(scope=RBACResourceWhitelistScope.ALL),
)
initialize_created_app_rbac_access_task.delay(current_tenant_id, current_user.id, dataset_id=dataset.id)
permission_keys_map = enterprise_rbac_service.RBACService.DatasetPermissions.batch_get(
str(current_tenant_id),
current_user.id,
@@ -614,7 +602,7 @@ class DatasetApi(Resource):
if dataset is None:
raise NotFound("Dataset not found.")
try:
DatasetService.check_dataset_permission(dataset, current_user)
DatasetService.check_dataset_permission(dataset, current_user, db.session)
except services.errors.account.NoPermissionError as e:
raise Forbidden(str(e))
permissions = enterprise_rbac_service.RBACService.MyPermissions.get(
@@ -786,7 +774,7 @@ class DatasetQueryApi(Resource):
raise NotFound("Dataset not found.")
try:
DatasetService.check_dataset_permission(dataset, current_user)
DatasetService.check_dataset_permission(dataset, current_user, db.session)
except services.errors.account.NoPermissionError as e:
raise Forbidden(str(e))
@@ -927,7 +915,7 @@ class DatasetRelatedAppListApi(Resource):
raise NotFound("Dataset not found.")
try:
DatasetService.check_dataset_permission(dataset, current_user)
DatasetService.check_dataset_permission(dataset, current_user, db.session)
except services.errors.account.NoPermissionError as e:
raise Forbidden(str(e))
@@ -1206,7 +1194,7 @@ class DatasetPermissionUserListApi(Resource):
if dataset is None:
raise NotFound("Dataset not found.")
try:
DatasetService.check_dataset_permission(dataset, current_user)
DatasetService.check_dataset_permission(dataset, current_user, db.session)
except services.errors.account.NoPermissionError as e:
raise Forbidden(str(e))
@@ -186,7 +186,7 @@ class DocumentResource(Resource):
raise NotFound("Dataset not found.")
try:
DatasetService.check_dataset_permission(dataset, current_user)
DatasetService.check_dataset_permission(dataset, current_user, db.session)
except services.errors.account.NoPermissionError as e:
raise Forbidden(str(e))
@@ -206,7 +206,7 @@ class DocumentResource(Resource):
raise NotFound("Dataset not found.")
try:
DatasetService.check_dataset_permission(dataset, current_user)
DatasetService.check_dataset_permission(dataset, current_user, db.session)
except services.errors.account.NoPermissionError as e:
raise Forbidden(str(e))
@@ -247,7 +247,7 @@ class GetProcessRuleApi(Resource):
raise NotFound("Dataset not found.")
try:
DatasetService.check_dataset_permission(dataset, current_user)
DatasetService.check_dataset_permission(dataset, current_user, db.session)
except services.errors.account.NoPermissionError as e:
raise Forbidden(str(e))
@@ -322,7 +322,7 @@ class DatasetDocumentListApi(Resource):
raise NotFound("Dataset not found.")
try:
DatasetService.check_dataset_permission(dataset, current_user)
DatasetService.check_dataset_permission(dataset, current_user, db.session)
except services.errors.account.NoPermissionError as e:
raise Forbidden(str(e))
@@ -431,7 +431,7 @@ class DatasetDocumentListApi(Resource):
raise Forbidden()
try:
DatasetService.check_dataset_permission(dataset, current_user)
DatasetService.check_dataset_permission(dataset, current_user, db.session)
except services.errors.account.NoPermissionError as e:
raise Forbidden(str(e))
@@ -1173,7 +1173,7 @@ class DocumentStatusApi(DocumentResource):
DatasetService.check_dataset_model_setting(dataset)
# check user's permission
DatasetService.check_dataset_permission(dataset, current_user)
DatasetService.check_dataset_permission(dataset, current_user, db.session)
document_ids = request.args.getlist("document_id")
@@ -1440,7 +1440,7 @@ class DocumentGenerateSummaryApi(Resource):
raise Forbidden()
try:
DatasetService.check_dataset_permission(dataset, current_user)
DatasetService.check_dataset_permission(dataset, current_user, db.session)
except services.errors.account.NoPermissionError as e:
raise Forbidden(str(e))
@@ -1537,7 +1537,7 @@ class DocumentSummaryStatusApi(DocumentResource):
# Check permissions
try:
DatasetService.check_dataset_permission(dataset, current_user)
DatasetService.check_dataset_permission(dataset, current_user, db.session)
except services.errors.account.NoPermissionError as e:
raise Forbidden(str(e))
@@ -181,7 +181,7 @@ class DatasetDocumentSegmentListApi(Resource):
raise NotFound("Dataset not found.")
try:
DatasetService.check_dataset_permission(dataset, current_user)
DatasetService.check_dataset_permission(dataset, current_user, db.session)
except services.errors.account.NoPermissionError as e:
raise Forbidden(str(e))
@@ -302,7 +302,7 @@ class DatasetDocumentSegmentListApi(Resource):
if not current_user.is_dataset_editor:
raise Forbidden()
try:
DatasetService.check_dataset_permission(dataset, current_user)
DatasetService.check_dataset_permission(dataset, current_user, db.session)
except services.errors.account.NoPermissionError as e:
raise Forbidden(str(e))
SegmentService.delete_segments(segment_ids, document, dataset)
@@ -345,7 +345,7 @@ class DatasetDocumentSegmentApi(Resource):
raise Forbidden()
try:
DatasetService.check_dataset_permission(dataset, current_user)
DatasetService.check_dataset_permission(dataset, current_user, db.session)
except services.errors.account.NoPermissionError as e:
raise Forbidden(str(e))
if dataset.indexing_technique == IndexTechniqueType.HIGH_QUALITY:
@@ -421,7 +421,7 @@ class DatasetDocumentSegmentAddApi(Resource):
except ProviderTokenNotInitError as ex:
raise ProviderNotInitializeError(ex.description)
try:
DatasetService.check_dataset_permission(dataset, current_user)
DatasetService.check_dataset_permission(dataset, current_user, db.session)
except services.errors.account.NoPermissionError as e:
raise Forbidden(str(e))
# validate args
@@ -494,7 +494,7 @@ class DatasetDocumentSegmentUpdateApi(Resource):
if not current_user.is_dataset_editor:
raise Forbidden()
try:
DatasetService.check_dataset_permission(dataset, current_user)
DatasetService.check_dataset_permission(dataset, current_user, db.session)
except services.errors.account.NoPermissionError as e:
raise Forbidden(str(e))
# validate args
@@ -550,7 +550,7 @@ class DatasetDocumentSegmentUpdateApi(Resource):
if not current_user.is_dataset_editor:
raise Forbidden()
try:
DatasetService.check_dataset_permission(dataset, current_user)
DatasetService.check_dataset_permission(dataset, current_user, db.session)
except services.errors.account.NoPermissionError as e:
raise Forbidden(str(e))
SegmentService.delete_segment(segment, document, dataset)
@@ -687,7 +687,7 @@ class ChildChunkAddApi(Resource):
except ProviderTokenNotInitError as ex:
raise ProviderNotInitializeError(ex.description)
try:
DatasetService.check_dataset_permission(dataset, current_user)
DatasetService.check_dataset_permission(dataset, current_user, db.session)
except services.errors.account.NoPermissionError as e:
raise Forbidden(str(e))
# validate args
@@ -789,7 +789,7 @@ class ChildChunkAddApi(Resource):
if not current_user.is_dataset_editor:
raise Forbidden()
try:
DatasetService.check_dataset_permission(dataset, current_user)
DatasetService.check_dataset_permission(dataset, current_user, db.session)
except services.errors.account.NoPermissionError as e:
raise Forbidden(str(e))
# validate args
@@ -862,7 +862,7 @@ class ChildChunkUpdateApi(Resource):
if not current_user.is_dataset_editor:
raise Forbidden()
try:
DatasetService.check_dataset_permission(dataset, current_user)
DatasetService.check_dataset_permission(dataset, current_user, db.session)
except services.errors.account.NoPermissionError as e:
raise Forbidden(str(e))
try:
@@ -930,7 +930,7 @@ class ChildChunkUpdateApi(Resource):
if not current_user.is_dataset_editor:
raise Forbidden()
try:
DatasetService.check_dataset_permission(dataset, current_user)
DatasetService.check_dataset_permission(dataset, current_user, db.session)
except services.errors.account.NoPermissionError as e:
raise Forbidden(str(e))
# validate args
+1 -1
View File
@@ -382,7 +382,7 @@ class ExternalKnowledgeHitTestingApi(Resource):
raise NotFound("Dataset not found.")
try:
DatasetService.check_dataset_permission(dataset, current_user)
DatasetService.check_dataset_permission(dataset, current_user, db.session)
except services.errors.account.NoPermissionError as e:
raise Forbidden(str(e))
@@ -90,7 +90,7 @@ class DatasetsHitTestingBase:
raise NotFound("Dataset not found.")
try:
DatasetService.check_dataset_permission(dataset, current_user)
DatasetService.check_dataset_permission(dataset, current_user, db.session)
except services.errors.account.NoPermissionError as e:
raise Forbidden(str(e))
+5 -5
View File
@@ -64,7 +64,7 @@ class DatasetMetadataCreateApi(Resource):
dataset = DatasetService.get_dataset(dataset_id_str)
if dataset is None:
raise NotFound("Dataset not found.")
DatasetService.check_dataset_permission(dataset, current_user)
DatasetService.check_dataset_permission(dataset, current_user, db.session)
metadata = MetadataService.create_metadata(
db.session(), dataset_id_str, metadata_args, current_user, current_tenant_id
@@ -108,7 +108,7 @@ class DatasetMetadataApi(Resource):
dataset = DatasetService.get_dataset(dataset_id_str)
if dataset is None:
raise NotFound("Dataset not found.")
DatasetService.check_dataset_permission(dataset, current_user)
DatasetService.check_dataset_permission(dataset, current_user, db.session)
metadata = MetadataService.update_metadata_name(
db.session(), dataset_id_str, metadata_id_str, name, current_user, current_tenant_id
@@ -128,7 +128,7 @@ class DatasetMetadataApi(Resource):
dataset = DatasetService.get_dataset(dataset_id_str)
if dataset is None:
raise NotFound("Dataset not found.")
DatasetService.check_dataset_permission(dataset, current_user)
DatasetService.check_dataset_permission(dataset, current_user, db.session)
MetadataService.delete_metadata(db.session(), dataset_id_str, metadata_id_str)
# Frontend callers only await success and invalidate metadata caches; no response body is consumed.
@@ -165,7 +165,7 @@ class DatasetMetadataBuiltInFieldActionApi(Resource):
dataset = DatasetService.get_dataset(dataset_id_str)
if dataset is None:
raise NotFound("Dataset not found.")
DatasetService.check_dataset_permission(dataset, current_user)
DatasetService.check_dataset_permission(dataset, current_user, db.session)
match action:
case "enable":
@@ -194,7 +194,7 @@ class DocumentMetadataEditApi(Resource):
dataset = DatasetService.get_dataset(dataset_id_str)
if dataset is None:
raise NotFound("Dataset not found.")
DatasetService.check_dataset_permission(dataset, current_user)
DatasetService.check_dataset_permission(dataset, current_user, db.session)
metadata_args = MetadataOperationData.model_validate(console_ns.payload or {})
-6
View File
@@ -58,12 +58,6 @@ class WorkspacesLimitExceeded(BaseHTTPException):
code = 400
class SeatsLimitExceeded(BaseHTTPException):
error_code = "limit_exceeded"
description = "Unable to create account because the licensed seats limit was exceeded"
code = 400
class AccountBannedError(BaseHTTPException):
error_code = "account_banned"
description = "Account is banned."
@@ -85,6 +85,7 @@ def _published_app_filter():
class InstalledAppInfoResponse(ResponseModel):
id: str
name: str | None = None
description: str | None = None
mode: str | None = None
icon_type: str | None = None
icon: str | None = None
@@ -123,6 +124,7 @@ class InstalledAppResponse(ResponseModel):
return {
"id": _safe_primitive(getattr(value, "id", "")) or "",
"name": _safe_primitive(getattr(value, "name", None)),
"description": _safe_primitive(getattr(value, "description", None)),
"mode": _safe_primitive(getattr(value, "mode", None)),
"icon_type": _safe_primitive(getattr(value, "icon_type", None)),
"icon": _safe_primitive(getattr(value, "icon", None)),
@@ -106,7 +106,7 @@ class RecommendedAppListApi(Resource):
language_prefix = _resolve_language(args.language, current_user)
return RecommendedAppListResponse.model_validate(
RecommendedAppService.get_recommended_apps_and_categories(language_prefix, session=db.session()),
RecommendedAppService.get_recommended_apps_and_categories(db.session, language_prefix),
from_attributes=True,
).model_dump(mode="json")
+3 -48
View File
@@ -44,9 +44,7 @@ from controllers.console.explore.error import (
NotWorkflowAppError,
)
from controllers.console.explore.wraps import TrialAppResource, trial_feature_enable
from controllers.console.files import FILE_UPLOAD_PARAMS, upload_file_from_request
from controllers.console.remote_files import RemoteFileUploadPayload, upload_remote_file_from_request
from controllers.console.wraps import cloud_edition_billing_resource_check, with_current_user
from controllers.console.wraps import with_current_user
from controllers.web.error import InvokeRateLimitError as InvokeRateLimitHttpError
from core.app.app_config.common.parameters_mapping import get_parameters_from_feature_dict
from core.app.apps.base_app_queue_manager import AppQueueManager
@@ -66,7 +64,6 @@ from fields.app_fields import (
tag_fields,
)
from fields.dataset_fields import dataset_fields
from fields.file_fields import FileResponse, FileWithSignedUrl
from fields.member_fields import simple_account_fields
from fields.message_fields import SuggestedQuestionsResponse
from fields.workflow_fields import (
@@ -78,8 +75,8 @@ from fields.workflow_fields import (
from graphon.graph_engine.manager import GraphEngineManager
from graphon.model_runtime.errors.invoke import InvokeError
from libs import helper
from libs.helper import dump_response, uuid_value
from models import Account, App
from libs.helper import uuid_value
from models import Account
from models.account import TenantStatus
from models.model import AppMode, Site
from models.workflow import Workflow
@@ -202,36 +199,6 @@ register_response_schema_models(
)
class TrialAppFileUploadApi(TrialAppResource):
@trial_feature_enable
@cloud_edition_billing_resource_check("documents")
@console_ns.doc(consumes=["multipart/form-data"], params=FILE_UPLOAD_PARAMS)
@console_ns.response(201, "File uploaded successfully", console_ns.models[FileResponse.__name__])
@with_current_user
def post(self, current_user: Account, app_model: App):
"""Upload a file into the tenant that owns the trial app."""
upload_file = upload_file_from_request(
current_user=current_user,
resource_tenant_id=app_model.tenant_id,
)
return dump_response(FileResponse, upload_file), 201
class TrialAppRemoteFileUploadApi(TrialAppResource):
@trial_feature_enable
@cloud_edition_billing_resource_check("documents")
@console_ns.expect(console_ns.models[RemoteFileUploadPayload.__name__])
@console_ns.response(201, "File uploaded successfully", console_ns.models[FileWithSignedUrl.__name__])
@with_current_user
def post(self, current_user: Account, app_model: App):
"""Upload a remote file into the tenant that owns the trial app."""
remote_file = upload_remote_file_from_request(
current_user=current_user,
resource_tenant_id=app_model.tenant_id,
)
return remote_file.model_dump(mode="json"), 201
class TrialAppWorkflowRunApi(TrialAppResource):
@trial_feature_enable
@console_ns.expect(console_ns.models[WorkflowRunRequest.__name__])
@@ -638,18 +605,6 @@ class DatasetListApi(Resource):
console_ns.add_resource(TrialChatApi, "/trial-apps/<uuid:app_id>/chat-messages", endpoint="trial_app_chat_completion")
console_ns.add_resource(
TrialAppFileUploadApi,
"/trial-apps/<uuid:app_id>/files/upload",
endpoint="trial_app_file_upload",
)
console_ns.add_resource(
TrialAppRemoteFileUploadApi,
"/trial-apps/<uuid:app_id>/remote-files/upload",
endpoint="trial_app_remote_file_upload",
)
console_ns.add_resource(
TrialMessageSuggestedQuestionApi,
"/trial-apps/<uuid:app_id>/messages/<uuid:message_id>/suggested-questions",
+33 -56
View File
@@ -28,7 +28,7 @@ from controllers.console.wraps import (
from extensions.ext_database import db
from fields.file_fields import FileResponse, UploadConfig
from libs.login import login_required
from models import Account, UploadFile
from models import Account
from services.file_service import FileService
from . import console_ns
@@ -38,59 +38,6 @@ register_response_schema_models(console_ns, AllowedExtensionsResponse, TextConte
PREVIEW_WORDS_LIMIT = 3000
FILE_UPLOAD_PARAMS = {
"file": {
"description": "File to upload",
"in": "formData",
"type": "file",
"required": True,
},
"source": {
"description": "Optional upload source",
"in": "formData",
"type": "string",
"enum": ["datasets"],
"required": False,
},
}
def upload_file_from_request(*, current_user: Account, resource_tenant_id: str | None = None) -> UploadFile:
"""Validate the multipart request and persist the file under the requested resource tenant."""
source_str = request.form.get("source")
source: Literal["datasets"] | None = "datasets" if source_str == "datasets" else None
if "file" not in request.files:
raise NoFileUploadedError()
if len(request.files) > 1:
raise TooManyFilesError()
file = request.files["file"]
if not file.filename:
raise FilenameNotExistsError
if source == "datasets" and not current_user.is_dataset_editor:
raise Forbidden()
if source not in ("datasets", None):
source = None
try:
return FileService(db.engine).upload_file(
filename=file.filename,
content=file.stream.read(),
mimetype=file.mimetype,
user=current_user,
tenant_id=resource_tenant_id,
source=source,
)
except services.errors.file.FileTooLargeError as file_too_large_error:
raise FileTooLargeError(file_too_large_error.description)
except services.errors.file.UnsupportedFileTypeError:
raise UnsupportedFileTypeError()
except services.errors.file.BlockedFileExtensionError as blocked_extension_error:
raise BlockedFileExtensionError(blocked_extension_error.description)
@console_ns.route("/files/upload")
class FileApi(Resource):
@@ -117,11 +64,41 @@ class FileApi(Resource):
@login_required
@account_initialization_required
@cloud_edition_billing_resource_check("documents")
@console_ns.doc(consumes=["multipart/form-data"], params=FILE_UPLOAD_PARAMS)
@console_ns.response(201, "File uploaded successfully", console_ns.models[FileResponse.__name__])
@with_current_user
def post(self, current_user: Account):
upload_file = upload_file_from_request(current_user=current_user)
source_str = request.form.get("source")
source: Literal["datasets"] | None = "datasets" if source_str == "datasets" else None
if "file" not in request.files:
raise NoFileUploadedError()
if len(request.files) > 1:
raise TooManyFilesError()
file = request.files["file"]
if not file.filename:
raise FilenameNotExistsError
if source == "datasets" and not current_user.is_dataset_editor:
raise Forbidden()
if source not in ("datasets", None):
source = None
try:
upload_file = FileService(db.engine).upload_file(
filename=file.filename,
content=file.stream.read(),
mimetype=file.mimetype,
user=current_user,
source=source,
)
except services.errors.file.FileTooLargeError as file_too_large_error:
raise FileTooLargeError(file_too_large_error.description)
except services.errors.file.UnsupportedFileTypeError:
raise UnsupportedFileTypeError()
except services.errors.file.BlockedFileExtensionError as blocked_extension_error:
raise BlockedFileExtensionError(blocked_extension_error.description)
response = FileResponse.model_validate(upload_file, from_attributes=True)
return response.model_dump(mode="json"), 201
+47 -57
View File
@@ -46,61 +46,6 @@ class GetRemoteFileInfo(Resource):
).model_dump(mode="json")
def upload_remote_file_from_request(
*,
current_user: Account,
resource_tenant_id: str | None = None,
) -> FileWithSignedUrl:
"""Validate the JSON request, fetch its remote file, and persist it under the requested tenant."""
payload = RemoteFileUploadPayload.model_validate(console_ns.payload)
url = payload.url
# Try to fetch remote file metadata/content first
try:
resp = remote_fetcher.make_request("HEAD", url=url)
if resp.status_code != httpx.codes.OK:
resp = remote_fetcher.make_request("GET", url=url, timeout=3, follow_redirects=True)
if resp.status_code != httpx.codes.OK:
# Normalize into a user-friendly error message expected by tests
raise RemoteFileUploadError(f"Failed to fetch file from {url}: {resp.text}")
except httpx.RequestError as e:
raise RemoteFileUploadError(f"Failed to fetch file from {url}: {str(e)}")
file_info = helpers.guess_file_info_from_response(resp)
# Enforce file size limit with 400 (Bad Request) per tests' expectation
if not FileService.is_file_size_within_limit(extension=file_info.extension, file_size=file_info.size):
raise FileTooLargeError()
# Load content if needed
content = resp.content if resp.request.method == "GET" else remote_fetcher.make_request("GET", url).content
try:
upload_file = FileService(db.engine).upload_file(
filename=file_info.filename,
content=content,
mimetype=file_info.mimetype,
user=current_user,
tenant_id=resource_tenant_id,
source_url=url,
)
except services.errors.file.FileTooLargeError as file_too_large_error:
raise FileTooLargeError(file_too_large_error.description)
except services.errors.file.UnsupportedFileTypeError:
raise UnsupportedFileTypeError()
return FileWithSignedUrl(
id=upload_file.id,
name=upload_file.name,
size=upload_file.size,
extension=upload_file.extension,
url=file_helpers.get_signed_file_url(upload_file_id=upload_file.id),
mime_type=upload_file.mime_type,
created_by=upload_file.created_by,
created_at=int(upload_file.created_at.timestamp()),
)
@console_ns.route("/remote-files/upload")
class RemoteFileUpload(Resource):
@console_ns.expect(console_ns.models[RemoteFileUploadPayload.__name__])
@@ -108,8 +53,53 @@ class RemoteFileUpload(Resource):
@login_required
@with_current_user
def post(self, current_user: Account):
remote_file = upload_remote_file_from_request(current_user=current_user)
payload = RemoteFileUploadPayload.model_validate(console_ns.payload)
url = payload.url
# Try to fetch remote file metadata/content first
try:
resp = remote_fetcher.make_request("HEAD", url=url)
if resp.status_code != httpx.codes.OK:
resp = remote_fetcher.make_request("GET", url=url, timeout=3, follow_redirects=True)
if resp.status_code != httpx.codes.OK:
# Normalize into a user-friendly error message expected by tests
raise RemoteFileUploadError(f"Failed to fetch file from {url}: {resp.text}")
except httpx.RequestError as e:
raise RemoteFileUploadError(f"Failed to fetch file from {url}: {str(e)}")
file_info = helpers.guess_file_info_from_response(resp)
# Enforce file size limit with 400 (Bad Request) per tests' expectation
if not FileService.is_file_size_within_limit(extension=file_info.extension, file_size=file_info.size):
raise FileTooLargeError()
# Load content if needed
content = resp.content if resp.request.method == "GET" else remote_fetcher.make_request("GET", url).content
try:
upload_file = FileService(db.engine).upload_file(
filename=file_info.filename,
content=content,
mimetype=file_info.mimetype,
user=current_user,
source_url=url,
)
except services.errors.file.FileTooLargeError as file_too_large_error:
raise FileTooLargeError(file_too_large_error.description)
except services.errors.file.UnsupportedFileTypeError:
raise UnsupportedFileTypeError()
# Success: return created resource with 201 status
return (
remote_file.model_dump(mode="json"),
FileWithSignedUrl(
id=upload_file.id,
name=upload_file.name,
size=upload_file.size,
extension=upload_file.extension,
url=file_helpers.get_signed_file_url(upload_file_id=upload_file.id),
mime_type=upload_file.mime_type,
created_by=upload_file.created_by,
created_at=int(upload_file.created_at.timestamp()),
).model_dump(mode="json"),
201,
)
+1 -9
View File
@@ -36,7 +36,7 @@ from libs.login import current_account_with_tenant, login_required
from models.account import Account, TenantAccountJoin, TenantAccountRole
from services.account_service import AccountService, RegisterService, TenantService
from services.enterprise import rbac_service as enterprise_rbac_service
from services.errors.account import AccountAlreadyInTenantError, SeatsLimitExceededError
from services.errors.account import AccountAlreadyInTenantError
from services.feature_service import FeatureService
@@ -291,14 +291,6 @@ class MemberInviteEmailApi(Resource):
"message": "Account already in workspace.",
}
)
except SeatsLimitExceededError:
invitation_results.append(
{
"status": "failed",
"email": invitee_email,
"message": "Licensed seats limit exceeded.",
}
)
except Exception as e:
invitation_results.append({"status": "failed", "email": invitee_email, "message": str(e)})
@@ -18,6 +18,7 @@ from controllers.console.wraps import (
with_current_tenant_id,
with_current_user,
)
from extensions.ext_database import db
from fields.base import ResponseModel
from graphon.model_runtime.entities.model_entities import ModelType
from graphon.model_runtime.errors.validate import CredentialsValidateFailedError
@@ -352,7 +353,7 @@ class ModelProviderPaymentCheckoutUrlApi(Resource):
def get(self, current_tenant_id: str, current_user: Account, provider: str):
if provider != "anthropic":
raise ValueError(f"provider name {provider} is invalid")
BillingService.is_tenant_owner_or_admin(current_user)
BillingService.is_tenant_owner_or_admin(db.session, current_user)
data = BillingService.get_model_provider_payment_link(
provider_name=provider,
tenant_id=current_tenant_id,
+17 -13
View File
@@ -1,5 +1,6 @@
from __future__ import annotations
from enum import StrEnum
from typing import Any
from flask import request
@@ -13,11 +14,9 @@ from controllers.common.schema import register_response_schema_models
from controllers.console import console_ns
from controllers.console.wraps import RBACPermission, RBACResourceScope, rbac_permission_required
from core.db.session_factory import session_factory
from core.rbac import RBACResourceWhitelistScope
from libs.login import current_account_with_tenant, login_required
from models import Account
from services.enterprise import rbac_service as svc
from tasks.initialize_created_app_rbac_access_task import initialize_created_app_rbac_access_task
class _RBACRoleList(svc.Paginated[svc.RBACRole]):
@@ -512,8 +511,14 @@ class RBACAccessPolicyBindingUnlockApi(Resource):
# ---------------------------------------------------------------------------
class _AccessScope(StrEnum):
ALL = "all"
SPECIFIC = "specific"
ONLY_ME = "only_me"
class _ResourceAccessScopeRequest(BaseModel):
scope: RBACResourceWhitelistScope
scope: _AccessScope
class _ReplaceBindingsRequest(BaseModel):
@@ -579,15 +584,14 @@ class RBACAppWhitelistApi(Resource):
def put(self, app_id):
tenant_id, account_id = _current_ids()
request = _payload(_ResourceAccessScopeRequest)
result = svc.RBACService.AppAccess.replace_whitelist(
tenant_id,
account_id,
str(app_id),
svc.ReplaceMemberBindings(scope=request.scope.value),
return _dump(
svc.RBACService.AppAccess.replace_whitelist(
tenant_id,
account_id,
str(app_id),
svc.ReplaceMemberBindings(scope=request.scope.value),
)
)
if dify_config.RBAC_ENABLED and request.scope is RBACResourceWhitelistScope.ALL:
initialize_created_app_rbac_access_task.delay(tenant_id, account_id, str(app_id))
return _dump(result)
@console_ns.route("/workspaces/current/rbac/apps/<uuid:app_id>/user-access-policies")
@@ -612,8 +616,8 @@ class RBACAppUserAccessPolicyAssignmentApi(Resource):
svc.RBACService.AppAccess.replace_user_access_policies(
tenant_id,
account_id,
app_id,
target_account_id,
str(app_id),
str(target_account_id),
payload,
)
)
+2 -1
View File
@@ -14,11 +14,12 @@ api = ExternalApi(
files_ns = Namespace("files", description="File operations", path="/")
from . import image_preview, tool_files, upload
from . import agent_drive_archive, image_preview, tool_files, upload
api.add_namespace(files_ns)
__all__ = [
"agent_drive_archive",
"api",
"bp",
"files_ns",
@@ -0,0 +1,67 @@
from urllib.parse import quote
from flask import Response, request
from flask_restx import Resource
from pydantic import BaseModel, Field
from werkzeug.exceptions import Forbidden, NotFound
from controllers.common.file_response import enforce_download_for_html
from controllers.common.schema import register_schema_models
from controllers.files import files_ns
from models.agent import AgentDriveFileKind
from services.agent_drive_service import AgentDriveError, AgentDriveService
class AgentDriveArchiveMemberQuery(BaseModel):
tenant_id: str = Field(..., description="Tenant ID")
agent_id: str = Field(..., description="Agent ID")
key: str = Field(..., description="Virtual drive key")
archive_file_kind: AgentDriveFileKind = Field(..., description="Archive file kind")
archive_file_id: str = Field(..., description="Archive file id")
member_path: str = Field(..., description="Zip member path")
timestamp: str = Field(..., description="Unix timestamp")
nonce: str = Field(..., description="Random nonce")
sign: str = Field(..., description="HMAC signature")
as_attachment: bool = Field(default=False, description="Download as attachment")
register_schema_models(files_ns, AgentDriveArchiveMemberQuery)
@files_ns.route("/agent-drive/archive-member")
class AgentDriveArchiveMemberApi(Resource):
@files_ns.doc("get_agent_drive_archive_member")
@files_ns.doc(description="Download a lazily resolved Agent Skill archive member by signed parameters")
def get(self):
args = AgentDriveArchiveMemberQuery.model_validate(request.args.to_dict(flat=True))
if not AgentDriveService.verify_archive_member_signature(
tenant_id=args.tenant_id,
agent_id=args.agent_id,
key=args.key,
archive_file_kind=args.archive_file_kind,
archive_file_id=args.archive_file_id,
member_path=args.member_path,
timestamp=args.timestamp,
nonce=args.nonce,
sign=args.sign,
):
raise Forbidden("Invalid request.")
try:
payload, mime_type, filename = AgentDriveService().load_archive_member_for_signed_request(
tenant_id=args.tenant_id,
agent_id=args.agent_id,
key=args.key,
archive_file_kind=args.archive_file_kind,
archive_file_id=args.archive_file_id,
member_path=args.member_path,
)
except AgentDriveError as exc:
raise NotFound(exc.message) from exc
response = Response(payload, mimetype=mime_type, direct_passthrough=True, headers={})
response.headers["Content-Length"] = str(len(payload))
if args.as_attachment and filename:
encoded_filename = quote(filename)
response.headers["Content-Disposition"] = f"attachment; filename*=UTF-8''{encoded_filename}"
enforce_download_for_html(response, mime_type=mime_type, filename=filename, extension="")
return response
+2 -1
View File
@@ -20,7 +20,7 @@ openapi_ns = Namespace("openapi", description="User-scoped operations", path="/"
# Register response/query models BEFORE importing controller modules so that
# @openapi_ns.response / @openapi_ns.expect decorators can resolve model names.
from controllers.common.fields import EventStreamResponse
from controllers.common.fields import EventStreamResponse, SimpleResultResponse
from controllers.common.schema import register_enum_models, register_response_schema_models, register_schema_models
from controllers.openapi._models import (
AccountPayload,
@@ -95,6 +95,7 @@ register_response_schema_models(
openapi_ns,
ErrorBody,
EventStreamResponse,
SimpleResultResponse,
UsageInfo,
MessageMetadata,
AppListRow,
+3 -2
View File
@@ -3,7 +3,7 @@
from __future__ import annotations
import logging
from collections.abc import Callable, Iterator
from collections.abc import Callable, Generator
from contextlib import contextmanager
from typing import Any
@@ -61,7 +61,7 @@ logger = logging.getLogger(__name__)
@contextmanager
def _translate_service_errors() -> Iterator[None]:
def _translate_service_errors() -> Generator[None, None, None]:
try:
yield
except WorkflowNotFoundError as ex:
@@ -166,6 +166,7 @@ class AppRunApi(Resource):
surface="apps",
)
# response-contract:ignore compact_generate_response
return helper.compact_generate_response(stream_obj)
+6 -3
View File
@@ -25,12 +25,13 @@ from controllers.openapi._models import (
AppListRow,
)
from controllers.openapi.auth.composition import auth_router
from controllers.openapi.auth.data import AuthData, RBACRequirement
from controllers.openapi.auth.data import AuthData, CallerKind, RBACRequirement
from controllers.service_api.app.error import AppUnavailableError
from core.app.app_config.common.parameters_mapping import get_parameters_from_feature_dict
from extensions.ext_database import db
from libs.oauth_bearer import Scope, TokenType
from models import App
from models.enums import AppStatus
from models.model import AppMode
from services.account_service import TenantService
from services.app_service import AppListParams, AppService
@@ -166,7 +167,9 @@ class AppListApi(Resource):
# an empty set or list means the caller has no accessible apps.
# End-users bypass RBAC here — their access is controlled by scope upstream.
apply_rbac_filter = (
dify_config.RBAC_ENABLED and auth_data.caller_kind != "end_user" and auth_data.account_id is not None
dify_config.RBAC_ENABLED
and auth_data.caller_kind != CallerKind.END_USER
and auth_data.account_id is not None
)
access_filter = AppAccessFilter.unrestricted()
if apply_rbac_filter:
@@ -203,7 +206,7 @@ class AppListApi(Resource):
limit=query.limit,
mode=query.mode.value if query.mode else "all", # type:ignore
name=query.name,
status="normal",
status=AppStatus.NORMAL,
# Visibility gate pushed into the query — pagination.total stays
# consistent across pages because invisible rows never count.
openapi_visible=True,
@@ -25,6 +25,7 @@ from controllers.openapi.auth.data import AuthData, Edition
from extensions.ext_database import db
from libs.oauth_bearer import Scope, TokenType
from models import App
from models.enums import AppStatus
from services.account_service import TenantService
from services.app_service import AppService
from services.enterprise.app_permitted_service import list_permitted_apps
@@ -62,7 +63,7 @@ class PermittedExternalAppsListApi(Resource):
items: list[AppListRow] = []
for app_id in page_result.app_ids:
app = apps_by_id.get(app_id)
if not app or app.status != "normal":
if not app or app.status != AppStatus.NORMAL:
continue
tenant = tenants_by_id.get(str(app.tenant_id))
items.append(
+7 -3
View File
@@ -2,7 +2,6 @@ from __future__ import annotations
import uuid
from enum import StrEnum
from typing import Literal
from pydantic import BaseModel, ConfigDict, Field
from werkzeug.exceptions import InternalServerError
@@ -21,6 +20,11 @@ class Edition(StrEnum):
SAAS = "saas"
class CallerKind(StrEnum):
ACCOUNT = "account"
END_USER = "end_user"
def current_edition() -> Edition:
if dify_config.EDITION == "CLOUD":
return Edition.SAAS
@@ -78,9 +82,9 @@ class AuthData(BaseModel):
tenant_role: TenantAccountRole | None = None
caller: Account | EndUser | None = None
caller_kind: Literal["account", "end_user"] | None = None
caller_kind: CallerKind | None = None
def require_app_context(self) -> tuple[App, Account | EndUser, Literal["account", "end_user"]]:
def require_app_context(self) -> tuple[App, Account | EndUser, CallerKind]:
if self.app is None or self.caller is None or self.caller_kind is None:
raise InternalServerError("pipeline_invariant_violated: app context missing")
return self.app, self.caller, self.caller_kind
+7 -7
View File
@@ -5,10 +5,10 @@ import uuid
from flask import request
from werkzeug.exceptions import Forbidden, InternalServerError, NotFound, Unauthorized
from controllers.openapi.auth.data import AuthData
from controllers.openapi.auth.data import AuthData, CallerKind
from extensions.ext_database import db
from models.account import TenantStatus
from models.enums import EndUserType
from models.account import AccountStatus, TenantStatus
from models.enums import AppStatus, EndUserType
from services.account_service import AccountService, TenantService
from services.app_service import AppService
from services.end_user_service import EndUserService
@@ -24,7 +24,7 @@ def load_app(data: AuthData) -> None:
except ValueError:
raise NotFound("app not found")
app = AppService.get_app_by_id(db.session, app_id)
if not app or app.status != "normal":
if not app or app.status != AppStatus.NORMAL:
raise NotFound("app not found")
data.app = app
@@ -65,7 +65,7 @@ def load_account(data: AuthData) -> None:
if data.tenant:
account.current_tenant = data.tenant
data.caller = account
data.caller_kind = "account"
data.caller_kind = CallerKind.ACCOUNT
def load_workspace_role(data: AuthData) -> None:
@@ -73,7 +73,7 @@ def load_workspace_role(data: AuthData) -> None:
return
if data.tenant is None or data.account_id is None:
return
if data.caller is not None and getattr(data.caller, "status", None) != "active":
if data.caller is not None and getattr(data.caller, "status", None) != AccountStatus.ACTIVE:
return
role = TenantService.get_account_role_in_tenant(db.session, str(data.account_id), str(data.tenant.id))
if role is None:
@@ -91,7 +91,7 @@ def resolve_external_user(data: AuthData) -> None:
user_id=data.external_identity.email,
)
data.caller = end_user
data.caller_kind = "end_user"
data.caller_kind = CallerKind.END_USER
def load_app_access_mode(data: AuthData) -> None:
+2 -2
View File
@@ -5,7 +5,7 @@ from werkzeug.exceptions import Forbidden, NotFound, UnprocessableEntity
from configs import dify_config
from controllers.common.wraps import enforce_rbac_access
from controllers.openapi.auth.data import AuthData
from controllers.openapi.auth.data import AuthData, CallerKind
from extensions.ext_database import db
from libs.oauth_bearer import Scope, TokenType
from services.account_service import AccountService, TenantService
@@ -58,7 +58,7 @@ def check_rbac_permission(data: AuthData) -> None:
if not dify_config.RBAC_ENABLED:
return
# Only account callers are subject to RBAC; end_user access is scope-controlled.
if data.caller_kind != "account":
if data.caller_kind != CallerKind.ACCOUNT:
return
if data.account_id is None or data.tenant is None:
raise Forbidden("rbac context missing")
+2 -2
View File
@@ -22,7 +22,7 @@ from controllers.openapi._contract import accepts, returns
from controllers.openapi._errors import HumanInputFormNotFound, RecipientSurfaceMismatch
from controllers.openapi._models import FormSubmitResponse, HumanInputFormDefinitionResponse
from controllers.openapi.auth.composition import auth_router
from controllers.openapi.auth.data import AuthData, RBACRequirement
from controllers.openapi.auth.data import AuthData, CallerKind, RBACRequirement
from core.workflow.human_input_policy import (
HumanInputSurface,
is_recipient_type_allowed_for_surface,
@@ -98,7 +98,7 @@ class OpenApiWorkflowHumanInputFormApi(Resource):
submission_user_id: str | None = None
submission_end_user_id: str | None = None
if caller_kind == "account":
if caller_kind == CallerKind.ACCOUNT:
submission_user_id = caller.id
else:
submission_end_user_id = caller.id
+2 -2
View File
@@ -22,7 +22,7 @@ from controllers.common.schema import query_params_from_model
from controllers.common.wraps import RBACPermission, RBACResourceScope
from controllers.openapi import openapi_ns
from controllers.openapi.auth.composition import auth_router
from controllers.openapi.auth.data import AuthData, RBACRequirement
from controllers.openapi.auth.data import AuthData, CallerKind, RBACRequirement
from core.app.apps.advanced_chat.app_generator import AdvancedChatAppGenerator
from core.app.apps.base_app_generator import BaseAppGenerator
from core.app.apps.common.workflow_response_converter import WorkflowResponseConverter
@@ -70,7 +70,7 @@ class OpenApiWorkflowEventsApi(Resource):
if workflow_run.app_id != app_model.id:
raise NotFound("Workflow run not found")
if caller_kind == "account":
if caller_kind == CallerKind.ACCOUNT:
if workflow_run.created_by_role != CreatorUserRole.ACCOUNT or workflow_run.created_by != caller.id:
raise NotFound("Workflow run not found")
else:
-3
View File
@@ -48,7 +48,6 @@ from services.errors.account import (
MemberNotInTenantError,
NoPermissionError,
RoleAlreadyAssignedError,
SeatsLimitExceededError,
)
from services.feature_service import FeatureService
@@ -191,8 +190,6 @@ class WorkspaceMembersApi(Resource):
raise BadRequest(str(exc))
except NoPermissionError as exc:
raise BadRequest(str(exc))
except SeatsLimitExceededError:
raise BadRequest("licensed seats limit exceeded")
except AccountRegisterError as exc:
raise BadRequest(str(exc))
+2 -13
View File
@@ -43,7 +43,6 @@ from models.dataset import DatasetPermissionEnum
from models.enums import TagType
from models.provider_ids import ModelProviderID
from services.dataset_service import DatasetPermissionService, DatasetService, DocumentService
from services.enterprise.rbac_service import RBACResourceWhitelistScope, RBACService, ReplaceMemberBindings
from services.entities.knowledge_entities.knowledge_entities import (
ExternalRetrievalModel,
KnowledgeProvider,
@@ -59,7 +58,6 @@ from services.tag_service import (
from services.tag_service import (
UpdateTagPayload as UpdateTagServicePayload,
)
from tasks.initialize_created_app_rbac_access_task import initialize_created_app_rbac_access_task
register_enum_models(service_api_ns, DatasetPermissionEnum)
@@ -525,15 +523,6 @@ class DatasetListApi(DatasetApiResource):
except services.errors.dataset.DatasetNameDuplicateError:
raise DatasetNameDuplicateError()
if payload.permission == DatasetPermissionEnum.ALL_TEAM and dify_config.RBAC_ENABLED:
RBACService.DatasetAccess.replace_whitelist(
tenant_id,
current_user.id,
dataset.id,
ReplaceMemberBindings(scope=RBACResourceWhitelistScope.ALL),
)
initialize_created_app_rbac_access_task.delay(tenant_id, current_user.id, dataset_id=dataset.id)
return _dump_service_dataset_detail(dataset), 200
@@ -576,7 +565,7 @@ class DatasetApi(DatasetApiResource):
if dataset is None:
raise NotFound("Dataset not found.")
try:
DatasetService.check_dataset_permission(dataset, current_user)
DatasetService.check_dataset_permission(dataset, current_user, db.session)
except services.errors.account.NoPermissionError as e:
raise Forbidden(str(e))
data = _dump_service_dataset_detail(dataset)
@@ -830,7 +819,7 @@ class DocumentStatusApi(DatasetApiResource):
# Check user's permission
try:
DatasetService.check_dataset_permission(dataset, current_user)
DatasetService.check_dataset_permission(dataset, current_user, db.session)
except services.errors.account.NoPermissionError as e:
raise Forbidden(str(e))
@@ -84,7 +84,7 @@ class DatasetMetadataCreateServiceApi(DatasetApiResource):
dataset = DatasetService.get_dataset(dataset_id_str)
if dataset is None:
raise NotFound("Dataset not found.")
DatasetService.check_dataset_permission(dataset, current_user)
DatasetService.check_dataset_permission(dataset, current_user, db.session)
metadata = MetadataService.create_metadata(db.session(), dataset_id_str, metadata_args)
return dump_response(DatasetMetadataResponse, metadata), 201
@@ -157,7 +157,7 @@ class DatasetMetadataServiceApi(DatasetApiResource):
dataset = DatasetService.get_dataset(dataset_id_str)
if dataset is None:
raise NotFound("Dataset not found.")
DatasetService.check_dataset_permission(dataset, current_user)
DatasetService.check_dataset_permission(dataset, current_user, db.session)
metadata = MetadataService.update_metadata_name(db.session(), dataset_id_str, metadata_id_str, payload.name)
return dump_response(DatasetMetadataResponse, metadata), 200
@@ -192,7 +192,7 @@ class DatasetMetadataServiceApi(DatasetApiResource):
dataset = DatasetService.get_dataset(dataset_id_str)
if dataset is None:
raise NotFound("Dataset not found.")
DatasetService.check_dataset_permission(dataset, current_user)
DatasetService.check_dataset_permission(dataset, current_user, db.session)
MetadataService.delete_metadata(db.session(), dataset_id_str, metadata_id_str)
return "", 204
@@ -260,7 +260,7 @@ class DatasetMetadataBuiltInFieldActionServiceApi(DatasetApiResource):
dataset = DatasetService.get_dataset(dataset_id_str)
if dataset is None:
raise NotFound("Dataset not found.")
DatasetService.check_dataset_permission(dataset, current_user)
DatasetService.check_dataset_permission(dataset, current_user, db.session)
match action:
case "enable":
@@ -306,7 +306,7 @@ class DocumentMetadataEditServiceApi(DatasetApiResource):
dataset = DatasetService.get_dataset(dataset_id_str)
if dataset is None:
raise NotFound("Dataset not found.")
DatasetService.check_dataset_permission(dataset, current_user)
DatasetService.check_dataset_permission(dataset, current_user, db.session)
metadata_args = MetadataOperationData.model_validate(service_api_ns.payload or {})
+2 -3
View File
@@ -12,7 +12,6 @@ from flask_restx import Resource
from flask_restx.utils import merge
from pydantic import BaseModel
from sqlalchemy import select
from sqlalchemy.orm import sessionmaker
from werkzeug.exceptions import Forbidden, NotFound, Unauthorized
from configs import dify_config
@@ -270,8 +269,8 @@ def cloud_edition_billing_rate_limit_check[**P, R](
subscription_plan=knowledge_rate_limit.subscription_plan,
operation="knowledge",
)
with sessionmaker(bind=db.engine, expire_on_commit=False).begin() as session:
session.add(rate_limit_log)
db.session.add(rate_limit_log)
db.session.commit()
raise Forbidden(
"Sorry, you have reached the knowledge base request rate limit of your subscription."
)
+2 -25
View File
@@ -7,26 +7,10 @@ from werkzeug.exceptions import NotFound, RequestEntityTooLarge
from controllers.trigger import bp
from core.trigger.debug.event_bus import TriggerDebugEventBus
from core.trigger.debug.events import WebhookDebugEvent, build_webhook_pool_key
from enums.quota_type import QuotaType
from services.errors.app import QuotaExceededError
from services.trigger.webhook_service import RawWebhookDataDict, WebhookService
logger = logging.getLogger(__name__)
_QUOTA_EXCEEDED_MESSAGES = {
QuotaType.TRIGGER: "Trigger event quota exceeded. Please upgrade your plan.",
QuotaType.WORKFLOW: "Workflow execution quota exceeded. Please upgrade your plan.",
}
_DEFAULT_QUOTA_EXCEEDED_MESSAGE = "Quota exceeded. Please upgrade your plan."
def _get_quota_exceeded_message(feature: str) -> str:
try:
quota_type = QuotaType(feature)
except ValueError:
return _DEFAULT_QUOTA_EXCEEDED_MESSAGE
return _QUOTA_EXCEEDED_MESSAGES.get(quota_type, _DEFAULT_QUOTA_EXCEEDED_MESSAGE)
def _prepare_webhook_execution(webhook_id: str, is_debug: bool = False):
"""Fetch trigger context, extract request data, and validate payload using unified processing.
@@ -76,15 +60,8 @@ def handle_webhook(webhook_id: str):
response_data, status_code = WebhookService.generate_webhook_response(node_config)
return jsonify(response_data), status_code
except QuotaExceededError as error:
return jsonify(
{
"error": "Too Many Requests",
"message": _get_quota_exceeded_message(error.feature),
}
), 429
except ValueError as error:
raise NotFound(str(error))
except ValueError as e:
raise NotFound(str(e))
except RequestEntityTooLarge:
raise
except Exception as e:
+1 -7
View File
@@ -29,7 +29,6 @@ from controllers.console.wraps import (
)
from controllers.web import web_ns
from controllers.web.wraps import decode_jwt_token
from core.logging.context import set_identity_context
from libs.helper import EmailStr, extract_remote_ip
from libs.passport import PassportService
from libs.password import valid_password
@@ -161,12 +160,7 @@ class LoginStatusApi(Resource):
user_logged_in = False
try:
_, end_user = decode_jwt_token(app_code=app_code, user_id=user_id)
set_identity_context(
tenant_id=end_user.tenant_id,
user_id=end_user.id,
user_type=end_user.type or "end_user",
)
_ = decode_jwt_token(app_code=app_code, user_id=user_id)
app_logged_in = True
except Exception:
app_logged_in = False
+29 -5
View File
@@ -14,7 +14,7 @@ from fields.base import ResponseModel
from libs.helper import AppIconUrlField
from models.account import TenantStatus
from models.model import App, EndUser, Site
from services.feature_service import FeatureService
from services.feature_service import FeatureModel, FeatureService
class AppSiteModelConfigResponse(ResponseModel):
@@ -38,6 +38,7 @@ class AppSiteResponse(ResponseModel):
description: str | None = None
copyright: str | None = None
privacy_policy: str | None = None
input_placeholder: str | None = None
custom_disclaimer: str | None = None
default_language: str | None = None
prompt_public: bool | None = None
@@ -84,6 +85,7 @@ class AppSiteApi(WebApiResource):
"description": fields.String,
"copyright": fields.String,
"privacy_policy": fields.String,
"input_placeholder": fields.String,
"custom_disclaimer": fields.String,
"default_language": fields.String,
"prompt_public": fields.Boolean,
@@ -127,9 +129,15 @@ class AppSiteApi(WebApiResource):
if app_model.tenant and app_model.tenant.status == TenantStatus.ARCHIVE:
raise Forbidden()
can_replace_logo = FeatureService.get_features(app_model.tenant_id, exclude_vector_space=True).can_replace_logo
features = FeatureService.get_features(app_model.tenant_id, exclude_vector_space=True)
return AppSiteInfo(app_model.tenant, app_model, site, end_user.id, can_replace_logo)
return AppSiteInfo(
app_model.tenant,
app_model,
serialize_runtime_site(site, features),
end_user.id,
features.can_replace_logo,
)
class AppSiteInfo:
@@ -164,7 +172,23 @@ def serialize_site(site: Site) -> dict[str, Any]:
return cast(dict[str, Any], marshal(site, AppSiteApi.site_fields))
def serialize_runtime_site(site: Site, features: FeatureModel) -> dict[str, Any]:
site_payload = serialize_site(site)
if not features.billing.enabled or features.webapp_copyright_enabled:
return site_payload
site_payload["copyright"] = None
site_payload["input_placeholder"] = None
return site_payload
def serialize_app_site_payload(app_model: App, site: Site, end_user_id: str | None) -> dict[str, Any]:
can_replace_logo = FeatureService.get_features(app_model.tenant_id, exclude_vector_space=True).can_replace_logo
app_site_info = AppSiteInfo(app_model.tenant, app_model, site, end_user_id, can_replace_logo)
features = FeatureService.get_features(app_model.tenant_id, exclude_vector_space=True)
app_site_info = AppSiteInfo(
app_model.tenant,
app_model,
serialize_runtime_site(site, features),
end_user_id,
features.can_replace_logo,
)
return cast(dict[str, Any], marshal(app_site_info, AppSiteApi.app_fields))
-6
View File
@@ -11,7 +11,6 @@ from werkzeug.exceptions import BadRequest, NotFound, Unauthorized
from constants import HEADER_NAME_APP_CODE
from controllers.web.error import WebAppAuthAccessDeniedError, WebAppAuthRequiredError
from core.logging.context import set_identity_context
from extensions.ext_database import db
from libs.passport import PassportService
from libs.token import extract_webapp_passport
@@ -29,11 +28,6 @@ def validate_jwt_token[**P, R](
@wraps(view)
def decorated(*args: P.args, **kwargs: P.kwargs) -> R:
app_model, end_user = decode_jwt_token()
set_identity_context(
tenant_id=end_user.tenant_id,
user_id=end_user.id,
user_type=end_user.type or "end_user",
)
return view(app_model, end_user, *args, **kwargs)
return decorated
@@ -47,7 +47,6 @@ from core.repositories import DifyCoreRepositoryFactory
from core.repositories.factory import WorkflowExecutionRepository, WorkflowNodeExecutionRepository
from extensions.ext_database import db
from factories import file_factory
from graphon.filters import ResponseStreamFilter
from graphon.graph_engine.layers import GraphEngineLayer
from graphon.model_runtime.errors.invoke import InvokeAuthorizationError
from graphon.runtime import GraphRuntimeState
@@ -232,7 +231,6 @@ class AdvancedChatAppGenerator(MessageBasedAppGenerator):
workflow_triggered_from = WorkflowRunTriggeredFrom.APP_RUN
workflow_execution_repository = DifyCoreRepositoryFactory.create_workflow_execution_repository(
session_factory=session_factory,
tenant_id=app_model.tenant_id,
user=user,
app_id=application_generate_entity.app_config.app_id,
triggered_from=workflow_triggered_from,
@@ -240,7 +238,6 @@ class AdvancedChatAppGenerator(MessageBasedAppGenerator):
# Create workflow node execution repository
workflow_node_execution_repository = DifyCoreRepositoryFactory.create_workflow_node_execution_repository(
session_factory=session_factory,
tenant_id=app_model.tenant_id,
user=user,
app_id=application_generate_entity.app_config.app_id,
triggered_from=WorkflowNodeExecutionTriggeredFrom.WORKFLOW_RUN,
@@ -271,7 +268,6 @@ class AdvancedChatAppGenerator(MessageBasedAppGenerator):
workflow_node_execution_repository: WorkflowNodeExecutionRepository,
graph_runtime_state: GraphRuntimeState,
pause_state_config: PauseStateLayerConfig | None = None,
response_stream_filter: ResponseStreamFilter | None = None,
):
"""
Resume a paused advanced chat execution.
@@ -301,7 +297,6 @@ class AdvancedChatAppGenerator(MessageBasedAppGenerator):
stream=application_generate_entity.stream,
pause_state_config=pause_state_config,
graph_runtime_state=graph_runtime_state,
response_stream_filter=response_stream_filter,
)
def single_iteration_generate(
@@ -361,7 +356,6 @@ class AdvancedChatAppGenerator(MessageBasedAppGenerator):
# Create workflow execution(aka workflow run) repository
workflow_execution_repository = DifyCoreRepositoryFactory.create_workflow_execution_repository(
session_factory=session_factory,
tenant_id=app_model.tenant_id,
user=user,
app_id=application_generate_entity.app_config.app_id,
triggered_from=WorkflowRunTriggeredFrom.DEBUGGING,
@@ -369,7 +363,6 @@ class AdvancedChatAppGenerator(MessageBasedAppGenerator):
# Create workflow node execution repository
workflow_node_execution_repository = DifyCoreRepositoryFactory.create_workflow_node_execution_repository(
session_factory=session_factory,
tenant_id=app_model.tenant_id,
user=user,
app_id=application_generate_entity.app_config.app_id,
triggered_from=WorkflowNodeExecutionTriggeredFrom.SINGLE_STEP,
@@ -450,7 +443,6 @@ class AdvancedChatAppGenerator(MessageBasedAppGenerator):
# Create workflow execution(aka workflow run) repository
workflow_execution_repository = DifyCoreRepositoryFactory.create_workflow_execution_repository(
session_factory=session_factory,
tenant_id=app_model.tenant_id,
user=user,
app_id=application_generate_entity.app_config.app_id,
triggered_from=WorkflowRunTriggeredFrom.DEBUGGING,
@@ -458,7 +450,6 @@ class AdvancedChatAppGenerator(MessageBasedAppGenerator):
# Create workflow node execution repository
workflow_node_execution_repository = DifyCoreRepositoryFactory.create_workflow_node_execution_repository(
session_factory=session_factory,
tenant_id=app_model.tenant_id,
user=user,
app_id=application_generate_entity.app_config.app_id,
triggered_from=WorkflowNodeExecutionTriggeredFrom.SINGLE_STEP,
@@ -500,7 +491,6 @@ class AdvancedChatAppGenerator(MessageBasedAppGenerator):
pause_state_config: PauseStateLayerConfig | None = None,
graph_runtime_state: GraphRuntimeState | None = None,
graph_engine_layers: Sequence[GraphEngineLayer] = (),
response_stream_filter: ResponseStreamFilter | None = None,
) -> Mapping[str, Any] | Generator[str | Mapping[str, Any], None, None]:
"""
Generate App response.
@@ -548,14 +538,12 @@ class AdvancedChatAppGenerator(MessageBasedAppGenerator):
)
graph_layers: list[GraphEngineLayer] = list(graph_engine_layers)
resolved_response_stream_filter = response_stream_filter or ResponseStreamFilter()
if pause_state_config is not None:
graph_layers.append(
PauseStatePersistenceLayer(
session_factory=pause_state_config.session_factory,
generate_entity=application_generate_entity,
state_owner_user_id=pause_state_config.state_owner_user_id,
response_stream_filter=resolved_response_stream_filter,
)
)
@@ -576,7 +564,6 @@ class AdvancedChatAppGenerator(MessageBasedAppGenerator):
"workflow_node_execution_repository": workflow_node_execution_repository,
"graph_engine_layers": tuple(graph_layers),
"graph_runtime_state": graph_runtime_state,
"response_stream_filter": resolved_response_stream_filter,
},
)
@@ -598,11 +585,7 @@ class AdvancedChatAppGenerator(MessageBasedAppGenerator):
message=message_snapshot,
user=user,
stream=stream,
draft_var_saver_factory=self._get_draft_var_saver_factory(
invoke_from,
account=user,
tenant_id=application_generate_entity.app_config.tenant_id,
),
draft_var_saver_factory=self._get_draft_var_saver_factory(invoke_from, account=user),
)
return AdvancedChatAppGenerateResponseConverter.convert(response=response, invoke_from=invoke_from)
@@ -620,7 +603,6 @@ class AdvancedChatAppGenerator(MessageBasedAppGenerator):
workflow_node_execution_repository: WorkflowNodeExecutionRepository,
graph_engine_layers: Sequence[GraphEngineLayer] = (),
graph_runtime_state: GraphRuntimeState | None = None,
response_stream_filter: ResponseStreamFilter | None = None,
):
"""
Generate worker in a new thread.
@@ -680,7 +662,6 @@ class AdvancedChatAppGenerator(MessageBasedAppGenerator):
workflow_node_execution_repository=workflow_node_execution_repository,
graph_engine_layers=graph_engine_layers,
graph_runtime_state=graph_runtime_state,
response_stream_filter=response_stream_filter,
)
try:
@@ -39,7 +39,6 @@ from extensions.ext_database import db
from extensions.ext_redis import redis_client
from extensions.otel import WorkflowAppRunnerHandler, trace_span
from graphon.enums import WorkflowType
from graphon.filters import ResponseStreamFilter
from graphon.graph_engine.command_channels import RedisChannel
from graphon.graph_engine.layers import GraphEngineLayer
from graphon.runtime import GraphRuntimeState, VariablePool
@@ -74,7 +73,6 @@ class AdvancedChatAppRunner(WorkflowBasedAppRunner):
workflow_node_execution_repository: WorkflowNodeExecutionRepository,
graph_engine_layers: Sequence[GraphEngineLayer] = (),
graph_runtime_state: GraphRuntimeState | None = None,
response_stream_filter: ResponseStreamFilter | None = None,
):
super().__init__(
queue_manager=queue_manager,
@@ -92,7 +90,6 @@ class AdvancedChatAppRunner(WorkflowBasedAppRunner):
self._workflow_execution_repository = workflow_execution_repository
self._workflow_node_execution_repository = workflow_node_execution_repository
self._resume_graph_runtime_state = graph_runtime_state
self._response_stream_filter = response_stream_filter
@trace_span(WorkflowAppRunnerHandler)
def run(self):
@@ -230,7 +227,6 @@ class AdvancedChatAppRunner(WorkflowBasedAppRunner):
variable_pool=variable_pool,
graph_runtime_state=graph_runtime_state,
command_channel=command_channel,
response_stream_filter=self._response_stream_filter,
)
self._queue_manager.graph_runtime_state = graph_runtime_state
+130 -27
View File
@@ -16,7 +16,7 @@ from collections.abc import Generator, Mapping
from typing import Any
from flask import Flask, current_app
from sqlalchemy import select
from sqlalchemy import and_, or_, select
from clients.agent_backend import AgentBackendRunEventAdapter
from clients.agent_backend.factory import create_agent_backend_run_client
@@ -42,7 +42,15 @@ from core.app.llm.model_access import build_dify_model_access
from core.ops.ops_trace_manager import TraceQueueManager
from extensions.ext_database import db
from models import Account, App, EndUser, Message
from models.agent import Agent, AgentConfigSnapshot, AgentScope, AgentSource, AgentStatus
from models.agent import (
Agent,
AgentConfigDraft,
AgentConfigDraftType,
AgentConfigSnapshot,
AgentScope,
AgentSource,
AgentStatus,
)
from models.agent_config_entities import AgentSoulConfig
from services.conversation_service import ConversationService
@@ -73,10 +81,15 @@ class AgentAppGenerator(MessageBasedAppGenerator):
inputs = args["inputs"]
# Resolve the bound roster Agent + its current Agent Soul snapshot.
agent, snapshot, agent_soul = self._resolve_agent(app_model)
agent, agent_config_id, agent_soul = self._resolve_agent(
app_model,
invoke_from=invoke_from,
draft_type=args.get("draft_type"),
user=user,
)
runtime_session_snapshot_id = self._runtime_session_snapshot_id(
invoke_from=invoke_from,
snapshot_id=snapshot.id,
snapshot_id=agent_config_id,
)
conversation = None
@@ -123,7 +136,7 @@ class AgentAppGenerator(MessageBasedAppGenerator):
call_depth=0,
trace_manager=trace_manager,
agent_id=agent.id,
agent_config_snapshot_id=snapshot.id,
agent_config_snapshot_id=agent_config_id,
agent_runtime_session_snapshot_id=runtime_session_snapshot_id,
)
@@ -179,7 +192,12 @@ class AgentAppGenerator(MessageBasedAppGenerator):
persisted to the conversation. Live streaming to a reconnected client is
out of scope here the message is persisted and can be re-fetched.
"""
agent, snapshot, agent_soul = self._resolve_agent(app_model)
agent, agent_config_id, agent_soul = self._resolve_agent(
app_model,
invoke_from=invoke_from,
draft_type="draft",
user=user,
)
conversation = ConversationService.get_conversation(
app_model=app_model, conversation_id=conversation_id, user=user
)
@@ -226,7 +244,7 @@ class AgentAppGenerator(MessageBasedAppGenerator):
call_depth=0,
trace_manager=trace_manager,
agent_id=agent.id,
agent_config_snapshot_id=snapshot.id,
agent_config_snapshot_id=agent_config_id,
)
conversation, message = self._init_generate_records(application_generate_entity, conversation)
@@ -421,50 +439,135 @@ class AgentAppGenerator(MessageBasedAppGenerator):
return False, query
def _resolve_agent(self, app_model: App) -> tuple[Agent, AgentConfigSnapshot, AgentSoulConfig]:
def _resolve_agent(
self,
app_model: App,
*,
invoke_from: InvokeFrom,
draft_type: Any,
user: Account | EndUser,
) -> tuple[Agent, str, AgentSoulConfig]:
agent = db.session.scalar(
select(Agent).where(
Agent.app_id == app_model.id,
Agent.scope == AgentScope.ROSTER,
Agent.source == AgentSource.AGENT_APP,
select(Agent)
.where(
Agent.tenant_id == app_model.tenant_id,
Agent.status == AgentStatus.ACTIVE,
or_(
and_(
Agent.app_id == app_model.id,
Agent.scope == AgentScope.ROSTER,
Agent.source == AgentSource.AGENT_APP,
),
Agent.backing_app_id == app_model.id,
),
)
.order_by(Agent.created_at.desc())
.limit(1)
)
if agent is None:
raise AgentAppGeneratorError("Agent App has no bound Agent")
return self._resolve_agent_by_id(
tenant_id=app_model.tenant_id, agent_id=agent.id, snapshot_id=agent.active_config_snapshot_id
if invoke_from == InvokeFrom.DEBUGGER:
draft = self._resolve_debug_draft(
tenant_id=app_model.tenant_id,
agent=agent,
draft_type=draft_type,
account_id=user.id if isinstance(user, Account) else None,
)
agent_soul = AgentSoulConfig.model_validate(draft.config_snapshot_dict)
return agent, draft.id, agent_soul
_, snapshot, agent_soul = self._resolve_agent_by_id(
tenant_id=app_model.tenant_id,
agent_id=agent.id,
snapshot_id=agent.active_config_snapshot_id,
)
return agent, snapshot.id, agent_soul
@staticmethod
def _runtime_session_snapshot_id(*, invoke_from: InvokeFrom, snapshot_id: str) -> str | None:
"""Return the session scope snapshot id for Agent App runtime state.
Console preview/debug chat is an editing workspace: saving Agent Soul
creates replacement snapshots, but the user expects the same preview
conversation to keep context while trying prompt changes. Use a stable
NULL snapshot scope for debugger runs so each turn can use the latest
Agent Soul while reusing the conversation history. Published/web/API
runs keep snapshot-scoped sessions for reproducible runtime state.
Console preview/debug chat uses a stable Agent draft row id; build mode
uses the current user's build-draft row id. Published/web/API runs use
immutable published snapshot ids. This keeps runtime session continuity
inside one editable surface without mixing draft/build/published state.
"""
if invoke_from == InvokeFrom.DEBUGGER:
return None
return snapshot_id
@staticmethod
def _resolve_debug_draft(
*, tenant_id: str, agent: Agent, draft_type: Any, account_id: str | None
) -> AgentConfigDraft:
effective_draft_type = (
AgentConfigDraftType.DEBUG_BUILD
if draft_type == AgentConfigDraftType.DEBUG_BUILD.value
else AgentConfigDraftType.DRAFT
)
stmt = select(AgentConfigDraft).where(
AgentConfigDraft.tenant_id == tenant_id,
AgentConfigDraft.agent_id == agent.id,
AgentConfigDraft.draft_type == effective_draft_type,
)
if effective_draft_type == AgentConfigDraftType.DEBUG_BUILD:
if not account_id:
raise AgentAppGeneratorError("Build draft requires an account user")
stmt = stmt.where(AgentConfigDraft.account_id == account_id)
else:
stmt = stmt.where(AgentConfigDraft.account_id.is_(None))
draft = db.session.scalar(stmt.order_by(AgentConfigDraft.updated_at.desc()).limit(1))
if draft is not None:
return draft
if effective_draft_type == AgentConfigDraftType.DEBUG_BUILD:
raise AgentAppGeneratorError("Agent build draft not found")
_, snapshot, agent_soul = AgentAppGenerator._resolve_agent_by_id(
tenant_id=tenant_id,
agent_id=agent.id,
snapshot_id=agent.active_config_snapshot_id,
)
draft = AgentConfigDraft(
tenant_id=tenant_id,
agent_id=agent.id,
draft_type=AgentConfigDraftType.DRAFT,
account_id=None,
draft_owner_key="",
base_snapshot_id=snapshot.id,
config_snapshot=agent_soul,
created_by=agent.created_by,
updated_by=agent.updated_by,
)
db.session.add(draft)
db.session.flush()
return draft
@staticmethod
def _resolve_agent_by_id(
*, tenant_id: str, agent_id: str, snapshot_id: str | None
) -> tuple[Agent, AgentConfigSnapshot, AgentSoulConfig]:
) -> tuple[Agent, AgentConfigSnapshot | AgentConfigDraft, AgentSoulConfig]:
agent = db.session.scalar(select(Agent).where(Agent.id == agent_id, Agent.tenant_id == tenant_id))
if agent is None:
raise AgentAppGeneratorError("Agent not found")
if not snapshot_id:
raise AgentAppGeneratorError("Agent has no published version")
snapshot = db.session.scalar(select(AgentConfigSnapshot).where(AgentConfigSnapshot.id == snapshot_id))
if snapshot is None:
snapshot = db.session.scalar(
select(AgentConfigSnapshot).where(
AgentConfigSnapshot.tenant_id == tenant_id,
AgentConfigSnapshot.agent_id == agent_id,
AgentConfigSnapshot.id == snapshot_id,
)
)
if snapshot is not None:
agent_soul = AgentSoulConfig.model_validate(snapshot.config_snapshot_dict)
return agent, snapshot, agent_soul
draft = db.session.scalar(
select(AgentConfigDraft).where(
AgentConfigDraft.tenant_id == tenant_id,
AgentConfigDraft.agent_id == agent_id,
AgentConfigDraft.id == snapshot_id,
)
)
if draft is None:
raise AgentAppGeneratorError("Agent published version not found")
agent_soul = AgentSoulConfig.model_validate(snapshot.config_snapshot_dict)
return agent, snapshot, agent_soul
agent_soul = AgentSoulConfig.model_validate(draft.config_snapshot_dict)
return agent, draft, agent_soul
__all__ = ["AgentAppGenerator", "AgentAppGeneratorError"]
+1 -10
View File
@@ -32,7 +32,6 @@ class _DebuggerDraftVariableSaver:
self,
*,
account: Account,
tenant_id: str,
app_id: str,
node_id: str,
node_type: NodeType,
@@ -40,7 +39,6 @@ class _DebuggerDraftVariableSaver:
enclosing_node_id: str | None = None,
) -> None:
self._account = account
self._tenant_id = tenant_id
self._app_id = app_id
self._node_id = node_id
self._node_type = node_type
@@ -51,7 +49,6 @@ class _DebuggerDraftVariableSaver:
with Session(db.engine) as session, session.begin():
DraftVariableSaverImpl(
session=session,
tenant_id=self._tenant_id,
app_id=self._app_id,
node_id=self._node_id,
node_type=self._node_type,
@@ -290,12 +287,7 @@ class BaseAppGenerator:
@final
@staticmethod
def _get_draft_var_saver_factory(
invoke_from: InvokeFrom,
account: Account | EndUser,
*,
tenant_id: str,
) -> DraftVariableSaverFactory:
def _get_draft_var_saver_factory(invoke_from: InvokeFrom, account: Account | EndUser) -> DraftVariableSaverFactory:
if invoke_from == InvokeFrom.DEBUGGER:
assert isinstance(account, Account)
@@ -308,7 +300,6 @@ class BaseAppGenerator:
) -> DraftVariableSaver:
return _DebuggerDraftVariableSaver(
account=account,
tenant_id=tenant_id,
app_id=app_id,
node_id=node_id,
node_type=node_type,
@@ -214,7 +214,6 @@ class PipelineGenerator(BaseAppGenerator):
session_factory = sessionmaker(bind=db.engine, expire_on_commit=False)
workflow_execution_repository = DifyCoreRepositoryFactory.create_workflow_execution_repository(
session_factory=session_factory,
tenant_id=pipeline.tenant_id,
user=user,
app_id=application_generate_entity.app_config.app_id,
triggered_from=workflow_triggered_from,
@@ -222,7 +221,6 @@ class PipelineGenerator(BaseAppGenerator):
workflow_node_execution_repository = DifyCoreRepositoryFactory.create_workflow_node_execution_repository(
session_factory=session_factory,
tenant_id=pipeline.tenant_id,
user=user,
app_id=application_generate_entity.app_config.app_id,
triggered_from=WorkflowNodeExecutionTriggeredFrom.RAG_PIPELINE_RUN,
@@ -343,7 +341,6 @@ class PipelineGenerator(BaseAppGenerator):
draft_var_saver_factory = self._get_draft_var_saver_factory(
invoke_from,
user,
tenant_id=pipeline.tenant_id,
)
# return response or stream generator
response = self._handle_response(
@@ -420,7 +417,6 @@ class PipelineGenerator(BaseAppGenerator):
workflow_execution_repository = DifyCoreRepositoryFactory.create_workflow_execution_repository(
session_factory=session_factory,
tenant_id=pipeline.tenant_id,
user=user,
app_id=application_generate_entity.app_config.app_id,
triggered_from=WorkflowRunTriggeredFrom.RAG_PIPELINE_DEBUGGING,
@@ -428,7 +424,6 @@ class PipelineGenerator(BaseAppGenerator):
workflow_node_execution_repository = DifyCoreRepositoryFactory.create_workflow_node_execution_repository(
session_factory=session_factory,
tenant_id=pipeline.tenant_id,
user=user,
app_id=application_generate_entity.app_config.app_id,
triggered_from=WorkflowNodeExecutionTriggeredFrom.SINGLE_STEP,
@@ -518,7 +513,6 @@ class PipelineGenerator(BaseAppGenerator):
workflow_execution_repository = DifyCoreRepositoryFactory.create_workflow_execution_repository(
session_factory=session_factory,
tenant_id=pipeline.tenant_id,
user=user,
app_id=application_generate_entity.app_config.app_id,
triggered_from=WorkflowRunTriggeredFrom.RAG_PIPELINE_DEBUGGING,
@@ -526,7 +520,6 @@ class PipelineGenerator(BaseAppGenerator):
workflow_node_execution_repository = DifyCoreRepositoryFactory.create_workflow_node_execution_repository(
session_factory=session_factory,
tenant_id=pipeline.tenant_id,
user=user,
app_id=application_generate_entity.app_config.app_id,
triggered_from=WorkflowNodeExecutionTriggeredFrom.SINGLE_STEP,
+1 -20
View File
@@ -42,7 +42,6 @@ from core.repositories import DifyCoreRepositoryFactory
from core.repositories.factory import WorkflowExecutionRepository, WorkflowNodeExecutionRepository
from extensions.ext_database import db
from factories import file_factory
from graphon.filters import ResponseStreamFilter
from graphon.graph_engine.layers import GraphEngineLayer
from graphon.model_runtime.errors.invoke import InvokeAuthorizationError
from graphon.runtime import GraphRuntimeState
@@ -242,7 +241,6 @@ class WorkflowAppGenerator(BaseAppGenerator):
workflow_triggered_from = WorkflowRunTriggeredFrom.APP_RUN
workflow_execution_repository = DifyCoreRepositoryFactory.create_workflow_execution_repository(
session_factory=session_factory,
tenant_id=app_model.tenant_id,
user=user,
app_id=application_generate_entity.app_config.app_id,
triggered_from=workflow_triggered_from,
@@ -250,7 +248,6 @@ class WorkflowAppGenerator(BaseAppGenerator):
# Create workflow node execution repository
workflow_node_execution_repository = DifyCoreRepositoryFactory.create_workflow_node_execution_repository(
session_factory=session_factory,
tenant_id=app_model.tenant_id,
user=user,
app_id=application_generate_entity.app_config.app_id,
triggered_from=WorkflowNodeExecutionTriggeredFrom.WORKFLOW_RUN,
@@ -283,7 +280,6 @@ class WorkflowAppGenerator(BaseAppGenerator):
graph_engine_layers: Sequence[GraphEngineLayer] = (),
pause_state_config: PauseStateLayerConfig | None = None,
variable_loader: VariableLoader = DUMMY_VARIABLE_LOADER,
response_stream_filter: ResponseStreamFilter | None = None,
) -> Mapping[str, Any] | Generator[str | Mapping[str, Any], None, None]:
"""
Resume a paused workflow execution using the persisted runtime state.
@@ -314,7 +310,6 @@ class WorkflowAppGenerator(BaseAppGenerator):
graph_engine_layers=graph_engine_layers,
graph_runtime_state=graph_runtime_state,
pause_state_config=pause_state_config,
response_stream_filter=response_stream_filter,
)
def _generate(
@@ -333,7 +328,6 @@ class WorkflowAppGenerator(BaseAppGenerator):
graph_engine_layers: Sequence[GraphEngineLayer] = (),
graph_runtime_state: GraphRuntimeState | None = None,
pause_state_config: PauseStateLayerConfig | None = None,
response_stream_filter: ResponseStreamFilter | None = None,
) -> Mapping[str, Any] | Generator[str | Mapping[str, Any], None, None]:
"""
Generate App response.
@@ -362,14 +356,12 @@ class WorkflowAppGenerator(BaseAppGenerator):
app_mode=app_model.mode,
)
resolved_response_stream_filter = response_stream_filter or ResponseStreamFilter()
if pause_state_config is not None:
graph_layers.append(
PauseStatePersistenceLayer(
session_factory=pause_state_config.session_factory,
generate_entity=application_generate_entity,
state_owner_user_id=pause_state_config.state_owner_user_id,
response_stream_filter=resolved_response_stream_filter,
)
)
@@ -392,17 +384,12 @@ class WorkflowAppGenerator(BaseAppGenerator):
"workflow_node_execution_repository": workflow_node_execution_repository,
"graph_engine_layers": tuple(graph_layers),
"graph_runtime_state": graph_runtime_state,
"response_stream_filter": resolved_response_stream_filter,
},
)
worker_thread.start()
draft_var_saver_factory = self._get_draft_var_saver_factory(
invoke_from,
user,
tenant_id=app_model.tenant_id,
)
draft_var_saver_factory = self._get_draft_var_saver_factory(invoke_from, user)
# return response or stream generator
response = self._handle_response(
@@ -472,7 +459,6 @@ class WorkflowAppGenerator(BaseAppGenerator):
# Create workflow execution(aka workflow run) repository
workflow_execution_repository = DifyCoreRepositoryFactory.create_workflow_execution_repository(
session_factory=session_factory,
tenant_id=app_model.tenant_id,
user=user,
app_id=application_generate_entity.app_config.app_id,
triggered_from=WorkflowRunTriggeredFrom.DEBUGGING,
@@ -480,7 +466,6 @@ class WorkflowAppGenerator(BaseAppGenerator):
# Create workflow node execution repository
workflow_node_execution_repository = DifyCoreRepositoryFactory.create_workflow_node_execution_repository(
session_factory=session_factory,
tenant_id=app_model.tenant_id,
user=user,
app_id=application_generate_entity.app_config.app_id,
triggered_from=WorkflowNodeExecutionTriggeredFrom.SINGLE_STEP,
@@ -561,7 +546,6 @@ class WorkflowAppGenerator(BaseAppGenerator):
# Create workflow execution(aka workflow run) repository
workflow_execution_repository = DifyCoreRepositoryFactory.create_workflow_execution_repository(
session_factory=session_factory,
tenant_id=app_model.tenant_id,
user=user,
app_id=application_generate_entity.app_config.app_id,
triggered_from=WorkflowRunTriggeredFrom.DEBUGGING,
@@ -569,7 +553,6 @@ class WorkflowAppGenerator(BaseAppGenerator):
# Create workflow node execution repository
workflow_node_execution_repository = DifyCoreRepositoryFactory.create_workflow_node_execution_repository(
session_factory=session_factory,
tenant_id=app_model.tenant_id,
user=user,
app_id=application_generate_entity.app_config.app_id,
triggered_from=WorkflowNodeExecutionTriggeredFrom.SINGLE_STEP,
@@ -607,7 +590,6 @@ class WorkflowAppGenerator(BaseAppGenerator):
root_node_id: str | None = None,
graph_engine_layers: Sequence[GraphEngineLayer] = (),
graph_runtime_state: GraphRuntimeState | None = None,
response_stream_filter: ResponseStreamFilter | None = None,
) -> None:
"""
Generate worker in a new thread.
@@ -656,7 +638,6 @@ class WorkflowAppGenerator(BaseAppGenerator):
root_node_id=root_node_id,
graph_engine_layers=graph_engine_layers,
graph_runtime_state=graph_runtime_state,
response_stream_filter=response_stream_filter,
)
try:
-4
View File
@@ -18,7 +18,6 @@ from core.workflow.workflow_entry import WorkflowEntry
from extensions.ext_redis import redis_client
from extensions.otel import WorkflowAppRunnerHandler, trace_span
from graphon.enums import WorkflowType
from graphon.filters import ResponseStreamFilter
from graphon.graph_engine.command_channels import RedisChannel
from graphon.graph_engine.layers import GraphEngineLayer
from graphon.runtime import GraphRuntimeState, VariablePool
@@ -47,7 +46,6 @@ class WorkflowAppRunner(WorkflowBasedAppRunner):
workflow_node_execution_repository: WorkflowNodeExecutionRepository,
graph_engine_layers: Sequence[GraphEngineLayer] = (),
graph_runtime_state: GraphRuntimeState | None = None,
response_stream_filter: ResponseStreamFilter | None = None,
):
super().__init__(
queue_manager=queue_manager,
@@ -62,7 +60,6 @@ class WorkflowAppRunner(WorkflowBasedAppRunner):
self._workflow_execution_repository = workflow_execution_repository
self._workflow_node_execution_repository = workflow_node_execution_repository
self._resume_graph_runtime_state = graph_runtime_state
self._response_stream_filter = response_stream_filter
@trace_span(WorkflowAppRunnerHandler)
def run(self):
@@ -166,7 +163,6 @@ class WorkflowAppRunner(WorkflowBasedAppRunner):
variable_pool=variable_pool,
graph_runtime_state=graph_runtime_state,
command_channel=command_channel,
response_stream_filter=self._response_stream_filter,
)
persistence_layer = WorkflowPersistenceLayer(
@@ -7,7 +7,6 @@ from sqlalchemy.orm import Session, sessionmaker
from core.app.entities.app_invoke_entities import AdvancedChatAppGenerateEntity, WorkflowAppGenerateEntity
from core.workflow.system_variables import SystemVariableKey, get_system_text
from graphon.filters import ResponseStreamFilter
from graphon.graph_engine.layers import GraphEngineLayer
from graphon.graph_events import GraphEngineEvent, GraphRunPausedEvent
from models.model import AppMode
@@ -42,10 +41,6 @@ class WorkflowResumptionContext(BaseModel):
# Only workflow / chatflow could be paused.
generate_entity: _GenerateEntityUnion
serialized_graph_runtime_state: str
# Optional so that a workflow run paused before this field existed still
# loads: it just degrades to fresh-filter behavior on resume for that one
# stale run.
serialized_response_stream_filter_state: str | None = None
def dumps(self) -> str:
return self.model_dump_json()
@@ -57,12 +52,6 @@ class WorkflowResumptionContext(BaseModel):
def get_generate_entity(self) -> WorkflowAppGenerateEntity | AdvancedChatAppGenerateEntity:
return self.generate_entity.entity
def get_response_stream_filter(self) -> ResponseStreamFilter:
response_stream_filter = ResponseStreamFilter()
if self.serialized_response_stream_filter_state is not None:
response_stream_filter.loads(self.serialized_response_stream_filter_state)
return response_stream_filter
@dataclass(frozen=True)
class PauseStateLayerConfig:
@@ -78,17 +67,11 @@ class PauseStatePersistenceLayer(GraphEngineLayer):
session_factory: Engine | sessionmaker[Session],
generate_entity: WorkflowAppGenerateEntity | AdvancedChatAppGenerateEntity,
state_owner_user_id: str,
response_stream_filter: ResponseStreamFilter,
):
"""Create a PauseStatePersistenceLayer.
The `state_owner_user_id` is used when creating state file for pause.
It generally should id of the creator of workflow.
`response_stream_filter` must be the exact same instance that
`WorkflowEntry` is using to stream this run's events — this layer
dumps its state on pause, and a different instance would silently
persist the wrong (empty) filter state.
"""
if isinstance(session_factory, Engine):
session_factory = sessionmaker(session_factory)
@@ -96,7 +79,6 @@ class PauseStatePersistenceLayer(GraphEngineLayer):
self._session_maker = session_factory
self._state_owner_user_id = state_owner_user_id
self._generate_entity = generate_entity
self._response_stream_filter = response_stream_filter
def _get_repo(self) -> APIWorkflowRunRepository:
return DifyAPIRepositoryFactory.create_api_workflow_run_repository(self._session_maker)
@@ -137,7 +119,6 @@ class PauseStatePersistenceLayer(GraphEngineLayer):
state = WorkflowResumptionContext(
serialized_graph_runtime_state=self.graph_runtime_state.dumps(),
generate_entity=entity_wrapper,
serialized_response_stream_filter_state=self._response_stream_filter.dumps(),
)
workflow_run_id = get_system_text(
@@ -1,7 +1,6 @@
from collections.abc import Generator, Iterable, Mapping
from typing import Any
from configs import dify_config
from core.callback_handler.agent_tool_callback_handler import DifyAgentCallbackHandler, print_text
from core.ops.ops_trace_manager import TraceQueueManager
from core.tools.entities.tool_entities import ToolInvokeMessage
@@ -20,9 +19,8 @@ class DifyWorkflowCallbackHandler(DifyAgentCallbackHandler):
trace_manager: TraceQueueManager | None = None,
) -> Generator[ToolInvokeMessage, None, None]:
for tool_output in tool_outputs:
if dify_config.DEBUG:
print_text("\n[on_tool_execution]\n", color=self.color)
print_text("Tool: " + tool_name + "\n", color=self.color)
print_text("Outputs: " + tool_output.model_dump_json()[:1000] + "\n", color=self.color)
print_text("\n")
print_text("\n[on_tool_execution]\n", color=self.color)
print_text("Tool: " + tool_name + "\n", color=self.color)
print_text("Outputs: " + tool_output.model_dump_json()[:1000] + "\n", color=self.color)
print_text("\n")
yield tool_output
+114 -15
View File
@@ -22,7 +22,10 @@ from core.entities.provider_entities import (
SystemConfigurationStatus,
)
from core.helper import encrypter
from core.helper.model_provider_cache import ProviderCredentialsCache, ProviderCredentialsCacheType
from core.helper.model_provider_cache import (
ProviderCredentialsCache,
ProviderCredentialsCacheType,
)
from core.plugin.impl.model_runtime_factory import create_model_type_instance, create_plugin_model_assembly
from graphon.model_runtime.entities.model_entities import AIModelEntity, FetchFrom, ModelType
from graphon.model_runtime.entities.provider_entities import (
@@ -473,6 +476,39 @@ class ProviderConfiguration(BaseModel):
provider_names.append(model_provider_id.provider_name)
return provider_names
def _invalidate_provider_configuration_cache(
self,
*,
provider_models: bool = False,
preferred_model_providers: bool = False,
provider_model_settings: bool = False,
provider_model_credentials: bool = False,
provider_credentials: bool = False,
provider_load_balancing_configs: bool = False,
) -> None:
"""Invalidate tenant-scoped provider snapshots after committing configuration writes."""
from core.provider_manager import ProviderConfigurationCacheSource, ProviderManager
sources: list[ProviderConfigurationCacheSource] = []
if provider_models:
sources.append(ProviderConfigurationCacheSource.PROVIDER_MODELS)
if preferred_model_providers:
sources.append(ProviderConfigurationCacheSource.PREFERRED_MODEL_PROVIDERS)
if provider_model_settings:
sources.append(ProviderConfigurationCacheSource.PROVIDER_MODEL_SETTINGS)
if provider_model_credentials:
sources.append(ProviderConfigurationCacheSource.PROVIDER_MODEL_CREDENTIALS)
if provider_credentials:
sources.append(ProviderConfigurationCacheSource.PROVIDER_CREDENTIALS)
if provider_load_balancing_configs:
sources.append(ProviderConfigurationCacheSource.PROVIDER_LOAD_BALANCING_CONFIGS)
if not sources:
logger.warning("No provider configuration cache source selected for invalidation")
return
ProviderManager.invalidate_configurations_cache(self.tenant_id, sources=sources)
def create_provider_credential(self, credentials: dict[str, Any], credential_name: str | None):
"""
Add custom provider credentials.
@@ -489,6 +525,7 @@ class ProviderConfiguration(BaseModel):
credentials = self.validate_provider_credentials(credentials=credentials)
preferred_model_providers_changed = False
with Session(db.engine) as session:
provider_record = self._get_provider_record(session)
try:
@@ -518,7 +555,9 @@ class ProviderConfiguration(BaseModel):
)
provider_model_credentials_cache.delete()
self.switch_preferred_provider_type(provider_type=ProviderType.CUSTOM, session=session)
preferred_model_providers_changed = self.switch_preferred_provider_type(
provider_type=ProviderType.CUSTOM, session=session
)
else:
provider_record.is_valid = True
@@ -533,12 +572,18 @@ class ProviderConfiguration(BaseModel):
)
provider_model_credentials_cache.delete()
self.switch_preferred_provider_type(provider_type=ProviderType.CUSTOM, session=session)
preferred_model_providers_changed = self.switch_preferred_provider_type(
provider_type=ProviderType.CUSTOM, session=session
)
session.commit()
except Exception:
session.rollback()
raise
self._invalidate_provider_configuration_cache(
preferred_model_providers=preferred_model_providers_changed,
provider_credentials=True,
)
def update_provider_credential(
self,
@@ -562,6 +607,7 @@ class ProviderConfiguration(BaseModel):
credentials = self.validate_provider_credentials(credentials=credentials, credential_id=credential_id)
load_balancing_configs_changed = False
with Session(db.engine) as session:
provider_record = self._get_provider_record(session)
stmt = select(ProviderCredential).where(
@@ -588,7 +634,7 @@ class ProviderConfiguration(BaseModel):
)
provider_model_credentials_cache.delete()
self._update_load_balancing_configs_with_credential(
load_balancing_configs_changed = self._update_load_balancing_configs_with_credential(
credential_id=credential_id,
credential_record=credential_record,
credential_source=CredentialSourceType.PROVIDER,
@@ -597,6 +643,10 @@ class ProviderConfiguration(BaseModel):
except Exception:
session.rollback()
raise
self._invalidate_provider_configuration_cache(
provider_credentials=True,
provider_load_balancing_configs=load_balancing_configs_changed,
)
def _update_load_balancing_configs_with_credential(
self,
@@ -604,7 +654,7 @@ class ProviderConfiguration(BaseModel):
credential_record: ProviderCredential | ProviderModelCredential,
credential_source: str,
session: Session,
):
) -> bool:
"""
Update load balancing configurations that reference the given credential_id.
@@ -625,7 +675,7 @@ class ProviderConfiguration(BaseModel):
load_balancing_configs = session.execute(stmt).scalars().all()
if not load_balancing_configs:
return
return False
# Update each load balancing config with the new credentials
for lb_config in load_balancing_configs:
@@ -643,6 +693,7 @@ class ProviderConfiguration(BaseModel):
lb_credentials_cache.delete()
session.commit()
return True
def delete_provider_credential(self, credential_id: str):
"""
@@ -651,6 +702,8 @@ class ProviderConfiguration(BaseModel):
:param credential_id: credential id
:return:
"""
preferred_model_providers_changed = False
load_balancing_configs_changed = False
with Session(db.engine) as session:
stmt = select(ProviderCredential).where(
ProviderCredential.id == credential_id,
@@ -671,6 +724,7 @@ class ProviderConfiguration(BaseModel):
LoadBalancingModelConfig.credential_source_type == CredentialSourceType.PROVIDER,
)
lb_configs_using_credential = session.execute(lb_stmt).scalars().all()
load_balancing_configs_changed = bool(lb_configs_using_credential)
try:
for lb_config in lb_configs_using_credential:
lb_credentials_cache = ProviderCredentialsCache(
@@ -703,7 +757,9 @@ class ProviderConfiguration(BaseModel):
cache_type=ProviderCredentialsCacheType.PROVIDER,
)
provider_model_credentials_cache.delete()
self.switch_preferred_provider_type(provider_type=ProviderType.SYSTEM, session=session)
preferred_model_providers_changed = self.switch_preferred_provider_type(
provider_type=ProviderType.SYSTEM, session=session
)
elif provider_record and provider_record.credential_id == credential_id:
provider_record.credential_id = None
provider_record.updated_at = naive_utc_now()
@@ -714,12 +770,19 @@ class ProviderConfiguration(BaseModel):
cache_type=ProviderCredentialsCacheType.PROVIDER,
)
provider_model_credentials_cache.delete()
self.switch_preferred_provider_type(provider_type=ProviderType.SYSTEM, session=session)
preferred_model_providers_changed = self.switch_preferred_provider_type(
provider_type=ProviderType.SYSTEM, session=session
)
session.commit()
except Exception:
session.rollback()
raise
self._invalidate_provider_configuration_cache(
preferred_model_providers=preferred_model_providers_changed,
provider_credentials=True,
provider_load_balancing_configs=load_balancing_configs_changed,
)
def switch_active_provider_credential(self, credential_id: str):
"""
@@ -728,6 +791,7 @@ class ProviderConfiguration(BaseModel):
:param credential_id: credential id
:return:
"""
preferred_model_providers_changed = False
with Session(db.engine) as session:
stmt = select(ProviderCredential).where(
ProviderCredential.id == credential_id,
@@ -753,10 +817,14 @@ class ProviderConfiguration(BaseModel):
cache_type=ProviderCredentialsCacheType.PROVIDER,
)
provider_model_credentials_cache.delete()
self.switch_preferred_provider_type(ProviderType.CUSTOM, session=session)
preferred_model_providers_changed = self.switch_preferred_provider_type(
ProviderType.CUSTOM, session=session
)
except Exception:
session.rollback()
raise
if preferred_model_providers_changed:
self._invalidate_provider_configuration_cache(preferred_model_providers=True)
def _get_custom_model_record(
self,
@@ -1017,6 +1085,10 @@ class ProviderConfiguration(BaseModel):
except Exception:
session.rollback()
raise
self._invalidate_provider_configuration_cache(
provider_models=True,
provider_model_credentials=True,
)
def update_custom_model_credential(
self,
@@ -1053,6 +1125,7 @@ class ProviderConfiguration(BaseModel):
credential_id=credential_id,
)
load_balancing_configs_changed = False
with Session(db.engine) as session:
provider_model_record = self._get_custom_model_record(model_type=model_type, model=model, session=session)
@@ -1082,7 +1155,7 @@ class ProviderConfiguration(BaseModel):
)
provider_model_credentials_cache.delete()
self._update_load_balancing_configs_with_credential(
load_balancing_configs_changed = self._update_load_balancing_configs_with_credential(
credential_id=credential_id,
credential_record=credential_record,
credential_source=CredentialSourceType.CUSTOM_MODEL,
@@ -1091,6 +1164,11 @@ class ProviderConfiguration(BaseModel):
except Exception:
session.rollback()
raise
self._invalidate_provider_configuration_cache(
provider_models=True,
provider_model_credentials=True,
provider_load_balancing_configs=load_balancing_configs_changed,
)
def delete_custom_model_credential(self, model_type: ModelType, model: str, credential_id: str):
"""
@@ -1099,6 +1177,7 @@ class ProviderConfiguration(BaseModel):
:param credential_id: credential id
:return:
"""
load_balancing_configs_changed = False
with Session(db.engine) as session:
stmt = select(ProviderModelCredential).where(
ProviderModelCredential.id == credential_id,
@@ -1118,6 +1197,7 @@ class ProviderConfiguration(BaseModel):
LoadBalancingModelConfig.credential_source_type == CredentialSourceType.CUSTOM_MODEL,
)
lb_configs_using_credential = session.execute(lb_stmt).scalars().all()
load_balancing_configs_changed = bool(lb_configs_using_credential)
try:
for lb_config in lb_configs_using_credential:
@@ -1161,6 +1241,11 @@ class ProviderConfiguration(BaseModel):
except Exception:
session.rollback()
raise
self._invalidate_provider_configuration_cache(
provider_models=True,
provider_model_credentials=True,
provider_load_balancing_configs=load_balancing_configs_changed,
)
def add_model_credential_to_model(self, model_type: ModelType, model: str, credential_id: str):
"""
@@ -1213,6 +1298,7 @@ class ProviderConfiguration(BaseModel):
session.add(provider_model_record)
session.commit()
self._invalidate_provider_configuration_cache(provider_models=True)
def switch_custom_model_credential(self, model_type: ModelType, model: str, credential_id: str):
"""
@@ -1251,6 +1337,7 @@ class ProviderConfiguration(BaseModel):
cache_type=ProviderCredentialsCacheType.MODEL,
)
provider_model_credentials_cache.delete()
self._invalidate_provider_configuration_cache(provider_models=True)
def delete_custom_model(self, model_type: ModelType, model: str):
"""
@@ -1259,6 +1346,7 @@ class ProviderConfiguration(BaseModel):
:param model: model name
:return:
"""
provider_models_changed = False
with Session(db.engine) as session:
# get provider model
provider_model_record = self._get_custom_model_record(model_type=model_type, model=model, session=session)
@@ -1267,6 +1355,7 @@ class ProviderConfiguration(BaseModel):
if provider_model_record:
session.delete(provider_model_record)
session.commit()
provider_models_changed = True
provider_model_credentials_cache = ProviderCredentialsCache(
tenant_id=self.tenant_id,
@@ -1275,6 +1364,8 @@ class ProviderConfiguration(BaseModel):
)
provider_model_credentials_cache.delete()
if provider_models_changed:
self._invalidate_provider_configuration_cache(provider_models=True)
def _get_provider_model_setting(
self, model_type: ModelType, model: str, session: Session
@@ -1314,6 +1405,7 @@ class ProviderConfiguration(BaseModel):
)
session.add(model_setting)
session.commit()
self._invalidate_provider_configuration_cache(provider_model_settings=True)
return model_setting
@@ -1340,6 +1432,7 @@ class ProviderConfiguration(BaseModel):
)
session.add(model_setting)
session.commit()
self._invalidate_provider_configuration_cache(provider_model_settings=True)
return model_setting
@@ -1392,6 +1485,7 @@ class ProviderConfiguration(BaseModel):
)
session.add(model_setting)
session.commit()
self._invalidate_provider_configuration_cache(provider_model_settings=True)
return model_setting
@@ -1419,6 +1513,7 @@ class ProviderConfiguration(BaseModel):
)
session.add(model_setting)
session.commit()
self._invalidate_provider_configuration_cache(provider_model_settings=True)
return model_setting
@@ -1454,19 +1549,19 @@ class ProviderConfiguration(BaseModel):
credentials=credentials or {},
)
def switch_preferred_provider_type(self, provider_type: ProviderType, session: Session | None = None):
def switch_preferred_provider_type(self, provider_type: ProviderType, session: Session | None = None) -> bool:
"""
Switch preferred provider type.
:param provider_type:
:return:
"""
if provider_type == self.preferred_provider_type:
return
return False
if provider_type == ProviderType.SYSTEM and not self.system_configuration.enabled:
return
return False
def _switch(s: Session):
def _switch(s: Session) -> bool:
stmt = select(TenantPreferredModelProvider).where(
TenantPreferredModelProvider.tenant_id == self.tenant_id,
TenantPreferredModelProvider.provider_name.in_(self._get_provider_names()),
@@ -1483,12 +1578,16 @@ class ProviderConfiguration(BaseModel):
)
s.add(preferred_model_provider)
s.commit()
return True
if session:
return _switch(session)
else:
with Session(db.engine) as session:
return _switch(session)
changed = _switch(session)
if changed:
self._invalidate_provider_configuration_cache(preferred_model_providers=True)
return changed
def extract_secret_variables(self, credential_form_schemas: list[CredentialFormSchema]) -> list[str]:
"""
@@ -13,7 +13,7 @@ from core.helper.code_executor.jinja2.jinja2_transformer import Jinja2TemplateTr
from core.helper.code_executor.python3.python3_transformer import Python3TemplateTransformer
from core.helper.code_executor.template_transformer import TemplateTransformer
from core.helper.http_client_pooling import get_pooled_http_client
from graphon.nodes.code.entities import CodeLanguage as CodeLanguage # noqa: PLC0414
from graphon.nodes.code.entities import CodeLanguage
logger = logging.getLogger(__name__)
code_execution_endpoint_url = URL(str(dify_config.CODE_EXECUTION_ENDPOINT))
@@ -133,9 +133,7 @@ class CodeExecutor:
return response_code.data.stdout or ""
@classmethod
def execute_workflow_code_template(
cls, language: CodeLanguage, code: str, inputs: Mapping[str, Any]
) -> dict[str, Any]:
def execute_workflow_code_template(cls, language: CodeLanguage, code: str, inputs: Mapping[str, Any]):
"""
Execute code
:param language: code language
@@ -11,7 +11,7 @@ class Jinja2TemplateTransformer(TemplateTransformer):
@classmethod
@override
def transform_response(cls, response: str) -> dict[str, Any]:
def transform_response(cls, response: str):
"""
Transform response to dict
:param response: response
@@ -36,14 +36,14 @@ class TemplateTransformer(ABC):
return runner_script, preload_script
@classmethod
def extract_result_str_from_response(cls, response: str) -> str:
def extract_result_str_from_response(cls, response: str):
result = re.search(rf"{cls._result_tag}(.*){cls._result_tag}", response, re.DOTALL)
if not result:
raise ValueError(f"Failed to parse result: no result tag found in response. Response: {response[:200]}...")
return result.group(1)
@classmethod
def transform_response(cls, response: str) -> dict[str, Any]:
def transform_response(cls, response: str) -> Mapping[str, Any]:
"""
Transform response to dict
:param response: response
@@ -71,7 +71,7 @@ class TemplateTransformer(ABC):
return result
@classmethod
def _post_process_result(cls, result: dict[str, Any]) -> dict[str, Any]:
def _post_process_result(cls, result: dict[Any, Any]) -> dict[Any, Any]:
"""
Post-process the result to convert scientific notation strings back to numbers
"""
@@ -89,7 +89,7 @@ class TemplateTransformer(ABC):
return [convert_scientific_notation(v) for v in value]
return value
return {key: convert_scientific_notation(value) for key, value in result.items()}
return convert_scientific_notation(result)
@classmethod
@abstractmethod
+1 -1
View File
@@ -24,7 +24,7 @@ def upload_dsl(dsl_file_bytes: bytes, filename: str = "template.yaml") -> str:
response.raise_for_status()
data = response.json()
claim_code = data.get("data", {}).get("claim_code")
if not isinstance(claim_code, str) or not claim_code:
if not claim_code:
raise ValueError("Creators Platform did not return a valid claim_code")
return claim_code
+4 -7
View File
@@ -10,21 +10,18 @@ def is_credential_exists(credential_id: str, credential_type: "PluginCredentialT
"""
Check if the credential still exists in the database.
Uses the configured SQLAlchemy session factory instead of Flask-SQLAlchemy's
``db.engine`` because workflow graph node construction may run without an
active Flask application context.
:param credential_id: The credential ID to check
:param credential_type: The type of credential (MODEL or TOOL)
:return: True if credential exists, False otherwise
"""
from sqlalchemy import select
from sqlalchemy.orm import Session
from core.db import session_factory
from extensions.ext_database import db
from models.provider import ProviderCredential, ProviderModelCredential
from models.tools import BuiltinToolProvider
with session_factory.create_session() as session:
with Session(db.engine) as session:
if credential_type == PluginCredentialType.MODEL:
# Check both pre-defined and custom model credentials using a single UNION query
stmt = (
@@ -45,7 +42,7 @@ def is_credential_exists(credential_id: str, credential_type: "PluginCredentialT
def runtime_check_credential_policy_compliance(
credential_id: str, provider: str, credential_type: "PluginCredentialType", check_existence: bool = True
) -> None:
):
if dify_config.ENTERPRISE_DISABLE_RUNTIME_CREDENTIAL_CHECK:
return
check_credential_policy_compliance(
+1 -4
View File
@@ -1,7 +1,4 @@
from typing import Any
def download_with_size_limit(url: str, max_download_size: int, **kwargs: Any) -> bytes:
def download_with_size_limit(url, max_download_size: int, **kwargs):
from core.file import remote_fetcher
response = remote_fetcher.make_request("GET", url, follow_redirects=True, **kwargs)
+6 -8
View File
@@ -1,7 +1,5 @@
import base64
from Crypto.PublicKey import RSA
from libs import rsa
@@ -13,13 +11,13 @@ def obfuscated_token(token: str) -> str:
return token[:6] + "*" * 12 + token[-2:]
def full_mask_token(token_length: int = 20) -> str:
def full_mask_token(token_length=20):
return "*" * token_length
def encrypt_token(tenant_id: str, token: str) -> str:
def encrypt_token(tenant_id: str, token: str):
from extensions.ext_database import db
from models.account import Tenant
from models.engine import db
if not (tenant := db.session.get(Tenant, tenant_id)):
raise ValueError(f"Tenant with id {tenant_id} not found")
@@ -32,15 +30,15 @@ def decrypt_token(tenant_id: str, token: str) -> str:
return rsa.decrypt(base64.b64decode(token), tenant_id)
def batch_decrypt_token(tenant_id: str, tokens: list[str]) -> list[str]:
def batch_decrypt_token(tenant_id: str, tokens: list[str]):
rsa_key, cipher_rsa = rsa.get_decrypt_decoding(tenant_id)
return [rsa.decrypt_token_with_decoding(base64.b64decode(token), rsa_key, cipher_rsa) for token in tokens]
def get_decrypt_decoding(tenant_id: str) -> tuple[RSA.RsaKey, object]:
def get_decrypt_decoding(tenant_id: str):
return rsa.get_decrypt_decoding(tenant_id)
def decrypt_token_with_decoding(token: str, rsa_key: RSA.RsaKey, cipher_rsa: object) -> str:
def decrypt_token_with_decoding(token: str, rsa_key, cipher_rsa):
return rsa.decrypt_token_with_decoding(base64.b64decode(token), rsa_key, cipher_rsa)

Some files were not shown because too many files have changed in this diff Show More