Compare commits

..
Author SHA1 Message Date
GareArc bc68e02711 feat(openapi): distinguish expired OAuth bearer from invalid token
Previously an expired OAuth bearer and an unknown/invalid one both
surfaced as an indistinguishable generic 401 (and an invalid token
actually leaked a 500), so a client could not tell "session expired,
re-authenticate" apart from "never authenticated."

The resolver now raises a distinct TokenExpiredError for expired DB
rows and records a separate `expired` negative-cache marker, so a
retry within the negative-cache TTL still reports expiry instead of
collapsing into a generic miss. The auth pipeline maps the two domain
errors to unified OpenApiError responses: SessionExpired (code
`token_expired`) and InvalidBearer (code `unauthorized`), both 401.
This also fixes the latent 500 on invalid bearers.

The `token_expired` code is synced through the contract codegen into
the generated types/zod, and the difyctl error mapper branches the
401 on it. The CLI `expired_token` taxonomy member (RFC 8628
device-flow code expiry) is merged into `token_expired`; the RFC 8628
wire value is unchanged.

Closes WTA-1062
2026-06-28 21:45:13 -07:00
Escape0707andGitHub eb4ec93cea chore: migrate account role test to testcontainers (#38010) 2026-06-26 05:51:14 +00:00
Xiyuan ChenandGitHub fa6f4b0ea5 refactor(api): enforce enums over string literals on openapi surface (#38009) 2026-06-26 05:48:21 +00:00
267b34caaf fix(web): keep HITL input-field save button visible in the edit dialog (#38007)
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-26 05:40:51 +00:00
yyhandGitHub 6ab1dd06ac fix: isolate node selector keyboard events (#37998) 2026-06-26 05:38:10 +00:00
Stephen ZhouandGitHub 113d6d7e00 refactor: align deployment detail state ownership (#38008) 2026-06-26 05:37:21 +00:00
Mohammed AqibGitHubautofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
a66b1de477 refactor: Use pytest caplog instead of logger patching in test_spec.py (#37997)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-06-26 04:48:22 +00:00
Stephen ZhouandGitHub 16b698b54d refactor: simplify deployment access ownership (#37994) 2026-06-26 04:10:56 +00:00
yyhandGitHub 4cd8b8c733 fix(web): make marketplace URL builder SSR-safe (#37944) 2026-06-26 03:35:22 +00:00
677ab01806 refactor(tests): use caplog in workflow collaboration service tests (#37468) (#37991)
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-26 03:14:06 +00:00
Stephen ZhouandGitHub f4e832f35c fix: decouple deployment access control dialog (#37936) 2026-06-26 02:16:29 +00:00
kurokoboGitHubautofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
1c5e1280cb feat(webapp): display app description on chat and text-generation app screens (#37345)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-06-26 02:03:16 +00:00
-LAN-GitHubautofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
0035d90e36 fix(api): require edit access for trace config changes (#37973)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-06-26 00:51:39 +00:00
Xiyuan ChenandGitHub 2047e0dc12 fix(cli): use a(n) article in difyctl prerelease warning (#37976) 2026-06-26 00:49:42 +00:00
chaririandGitHub 2e9c0a3c7a refactor(api): migrate response contract tooling to BaseModel (#37950) 2026-06-26 00:11:04 +00:00
ojasarora.ethandGitHub a246dc8b17 ci: emit pyrefly diagnostics as GitHub workflow commands (#37974) 2026-06-25 18:27:46 +00:00
ojasarora.ethandGitHub bb921bcc45 test: replace logger mock with caplog in workflow collaboration test (#37971) 2026-06-25 16:45:59 +00:00
QuantumGhostandGitHub 4f4ac27de2 chore(api): surface pyrefly output on type-check failures (#37934) 2026-06-25 13:47:10 +00:00
Yunlu WenandGitHub 3aa26fb637 chore: bump versions to 1.15.0 (#37929) 2026-06-25 12:02:38 +00:00
Wu TianweiandGitHub 1d2cc1e475 feat: add hydration support and fallback for Countdown component (#37943) 2026-06-25 10:40:48 +00:00
林玮 (Jade Lin)andGitHub aa37c1d833 fix(api): tongyi credential compatibility (#37942) 2026-06-25 10:25:24 +00:00
yyhandGitHub 48dfbd60d6 fix: polish tool provider selection and detail drawer (#37940) 2026-06-25 10:07:32 +00:00
Mr_xieGitHubautofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
e90c7ab8a7 refactor(tests): replace logger mocks with caplog (#37468) (#37922)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-06-25 09:48:43 +00:00
wangxiaoleiGitHubautofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
40119fef44 feat: update monitor permission key and plugin permission key (#37937)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-06-25 09:41:38 +00:00
72a03c2d6a refactor(test): replace SimpleNamespace with real UploadFile model in test_dataset_models.py (#37935)
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-25 09:35:24 +00:00
Wu TianweiandGitHub affdc89f84 feat: enhance app permissions and access controls (#37933) 2026-06-25 09:28:58 +00:00
QuantumGhostandGitHub b33e8f0ddb fix: improve error handling for workflow execution (#37919) 2026-06-25 07:36:34 +00:00
FFXNGitHubgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>Copilot Autofix powered by AI
8f74e176ca fix: remove redundant snippets permissions (#37921)
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
2026-06-25 07:27:16 +00:00
github-actions[bot]GitHubclaude[bot] <41898282+claude[bot]@users.noreply.github.com>yyh
b9bcf31c72 chore(i18n): sync translations with en-US (#37916)
Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: yyh <92089059+lyzno1@users.noreply.github.com>
2026-06-25 07:07:23 +00:00
GatewayandGitHub abf2986299 build(deps): update Bleach sanitizer security fix (#37860) 2026-06-25 06:32:48 +00:00
Wu TianweiGitHubautofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
599d92ef6b fix: Fix frontend rbac issues (#37872)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-06-25 05:51:39 +00:00
JingyiandGitHub 93dd955deb docs: add Dify Cloud support contact (#37913) 2026-06-25 05:19:31 +00:00
github-actions[bot]GitHubclaude[bot] <41898282+claude[bot]@users.noreply.github.com>yyh
75909ce10e chore(i18n): sync translations with en-US (#37891)
Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: yyh <92089059+lyzno1@users.noreply.github.com>
2026-06-25 03:24:37 +00:00
Asuka MinatoGitHubautofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
d93989bfc0 chore: add type to test (#37876)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-06-25 03:01:09 +00:00
31a50a3b20 fix: eagerly validate conversation before generator to prevent hanging (#37224)
Co-authored-by: yunlu.wen <yunlu.wen@dify.ai>
2026-06-25 02:57:33 +00:00
Stephen ZhouandGitHub 3d8316333f fix: keep body background consistent on overscroll (#37909) 2026-06-25 02:20:39 +00:00
wangxiaoleiGitHubautofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
9fc2925b00 feat: delete member delete rbac binding (#37904)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-06-25 02:08:07 +00:00
JingyiandGitHub d349e892f4 fix: respect legacy plugin permissions without RBAC (#37903) 2026-06-25 01:18:40 +00:00
Willow LopezGitHubautofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>Asuka Minato
2483c091aa refactor: replace logger patches with pytest caplog in tests (#37890)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Asuka Minato <i@asukaminato.eu.org>
2026-06-24 21:40:06 +00:00
Stephen ZhouandGitHub a421362847 chore(web): prune unused i18n translations (#37888) 2026-06-24 16:13:22 +00:00
4964359961 refactor(tests): use caplog in web login tests (#37889)
Co-authored-by: lin-hongkuan <lin-hongkuan@users.noreply.github.com>
2026-06-24 14:44:01 +00:00
Stephen ZhouGitHubautofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
1b81ac033f chore(knip): add mdx support clean unused code (#37882)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-06-24 12:30:11 +00:00
JoelandGitHub 2eb564696e fix: banner has ui problem in small screen (#37879) 2026-06-24 11:19:33 +00:00
wangxiaoleiGitHubautofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
d87764b0f8 feat: filter dataset operator and add miss permission key (#37867)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-06-24 09:56:28 +00:00
JiangandGitHub d135dab241 fix: Lindorm vector store errors caused by the update of opensearch-py (#37862) 2026-06-24 09:54:06 +00:00
1033 changed files with 11470 additions and 37226 deletions
+6 -5
View File
@@ -102,11 +102,11 @@ describe('ComponentName', () => {
})
})
// Props tests (REQUIRED)
// Props tests (REQUIRED when props change observable behavior)
describe('Props', () => {
it('should apply custom className', () => {
render(<Component className="custom" />)
expect(screen.getByRole('button')).toHaveClass('custom')
it('should disable the action when disabled', () => {
render(<Component disabled />)
expect(screen.getByRole('button')).toBeDisabled()
})
})
@@ -220,6 +220,7 @@ Every test should clearly separate:
### 2. Black-Box Testing
- Test observable behavior, not implementation details
- Test product contracts, not cosmetic implementation. Do not add or expand unit tests only to lock pure style classes, spacing, colors, backgrounds, or layout micro-adjustments. Cover visual-only fixes with browser/manual verification, screenshots, or E2E/visual checks when risk justifies it. Add unit tests only when the change affects user-observable behavior, accessibility semantics, state, data flow, routing, or a stable component API contract.
- Use semantic queries (`getByRole` with accessible `name`, `getByLabelText`, `getByPlaceholderText`, `getByText`, and scoped `within(...)`)
- Treat `getByTestId` as a last resort. If a control cannot be found by role/name, label, landmark, or dialog scope, fix the component accessibility first instead of adding or relying on `data-testid`.
- Remove production `data-testid` attributes when semantic selectors can cover the behavior. Keep them only for non-visual mocked boundaries, editor/browser shims such as Monaco, canvas/chart output, or third-party widgets with no accessible DOM in the test environment.
@@ -273,7 +274,7 @@ it('should disable input when isReadOnly is true')
### Always Required (All Components)
1. **Rendering**: Component renders without crashing
1. **Props**: Required props, optional props, default values
1. **Props**: Required props, optional props, default values that change observable behavior. Do not test pass-through styling props such as `className` unless they are an explicit, stable component API whose absence would break a real integration contract.
1. **Edge Cases**: null, undefined, empty values, boundary conditions
### Conditional (When Present)
+10 -2
View File
@@ -12,6 +12,7 @@ Use this as the component decision guide for Dify web. Existing code is referenc
| Question | Default | Promote or extract only when |
| --- | --- | --- |
| Where should code live? | Keep it local to the feature workflow, route, or owner. | Multiple verticals need the same stable primitive. |
| How should route/tab folders be named? | Match the current route segment, tab name, or user-visible surface. | Keep a historical or broader parent only when it still owns multiple surfaces. |
| Who owns state, data, and handlers? | The lowest component that uses them. | A parent coordinates shared loading, errors, empty UI, selection, submission, navigation, or one consistent snapshot. |
| Should this become Jotai state? | Keep synchronous UI/form state in component or DOM state. | Siblings need one source of truth, the value drives atoms, or scoped workflow state must survive hidden/unmounted steps. |
| Should URL state enter Jotai? | Let Next.js route params and `nuqs` own URL state and updates. | Query atoms or shared derived atoms need a read-only bridge hydrated at the route/surface boundary. |
@@ -23,7 +24,7 @@ Use this as the component decision guide for Dify web. Existing code is referenc
- Search before adding UI, hooks, helpers, query utilities, or styling patterns. Reuse existing base components, feature components, hooks, utilities, and design styles when they fit.
- Follow Dify's CSS-first Tailwind v4 contract from `packages/dify-ui/README.md` and `packages/dify-ui/AGENTS.md`. Prefer design-system tokens, utilities, and radius mappings over generic Tailwind choices.
- Group feature code by workflow, route, or ownership area: components, hooks, local types, query helpers, atoms, constants, and small utilities should live near the code that changes with them.
- Group feature code by workflow, route, or ownership area with route-aligned names: components, hooks, local types, query helpers, atoms, constants, tests, and small utilities should live near the code that changes with them.
- Keep source/default selection, validation, dirty checks, and payload shaping close to the workflow that owns submit behavior. Do not hide flow-specific priority order, fallback behavior, or submit semantics in generic utilities.
- Prefer direct conditionals for small branch-specific decisions, especially form source selection and request payload assembly.
- Loading states for page sections, cards, lists, tables, forms, and drawers should be skeletons scoped to the content being loaded. Use spinners only for small inline busy indicators.
@@ -32,6 +33,8 @@ Use this as the component decision guide for Dify web. Existing code is referenc
- State-heavy wizards, drawers, modals, and secondary workflows can be a small feature surface: an entry file, one feature-local state file when Jotai is actually needed, and shallow `ui/` owners that match real visual regions.
- The entry file handles route integration, provider wiring, close behavior, and surface mounting. The composition owner handles high-level workflow branching. The closest visual owner handles section branching.
- When a page or tab maps to a route segment, name its feature folder after that route/tab surface instead of a stale parent grouping. Remove misleading intermediate folders when only one surface remains.
- When a tab folder grows into several independent sections or action areas, split the first level by product/visual owners. Keep the root for the entry component and cross-owner state, colocate tests with the owner folder, and put truly shared local UI under a specifically named `components/` file.
- Repeated TanStack query calls in sibling components are acceptable when each component independently consumes the data; TanStack Query deduplicates and shares cache.
- Pass stable domain identity across boundaries. Do not forward derived presentation state when the receiver can derive it from its own data source.
- A component that owns a visual surface should also own data access, loading, empty, and error states for content rendered inside it unless a parent truly coordinates that state.
@@ -46,6 +49,7 @@ Use this as the component decision guide for Dify web. Existing code is referenc
- Use uncontrolled `@langgenius/dify-ui/form` and `@langgenius/dify-ui/field` controls for edit/create forms whose fields are read only at submit time. Initialize query-backed defaults with `defaultValue` and keyed remounts.
- Promote form state to atoms only when another component must react to in-progress values, a draft must survive unmount/remount in the scoped workflow, or multiple steps share the same editable draft before submit.
- Treat `useParams`, route args, and `nuqs` query state as framework-owned state. When atom logic needs those values, hydrate primitive atoms at the route or surface boundary, such as with `useHydrateAtoms(..., { dangerouslyForceHydrate: true })`; keep URL updates in the route/query-state APIs instead of write atoms.
- Within a route-owned feature, choose one source for route identity. If route params are bridged into feature atoms, use that bridge consistently for route-derived queries and actions instead of also threading the same route id through page, tab, and section props.
- For async work tied to atom state, use `atomWithQuery` or `atomWithMutation`; write atoms should update only the inputs that drive those atoms. This applies to pure frontend async work as well as network requests, so do not hand-roll loading/error/in-flight state with `useState` or `useRef` for atom-orchestrated async behavior. For component-owned remote work, use `useQuery` or `useMutation` directly.
- Row-local async state belongs to the row owner unless it participates in a shared Jotai workflow or needs atom-scoped reset semantics.
- Leave query and mutation atoms unscoped so they keep shared QueryClient cache and invalidation behavior. Scope resettable primitives and explicit hydration tuples; scope a derived atom only when every dependency should be private to that surface.
@@ -60,8 +64,10 @@ Use this as the component decision guide for Dify web. Existing code is referenc
- Type component signatures directly; do not use `FC` or `React.FC`.
- Prefer `function` for top-level components and module helpers. Use arrow functions for local callbacks, handlers, and lambda-style APIs.
- Prefer named exports. Use default exports only where the framework requires them, such as Next.js route files.
- Avoid barrel files that only re-export secondary owners. `index.tsx` is acceptable for a route/tab entry component; import header controls, switches, sections, and row owners from their concrete owner files.
- Type simple one-off props inline. Use a named `Props` type only when reused, exported, complex, or clearer.
- Use API-generated or API-returned types at component boundaries. Keep small UI conversion helpers and one-off UI extensions beside the component that needs them.
- Avoid `common.tsx` buckets for shared UI. Use a feature-local `components/` folder with concrete filenames that describe the shared role.
- Do not create type aliases that only rename another type. Use aliases only for real UI concepts, refinements, or reusable local contracts.
- Name values by their domain role and backend API contract, especially persistent IDs and route params. Normalize framework or route params at the boundary.
- Put fallback and invariant checks in the lowest component that already handles that state. Do not extract helpers whose only behavior is hiding missing display data.
@@ -83,11 +89,13 @@ Use this as the component decision guide for Dify web. Existing code is referenc
- Keep `web/contract/*` as the API shape source of truth and follow the `{ params, query?, body? }` input shape.
- Consume generated queries with `useQuery(consoleQuery.xxx.queryOptions(...))` or `useQuery(marketplaceQuery.xxx.queryOptions(...))`.
- If a generated query input comes from an atom, including a route-identity bridge atom, keep the query in `atomWithQuery`; do not unwrap the atom in a component just to call `useQuery`.
- Consume owner-local mutations with `useMutation(consoleQuery.xxx.mutationOptions(...))` or `useMutation(marketplaceQuery.xxx.mutationOptions(...))` when pending/error state is not consumed by feature atoms.
- In `atomWithQuery`, `atomWithInfiniteQuery`, and `atomWithMutation`, return generated `queryOptions()`, `infiniteOptions()`, or `mutationOptions()` directly. Pass `enabled`, `retry`, `placeholderData`, `select`, and pagination options into the generated call instead of spreading options into a hand-built object.
- For generated oRPC options with missing required input, branch the whole input with `input: condition ? validInput : skipToken` and `enabled: Boolean(condition)`. Never place `skipToken` inside a nested placeholder payload or coerce required IDs to `''`.
- When prefetch and render use the same request, extract local query options or a query-options atom so `prefetchQuery` and `useQuery`/`atomWithQuery` share the exact options.
- For custom query or mutation functions, wrap options with TanStack `queryOptions(...)` or `mutationOptions(...)`.
- Do not extract generated `queryOptions(...)` into a helper solely to share input construction; extract only when prefetch/render must share exact options or the helper owns real domain behavior.
- Avoid pass-through hooks and thin `web/service/use-*` wrappers that only rename generated options. Keep feature hooks for real orchestration, workflow state, or shared domain behavior.
- Put shared cache behavior in `createTanstackQueryUtils(...experimental_defaults...)`. Component or atom callbacks may handle local toasts, closing dialogs, and navigation, but should not replace shared invalidation or patch shared server state locally.
- For overlays that may open heavier secondary content, prefetch from the trigger/menu open event with `queryClient.prefetchQuery(queryOptions)` when `onOpenChange` is available. Do not mount hidden subscribers just to warm cache.
@@ -96,7 +104,7 @@ Use this as the component decision guide for Dify web. Existing code is referenc
## Boundaries And Overlays
- Use the first level below a page or tab to organize independent page sections when it adds structure. This layer is layout/semantic first, not automatically the data owner.
- Use the first level below a page or tab to organize independent page sections when it adds structure or the root folder becomes noisy. This layer is layout/semantic first, not automatically the data owner.
- Treat component names, semantic roles, and user- or design-marked visual regions as boundary constraints. Keep adjacent UI as a sibling owner or introduce a correctly named broader owner.
- Keep cohesive forms, menu bodies, and one-off helpers local unless they need their own state, reuse, or semantic boundary.
- Separate hidden secondary surfaces from the trigger's main flow. For dialogs, dropdowns, popovers, and similar branches, extract a small local component when hidden content would obscure the parent.
+6
View File
@@ -53,6 +53,8 @@ jobs:
- name: Run Type Checks
if: steps.changed-files.outputs.any_changed == 'true'
env:
PYREFLY_OUTPUT_FORMAT: github
run: make type-check-core
- name: Dotenv check
@@ -109,6 +111,10 @@ jobs:
if: steps.changed-files.outputs.any_changed == 'true'
run: vp run knip:production
- name: Web production unused declarations check
if: steps.changed-files.outputs.any_changed == 'true'
run: vp run knip:production-unused-check
ts-common-style:
name: TS Common
runs-on: depot-ubuntu-24.04
+1 -1
View File
@@ -116,7 +116,7 @@ All of Dify's offerings come with corresponding APIs, so you could effortlessly
## Using Dify
- **Cloud <br/>**
We host a [Dify Cloud](https://dify.ai) service for anyone to try with zero setup. It provides all the capabilities of the self-deployed version, and includes 200 free GPT-4 calls in the sandbox plan.
We host a [Dify Cloud](https://dify.ai) service for anyone to try with zero setup. It provides all the capabilities of the self-deployed version, and includes 200 free GPT-4 calls in the sandbox plan. If you run into issues with Dify Cloud, [contact our Cloud support team](mailto:cloud@dify.ai?subject=%5BGitHub%5DDify%20Cloud%20Support).
- **Self-hosting Dify Community Edition<br/>**
Quickly get Dify running in your environment with this [starter guide](#quick-start).
+2 -2
View File
@@ -312,7 +312,7 @@ class AgentBackendRunRequestBuilder:
)
)
if run_input.knowledge is not None and run_input.knowledge.sets:
if run_input.knowledge is not None and run_input.knowledge.dataset_ids:
layers.append(
RunLayerSpec(
name=DIFY_KNOWLEDGE_BASE_LAYER_ID,
@@ -513,7 +513,7 @@ class AgentBackendRunRequestBuilder:
)
)
if run_input.knowledge is not None and run_input.knowledge.sets:
if run_input.knowledge is not None and run_input.knowledge.dataset_ids:
layers.append(
RunLayerSpec(
name=DIFY_KNOWLEDGE_BASE_LAYER_ID,
@@ -137,7 +137,6 @@ class WorkflowAgentComposerValidateApi(Resource):
def post(self, tenant_id: str, app_model: App, node_id: str):
payload = ComposerSavePayload.model_validate(console_ns.payload or {})
ComposerConfigValidator.validate_publish_payload(payload)
AgentComposerService.validate_knowledge_datasets(tenant_id=tenant_id, agent_soul=payload.agent_soul)
findings = AgentComposerService.collect_validation_findings(
tenant_id=tenant_id,
payload=payload,
@@ -272,7 +271,6 @@ class AgentComposerValidateApi(Resource):
_resolve_agent_app_id(tenant_id=tenant_id, agent_id=agent_id)
payload = ComposerSavePayload.model_validate(console_ns.payload or {})
ComposerConfigValidator.validate_publish_payload(payload)
AgentComposerService.validate_knowledge_datasets(tenant_id=tenant_id, agent_soul=payload.agent_soul)
findings = AgentComposerService.collect_validation_findings(
tenant_id=tenant_id,
payload=payload,
+1 -145
View File
@@ -56,7 +56,6 @@ from libs.login import login_required
from models import Account
from models.enums import ApiTokenType
from models.model import ApiToken, App, IconType
from services.agent.composer_service import AgentComposerService
from services.agent.errors import AgentNotFoundError
from services.agent.observability_service import (
AgentLogQueryParams,
@@ -66,7 +65,7 @@ from services.agent.observability_service import (
from services.agent.roster_service import AgentRosterService
from services.app_service import AppListParams, AppService, CreateAppParams
from services.enterprise.enterprise_service import EnterpriseService
from services.entities.agent_entities import ComposerSavePayload, RosterListQuery
from services.entities.agent_entities import RosterListQuery
from services.feature_service import FeatureService
@@ -251,36 +250,6 @@ class AgentDebugConversationRefreshResponse(BaseModel):
debug_conversation_id: str
class AgentPublishPayload(BaseModel):
version_note: str | None = Field(default=None, description="Optional note for this published Agent version")
class AgentPublishResponse(BaseModel):
result: str
active_config_snapshot_id: str
active_config_snapshot: dict[str, object] | None = None
draft: dict[str, object] | None = None
class AgentBuildDraftCheckoutPayload(BaseModel):
force: bool = Field(default=False, description="Overwrite the existing current-user build draft")
class AgentBuildDraftResponse(BaseModel):
variant: str
draft: dict[str, object]
agent_soul: dict[str, object]
class AgentBuildDraftApplyResponse(BaseModel):
result: str
draft: dict[str, object]
class AgentSimpleResultResponse(BaseModel):
result: str
class AgentAppPagination(GenericAppPagination):
data: list[AgentAppPartial] = Field( # type: ignore[assignment] # pyrefly: ignore[bad-override-mutable-attribute]
validation_alias=AliasChoices("items", "data")
@@ -292,9 +261,6 @@ register_schema_models(
AgentAppCreatePayload,
AgentAppUpdatePayload,
AgentAppCopyPayload,
AgentPublishPayload,
AgentBuildDraftCheckoutPayload,
ComposerSavePayload,
AgentApiStatusPayload,
AgentInviteOptionsQuery,
AgentLogsQuery,
@@ -311,10 +277,6 @@ register_response_schema_models(
AgentAppDetailWithSite,
AgentAppPartial,
AgentDebugConversationRefreshResponse,
AgentPublishResponse,
AgentBuildDraftResponse,
AgentBuildDraftApplyResponse,
AgentSimpleResultResponse,
AgentConfigSnapshotDetailResponse,
AgentConfigSnapshotListResponse,
AgentConfigSnapshotRestoreResponse,
@@ -621,112 +583,6 @@ class AgentDebugConversationRefreshApi(Resource):
)
@console_ns.route("/agent/<uuid:agent_id>/publish")
class AgentPublishApi(Resource):
@console_ns.expect(console_ns.models[AgentPublishPayload.__name__])
@console_ns.response(200, "Agent draft published", console_ns.models[AgentPublishResponse.__name__])
@console_ns.response(403, "Insufficient permissions")
@setup_required
@login_required
@account_initialization_required
@edit_permission_required
@with_current_user
@with_current_tenant_id
def post(self, tenant_id: str, current_user: Account, agent_id: UUID):
args = AgentPublishPayload.model_validate(console_ns.payload or {})
return AgentComposerService.publish_agent_app_draft(
tenant_id=tenant_id,
agent_id=str(agent_id),
account_id=current_user.id,
version_note=args.version_note,
)
@console_ns.route("/agent/<uuid:agent_id>/build-draft/checkout")
class AgentBuildDraftCheckoutApi(Resource):
@console_ns.expect(console_ns.models[AgentBuildDraftCheckoutPayload.__name__])
@console_ns.response(200, "Agent build draft checked out", console_ns.models[AgentBuildDraftResponse.__name__])
@setup_required
@login_required
@account_initialization_required
@edit_permission_required
@with_current_user
@with_current_tenant_id
def post(self, tenant_id: str, current_user: Account, agent_id: UUID):
args = AgentBuildDraftCheckoutPayload.model_validate(console_ns.payload or {})
return AgentComposerService.checkout_agent_app_build_draft(
tenant_id=tenant_id,
agent_id=str(agent_id),
account_id=current_user.id,
force=args.force,
)
@console_ns.route("/agent/<uuid:agent_id>/build-draft")
class AgentBuildDraftApi(Resource):
@console_ns.response(200, "Agent build draft", console_ns.models[AgentBuildDraftResponse.__name__])
@setup_required
@login_required
@account_initialization_required
@edit_permission_required
@with_current_user
@with_current_tenant_id
def get(self, tenant_id: str, current_user: Account, agent_id: UUID):
return AgentComposerService.load_agent_app_build_draft(
tenant_id=tenant_id,
agent_id=str(agent_id),
account_id=current_user.id,
)
@console_ns.expect(console_ns.models[ComposerSavePayload.__name__])
@console_ns.response(200, "Agent build draft saved", console_ns.models[AgentBuildDraftResponse.__name__])
@setup_required
@login_required
@account_initialization_required
@edit_permission_required
@with_current_user
@with_current_tenant_id
def put(self, tenant_id: str, current_user: Account, agent_id: UUID):
payload = ComposerSavePayload.model_validate(console_ns.payload or {})
return AgentComposerService.save_agent_app_build_draft(
tenant_id=tenant_id,
agent_id=str(agent_id),
account_id=current_user.id,
payload=payload,
)
@console_ns.response(200, "Agent build draft discarded", console_ns.models[AgentSimpleResultResponse.__name__])
@setup_required
@login_required
@account_initialization_required
@edit_permission_required
@with_current_user
@with_current_tenant_id
def delete(self, tenant_id: str, current_user: Account, agent_id: UUID):
return AgentComposerService.discard_agent_app_build_draft(
tenant_id=tenant_id,
agent_id=str(agent_id),
account_id=current_user.id,
)
@console_ns.route("/agent/<uuid:agent_id>/build-draft/apply")
class AgentBuildDraftApplyApi(Resource):
@console_ns.response(200, "Agent build draft applied", console_ns.models[AgentBuildDraftApplyResponse.__name__])
@setup_required
@login_required
@account_initialization_required
@edit_permission_required
@with_current_user
@with_current_tenant_id
def post(self, tenant_id: str, current_user: Account, agent_id: UUID):
return AgentComposerService.apply_agent_app_build_draft(
tenant_id=tenant_id,
agent_id=str(agent_id),
account_id=current_user.id,
)
@console_ns.route("/agent/<uuid:agent_id>/copy")
class AgentAppCopyApi(Resource):
@console_ns.expect(console_ns.models[AgentAppCopyPayload.__name__])
+17 -61
View File
@@ -38,7 +38,6 @@ from services.agent.skill_tool_inference_service import (
SkillToolInferenceResult,
SkillToolInferenceService,
)
from services.agent.soul_files_service import AgentSoulFilesService
from services.agent_drive_service import (
AgentDriveError,
AgentDriveService,
@@ -182,22 +181,6 @@ def _agent_not_bound() -> tuple[dict[str, str], int]:
return {"code": "agent_not_bound", "message": "no agent is bound for this app/node"}, 400
def _sync_active_soul_files(
*,
tenant_id: str,
agent_id: str,
account_id: str,
committed_items: list[dict[str, Any]],
) -> None:
AgentSoulFilesService.sync_drive_commit_to_active_soul(
tenant_id=tenant_id,
agent_id=agent_id,
account_id=account_id,
committed_items=committed_items,
)
db.session.commit()
def _upload_skill_for_app(*, current_user: Account, app_model: App):
"""Upload one skill package and commit its normalized files into the agent drive."""
@@ -212,9 +195,8 @@ def _upload_skill_for_app(*, current_user: Account, app_model: App):
upload = request.files["file"]
content = upload.stream.read()
standardize_service = SkillStandardizeService()
try:
result = standardize_service.standardize(
result = SkillStandardizeService().standardize(
content=content,
filename=upload.filename or "",
tenant_id=app_model.tenant_id,
@@ -223,12 +205,6 @@ def _upload_skill_for_app(*, current_user: Account, app_model: App):
)
except (SkillPackageError, AgentDriveError) as exc:
return {"code": exc.code, "message": exc.message}, exc.status_code
_sync_active_soul_files(
tenant_id=app_model.tenant_id,
agent_id=agent_id,
account_id=current_user.id,
committed_items=standardize_service.last_committed_items,
)
return result, 201
@@ -267,12 +243,6 @@ def _commit_drive_file_for_app(*, current_user: Account, app_model: App, allow_n
)
except AgentDriveError as exc:
return {"code": exc.code, "message": exc.message}, exc.status_code
_sync_active_soul_files(
tenant_id=app_model.tenant_id,
agent_id=agent_id,
account_id=current_user.id,
committed_items=committed,
)
row = committed[0]
return {
@@ -297,13 +267,15 @@ def _delete_drive_file_for_app(*, current_user: Account, app_model: App, allow_n
except AgentDriveError as exc:
return {"code": exc.code, "message": exc.message}, exc.status_code
result = [{"key": key, "removed": True}]
_sync_active_soul_files(
tenant_id=app_model.tenant_id,
agent_id=agent_id,
account_id=current_user.id,
committed_items=result,
)
try:
result = AgentDriveService().commit(
tenant_id=app_model.tenant_id,
user_id=current_user.id,
agent_id=agent_id,
items=[DriveCommitItem(key=key, file_ref=None)],
)
except AgentDriveError as exc:
return {"code": exc.code, "message": exc.message}, exc.status_code
removed_keys = [item["key"] for item in result if item.get("removed")]
return {"result": "success", "removed_keys": removed_keys}
@@ -318,34 +290,18 @@ def _delete_skill_for_app(*, current_user: Account, app_model: App, slug: str, a
return {"code": "drive_key_invalid", "message": "skill slug must be a single path segment"}, 400
try:
agent_soul = AgentSoulFilesService.active_agent_soul(
session=db.session,
result = AgentDriveService().commit(
tenant_id=app_model.tenant_id,
user_id=current_user.id,
agent_id=agent_id,
items=[
DriveCommitItem(key=f"{slug}/SKILL.md", file_ref=None),
DriveCommitItem(key=f"{slug}/.DIFY-SKILL-FULL.zip", file_ref=None),
],
)
except AgentDriveError as exc:
return {"code": exc.code, "message": exc.message}, exc.status_code
skill_prefix = f"{slug}/"
removed_keys = [
file_ref.drive_key
for skill in agent_soul.files.skills
if (
skill.path
or (AgentSoulFilesService.skill_path_from_key(skill.skill_md_key) if skill.skill_md_key else "")
).strip("/")
== slug
for file_ref in skill.file_refs
if file_ref.drive_key
]
if f"{slug}/SKILL.md" not in removed_keys:
removed_keys.append(f"{slug}/SKILL.md")
result = [{"key": key, "removed": True} for key in removed_keys if key.startswith(skill_prefix)]
_sync_active_soul_files(
tenant_id=app_model.tenant_id,
agent_id=agent_id,
account_id=current_user.id,
committed_items=result,
)
removed_keys = [item["key"] for item in result if item.get("removed")]
return {"result": "success", "removed_keys": removed_keys}
@@ -28,13 +28,10 @@ from controllers.console import console_ns
from controllers.console.agent.app_helpers import resolve_agent_app_model
from controllers.console.app.wraps import get_app_model
from controllers.console.wraps import account_initialization_required, setup_required, with_current_tenant_id
from extensions.ext_database import db
from fields.base import ResponseModel
from libs.login import login_required
from models.agent import AgentDriveFileKind
from models.model import App, AppMode
from services.agent.composer_service import AgentComposerService
from services.agent.soul_files_service import AgentSoulFilesService
from services.agent_drive_service import AgentDriveError, AgentDriveService
@@ -163,111 +160,6 @@ def _handle(exc: AgentDriveError) -> tuple[dict[str, object], int]:
return {"code": exc.code, "message": exc.message}, exc.status_code
def _versioned_manifest(*, tenant_id: str, agent_id: str, prefix: str = "") -> list[dict[str, Any]]:
agent_soul = AgentSoulFilesService.active_agent_soul(session=db.session, tenant_id=tenant_id, agent_id=agent_id)
normalized_prefix = prefix.strip().lstrip("/")
skill_prefixes = AgentSoulFilesService.allowed_skill_prefixes(agent_soul)
if normalized_prefix and any(normalized_prefix.startswith(p) for p in skill_prefixes):
return AgentSoulFilesService.list_manifest_items(
session=db.session,
tenant_id=tenant_id,
agent_id=agent_id,
prefix=normalized_prefix,
)
return AgentSoulFilesService.list_files(
session=db.session,
tenant_id=tenant_id,
agent_id=agent_id,
prefix=normalized_prefix,
)
def _versioned_skills(*, tenant_id: str, agent_id: str) -> list[dict[str, Any]]:
return AgentSoulFilesService.list_skills(session=db.session, tenant_id=tenant_id, agent_id=agent_id)
def _assert_key_in_active_soul(*, tenant_id: str, agent_id: str, key: str) -> None:
agent_soul = AgentSoulFilesService.active_agent_soul(session=db.session, tenant_id=tenant_id, agent_id=agent_id)
if not AgentSoulFilesService.key_allowed_by_soul(agent_soul=agent_soul, key=key):
raise AgentDriveError(
"drive_key_not_in_agent_soul",
"drive key is not part of the active Agent Soul version",
status_code=404,
)
def _file_ref_for_active_soul(*, tenant_id: str, agent_id: str, key: str):
agent_soul = AgentSoulFilesService.active_agent_soul(session=db.session, tenant_id=tenant_id, agent_id=agent_id)
file_ref = AgentSoulFilesService.file_ref_for_key(agent_soul=agent_soul, key=key)
if file_ref is None and not AgentSoulFilesService.key_allowed_by_soul(agent_soul=agent_soul, key=key):
raise AgentDriveError(
"drive_key_not_in_agent_soul",
"drive key is not part of the active Agent Soul version",
status_code=404,
)
return file_ref
def _file_kind_from_ref(file_ref) -> AgentDriveFileKind | None:
raw = file_ref.transfer_method or ("upload_file" if file_ref.upload_file_id else None)
if raw is None:
return None
try:
return AgentDriveFileKind(raw)
except ValueError as exc:
raise AgentDriveError("invalid_drive_file_ref", "Agent Soul file ref has invalid transfer method") from exc
def _preview_versioned_file(*, tenant_id: str, agent_id: str, key: str) -> dict[str, Any]:
file_ref = _file_ref_for_active_soul(tenant_id=tenant_id, agent_id=agent_id, key=key)
if file_ref is None:
return AgentDriveService().preview(tenant_id=tenant_id, agent_id=agent_id, key=key)
file_kind = _file_kind_from_ref(file_ref)
file_id = file_ref.file_id or file_ref.upload_file_id
if file_kind is None or not file_id:
raise AgentDriveError("invalid_drive_file_ref", "Agent Soul file ref is missing file id", status_code=404)
return AgentDriveService().preview_file_ref(
tenant_id=tenant_id,
agent_id=agent_id,
key=key,
file_kind=file_kind,
file_id=file_id,
size=file_ref.get("size"),
)
def _download_versioned_file(*, tenant_id: str, agent_id: str, key: str) -> str:
file_ref = _file_ref_for_active_soul(tenant_id=tenant_id, agent_id=agent_id, key=key)
if file_ref is None:
return AgentDriveService().download_url(tenant_id=tenant_id, agent_id=agent_id, key=key)
file_kind = _file_kind_from_ref(file_ref)
file_id = file_ref.file_id or file_ref.upload_file_id
if file_kind is None or not file_id:
raise AgentDriveError("invalid_drive_file_ref", "Agent Soul file ref is missing file id", status_code=404)
return AgentDriveService().download_url_for_ref(
tenant_id=tenant_id,
agent_id=agent_id,
file_kind=file_kind,
file_id=file_id,
)
def _assert_skill_in_active_soul(*, tenant_id: str, agent_id: str, skill_path: str) -> None:
agent_soul = AgentSoulFilesService.active_agent_soul(session=db.session, tenant_id=tenant_id, agent_id=agent_id)
wanted = skill_path.strip().strip("/")
for skill in agent_soul.files.skills:
path = skill.path
if not path and skill.skill_md_key:
path = AgentSoulFilesService.skill_path_from_key(skill.skill_md_key)
if path == wanted:
return
raise AgentDriveError(
"skill_not_in_agent_soul",
"skill is not part of the active Agent Soul version",
status_code=404,
)
def _json_response(data: Mapping[str, Any]):
return Response(
response=json.dumps(data, ensure_ascii=False, separators=(",", ":")),
@@ -292,7 +184,7 @@ class AgentDriveListByAgentApi(Resource):
query = query_params_from_request(AgentDriveListByAgentQuery)
resolve_agent_app_model(tenant_id=tenant_id, agent_id=agent_id)
try:
items = _versioned_manifest(tenant_id=tenant_id, agent_id=str(agent_id), prefix=query.prefix)
items = AgentDriveService().manifest(tenant_id=tenant_id, agent_id=str(agent_id), prefix=query.prefix)
except AgentDriveError as exc:
return _handle(exc)
return {"items": [{k: v for k, v in item.items() if k != "file_id"} for item in items]}
@@ -311,7 +203,7 @@ class AgentDriveSkillListByAgentApi(Resource):
def get(self, tenant_id: str, agent_id: UUID):
resolve_agent_app_model(tenant_id=tenant_id, agent_id=agent_id)
try:
items = _versioned_skills(tenant_id=tenant_id, agent_id=str(agent_id))
items = AgentDriveService().list_skills(tenant_id=tenant_id, agent_id=str(agent_id))
except AgentDriveError as exc:
return _handle(exc)
return {"items": items}
@@ -330,7 +222,6 @@ class AgentDriveSkillInspectByAgentApi(Resource):
def get(self, tenant_id: str, agent_id: UUID, skill_path: str):
resolve_agent_app_model(tenant_id=tenant_id, agent_id=agent_id)
try:
_assert_skill_in_active_soul(tenant_id=tenant_id, agent_id=str(agent_id), skill_path=skill_path)
return _json_response(
AgentDriveService().inspect_skill(
tenant_id=tenant_id,
@@ -356,7 +247,7 @@ class AgentDrivePreviewByAgentApi(Resource):
query = query_params_from_request(AgentDriveFileByAgentQuery)
resolve_agent_app_model(tenant_id=tenant_id, agent_id=agent_id)
try:
return _preview_versioned_file(tenant_id=tenant_id, agent_id=str(agent_id), key=query.key)
return AgentDriveService().preview(tenant_id=tenant_id, agent_id=str(agent_id), key=query.key)
except AgentDriveError as exc:
return _handle(exc)
@@ -375,7 +266,7 @@ class AgentDriveDownloadByAgentApi(Resource):
query = query_params_from_request(AgentDriveFileByAgentQuery)
resolve_agent_app_model(tenant_id=tenant_id, agent_id=agent_id)
try:
url = _download_versioned_file(tenant_id=tenant_id, agent_id=str(agent_id), key=query.key)
url = AgentDriveService().download_url(tenant_id=tenant_id, agent_id=str(agent_id), key=query.key)
except AgentDriveError as exc:
return _handle(exc)
return {"url": url}
@@ -397,7 +288,7 @@ class AgentDriveListApi(Resource):
if not agent_id:
return _agent_not_bound()
try:
items = _versioned_manifest(tenant_id=app_model.tenant_id, agent_id=agent_id, prefix=query.prefix)
items = AgentDriveService().manifest(tenant_id=app_model.tenant_id, agent_id=agent_id, prefix=query.prefix)
except AgentDriveError as exc:
return _handle(exc)
# the inner manifest exposes file_id for agent-side pulls; the console
@@ -421,7 +312,7 @@ class AgentDriveSkillListApi(Resource):
if not agent_id:
return _agent_not_bound()
try:
items = _versioned_skills(tenant_id=app_model.tenant_id, agent_id=agent_id)
items = AgentDriveService().list_skills(tenant_id=app_model.tenant_id, agent_id=agent_id)
except AgentDriveError as exc:
return _handle(exc)
return {"items": items}
@@ -449,7 +340,6 @@ class AgentDriveSkillInspectApi(Resource):
if not agent_id:
return _agent_not_bound()
try:
_assert_skill_in_active_soul(tenant_id=app_model.tenant_id, agent_id=agent_id, skill_path=skill_path)
return _json_response(
AgentDriveService().inspect_skill(
tenant_id=app_model.tenant_id,
@@ -477,7 +367,7 @@ class AgentDrivePreviewApi(Resource):
if not agent_id:
return _agent_not_bound()
try:
return _preview_versioned_file(tenant_id=app_model.tenant_id, agent_id=agent_id, key=query.key)
return AgentDriveService().preview(tenant_id=app_model.tenant_id, agent_id=agent_id, key=query.key)
except AgentDriveError as exc:
return _handle(exc)
@@ -498,7 +388,7 @@ class AgentDriveDownloadApi(Resource):
if not agent_id:
return _agent_not_bound()
try:
url = _download_versioned_file(tenant_id=app_model.tenant_id, agent_id=agent_id, key=query.key)
url = AgentDriveService().download_url(tenant_id=app_model.tenant_id, agent_id=agent_id, key=query.key)
except AgentDriveError as exc:
return _handle(exc)
return {"url": url}
+4 -4
View File
@@ -331,7 +331,7 @@ class ModelConfig(ResponseModel):
return to_timestamp(value)
class AppDetailSiteResponse(ResponseModel):
class Site(ResponseModel):
access_token: str | None = Field(default=None, validation_alias="code")
code: str | None = None
title: str | None = None
@@ -461,7 +461,7 @@ class AppDetailWithSite(AppDetail):
api_base_url: str | None = None
max_active_requests: int | None = None
deleted_tools: list[DeletedTool] = Field(default_factory=list)
site: AppDetailSiteResponse | None = None
site: Site | None = None
# For Agent App type: the roster Agent backing this app (None otherwise).
bound_agent_id: str | None = None
# For Agent App responses exposed through /agent.
@@ -546,7 +546,7 @@ register_schema_models(
WorkflowPartial,
ModelConfigPartial,
ModelConfig,
AppDetailSiteResponse,
Site,
DeletedTool,
AppDetail,
AppExportResponse,
@@ -1094,7 +1094,7 @@ class AppTraceApi(Resource):
@login_required
@account_initialization_required
@edit_permission_required
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_MONITOR)
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_TRACING_CONFIG)
@get_app_model
def post(self, app_model: App):
# add app trace
@@ -93,10 +93,6 @@ class ChatMessagePayload(BaseMessagePayload):
query: str = Field(..., description="User query")
conversation_id: str | None = Field(default=None, description="Conversation ID")
parent_message_id: str | None = Field(default=None, description="Parent message ID")
draft_type: Literal["draft", "debug_build"] = Field(
default="draft",
description="Agent App debug config source. Use debug_build while the Agent is in build mode.",
)
@field_validator("conversation_id", "parent_message_id")
@classmethod
+5 -1
View File
@@ -167,12 +167,16 @@ register_schema_models(
ChatMessagesQuery,
MessageFeedbackPayload,
FeedbackExportQuery,
)
register_response_schema_models(
console_ns,
AnnotationCountResponse,
SuggestedQuestionsResponse,
MessageDetailResponse,
MessageInfiniteScrollPaginationResponse,
SimpleResultResponse,
TextFileResponse,
)
register_response_schema_models(console_ns, SimpleResultResponse, TextFileResponse)
@console_ns.route("/apps/<uuid:app_id>/chat-messages")
+11 -1
View File
@@ -13,6 +13,7 @@ from controllers.console.wraps import (
RBACPermission,
RBACResourceScope,
account_initialization_required,
edit_permission_required,
rbac_permission_required,
setup_required,
)
@@ -70,7 +71,7 @@ class TraceAppConfigApi(Resource):
@setup_required
@login_required
@account_initialization_required
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_MONITOR)
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_TRACING_CONFIG)
@get_app_model
def get(self, app_model: App):
args = TraceProviderQuery.model_validate(request.args.to_dict(flat=True)) # type: ignore
@@ -95,9 +96,12 @@ class TraceAppConfigApi(Resource):
console_ns.models[TraceAppConfigResponse.__name__],
)
@console_ns.response(400, "Invalid request parameters or configuration already exists")
@console_ns.response(403, "Insufficient permissions")
@setup_required
@login_required
@account_initialization_required
@edit_permission_required
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_TRACING_CONFIG)
@get_app_model
def post(self, app_model: App):
"""Create a new trace app configuration"""
@@ -125,9 +129,12 @@ class TraceAppConfigApi(Resource):
console_ns.models[TraceAppConfigResponse.__name__],
)
@console_ns.response(400, "Invalid request parameters or configuration not found")
@console_ns.response(403, "Insufficient permissions")
@setup_required
@login_required
@account_initialization_required
@edit_permission_required
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_TRACING_CONFIG)
@get_app_model
def patch(self, app_model: App):
"""Update an existing trace app configuration"""
@@ -149,9 +156,12 @@ class TraceAppConfigApi(Resource):
@console_ns.doc(params=query_params_from_model(TraceProviderQuery))
@console_ns.response(204, "Tracing configuration deleted successfully")
@console_ns.response(400, "Invalid request parameters or configuration not found")
@console_ns.response(403, "Insufficient permissions")
@setup_required
@login_required
@account_initialization_required
@edit_permission_required
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_TRACING_CONFIG)
@get_app_model
def delete(self, app_model: App):
"""Delete an existing trace app configuration"""
@@ -181,7 +181,7 @@ class WorkflowAppLogApi(Resource):
@setup_required
@login_required
@account_initialization_required
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_MONITOR)
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_LOG_AND_ANNOTATION)
@get_app_model(mode=[AppMode.WORKFLOW])
def get(self, app_model: App):
"""
@@ -225,7 +225,7 @@ class WorkflowArchivedLogApi(Resource):
@setup_required
@login_required
@account_initialization_required
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_MONITOR)
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_LOG_AND_ANNOTATION)
@get_app_model(mode=[AppMode.WORKFLOW])
def get(self, app_model: App):
"""
@@ -85,6 +85,7 @@ def _published_app_filter():
class InstalledAppInfoResponse(ResponseModel):
id: str
name: str | None = None
description: str | None = None
mode: str | None = None
icon_type: str | None = None
icon: str | None = None
@@ -123,6 +124,7 @@ class InstalledAppResponse(ResponseModel):
return {
"id": _safe_primitive(getattr(value, "id", "")) or "",
"name": _safe_primitive(getattr(value, "name", None)),
"description": _safe_primitive(getattr(value, "description", None)),
"mode": _safe_primitive(getattr(value, "mode", None)),
"icon_type": _safe_primitive(getattr(value, "icon_type", None)),
"icon": _safe_primitive(getattr(value, "icon", None)),
@@ -169,7 +169,7 @@ class EndpointCollectionApi(Resource):
@setup_required
@login_required
@is_admin_or_owner_required
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.PLUGIN_MANAGE, resource_required=False)
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.PLUGIN_MODEL_CONFIG, resource_required=False)
@account_initialization_required
@with_current_user_id
@with_current_tenant_id
@@ -198,7 +198,7 @@ class DeprecatedEndpointCreateApi(Resource):
@setup_required
@login_required
@is_admin_or_owner_required
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.PLUGIN_MANAGE, resource_required=False)
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.PLUGIN_MODEL_CONFIG, resource_required=False)
@account_initialization_required
@with_current_user_id
@with_current_tenant_id
@@ -290,7 +290,7 @@ class EndpointItemApi(Resource):
@setup_required
@login_required
@is_admin_or_owner_required
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.PLUGIN_MANAGE, resource_required=False)
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.PLUGIN_MODEL_CONFIG, resource_required=False)
@account_initialization_required
@with_current_user_id
@with_current_tenant_id
@@ -310,7 +310,7 @@ class EndpointItemApi(Resource):
@setup_required
@login_required
@is_admin_or_owner_required
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.PLUGIN_MANAGE, resource_required=False)
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.PLUGIN_MODEL_CONFIG, resource_required=False)
@account_initialization_required
@with_current_user_id
@with_current_tenant_id
@@ -340,7 +340,7 @@ class DeprecatedEndpointDeleteApi(Resource):
@setup_required
@login_required
@is_admin_or_owner_required
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.PLUGIN_MANAGE, resource_required=False)
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.PLUGIN_MODEL_CONFIG, resource_required=False)
@account_initialization_required
@with_current_user_id
@with_current_tenant_id
@@ -371,7 +371,7 @@ class DeprecatedEndpointUpdateApi(Resource):
@setup_required
@login_required
@is_admin_or_owner_required
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.PLUGIN_MANAGE, resource_required=False)
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.PLUGIN_MODEL_CONFIG, resource_required=False)
@account_initialization_required
@with_current_user_id
@with_current_tenant_id
@@ -394,7 +394,7 @@ class EndpointEnableApi(Resource):
@setup_required
@login_required
@is_admin_or_owner_required
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.PLUGIN_MANAGE, resource_required=False)
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.PLUGIN_MODEL_CONFIG, resource_required=False)
@account_initialization_required
@with_current_user_id
@with_current_tenant_id
@@ -422,7 +422,7 @@ class EndpointDisableApi(Resource):
@setup_required
@login_required
@is_admin_or_owner_required
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.PLUGIN_MANAGE, resource_required=False)
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.PLUGIN_MODEL_CONFIG, resource_required=False)
@account_initialization_required
@with_current_user_id
@with_current_tenant_id
+4 -9
View File
@@ -793,7 +793,6 @@ class PluginFetchInstallTasksApi(Resource):
@setup_required
@login_required
@account_initialization_required
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.PLUGIN_INSTALL, resource_required=False)
@plugin_permission_required(install_required=True)
@with_current_tenant_id
def get(self, tenant_id: str):
@@ -811,7 +810,6 @@ class PluginFetchInstallTaskApi(Resource):
@setup_required
@login_required
@account_initialization_required
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.PLUGIN_INSTALL, resource_required=False)
@plugin_permission_required(install_required=True)
@with_current_tenant_id
def get(self, tenant_id: str, task_id: str):
@@ -827,7 +825,6 @@ class PluginDeleteInstallTaskApi(Resource):
@setup_required
@login_required
@account_initialization_required
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.PLUGIN_INSTALL, resource_required=False)
@plugin_permission_required(install_required=True)
@with_current_tenant_id
def post(self, tenant_id: str, task_id: str):
@@ -843,7 +840,6 @@ class PluginDeleteAllInstallTaskItemsApi(Resource):
@setup_required
@login_required
@account_initialization_required
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.PLUGIN_INSTALL, resource_required=False)
@plugin_permission_required(install_required=True)
@with_current_tenant_id
def post(self, tenant_id: str):
@@ -859,7 +855,6 @@ class PluginDeleteInstallTaskItemApi(Resource):
@setup_required
@login_required
@account_initialization_required
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.PLUGIN_INSTALL, resource_required=False)
@plugin_permission_required(install_required=True)
@with_current_tenant_id
def post(self, tenant_id: str, task_id: str, identifier: str):
@@ -876,7 +871,7 @@ class PluginUpgradeFromMarketplaceApi(Resource):
@setup_required
@login_required
@account_initialization_required
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.PLUGIN_INSTALL, resource_required=False)
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.PLUGIN_MODEL_CONFIG, resource_required=False)
@plugin_permission_required(install_required=True)
@with_current_tenant_id
def post(self, tenant_id: str):
@@ -899,7 +894,7 @@ class PluginUpgradeFromGithubApi(Resource):
@setup_required
@login_required
@account_initialization_required
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.PLUGIN_INSTALL, resource_required=False)
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.PLUGIN_MODEL_CONFIG, resource_required=False)
@plugin_permission_required(install_required=True)
@with_current_tenant_id
def post(self, tenant_id: str):
@@ -927,7 +922,7 @@ class PluginUninstallApi(Resource):
@setup_required
@login_required
@account_initialization_required
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.PLUGIN_INSTALL, resource_required=False)
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.PLUGIN_DELETE, resource_required=False)
@plugin_permission_required(install_required=True)
@with_current_tenant_id
def post(self, tenant_id: str):
@@ -995,7 +990,7 @@ class PluginFetchDynamicSelectOptionsApi(Resource):
@setup_required
@login_required
@is_admin_or_owner_required
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.PLUGIN_MANAGE, resource_required=False)
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.PLUGIN_MODEL_CONFIG, resource_required=False)
@account_initialization_required
@with_current_user
@with_current_tenant_id
+16 -14
View File
@@ -201,21 +201,23 @@ def _legacy_workspace_roles(
This keeps the new `/rbac/roles` endpoint compatible with the original
Dify role model when enterprise RBAC is disabled.
"""
legacy_roles = [
svc.RBACRole(
id=role_name,
tenant_id="",
type=svc.RBACRoleType.WORKSPACE.value,
category="global_system_default",
name=role_name,
description="",
is_builtin=True,
permission_keys=list(dict.fromkeys(_LEGACY_ROLE_PERMISSION_KEYS[role_name])),
role_tag="owner" if role_name == "owner" else "",
legacy_roles = []
for role_name in ("owner", "admin", "editor", "normal", "dataset_operator"):
if not dify_config.DATASET_OPERATOR_ENABLED and role_name == "dataset_operator":
continue
legacy_roles.append(
svc.RBACRole(
id=role_name,
tenant_id="",
type=svc.RBACRoleType.WORKSPACE.value,
category="global_system_default",
name=role_name,
description="",
is_builtin=True,
permission_keys=list(dict.fromkeys(_LEGACY_ROLE_PERMISSION_KEYS[role_name])),
role_tag="owner" if role_name == "owner" else "",
)
)
for role_name in ("owner", "admin", "editor", "normal", "dataset_operator")
]
if not include_owner:
legacy_roles = [r for r in legacy_roles if r.name != "owner"]
@@ -17,9 +17,6 @@ from controllers.console.wraps import setup_required
from controllers.inner_api import inner_api_ns
from controllers.inner_api.plugin.wraps import get_user
from controllers.inner_api.wraps import plugin_inner_api_only
from extensions.ext_database import db
from models.agent import AgentDriveFileKind
from services.agent.soul_files_service import AgentSoulFilesService
from services.agent_drive_service import (
AgentDriveError,
AgentDriveService,
@@ -38,42 +35,6 @@ def _error_response(exc: AgentDriveError) -> tuple[dict[str, str], int]:
return {"code": exc.code, "message": exc.message}, exc.status_code
def _versioned_manifest(
*,
tenant_id: str,
agent_id: str,
prefix: str = "",
include_download_url: bool = False,
) -> list[dict[str, object]]:
agent_soul = AgentSoulFilesService.active_agent_soul(session=db.session, tenant_id=tenant_id, agent_id=agent_id)
normalized_prefix = prefix.strip().lstrip("/")
items = AgentSoulFilesService.list_manifest_items(
session=db.session,
tenant_id=tenant_id,
agent_id=agent_id,
prefix=normalized_prefix,
)
skill_prefixes = AgentSoulFilesService.allowed_skill_prefixes(agent_soul)
if normalized_prefix and not any(normalized_prefix.startswith(p) for p in skill_prefixes):
allowed_keys = AgentSoulFilesService.allowed_drive_keys(agent_soul)
items = [item for item in items if item.get("key") in allowed_keys]
if include_download_url:
for item in items:
file_kind = item.get("file_kind")
file_id = item.get("file_id")
if not file_kind or not file_id:
continue
try:
item["download_url"] = AgentDriveService.resolve_download_url_for_ref(
tenant_id=tenant_id,
file_kind=AgentDriveFileKind(str(file_kind)),
file_id=str(file_id),
)
except ValueError:
item["download_url"] = None
return items
@inner_api_ns.route("/drive/<string:drive_ref>/manifest")
class AgentDriveManifestApi(Resource):
@setup_required
@@ -87,7 +48,7 @@ class AgentDriveManifestApi(Resource):
if not tenant_id:
raise AgentDriveError("missing_tenant_id", "tenant_id is required", status_code=400)
include_download_url = (request.args.get("include_download_url") or "").lower() in ("1", "true", "yes")
items = _versioned_manifest(
items = AgentDriveService().manifest(
tenant_id=tenant_id,
agent_id=agent_id,
prefix=request.args.get("prefix", ""),
@@ -110,7 +71,7 @@ class AgentDriveSkillsApi(Resource):
tenant_id = (request.args.get("tenant_id") or "").strip()
if not tenant_id:
raise AgentDriveError("missing_tenant_id", "tenant_id is required", status_code=400)
items = AgentSoulFilesService.list_skills(session=db.session, tenant_id=tenant_id, agent_id=agent_id)
items = AgentDriveService().list_skills(tenant_id=tenant_id, agent_id=agent_id)
except AgentDriveError as exc:
return _error_response(exc)
return {"items": items}
@@ -136,13 +97,6 @@ class AgentDriveCommitApi(Resource):
agent_id=agent_id,
items=body.items,
)
AgentSoulFilesService.sync_drive_commit_to_active_soul(
tenant_id=body.tenant_id,
agent_id=agent_id,
account_id=user.id,
committed_items=items,
)
db.session.commit()
except AgentDriveError as exc:
return _error_response(exc)
return {"items": items}
+2 -1
View File
@@ -20,7 +20,7 @@ openapi_ns = Namespace("openapi", description="User-scoped operations", path="/"
# Register response/query models BEFORE importing controller modules so that
# @openapi_ns.response / @openapi_ns.expect decorators can resolve model names.
from controllers.common.fields import EventStreamResponse
from controllers.common.fields import EventStreamResponse, SimpleResultResponse
from controllers.common.schema import register_enum_models, register_response_schema_models, register_schema_models
from controllers.openapi._models import (
AccountPayload,
@@ -95,6 +95,7 @@ register_response_schema_models(
openapi_ns,
ErrorBody,
EventStreamResponse,
SimpleResultResponse,
UsageInfo,
MessageMetadata,
AppListRow,
+14
View File
@@ -34,6 +34,7 @@ class OpenApiErrorCode(StrEnum):
# transport-generic (resolved from HTTP status for plain werkzeug raises)
BAD_REQUEST = "bad_request"
UNAUTHORIZED = "unauthorized"
TOKEN_EXPIRED = "token_expired"
FORBIDDEN = "forbidden"
NOT_FOUND = "not_found"
METHOD_NOT_ALLOWED = "method_not_allowed"
@@ -223,6 +224,19 @@ class OpenApiErrorFormatter:
return isinstance(part, (str, int)) and not isinstance(part, bool)
class InvalidBearer(OpenApiError): # noqa: N818
code = 401
error_code = OpenApiErrorCode.UNAUTHORIZED
description = "Invalid or unknown bearer token."
class SessionExpired(OpenApiError): # noqa: N818
code = 401
error_code = OpenApiErrorCode.TOKEN_EXPIRED
description = "Your session has expired."
hint = "Re-authenticate to continue (e.g. re-run your login command)."
class FilenameNotExists(OpenApiError): # noqa: N818
code = 400
error_code = OpenApiErrorCode.FILENAME_NOT_EXISTS
+3 -2
View File
@@ -3,7 +3,7 @@
from __future__ import annotations
import logging
from collections.abc import Callable, Iterator
from collections.abc import Callable, Generator
from contextlib import contextmanager
from typing import Any
@@ -61,7 +61,7 @@ logger = logging.getLogger(__name__)
@contextmanager
def _translate_service_errors() -> Iterator[None]:
def _translate_service_errors() -> Generator[None, None, None]:
try:
yield
except WorkflowNotFoundError as ex:
@@ -166,6 +166,7 @@ class AppRunApi(Resource):
surface="apps",
)
# response-contract:ignore compact_generate_response
return helper.compact_generate_response(stream_obj)
+6 -3
View File
@@ -25,12 +25,13 @@ from controllers.openapi._models import (
AppListRow,
)
from controllers.openapi.auth.composition import auth_router
from controllers.openapi.auth.data import AuthData, RBACRequirement
from controllers.openapi.auth.data import AuthData, CallerKind, RBACRequirement
from controllers.service_api.app.error import AppUnavailableError
from core.app.app_config.common.parameters_mapping import get_parameters_from_feature_dict
from extensions.ext_database import db
from libs.oauth_bearer import Scope, TokenType
from models import App
from models.enums import AppStatus
from models.model import AppMode
from services.account_service import TenantService
from services.app_service import AppListParams, AppService
@@ -166,7 +167,9 @@ class AppListApi(Resource):
# an empty set or list means the caller has no accessible apps.
# End-users bypass RBAC here — their access is controlled by scope upstream.
apply_rbac_filter = (
dify_config.RBAC_ENABLED and auth_data.caller_kind != "end_user" and auth_data.account_id is not None
dify_config.RBAC_ENABLED
and auth_data.caller_kind != CallerKind.END_USER
and auth_data.account_id is not None
)
access_filter = AppAccessFilter.unrestricted()
if apply_rbac_filter:
@@ -203,7 +206,7 @@ class AppListApi(Resource):
limit=query.limit,
mode=query.mode.value if query.mode else "all", # type:ignore
name=query.name,
status="normal",
status=AppStatus.NORMAL,
# Visibility gate pushed into the query — pagination.total stays
# consistent across pages because invisible rows never count.
openapi_visible=True,
@@ -25,6 +25,7 @@ from controllers.openapi.auth.data import AuthData, Edition
from extensions.ext_database import db
from libs.oauth_bearer import Scope, TokenType
from models import App
from models.enums import AppStatus
from services.account_service import TenantService
from services.app_service import AppService
from services.enterprise.app_permitted_service import list_permitted_apps
@@ -62,7 +63,7 @@ class PermittedExternalAppsListApi(Resource):
items: list[AppListRow] = []
for app_id in page_result.app_ids:
app = apps_by_id.get(app_id)
if not app or app.status != "normal":
if not app or app.status != AppStatus.NORMAL:
continue
tenant = tenants_by_id.get(str(app.tenant_id))
items.append(
+7 -3
View File
@@ -2,7 +2,6 @@ from __future__ import annotations
import uuid
from enum import StrEnum
from typing import Literal
from pydantic import BaseModel, ConfigDict, Field
from werkzeug.exceptions import InternalServerError
@@ -21,6 +20,11 @@ class Edition(StrEnum):
SAAS = "saas"
class CallerKind(StrEnum):
ACCOUNT = "account"
END_USER = "end_user"
def current_edition() -> Edition:
if dify_config.EDITION == "CLOUD":
return Edition.SAAS
@@ -78,9 +82,9 @@ class AuthData(BaseModel):
tenant_role: TenantAccountRole | None = None
caller: Account | EndUser | None = None
caller_kind: Literal["account", "end_user"] | None = None
caller_kind: CallerKind | None = None
def require_app_context(self) -> tuple[App, Account | EndUser, Literal["account", "end_user"]]:
def require_app_context(self) -> tuple[App, Account | EndUser, CallerKind]:
if self.app is None or self.caller is None or self.caller_kind is None:
raise InternalServerError("pipeline_invariant_violated: app context missing")
return self.app, self.caller, self.caller_kind
+9 -1
View File
@@ -17,6 +17,7 @@ from flask_login import user_logged_in
from werkzeug.exceptions import Forbidden, NotFound, Unauthorized
from controllers.openapi._audit import emit_wrong_surface
from controllers.openapi._errors import InvalidBearer, SessionExpired
from controllers.openapi.auth.data import (
AuthData,
Edition,
@@ -28,7 +29,9 @@ from controllers.openapi.auth.data import (
from controllers.openapi.auth.flow import When
from libs.oauth_bearer import (
AuthContext,
InvalidBearerError,
Scope,
TokenExpiredError,
TokenType,
extract_bearer,
get_authenticator,
@@ -217,7 +220,12 @@ class PipelineRouter:
if not token:
raise Unauthorized("bearer required")
identity = get_authenticator().authenticate(token)
try:
identity = get_authenticator().authenticate(token)
except TokenExpiredError:
raise SessionExpired()
except InvalidBearerError:
raise InvalidBearer()
if allowed_token_types is not None and identity.token_type not in allowed_token_types:
emit_wrong_surface(
+7 -7
View File
@@ -5,10 +5,10 @@ import uuid
from flask import request
from werkzeug.exceptions import Forbidden, InternalServerError, NotFound, Unauthorized
from controllers.openapi.auth.data import AuthData
from controllers.openapi.auth.data import AuthData, CallerKind
from extensions.ext_database import db
from models.account import TenantStatus
from models.enums import EndUserType
from models.account import AccountStatus, TenantStatus
from models.enums import AppStatus, EndUserType
from services.account_service import AccountService, TenantService
from services.app_service import AppService
from services.end_user_service import EndUserService
@@ -24,7 +24,7 @@ def load_app(data: AuthData) -> None:
except ValueError:
raise NotFound("app not found")
app = AppService.get_app_by_id(db.session, app_id)
if not app or app.status != "normal":
if not app or app.status != AppStatus.NORMAL:
raise NotFound("app not found")
data.app = app
@@ -65,7 +65,7 @@ def load_account(data: AuthData) -> None:
if data.tenant:
account.current_tenant = data.tenant
data.caller = account
data.caller_kind = "account"
data.caller_kind = CallerKind.ACCOUNT
def load_workspace_role(data: AuthData) -> None:
@@ -73,7 +73,7 @@ def load_workspace_role(data: AuthData) -> None:
return
if data.tenant is None or data.account_id is None:
return
if data.caller is not None and getattr(data.caller, "status", None) != "active":
if data.caller is not None and getattr(data.caller, "status", None) != AccountStatus.ACTIVE:
return
role = TenantService.get_account_role_in_tenant(db.session, str(data.account_id), str(data.tenant.id))
if role is None:
@@ -91,7 +91,7 @@ def resolve_external_user(data: AuthData) -> None:
user_id=data.external_identity.email,
)
data.caller = end_user
data.caller_kind = "end_user"
data.caller_kind = CallerKind.END_USER
def load_app_access_mode(data: AuthData) -> None:
+2 -2
View File
@@ -5,7 +5,7 @@ from werkzeug.exceptions import Forbidden, NotFound, UnprocessableEntity
from configs import dify_config
from controllers.common.wraps import enforce_rbac_access
from controllers.openapi.auth.data import AuthData
from controllers.openapi.auth.data import AuthData, CallerKind
from extensions.ext_database import db
from libs.oauth_bearer import Scope, TokenType
from services.account_service import AccountService, TenantService
@@ -58,7 +58,7 @@ def check_rbac_permission(data: AuthData) -> None:
if not dify_config.RBAC_ENABLED:
return
# Only account callers are subject to RBAC; end_user access is scope-controlled.
if data.caller_kind != "account":
if data.caller_kind != CallerKind.ACCOUNT:
return
if data.account_id is None or data.tenant is None:
raise Forbidden("rbac context missing")
+2 -2
View File
@@ -22,7 +22,7 @@ from controllers.openapi._contract import accepts, returns
from controllers.openapi._errors import HumanInputFormNotFound, RecipientSurfaceMismatch
from controllers.openapi._models import FormSubmitResponse, HumanInputFormDefinitionResponse
from controllers.openapi.auth.composition import auth_router
from controllers.openapi.auth.data import AuthData, RBACRequirement
from controllers.openapi.auth.data import AuthData, CallerKind, RBACRequirement
from core.workflow.human_input_policy import (
HumanInputSurface,
is_recipient_type_allowed_for_surface,
@@ -98,7 +98,7 @@ class OpenApiWorkflowHumanInputFormApi(Resource):
submission_user_id: str | None = None
submission_end_user_id: str | None = None
if caller_kind == "account":
if caller_kind == CallerKind.ACCOUNT:
submission_user_id = caller.id
else:
submission_end_user_id = caller.id
+2 -2
View File
@@ -22,7 +22,7 @@ from controllers.common.schema import query_params_from_model
from controllers.common.wraps import RBACPermission, RBACResourceScope
from controllers.openapi import openapi_ns
from controllers.openapi.auth.composition import auth_router
from controllers.openapi.auth.data import AuthData, RBACRequirement
from controllers.openapi.auth.data import AuthData, CallerKind, RBACRequirement
from core.app.apps.advanced_chat.app_generator import AdvancedChatAppGenerator
from core.app.apps.base_app_generator import BaseAppGenerator
from core.app.apps.common.workflow_response_converter import WorkflowResponseConverter
@@ -70,7 +70,7 @@ class OpenApiWorkflowEventsApi(Resource):
if workflow_run.app_id != app_model.id:
raise NotFound("Workflow run not found")
if caller_kind == "account":
if caller_kind == CallerKind.ACCOUNT:
if workflow_run.created_by_role != CreatorUserRole.ACCOUNT or workflow_run.created_by != caller.id:
raise NotFound("Workflow run not found")
else:
+7
View File
@@ -32,6 +32,7 @@ from libs.helper import uuid_value
from models.model import App, AppMode, EndUser
from services.app_generate_service import AppGenerateService
from services.app_task_service import AppTaskService
from services.conversation_service import ConversationService
from services.errors.llm import InvokeRateLimitError
logger = logging.getLogger(__name__)
@@ -202,6 +203,12 @@ class ChatApi(WebApiResource):
args["auto_generate_name"] = False
try:
# Eagerly validate conversation to avoid hanging on invalid conversation_id
if payload.conversation_id:
ConversationService.get_conversation(
app_model=app_model, conversation_id=payload.conversation_id, user=end_user
)
response = AppGenerateService.generate(
app_model=app_model, user=end_user, args=args, invoke_from=InvokeFrom.WEB_APP, streaming=streaming
)
+22 -116
View File
@@ -42,15 +42,7 @@ from core.app.llm.model_access import build_dify_model_access
from core.ops.ops_trace_manager import TraceQueueManager
from extensions.ext_database import db
from models import Account, App, EndUser, Message
from models.agent import (
Agent,
AgentConfigDraft,
AgentConfigDraftType,
AgentConfigSnapshot,
AgentScope,
AgentSource,
AgentStatus,
)
from models.agent import Agent, AgentConfigSnapshot, AgentScope, AgentSource, AgentStatus
from models.agent_config_entities import AgentSoulConfig
from services.conversation_service import ConversationService
@@ -81,15 +73,10 @@ class AgentAppGenerator(MessageBasedAppGenerator):
inputs = args["inputs"]
# Resolve the bound roster Agent + its current Agent Soul snapshot.
agent, agent_config_id, agent_soul = self._resolve_agent(
app_model,
invoke_from=invoke_from,
draft_type=args.get("draft_type"),
user=user,
)
agent, snapshot, agent_soul = self._resolve_agent(app_model)
runtime_session_snapshot_id = self._runtime_session_snapshot_id(
invoke_from=invoke_from,
snapshot_id=agent_config_id,
snapshot_id=snapshot.id,
)
conversation = None
@@ -136,7 +123,7 @@ class AgentAppGenerator(MessageBasedAppGenerator):
call_depth=0,
trace_manager=trace_manager,
agent_id=agent.id,
agent_config_snapshot_id=agent_config_id,
agent_config_snapshot_id=snapshot.id,
agent_runtime_session_snapshot_id=runtime_session_snapshot_id,
)
@@ -192,12 +179,7 @@ class AgentAppGenerator(MessageBasedAppGenerator):
persisted to the conversation. Live streaming to a reconnected client is
out of scope here — the message is persisted and can be re-fetched.
"""
agent, agent_config_id, agent_soul = self._resolve_agent(
app_model,
invoke_from=invoke_from,
draft_type="draft",
user=user,
)
agent, snapshot, agent_soul = self._resolve_agent(app_model)
conversation = ConversationService.get_conversation(
app_model=app_model, conversation_id=conversation_id, user=user
)
@@ -244,7 +226,7 @@ class AgentAppGenerator(MessageBasedAppGenerator):
call_depth=0,
trace_manager=trace_manager,
agent_id=agent.id,
agent_config_snapshot_id=agent_config_id,
agent_config_snapshot_id=snapshot.id,
)
conversation, message = self._init_generate_records(application_generate_entity, conversation)
@@ -439,14 +421,7 @@ class AgentAppGenerator(MessageBasedAppGenerator):
return False, query
def _resolve_agent(
self,
app_model: App,
*,
invoke_from: InvokeFrom,
draft_type: Any,
user: Account | EndUser,
) -> tuple[Agent, str, AgentSoulConfig]:
def _resolve_agent(self, app_model: App) -> tuple[Agent, AgentConfigSnapshot, AgentSoulConfig]:
agent = db.session.scalar(
select(Agent).where(
Agent.app_id == app_model.id,
@@ -457,108 +432,39 @@ class AgentAppGenerator(MessageBasedAppGenerator):
)
if agent is None:
raise AgentAppGeneratorError("Agent App has no bound Agent")
if invoke_from == InvokeFrom.DEBUGGER:
draft = self._resolve_debug_draft(
tenant_id=app_model.tenant_id,
agent=agent,
draft_type=draft_type,
account_id=user.id if isinstance(user, Account) else None,
)
agent_soul = AgentSoulConfig.model_validate(draft.config_snapshot_dict)
return agent, draft.id, agent_soul
_, snapshot, agent_soul = self._resolve_agent_by_id(
tenant_id=app_model.tenant_id,
agent_id=agent.id,
snapshot_id=agent.active_config_snapshot_id,
return self._resolve_agent_by_id(
tenant_id=app_model.tenant_id, agent_id=agent.id, snapshot_id=agent.active_config_snapshot_id
)
return agent, snapshot.id, agent_soul
@staticmethod
def _runtime_session_snapshot_id(*, invoke_from: InvokeFrom, snapshot_id: str) -> str | None:
"""Return the session scope snapshot id for Agent App runtime state.
Console preview/debug chat uses a stable Agent draft row id; build mode
uses the current user's build-draft row id. Published/web/API runs use
immutable published snapshot ids. This keeps runtime session continuity
inside one editable surface without mixing draft/build/published state.
Console preview/debug chat is an editing workspace: saving Agent Soul
creates replacement snapshots, but the user expects the same preview
conversation to keep context while trying prompt changes. Use a stable
NULL snapshot scope for debugger runs so each turn can use the latest
Agent Soul while reusing the conversation history. Published/web/API
runs keep snapshot-scoped sessions for reproducible runtime state.
"""
if invoke_from == InvokeFrom.DEBUGGER:
return None
return snapshot_id
@staticmethod
def _resolve_debug_draft(
*, tenant_id: str, agent: Agent, draft_type: Any, account_id: str | None
) -> AgentConfigDraft:
effective_draft_type = (
AgentConfigDraftType.DEBUG_BUILD
if draft_type == AgentConfigDraftType.DEBUG_BUILD.value
else AgentConfigDraftType.DRAFT
)
stmt = select(AgentConfigDraft).where(
AgentConfigDraft.tenant_id == tenant_id,
AgentConfigDraft.agent_id == agent.id,
AgentConfigDraft.draft_type == effective_draft_type,
)
if effective_draft_type == AgentConfigDraftType.DEBUG_BUILD:
if not account_id:
raise AgentAppGeneratorError("Build draft requires an account user")
stmt = stmt.where(AgentConfigDraft.account_id == account_id)
else:
stmt = stmt.where(AgentConfigDraft.account_id.is_(None))
draft = db.session.scalar(stmt.order_by(AgentConfigDraft.updated_at.desc()).limit(1))
if draft is not None:
return draft
if effective_draft_type == AgentConfigDraftType.DEBUG_BUILD:
raise AgentAppGeneratorError("Agent build draft not found")
_, snapshot, agent_soul = AgentAppGenerator._resolve_agent_by_id(
tenant_id=tenant_id,
agent_id=agent.id,
snapshot_id=agent.active_config_snapshot_id,
)
draft = AgentConfigDraft(
tenant_id=tenant_id,
agent_id=agent.id,
draft_type=AgentConfigDraftType.DRAFT,
account_id=None,
draft_owner_key="",
base_snapshot_id=snapshot.id,
config_snapshot=agent_soul,
created_by=agent.created_by,
updated_by=agent.updated_by,
)
db.session.add(draft)
db.session.flush()
return draft
@staticmethod
def _resolve_agent_by_id(
*, tenant_id: str, agent_id: str, snapshot_id: str | None
) -> tuple[Agent, AgentConfigSnapshot | AgentConfigDraft, AgentSoulConfig]:
) -> tuple[Agent, AgentConfigSnapshot, AgentSoulConfig]:
agent = db.session.scalar(select(Agent).where(Agent.id == agent_id, Agent.tenant_id == tenant_id))
if agent is None:
raise AgentAppGeneratorError("Agent not found")
if not snapshot_id:
raise AgentAppGeneratorError("Agent has no published version")
snapshot = db.session.scalar(
select(AgentConfigSnapshot).where(
AgentConfigSnapshot.tenant_id == tenant_id,
AgentConfigSnapshot.agent_id == agent_id,
AgentConfigSnapshot.id == snapshot_id,
)
)
if snapshot is not None:
agent_soul = AgentSoulConfig.model_validate(snapshot.config_snapshot_dict)
return agent, snapshot, agent_soul
draft = db.session.scalar(
select(AgentConfigDraft).where(
AgentConfigDraft.tenant_id == tenant_id,
AgentConfigDraft.agent_id == agent_id,
AgentConfigDraft.id == snapshot_id,
)
)
if draft is None:
snapshot = db.session.scalar(select(AgentConfigSnapshot).where(AgentConfigSnapshot.id == snapshot_id))
if snapshot is None:
raise AgentAppGeneratorError("Agent published version not found")
agent_soul = AgentSoulConfig.model_validate(draft.config_snapshot_dict)
return agent, draft, agent_soul
agent_soul = AgentSoulConfig.model_validate(snapshot.config_snapshot_dict)
return agent, snapshot, agent_soul
__all__ = ["AgentAppGenerator", "AgentAppGeneratorError"]
+2 -1
View File
@@ -240,7 +240,8 @@ class HostingConfiguration:
if len(quotas) > 0:
credentials = {
"dashscope_api_key": dify_config.HOSTED_TONGYI_API_KEY,
"use_international_endpoint": dify_config.HOSTED_TONGYI_USE_INTERNATIONAL_ENDPOINT,
# SNP-494: keep temporary compatibility with tongyi plugin string credential checks.
"use_international_endpoint": str(dify_config.HOSTED_TONGYI_USE_INTERNATIONAL_ENDPOINT).lower(),
}
return HostingProvider(enabled=True, credentials=credentials, quota_unit=quota_unit, quotas=quotas)
+4
View File
@@ -28,6 +28,8 @@ class RBACPermission(StrEnum):
APP_IMPORT_EXPORT_DSL = "app_import_export_dsl"
APP_EDIT = "app_edit"
APP_MONITOR = "app_monitor"
APP_TRACING_CONFIG = "app_tracing_config"
APP_LOG_AND_ANNOTATION = "app_log_and_annotation"
APP_DELETE = "app_delete"
APP_ACCESS_CONFIG = "app_access_config"
@@ -57,7 +59,9 @@ class RBACPermission(StrEnum):
PLUGIN_INSTALL = "plugin_install"
PLUGIN_PREFERENCES = "plugin_preferences"
PLUGIN_MODEL_CONFIG = "plugin_model_config"
PLUGIN_MANAGE = "plugin_manage"
PLUGIN_DELETE = "plugin_delete"
PLUGIN_DEBUG = "plugin_debug"
CREDENTIAL_USE = "credential_use"
@@ -3,7 +3,6 @@ from __future__ import annotations
from typing import Any
from models.agent_config_entities import AgentSoulConfig
from services.agent.knowledge_datasets import list_agent_soul_knowledge_dataset_ids
SUPPORTED_AGENT_BACKEND_FEATURES = frozenset(
{
@@ -49,7 +48,9 @@ def build_runtime_feature_manifest(agent_soul: AgentSoulConfig) -> dict[str, Any
)
reserved_status = dict.fromkeys(sorted(RESERVED_AGENT_BACKEND_FEATURES), "reserved_not_executed")
reserved_status["knowledge"] = "supported_by_knowledge_layer" if agent_soul.knowledge.sets else "not_configured"
reserved_status["knowledge"] = (
"supported_by_knowledge_layer" if list_configured_knowledge_dataset_ids(agent_soul) else "not_configured"
)
reserved_status["tools.dify_tools"] = "supported_when_config_valid"
reserved_status["tools.cli_tools"] = "supported_by_shell_bootstrap"
reserved_status["env"] = "supported_by_shell_bootstrap"
@@ -65,14 +66,14 @@ def build_runtime_feature_manifest(agent_soul: AgentSoulConfig) -> dict[str, Any
def list_configured_knowledge_dataset_ids(agent_soul: AgentSoulConfig) -> list[str]:
"""Return normalized dataset ids selected by Agent v2 knowledge sets.
"""Return the normalized knowledge dataset ids that can produce a runtime layer.
``build_runtime_feature_manifest()`` and ``build_knowledge_layer_config()``
stay aligned on the set-based contract: DTO validation rejects blank dataset
ids before runtime, so this helper only flattens configured set datasets for
metadata/diagnostic surfaces that still need a dataset-id summary.
must stay aligned: both decide knowledge support from this effective,
non-blank dataset-id set rather than from raw
``agent_soul.knowledge.datasets`` entries.
"""
return list_agent_soul_knowledge_dataset_ids(agent_soul)
return [dataset_id for dataset in agent_soul.knowledge.datasets if (dataset_id := (dataset.id or "").strip())]
def _get_nested(value: dict[str, Any], path: str) -> Any:
@@ -15,16 +15,7 @@ from dify_agent.layers.execution_context import (
DifyExecutionContextLayerConfig,
DifyExecutionContextUserFrom,
)
from dify_agent.layers.knowledge import (
DifyKnowledgeBaseLayerConfig,
DifyKnowledgeDatasetConfig,
DifyKnowledgeMetadataFilteringConfig,
DifyKnowledgeModelConfig,
DifyKnowledgeQueryConfig,
DifyKnowledgeRerankingModelConfig,
DifyKnowledgeRetrievalConfig,
DifyKnowledgeSetConfig,
)
from dify_agent.layers.knowledge import DifyKnowledgeBaseLayerConfig, DifyKnowledgeRetrievalConfig
from dify_agent.layers.shell import (
DifyShellCliToolConfig,
DifyShellEnvVarConfig,
@@ -49,9 +40,7 @@ from graphon.file import FileTransferMethod
from graphon.variables.segments import Segment
from models.agent import Agent, AgentConfigSnapshot, WorkflowAgentNodeBinding
from models.agent_config_entities import (
AgentKnowledgeMetadataFilteringConfig,
AgentKnowledgeModelConfig,
AgentKnowledgeRetrievalConfig,
AgentKnowledgeQueryConfig,
AgentSoulConfig,
DeclaredArrayItem,
DeclaredOutputChildConfig,
@@ -71,12 +60,11 @@ from services.agent.prompt_mentions import (
expand_prompt_mentions,
parse_prompt_mentions,
)
from services.agent.soul_files_service import AgentSoulFilesService
from services.agent_drive_service import decode_drive_mention_ref
from services.agent_drive_service import AgentDriveService, decode_drive_mention_ref
from .output_failure_orchestrator import retry_idempotency_key
from .plugin_tools_builder import WorkflowAgentPluginToolsBuilder, WorkflowAgentPluginToolsBuildError
from .runtime_feature_manifest import build_runtime_feature_manifest
from .runtime_feature_manifest import build_runtime_feature_manifest, list_configured_knowledge_dataset_ids
_DENIED_PERMISSION_STATUSES = frozenset({"unauthorized", "denied", "forbidden", "invalid", "unavailable"})
_DANGEROUS_FLAG_KEYS = ("dangerous", "dangerous_command", "requires_confirmation")
@@ -559,84 +547,42 @@ def build_shell_layer_config(agent_soul: AgentSoulConfig) -> DifyShellLayerConfi
def build_knowledge_layer_config(agent_soul: AgentSoulConfig) -> DifyKnowledgeBaseLayerConfig | None:
"""Map Agent Soul knowledge sets into one Dify knowledge-base layer.
"""Map Agent Soul knowledge config into the fixed Dify knowledge-base layer.
Agent Soul DTO validation owns malformed set rejection. Runtime mapping is
intentionally lossless: every configured set is forwarded with its query
policy, dataset refs, retrieval controls, and metadata-filtering controls.
``score_threshold=None`` means disabled threshold filtering and maps to the
inner retrieval request's ``0.0`` default through the Agent backend DTO.
Normalization intentionally matches the current dify-agent runtime contract:
- blank or missing dataset ids are ignored;
- if no valid dataset ids remain, no knowledge layer is injected;
- retrieval mode is always forced to ``multiple`` in this first wiring pass;
- ``top_k`` falls back to a stable runtime default when the soul omits it;
- ``score_threshold`` is only forwarded when the product config explicitly
enables it, otherwise the layer keeps the disabled/default ``0.0`` value;
- metadata filtering stays at the layer DTO default (disabled).
"""
if not agent_soul.knowledge.sets:
dataset_ids = list_configured_knowledge_dataset_ids(agent_soul)
if not dataset_ids:
return None
query_config = agent_soul.knowledge.query_config
return DifyKnowledgeBaseLayerConfig(
sets=[
DifyKnowledgeSetConfig(
id=knowledge_set.id,
name=knowledge_set.name,
description=knowledge_set.description,
datasets=[
DifyKnowledgeDatasetConfig(
id=dataset.id or "",
name=dataset.name,
description=dataset.description,
)
for dataset in knowledge_set.datasets
],
query=DifyKnowledgeQueryConfig(
mode=cast(Literal["user_query", "generated_query"], knowledge_set.query.mode.value),
value=knowledge_set.query.value,
),
retrieval=_knowledge_retrieval_config(knowledge_set.retrieval),
metadata_filtering=_knowledge_metadata_filtering_config(knowledge_set.metadata_filtering),
)
for knowledge_set in agent_soul.knowledge.sets
],
dataset_ids=dataset_ids,
retrieval=DifyKnowledgeRetrievalConfig(
mode="multiple",
top_k=_knowledge_top_k(query_config),
score_threshold=_knowledge_score_threshold(query_config),
),
)
def _knowledge_retrieval_config(retrieval: AgentKnowledgeRetrievalConfig) -> DifyKnowledgeRetrievalConfig:
return DifyKnowledgeRetrievalConfig(
mode=retrieval.mode,
top_k=retrieval.top_k,
score_threshold=retrieval.score_threshold or 0.0,
reranking_mode=retrieval.reranking_mode,
reranking_enable=retrieval.reranking_enable,
reranking_model=DifyKnowledgeRerankingModelConfig(
provider=retrieval.reranking_model.provider,
model=retrieval.reranking_model.model,
)
if retrieval.reranking_model is not None
else None,
weights=cast(dict[str, Any], retrieval.weights.model_dump(mode="json", exclude_none=True))
if retrieval.weights is not None
else None,
model=_knowledge_model_config(retrieval.model),
)
def _knowledge_top_k(query_config: AgentKnowledgeQueryConfig) -> int:
top_k = query_config.top_k
return top_k if isinstance(top_k, int) and top_k >= 1 else 4
def _knowledge_metadata_filtering_config(
metadata_filtering: AgentKnowledgeMetadataFilteringConfig,
) -> DifyKnowledgeMetadataFilteringConfig:
return DifyKnowledgeMetadataFilteringConfig(
mode=metadata_filtering.mode,
model_config=_knowledge_model_config(metadata_filtering.metadata_model_config),
conditions=cast(Any, metadata_filtering.conditions.model_dump(mode="json"))
if metadata_filtering.conditions is not None
else None,
)
def _knowledge_model_config(model: AgentKnowledgeModelConfig | None) -> DifyKnowledgeModelConfig | None:
if model is None:
return None
return DifyKnowledgeModelConfig(
provider=model.provider,
name=model.name,
mode=model.mode,
completion_params=model.completion_params,
)
def _knowledge_score_threshold(query_config: AgentKnowledgeQueryConfig) -> float:
if query_config.score_threshold_enabled and query_config.score_threshold is not None:
return query_config.score_threshold
return 0.0
def build_ask_human_layer_config(agent_soul: AgentSoulConfig) -> DifyAskHumanLayerConfig | None:
@@ -670,19 +616,13 @@ def build_drive_aware_soul_mention_resolver(
tenant_id: str,
agent_id: str,
):
"""Resolve skill/file mentions against versioned Agent Soul refs and everything else via Agent Soul."""
"""Resolve skill/file mentions against the agent drive and everything else via Agent Soul."""
base_resolver = build_soul_mention_resolver(agent_soul)
skill_names_by_key = {
skill.skill_md_key: skill.name
for skill in agent_soul.files.skills
if skill.skill_md_key and skill.name
}
file_names_by_key = {
file_ref.drive_key: file_ref.name or file_ref.drive_key.rsplit("/", 1)[-1]
for file_ref in agent_soul.files.files
if file_ref.drive_key
}
drive_service = AgentDriveService()
skill_catalog = drive_service.list_skills(tenant_id=tenant_id, agent_id=agent_id)
skill_names_by_key = {skill["skill_md_key"]: skill["name"] for skill in skill_catalog}
drive_keys = {item["key"] for item in drive_service.manifest(tenant_id=tenant_id, agent_id=agent_id)}
def _resolve(mention: object) -> str | None:
if not hasattr(mention, "kind") or not hasattr(mention, "ref_id"):
@@ -695,7 +635,9 @@ def build_drive_aware_soul_mention_resolver(
return skill_names_by_key.get(decoded_key) or label or decoded_key
if kind == MentionKind.FILE:
decoded_key = decode_drive_mention_ref(ref_id)
return file_names_by_key.get(decoded_key) or label or decoded_key
if decoded_key in drive_keys:
return decoded_key.rsplit("/", 1)[-1]
return label or decoded_key
return base_resolver(cast(Any, mention))
return _resolve
@@ -707,7 +649,7 @@ def build_drive_layer_config(
tenant_id: str,
agent_id: str | None,
) -> tuple[DifyDriveLayerConfig | None, list[dict[str, str]]]:
"""Derive drive runtime catalog + prompt-mentioned eager-pull keys from Agent Soul refs."""
"""Derive drive runtime catalog + prompt-mentioned eager-pull keys from the drive."""
mentioned_drive_refs = [
decode_drive_mention_ref(mention.ref_id)
@@ -726,22 +668,10 @@ def build_drive_layer_config(
}
]
skills_catalog = [
{
"path": skill.path or AgentSoulFilesService.skill_path_from_key(skill.skill_md_key),
"name": skill.name or skill.path or skill.skill_md_key,
"description": skill.description or "",
"skill_md_key": skill.skill_md_key,
"archive_key": skill.full_archive_key,
}
for skill in agent_soul.files.skills
if skill.skill_md_key
]
soul_file_keys = {
key
for key in AgentSoulFilesService.allowed_drive_keys(agent_soul)
if key not in {skill["skill_md_key"] for skill in skills_catalog}
}
drive_service = AgentDriveService()
skills_catalog = drive_service.list_skills(tenant_id=tenant_id, agent_id=agent_id)
manifest_items = drive_service.manifest(tenant_id=tenant_id, agent_id=agent_id)
manifest_by_key = {item["key"]: item for item in manifest_items}
skill_keys = {skill["skill_md_key"] for skill in skills_catalog}
warnings: list[dict[str, str]] = []
mentioned_skill_keys: list[str] = []
@@ -750,7 +680,7 @@ def build_drive_layer_config(
if drive_key in skill_keys:
mentioned_skill_keys.append(drive_key)
continue
if drive_key in soul_file_keys:
if drive_key in manifest_by_key:
mentioned_file_keys.append(drive_key)
continue
warnings.append(
@@ -18,7 +18,6 @@ from models.agent_config_entities import (
)
from models.model import UploadFile
from models.workflow import Workflow
from services.agent.knowledge_datasets import list_missing_tenant_knowledge_dataset_ids
from .entities import DifyAgentNodeData
@@ -147,7 +146,6 @@ class WorkflowAgentNodeValidator:
)
cls._validate_agent_soul_env(binding=binding, agent_soul=agent_soul)
cls._validate_agent_soul_tools(binding=binding, agent_soul=agent_soul)
cls._validate_agent_soul_knowledge(binding=binding, agent_soul=agent_soul)
node_job = WorkflowNodeJobConfig.model_validate(binding.node_job_config_dict)
cls.validate_node_job(session=session, binding=binding, node_job=node_job, topology=topology)
@@ -366,24 +364,6 @@ class WorkflowAgentNodeValidator:
)
cli_tool_names.add(normalized_name)
@classmethod
def _validate_agent_soul_knowledge(
cls,
*,
binding: WorkflowAgentNodeBinding,
agent_soul: AgentSoulConfig,
) -> None:
"""Validate knowledge set dataset rows against the publishing tenant."""
missing_ids = list_missing_tenant_knowledge_dataset_ids(
tenant_id=binding.tenant_id,
agent_soul=agent_soul,
)
if missing_ids:
raise WorkflowAgentNodeValidationError(
f"Workflow Agent node {binding.node_id} references missing or out-of-scope knowledge datasets: "
f"{', '.join(missing_ids)}."
)
@classmethod
def _validate_agent_soul_env(
cls,
+96 -20
View File
@@ -2,8 +2,8 @@
This checker intentionally stays conservative. It only reports a hard schema
mismatch when both sides are statically known for the same 2xx status code:
a documented ``@ns.response(..., Model)`` and an actual ``dump_response(Model, ...)``
or ``Model.model_validate(...).model_dump()`` return.
a documented ``@ns.response(..., Model)`` and an actual ``dump_response(Model, ...)``,
``Model(...).model_dump()``, or ``Model.model_validate(...).model_dump()`` return.
Raw dictionaries, raw lists, ``None`` responses, streaming helpers, missing
response schemas, and returns with non-literal status codes are classified as
@@ -28,6 +28,7 @@ from typing import Any, Literal
HTTP_METHODS = {"delete", "get", "head", "options", "patch", "post", "put"}
NO_BODY_STATUSES = {HTTPStatus.NO_CONTENT.value, HTTPStatus.RESET_CONTENT.value, HTTPStatus.NOT_MODIFIED.value}
DEFAULT_CONTROLLER_DIRS = ("controllers/console", "controllers/service_api", "controllers/web")
IGNORE_COMMENT_MARKERS = ("response-contract:ignore",)
type Classification = Literal["valid", "mismatch", "unknown", "refactorable"]
type ActualKind = Literal[
@@ -41,6 +42,7 @@ type ActualKind = Literal[
"unknown",
]
type MethodNode = ast.FunctionDef | ast.AsyncFunctionDef
type ModelValueSource = Literal["constructor", "model_validate"]
HTTP_STATUS_NAMES = {status.name: status.value for status in HTTPStatus}
HTTP_STATUS_NAMES.update({f"HTTP_{status.value}_{status.name}": status.value for status in HTTPStatus})
@@ -109,18 +111,22 @@ class VariableAssignmentSummary:
"""Track whether a local name is safe to treat as one specific response model."""
known_models: set[str] = field(default_factory=set)
known_sources: set[ModelValueSource] = field(default_factory=set)
has_unknown_assignment: bool = False
def add_known(self, model: str) -> None:
def add_known(self, model: str, source: ModelValueSource) -> None:
self.known_models.add(model)
self.known_sources.add(source)
def add_unknown(self) -> None:
self.has_unknown_assignment = True
def single_known_model(self) -> str | None:
def single_known_model(self) -> tuple[str, ModelValueSource] | None:
if self.has_unknown_assignment or len(self.known_models) != 1:
return None
return next(iter(self.known_models))
model = next(iter(self.known_models))
source: ModelValueSource = "constructor" if self.known_sources == {"constructor"} else "model_validate"
return model, source
def dotted_name(node: ast.AST) -> str | None:
@@ -249,6 +255,12 @@ def model_name_from_model_validate_call(node: ast.AST) -> str | None:
return None
def model_value_from_model_validate_call(node: ast.AST) -> tuple[str, ModelValueSource] | None:
if model_name := model_name_from_model_validate_call(node):
return model_name, "model_validate"
return None
def model_name_from_constructor_call(node: ast.AST) -> str | None:
if not isinstance(node, ast.Call):
return None
@@ -257,6 +269,12 @@ def model_name_from_constructor_call(node: ast.AST) -> str | None:
return None
def model_value_from_constructor_call(node: ast.AST) -> tuple[str, ModelValueSource] | None:
if model_name := model_name_from_constructor_call(node):
return model_name, "constructor"
return None
def model_name_from_model_dump(node: ast.AST) -> str | None:
if not isinstance(node, ast.Call) or not isinstance(node.func, ast.Attribute) or node.func.attr != "model_dump":
return None
@@ -272,6 +290,10 @@ def model_name_from_model_value(node: ast.AST) -> str | None:
return model_name_from_model_validate_call(node) or model_name_from_constructor_call(node)
def model_value_from_model_value(node: ast.AST) -> tuple[str, ModelValueSource] | None:
return model_value_from_model_validate_call(node) or model_value_from_constructor_call(node)
def model_name_from_dump_response(node: ast.AST) -> str | None:
if not isinstance(node, ast.Call):
return None
@@ -287,7 +309,7 @@ def model_name_from_dump_response(node: ast.AST) -> str | None:
def actual_kind_from_expr(
expr: ast.AST | None, variable_models: dict[str, str] | None = None
expr: ast.AST | None, variable_models: dict[str, tuple[str, ModelValueSource]] | None = None
) -> tuple[ActualKind, str | None]:
if expr is None:
return "none", None
@@ -299,10 +321,14 @@ def actual_kind_from_expr(
if isinstance(expr, ast.Call) and isinstance(expr.func, ast.Attribute) and expr.func.attr == "model_dump":
dumped_value = expr.func.value
if isinstance(dumped_value, ast.Name) and variable_models:
# A variable dump can match today, but it bypasses dump_response and
# is easier to drift; keep it visible as refactorable.
model_name = variable_models.get(dumped_value.id)
if model_name:
model_assignment = variable_models.get(dumped_value.id)
if model_assignment:
model_name, source = model_assignment
if source == "constructor":
return "model", model_name
# A variable dump from model_validate can match today, but it
# bypasses dump_response and is easier to drift; keep it visible
# as refactorable.
return "model_dump_variable", model_name
model_dump_model = model_name_from_model_dump(expr)
@@ -325,7 +351,9 @@ def actual_kind_from_expr(
return "unknown", None
def actual_response_from_return(return_node: ast.Return, variable_models: dict[str, str]) -> ActualResponse:
def actual_response_from_return(
return_node: ast.Return, variable_models: dict[str, tuple[str, ModelValueSource]]
) -> ActualResponse:
status: int | None = 200
body_expr = return_node.value
@@ -363,18 +391,21 @@ def target_names(target: ast.AST) -> Iterable[str]:
def record_assignment(
assignments: defaultdict[str, VariableAssignmentSummary], targets: Iterable[str], model_name: str | None
assignments: defaultdict[str, VariableAssignmentSummary],
targets: Iterable[str],
model_assignment: tuple[str, ModelValueSource] | None,
) -> None:
for target in targets:
if model_name is None:
if model_assignment is None:
# Once a name receives an unknown value, later model_dump() calls on it
# are no longer a reliable signal for the returned schema.
assignments[target].add_unknown()
else:
assignments[target].add_known(model_name)
model_name, source = model_assignment
assignments[target].add_known(model_name, source)
def variable_model_assignments_for_method(method: MethodNode) -> dict[str, str]:
def variable_model_assignments_for_method(method: MethodNode) -> dict[str, tuple[str, ModelValueSource]]:
"""Infer local variables that are unambiguously assigned one response model."""
assignments: defaultdict[str, VariableAssignmentSummary] = defaultdict(VariableAssignmentSummary)
@@ -385,10 +416,10 @@ def variable_model_assignments_for_method(method: MethodNode) -> dict[str, str]:
record_assignment(
assignments,
(name for target in targets for name in target_names(target)),
model_name_from_model_value(value),
model_value_from_model_value(value),
)
case ast.AnnAssign(target=target, value=value) if value is not None:
record_assignment(assignments, target_names(target), model_name_from_model_value(value))
record_assignment(assignments, target_names(target), model_value_from_model_value(value))
case ast.AugAssign(target=target) | ast.For(target=target) | ast.AsyncFor(target=target):
# Mutation and loop targets overwrite prior values with runtime-dependent data.
record_assignment(assignments, target_names(target), None)
@@ -399,9 +430,13 @@ def variable_model_assignments_for_method(method: MethodNode) -> dict[str, str]:
case ast.ExceptHandler(name=name) if name:
assignments[name].add_unknown()
case ast.NamedExpr(target=target, value=value):
record_assignment(assignments, target_names(target), model_name_from_model_value(value))
record_assignment(assignments, target_names(target), model_value_from_model_value(value))
return {name: model for name, summary in assignments.items() if (model := summary.single_known_model()) is not None}
return {
name: assignment
for name, summary in assignments.items()
if (assignment := summary.single_known_model()) is not None
}
def actual_responses_for_method(method: MethodNode) -> list[ActualResponse]:
@@ -545,13 +580,52 @@ def iter_controller_files(paths: Iterable[Path]) -> Iterable[Path]:
yield from sorted(child for child in path.rglob("*.py") if child.is_file())
def node_start_lineno(node: ast.ClassDef | MethodNode) -> int:
decorator_lines = [decorator.lineno for decorator in node.decorator_list]
if decorator_lines:
return min(decorator_lines)
return node.lineno
def line_has_ignore_marker(line: str) -> bool:
_, marker, comment = line.partition("#")
if not marker:
return False
normalized = comment.lower()
return any(ignore_marker in normalized for ignore_marker in IGNORE_COMMENT_MARKERS)
def node_has_ignore_comment(lines: Sequence[str], node: ast.ClassDef | MethodNode) -> bool:
start = node_start_lineno(node)
end = node.end_lineno or node.lineno
if any(line_has_ignore_marker(line) for line in lines[start - 1 : end]):
return True
line_index = start - 2
while line_index >= 0:
stripped = lines[line_index].strip()
if not stripped:
line_index -= 1
continue
if not stripped.startswith("#"):
break
if line_has_ignore_marker(lines[line_index]):
return True
line_index -= 1
return False
def checks_for_file(file_path: Path, repo_root: Path) -> list[ContractCheck]:
module = ast.parse(file_path.read_text(encoding="utf-8"), filename=str(file_path))
source = file_path.read_text(encoding="utf-8")
lines = source.splitlines()
module = ast.parse(source, filename=str(file_path))
checks: list[ContractCheck] = []
for node in module.body:
if not isinstance(node, ast.ClassDef):
continue
if node_has_ignore_comment(lines, node):
continue
class_routes = routes_from_decorators(node.decorator_list)
class_documented = response_docs_from_decorators(node.decorator_list)
@@ -559,6 +633,8 @@ def checks_for_file(file_path: Path, repo_root: Path) -> list[ContractCheck]:
for item in node.body:
if not isinstance(item, ast.FunctionDef | ast.AsyncFunctionDef) or item.name not in HTTP_METHODS:
continue
if node_has_ignore_comment(lines, item):
continue
routes = routes_from_decorators(item.decorator_list) or class_routes
if not routes:
+2 -38
View File
@@ -6,7 +6,6 @@ from pydantic import Field, field_validator
from fields.base import ResponseModel
from libs.helper import to_timestamp
from models.agent import (
AgentConfigDraftType,
AgentConfigRevisionOperation,
AgentIconType,
AgentKind,
@@ -17,10 +16,8 @@ from models.agent import (
)
from models.agent_config_entities import (
AgentCliToolConfig,
AgentFileRefConfig,
AgentHumanContactConfig,
AgentKnowledgeDatasetConfig,
AgentSkillRefConfig,
AgentSoulConfig,
DeclaredOutputConfig,
DeclaredOutputType,
@@ -50,18 +47,6 @@ class AgentConfigSnapshotSummaryResponse(ResponseModel):
created_at: int | None = None
class AgentConfigDraftSummaryResponse(ResponseModel):
id: str
agent_id: str
draft_type: AgentConfigDraftType
account_id: str | None = None
base_snapshot_id: str | None = None
created_by: str | None = None
updated_by: str | None = None
created_at: int | None = None
updated_at: int | None = None
class AgentPublishedReferenceResponse(ResponseModel):
app_id: str
app_name: str
@@ -307,18 +292,12 @@ class AgentConfigSnapshotListResponse(ResponseModel):
class AgentConfigSnapshotRestoreResponse(ResponseModel):
result: Literal["success"]
active_config_snapshot_id: str
draft_config_id: str | None = None
restored_version_id: str | None = None
class AgentComposerAgentResponse(ResponseModel):
id: str
name: str
description: str
role: str | None = None
icon_type: str | None = None
icon: str | None = None
icon_background: str | None = None
scope: AgentScope
status: AgentStatus
active_config_snapshot_id: str | None = None
@@ -371,8 +350,7 @@ class WorkflowAgentComposerResponse(ResponseModel):
class AgentAppComposerResponse(ResponseModel):
variant: Literal[ComposerVariant.AGENT_APP]
agent: AgentComposerAgentResponse
active_config_snapshot: AgentConfigSnapshotSummaryResponse | None = None
draft: AgentConfigDraftSummaryResponse | None = None
active_config_snapshot: AgentConfigSnapshotSummaryResponse
agent_soul: AgentSoulConfig
save_options: list[ComposerSaveStrategy]
validation: "ComposerValidationFindingsResponse | None" = None
@@ -422,25 +400,11 @@ class AgentComposerNodeJobCandidatesResponse(ResponseModel):
human_contacts: list[AgentHumanContactConfig] = Field(default_factory=list)
class AgentComposerKnowledgeDatasetCandidateResponse(AgentKnowledgeDatasetConfig):
missing: bool = False
class AgentComposerKnowledgeSetCandidateResponse(ResponseModel):
id: str
name: str
description: str | None = None
datasets: list[AgentComposerKnowledgeDatasetCandidateResponse] = Field(default_factory=list)
missing_dataset_ids: list[str] = Field(default_factory=list)
class AgentComposerSoulCandidatesResponse(ResponseModel):
dify_tools: list[AgentComposerDifyToolCandidateResponse] = Field(default_factory=list)
cli_tools: list[AgentCliToolConfig] = Field(default_factory=list)
knowledge_sets: list[AgentComposerKnowledgeSetCandidateResponse] = Field(default_factory=list)
knowledge_datasets: list[AgentKnowledgeDatasetConfig] = Field(default_factory=list)
human_contacts: list[AgentHumanContactConfig] = Field(default_factory=list)
skills: list[AgentSkillRefConfig] = Field(default_factory=list)
files: list[AgentFileRefConfig] = Field(default_factory=list)
class AgentComposerCandidatesResponse(ResponseModel):
+2 -1
View File
@@ -7,7 +7,8 @@ class ResponseModel(BaseModel):
model_config = ConfigDict(
from_attributes=True,
extra="ignore",
populate_by_name=True,
validate_by_name=True,
validate_by_alias=True,
serialize_by_alias=True,
protected_namespaces=(),
)
+26 -10
View File
@@ -236,6 +236,16 @@ class TokenExpiredError(Exception):
"""Hard-expire bookkeeping is the resolver's job before raising."""
class NegativeCache(StrEnum):
"""Negative cache markers. ``EXPIRED`` is distinct from ``INVALID`` so a
retry inside ``NEGATIVE_TTL`` still reports expiry instead of collapsing
into a generic unknown-token miss.
"""
INVALID = "invalid"
EXPIRED = "expired"
# ============================================================================
# Registry
# ============================================================================
@@ -343,13 +353,15 @@ class OAuthAccessTokenResolver:
def _cache_key(self, token_hash: str) -> str:
return TOKEN_CACHE_KEY_FMT.format(hash=token_hash)
def cache_get(self, token_hash: str) -> ResolvedRow | None | Literal["invalid"]:
def cache_get(self, token_hash: str) -> ResolvedRow | None | NegativeCache:
raw = self._redis.get(self._cache_key(token_hash))
if raw is None:
return None
text = raw.decode() if isinstance(raw, (bytes, bytearray)) else raw
if text == "invalid":
return "invalid"
try:
return NegativeCache(text)
except ValueError:
pass
try:
return ResolvedRow.from_cache(json.loads(text))
except (ValueError, KeyError):
@@ -363,8 +375,8 @@ class OAuthAccessTokenResolver:
json.dumps(row.to_cache()),
)
def cache_set_negative(self, token_hash: str) -> None:
self._redis.setex(self._cache_key(token_hash), self._negative_ttl, "invalid")
def cache_set_negative(self, token_hash: str, marker: NegativeCache = NegativeCache.INVALID) -> None:
self._redis.setex(self._cache_key(token_hash), self._negative_ttl, str(marker))
def hard_expire(self, session: Session, row_id: uuid.UUID | str, token_hash: str) -> None:
"""Atomic CAS — only the worker that flips revoked_at emits audit;
@@ -385,7 +397,7 @@ class OAuthAccessTokenResolver:
extra={"audit": True, "token_id": str(row_id)},
)
self._redis.delete(self._cache_key(token_hash))
self.cache_set_negative(token_hash)
self.cache_set_negative(token_hash, NegativeCache.EXPIRED)
class _VariantResolver:
@@ -395,9 +407,11 @@ class _VariantResolver:
def resolve(self, token_hash: str) -> ResolvedRow | None:
cached = self._parent.cache_get(token_hash)
if cached == "invalid":
if isinstance(cached, NegativeCache):
if cached is NegativeCache.EXPIRED:
raise TokenExpiredError("token_expired")
return None
if cached is not None and not isinstance(cached, str):
if cached is not None:
if not self._matches_variant(cached):
return None
return cached
@@ -413,7 +427,7 @@ class _VariantResolver:
now = datetime.now(UTC)
if row.expires_at is not None and row.expires_at <= now:
self._parent.hard_expire(session, row.id, token_hash)
return None
raise TokenExpiredError("token_expired")
if not self._matches_variant_model(row):
logger.error(
@@ -472,7 +486,7 @@ def record_layer0_verdict(token_hash: str, tenant_id: str, verdict: bool) -> Non
if raw is None:
return
text = raw.decode() if isinstance(raw, (bytes, bytearray)) else raw
if text == "invalid":
if text in (NegativeCache.INVALID, NegativeCache.EXPIRED):
return
try:
data = json.loads(text)
@@ -601,6 +615,8 @@ def validate_bearer(*, accept: frozenset[Accepts]) -> Callable[[Callable[_DP, _D
try:
ctx = get_authenticator().authenticate(token)
except TokenExpiredError:
raise Unauthorized("token_expired")
except InvalidBearerError as e:
raise Unauthorized(str(e))
+21 -3
View File
@@ -2,9 +2,10 @@
from __future__ import annotations
import argparse
import sys
_DIAGNOSTIC_PREFIXES = ("ERROR ", "WARNING ")
_DIAGNOSTIC_PREFIXES = ("ERROR ", "WARN ", "WARNING ")
_LOCATION_PREFIX = "-->"
@@ -13,7 +14,7 @@ def extract_diagnostics(raw_output: str) -> str:
The full pyrefly output includes code excerpts and carets, which create noisy
diffs. This helper keeps only:
- diagnostic headline lines (``ERROR ...`` / ``WARNING ...``)
- diagnostic headline lines (``ERROR ...`` / ``WARN ...`` / ``WARNING ...``)
- the following location line (``--> path:line:column``), when present
"""
@@ -36,11 +37,28 @@ def extract_diagnostics(raw_output: str) -> str:
return "\n".join(diagnostics) + "\n"
def render_diagnostics(raw_output: str, exit_code: int) -> str:
"""Render concise diagnostics and fall back to raw output on unmatched failures."""
diagnostics = extract_diagnostics(raw_output)
if diagnostics:
return diagnostics
if exit_code != 0:
return raw_output
return ""
def main() -> int:
"""Read pyrefly output from stdin and print normalized diagnostics."""
parser = argparse.ArgumentParser()
parser.add_argument("--status", type=int, default=0)
args = parser.parse_args()
raw_output = sys.stdin.read()
sys.stdout.write(extract_diagnostics(raw_output))
sys.stdout.write(render_diagnostics(raw_output, exit_code=args.status))
return 0
@@ -1,135 +0,0 @@
"""add agent config drafts
Revision ID: e4f5a6b7c8d9
Revises: d9e8f7a6b5c4
Create Date: 2026-06-24 20:15:00.000000
"""
from datetime import UTC, datetime
import sqlalchemy as sa
from alembic import op
import models
from libs.uuid_utils import uuidv7
# revision identifiers, used by Alembic.
revision = "e4f5a6b7c8d9"
down_revision = "d9e8f7a6b5c4"
branch_labels = None
depends_on = None
def _is_pg(conn) -> bool:
return conn.dialect.name == "postgresql"
def _uuid_column(name: str, *, nullable: bool = False, primary_key: bool = False) -> sa.Column:
kwargs = {"nullable": nullable, "primary_key": primary_key}
if primary_key and _is_pg(op.get_bind()):
kwargs["server_default"] = sa.text("uuidv7()")
return sa.Column(name, models.types.StringUUID(), **kwargs)
def upgrade():
op.create_table(
"agent_config_drafts",
_uuid_column("id", primary_key=True),
sa.Column("tenant_id", models.types.StringUUID(), nullable=False),
sa.Column("agent_id", models.types.StringUUID(), nullable=False),
sa.Column("draft_type", sa.String(length=32), nullable=False),
sa.Column("account_id", models.types.StringUUID(), nullable=True),
sa.Column("draft_owner_key", sa.String(length=255), server_default="", nullable=False),
sa.Column("base_snapshot_id", models.types.StringUUID(), nullable=True),
sa.Column("config_snapshot", models.types.LongText(), nullable=False),
sa.Column("created_by", models.types.StringUUID(), nullable=True),
sa.Column("updated_by", models.types.StringUUID(), nullable=True),
sa.Column("created_at", sa.DateTime(), server_default=sa.func.current_timestamp(), nullable=False),
sa.Column("updated_at", sa.DateTime(), server_default=sa.func.current_timestamp(), nullable=False),
sa.PrimaryKeyConstraint("id", name=op.f("agent_config_draft_pkey")),
sa.UniqueConstraint(
"tenant_id",
"agent_id",
"draft_type",
"draft_owner_key",
name=op.f("agent_config_draft_agent_type_account_unique"),
),
)
op.create_index("agent_config_draft_tenant_agent_idx", "agent_config_drafts", ["tenant_id", "agent_id"])
op.create_index(
"agent_config_draft_base_snapshot_idx",
"agent_config_drafts",
["tenant_id", "base_snapshot_id"],
)
bind = op.get_bind()
now = datetime.now(UTC).replace(tzinfo=None)
if bind.dialect.name == "postgresql":
op.execute(
sa.text(
"""
INSERT INTO agent_config_drafts (
id, tenant_id, agent_id, draft_type, account_id, draft_owner_key, base_snapshot_id,
config_snapshot, created_by, updated_by, created_at, updated_at
)
SELECT
uuidv7(), a.tenant_id, a.id, 'draft', NULL, '', s.id,
s.config_snapshot, a.created_by, a.updated_by, CURRENT_TIMESTAMP, CURRENT_TIMESTAMP
FROM agents a
JOIN agent_config_snapshots s
ON s.tenant_id = a.tenant_id
AND s.agent_id = a.id
AND s.id = a.active_config_snapshot_id
WHERE a.active_config_snapshot_id IS NOT NULL
"""
)
)
else:
agents = bind.execute(
sa.text(
"""
SELECT
a.tenant_id, a.id AS agent_id, a.created_by, a.updated_by,
s.id AS snapshot_id, s.config_snapshot
FROM agents a
JOIN agent_config_snapshots s
ON s.tenant_id = a.tenant_id
AND s.agent_id = a.id
AND s.id = a.active_config_snapshot_id
WHERE a.active_config_snapshot_id IS NOT NULL
"""
)
).mappings()
for row in agents:
bind.execute(
sa.text(
"""
INSERT INTO agent_config_drafts (
id, tenant_id, agent_id, draft_type, account_id, draft_owner_key, base_snapshot_id,
config_snapshot, created_by, updated_by, created_at, updated_at
)
VALUES (
:id, :tenant_id, :agent_id, 'draft', NULL, '', :snapshot_id,
:config_snapshot, :created_by, :updated_by, :created_at, :updated_at
)
"""
),
{
"id": str(uuidv7()),
"tenant_id": row["tenant_id"],
"agent_id": row["agent_id"],
"snapshot_id": row["snapshot_id"],
"config_snapshot": row["config_snapshot"],
"created_by": row["created_by"],
"updated_by": row["updated_by"],
"created_at": now,
"updated_at": now,
},
)
def downgrade():
op.drop_index("agent_config_draft_base_snapshot_idx", table_name="agent_config_drafts")
op.drop_index("agent_config_draft_tenant_agent_idx", table_name="agent_config_drafts")
op.drop_table("agent_config_drafts")
-4
View File
@@ -10,8 +10,6 @@ from .account import (
)
from .agent import (
Agent,
AgentConfigDraft,
AgentConfigDraftType,
AgentConfigRevision,
AgentConfigRevisionOperation,
AgentConfigSnapshot,
@@ -156,8 +154,6 @@ __all__ = [
"AccountStatus",
"AccountTrialAppRecord",
"Agent",
"AgentConfigDraft",
"AgentConfigDraftType",
"AgentConfigRevision",
"AgentConfigRevisionOperation",
"AgentConfigSnapshot",
+3 -54
View File
@@ -85,17 +85,6 @@ class AgentConfigRevisionOperation(StrEnum):
SAVE_TO_ROSTER = "save_to_roster"
# Switches the Agent's current published config back to an existing version.
RESTORE_VERSION = "restore_version"
# Publishes the editable Agent Soul draft as a new immutable version.
PUBLISH_DRAFT = "publish_draft"
class AgentConfigDraftType(StrEnum):
"""Editable Agent Soul draft workspace type."""
# Shared Agent Console draft edited by users before publishing.
DRAFT = "draft"
# Per-editor build draft mutated during debug/build mode.
DEBUG_BUILD = "debug_build"
class WorkflowAgentBindingType(StrEnum):
@@ -221,46 +210,6 @@ class AgentDebugConversation(DefaultFieldsMixin, Base):
conversation_id: Mapped[str] = mapped_column(StringUUID, nullable=False)
class AgentConfigDraft(DefaultFieldsMixin, Base):
"""Editable Agent Soul draft separated from immutable published snapshots."""
__tablename__ = "agent_config_drafts"
__table_args__ = (
sa.PrimaryKeyConstraint("id", name="agent_config_draft_pkey"),
UniqueConstraint(
"tenant_id",
"agent_id",
"draft_type",
"draft_owner_key",
name="agent_config_draft_agent_type_account_unique",
),
Index("agent_config_draft_tenant_agent_idx", "tenant_id", "agent_id"),
Index("agent_config_draft_base_snapshot_idx", "tenant_id", "base_snapshot_id"),
)
tenant_id: Mapped[str] = mapped_column(StringUUID, nullable=False)
agent_id: Mapped[str] = mapped_column(StringUUID, nullable=False)
draft_type: Mapped[AgentConfigDraftType] = mapped_column(
EnumText(AgentConfigDraftType, length=32), nullable=False
)
account_id: Mapped[str | None] = mapped_column(StringUUID, nullable=True)
draft_owner_key: Mapped[str] = mapped_column(String(255), nullable=False, default="")
base_snapshot_id: Mapped[str | None] = mapped_column(StringUUID, nullable=True)
config_snapshot: Mapped[Any] = mapped_column(JSONModelColumn(AgentSoulConfig), nullable=False)
created_by: Mapped[str | None] = mapped_column(StringUUID, nullable=True)
updated_by: Mapped[str | None] = mapped_column(StringUUID, nullable=True)
@property
def config_snapshot_dict(self) -> dict[str, Any]:
if not self.config_snapshot:
return {}
if hasattr(self.config_snapshot, "model_dump"):
return self.config_snapshot.model_dump(mode="json")
if isinstance(self.config_snapshot, str):
return json.loads(self.config_snapshot)
return dict(self.config_snapshot)
class AgentConfigSnapshot(DefaultFieldsMixin, Base):
"""Immutable Agent Soul snapshot.
@@ -406,9 +355,9 @@ class AgentRuntimeSession(DefaultFieldsMixin, Base):
agent_config_snapshot_id / composition_layer_specs`` columns are set.
- Agent App conversations: ``owner_type = conversation``; the
``conversation_id`` column is set and the workflow columns stay NULL.
Runtime state is scoped by ``agent_config_snapshot_id``. For published
web/API runs this points to an immutable AgentConfigSnapshot; for console
debugger/build runs it points to the editable AgentConfigDraft row.
Published/web/API runs scope runtime state by ``agent_config_snapshot_id``;
console debugger runs may keep it NULL so prompt-only draft saves can reuse
the same preview conversation state while executing the latest Agent Soul.
The snapshot is runtime state returned by Agent backend, kept separate from
Agent Soul snapshots and workflow node-job config.
+8 -189
View File
@@ -2,11 +2,10 @@ from __future__ import annotations
import re
from enum import StrEnum
from typing import Annotated, Any, Final, Literal, Self
from typing import Annotated, Any, Final, Literal
from pydantic import BaseModel, ConfigDict, Field, WithJsonSchema, field_validator, model_validator
from core.rag.entities.metadata_entities import ConditionValue, SupportedComparisonOperator
from core.workflow.file_reference import is_canonical_file_reference
from graphon.file import FileTransferMethod
@@ -160,22 +159,6 @@ class AgentSkillRefConfig(AgentFlexibleConfig):
# Zip member path listing from standardization (ENG-371): lets infer-tools
# show the model strong signals like ``scripts/*.sh`` without unpacking.
manifest_files: list[str] | None = None
# Versioned drive KV entries that belong to this skill package. The drive
# table remains the storage index, but Agent Soul owns which concrete file
# ids are visible for a snapshot/version.
file_refs: list[AgentFileRefConfig] = Field(default_factory=list)
class AgentSoulFilesConfig(BaseModel):
"""Versioned Agent Soul references to drive-backed skills and files.
File bytes and drive value pointers stay in ``agent_drive_files``. This
section records which drive keys belong to one Agent Soul snapshot so version
restore/copy/runtime use the same skills/files view the user published.
"""
skills: list[AgentSkillRefConfig] = Field(default_factory=list)
files: list[AgentFileRefConfig] = Field(default_factory=list)
class AgentPermissionConfig(BaseModel):
@@ -253,161 +236,17 @@ class AgentCliToolConfig(AgentFlexibleConfig):
inferred_from: str | None = Field(default=None, max_length=255)
class AgentKnowledgeDatasetConfig(BaseModel):
model_config = ConfigDict(extra="forbid")
class AgentKnowledgeDatasetConfig(AgentFlexibleConfig):
id: str | None = Field(default=None, max_length=255)
name: str | None = Field(default=None, max_length=255)
description: str | None = None
class AgentKnowledgeQueryConfig(BaseModel):
"""Per-set query policy for Agent v2 knowledge retrieval.
Agent v2 stores knowledge as explicit ``knowledge.sets`` rather than the
legacy flat ``datasets`` / ``query_mode`` / ``query_config`` shape. Each
set owns its own query policy, so ``user_query`` must carry an explicit
``value`` while ``generated_query`` leaves that value empty.
"""
model_config = ConfigDict(extra="forbid")
mode: AgentKnowledgeQueryMode
value: str | None = None
@model_validator(mode="after")
def validate_query(self) -> Self:
if self.mode == AgentKnowledgeQueryMode.USER_QUERY and not (self.value or "").strip():
raise ValueError("knowledge query.value is required for user_query mode")
return self
class AgentKnowledgeModelConfig(BaseModel):
model_config = ConfigDict(extra="forbid")
provider: str = Field(min_length=1, max_length=255)
name: str = Field(min_length=1, max_length=255)
mode: str = Field(min_length=1, max_length=64)
completion_params: dict[str, Any] = Field(default_factory=dict)
class AgentKnowledgeRerankingModelConfig(BaseModel):
model_config = ConfigDict(extra="forbid")
provider: str = Field(min_length=1, max_length=255)
model: str = Field(min_length=1, max_length=255)
class AgentKnowledgeWeightedScoreConfig(AgentFlexibleConfig):
weight_type: str | None = Field(default=None, max_length=64)
vector_setting: dict[str, Any] | None = None
keyword_setting: dict[str, Any] | None = None
class AgentKnowledgeRetrievalConfig(BaseModel):
"""Per-set retrieval policy for Agent v2 knowledge retrieval.
Retrieval settings now live on each knowledge set instead of one shared
flat config. A set may use either ``multiple`` retrieval with ``top_k`` or
``single`` retrieval with a required model config.
"""
model_config = ConfigDict(extra="forbid")
mode: Literal["single", "multiple"]
class AgentKnowledgeQueryConfig(AgentFlexibleConfig):
query: str | None = None
top_k: int | None = Field(default=None, ge=1)
score_threshold: float | None = Field(default=None, ge=0, le=1)
reranking_mode: str = "reranking_model"
reranking_enable: bool = True
reranking_model: AgentKnowledgeRerankingModelConfig | None = None
weights: AgentKnowledgeWeightedScoreConfig | None = None
model: AgentKnowledgeModelConfig | None = None
@model_validator(mode="after")
def validate_mode_fields(self) -> Self:
if self.mode == "multiple" and self.top_k is None:
raise ValueError("knowledge retrieval.top_k is required for multiple mode")
if self.mode == "single" and self.model is None:
raise ValueError("knowledge retrieval.model is required for single mode")
return self
class AgentKnowledgeMetadataCondition(BaseModel):
model_config = ConfigDict(extra="forbid")
name: str = Field(min_length=1, max_length=255)
comparison_operator: SupportedComparisonOperator
value: ConditionValue = None
class AgentKnowledgeMetadataConditions(BaseModel):
model_config = ConfigDict(extra="forbid")
logical_operator: Literal["and", "or"] = "and"
conditions: list[AgentKnowledgeMetadataCondition] = Field(default_factory=list)
class AgentKnowledgeMetadataFilteringConfig(BaseModel):
"""Per-set metadata filtering policy.
The Python attribute uses ``metadata_model_config`` for clarity because the
model belongs to metadata filtering specifically, while the external API and
generated schema keep the historical ``model_config`` field name via alias.
"""
model_config = ConfigDict(extra="forbid", populate_by_name=True)
mode: Literal["disabled", "automatic", "manual"] = "disabled"
# Internal name is explicit; wire format remains ``model_config``.
metadata_model_config: AgentKnowledgeModelConfig | None = Field(default=None, alias="model_config")
conditions: AgentKnowledgeMetadataConditions | None = None
@model_validator(mode="after")
def validate_mode_fields(self) -> Self:
if self.mode == "automatic" and self.metadata_model_config is None:
raise ValueError("metadata_filtering.model_config is required for automatic mode")
if self.mode == "manual" and (self.conditions is None or not self.conditions.conditions):
raise ValueError("metadata_filtering.conditions is required for manual mode")
return self
class AgentKnowledgeSetConfig(BaseModel):
"""One explicit knowledge set in Agent v2.
``knowledge.sets`` replaces the old flat knowledge config. Each set owns
its datasets plus query, retrieval, and metadata policies. An individual
set must contain at least one dataset id even though the overall knowledge
section may be empty, which is how callers express "no knowledge layer".
"""
model_config = ConfigDict(extra="forbid")
id: str = Field(min_length=1, max_length=255)
name: str = Field(min_length=1, max_length=255)
description: str | None = None
datasets: list[AgentKnowledgeDatasetConfig]
query: AgentKnowledgeQueryConfig
retrieval: AgentKnowledgeRetrievalConfig
metadata_filtering: AgentKnowledgeMetadataFilteringConfig = Field(
default_factory=AgentKnowledgeMetadataFilteringConfig
)
@field_validator("id", "name")
@classmethod
def validate_non_blank_identity(cls, value: str) -> str:
normalized = value.strip()
if not normalized:
raise ValueError("knowledge set id and name must not be blank")
return normalized
@model_validator(mode="after")
def validate_datasets(self) -> Self:
dataset_ids = [(dataset.id or "").strip() for dataset in self.datasets]
if not dataset_ids or any(not dataset_id for dataset_id in dataset_ids):
raise ValueError("knowledge set requires at least one dataset id")
if len(dataset_ids) != len(set(dataset_ids)):
raise ValueError("knowledge set dataset ids must be unique")
return self
score_threshold_enabled: bool | None = None
class AgentHumanContactConfig(AgentFlexibleConfig):
@@ -614,28 +453,9 @@ class AgentSoulToolsConfig(BaseModel):
class AgentSoulKnowledgeConfig(BaseModel):
"""Top-level Agent v2 knowledge config.
Agent v2 models knowledge as explicit sets instead of one flat
``datasets`` / ``query_mode`` / ``query_config`` block. An empty ``sets``
list means no knowledge layer should be emitted at runtime, while set-name
uniqueness stays case-insensitive because runtime selection addresses sets
by name.
"""
model_config = ConfigDict(extra="forbid")
sets: list[AgentKnowledgeSetConfig] = Field(default_factory=list)
@model_validator(mode="after")
def validate_unique_sets(self) -> Self:
set_ids = [item.id.strip() for item in self.sets]
if len(set_ids) != len(set(set_ids)):
raise ValueError("knowledge set ids must be unique")
set_names = [item.name.strip().lower() for item in self.sets]
if len(set_names) != len(set(set_names)):
raise ValueError("knowledge set names must be unique")
return self
datasets: list[AgentKnowledgeDatasetConfig] = Field(default_factory=list)
query_mode: AgentKnowledgeQueryMode | None = None
query_config: AgentKnowledgeQueryConfig = Field(default_factory=AgentKnowledgeQueryConfig)
class AgentSoulHumanConfig(BaseModel):
@@ -695,7 +515,6 @@ class AgentSoulConfig(BaseModel):
env: AgentSoulEnvConfig = Field(default_factory=AgentSoulEnvConfig)
sandbox: AgentSoulSandboxConfig = Field(default_factory=AgentSoulSandboxConfig)
memory: AgentSoulMemoryConfig = Field(default_factory=AgentSoulMemoryConfig)
files: AgentSoulFilesConfig = Field(default_factory=AgentSoulFilesConfig)
model: AgentSoulModelConfig | None = None
app_features: AgentSoulAppFeaturesConfig = Field(default_factory=AgentSoulAppFeaturesConfig)
app_variables: list[AppVariableConfig] = Field(default_factory=list)
+15 -99
View File
@@ -2868,6 +2868,7 @@ Delete an existing tracing configuration for an application
| ---- | ----------- |
| 204 | Tracing configuration deleted successfully |
| 400 | Invalid request parameters or configuration not found |
| 403 | Insufficient permissions |
### [GET] /apps/{app_id}/trace-config
Get tracing configuration for an application
@@ -2909,6 +2910,7 @@ Update an existing tracing configuration for an application
| ---- | ----------- | ------ |
| 200 | Tracing configuration updated successfully | **application/json**: [TraceAppConfigResponse](#traceappconfigresponse)<br> |
| 400 | Invalid request parameters or configuration not found | |
| 403 | Insufficient permissions | |
### [POST] /apps/{app_id}/trace-config
**Create a new trace app configuration**
@@ -2933,6 +2935,7 @@ Create a new tracing configuration for an application
| ---- | ----------- | ------ |
| 201 | Tracing configuration created successfully | **application/json**: [TraceAppConfigResponse](#traceappconfigresponse)<br> |
| 400 | Invalid request parameters or configuration already exists | |
| 403 | Insufficient permissions | |
### [POST] /apps/{app_id}/trigger-enable
**Update app trigger (enable/disable)**
@@ -12453,25 +12456,6 @@ Risk marker for CLI tool bootstrap commands.
| current_snapshot_id | string | | No |
| workflow_node_count | integer | | Yes |
#### AgentComposerKnowledgeDatasetCandidateResponse
| Name | Type | Description | Required |
| ---- | ---- | ----------- | -------- |
| description | string | | No |
| id | string | | No |
| missing | boolean | | No |
| name | string | | No |
#### AgentComposerKnowledgeSetCandidateResponse
| Name | Type | Description | Required |
| ---- | ---- | ----------- | -------- |
| datasets | [ [AgentComposerKnowledgeDatasetCandidateResponse](#agentcomposerknowledgedatasetcandidateresponse) ] | | No |
| description | string | | No |
| id | string | | Yes |
| missing_dataset_ids | [ string ] | | No |
| name | string | | Yes |
#### AgentComposerNodeJobCandidatesResponse
| Name | Type | Description | Required |
@@ -12487,7 +12471,7 @@ Risk marker for CLI tool bootstrap commands.
| cli_tools | [ [AgentCliToolConfig](#agentclitoolconfig) ] | | No |
| dify_tools | [ [AgentComposerDifyToolCandidateResponse](#agentcomposerdifytoolcandidateresponse) ] | | No |
| human_contacts | [ [AgentHumanContactConfig](#agenthumancontactconfig) ] | | No |
| knowledge_sets | [ [AgentComposerKnowledgeSetCandidateResponse](#agentcomposerknowledgesetcandidateresponse) ] | | No |
| knowledge_datasets | [ [AgentKnowledgeDatasetConfig](#agentknowledgedatasetconfig) ] | | No |
#### AgentComposerSoulLockResponse
@@ -12881,44 +12865,14 @@ the current roster/workflow APIs scoped to Dify Agent.
| id | string | | No |
| name | string | | No |
#### AgentKnowledgeMetadataCondition
| Name | Type | Description | Required |
| ---- | ---- | ----------- | -------- |
| comparison_operator | string, <br>**Available values:** "<", "=", ">", "after", "before", "contains", "empty", "end with", "in", "is", "is not", "not contains", "not empty", "not in", "start with", "≠", "≤", "≥" | *Enum:* `"<"`, `"="`, `">"`, `"after"`, `"before"`, `"contains"`, `"empty"`, `"end with"`, `"in"`, `"is"`, `"is not"`, `"not contains"`, `"not empty"`, `"not in"`, `"start with"`, `"≠"`, `"≤"`, `"≥"` | Yes |
| name | string | | Yes |
| value | string<br>[ string ]<br>number | | No |
#### AgentKnowledgeMetadataConditions
| Name | Type | Description | Required |
| ---- | ---- | ----------- | -------- |
| conditions | [ [AgentKnowledgeMetadataCondition](#agentknowledgemetadatacondition) ] | | No |
| logical_operator | string, <br>**Available values:** "and", "or", <br>**Default:** and | *Enum:* `"and"`, `"or"` | No |
#### AgentKnowledgeMetadataFilteringConfig
| Name | Type | Description | Required |
| ---- | ---- | ----------- | -------- |
| conditions | [AgentKnowledgeMetadataConditions](#agentknowledgemetadataconditions) | | No |
| mode | string, <br>**Available values:** "automatic", "disabled", "manual", <br>**Default:** disabled | *Enum:* `"automatic"`, `"disabled"`, `"manual"` | No |
| model_config | [AgentKnowledgeModelConfig](#agentknowledgemodelconfig) | | No |
#### AgentKnowledgeModelConfig
| Name | Type | Description | Required |
| ---- | ---- | ----------- | -------- |
| completion_params | object | | No |
| mode | string | | Yes |
| name | string | | Yes |
| provider | string | | Yes |
#### AgentKnowledgeQueryConfig
| Name | Type | Description | Required |
| ---- | ---- | ----------- | -------- |
| mode | [AgentKnowledgeQueryMode](#agentknowledgequerymode) | | Yes |
| value | string | | No |
| query | string | | No |
| score_threshold | number | | No |
| score_threshold_enabled | boolean | | No |
| top_k | integer | | No |
#### AgentKnowledgeQueryMode
@@ -12926,46 +12880,6 @@ the current roster/workflow APIs scoped to Dify Agent.
| ---- | ---- | ----------- | -------- |
| AgentKnowledgeQueryMode | string | | |
#### AgentKnowledgeRerankingModelConfig
| Name | Type | Description | Required |
| ---- | ---- | ----------- | -------- |
| model | string | | Yes |
| provider | string | | Yes |
#### AgentKnowledgeRetrievalConfig
| Name | Type | Description | Required |
| ---- | ---- | ----------- | -------- |
| mode | string, <br>**Available values:** "multiple", "single" | *Enum:* `"multiple"`, `"single"` | Yes |
| model | [AgentKnowledgeModelConfig](#agentknowledgemodelconfig) | | No |
| reranking_enable | boolean, <br>**Default:** true | | No |
| reranking_mode | string, <br>**Default:** reranking_model | | No |
| reranking_model | [AgentKnowledgeRerankingModelConfig](#agentknowledgererankingmodelconfig) | | No |
| score_threshold | number | | No |
| top_k | integer | | No |
| weights | [AgentKnowledgeWeightedScoreConfig](#agentknowledgeweightedscoreconfig) | | No |
#### AgentKnowledgeSetConfig
| Name | Type | Description | Required |
| ---- | ---- | ----------- | -------- |
| datasets | [ [AgentKnowledgeDatasetConfig](#agentknowledgedatasetconfig) ] | | Yes |
| description | string | | No |
| id | string | | Yes |
| metadata_filtering | [AgentKnowledgeMetadataFilteringConfig](#agentknowledgemetadatafilteringconfig) | | No |
| name | string | | Yes |
| query | [AgentKnowledgeQueryConfig](#agentknowledgequeryconfig) | | Yes |
| retrieval | [AgentKnowledgeRetrievalConfig](#agentknowledgeretrievalconfig) | | Yes |
#### AgentKnowledgeWeightedScoreConfig
| Name | Type | Description | Required |
| ---- | ---- | ----------- | -------- |
| keyword_setting | object | | No |
| vector_setting | object | | No |
| weight_type | string | | No |
#### AgentLogConversationItemResponse
| Name | Type | Description | Required |
@@ -13367,7 +13281,9 @@ old Agent tool payloads can be read while new payloads stay explicit.
| Name | Type | Description | Required |
| ---- | ---- | ----------- | -------- |
| sets | [ [AgentKnowledgeSetConfig](#agentknowledgesetconfig) ] | | No |
| datasets | [ [AgentKnowledgeDatasetConfig](#agentknowledgedatasetconfig) ] | | No |
| query_config | [AgentKnowledgeQueryConfig](#agentknowledgequeryconfig) | | No |
| query_mode | [AgentKnowledgeQueryMode](#agentknowledgequerymode) | | No |
#### AgentSoulMemoryConfig
@@ -13516,7 +13432,6 @@ Soft lifecycle state for Agent records.
| created_at | integer | | No |
| files | [ string ] | | Yes |
| id | string | | Yes |
| message_chain_id | string | | No |
| message_id | string | | Yes |
| observation | string | | No |
| position | integer | | Yes |
@@ -14627,8 +14542,8 @@ Enum class for configurate method of provider model.
| Name | Type | Description | Required |
| ---- | ---- | ----------- | -------- |
| annotation_create_account | [SimpleAccount](#simpleaccount) | | No |
| annotation_id | string | | Yes |
| created_at | integer | | No |
| id | string | | Yes |
#### ConversationDetail
@@ -16797,6 +16712,7 @@ Input field definition for snippet parameters.
| Name | Type | Description | Required |
| ---- | ---- | ----------- | -------- |
| description | string | | No |
| icon | string | | No |
| icon_background | string | | No |
| icon_type | string | | No |
@@ -17166,6 +17082,7 @@ Enum class for large language model mode.
| agent_thoughts | [ [AgentThought](#agentthought) ] | | No |
| annotation | [ConversationAnnotation](#conversationannotation) | | No |
| annotation_hit_history | [ConversationAnnotationHitHistory](#conversationannotationhithistory) | | No |
| answer | string | | Yes |
| answer_tokens | integer | | No |
| conversation_id | string | | Yes |
| created_at | integer | | No |
@@ -17179,12 +17096,11 @@ Enum class for large language model mode.
| inputs | object | | Yes |
| message | [JSONValue](#jsonvalue) | | No |
| message_files | [ [MessageFile](#messagefile) ] | | No |
| message_metadata_dict | [JSONValue](#jsonvalue) | | No |
| message_tokens | integer | | No |
| metadata | [JSONValue](#jsonvalue) | | No |
| parent_message_id | string | | No |
| provider_response_latency | number | | No |
| query | string | | Yes |
| re_sign_file_url_answer | string | | Yes |
| status | string | | Yes |
| workflow_run_id | string | | No |
+6
View File
@@ -990,6 +990,12 @@ Pagination for GET /account/sessions. Strict (extra='forbid').
| last_used_at | string | | No |
| prefix | string | | Yes |
#### SimpleResultResponse
| Name | Type | Description | Required |
| ---- | ---- | ----------- | -------- |
| result | string | | Yes |
#### SupportedAppType
App types the ``app`` usage face (``get app``) lists and filters.
@@ -181,30 +181,34 @@ class TestTencentDataTrace:
mock_trace_utils.convert_to_trace_id.return_value = 123
mock_trace_utils.create_link.return_value = "link"
with patch.object(tencent_data_trace, "_get_user_id", return_value="user-1"):
with patch.object(tencent_data_trace, "_process_workflow_nodes") as mock_proc:
with patch.object(tencent_data_trace, "_record_workflow_trace_duration") as mock_dur:
mock_span_builder.build_workflow_spans.return_value = [MagicMock(), MagicMock()]
with (
patch.object(tencent_data_trace, "_get_user_id", return_value="user-1"),
patch.object(tencent_data_trace, "_process_workflow_nodes") as mock_proc,
patch.object(tencent_data_trace, "_record_workflow_trace_duration") as mock_dur,
):
mock_span_builder.build_workflow_spans.return_value = [MagicMock(), MagicMock()]
tencent_data_trace.workflow_trace(trace_info)
tencent_data_trace.workflow_trace(trace_info)
mock_trace_utils.convert_to_trace_id.assert_called_once_with("run-id")
mock_trace_utils.create_link.assert_called_once_with("parent-trace-id")
mock_span_builder.build_workflow_spans.assert_called_once()
assert tencent_data_trace.trace_client.add_span.call_count == 2
mock_proc.assert_called_once_with(trace_info, 123)
mock_dur.assert_called_once_with(trace_info)
mock_trace_utils.convert_to_trace_id.assert_called_once_with("run-id")
mock_trace_utils.create_link.assert_called_once_with("parent-trace-id")
mock_span_builder.build_workflow_spans.assert_called_once()
assert tencent_data_trace.trace_client.add_span.call_count == 2
mock_proc.assert_called_once_with(trace_info, 123)
mock_dur.assert_called_once_with(trace_info)
def test_workflow_trace_exception(self, tencent_data_trace):
def test_workflow_trace_exception(self, tencent_data_trace, caplog: pytest.LogCaptureFixture):
trace_info = MagicMock(spec=WorkflowTraceInfo)
trace_info.workflow_run_id = "run-id"
with patch(
"dify_trace_tencent.tencent_trace.TencentTraceUtils.convert_to_trace_id", side_effect=Exception("error")
with (
patch(
"dify_trace_tencent.tencent_trace.TencentTraceUtils.convert_to_trace_id", side_effect=Exception("error")
),
caplog.at_level(logging.ERROR),
):
with patch("dify_trace_tencent.tencent_trace.logger.exception") as mock_log:
tencent_data_trace.workflow_trace(trace_info)
mock_log.assert_called_once_with("[Tencent APM] Failed to process workflow trace")
tencent_data_trace.workflow_trace(trace_info)
assert "[Tencent APM] Failed to process workflow trace" in caplog.text
def test_message_trace(self, tencent_data_trace, mock_trace_utils, mock_span_builder):
trace_info = MagicMock(spec=MessageTraceInfo)
@@ -214,29 +218,33 @@ class TestTencentDataTrace:
mock_trace_utils.convert_to_trace_id.return_value = 123
mock_trace_utils.create_link.return_value = "link"
with patch.object(tencent_data_trace, "_get_user_id", return_value="user-1"):
with patch.object(tencent_data_trace, "_record_message_llm_metrics") as mock_metrics:
with patch.object(tencent_data_trace, "_record_message_trace_duration") as mock_dur:
mock_span_builder.build_message_span.return_value = MagicMock()
with (
patch.object(tencent_data_trace, "_get_user_id", return_value="user-1"),
patch.object(tencent_data_trace, "_record_message_llm_metrics") as mock_metrics,
patch.object(tencent_data_trace, "_record_message_trace_duration") as mock_dur,
):
mock_span_builder.build_message_span.return_value = MagicMock()
tencent_data_trace.message_trace(trace_info)
tencent_data_trace.message_trace(trace_info)
mock_trace_utils.convert_to_trace_id.assert_called_once_with("msg-id")
mock_trace_utils.create_link.assert_called_once_with("parent-trace-id")
mock_span_builder.build_message_span.assert_called_once()
tencent_data_trace.trace_client.add_span.assert_called_once()
mock_metrics.assert_called_once_with(trace_info)
mock_dur.assert_called_once_with(trace_info)
mock_trace_utils.convert_to_trace_id.assert_called_once_with("msg-id")
mock_trace_utils.create_link.assert_called_once_with("parent-trace-id")
mock_span_builder.build_message_span.assert_called_once()
tencent_data_trace.trace_client.add_span.assert_called_once()
mock_metrics.assert_called_once_with(trace_info)
mock_dur.assert_called_once_with(trace_info)
def test_message_trace_exception(self, tencent_data_trace):
def test_message_trace_exception(self, tencent_data_trace, caplog: pytest.LogCaptureFixture):
trace_info = MagicMock(spec=MessageTraceInfo)
with patch(
"dify_trace_tencent.tencent_trace.TencentTraceUtils.convert_to_trace_id", side_effect=Exception("error")
with (
patch(
"dify_trace_tencent.tencent_trace.TencentTraceUtils.convert_to_trace_id", side_effect=Exception("error")
),
caplog.at_level(logging.ERROR),
):
with patch("dify_trace_tencent.tencent_trace.logger.exception") as mock_log:
tencent_data_trace.message_trace(trace_info)
mock_log.assert_called_once_with("[Tencent APM] Failed to process message trace")
tencent_data_trace.message_trace(trace_info)
assert "[Tencent APM] Failed to process message trace" in caplog.text
def test_tool_trace(self, tencent_data_trace, mock_trace_utils, mock_span_builder):
trace_info = MagicMock(spec=ToolTraceInfo)
@@ -259,16 +267,18 @@ class TestTencentDataTrace:
tencent_data_trace.tool_trace(trace_info)
tencent_data_trace.trace_client.add_span.assert_not_called()
def test_tool_trace_exception(self, tencent_data_trace):
def test_tool_trace_exception(self, tencent_data_trace, caplog: pytest.LogCaptureFixture):
trace_info = MagicMock(spec=ToolTraceInfo)
trace_info.message_id = "msg-id"
with patch(
"dify_trace_tencent.tencent_trace.TencentTraceUtils.convert_to_span_id", side_effect=Exception("error")
with (
patch(
"dify_trace_tencent.tencent_trace.TencentTraceUtils.convert_to_span_id", side_effect=Exception("error")
),
caplog.at_level(logging.ERROR),
):
with patch("dify_trace_tencent.tencent_trace.logger.exception") as mock_log:
tencent_data_trace.tool_trace(trace_info)
mock_log.assert_called_once_with("[Tencent APM] Failed to process tool trace")
tencent_data_trace.tool_trace(trace_info)
assert "[Tencent APM] Failed to process tool trace" in caplog.text
def test_dataset_retrieval_trace(self, tencent_data_trace, mock_trace_utils, mock_span_builder):
trace_info = MagicMock(spec=DatasetRetrievalTraceInfo)
@@ -291,29 +301,34 @@ class TestTencentDataTrace:
tencent_data_trace.dataset_retrieval_trace(trace_info)
tencent_data_trace.trace_client.add_span.assert_not_called()
def test_dataset_retrieval_trace_exception(self, tencent_data_trace):
def test_dataset_retrieval_trace_exception(self, tencent_data_trace, caplog: pytest.LogCaptureFixture):
trace_info = MagicMock(spec=DatasetRetrievalTraceInfo)
trace_info.message_id = "msg-id"
with patch(
"dify_trace_tencent.tencent_trace.TencentTraceUtils.convert_to_span_id", side_effect=Exception("error")
with (
patch(
"dify_trace_tencent.tencent_trace.TencentTraceUtils.convert_to_span_id", side_effect=Exception("error")
),
caplog.at_level(logging.ERROR),
):
with patch("dify_trace_tencent.tencent_trace.logger.exception") as mock_log:
tencent_data_trace.dataset_retrieval_trace(trace_info)
mock_log.assert_called_once_with("[Tencent APM] Failed to process dataset retrieval trace")
tencent_data_trace.dataset_retrieval_trace(trace_info)
assert "[Tencent APM] Failed to process dataset retrieval trace" in caplog.text
def test_suggested_question_trace(self, tencent_data_trace):
def test_suggested_question_trace(self, tencent_data_trace, caplog: pytest.LogCaptureFixture):
trace_info = MagicMock(spec=SuggestedQuestionTraceInfo)
with patch("dify_trace_tencent.tencent_trace.logger.info") as mock_log:
with caplog.at_level(logging.INFO):
tencent_data_trace.suggested_question_trace(trace_info)
mock_log.assert_called_once_with("[Tencent APM] Processing suggested question trace")
assert "[Tencent APM] Processing suggested question trace" in caplog.text
def test_suggested_question_trace_exception(self, tencent_data_trace):
def test_suggested_question_trace_exception(
self, tencent_data_trace, monkeypatch, caplog: pytest.LogCaptureFixture
):
trace_info = MagicMock(spec=SuggestedQuestionTraceInfo)
with patch("dify_trace_tencent.tencent_trace.logger.info", side_effect=Exception("error")):
with patch("dify_trace_tencent.tencent_trace.logger.exception") as mock_log:
tencent_data_trace.suggested_question_trace(trace_info)
mock_log.assert_called_once_with("[Tencent APM] Failed to process suggested question trace")
target_logger = logging.getLogger("dify_trace_tencent.tencent_trace")
monkeypatch.setattr(target_logger, "info", MagicMock(side_effect=Exception("error")))
with caplog.at_level(logging.ERROR):
tencent_data_trace.suggested_question_trace(trace_info)
assert "[Tencent APM] Failed to process suggested question trace" in caplog.text
def test_process_workflow_nodes(self, tencent_data_trace, mock_trace_utils):
trace_info = MagicMock(spec=WorkflowTraceInfo)
@@ -327,35 +342,42 @@ class TestTencentDataTrace:
node2.id = "n2"
node2.node_type = BuiltinNodeTypes.TOOL
with patch.object(tencent_data_trace, "_get_workflow_node_executions", return_value=[node1, node2]):
with patch.object(tencent_data_trace, "_build_workflow_node_span", side_effect=["span1", "span2"]):
with patch.object(tencent_data_trace, "_record_llm_metrics") as mock_metrics:
tencent_data_trace._process_workflow_nodes(trace_info, 123)
with (
patch.object(tencent_data_trace, "_get_workflow_node_executions", return_value=[node1, node2]),
patch.object(tencent_data_trace, "_build_workflow_node_span", side_effect=["span1", "span2"]),
patch.object(tencent_data_trace, "_record_llm_metrics") as mock_metrics,
):
tencent_data_trace._process_workflow_nodes(trace_info, 123)
assert tencent_data_trace.trace_client.add_span.call_count == 2
mock_metrics.assert_called_once_with(node1)
assert tencent_data_trace.trace_client.add_span.call_count == 2
mock_metrics.assert_called_once_with(node1)
def test_process_workflow_nodes_node_exception(self, tencent_data_trace, mock_trace_utils):
def test_process_workflow_nodes_node_exception(
self, tencent_data_trace, mock_trace_utils, caplog: pytest.LogCaptureFixture
):
trace_info = MagicMock(spec=WorkflowTraceInfo)
mock_trace_utils.convert_to_span_id.return_value = 111
node = MagicMock(spec=WorkflowNodeExecution)
node.id = "n1"
with patch.object(tencent_data_trace, "_get_workflow_node_executions", return_value=[node]):
with patch.object(tencent_data_trace, "_build_workflow_node_span", side_effect=Exception("node error")):
with patch("dify_trace_tencent.tencent_trace.logger.exception") as mock_log:
tencent_data_trace._process_workflow_nodes(trace_info, 123)
# The exception should be caught by the outer handler since convert_to_span_id is called first
mock_log.assert_called_once_with("[Tencent APM] Failed to process workflow nodes")
with (
patch.object(tencent_data_trace, "_get_workflow_node_executions", return_value=[node]),
patch.object(tencent_data_trace, "_build_workflow_node_span", side_effect=Exception("node error")),
caplog.at_level(logging.ERROR),
):
tencent_data_trace._process_workflow_nodes(trace_info, 123)
assert "[Tencent APM] Failed to process workflow nodes" in caplog.text
def test_process_workflow_nodes_exception(self, tencent_data_trace, mock_trace_utils):
def test_process_workflow_nodes_exception(
self, tencent_data_trace, mock_trace_utils, caplog: pytest.LogCaptureFixture
):
trace_info = MagicMock(spec=WorkflowTraceInfo)
mock_trace_utils.convert_to_span_id.side_effect = Exception("outer error")
with patch("dify_trace_tencent.tencent_trace.logger.exception") as mock_log:
with caplog.at_level(logging.ERROR):
tencent_data_trace._process_workflow_nodes(trace_info, 123)
mock_log.assert_called_once_with("[Tencent APM] Failed to process workflow nodes")
assert "[Tencent APM] Failed to process workflow nodes" in caplog.text
def test_build_workflow_node_span(self, tencent_data_trace, mock_span_builder):
trace_info = MagicMock(spec=WorkflowTraceInfo)
@@ -377,16 +399,18 @@ class TestTencentDataTrace:
assert result == "span"
builder_method.assert_called_once_with(123, 456, trace_info, node)
def test_build_workflow_node_span_exception(self, tencent_data_trace, mock_span_builder):
def test_build_workflow_node_span_exception(
self, tencent_data_trace, mock_span_builder, caplog: pytest.LogCaptureFixture
):
node = MagicMock(spec=WorkflowNodeExecution)
node.node_type = BuiltinNodeTypes.LLM
node.id = "n1"
mock_span_builder.build_workflow_llm_span.side_effect = Exception("error")
with patch("dify_trace_tencent.tencent_trace.logger.debug") as mock_log:
with caplog.at_level(logging.DEBUG):
result = tencent_data_trace._build_workflow_node_span(node, 123, MagicMock(), 456)
assert result is None
mock_log.assert_called_once()
assert result is None
assert len([r for r in caplog.records if r.levelno == logging.DEBUG]) >= 1
def test_get_workflow_node_executions(self, tencent_data_trace):
trace_info = MagicMock(spec=WorkflowTraceInfo)
@@ -419,16 +443,16 @@ class TestTencentDataTrace:
assert results == mock_executions
account.set_tenant_id.assert_called_once_with("tenant-1")
def test_get_workflow_node_executions_no_app_id(self, tencent_data_trace):
def test_get_workflow_node_executions_no_app_id(self, tencent_data_trace, caplog: pytest.LogCaptureFixture):
trace_info = MagicMock(spec=WorkflowTraceInfo)
trace_info.metadata = {}
with patch("dify_trace_tencent.tencent_trace.logger.exception") as mock_log:
with caplog.at_level(logging.ERROR):
results = tencent_data_trace._get_workflow_node_executions(trace_info)
assert results == []
mock_log.assert_called_once()
assert results == []
assert len([r for r in caplog.records if r.levelno == logging.ERROR]) >= 1
def test_get_workflow_node_executions_app_not_found(self, tencent_data_trace):
def test_get_workflow_node_executions_app_not_found(self, tencent_data_trace, caplog: pytest.LogCaptureFixture):
trace_info = MagicMock(spec=WorkflowTraceInfo)
trace_info.metadata = {"app_id": "app-1"}
@@ -439,23 +463,25 @@ class TestTencentDataTrace:
session = mock_session_ctx.return_value.__enter__.return_value
session.scalar.return_value = None
with patch("dify_trace_tencent.tencent_trace.logger.exception") as mock_log:
with caplog.at_level(logging.ERROR):
results = tencent_data_trace._get_workflow_node_executions(trace_info)
assert results == []
mock_log.assert_called_once()
assert results == []
assert len([r for r in caplog.records if r.levelno == logging.ERROR]) >= 1
def test_get_user_id_workflow(self, tencent_data_trace):
trace_info = MagicMock(spec=WorkflowTraceInfo)
trace_info.tenant_id = "tenant-1"
trace_info.metadata = {"user_id": "user-1"}
with patch("dify_trace_tencent.tencent_trace.sessionmaker", side_effect=Exception("Database error")):
with patch("dify_trace_tencent.tencent_trace.db") as mock_db:
mock_db.init_app = MagicMock()
mock_db.engine = MagicMock()
with (
patch("dify_trace_tencent.tencent_trace.sessionmaker", side_effect=Exception("Database error")),
patch("dify_trace_tencent.tencent_trace.db") as mock_db,
):
mock_db.init_app = MagicMock()
mock_db.engine = MagicMock()
user_id = tencent_data_trace._get_user_id(trace_info)
assert user_id == "unknown"
user_id = tencent_data_trace._get_user_id(trace_info)
assert user_id == "unknown"
def test_get_user_id_only_user_id(self, tencent_data_trace):
trace_info = MagicMock(spec=MessageTraceInfo)
@@ -471,16 +497,18 @@ class TestTencentDataTrace:
user_id = tencent_data_trace._get_user_id(trace_info)
assert user_id == "anonymous"
def test_get_user_id_exception(self, tencent_data_trace):
def test_get_user_id_exception(self, tencent_data_trace, caplog: pytest.LogCaptureFixture):
trace_info = MagicMock(spec=WorkflowTraceInfo)
trace_info.tenant_id = "t"
trace_info.metadata = {"user_id": "u"}
with patch("dify_trace_tencent.tencent_trace.sessionmaker", side_effect=Exception("error")):
with patch("dify_trace_tencent.tencent_trace.logger.exception") as mock_log:
user_id = tencent_data_trace._get_user_id(trace_info)
assert user_id == "unknown"
mock_log.assert_called_once_with("[Tencent APM] Failed to get user ID")
with (
patch("dify_trace_tencent.tencent_trace.sessionmaker", side_effect=Exception("error")),
caplog.at_level(logging.ERROR),
):
user_id = tencent_data_trace._get_user_id(trace_info)
assert user_id == "unknown"
assert "[Tencent APM] Failed to get user ID" in caplog.text
def test_record_llm_metrics_usage_in_process_data(self, tencent_data_trace):
node = MagicMock(spec=WorkflowNodeExecution)
@@ -514,14 +542,14 @@ class TestTencentDataTrace:
tencent_data_trace.trace_client.record_llm_duration.assert_called_once()
tencent_data_trace.trace_client.record_token_usage.assert_called_once()
def test_record_llm_metrics_exception(self, tencent_data_trace):
def test_record_llm_metrics_exception(self, tencent_data_trace, caplog: pytest.LogCaptureFixture):
node = MagicMock(spec=WorkflowNodeExecution)
node.process_data = None
node.outputs = None
with patch("dify_trace_tencent.tencent_trace.logger.debug") as mock_log:
with caplog.at_level(logging.DEBUG):
tencent_data_trace._record_llm_metrics(node)
# Should not crash
# Should not crash
def test_record_message_llm_metrics(self, tencent_data_trace):
trace_info = MagicMock(spec=MessageTraceInfo)
@@ -553,13 +581,13 @@ class TestTencentDataTrace:
tencent_data_trace._record_message_llm_metrics(trace_info)
tencent_data_trace.trace_client.record_llm_duration.assert_called_once()
def test_record_message_llm_metrics_exception(self, tencent_data_trace):
def test_record_message_llm_metrics_exception(self, tencent_data_trace, caplog: pytest.LogCaptureFixture):
trace_info = MagicMock(spec=MessageTraceInfo)
trace_info.metadata = None
with patch("dify_trace_tencent.tencent_trace.logger.debug") as mock_log:
with caplog.at_level(logging.DEBUG):
tencent_data_trace._record_message_llm_metrics(trace_info)
# Should not crash
# Should not crash
def test_record_workflow_trace_duration(self, tencent_data_trace):
trace_info = MagicMock(spec=WorkflowTraceInfo)
@@ -605,11 +633,11 @@ class TestTencentDataTrace:
attributes = kwargs["attributes"] if "attributes" in kwargs else args[1] if len(args) > 1 else {}
assert attributes["has_conversation"] == "false"
def test_record_workflow_trace_duration_exception(self, tencent_data_trace):
def test_record_workflow_trace_duration_exception(self, tencent_data_trace, caplog: pytest.LogCaptureFixture):
trace_info = MagicMock(spec=WorkflowTraceInfo)
trace_info.start_time = MagicMock() # This might cause total_seconds() to fail if not mocked right
with patch("dify_trace_tencent.tencent_trace.logger.debug") as mock_log:
with caplog.at_level(logging.DEBUG):
tencent_data_trace._record_workflow_trace_duration(trace_info)
def test_record_message_trace_duration(self, tencent_data_trace):
@@ -627,11 +655,11 @@ class TestTencentDataTrace:
2.0, {"conversation_mode": "chat", "stream": "true"}
)
def test_record_message_trace_duration_exception(self, tencent_data_trace):
def test_record_message_trace_duration_exception(self, tencent_data_trace, caplog: pytest.LogCaptureFixture):
trace_info = MagicMock(spec=MessageTraceInfo)
trace_info.start_time = None
with patch("dify_trace_tencent.tencent_trace.logger.debug") as mock_log:
with caplog.at_level(logging.DEBUG):
tencent_data_trace._record_message_trace_duration(trace_info)
def test_close(self, tencent_data_trace):
@@ -647,11 +675,11 @@ class TestTencentDataTrace:
client.shutdown.assert_called_once()
def test_close_exception(self, tencent_data_trace):
def test_close_exception(self, tencent_data_trace, caplog: pytest.LogCaptureFixture):
tencent_data_trace.trace_client.shutdown.side_effect = Exception("error")
with patch("dify_trace_tencent.tencent_trace.logger.exception") as mock_log:
with caplog.at_level(logging.ERROR):
tencent_data_trace.close()
mock_log.assert_called_once_with("[Tencent APM] Failed to shutdown trace client during cleanup")
assert "[Tencent APM] Failed to shutdown trace client during cleanup" in caplog.text
def test_close_handles_async_shutdown_mock(self, tencent_data_trace):
shutdown = AsyncMock()
@@ -113,7 +113,7 @@ class LindormVectorStore(BaseVector):
)
def _bulk_with_retry(actions):
try:
response = self._client.bulk(actions, timeout=timeout)
response = self._client.bulk(body=actions, timeout=timeout)
if response["errors"]:
error_items = [item for item in response["items"] if "error" in item["index"]]
error_msg = f"Bulk indexing had {len(error_items)} errors"
@@ -231,7 +231,7 @@ class LindormVectorStore(BaseVector):
routing_filter_query = {
"query": {"bool": {"must": [{"term": {f"{ROUTING_FIELD}.keyword": self._routing}}]}}
}
self._client.delete_by_query(self._collection_name, body=routing_filter_query)
self._client.delete_by_query(index=self._collection_name, body=routing_filter_query)
self.refresh()
else:
if self._client.indices.exists(index=self._collection_name):
@@ -127,7 +127,7 @@ def test_create_refresh_and_add_texts_success(lindorm_module, monkeypatch: pytes
vector.add_texts(docs, embeddings, batch_size=2, timeout=9)
assert vector._client.bulk.call_count == 2
actions = vector._client.bulk.call_args_list[0].args[0]
actions = vector._client.bulk.call_args_list[0].kwargs["body"]
assert actions[0]["index"]["routing"] == "route"
assert actions[1][lindorm_module.ROUTING_FIELD] == "route"
vector.refresh()
@@ -268,9 +268,11 @@ class TestWeaviateVector(unittest.TestCase):
wv._client = MagicMock()
wv._client.collections.exists.side_effect = RuntimeError("create failed")
with patch.object(weaviate_vector_module.logger, "exception") as mock_exception:
with pytest.raises(RuntimeError, match="create failed"):
wv._create_collection()
with (
patch.object(weaviate_vector_module.logger, "exception") as mock_exception,
pytest.raises(RuntimeError, match="create failed"),
):
wv._create_collection()
mock_exception.assert_called_once()
@@ -835,9 +837,11 @@ class TestWeaviateVector(unittest.TestCase):
wv._client.collections.use.return_value = mock_col
mock_col.data.delete_by_id.side_effect = FakeUnexpectedStatusCodeError(500)
with patch.object(weaviate_vector_module, "UnexpectedStatusCodeError", FakeUnexpectedStatusCodeError):
with pytest.raises(FakeUnexpectedStatusCodeError, match="status=500"):
wv.delete_by_ids(["bad-id"])
with (
patch.object(weaviate_vector_module, "UnexpectedStatusCodeError", FakeUnexpectedStatusCodeError),
pytest.raises(FakeUnexpectedStatusCodeError, match="status=500"),
):
wv.delete_by_ids(["bad-id"])
def test_json_serializable_converts_datetime(self):
wv = WeaviateVector.__new__(WeaviateVector)
+2 -2
View File
@@ -1,11 +1,11 @@
[project]
name = "dify-api"
version = "1.14.2"
version = "1.15.0"
requires-python = "~=3.12.0"
dependencies = [
# Legacy: mature and widely deployed
"bleach>=6.3.0,<7.0.0",
"bleach>=6.4.0,<7.0.0",
"boto3>=1.43.24,<2.0.0",
"celery>=5.6.3,<6.0.0",
"croniter>=6.2.2,<7.0.0",
+3
View File
@@ -1788,6 +1788,9 @@ class TenantService:
account_id,
)
if dify_config.RBAC_ENABLED:
RBACService.MemberRoles.delete_rbac_bindings(tenant_id=tenant.id, account_id=account_id)
@staticmethod
def update_member_role(
tenant: Tenant, member: Account, new_role: str, operator: Account, *, session: scoped_session | Session
+12 -32
View File
@@ -25,7 +25,6 @@ from models.agent_config_entities import (
AgentSoulConfig,
DeclaredOutputConfig,
)
from services.agent.knowledge_datasets import list_agent_soul_knowledge_dataset_ids
MAX_CANDIDATES_PER_LIST = 200
@@ -140,49 +139,30 @@ def soul_candidates(
cli_tools = [tool.model_dump(exclude_none=True) for tool in soul.tools.cli_tools if tool.enabled]
dataset_ids = list_agent_soul_knowledge_dataset_ids(soul)
dataset_ids = [dataset.id for dataset in soul.knowledge.datasets if dataset.id]
dataset_rows = dataset_lookup(dataset_ids) if dataset_ids else {}
knowledge_sets: list[dict[str, Any]] = []
for knowledge_set in soul.knowledge.sets:
missing_dataset_ids: list[str] = []
datasets: list[dict[str, Any]] = []
for dataset in knowledge_set.datasets:
dataset_id = (dataset.id or "").strip()
if not dataset_id:
continue
row = dataset_rows.get(dataset_id)
if row is None:
missing_dataset_ids.append(dataset_id)
datasets.append(
{
"id": dataset_id,
"name": (getattr(row, "name", None) or dataset.name or dataset_id),
"description": getattr(row, "description", None) or dataset.description,
"missing": row is None,
}
)
knowledge_sets.append(
knowledge_datasets: list[dict[str, Any]] = []
for dataset in soul.knowledge.datasets:
if not dataset.id:
continue
row = dataset_rows.get(dataset.id)
knowledge_datasets.append(
{
"id": knowledge_set.id,
"name": knowledge_set.name,
"description": knowledge_set.description,
"datasets": datasets,
"missing_dataset_ids": missing_dataset_ids,
"id": dataset.id,
"name": (getattr(row, "name", None) or dataset.name or dataset.id),
"description": getattr(row, "description", None) or dataset.description,
"missing": row is None,
}
)
human_contacts = [contact.model_dump(exclude_none=True) for contact in soul.human.contacts]
skills = [skill.model_dump(exclude_none=True) for skill in soul.files.skills]
files = [file_ref.model_dump(exclude_none=True) for file_ref in soul.files.files]
dify_tools = workspace_tools_loader()
lists = {
"dify_tools": dify_tools,
"cli_tools": cli_tools,
"knowledge_sets": knowledge_sets,
"knowledge_datasets": knowledge_datasets,
"human_contacts": human_contacts,
"skills": skills,
"files": files,
}
capped: dict[str, list[dict[str, Any]]] = {}
for key, values in lists.items():
+123 -486
View File
@@ -11,8 +11,6 @@ from libs.helper import to_timestamp
from models import Account
from models.agent import (
Agent,
AgentConfigDraft,
AgentConfigDraftType,
AgentConfigRevision,
AgentConfigRevisionOperation,
AgentConfigSnapshot,
@@ -39,12 +37,7 @@ from services.agent.errors import (
AgentVersionNotFoundError,
InvalidComposerConfigError,
)
from services.agent.knowledge_datasets import (
get_tenant_knowledge_dataset_rows,
list_missing_tenant_knowledge_dataset_ids,
)
from services.agent.roster_service import AgentRosterService
from services.agent.soul_files_service import AgentSoulFilesService
from services.app_service import AppService, CreateAppParams
from services.entities.agent_entities import (
AgentSoulConfig,
@@ -127,7 +120,6 @@ class AgentComposerService:
_backfill_cli_tool_ids(payload.agent_soul)
_validate_composer_payload_for_strategy(payload)
cls.validate_knowledge_datasets(tenant_id=tenant_id, agent_soul=payload.agent_soul)
workflow = cls._get_draft_workflow(tenant_id=tenant_id, app_id=app_id)
binding = cls._get_workflow_binding(tenant_id=tenant_id, workflow_id=workflow.id, node_id=node_id)
@@ -267,23 +259,27 @@ class AgentComposerService:
@classmethod
def load_agent_app_composer(cls, *, tenant_id: str, app_id: str) -> dict[str, Any]:
agent = cls._require_agent_app_agent(tenant_id=tenant_id, app_id=app_id)
draft = cls._get_or_create_agent_draft(
tenant_id=tenant_id,
agent=agent,
draft_type=AgentConfigDraftType.DRAFT,
account_id=None,
created_by=agent.updated_by or agent.created_by,
agent = db.session.scalar(
select(Agent)
.where(
Agent.tenant_id == tenant_id,
Agent.app_id == app_id,
Agent.scope == AgentScope.ROSTER,
Agent.status == AgentStatus.ACTIVE,
)
.order_by(Agent.created_at.desc())
.limit(1)
)
version = cls._get_version_if_present(
if not agent:
raise AgentNotFoundError()
version = cls._require_version(
tenant_id=tenant_id, agent_id=agent.id, version_id=agent.active_config_snapshot_id
)
return {
"variant": ComposerVariant.AGENT_APP.value,
"agent": cls._serialize_agent(agent),
"active_config_snapshot": cls._serialize_version(version),
"draft": cls._serialize_draft(draft),
"agent_soul": draft.config_snapshot_dict,
"agent_soul": version.config_snapshot_dict,
"save_options": [
ComposerSaveStrategy.SAVE_TO_CURRENT_VERSION.value,
ComposerSaveStrategy.SAVE_AS_NEW_VERSION.value,
@@ -298,11 +294,20 @@ class AgentComposerService:
raise ValueError("Agent App composer endpoint only accepts agent_app variant")
_backfill_cli_tool_ids(payload.agent_soul)
_validate_composer_payload_for_strategy(payload)
cls.validate_knowledge_datasets(tenant_id=tenant_id, agent_soul=payload.agent_soul)
if payload.agent_soul is None:
raise ValueError("agent_soul is required")
agent = cls._get_agent_app_agent(tenant_id=tenant_id, app_id=app_id)
agent = db.session.scalar(
select(Agent)
.where(
Agent.tenant_id == tenant_id,
Agent.app_id == app_id,
Agent.scope == AgentScope.ROSTER,
Agent.status == AgentStatus.ACTIVE,
)
.order_by(Agent.created_at.desc())
.limit(1)
)
if not agent:
agent = Agent(
tenant_id=tenant_id,
@@ -322,20 +327,32 @@ class AgentComposerService:
except IntegrityError as exc:
db.session.rollback()
raise AgentNameConflictError() from exc
payload.agent_soul = cls._preserve_agent_draft_soul_files(
tenant_id=tenant_id,
agent_id=agent.id,
agent_soul=payload.agent_soul,
)
cls._save_agent_draft(
tenant_id=tenant_id,
agent=agent,
draft_type=AgentConfigDraftType.DRAFT,
account_id=None,
agent_soul=payload.agent_soul,
account_id_for_audit=account_id,
)
agent.updated_by = account_id
if payload.save_strategy == ComposerSaveStrategy.SAVE_AS_NEW_VERSION or not agent.active_config_snapshot_id:
version = cls._create_config_version(
tenant_id=tenant_id,
agent_id=agent.id,
account_id=account_id,
agent_soul=payload.agent_soul,
operation=AgentConfigRevisionOperation.SAVE_NEW_VERSION,
version_note=payload.version_note,
)
agent.active_config_snapshot_id = version.id
agent.active_config_has_model = agent_soul_has_model(payload.agent_soul)
else:
current_snapshot = cls._require_version(
tenant_id=tenant_id, agent_id=agent.id, version_id=agent.active_config_snapshot_id
)
version = cls._update_current_version(
current_snapshot=current_snapshot,
account_id=account_id,
agent_soul=payload.agent_soul,
operation=AgentConfigRevisionOperation.SAVE_CURRENT_VERSION,
version_note=payload.version_note,
)
agent.active_config_snapshot_id = version.id
agent.active_config_has_model = agent_soul_has_model(payload.agent_soul)
agent.updated_by = account_id
db.session.commit()
state = cls.load_agent_app_composer(tenant_id=tenant_id, app_id=app_id)
@@ -346,167 +363,6 @@ class AgentComposerService:
)
return state
@classmethod
def publish_agent_app_draft(
cls, *, tenant_id: str, agent_id: str, account_id: str, version_note: str | None = None
) -> dict[str, Any]:
agent = cls._require_agent(tenant_id=tenant_id, agent_id=agent_id)
if agent.scope != AgentScope.ROSTER or agent.source != AgentSource.AGENT_APP:
raise AgentNotFoundError()
draft = cls._get_or_create_agent_draft(
tenant_id=tenant_id,
agent=agent,
draft_type=AgentConfigDraftType.DRAFT,
account_id=None,
created_by=account_id,
)
agent_soul = AgentSoulConfig.model_validate(draft.config_snapshot_dict)
ComposerConfigValidator.validate_publish_payload(
ComposerSavePayload(
variant=ComposerVariant.AGENT_APP,
agent_soul=agent_soul,
save_strategy=ComposerSaveStrategy.SAVE_AS_NEW_VERSION,
version_note=version_note,
)
)
cls.validate_knowledge_datasets(tenant_id=tenant_id, agent_soul=agent_soul)
version = cls._create_config_version(
tenant_id=tenant_id,
agent_id=agent.id,
account_id=account_id,
agent_soul=agent_soul,
operation=AgentConfigRevisionOperation.PUBLISH_DRAFT,
version_note=version_note,
previous_snapshot_id=agent.active_config_snapshot_id,
)
agent.active_config_snapshot_id = version.id
agent.active_config_has_model = agent_soul_has_model(agent_soul)
agent.updated_by = account_id
draft.base_snapshot_id = version.id
draft.updated_by = account_id
db.session.commit()
return {
"result": "success",
"active_config_snapshot_id": version.id,
"active_config_snapshot": cls._serialize_version(version),
"draft": cls._serialize_draft(draft),
}
@classmethod
def checkout_agent_app_build_draft(
cls, *, tenant_id: str, agent_id: str, account_id: str, force: bool = False
) -> dict[str, Any]:
agent = cls._require_agent(tenant_id=tenant_id, agent_id=agent_id)
if agent.scope != AgentScope.ROSTER or agent.source != AgentSource.AGENT_APP:
raise AgentNotFoundError()
normal_draft = cls._get_or_create_agent_draft(
tenant_id=tenant_id,
agent=agent,
draft_type=AgentConfigDraftType.DRAFT,
account_id=None,
created_by=account_id,
)
build_draft = cls._get_agent_draft(
tenant_id=tenant_id,
agent_id=agent.id,
draft_type=AgentConfigDraftType.DEBUG_BUILD,
account_id=account_id,
)
if build_draft is not None and not force:
return cls._serialize_build_draft_state(build_draft)
if build_draft is None:
build_draft = AgentConfigDraft(
tenant_id=tenant_id,
agent_id=agent.id,
draft_type=AgentConfigDraftType.DEBUG_BUILD,
account_id=account_id,
draft_owner_key=account_id,
created_by=account_id,
)
db.session.add(build_draft)
build_draft.base_snapshot_id = normal_draft.base_snapshot_id
build_draft.config_snapshot = AgentSoulConfig.model_validate(normal_draft.config_snapshot_dict)
build_draft.updated_by = account_id
db.session.commit()
return cls._serialize_build_draft_state(build_draft)
@classmethod
def load_agent_app_build_draft(cls, *, tenant_id: str, agent_id: str, account_id: str) -> dict[str, Any]:
build_draft = cls._get_agent_draft(
tenant_id=tenant_id,
agent_id=agent_id,
draft_type=AgentConfigDraftType.DEBUG_BUILD,
account_id=account_id,
)
if build_draft is None:
raise AgentVersionNotFoundError()
return cls._serialize_build_draft_state(build_draft)
@classmethod
def save_agent_app_build_draft(
cls, *, tenant_id: str, agent_id: str, account_id: str, payload: ComposerSavePayload
) -> dict[str, Any]:
if payload.agent_soul is None:
raise ValueError("agent_soul is required")
_backfill_cli_tool_ids(payload.agent_soul)
ComposerConfigValidator.validate_draft_save_payload(payload)
cls.validate_knowledge_datasets(tenant_id=tenant_id, agent_soul=payload.agent_soul)
agent = cls._require_agent(tenant_id=tenant_id, agent_id=agent_id)
payload.agent_soul = cls._preserve_agent_draft_soul_files(
tenant_id=tenant_id,
agent_id=agent.id,
agent_soul=payload.agent_soul,
draft_type=AgentConfigDraftType.DEBUG_BUILD,
account_id=account_id,
)
build_draft = cls._save_agent_draft(
tenant_id=tenant_id,
agent=agent,
draft_type=AgentConfigDraftType.DEBUG_BUILD,
account_id=account_id,
agent_soul=payload.agent_soul,
account_id_for_audit=account_id,
)
db.session.commit()
return cls._serialize_build_draft_state(build_draft)
@classmethod
def apply_agent_app_build_draft(cls, *, tenant_id: str, agent_id: str, account_id: str) -> dict[str, Any]:
agent = cls._require_agent(tenant_id=tenant_id, agent_id=agent_id)
build_draft = cls._get_agent_draft(
tenant_id=tenant_id,
agent_id=agent.id,
draft_type=AgentConfigDraftType.DEBUG_BUILD,
account_id=account_id,
)
if build_draft is None:
raise AgentVersionNotFoundError()
normal_draft = cls._save_agent_draft(
tenant_id=tenant_id,
agent=agent,
draft_type=AgentConfigDraftType.DRAFT,
account_id=None,
agent_soul=AgentSoulConfig.model_validate(build_draft.config_snapshot_dict),
account_id_for_audit=account_id,
base_snapshot_id=build_draft.base_snapshot_id,
)
db.session.delete(build_draft)
db.session.commit()
return {"result": "success", "draft": cls._serialize_draft(normal_draft)}
@classmethod
def discard_agent_app_build_draft(cls, *, tenant_id: str, agent_id: str, account_id: str) -> dict[str, Any]:
build_draft = cls._get_agent_draft(
tenant_id=tenant_id,
agent_id=agent_id,
draft_type=AgentConfigDraftType.DEBUG_BUILD,
account_id=account_id,
)
if build_draft is not None:
db.session.delete(build_draft)
db.session.commit()
return {"result": "success"}
@classmethod
def collect_validation_findings(
cls,
@@ -516,43 +372,29 @@ class AgentComposerService:
agent_id: str | None = None,
) -> dict[str, Any]:
"""ENG-617 soft findings, with DB-backed dataset and drive mention checks."""
existing_knowledge_set_ids = (
{knowledge_set.id for knowledge_set in payload.agent_soul.knowledge.sets}
if payload.agent_soul is not None
else None
)
findings = ComposerConfigValidator.collect_soft_findings(
payload,
existing_knowledge_set_ids=existing_knowledge_set_ids,
)
from services.agent.prompt_mentions import MentionKind, parse_prompt_mentions
mentioned_ids: set[str] = set()
if payload.agent_soul is not None:
mentioned_ids |= {
mention.ref_id
for mention in parse_prompt_mentions(payload.agent_soul.prompt.system_prompt)
if mention.kind == MentionKind.KNOWLEDGE
}
existing_dataset_ids: set[str] | None = None
if mentioned_ids:
existing_dataset_ids = set(cls._dataset_rows(tenant_id=tenant_id, dataset_ids=sorted(mentioned_ids)))
findings = ComposerConfigValidator.collect_soft_findings(payload, existing_dataset_ids=existing_dataset_ids)
if agent_id and payload.agent_soul is not None:
findings["warnings"].extend(
cls._drive_mention_findings(
tenant_id=tenant_id,
agent_id=agent_id,
agent_soul=payload.agent_soul,
prompt=payload.agent_soul.prompt.system_prompt,
)
)
return findings
@classmethod
def validate_knowledge_datasets(cls, *, tenant_id: str, agent_soul: AgentSoulConfig | None) -> None:
"""Hard-validate tenant-scoped knowledge set datasets before saving.
DTO validators own set shape, duplicate set ids/names, and duplicate
dataset ids within one set. This service-level check owns database
existence and tenant ownership so invalid or cross-tenant datasets fail
before Agent Soul snapshots are persisted.
"""
if agent_soul is None:
return
missing_ids = list_missing_tenant_knowledge_dataset_ids(tenant_id=tenant_id, agent_soul=agent_soul)
if missing_ids:
raise InvalidComposerConfigError(
"knowledge_dataset_not_found: knowledge sets reference missing or out-of-scope datasets: "
+ ", ".join(missing_ids)
)
@classmethod
def resolve_bound_agent_id(cls, *, tenant_id: str, app_id: str) -> str | None:
"""The Agent App's bound roster agent id, if any (validate-endpoint context)."""
@@ -584,16 +426,14 @@ class AgentComposerService:
*,
tenant_id: str,
agent_id: str,
agent_soul: AgentSoulConfig,
prompt: str,
) -> list[dict[str, str | None]]:
"""Soft warnings for missing drive-backed prompt mentions."""
from services.agent.prompt_mentions import MentionKind, parse_prompt_mentions
from services.agent_drive_service import decode_drive_mention_ref
soul_skill_keys = {skill.skill_md_key for skill in agent_soul.files.skills if skill.skill_md_key}
soul_file_keys = {file_ref.drive_key for file_ref in agent_soul.files.files if file_ref.drive_key}
wanted_keys: dict[str, tuple[str, str]] = {}
for mention in parse_prompt_mentions(agent_soul.prompt.system_prompt):
for mention in parse_prompt_mentions(prompt):
if mention.kind not in {MentionKind.SKILL, MentionKind.FILE}:
continue
decoded_key = decode_drive_mention_ref(mention.ref_id)
@@ -614,28 +454,6 @@ class AgentComposerService:
)
findings: list[dict[str, str | None]] = []
for key, (kind, display) in wanted_keys.items():
if kind == MentionKind.SKILL.value and key not in soul_skill_keys:
findings.append(
{
"code": "mention_target_missing",
"surface": "agent_soul",
"kind": kind,
"id": key,
"message": f"{kind} '{display}' is not recorded in this Agent Soul version.",
}
)
continue
if kind == MentionKind.FILE.value and key not in soul_file_keys:
findings.append(
{
"code": "mention_target_missing",
"surface": "agent_soul",
"kind": kind,
"id": key,
"message": f"{kind} '{display}' is not recorded in this Agent Soul version.",
}
)
continue
if key in existing_keys:
continue
findings.append(
@@ -691,7 +509,7 @@ class AgentComposerService:
soul_lists, soul_truncated = soul_candidates(
agent_soul=agent_soul,
dataset_lookup=lambda ids: get_tenant_knowledge_dataset_rows(tenant_id=tenant_id, dataset_ids=ids),
dataset_lookup=lambda ids: cls._dataset_rows(tenant_id=tenant_id, dataset_ids=ids),
workspace_tools_loader=lambda: cls._workspace_dify_tools(tenant_id=tenant_id, user_id=user_id),
)
truncated = truncated or soul_truncated
@@ -718,7 +536,7 @@ class AgentComposerService:
agent_soul = cls._load_agent_app_soul(tenant_id=tenant_id, app_id=app_id)
soul_lists, truncated = soul_candidates(
agent_soul=agent_soul,
dataset_lookup=lambda ids: get_tenant_knowledge_dataset_rows(tenant_id=tenant_id, dataset_ids=ids),
dataset_lookup=lambda ids: cls._dataset_rows(tenant_id=tenant_id, dataset_ids=ids),
workspace_tools_loader=lambda: cls._workspace_dify_tools(tenant_id=tenant_id, user_id=user_id),
)
response = ComposerCandidatesResponse(
@@ -751,17 +569,23 @@ class AgentComposerService:
@classmethod
def _load_agent_app_soul(cls, *, tenant_id: str, app_id: str) -> AgentSoulConfig | None:
agent = cls._get_agent_app_agent(tenant_id=tenant_id, app_id=app_id)
agent = db.session.scalar(
select(Agent)
.where(
Agent.tenant_id == tenant_id,
Agent.app_id == app_id,
Agent.scope == AgentScope.ROSTER,
Agent.status == AgentStatus.ACTIVE,
)
.order_by(Agent.created_at.desc())
.limit(1)
)
if agent is None:
return None
draft = cls._get_or_create_agent_draft(
tenant_id=tenant_id,
agent=agent,
draft_type=AgentConfigDraftType.DRAFT,
account_id=None,
created_by=agent.updated_by or agent.created_by,
version = cls._get_version_if_present(
tenant_id=tenant_id, agent_id=agent.id, version_id=agent.active_config_snapshot_id
)
return AgentSoulConfig.model_validate(draft.config_snapshot_dict)
return cls._parse_soul_snapshot(version)
@staticmethod
def _parse_soul_snapshot(version: AgentConfigSnapshot | None) -> AgentSoulConfig | None:
@@ -805,6 +629,30 @@ class AgentComposerService:
variables = WorkflowDraftVariableService(session=session).list_system_variables(app_id, user_id)
return [(variable.name, variable.value_type.value) for variable in variables.variables]
@staticmethod
def _dataset_rows(*, tenant_id: str, dataset_ids: list[str]) -> dict[str, Any]:
"""Tenant-scoped dataset lookup tolerating malformed ids.
Mention ids come from user-editable prompt text; a non-UUID id can never
match a dataset row, so it is simply absent from the result (-> missing/
placeholder semantics) instead of breaking the UUID-typed query.
"""
from uuid import UUID
from services.dataset_service import DatasetService
valid_ids: list[str] = []
for dataset_id in dataset_ids:
try:
UUID(dataset_id)
except (ValueError, TypeError):
continue
valid_ids.append(dataset_id)
if not valid_ids:
return {}
rows, _ = DatasetService.get_datasets_by_ids(valid_ids, tenant_id)
return {str(row.id): row for row in rows}
@staticmethod
def _workspace_dify_tools(*, tenant_id: str, user_id: str) -> list[dict[str, Any]]:
"""Workspace Dify Plugin tools, same source as the tool selector.
@@ -915,11 +763,6 @@ class AgentComposerService:
)
binding.node_job_config = node_job
if payload.agent_soul is not None and binding.binding_type == WorkflowAgentBindingType.INLINE_AGENT:
payload.agent_soul = cls._preserve_active_soul_files(
tenant_id=tenant_id,
agent_id=binding.agent_id,
agent_soul=payload.agent_soul,
)
current_snapshot = cls._require_version(
tenant_id=tenant_id,
agent_id=binding.agent_id,
@@ -1020,11 +863,6 @@ class AgentComposerService:
binding = cls._require_binding(binding)
if payload.agent_soul is None:
raise ValueError("agent_soul is required")
payload.agent_soul = cls._preserve_active_soul_files(
tenant_id=tenant_id,
agent_id=binding.agent_id,
agent_soul=payload.agent_soul,
)
current_snapshot = cls._require_version(
tenant_id=tenant_id,
agent_id=binding.agent_id,
@@ -1059,11 +897,6 @@ class AgentComposerService:
binding = cls._require_binding(binding)
if not binding.agent_id or payload.agent_soul is None:
raise ValueError("agent_id and agent_soul are required")
payload.agent_soul = cls._preserve_active_soul_files(
tenant_id=tenant_id,
agent_id=binding.agent_id,
agent_soul=payload.agent_soul,
)
version = cls._create_config_version(
tenant_id=tenant_id,
agent_id=binding.agent_id,
@@ -1096,12 +929,6 @@ class AgentComposerService:
) -> WorkflowAgentNodeBinding:
if payload.agent_soul is None:
raise ValueError("agent_soul is required")
if binding and binding.agent_id:
payload.agent_soul = cls._preserve_active_soul_files(
tenant_id=tenant_id,
agent_id=binding.agent_id,
agent_soul=payload.agent_soul,
)
agent_name = payload.new_agent_name or "Untitled Agent"
agent = cls._create_roster_agent_for_composer(
tenant_id=tenant_id,
@@ -1117,15 +944,6 @@ class AgentComposerService:
version_note=payload.version_note,
)
node_job = payload.node_job or WorkflowNodeJobConfig()
if binding and binding.agent_id:
cls._copy_agent_drive_rows(
tenant_id=tenant_id,
source_agent_id=binding.agent_id,
target_agent_id=agent.id,
account_id=account_id,
agent_soul=payload.agent_soul,
node_job=node_job,
)
if not binding:
binding = WorkflowAgentNodeBinding(
tenant_id=tenant_id,
@@ -1161,9 +979,6 @@ class AgentComposerService:
version_id=binding.current_snapshot_id,
)
agent_soul = payload.agent_soul or AgentSoulConfig.model_validate(source_version.config_snapshot_dict)
source_soul = AgentSoulConfig.model_validate(source_version.config_snapshot_dict)
agent_soul = agent_soul.model_copy(deep=True)
agent_soul.files = source_soul.files
agent_name = payload.new_agent_name or source_agent.name
roster_agent = cls._create_roster_agent_for_composer(
tenant_id=tenant_id,
@@ -1305,63 +1120,26 @@ class AgentComposerService:
)
)
@classmethod
def _preserve_active_soul_files(
cls,
*,
tenant_id: str,
agent_id: str | None,
agent_soul: AgentSoulConfig,
) -> AgentSoulConfig:
"""Keep drive refs owned by drive APIs when saving non-file composer changes."""
if not agent_id:
return agent_soul
agent = cls._get_agent_if_present(tenant_id=tenant_id, agent_id=agent_id)
if agent is None or not agent.active_config_snapshot_id:
return agent_soul
version = cls._get_version_if_present(
tenant_id=tenant_id,
agent_id=agent.id,
version_id=agent.active_config_snapshot_id,
)
if version is None:
return agent_soul
existing_soul = AgentSoulConfig.model_validate(version.config_snapshot_dict)
preserved = agent_soul.model_copy(deep=True)
preserved.files = existing_soul.files
return preserved
@classmethod
def _preserve_agent_draft_soul_files(
cls,
*,
tenant_id: str,
agent_id: str | None,
agent_soul: AgentSoulConfig,
draft_type: AgentConfigDraftType = AgentConfigDraftType.DRAFT,
account_id: str | None = None,
) -> AgentSoulConfig:
if not agent_id:
return agent_soul
draft = cls._get_agent_draft(
tenant_id=tenant_id,
agent_id=agent_id,
draft_type=draft_type,
account_id=account_id,
)
if draft is not None:
existing_soul = AgentSoulConfig.model_validate(draft.config_snapshot_dict)
preserved = agent_soul.model_copy(deep=True)
preserved.files = existing_soul.files
return preserved
return cls._preserve_active_soul_files(tenant_id=tenant_id, agent_id=agent_id, agent_soul=agent_soul)
@staticmethod
def _drive_copy_scopes_from_agent_configs(
*, agent_soul: AgentSoulConfig, node_job: WorkflowNodeJobConfig | None = None
) -> tuple[set[str], set[str]]:
exact_keys, prefixes = AgentSoulFilesService.drive_copy_scopes(agent_soul=agent_soul)
from services.agent.prompt_mentions import MentionKind, parse_prompt_mentions
from services.agent_drive_service import decode_drive_mention_ref
exact_keys: set[str] = set()
prefixes: set[str] = set()
for mention in parse_prompt_mentions(agent_soul.prompt.system_prompt):
if mention.kind not in {MentionKind.SKILL, MentionKind.FILE}:
continue
drive_key = decode_drive_mention_ref(mention.ref_id)
if not drive_key:
continue
if mention.kind == MentionKind.SKILL and "/" in drive_key:
prefixes.add(f"{drive_key.rsplit('/', 1)[0]}/")
else:
exact_keys.add(drive_key)
if node_job is not None:
for file_ref in node_job.metadata.file_refs or []:
@@ -1507,143 +1285,6 @@ class AgentComposerService:
or 0
) + 1
@classmethod
def _get_agent_app_agent(cls, *, tenant_id: str, app_id: str) -> Agent | None:
return db.session.scalar(
select(Agent)
.where(
Agent.tenant_id == tenant_id,
Agent.app_id == app_id,
Agent.scope == AgentScope.ROSTER,
Agent.source == AgentSource.AGENT_APP,
Agent.status == AgentStatus.ACTIVE,
)
.order_by(Agent.created_at.desc())
.limit(1)
)
@classmethod
def _require_agent_app_agent(cls, *, tenant_id: str, app_id: str) -> Agent:
agent = cls._get_agent_app_agent(tenant_id=tenant_id, app_id=app_id)
if agent is None:
raise AgentNotFoundError()
return agent
@classmethod
def _get_agent_draft(
cls,
*,
tenant_id: str,
agent_id: str,
draft_type: AgentConfigDraftType,
account_id: str | None,
) -> AgentConfigDraft | None:
stmt = select(AgentConfigDraft).where(
AgentConfigDraft.tenant_id == tenant_id,
AgentConfigDraft.agent_id == agent_id,
AgentConfigDraft.draft_type == draft_type,
)
if draft_type == AgentConfigDraftType.DEBUG_BUILD:
stmt = stmt.where(AgentConfigDraft.account_id == account_id)
else:
stmt = stmt.where(AgentConfigDraft.account_id.is_(None))
return db.session.scalar(stmt.order_by(AgentConfigDraft.updated_at.desc()).limit(1))
@classmethod
def _get_or_create_agent_draft(
cls,
*,
tenant_id: str,
agent: Agent,
draft_type: AgentConfigDraftType,
account_id: str | None,
created_by: str | None,
) -> AgentConfigDraft:
draft = cls._get_agent_draft(
tenant_id=tenant_id,
agent_id=agent.id,
draft_type=draft_type,
account_id=account_id,
)
if draft is not None:
return draft
base_snapshot = cls._get_version_if_present(
tenant_id=tenant_id,
agent_id=agent.id,
version_id=agent.active_config_snapshot_id,
)
agent_soul = (
AgentSoulConfig.model_validate(base_snapshot.config_snapshot_dict)
if base_snapshot is not None
else AgentSoulConfig()
)
draft = AgentConfigDraft(
tenant_id=tenant_id,
agent_id=agent.id,
draft_type=draft_type,
account_id=account_id if draft_type == AgentConfigDraftType.DEBUG_BUILD else None,
draft_owner_key=account_id if draft_type == AgentConfigDraftType.DEBUG_BUILD and account_id else "",
base_snapshot_id=base_snapshot.id if base_snapshot else None,
config_snapshot=agent_soul,
created_by=created_by,
updated_by=created_by,
)
db.session.add(draft)
db.session.flush()
return draft
@classmethod
def _save_agent_draft(
cls,
*,
tenant_id: str,
agent: Agent,
draft_type: AgentConfigDraftType,
account_id: str | None,
agent_soul: AgentSoulConfig,
account_id_for_audit: str,
base_snapshot_id: str | None = None,
) -> AgentConfigDraft:
draft = cls._get_or_create_agent_draft(
tenant_id=tenant_id,
agent=agent,
draft_type=draft_type,
account_id=account_id,
created_by=account_id_for_audit,
)
draft.config_snapshot = agent_soul
if base_snapshot_id is not None:
draft.base_snapshot_id = base_snapshot_id
elif draft.base_snapshot_id is None:
draft.base_snapshot_id = agent.active_config_snapshot_id
draft.updated_by = account_id_for_audit
db.session.flush()
return draft
@classmethod
def _serialize_draft(cls, draft: AgentConfigDraft | None) -> dict[str, Any] | None:
if draft is None:
return None
return {
"id": draft.id,
"agent_id": draft.agent_id,
"draft_type": draft.draft_type.value,
"account_id": draft.account_id,
"base_snapshot_id": draft.base_snapshot_id,
"created_by": draft.created_by,
"updated_by": draft.updated_by,
"created_at": to_timestamp(draft.created_at),
"updated_at": to_timestamp(draft.updated_at),
}
@classmethod
def _serialize_build_draft_state(cls, draft: AgentConfigDraft) -> dict[str, Any]:
return {
"variant": ComposerVariant.AGENT_APP.value,
"draft": cls._serialize_draft(draft),
"agent_soul": draft.config_snapshot_dict,
}
@classmethod
def _get_draft_workflow(cls, *, tenant_id: str, app_id: str) -> Workflow:
workflow = db.session.scalar(
@@ -1838,10 +1479,6 @@ class AgentComposerService:
"id": agent.id,
"name": agent.name,
"description": agent.description,
"role": agent.role,
"icon_type": agent.icon_type,
"icon": agent.icon,
"icon_background": agent.icon_background,
"scope": agent.scope.value,
"status": agent.status.value,
"active_config_snapshot_id": agent.active_config_snapshot_id,
+5 -5
View File
@@ -148,15 +148,15 @@ class ComposerConfigValidator:
cls,
payload: ComposerSavePayload,
*,
existing_knowledge_set_ids: set[str] | None = None,
existing_dataset_ids: set[str] | None = None,
) -> dict[str, Any]:
"""ENG-617 §5.3/§5.4 soft findings — never block save.
``warnings`` carries ``mention_target_missing`` / ``mention_malformed``
entries; ``knowledge_retrieval_placeholder`` keeps dangling knowledge-set
entries; ``knowledge_retrieval_placeholder`` keeps dangling knowledge
mentions with a placeholder name (0522 consensus) instead of dropping or
rejecting them. With ``existing_knowledge_set_ids`` provided, mentions
that no longer exist in the current Agent Soul surface as placeholders too.
rejecting them. With ``existing_dataset_ids`` provided, configured-but-
deleted datasets surface as placeholders too.
"""
warnings: list[dict[str, Any]] = []
placeholders: list[dict[str, str]] = []
@@ -188,7 +188,7 @@ class ComposerConfigValidator:
resolved = resolver(mention)
if mention.kind == MentionKind.KNOWLEDGE:
dangling = resolved is None or (
existing_knowledge_set_ids is not None and mention.ref_id not in existing_knowledge_set_ids
existing_dataset_ids is not None and mention.ref_id not in existing_dataset_ids
)
if dangling:
placeholders.append(
-63
View File
@@ -1,63 +0,0 @@
from __future__ import annotations
from typing import Any
from uuid import UUID
from models.agent_config_entities import AgentSoulConfig
def list_agent_soul_knowledge_dataset_ids(agent_soul: AgentSoulConfig) -> list[str]:
"""Return normalized unique knowledge dataset ids in config order.
Agent v2 knowledge dataset selection is owned by ``knowledge.sets``. This
helper keeps composer, workflow validation, candidates, and runtime
diagnostics aligned on the same normalization rules: strip whitespace, drop
blanks, preserve first-seen order, and deduplicate.
"""
dataset_ids: list[str] = []
seen: set[str] = set()
for knowledge_set in agent_soul.knowledge.sets:
for dataset in knowledge_set.datasets:
dataset_id = (dataset.id or "").strip()
if not dataset_id or dataset_id in seen:
continue
seen.add(dataset_id)
dataset_ids.append(dataset_id)
return dataset_ids
def get_tenant_knowledge_dataset_rows(*, tenant_id: str, dataset_ids: list[str]) -> dict[str, Any]:
"""Return tenant-scoped dataset rows for normalized knowledge dataset ids.
Knowledge ids come from user-editable config. Malformed ids can never match
a dataset row, so they are treated as missing instead of breaking the
UUID-typed dataset lookup.
"""
from services.dataset_service import DatasetService
valid_ids: list[str] = []
for dataset_id in dataset_ids:
try:
UUID(dataset_id)
except (TypeError, ValueError):
continue
valid_ids.append(dataset_id)
if not valid_ids:
return {}
rows, _ = DatasetService.get_datasets_by_ids(valid_ids, tenant_id)
return {str(row.id): row for row in rows}
def list_missing_tenant_knowledge_dataset_ids(*, tenant_id: str, agent_soul: AgentSoulConfig | None) -> list[str]:
"""Return normalized knowledge dataset ids missing from the tenant scope."""
if agent_soul is None:
return []
dataset_ids = list_agent_soul_knowledge_dataset_ids(agent_soul)
if not dataset_ids:
return []
rows = get_tenant_knowledge_dataset_rows(tenant_id=tenant_id, dataset_ids=dataset_ids)
return [dataset_id for dataset_id in dataset_ids if dataset_id not in rows]
+4 -4
View File
@@ -6,7 +6,7 @@ Slash-menu insertions are stored inline in the plain-string prompt as tokens:
``kind`` is a fixed lowercase word; ``id`` points at an item in the Agent
runtime context. For prompt-owned entities that means Agent Soul lists such as
``tools`` / ``knowledge.sets`` / ``human.contacts`` and workflow job lists
``tools`` / ``knowledge.datasets`` / ``human.contacts`` and workflow job lists
such as ``previous_node_output_refs`` / ``declared_outputs``. For drive-backed
``skill`` / ``file`` mentions the field stores a URL-encoded drive key and is
resolved against ``agent_drive_files`` at runtime. ``label`` is an optional
@@ -211,9 +211,9 @@ def build_soul_mention_resolver(agent_soul: AgentSoulConfig) -> MentionResolver:
if mention.ref_id in (cli_tool.id, cli_tool.name):
return cli_tool.name or cli_tool.id
case MentionKind.KNOWLEDGE:
for knowledge_set in agent_soul.knowledge.sets:
if mention.ref_id == knowledge_set.id:
return knowledge_set.name or knowledge_set.id
for dataset in agent_soul.knowledge.datasets:
if mention.ref_id == dataset.id:
return dataset.name or dataset.id
case MentionKind.HUMAN:
return _resolve_human_contact(agent_soul.human.contacts, mention.ref_id)
case _:
+19 -54
View File
@@ -8,8 +8,6 @@ from libs.datetime_utils import naive_utc_now
from libs.helper import to_timestamp
from models.agent import (
Agent,
AgentConfigDraft,
AgentConfigDraftType,
AgentConfigRevision,
AgentConfigRevisionOperation,
AgentConfigSnapshot,
@@ -576,7 +574,7 @@ class AgentRosterService:
Agent.status == AgentStatus.ACTIVE,
)
).all()
return {agent.app_id: agent for agent in agents if agent.app_id and agent.id}
return {agent.app_id: agent for agent in agents if agent.app_id}
def get_app_backing_agent(self, *, tenant_id: str, app_id: str) -> Agent | None:
"""Return the roster Agent that backs the given Agent App, if any."""
@@ -838,7 +836,6 @@ class AgentRosterService:
def _visible_version_operations(agent: Agent) -> set[AgentConfigRevisionOperation]:
if agent.source == AgentSource.AGENT_APP:
return {
AgentConfigRevisionOperation.PUBLISH_DRAFT,
AgentConfigRevisionOperation.SAVE_NEW_VERSION,
AgentConfigRevisionOperation.SAVE_TO_ROSTER,
AgentConfigRevisionOperation.RESTORE_VERSION,
@@ -852,37 +849,16 @@ class AgentRosterService:
}
def active_config_is_published(self, *, tenant_id: str, agent: Agent) -> bool:
"""Return whether the editable draft matches the active published snapshot."""
"""Return whether the Agent's current active snapshot is a visible published version."""
return self.load_active_config_is_published_by_agent_id(tenant_id=tenant_id, agents=[agent]).get(
agent.id,
False,
)
def load_active_config_is_published_by_agent_id(self, *, tenant_id: str, agents: list[Agent]) -> dict[str, bool]:
"""Return whether each Agent's normal draft is aligned with its active published snapshot."""
agents = [agent for agent in agents if agent.id]
if not agents:
return {}
"""Return publish-state flags for the active config snapshots of the given Agents."""
published_agent_ids = self._load_published_active_snapshot_agent_ids(tenant_id=tenant_id, agents=agents)
drafts = self._session.scalars(
select(AgentConfigDraft).where(
AgentConfigDraft.tenant_id == tenant_id,
AgentConfigDraft.agent_id.in_([agent.id for agent in agents]),
AgentConfigDraft.draft_type == AgentConfigDraftType.DRAFT,
AgentConfigDraft.account_id.is_(None),
)
).all()
drafts_by_agent_id = {draft.agent_id: draft for draft in drafts}
result: dict[str, bool] = {}
for agent in agents:
draft = drafts_by_agent_id.get(agent.id)
result[agent.id] = (
agent.id in published_agent_ids
and bool(agent.active_config_snapshot_id)
and (draft is None or draft.base_snapshot_id == agent.active_config_snapshot_id)
)
return result
return {agent.id: agent.id in published_agent_ids for agent in agents}
def list_agent_versions(self, *, tenant_id: str, agent_id: str) -> list[dict[str, Any]]:
agent = self._get_agent(tenant_id=tenant_id, agent_id=agent_id, roster_only=True)
@@ -981,37 +957,26 @@ class AgentRosterService:
raise AgentVersionNotFoundError()
version = self._get_version(tenant_id=tenant_id, agent_id=agent_id, version_id=version_id)
draft = self._session.scalar(
select(AgentConfigDraft)
.where(
AgentConfigDraft.tenant_id == tenant_id,
AgentConfigDraft.agent_id == agent_id,
AgentConfigDraft.draft_type == AgentConfigDraftType.DRAFT,
AgentConfigDraft.account_id.is_(None),
)
.limit(1)
)
if draft is None:
draft = AgentConfigDraft(
if agent.active_config_snapshot_id == version.id:
return {"result": "success", "active_config_snapshot_id": version.id}
previous_snapshot_id = agent.active_config_snapshot_id
agent.active_config_snapshot_id = version.id
agent.active_config_has_model = agent_soul_has_model(version.config_snapshot)
agent.updated_by = account_id
self._session.add(
AgentConfigRevision(
tenant_id=tenant_id,
agent_id=agent_id,
draft_type=AgentConfigDraftType.DRAFT,
account_id=None,
draft_owner_key="",
previous_snapshot_id=previous_snapshot_id,
current_snapshot_id=version.id,
revision=self._next_revision(tenant_id=tenant_id, agent_id=agent_id),
operation=AgentConfigRevisionOperation.RESTORE_VERSION,
created_by=account_id,
)
self._session.add(draft)
draft.base_snapshot_id = version.id
draft.config_snapshot = AgentSoulConfig.model_validate(version.config_snapshot_dict)
draft.updated_by = account_id
agent.updated_by = account_id
)
self._session.commit()
return {
"result": "success",
"active_config_snapshot_id": agent.active_config_snapshot_id or version.id,
"draft_config_id": draft.id,
"restored_version_id": version.id,
}
return {"result": "success", "active_config_snapshot_id": version.id}
def _get_agent(self, *, tenant_id: str, agent_id: str, roster_only: bool = False) -> Agent:
stmt = select(Agent).where(Agent.tenant_id == tenant_id, Agent.id == agent_id)
@@ -50,7 +50,6 @@ class SkillStandardizeService:
self._package = package_service or SkillPackageService()
self._drive = drive_service or AgentDriveService()
self._tool_files = tool_file_manager or ToolFileManager()
self.last_committed_items: list[dict[str, Any]] = []
def standardize(
self,
@@ -110,7 +109,7 @@ class SkillStandardizeService:
)
)
committed_items = self._drive.commit(
self._drive.commit(
tenant_id=tenant_id,
user_id=user_id,
agent_id=agent_id,
@@ -134,7 +133,6 @@ class SkillStandardizeService:
*member_items,
],
)
self.last_committed_items = committed_items
drive_skill = next(
skill
-528
View File
@@ -1,528 +0,0 @@
from __future__ import annotations
import json
from typing import Any
from sqlalchemy import func, select
from sqlalchemy.orm import Session
from extensions.ext_database import db
from models.agent import (
Agent,
AgentConfigRevision,
AgentConfigRevisionOperation,
AgentConfigSnapshot,
AgentDriveFile,
)
from models.agent_config_entities import AgentFileRefConfig, AgentSkillRefConfig, AgentSoulConfig
from services.agent.agent_soul_state import agent_soul_has_model
from services.agent_drive_service import AgentDriveError, DriveSkillMetadata, normalize_drive_key
_SKILL_MD_SUFFIX = "/SKILL.md"
_SKILL_ARCHIVE_NAME = ".DIFY-SKILL-FULL.zip"
_FILES_PREFIX = "files/"
class AgentSoulFilesService:
"""Versioned Agent Soul view of drive-backed skills and files.
``agent_drive_files`` remains the storage/index for bytes and drive values.
``AgentSoulConfig.files`` records the versioned pointers that a specific
Agent Soul snapshot owns, so restore/publish/runtime do not accidentally see
later drive mutations.
"""
@classmethod
def sync_drive_commit_to_active_soul(
cls,
*,
tenant_id: str,
agent_id: str,
account_id: str,
committed_items: list[dict[str, Any]],
) -> AgentConfigSnapshot | None:
if not committed_items:
return None
agent = db.session.scalar(select(Agent).where(Agent.tenant_id == tenant_id, Agent.id == agent_id))
if agent is None or not agent.active_config_snapshot_id:
return None
current_snapshot = db.session.scalar(
select(AgentConfigSnapshot).where(
AgentConfigSnapshot.tenant_id == tenant_id,
AgentConfigSnapshot.agent_id == agent_id,
AgentConfigSnapshot.id == agent.active_config_snapshot_id,
)
)
if current_snapshot is None:
return None
agent_soul = AgentSoulConfig.model_validate(current_snapshot.config_snapshot_dict).model_copy(deep=True)
before = agent_soul.files.model_dump(mode="json")
for item in committed_items:
cls._apply_commit_item(agent_soul=agent_soul, item=item)
if agent_soul.files.model_dump(mode="json") == before:
return None
version = cls._create_config_version(
tenant_id=tenant_id,
agent_id=agent_id,
account_id=account_id,
agent_soul=agent_soul,
previous_snapshot_id=current_snapshot.id,
)
agent.active_config_snapshot_id = version.id
agent.active_config_has_model = agent_soul_has_model(agent_soul)
agent.updated_by = account_id
db.session.flush()
return version
@classmethod
def list_files(
cls,
*,
session: Session,
tenant_id: str,
agent_id: str,
prefix: str = "",
) -> list[dict[str, Any]]:
agent_soul = cls.active_agent_soul(session=session, tenant_id=tenant_id, agent_id=agent_id)
file_keys = [file.drive_key for file in agent_soul.files.files if file.drive_key]
if prefix:
normalized_prefix = normalize_drive_key(prefix)
file_keys = [key for key in file_keys if key.startswith(normalized_prefix)]
if not file_keys:
return []
rows = cls._drive_rows_by_key(session=session, tenant_id=tenant_id, agent_id=agent_id, keys=file_keys)
items: list[dict[str, Any]] = []
for file_ref in agent_soul.files.files:
key = file_ref.drive_key
if not key or key not in file_keys:
continue
row = rows.get(key)
item = cls._file_item_from_ref(file_ref)
item.update(cls._row_item(row) if row is not None else {"key": key, "missing": True})
items.append(item)
return items
@classmethod
def list_manifest_items(
cls,
*,
session: Session,
tenant_id: str,
agent_id: str,
prefix: str = "",
) -> list[dict[str, Any]]:
agent_soul = cls.active_agent_soul(session=session, tenant_id=tenant_id, agent_id=agent_id)
refs = cls._all_file_refs(agent_soul)
if prefix:
normalized_prefix = normalize_drive_key(prefix)
refs = [ref for ref in refs if (ref.drive_key or "").startswith(normalized_prefix)]
keys = [ref.drive_key for ref in refs if ref.drive_key]
rows = cls._drive_rows_by_key(session=session, tenant_id=tenant_id, agent_id=agent_id, keys=keys)
items: list[dict[str, Any]] = []
for file_ref in refs:
key = file_ref.drive_key
if not key:
continue
row = rows.get(key)
item = cls._file_item_from_ref(file_ref)
item.update(cls._row_item(row) if row is not None else {"key": key, "missing": True})
item["file_id"] = file_ref.file_id or file_ref.upload_file_id
items.append(item)
return sorted(items, key=lambda item: str(item.get("key") or ""))
@classmethod
def list_skills(
cls,
*,
session: Session,
tenant_id: str,
agent_id: str,
) -> list[dict[str, Any]]:
agent_soul = cls.active_agent_soul(session=session, tenant_id=tenant_id, agent_id=agent_id)
skill_keys = [skill.skill_md_key for skill in agent_soul.files.skills if skill.skill_md_key]
archive_keys = [skill.full_archive_key for skill in agent_soul.files.skills if skill.full_archive_key]
rows = cls._drive_rows_by_key(
session=session, tenant_id=tenant_id, agent_id=agent_id, keys=[*skill_keys, *archive_keys]
)
items: list[dict[str, Any]] = []
for skill in agent_soul.files.skills:
if not skill.skill_md_key:
continue
row = rows.get(skill.skill_md_key)
archive_key = skill.full_archive_key if skill.full_archive_key in rows else None
skill_md_ref = cls.file_ref_for_key(agent_soul=agent_soul, key=skill.skill_md_key)
items.append(
{
"path": skill.path or cls.skill_path_from_key(skill.skill_md_key),
"skill_md_key": skill.skill_md_key,
"archive_key": archive_key or skill.full_archive_key,
"name": skill.name,
"description": skill.description,
"size": row.size if row is not None else None,
"mime_type": row.mime_type if row is not None else (skill_md_ref.type if skill_md_ref else None),
"hash": row.hash if row is not None else None,
"created_at": int(row.created_at.timestamp()) if row is not None and row.created_at else None,
"missing": row is None,
}
)
return items
@classmethod
def allowed_drive_keys(cls, agent_soul: AgentSoulConfig) -> set[str]:
keys: set[str] = set()
for file_ref in agent_soul.files.files:
if file_ref.drive_key:
keys.add(file_ref.drive_key)
for skill in agent_soul.files.skills:
if skill.skill_md_key:
keys.add(skill.skill_md_key)
if skill.full_archive_key:
keys.add(skill.full_archive_key)
for file_ref in skill.file_refs:
if file_ref.drive_key:
keys.add(file_ref.drive_key)
return keys
@classmethod
def allowed_skill_prefixes(cls, agent_soul: AgentSoulConfig) -> set[str]:
prefixes: set[str] = set()
for skill in agent_soul.files.skills:
path = skill.path or (cls.skill_path_from_key(skill.skill_md_key) if skill.skill_md_key else None)
if path:
prefixes.add(f"{path}/")
return prefixes
@classmethod
def key_allowed_by_soul(cls, *, agent_soul: AgentSoulConfig, key: str) -> bool:
normalized_key = normalize_drive_key(key)
if normalized_key in cls.allowed_drive_keys(agent_soul):
return True
return any(normalized_key.startswith(prefix) for prefix in cls.allowed_skill_prefixes(agent_soul))
@classmethod
def file_ref_for_key(cls, *, agent_soul: AgentSoulConfig, key: str) -> AgentFileRefConfig | None:
normalized_key = normalize_drive_key(key)
for file_ref in agent_soul.files.files:
if file_ref.drive_key == normalized_key:
return file_ref
for skill in agent_soul.files.skills:
for file_ref in skill.file_refs:
if file_ref.drive_key == normalized_key:
return file_ref
return None
@classmethod
def drive_copy_scopes(cls, *, agent_soul: AgentSoulConfig) -> tuple[set[str], set[str]]:
exact_keys = cls.allowed_drive_keys(agent_soul)
prefixes = cls.allowed_skill_prefixes(agent_soul)
return exact_keys, prefixes
@staticmethod
def active_agent_soul(*, session: Session, tenant_id: str, agent_id: str) -> AgentSoulConfig:
snapshot = session.scalar(
select(AgentConfigSnapshot)
.join(Agent, Agent.active_config_snapshot_id == AgentConfigSnapshot.id)
.where(
Agent.tenant_id == tenant_id,
Agent.id == agent_id,
AgentConfigSnapshot.tenant_id == tenant_id,
AgentConfigSnapshot.agent_id == agent_id,
)
)
if snapshot is None:
raise AgentDriveError(
"agent_snapshot_not_found",
"agent has no active Agent Soul snapshot",
status_code=404,
)
return AgentSoulConfig.model_validate(snapshot.config_snapshot_dict)
@staticmethod
def skill_path_from_key(key: str) -> str:
if not key.endswith(_SKILL_MD_SUFFIX):
raise AgentDriveError(
"invalid_skill_key",
"skill rows must use the canonical '<path>/SKILL.md' key",
status_code=500,
)
return key[: -len(_SKILL_MD_SUFFIX)]
@staticmethod
def skill_archive_key(skill_md_key: str) -> str:
return f"{AgentSoulFilesService.skill_path_from_key(skill_md_key)}/{_SKILL_ARCHIVE_NAME}"
@classmethod
def _apply_commit_item(cls, *, agent_soul: AgentSoulConfig, item: dict[str, Any]) -> None:
key = normalize_drive_key(str(item.get("key") or ""))
if item.get("removed"):
cls._remove_ref(agent_soul=agent_soul, key=key)
return
if item.get("is_skill"):
cls._upsert_skill_ref(agent_soul=agent_soul, key=key, item=item)
return
if key.startswith(_FILES_PREFIX):
cls._upsert_file_ref(agent_soul=agent_soul, key=key, item=item)
return
cls._upsert_skill_file_ref(agent_soul=agent_soul, key=key, item=item)
@classmethod
def _upsert_skill_ref(cls, *, agent_soul: AgentSoulConfig, key: str, item: dict[str, Any]) -> None:
metadata = cls._parse_skill_metadata(item.get("skill_metadata"))
path = cls.skill_path_from_key(key)
ref = AgentSkillRefConfig(
id=path,
name=metadata.name,
description=metadata.description,
file_id=str(item.get("file_id") or ""),
path=path,
skill_md_key=key,
skill_md_file_id=str(item.get("file_id") or ""),
full_archive_key=cls.skill_archive_key(key),
manifest_files=metadata.manifest_files,
)
existing_ref = next(
(
existing
for existing in agent_soul.files.skills
if existing.skill_md_key == key or existing.path == path
),
None,
)
file_refs = list(existing_ref.file_refs) if existing_ref else []
file_refs = [file_ref for file_ref in file_refs if file_ref.drive_key != key]
file_refs.append(cls._file_ref_from_item(key=key, item=item, name="SKILL.md"))
archive_key = cls.skill_archive_key(key)
archive_ref = next((file_ref for file_ref in file_refs if file_ref.drive_key == archive_key), None)
if archive_ref:
ref.full_archive_file_id = archive_ref.file_id
ref.file_refs = sorted(file_refs, key=lambda value: value.drive_key or value.name)
skills = [
existing
for existing in agent_soul.files.skills
if existing.skill_md_key != key and existing.path != path
]
skills.append(ref)
skills.sort(key=lambda value: value.path or value.skill_md_key or "")
agent_soul.files.skills = skills
@staticmethod
def _upsert_file_ref(*, agent_soul: AgentSoulConfig, key: str, item: dict[str, Any]) -> None:
name = key.removeprefix(_FILES_PREFIX) or key.rsplit("/", 1)[-1]
file_id = str(item.get("file_id") or "")
ref = AgentFileRefConfig(
id=key,
file_id=file_id,
upload_file_id=file_id if item.get("file_kind") == "upload_file" else None,
name=name,
type=str(item.get("mime_type") or ""),
transfer_method=str(item.get("file_kind") or ""),
drive_key=key,
size=item.get("size"),
hash=item.get("hash"),
)
files = [existing for existing in agent_soul.files.files if existing.drive_key != key]
files.append(ref)
files.sort(key=lambda value: value.drive_key or value.name)
agent_soul.files.files = files
@classmethod
def _upsert_skill_file_ref(cls, *, agent_soul: AgentSoulConfig, key: str, item: dict[str, Any]) -> None:
path = key.split("/", 1)[0]
if not path:
return
updated: list[AgentSkillRefConfig] = []
changed = False
for skill in agent_soul.files.skills:
skill_path = skill.path or (cls.skill_path_from_key(skill.skill_md_key) if skill.skill_md_key else "")
if skill_path != path:
updated.append(skill)
continue
file_ref = cls._file_ref_from_item(key=key, item=item)
file_refs = [existing for existing in skill.file_refs if existing.drive_key != key]
file_refs.append(file_ref)
replacement = skill.model_copy(
update={"file_refs": sorted(file_refs, key=lambda value: value.drive_key or value.name)}
)
if key.endswith(f"/{_SKILL_ARCHIVE_NAME}"):
replacement = replacement.model_copy(
update={
"full_archive_key": key,
"full_archive_file_id": file_ref.file_id,
}
)
updated.append(replacement)
changed = True
if changed:
agent_soul.files.skills = updated
@staticmethod
def _file_ref_from_item(*, key: str, item: dict[str, Any], name: str | None = None) -> AgentFileRefConfig:
file_id = str(item.get("file_id") or "")
return AgentFileRefConfig(
id=key,
file_id=file_id,
upload_file_id=file_id if item.get("file_kind") == "upload_file" else None,
name=name or key.rsplit("/", 1)[-1],
type=str(item.get("mime_type") or ""),
transfer_method=str(item.get("file_kind") or ""),
drive_key=key,
size=item.get("size"),
hash=item.get("hash"),
)
@classmethod
def _remove_ref(cls, *, agent_soul: AgentSoulConfig, key: str) -> None:
agent_soul.files.files = [file_ref for file_ref in agent_soul.files.files if file_ref.drive_key != key]
if key.endswith(_SKILL_MD_SUFFIX):
path = cls.skill_path_from_key(key)
agent_soul.files.skills = [
skill for skill in agent_soul.files.skills if skill.skill_md_key != key and skill.path != path
]
return
if key.endswith(f"/{_SKILL_ARCHIVE_NAME}"):
agent_soul.files.skills = [
skill.model_copy(
update={
"full_archive_key": None,
"full_archive_file_id": None,
"file_refs": [file_ref for file_ref in skill.file_refs if file_ref.drive_key != key],
}
)
if skill.full_archive_key == key
else skill
for skill in agent_soul.files.skills
]
return
path = key.split("/", 1)[0]
if path:
agent_soul.files.skills = [
skill.model_copy(
update={"file_refs": [file_ref for file_ref in skill.file_refs if file_ref.drive_key != key]}
)
if (skill.path or (cls.skill_path_from_key(skill.skill_md_key) if skill.skill_md_key else "")) == path
else skill
for skill in agent_soul.files.skills
]
@staticmethod
def _all_file_refs(agent_soul: AgentSoulConfig) -> list[AgentFileRefConfig]:
refs = list(agent_soul.files.files)
for skill in agent_soul.files.skills:
refs.extend(skill.file_refs)
return refs
@staticmethod
def _parse_skill_metadata(raw_metadata: Any) -> DriveSkillMetadata:
if isinstance(raw_metadata, DriveSkillMetadata):
return raw_metadata
if isinstance(raw_metadata, str):
return DriveSkillMetadata.model_validate(json.loads(raw_metadata))
return DriveSkillMetadata.model_validate(raw_metadata or {})
@staticmethod
def _drive_rows_by_key(
*,
session: Session,
tenant_id: str,
agent_id: str,
keys: list[str],
) -> dict[str, AgentDriveFile]:
if not keys:
return {}
return {
row.key: row
for row in session.scalars(
select(AgentDriveFile).where(
AgentDriveFile.tenant_id == tenant_id,
AgentDriveFile.agent_id == agent_id,
AgentDriveFile.key.in_(sorted(set(keys))),
)
)
}
@staticmethod
def _row_item(row: AgentDriveFile | None) -> dict[str, Any]:
if row is None:
return {}
return {
"key": row.key,
"size": row.size,
"hash": row.hash,
"mime_type": row.mime_type,
"file_kind": row.file_kind.value,
"is_skill": row.is_skill,
"skill_metadata": row.skill_metadata,
"created_at": int(row.created_at.timestamp()) if row.created_at else None,
}
@staticmethod
def _file_item_from_ref(file_ref: AgentFileRefConfig) -> dict[str, Any]:
key = file_ref.drive_key or file_ref.name
return {
"key": key,
"name": file_ref.name,
"mime_type": file_ref.type,
"file_kind": file_ref.transfer_method,
"is_skill": False,
"size": file_ref.get("size"),
"hash": file_ref.get("hash"),
}
@classmethod
def _create_config_version(
cls,
*,
tenant_id: str,
agent_id: str,
account_id: str,
agent_soul: AgentSoulConfig,
previous_snapshot_id: str,
) -> AgentConfigSnapshot:
next_version = (
db.session.scalar(
select(func.max(AgentConfigSnapshot.version)).where(
AgentConfigSnapshot.tenant_id == tenant_id,
AgentConfigSnapshot.agent_id == agent_id,
)
)
or 0
) + 1
version = AgentConfigSnapshot(
tenant_id=tenant_id,
agent_id=agent_id,
version=next_version,
config_snapshot=agent_soul,
created_by=account_id,
)
db.session.add(version)
db.session.flush()
revision = AgentConfigRevision(
tenant_id=tenant_id,
agent_id=agent_id,
previous_snapshot_id=previous_snapshot_id,
current_snapshot_id=version.id,
revision=cls._next_revision(tenant_id=tenant_id, agent_id=agent_id),
operation=AgentConfigRevisionOperation.SAVE_CURRENT_VERSION,
created_by=account_id,
)
db.session.add(revision)
db.session.flush()
return version
@staticmethod
def _next_revision(*, tenant_id: str, agent_id: str) -> int:
return (
db.session.scalar(
select(func.max(AgentConfigRevision.revision)).where(
AgentConfigRevision.tenant_id == tenant_id,
AgentConfigRevision.agent_id == agent_id,
)
)
or 0
) + 1
+13 -72
View File
@@ -9,11 +9,10 @@ from sqlalchemy import select
from sqlalchemy.orm import Session
from core.workflow.nodes.agent_v2.validators import WorkflowAgentNodeValidationError, WorkflowAgentNodeValidator
from models import ToolFile, UploadFile
from models.agent import (
Agent,
AgentConfigSnapshot,
AgentDriveFileKind,
AgentDriveFile,
AgentScope,
AgentStatus,
WorkflowAgentBindingType,
@@ -22,7 +21,6 @@ from models.agent import (
from models.agent_config_entities import AgentSoulConfig, DeclaredOutputConfig, WorkflowNodeJobConfig
from models.workflow import Workflow
from services.agent.composer_validator import ComposerConfigValidator
from services.agent.soul_files_service import AgentSoulFilesService
from services.entities.agent_entities import (
ComposerSavePayload,
ComposerSaveStrategy,
@@ -41,10 +39,10 @@ class WorkflowAgentPublishService:
@classmethod
def project_draft_bindings_to_graph(cls, *, session: Session, draft_workflow: Workflow) -> dict[str, Any]:
"""Return draft graph with persisted Agent binding fields projected into node data.
"""Return draft graph with persisted Agent node job config projected into node data.
Workflow draft graph is the front-end's editing source of truth, while
runtime/publish reads WorkflowAgentNodeBinding. This
runtime/publish reads WorkflowAgentNodeBinding.node_job_config. This
response-only projection keeps reads aligned without writing binding
details back into the stored graph JSON.
"""
@@ -66,18 +64,6 @@ class WorkflowAgentPublishService:
node_data = agent_nodes.get(binding.node_id)
if not isinstance(node_data, dict):
continue
graph_binding = node_data.get(cls._AGENT_BINDING_KEY)
is_pending_inline_graph_binding = (
isinstance(graph_binding, Mapping)
and graph_binding.get("binding_type") == WorkflowAgentBindingType.INLINE_AGENT.value
and (not graph_binding.get("agent_id") or not graph_binding.get("current_snapshot_id"))
)
if not is_pending_inline_graph_binding or binding.binding_type == WorkflowAgentBindingType.INLINE_AGENT:
node_data[cls._AGENT_BINDING_KEY] = {
"binding_type": binding.binding_type.value,
"agent_id": binding.agent_id,
"current_snapshot_id": binding.current_snapshot_id,
}
node_job = WorkflowNodeJobConfig.model_validate(binding.node_job_config_dict)
if node_job.workflow_prompt is not None:
node_data[cls._AGENT_TASK_KEY] = node_job.workflow_prompt
@@ -183,8 +169,6 @@ class WorkflowAgentPublishService:
from services.agent_drive_service import decode_drive_mention_ref
wanted_keys: dict[str, tuple[str, str]] = {}
soul_skill_keys = {skill.skill_md_key for skill in agent_soul.files.skills if skill.skill_md_key}
soul_file_keys = AgentSoulFilesService.allowed_drive_keys(agent_soul=agent_soul) - soul_skill_keys
for mention in parse_prompt_mentions(agent_soul.prompt.system_prompt):
if mention.kind not in {MentionKind.SKILL, MentionKind.FILE}:
continue
@@ -193,67 +177,29 @@ class WorkflowAgentPublishService:
continue
code = "skill_ref_dangling" if mention.kind == MentionKind.SKILL else "file_ref_dangling"
wanted_keys[drive_key] = (code, mention.label or drive_key)
if not binding.agent_id:
if not wanted_keys or not binding.agent_id:
return
declared_keys, _ = AgentSoulFilesService.drive_copy_scopes(agent_soul=agent_soul)
check_keys = sorted(set(wanted_keys) | declared_keys)
if not check_keys:
return
existing_keys = {
key
for key in check_keys
if cls._soul_file_ref_exists(session=session, tenant_id=binding.tenant_id, agent_soul=agent_soul, key=key)
}
existing_keys = set(
session.scalars(
select(AgentDriveFile.key).where(
AgentDriveFile.tenant_id == binding.tenant_id,
AgentDriveFile.agent_id == binding.agent_id,
AgentDriveFile.key.in_(sorted(wanted_keys)),
)
).all()
)
messages: list[str] = []
for key, (code, display) in wanted_keys.items():
if code == "skill_ref_dangling" and key not in soul_skill_keys:
messages.append(f"{code}: skill '{display}' is not recorded in this Agent Soul version.")
continue
if code == "file_ref_dangling" and key not in soul_file_keys:
messages.append(f"{code}: file '{display}' is not recorded in this Agent Soul version.")
continue
if key in existing_keys:
continue
kind = "skill" if code == "skill_ref_dangling" else "file"
messages.append(f"{code}: {kind} '{display}' has no drive entry for key '{key}'.")
for key in declared_keys:
if key not in existing_keys:
messages.append(f"drive_ref_dangling: Agent Soul drive ref '{key}' has no backing drive entry.")
if messages:
raise WorkflowAgentNodeValidationError(
f"Workflow Agent node {binding.node_id} has invalid Agent Soul drive refs: {'; '.join(messages)}"
)
@staticmethod
def _soul_file_ref_exists(
*,
session: Session,
tenant_id: str,
agent_soul: AgentSoulConfig,
key: str,
) -> bool:
file_ref = AgentSoulFilesService.file_ref_for_key(agent_soul=agent_soul, key=key)
if file_ref is None:
return False
file_id = file_ref.file_id or file_ref.upload_file_id
if not file_id:
return False
raw_kind = file_ref.transfer_method or ("upload_file" if file_ref.upload_file_id else None)
try:
file_kind = AgentDriveFileKind(str(raw_kind))
except ValueError:
return False
if file_kind == AgentDriveFileKind.TOOL_FILE:
return (
session.scalar(select(ToolFile.id).where(ToolFile.tenant_id == tenant_id, ToolFile.id == file_id))
is not None
)
return (
session.scalar(select(UploadFile.id).where(UploadFile.tenant_id == tenant_id, UploadFile.id == file_id))
is not None
)
@classmethod
def sync_agent_bindings_for_draft(
cls,
@@ -285,11 +231,6 @@ class WorkflowAgentPublishService:
continue
if not isinstance(binding_payload, Mapping):
raise ValueError(f"Workflow Agent node {node_id} has invalid agent_binding.")
if (
binding_payload.get("binding_type") == WorkflowAgentBindingType.INLINE_AGENT.value
and (not binding_payload.get("agent_id") or not binding_payload.get("current_snapshot_id"))
):
continue
cls._sync_agent_binding_for_node(
session=session,
draft_workflow=draft_workflow,
+3 -98
View File
@@ -825,24 +825,6 @@ class AgentDriveService:
except ValueError:
return None
@classmethod
def resolve_download_url_for_ref(
cls,
*,
tenant_id: str,
file_kind: AgentDriveFileKind,
file_id: str,
for_external: bool = False,
as_attachment: bool = False,
) -> str | None:
return cls._resolve_download_url(
tenant_id=tenant_id,
file_kind=file_kind,
file_id=file_id,
for_external=for_external,
as_attachment=as_attachment,
)
# ── console drive inspector (ENG-624) ────────────────────────────────────
# SKILL.md is the primary preview use case; 64 KiB covers it with headroom
@@ -862,30 +844,15 @@ class AgentDriveService:
return row
def _storage_key_for_row(self, session: Session, *, tenant_id: str, row: AgentDriveFile) -> str:
return self._storage_key_for_ref(
session,
tenant_id=tenant_id,
file_kind=row.file_kind,
file_id=row.file_id,
)
def _storage_key_for_ref(
self,
session: Session,
*,
tenant_id: str,
file_kind: AgentDriveFileKind,
file_id: str,
) -> str:
if file_kind == AgentDriveFileKind.TOOL_FILE:
if row.file_kind == AgentDriveFileKind.TOOL_FILE:
tool_file = session.scalar(
select(ToolFile).where(ToolFile.id == file_id, ToolFile.tenant_id == tenant_id)
select(ToolFile).where(ToolFile.id == row.file_id, ToolFile.tenant_id == tenant_id)
)
if tool_file is None:
raise AgentDriveError("drive_key_not_found", "drive value record is missing", status_code=404)
return tool_file.file_key
upload_file = session.scalar(
select(UploadFile).where(UploadFile.id == file_id, UploadFile.tenant_id == tenant_id)
select(UploadFile).where(UploadFile.id == row.file_id, UploadFile.tenant_id == tenant_id)
)
if upload_file is None:
raise AgentDriveError("drive_key_not_found", "drive value record is missing", status_code=404)
@@ -922,47 +889,6 @@ class AgentDriveService:
return {"key": row.key, "size": size, "truncated": truncated, "binary": True, "text": None}
return {"key": row.key, "size": size, "truncated": truncated, "binary": False, "text": text}
def preview_file_ref(
self,
*,
tenant_id: str,
agent_id: str,
key: str,
file_kind: AgentDriveFileKind,
file_id: str,
size: int | None = None,
) -> dict[str, Any]:
"""Preview a concrete versioned file ref recorded in Agent Soul."""
with session_factory.create_session() as session:
self._assert_agent_belongs_to_tenant(session, tenant_id=tenant_id, agent_id=agent_id)
storage_key = self._storage_key_for_ref(
session,
tenant_id=tenant_id,
file_kind=file_kind,
file_id=file_id,
)
data = bytearray()
for chunk in storage.load_stream(storage_key):
data.extend(chunk)
if len(data) > self.PREVIEW_MAX_BYTES:
break
truncated = len(data) > self.PREVIEW_MAX_BYTES
sample = bytes(data[: self.PREVIEW_MAX_BYTES])
if b"\x00" in sample:
return {"key": key, "size": size, "truncated": truncated, "binary": True, "text": None}
try:
text = sample.decode("utf-8")
except UnicodeDecodeError:
if truncated:
try:
text = sample[:-3].decode("utf-8", errors="strict")
except UnicodeDecodeError:
return {"key": key, "size": size, "truncated": truncated, "binary": True, "text": None}
else:
return {"key": key, "size": size, "truncated": truncated, "binary": True, "text": None}
return {"key": key, "size": size, "truncated": truncated, "binary": False, "text": text}
def download_url(self, *, tenant_id: str, agent_id: str, key: str) -> str:
"""External signed URL for a browser download of one drive value."""
with session_factory.create_session() as session:
@@ -979,27 +905,6 @@ class AgentDriveService:
raise AgentDriveError("drive_key_not_found", "drive value cannot be resolved", status_code=404)
return url
def download_url_for_ref(
self,
*,
tenant_id: str,
agent_id: str,
file_kind: AgentDriveFileKind,
file_id: str,
) -> str:
with session_factory.create_session() as session:
self._assert_agent_belongs_to_tenant(session, tenant_id=tenant_id, agent_id=agent_id)
url = self._resolve_download_url(
tenant_id=tenant_id,
file_kind=file_kind,
file_id=file_id,
for_external=True,
as_attachment=True,
)
if url is None:
raise AgentDriveError("drive_key_not_found", "drive value cannot be resolved", status_code=404)
return url
__all__ = [
"AgentDriveError",
+27 -10
View File
@@ -309,7 +309,8 @@ _LEGACY_WORKSPACE_OWNER_KEYS: list[str] = [
"customization.manage",
"plugin.install",
"plugin.plugin_preferences",
"plugin.manage",
"plugin.model_config",
"plugin.delete",
"plugin.debug",
"credential.use",
"credential.create",
@@ -330,8 +331,6 @@ _LEGACY_WORKSPACE_OWNER_KEYS: list[str] = [
"snippets.management",
"tool.manage",
"mcp.manage",
"snippets.create_and_modify",
"snippets.management",
]
_LEGACY_WORKSPACE_ADMIN_KEYS: list[str] = [
@@ -342,7 +341,8 @@ _LEGACY_WORKSPACE_ADMIN_KEYS: list[str] = [
"customization.manage",
"plugin.install",
"plugin.plugin_preferences",
"plugin.manage",
"plugin.model_config",
"plugin.delete",
"plugin.debug",
"credential.use",
"credential.create",
@@ -361,8 +361,6 @@ _LEGACY_WORKSPACE_ADMIN_KEYS: list[str] = [
"snippets.management",
"tool.manage",
"mcp.manage",
"snippets.create_and_modify",
"snippets.management",
]
_LEGACY_WORKSPACE_EDITOR_KEYS: list[str] = [
@@ -378,7 +376,9 @@ _LEGACY_WORKSPACE_EDITOR_KEYS: list[str] = [
"dataset.external.connect",
"snippets.create_and_modify",
"tool.manage",
"snippets.create_and_modify",
"billing.view",
"billing.subscription.manage",
"billing.manage",
]
_LEGACY_WORKSPACE_NORMAL_KEYS: list[str] = [
@@ -386,6 +386,9 @@ _LEGACY_WORKSPACE_NORMAL_KEYS: list[str] = [
"plugin.install",
"credential.use",
"app_library.access",
"billing.view",
"billing.subscription.manage",
"billing.manage",
]
_LEGACY_WORKSPACE_DATASET_OPERATOR_KEYS: list[str] = [
@@ -404,6 +407,8 @@ _LEGACY_APP_OWNER_KEYS: list[str] = [
"app.acl.release_and_version",
"app.acl.monitor",
"app.acl.access_config",
"app.acl.tracing_config",
"app.acl.log_and_annotation",
]
_LEGACY_APP_ADMIN_KEYS: list[str] = [
@@ -416,6 +421,9 @@ _LEGACY_APP_ADMIN_KEYS: list[str] = [
"app.acl.release_and_version",
"app.acl.monitor",
"app.acl.access_config",
"app.acl.access_config",
"app.acl.tracing_config",
"app.acl.log_and_annotation",
]
_LEGACY_APP_EDITOR_KEYS: list[str] = [
@@ -431,9 +439,6 @@ _LEGACY_APP_EDITOR_KEYS: list[str] = [
]
_LEGACY_APP_NORMAL_KEYS: list[str] = [
"app.acl.preview",
"app.acl.view_layout",
"app.acl.test_and_run",
"app.acl.monitor",
]
@@ -834,6 +839,7 @@ class RBACService:
options: ListOption | None = None,
) -> Paginated[RBACRole]:
params = (options or ListOption()).to_params({"include_owner": include_owner})
params["dataset_operator_enabled"] = dify_config.DATASET_OPERATOR_ENABLED
data = _inner_call(
"GET",
f"{_INNER_PREFIX}/roles",
@@ -1678,6 +1684,17 @@ class RBACService:
)
return MemberRolesResponse.model_validate(data or {})
@staticmethod
def delete_rbac_bindings(tenant_id: str, account_id: str):
data = _inner_call(
"DELETE",
f"{_INNER_PREFIX}/members/rbac-bindings",
tenant_id=tenant_id,
account_id=account_id,
params={"account_id": account_id},
)
return data
class CheckAccess:
"""Call the ``/inner/api/rbac/check-access`` endpoint."""
@@ -173,6 +173,14 @@ class InnerKnowledgeRetrieveRequest(BaseModel):
class InnerKnowledgeRetrieveUsage(ResponseModel):
"""Serialized LLM usage payload returned by dataset retrieval."""
model_config = ConfigDict(
from_attributes=True,
extra="forbid",
populate_by_name=True,
serialize_by_alias=True,
protected_namespaces=(),
)
prompt_tokens: int
completion_tokens: int
total_tokens: int
+217 -20
View File
@@ -19,11 +19,16 @@ from core.app.entities.app_invoke_entities import (
InvokeFrom,
WorkflowAppGenerateEntity,
)
from core.app.entities.task_entities import WorkflowFinishStreamResponse, WorkflowStartStreamResponse
from core.app.layers.pause_state_persist_layer import PauseStateLayerConfig, WorkflowResumptionContext
from core.repositories import DifyCoreRepositoryFactory
from extensions.ext_database import db
from graphon.entities import WorkflowStartReason
from graphon.enums import WorkflowExecutionStatus
from graphon.runtime import GraphRuntimeState
from libs.datetime_utils import naive_utc_now
from libs.flask_utils import set_login_user
from libs.helper import to_timestamp
from models.account import Account
from models.enums import CreatorUserRole, WorkflowRunTriggeredFrom
from models.model import App, AppMode, Conversation, EndUser, Message
@@ -173,14 +178,24 @@ class _AppRunner:
)
except Exception as exc:
if exec_params.streaming:
_publish_error_event(exc, exec_params.workflow_run_id, exec_params.app_mode)
_publish_failed_workflow_terminal_events(
exc=exc,
exec_params=exec_params,
)
raise
if not exec_params.streaming:
return response
assert isinstance(response, Generator)
_publish_streaming_response(response, exec_params.workflow_run_id, exec_params.app_mode)
_publish_streaming_response(
response,
exec_params.workflow_run_id,
exec_params.app_mode,
exec_params.workflow_id,
exec_params.args.get("inputs", {}),
WorkflowStartReason.INITIAL,
)
def _run_app(
self,
@@ -246,29 +261,197 @@ def _resolve_user_for_run(session: Session, workflow_run: WorkflowRun) -> Accoun
return session.get(EndUser, workflow_run.created_by)
def _publish_error_event(exc: Exception, workflow_run_id: str, app_mode: AppMode) -> None:
topic = MessageBasedAppGenerator.get_response_topic(app_mode, workflow_run_id)
payload = json.dumps({"event": "error", "message": str(exc), "status": 500})
topic.publish(payload.encode())
def _publish_failed_workflow_terminal_events(exc: Exception, exec_params: AppExecutionParams) -> None:
"""Publish synthetic workflow lifecycle events for pre-runtime failures.
Early failures can happen before the app generator creates a task entity or
emits any workflow queue events. In that window SSE consumers still need a
normal terminal event to close their state machines, so we synthesize a
minimal `workflow_started -> workflow_finished(failed)` sequence here.
`workflow_run_id` is reused as a synthetic `task_id` because no application
task id exists yet on this failure path.
"""
timestamp = to_timestamp(naive_utc_now())
assert timestamp is not None
topic = MessageBasedAppGenerator.get_response_topic(exec_params.app_mode, exec_params.workflow_run_id)
started_payload = WorkflowStartStreamResponse(
task_id=exec_params.workflow_run_id,
workflow_run_id=exec_params.workflow_run_id,
data=WorkflowStartStreamResponse.Data(
id=exec_params.workflow_run_id,
workflow_id=exec_params.workflow_id,
inputs=exec_params.args.get("inputs", {}),
created_at=timestamp,
reason=WorkflowStartReason.INITIAL,
),
)
topic.publish(json.dumps(started_payload.model_dump(mode="json"), ensure_ascii=False).encode())
finished_payload = WorkflowFinishStreamResponse(
task_id=exec_params.workflow_run_id,
workflow_run_id=exec_params.workflow_run_id,
data=WorkflowFinishStreamResponse.Data(
id=exec_params.workflow_run_id,
workflow_id=exec_params.workflow_id,
status=WorkflowExecutionStatus.FAILED,
outputs=None,
error=str(exc),
elapsed_time=0.0,
total_tokens=0,
total_steps=0,
created_by={},
created_at=timestamp,
finished_at=timestamp,
exceptions_count=1,
files=[],
),
)
topic.publish(json.dumps(finished_payload.model_dump(mode="json"), ensure_ascii=False).encode())
def _get_event_name(event: str | Mapping[str, Any] | BaseModel) -> str | None:
if isinstance(event, BaseModel):
# Temporary compatibility for legacy BaseModel stream events; remove after confirming generators always emit
# str / Mapping responses.
event_name = getattr(event, "event", None)
elif isinstance(event, Mapping):
event_name = event.get("event")
else:
return None
if event_name is None:
return None
return str(event_name)
def _get_task_id(event: str | Mapping[str, Any] | BaseModel) -> str | None:
if isinstance(event, BaseModel):
# Temporary compatibility for legacy BaseModel stream events; remove after confirming generators always emit
# str / Mapping responses.
task_id = getattr(event, "task_id", None)
elif isinstance(event, Mapping):
task_id = event.get("task_id")
else:
return None
return task_id if isinstance(task_id, str) and task_id else None
def _publish_streaming_response(
response_stream: Generator[str | Mapping[str, Any] | BaseModel, None, None],
workflow_run_id: str,
workflow_run_id: str | uuid.UUID,
app_mode: AppMode,
workflow_id: str,
inputs: Mapping[str, Any],
started_reason: WorkflowStartReason,
) -> None:
topic = MessageBasedAppGenerator.get_response_topic(app_mode, workflow_run_id)
for event in response_stream:
try:
if isinstance(event, BaseModel):
payload = json.dumps(event.model_dump(mode="json"), ensure_ascii=False)
else:
payload = json.dumps(event, ensure_ascii=False, default=str)
except (TypeError, ValueError):
logger.exception("error while encoding event")
continue
"""Publish workflow stream events and close broken streams with a failed terminal event.
topic.publish(payload.encode())
`_AppRunner.run()` only handles failures before the generator is returned.
Once we start iterating the runtime stream, this helper becomes the last
place that can guarantee SSE consumers eventually see a terminal workflow
lifecycle event.
"""
normalized_workflow_run_id = str(workflow_run_id)
def _publish_failed_terminal_event(error_message: str, task_id: str, publish_started: bool) -> None:
timestamp = to_timestamp(naive_utc_now())
assert timestamp is not None
if publish_started:
started_payload = WorkflowStartStreamResponse(
task_id=task_id,
workflow_run_id=normalized_workflow_run_id,
data=WorkflowStartStreamResponse.Data(
id=normalized_workflow_run_id,
workflow_id=workflow_id,
inputs=inputs,
created_at=timestamp,
reason=started_reason,
),
)
topic.publish(
json.dumps(
started_payload.model_dump(mode="json", fallback=str),
ensure_ascii=False,
).encode()
)
finished_payload = WorkflowFinishStreamResponse(
task_id=task_id,
workflow_run_id=normalized_workflow_run_id,
data=WorkflowFinishStreamResponse.Data(
id=normalized_workflow_run_id,
workflow_id=workflow_id,
status=WorkflowExecutionStatus.FAILED,
outputs=None,
error=error_message,
elapsed_time=0.0,
total_tokens=0,
total_steps=0,
created_by={},
created_at=timestamp,
finished_at=timestamp,
exceptions_count=1,
files=[],
),
)
topic.publish(json.dumps(finished_payload.model_dump(mode="json"), ensure_ascii=False).encode())
terminal_events = {"workflow_finished", "workflow_paused"}
unexpected_stream_end_message = "Workflow stream ended without a terminal event"
topic = MessageBasedAppGenerator.get_response_topic(app_mode, normalized_workflow_run_id)
started_published = False
terminal_published = False
last_task_id = normalized_workflow_run_id
try:
for event in response_stream:
event_name = _get_event_name(event)
task_id = _get_task_id(event)
if task_id is not None:
last_task_id = task_id
try:
if isinstance(event, BaseModel):
payload = json.dumps(event.model_dump(mode="json"), ensure_ascii=False)
else:
payload = json.dumps(event, ensure_ascii=False, default=str)
except (TypeError, ValueError):
logger.exception("error while encoding event")
continue
topic.publish(payload.encode())
if event_name == "workflow_started":
started_published = True
elif event_name in terminal_events:
terminal_published = True
except Exception as exc:
if not terminal_published:
logger.exception(
"Workflow stream for run %s failed before terminal event; publishing fallback terminal event",
normalized_workflow_run_id,
)
_publish_failed_terminal_event(
error_message=str(exc) or exc.__class__.__name__,
task_id=last_task_id,
publish_started=not started_published,
)
raise
if not terminal_published:
logger.warning(
"Workflow stream for run %s ended without a terminal event; publishing fallback terminal event",
normalized_workflow_run_id,
)
_publish_failed_terminal_event(
error_message=unexpected_stream_end_message,
task_id=last_task_id,
publish_started=not started_published,
)
@shared_task(queue=WORKFLOW_BASED_APP_EXECUTION_QUEUE)
@@ -454,7 +637,14 @@ def _resume_advanced_chat(
raise
assert isinstance(response, Generator)
_publish_streaming_response(response, workflow_run_id, AppMode.ADVANCED_CHAT)
_publish_streaming_response(
response,
workflow_run_id,
AppMode.ADVANCED_CHAT,
workflow.id,
generate_entity.inputs,
WorkflowStartReason.RESUMPTION,
)
def _resume_workflow(
@@ -509,7 +699,14 @@ def _resume_workflow(
raise
assert isinstance(response, Generator)
_publish_streaming_response(response, workflow_run_id, AppMode.WORKFLOW)
_publish_streaming_response(
response,
workflow_run_id,
AppMode.WORKFLOW,
workflow.id,
generate_entity.inputs,
WorkflowStartReason.RESUMPTION,
)
try:
workflow_run_repo.delete_workflow_pause(pause_entity)
@@ -72,6 +72,7 @@ def mint_token(flask_app: Flask):
prefix: str,
subject_email: str,
subject_issuer: str | None,
expires_at: datetime | None = None,
) -> OAuthAccessToken:
with flask_app.app_context():
row = OAuthAccessToken(
@@ -82,7 +83,7 @@ def mint_token(flask_app: Flask):
subject_issuer=subject_issuer,
client_id="difyctl",
device_label="test-device",
expires_at=datetime.now(UTC) + timedelta(hours=1),
expires_at=expires_at or (datetime.now(UTC) + timedelta(hours=1)),
)
db.session.add(row)
db.session.commit()
@@ -111,6 +112,21 @@ def account_token(workspace_account, mint_token) -> str:
return token
@pytest.fixture
def expired_account_token(workspace_account, mint_token) -> str:
account, _, _ = workspace_account
token = "dfoa_" + uuid.uuid4().hex
mint_token(
token,
account_id=account.id,
prefix="dfoa_",
subject_email=account.email,
subject_issuer="dify:account",
expires_at=datetime.now(UTC) - timedelta(minutes=1),
)
return token
@pytest.fixture(autouse=True)
def _flush_auth_redis(flask_app: Flask) -> Generator[None, None, None]:
def _flush():
@@ -6,6 +6,7 @@ acceptance/rejection on app-scoped routes.
from __future__ import annotations
import uuid
from collections.abc import Generator
import pytest
@@ -16,6 +17,50 @@ from extensions.ext_database import db
from models import App, Tenant
def test_expired_token_returns_401_token_expired(
test_client: FlaskClient,
expired_account_token: str,
) -> None:
"""An expired bearer is distinguishable from an unknown one: 401 with the
domain code ``token_expired`` (+ actionable hint), not a generic 401 or 500."""
res = test_client.get(
"/openapi/v1/account",
headers={"Authorization": f"Bearer {expired_account_token}"},
)
assert res.status_code == 401
assert res.json["code"] == "token_expired"
assert res.json["hint"]
def test_expired_token_replay_stays_token_expired(
test_client: FlaskClient,
expired_account_token: str,
) -> None:
"""The distinct ``expired`` negative-cache marker keeps the second hit (served
from cache, inside NEGATIVE_TTL) reporting ``token_expired`` rather than
collapsing into a generic unknown-token 401."""
headers = {"Authorization": f"Bearer {expired_account_token}"}
first = test_client.get("/openapi/v1/account", headers=headers)
second = test_client.get("/openapi/v1/account", headers=headers)
assert first.json["code"] == "token_expired"
assert second.status_code == 401
assert second.json["code"] == "token_expired"
def test_unknown_token_returns_401_unauthorized_not_500(
test_client: FlaskClient,
workspace_account,
) -> None:
"""An unknown bearer is a clean 401 ``unauthorized`` — not the latent 500 the
pipeline used to leak for unmapped InvalidBearerError."""
res = test_client.get(
"/openapi/v1/account",
headers={"Authorization": "Bearer dfoa_" + uuid.uuid4().hex},
)
assert res.status_code == 401
assert res.json["code"] == "unauthorized"
def test_info_accepts_account_bearer_with_apps_read_scope(
test_client: FlaskClient,
app_in_workspace: App,
@@ -27,7 +27,6 @@ extend-select = ["ANN401", "ARG", "TID251"]
"controllers/web/test_wraps.py" = ["ARG"]
"core/app/layers/test_pause_state_persist_layer.py" = ["ARG"]
"core/rag/retrieval/test_dataset_retrieval_integration.py" = ["ARG"]
"models/test_account.py" = ["ARG"]
"models/test_conversation_message_inputs.py" = ["ARG"]
"models/test_conversation_status_count.py" = ["ARG"]
"repositories/test_sqlalchemy_api_workflow_run_repository.py" = ["ARG"]
@@ -3,6 +3,7 @@ Integration tests for Account and Tenant model methods that interact with the da
Migrated from unit_tests/models/test_account_models.py, replacing
@patch("models.account.db") mock patches with real PostgreSQL operations.
Also absorbs unit_tests/models/test_account.py role helper coverage.
Covers:
- Account.current_tenant setter (sets _current_tenant and role from TenantAccountJoin)
@@ -12,6 +13,7 @@ Covers:
"""
from collections.abc import Generator
from typing import cast
from uuid import uuid4
import pytest
@@ -20,8 +22,10 @@ from sqlalchemy.orm import Session
from models.account import Account, AccountIntegrate, Tenant, TenantAccountJoin, TenantAccountRole
TrackedRow = Account | AccountIntegrate | Tenant | TenantAccountJoin
def _cleanup_tracked_rows(db_session: Session, tracked: list) -> None:
def _cleanup_tracked_rows(db_session: Session, tracked: list[TrackedRow]) -> None:
"""Delete rows tracked during the test so committed state does not leak into the DB.
Rolls back any pending (uncommitted) session state first, then issues DELETE
@@ -52,7 +56,7 @@ def _build_account(email_prefix: str = "account") -> Account:
class _DBTrackingTestBase:
"""Base class providing a tracker list and shared row factories for account/tenant tests."""
_tracked: list
_tracked: list[TrackedRow]
@pytest.fixture(autouse=True)
def _setup_cleanup(self, db_session_with_containers: Session) -> Generator[None, None, None]:
@@ -84,6 +88,22 @@ class _DBTrackingTestBase:
return join
class TestTenantAccountRole:
"""Tests for TenantAccountRole helper methods."""
def test_account_is_privileged_role(self) -> None:
assert TenantAccountRole.ADMIN == "admin"
assert TenantAccountRole.OWNER == "owner"
assert TenantAccountRole.EDITOR == "editor"
assert TenantAccountRole.NORMAL == "normal"
assert TenantAccountRole.is_privileged_role(TenantAccountRole.ADMIN)
assert TenantAccountRole.is_privileged_role(TenantAccountRole.OWNER)
assert not TenantAccountRole.is_privileged_role(TenantAccountRole.NORMAL)
assert not TenantAccountRole.is_privileged_role(TenantAccountRole.EDITOR)
assert not TenantAccountRole.is_privileged_role(cast(TenantAccountRole, ""))
class TestAccountCurrentTenantSetter(_DBTrackingTestBase):
"""Integration tests for Account.current_tenant property setter."""
@@ -176,7 +196,7 @@ class TestAccountGetByOpenId(_DBTrackingTestBase):
assert result is not None
assert result.id == account.id
def test_get_by_openid_returns_none_when_no_integrate_exists(self, db_session_with_containers: Session) -> None:
def test_get_by_openid_returns_none_when_no_integrate_exists(self) -> None:
"""get_by_openid returns None when no AccountIntegrate row matches."""
result = Account.get_by_openid("github", f"github_{uuid4()}")
@@ -50,7 +50,6 @@ project-excludes = [
"libs/broadcast_channel/redis/test_streams_channel.py",
"libs/test_auto_renew_redis_lock_integration.py",
"libs/test_rate_limiter_integration.py",
"models/test_account.py",
"models/test_conversation_message_inputs.py",
"models/test_types_enum_text.py",
"repositories/test_sqlalchemy_api_workflow_node_execution_repository.py",
@@ -162,15 +162,8 @@ def test_request_builder_adds_knowledge_layer_when_configured():
run_input = _run_input()
run_input.knowledge = DifyKnowledgeBaseLayerConfig.model_validate(
{
"sets": [
{
"id": "support",
"name": "Support KB",
"datasets": [{"id": "dataset-1"}],
"query": {"mode": "generated_query"},
"retrieval": {"mode": "multiple", "top_k": 4},
}
],
"dataset_ids": ["dataset-1"],
"retrieval": {"mode": "multiple", "top_k": 4},
}
)
@@ -181,7 +174,7 @@ def test_request_builder_adds_knowledge_layer_when_configured():
assert layers[DIFY_KNOWLEDGE_BASE_LAYER_ID].type == DIFY_KNOWLEDGE_BASE_LAYER_TYPE_ID
assert layers[DIFY_KNOWLEDGE_BASE_LAYER_ID].deps == {"execution_context": DIFY_EXECUTION_CONTEXT_LAYER_ID}
knowledge_config = cast(DifyKnowledgeBaseLayerConfig, layers[DIFY_KNOWLEDGE_BASE_LAYER_ID].config)
assert knowledge_config.sets[0].dataset_ids == ["dataset-1"]
assert knowledge_config.dataset_ids == ["dataset-1"]
def test_request_builder_can_delete_on_exit_for_cleanup_paths():
@@ -393,15 +386,8 @@ def test_agent_app_request_builder_adds_knowledge_layer_when_configured():
run_input = _agent_app_input()
run_input.knowledge = DifyKnowledgeBaseLayerConfig.model_validate(
{
"sets": [
{
"id": "support",
"name": "Support KB",
"datasets": [{"id": "dataset-1"}, {"id": "dataset-2"}],
"query": {"mode": "generated_query"},
"retrieval": {"mode": "multiple", "top_k": 2},
}
],
"dataset_ids": ["dataset-1", "dataset-2"],
"retrieval": {"mode": "multiple", "top_k": 2},
}
)
@@ -412,7 +398,7 @@ def test_agent_app_request_builder_adds_knowledge_layer_when_configured():
assert layers[DIFY_KNOWLEDGE_BASE_LAYER_ID].type == DIFY_KNOWLEDGE_BASE_LAYER_TYPE_ID
assert layers[DIFY_KNOWLEDGE_BASE_LAYER_ID].deps == {"execution_context": DIFY_EXECUTION_CONTEXT_LAYER_ID}
knowledge_config = cast(DifyKnowledgeBaseLayerConfig, layers[DIFY_KNOWLEDGE_BASE_LAYER_ID].config)
assert knowledge_config.sets[0].dataset_ids == ["dataset-1", "dataset-2"]
assert knowledge_config.dataset_ids == ["dataset-1", "dataset-2"]
# ── ENG-635 / ENG-638: ask_human layer injection + deferred_tool_results ─────
@@ -149,55 +149,3 @@ def test_generate_specs_is_idempotent(tmp_path):
assert [path.name for path in first_paths] == [path.name for path in second_paths]
for first_path, second_path in zip(first_paths, second_paths):
assert first_path.read_text(encoding="utf-8") == second_path.read_text(encoding="utf-8")
def test_generate_specs_include_agent_v2_knowledge_set_schema_and_query_enums(tmp_path):
module = _load_generate_swagger_specs_module()
written_paths = module.generate_specs(tmp_path)
console_path = next(path for path in written_paths if path.name == "console-openapi.json")
payload = json.loads(console_path.read_text(encoding="utf-8"))
schemas = payload["components"]["schemas"]
assert "AgentKnowledgeSetConfig" in schemas
assert schemas["AgentSoulKnowledgeConfig"]["properties"]["sets"]["items"]["$ref"] == (
"#/components/schemas/AgentKnowledgeSetConfig"
)
assert schemas["AgentKnowledgeQueryMode"]["enum"] == ["generated_query", "user_query"]
def test_checked_in_agent_v2_knowledge_openapi_and_generated_contracts_are_in_sync():
api_dir = Path(__file__).resolve().parents[3]
repo_root = api_dir.parent
markdown = (api_dir / "openapi" / "markdown" / "console-openapi.md").read_text(encoding="utf-8")
agent_types = (
repo_root / "packages" / "contracts" / "generated" / "api" / "console" / "agent" / "types.gen.ts"
).read_text(encoding="utf-8")
apps_types = (
repo_root / "packages" / "contracts" / "generated" / "api" / "console" / "apps" / "types.gen.ts"
).read_text(encoding="utf-8")
agent_zod = (
repo_root / "packages" / "contracts" / "generated" / "api" / "console" / "agent" / "zod.gen.ts"
).read_text(encoding="utf-8")
apps_zod = (
repo_root / "packages" / "contracts" / "generated" / "api" / "console" / "apps" / "zod.gen.ts"
).read_text(encoding="utf-8")
assert "#### AgentKnowledgeSetConfig" in markdown
assert "#### AgentSoulKnowledgeConfig" in markdown
assert "#### AgentKnowledgeQueryMode" in markdown
for content in (agent_types, apps_types):
assert "export type AgentKnowledgeSetConfig = {" in content
assert "export type AgentSoulKnowledgeConfig = {" in content
assert "AgentKnowledgeQueryMode" in content
assert "generated_query" in content
assert "user_query" in content
for content in (agent_zod, apps_zod):
assert "export const zAgentKnowledgeSetConfig = z.object({" in content
assert "export const zAgentSoulKnowledgeConfig = z.object({" in content
assert "zAgentKnowledgeQueryMode = z.enum([" in content
assert "generated_query" in content
assert "user_query" in content
@@ -77,6 +77,25 @@ class AnnotationApi(Resource):
assert "prefer dump_response" in checks[0].reason
def test_constructor_variable_model_dump_is_valid(tmp_path: Path):
checks = _checks_for_source(
tmp_path,
"""
@ns.route("/annotations")
class AnnotationApi(Resource):
@ns.response(201, "Created", ns.models[AnnotationResponse.__name__])
def post(self):
response = AnnotationResponse(id="new", name=name)
return response.model_dump(mode="json"), 201
""",
)
assert len(checks) == 1
assert checks[0].classification == "valid"
assert checks[0].actual[0].kind == "model"
assert checks[0].actual[0].model == "AnnotationResponse"
def test_variable_model_dump_with_wrong_documented_schema_is_mismatch(tmp_path: Path):
checks = _checks_for_source(
tmp_path,
@@ -117,6 +136,38 @@ class StreamApi(Resource):
assert {actual.model for actual in checks[0].actual} == {"StreamResponse"}
def test_response_contract_ignore_comment_skips_route_method(tmp_path: Path):
checks = _checks_for_source(
tmp_path,
"""
@ns.route("/binary")
class BinaryApi(Resource):
# response-contract:ignore binary response
@ns.response(200, "Binary file")
def get(self):
return send_file(path)
# response-contract:ignore compact Flask response
@ns.route("/compact")
class CompactApi(Resource):
def get(self):
return make_response({"url": "https://example.com"})
@ns.route("/regular")
class RegularApi(Resource):
@ns.response(200, "OK", ns.models[RegularResponse.__name__])
def get(self):
return dump_response(RegularResponse, {})
""",
)
assert len(checks) == 1
assert checks[0].class_name == "RegularApi"
assert checks[0].classification == "valid"
def test_main_is_report_only_by_default_for_mismatches(tmp_path: Path, monkeypatch: pytest.MonkeyPatch):
module = _load_lint_response_contracts_module()
controller_path = tmp_path / "controllers" / "sample.py"
@@ -1,5 +1,4 @@
import builtins
from types import SimpleNamespace
from unittest.mock import patch
from flask.views import MethodView as FlaskMethodView
@@ -22,7 +21,7 @@ def test_parameters_model_round_trip():
def test_site_icon_url_uses_signed_url_for_image_icon():
site = SimpleNamespace(
site = Site(
title="Example",
chat_color_theme=None,
chat_color_theme_inverted=False,
@@ -46,7 +45,7 @@ def test_site_icon_url_uses_signed_url_for_image_icon():
def test_site_icon_url_is_none_for_non_image_icon():
site = SimpleNamespace(
site = Site(
title="Example",
chat_color_theme=None,
chat_color_theme_inverted=False,
@@ -28,15 +28,11 @@ from controllers.console.agent.roster import (
AgentAppApi,
AgentAppCopyApi,
AgentAppListApi,
AgentBuildDraftApi,
AgentBuildDraftApplyApi,
AgentBuildDraftCheckoutApi,
AgentDebugConversationRefreshApi,
AgentInviteOptionsApi,
AgentLogMessagesApi,
AgentLogsApi,
AgentLogSourcesApi,
AgentPublishApi,
AgentRosterVersionDetailApi,
AgentRosterVersionRestoreApi,
AgentRosterVersionsApi,
@@ -155,10 +151,6 @@ def test_agent_v2_console_routes_are_agent_id_first() -> None:
"/agent/<uuid:agent_id>/composer/candidates",
"/agent/<uuid:agent_id>/features",
"/agent/<uuid:agent_id>/copy",
"/agent/<uuid:agent_id>/publish",
"/agent/<uuid:agent_id>/build-draft/checkout",
"/agent/<uuid:agent_id>/build-draft",
"/agent/<uuid:agent_id>/build-draft/apply",
"/agent/<uuid:agent_id>/referencing-workflows",
"/agent/<uuid:agent_id>/drive/files",
"/agent/<uuid:agent_id>/sandbox/files",
@@ -528,129 +520,6 @@ def test_agent_debug_conversation_refresh_uses_current_user(
}
def test_agent_publish_and_build_draft_routes_call_composer_service(
app: Flask, monkeypatch: pytest.MonkeyPatch, account_id: str
) -> None:
agent_id = "00000000-0000-0000-0000-000000000001"
current_user = SimpleNamespace(id=account_id)
captured: dict[str, object] = {}
def publish_agent_app_draft(**kwargs: object) -> dict[str, object]:
captured["publish"] = kwargs
return {"result": "success", "active_config_snapshot_id": "version-1"}
def checkout_agent_app_build_draft(**kwargs: object) -> dict[str, object]:
captured["checkout"] = kwargs
return {"variant": "agent_app", "draft": {"id": "build-draft-1"}, "agent_soul": {}}
def load_agent_app_build_draft(**kwargs: object) -> dict[str, object]:
captured["load"] = kwargs
return {"variant": "agent_app", "draft": {"id": "build-draft-1"}, "agent_soul": {}}
def save_agent_app_build_draft(**kwargs: object) -> dict[str, object]:
captured["save"] = kwargs
return {"variant": "agent_app", "draft": {"id": "build-draft-1"}, "agent_soul": {}}
def apply_agent_app_build_draft(**kwargs: object) -> dict[str, object]:
captured["apply"] = kwargs
return {"result": "success", "draft": {"id": "draft-1"}}
def discard_agent_app_build_draft(**kwargs: object) -> dict[str, object]:
captured["discard"] = kwargs
return {"result": "success"}
monkeypatch.setattr(
roster_controller.AgentComposerService,
"publish_agent_app_draft",
publish_agent_app_draft,
)
monkeypatch.setattr(
roster_controller.AgentComposerService,
"checkout_agent_app_build_draft",
checkout_agent_app_build_draft,
)
monkeypatch.setattr(
roster_controller.AgentComposerService,
"load_agent_app_build_draft",
load_agent_app_build_draft,
)
monkeypatch.setattr(
roster_controller.AgentComposerService,
"save_agent_app_build_draft",
save_agent_app_build_draft,
)
monkeypatch.setattr(
roster_controller.AgentComposerService,
"apply_agent_app_build_draft",
apply_agent_app_build_draft,
)
monkeypatch.setattr(
roster_controller.AgentComposerService,
"discard_agent_app_build_draft",
discard_agent_app_build_draft,
)
with app.test_request_context(
"/console/api/agent/00000000-0000-0000-0000-000000000001/publish",
json={"version_note": "publish v1"},
):
published = unwrap(AgentPublishApi.post)(AgentPublishApi(), "tenant-1", current_user, agent_id)
assert published["active_config_snapshot_id"] == "version-1"
assert captured["publish"] == {
"tenant_id": "tenant-1",
"agent_id": agent_id,
"account_id": account_id,
"version_note": "publish v1",
}
with app.test_request_context(
"/console/api/agent/00000000-0000-0000-0000-000000000001/build-draft/checkout",
json={"force": True},
):
checked_out = unwrap(AgentBuildDraftCheckoutApi.post)(
AgentBuildDraftCheckoutApi(), "tenant-1", current_user, agent_id
)
assert checked_out["draft"]["id"] == "build-draft-1"
assert captured["checkout"] == {
"tenant_id": "tenant-1",
"agent_id": agent_id,
"account_id": account_id,
"force": True,
}
with app.test_request_context("/console/api/agent/00000000-0000-0000-0000-000000000001/build-draft"):
loaded = unwrap(AgentBuildDraftApi.get)(AgentBuildDraftApi(), "tenant-1", current_user, agent_id)
assert loaded["draft"]["id"] == "build-draft-1"
assert captured["load"] == {"tenant_id": "tenant-1", "agent_id": agent_id, "account_id": account_id}
with app.test_request_context(
"/console/api/agent/00000000-0000-0000-0000-000000000001/build-draft",
json={"variant": "agent_app", "save_strategy": "save_to_current_version", "agent_soul": {}},
):
saved = unwrap(AgentBuildDraftApi.put)(AgentBuildDraftApi(), "tenant-1", current_user, agent_id)
assert saved["draft"]["id"] == "build-draft-1"
assert captured["save"]["tenant_id"] == "tenant-1"
assert captured["save"]["agent_id"] == agent_id
assert captured["save"]["account_id"] == account_id
assert captured["save"]["payload"].variant == ComposerVariant.AGENT_APP
with app.test_request_context(
"/console/api/agent/00000000-0000-0000-0000-000000000001/build-draft/apply",
method="POST",
):
applied = unwrap(AgentBuildDraftApplyApi.post)(AgentBuildDraftApplyApi(), "tenant-1", current_user, agent_id)
assert applied == {"result": "success", "draft": {"id": "draft-1"}}
assert captured["apply"] == {"tenant_id": "tenant-1", "agent_id": agent_id, "account_id": account_id}
with app.test_request_context(
"/console/api/agent/00000000-0000-0000-0000-000000000001/build-draft",
method="DELETE",
):
discarded = unwrap(AgentBuildDraftApi.delete)(AgentBuildDraftApi(), "tenant-1", current_user, agent_id)
assert discarded == {"result": "success"}
assert captured["discard"] == {"tenant_id": "tenant-1", "agent_id": agent_id, "account_id": account_id}
def test_agent_api_access_uses_agent_id_and_returns_service_api_metadata(
monkeypatch: pytest.MonkeyPatch,
) -> None:
@@ -0,0 +1,154 @@
from __future__ import annotations
from contextlib import nullcontext
from types import SimpleNamespace
from unittest.mock import MagicMock, PropertyMock, patch
import pytest
from flask import Flask
from werkzeug.exceptions import Forbidden
from controllers.common import wraps as common_wraps
from controllers.console import console_ns
from controllers.console import wraps as console_wraps
from controllers.console.app import ops_trace as ops_trace_module
from controllers.console.app import wraps as app_wraps
from libs import login as login_lib
from models.account import Account, AccountStatus, TenantAccountRole
def _make_account(role: TenantAccountRole) -> Account:
account = Account(name="tester", email="tester@example.com")
account.id = "account-123" # type: ignore[assignment]
account.status = AccountStatus.ACTIVE
account.role = role
account._current_tenant = SimpleNamespace(id="tenant-123") # type: ignore[assignment]
account._get_current_object = lambda: account # type: ignore[attr-defined]
return account
def _make_app() -> SimpleNamespace:
return SimpleNamespace(id="app-123", tenant_id="tenant-123", status="normal", mode="chat")
def _patch_console_guards(
monkeypatch: pytest.MonkeyPatch,
account: Account,
app_model: SimpleNamespace,
*,
rbac_enabled: bool = False,
) -> None:
monkeypatch.setattr(login_lib.dify_config, "LOGIN_DISABLED", True)
monkeypatch.setattr(login_lib.dify_config, "RBAC_ENABLED", rbac_enabled)
monkeypatch.setattr(console_wraps.dify_config, "EDITION", "CLOUD")
monkeypatch.setattr(login_lib, "current_user", account)
monkeypatch.setattr(login_lib, "current_account_with_tenant", lambda: (account, account.current_tenant_id))
monkeypatch.setattr(console_wraps, "current_account_with_tenant", lambda: (account, account.current_tenant_id))
monkeypatch.setattr(common_wraps, "current_account_with_tenant", lambda: (account, account.current_tenant_id))
monkeypatch.setattr(app_wraps, "_load_app_model_from_scoped_session", lambda _app_id: app_model)
def _patch_payload(payload: dict[str, object] | None):
if payload is None:
return nullcontext()
return patch.object(type(console_ns), "payload", new_callable=PropertyMock, return_value=payload)
@pytest.mark.parametrize(
("method_name", "path", "payload", "service_method_name", "service_result"),
[
(
"post",
"/console/api/apps/app-123/trace-config",
{"tracing_provider": "mlflow", "tracing_config": {"endpoint": "https://trace.example.com"}},
"create_tracing_app_config",
{"id": "trace-config-1"},
),
(
"patch",
"/console/api/apps/app-123/trace-config",
{"tracing_provider": "mlflow", "tracing_config": {"endpoint": "https://trace.example.com"}},
"update_tracing_app_config",
True,
),
(
"delete",
"/console/api/apps/app-123/trace-config?tracing_provider=mlflow",
None,
"delete_tracing_app_config",
True,
),
],
)
def test_trace_config_mutations_require_edit_permission(
app: Flask,
monkeypatch: pytest.MonkeyPatch,
method_name: str,
path: str,
payload: dict[str, object] | None,
service_method_name: str,
service_result: object,
) -> None:
app.config.setdefault("RESTX_MASK_HEADER", "X-Fields")
account = _make_account(TenantAccountRole.NORMAL)
_patch_console_guards(monkeypatch, account, _make_app())
service_mock = MagicMock(return_value=service_result)
monkeypatch.setattr(ops_trace_module.OpsService, service_method_name, service_mock)
with app.test_request_context(path, method=method_name.upper(), json=payload):
with _patch_payload(payload):
with pytest.raises(Forbidden):
getattr(ops_trace_module.TraceAppConfigApi(), method_name)(app_id="app-123")
service_mock.assert_not_called()
@pytest.mark.parametrize(
("method_name", "path", "payload", "service_method_name", "service_result"),
[
(
"post",
"/console/api/apps/app-123/trace-config",
{"tracing_provider": "mlflow", "tracing_config": {"endpoint": "https://trace.example.com"}},
"create_tracing_app_config",
{"id": "trace-config-1"},
),
(
"patch",
"/console/api/apps/app-123/trace-config",
{"tracing_provider": "mlflow", "tracing_config": {"endpoint": "https://trace.example.com"}},
"update_tracing_app_config",
True,
),
(
"delete",
"/console/api/apps/app-123/trace-config?tracing_provider=mlflow",
None,
"delete_tracing_app_config",
True,
),
],
)
def test_trace_config_mutations_require_rbac_permission(
app: Flask,
monkeypatch: pytest.MonkeyPatch,
method_name: str,
path: str,
payload: dict[str, object] | None,
service_method_name: str,
service_result: object,
) -> None:
app.config.setdefault("RESTX_MASK_HEADER", "X-Fields")
account = _make_account(TenantAccountRole.NORMAL)
_patch_console_guards(monkeypatch, account, _make_app(), rbac_enabled=True)
monkeypatch.setattr(common_wraps.db, "session", SimpleNamespace(scalar=lambda _stmt: "other-account"))
monkeypatch.setattr(common_wraps.RBACService.CheckAccess, "check", MagicMock(return_value=False))
service_mock = MagicMock(return_value=service_result)
monkeypatch.setattr(ops_trace_module.OpsService, service_method_name, service_mock)
with app.test_request_context(path, method=method_name.upper(), json=payload):
with _patch_payload(payload):
with pytest.raises(Forbidden):
getattr(ops_trace_module.TraceAppConfigApi(), method_name)(app_id="app-123")
service_mock.assert_not_called()
@@ -2,21 +2,14 @@
from __future__ import annotations
from inspect import unwrap
from types import SimpleNamespace
import pytest
from flask import Flask
from controllers.console.app import workflow as workflow_module
def _unwrap(func):
bound_self = getattr(func, "__self__", None)
while hasattr(func, "__wrapped__"):
func = func.__wrapped__
if bound_self is not None:
return func.__get__(bound_self, bound_self.__class__)
return func
from controllers.console.app.workflow import ConvertToWorkflowApi
class TestConvertToWorkflowApi:
@@ -25,9 +18,9 @@ class TestConvertToWorkflowApi:
return workflow_module.ConvertToWorkflowApi()
def test_convert_to_workflow_attaches_permission_keys_when_rbac_enabled(
self, api, app: Flask, monkeypatch: pytest.MonkeyPatch
self, api: ConvertToWorkflowApi, app: Flask, monkeypatch: pytest.MonkeyPatch
) -> None:
method = _unwrap(api.post)
method = unwrap(api.post)
monkeypatch.setattr(
workflow_module,
@@ -46,6 +39,7 @@ class TestConvertToWorkflowApi:
json={},
):
response = method(
api,
current_tenant_id="tenant-1",
current_user=SimpleNamespace(id="u1"),
app_model=SimpleNamespace(id="app-1"),
@@ -9,6 +9,7 @@ This module tests the core authentication endpoints including:
"""
import base64
import logging
from unittest.mock import ANY, MagicMock, Mock, patch
import pytest
@@ -191,7 +192,9 @@ class TestLoginApi:
@patch("controllers.console.auth.login.dify_config.BILLING_ENABLED", False)
@patch("controllers.console.auth.login.AccountService.is_login_error_rate_limit")
@patch("controllers.console.auth.login.RegisterService.get_invitation_with_case_fallback")
def test_login_fails_when_rate_limited(self, mock_get_invitation, mock_is_rate_limit, mock_db, app: Flask):
def test_login_fails_when_rate_limited(
self, mock_get_invitation, mock_is_rate_limit, mock_db, app: Flask, caplog: pytest.LogCaptureFixture
):
"""
Test login rejection when rate limit is exceeded.
@@ -204,22 +207,26 @@ class TestLoginApi:
mock_get_invitation.return_value = None
# Act & Assert
with patch("controllers.console.auth.login.logger.warning") as mock_log_warning:
with app.test_request_context(
"/login", method="POST", json={"email": "test@example.com", "password": encode_password("password")}
):
login_api = LoginApi()
with pytest.raises(EmailPasswordLoginLimitError):
login_api.post()
with app.test_request_context(
"/login", method="POST", json={"email": "test@example.com", "password": encode_password("password")}
):
login_api = LoginApi()
with pytest.raises(EmailPasswordLoginLimitError):
login_api.post()
assert mock_log_warning.call_count == 1
assert mock_log_warning.call_args.args[1] == "test@example.com"
assert mock_log_warning.call_args.args[2] == LoginFailureReason.LOGIN_RATE_LIMITED
warn_records = [
r for r in caplog.records if r.name == "controllers.console.auth.login" and r.levelno == logging.WARNING
]
assert len(warn_records) == 1
assert warn_records[0].args[0] == "test@example.com"
assert warn_records[0].args[1] == LoginFailureReason.LOGIN_RATE_LIMITED
@patch("controllers.console.wraps.db")
@patch("controllers.console.auth.login.dify_config.BILLING_ENABLED", True)
@patch("controllers.console.auth.login.BillingService.is_email_in_freeze")
def test_login_fails_when_account_frozen(self, mock_is_frozen, mock_db, app: Flask):
def test_login_fails_when_account_frozen(
self, mock_is_frozen, mock_db, app: Flask, caplog: pytest.LogCaptureFixture
):
"""
Test login rejection for frozen accounts.
@@ -231,17 +238,19 @@ class TestLoginApi:
mock_is_frozen.return_value = True
# Act & Assert
with patch("controllers.console.auth.login.logger.warning") as mock_log_warning:
with app.test_request_context(
"/login", method="POST", json={"email": "frozen@example.com", "password": encode_password("password")}
):
login_api = LoginApi()
with pytest.raises(AccountInFreezeError):
login_api.post()
with app.test_request_context(
"/login", method="POST", json={"email": "frozen@example.com", "password": encode_password("password")}
):
login_api = LoginApi()
with pytest.raises(AccountInFreezeError):
login_api.post()
assert mock_log_warning.call_count == 1
assert mock_log_warning.call_args.args[1] == "frozen@example.com"
assert mock_log_warning.call_args.args[2] == LoginFailureReason.ACCOUNT_IN_FREEZE
warn_records = [
r for r in caplog.records if r.name == "controllers.console.auth.login" and r.levelno == logging.WARNING
]
assert len(warn_records) == 1
assert warn_records[0].args[0] == "frozen@example.com"
assert warn_records[0].args[1] == LoginFailureReason.ACCOUNT_IN_FREEZE
@patch("controllers.console.wraps.db")
@patch("controllers.console.auth.login.dify_config.BILLING_ENABLED", False)
@@ -257,6 +266,7 @@ class TestLoginApi:
mock_is_rate_limit,
mock_db,
app: Flask,
caplog: pytest.LogCaptureFixture,
):
"""
Test login failure with invalid credentials.
@@ -272,20 +282,22 @@ class TestLoginApi:
mock_authenticate.side_effect = AccountPasswordError("Invalid password")
# Act & Assert
with patch("controllers.console.auth.login.logger.warning") as mock_log_warning:
with app.test_request_context(
"/login",
method="POST",
json={"email": "test@example.com", "password": encode_password("WrongPass123!")},
):
login_api = LoginApi()
with pytest.raises(AuthenticationFailedError):
login_api.post()
with app.test_request_context(
"/login",
method="POST",
json={"email": "test@example.com", "password": encode_password("WrongPass123!")},
):
login_api = LoginApi()
with pytest.raises(AuthenticationFailedError):
login_api.post()
mock_add_rate_limit.assert_called_once_with("test@example.com")
assert mock_log_warning.call_count == 1
assert mock_log_warning.call_args.args[1] == "test@example.com"
assert mock_log_warning.call_args.args[2] == LoginFailureReason.INVALID_CREDENTIALS
warn_records = [
r for r in caplog.records if r.name == "controllers.console.auth.login" and r.levelno == logging.WARNING
]
assert len(warn_records) == 1
assert warn_records[0].args[0] == "test@example.com"
assert warn_records[0].args[1] == LoginFailureReason.INVALID_CREDENTIALS
@patch("controllers.console.wraps.db")
@patch("controllers.console.auth.login.dify_config.BILLING_ENABLED", False)
@@ -293,7 +305,7 @@ class TestLoginApi:
@patch("controllers.console.auth.login.RegisterService.get_invitation_with_case_fallback")
@patch("controllers.console.auth.login.AccountService.authenticate")
def test_login_fails_for_banned_account(
self, mock_authenticate, mock_get_invitation, mock_is_rate_limit, mock_db, app: Flask
self, mock_authenticate, mock_get_invitation, mock_is_rate_limit, mock_db, app: Flask, caplog
):
"""
Test login rejection for banned accounts.
@@ -308,19 +320,21 @@ class TestLoginApi:
mock_authenticate.side_effect = AccountLoginError("Account is banned")
# Act & Assert
with patch("controllers.console.auth.login.logger.warning") as mock_log_warning:
with app.test_request_context(
"/login",
method="POST",
json={"email": "banned@example.com", "password": encode_password("ValidPass123!")},
):
login_api = LoginApi()
with pytest.raises(AccountBannedError):
login_api.post()
with app.test_request_context(
"/login",
method="POST",
json={"email": "banned@example.com", "password": encode_password("ValidPass123!")},
):
login_api = LoginApi()
with pytest.raises(AccountBannedError):
login_api.post()
assert mock_log_warning.call_count == 1
assert mock_log_warning.call_args.args[1] == "banned@example.com"
assert mock_log_warning.call_args.args[2] == LoginFailureReason.ACCOUNT_BANNED
warn_records = [
r for r in caplog.records if r.name == "controllers.console.auth.login" and r.levelno == logging.WARNING
]
assert len(warn_records) == 1
assert warn_records[0].args[0] == "banned@example.com"
assert warn_records[0].args[1] == LoginFailureReason.ACCOUNT_BANNED
@patch("controllers.console.wraps.db")
@patch("controllers.console.auth.login.dify_config.BILLING_ENABLED", False)
@@ -452,23 +466,26 @@ class TestLoginApi:
mock_get_token_data: MagicMock,
mock_db: MagicMock,
app: Flask,
caplog: pytest.LogCaptureFixture,
):
mock_get_token_data.return_value = {"email": "User@Example.com", "code": "123456"}
mock_get_account.side_effect = Unauthorized("Account is banned.")
with patch("controllers.console.auth.login.logger.warning") as mock_log_warning:
with app.test_request_context(
"/email-code-login/validity",
method="POST",
json={"email": "User@Example.com", "code": encode_code("123456"), "token": "token-123"},
):
with pytest.raises(AccountBannedError):
EmailCodeLoginApi().post()
with app.test_request_context(
"/email-code-login/validity",
method="POST",
json={"email": "User@Example.com", "code": encode_code("123456"), "token": "token-123"},
):
with pytest.raises(AccountBannedError):
EmailCodeLoginApi().post()
mock_revoke_token.assert_called_once_with("token-123")
assert mock_log_warning.call_count == 1
assert mock_log_warning.call_args.args[1] == "user@example.com"
assert mock_log_warning.call_args.args[2] == LoginFailureReason.ACCOUNT_BANNED
warn_records = [
r for r in caplog.records if r.name == "controllers.console.auth.login" and r.levelno == logging.WARNING
]
assert len(warn_records) == 1
assert warn_records[0].args[0] == "user@example.com"
assert warn_records[0].args[1] == LoginFailureReason.ACCOUNT_BANNED
class TestLogoutApi:
@@ -1,3 +1,4 @@
from inspect import unwrap
from types import SimpleNamespace
from unittest.mock import Mock
@@ -10,12 +11,6 @@ from models.account import Account, AccountStatus
from services.workflow_draft_variable_service import WorkflowDraftVariableList
def _unwrap(func):
while hasattr(func, "__wrapped__"):
func = func.__wrapped__
return func
def _make_account() -> Account:
account = Account(
name="tester",
@@ -66,7 +61,7 @@ def test_ensure_snippet_draft_variable_row_allowed_accepts_canvas_node_variable(
def test_conversation_variables_returns_empty_list(app: Flask):
api = module.SnippetConversationVariableCollectionApi()
handler = _unwrap(api.get)
handler = unwrap(api.get)
with app.test_request_context("/"):
result = handler(api, _make_account(), snippet=SimpleNamespace(id="snippet-1"))
@@ -76,7 +71,7 @@ def test_conversation_variables_returns_empty_list(app: Flask):
def test_system_variables_returns_empty_list(app: Flask):
api = module.SnippetSystemVariableCollectionApi()
handler = _unwrap(api.get)
handler = unwrap(api.get)
with app.test_request_context("/"):
result = handler(api, _make_account(), snippet=SimpleNamespace(id="snippet-1"))
@@ -91,7 +86,7 @@ def test_delete_variable_collection_deletes_current_user_variables(app: Flask, m
db_session.return_value = SimpleNamespace()
monkeypatch.setattr(module.db, "session", db_session)
api = module.SnippetWorkflowVariableCollectionApi()
handler = _unwrap(api.delete)
handler = unwrap(api.delete)
with app.test_request_context("/", method="DELETE"):
response = handler(api, _make_account(), snippet=SimpleNamespace(id="snippet-1"))
@@ -109,7 +104,7 @@ def test_variable_collection_get_raises_when_draft_workflow_missing(app: Flask,
)
api = module.SnippetWorkflowVariableCollectionApi()
handler = _unwrap(api.get)
handler = unwrap(api.get)
with app.test_request_context("/?page=1&limit=20"):
with pytest.raises(module.DraftWorkflowNotExist):
@@ -140,7 +135,7 @@ def test_node_variable_collection_get_lists_node_variables(app: Flask, monkeypat
)
api = module.SnippetNodeVariableCollectionApi()
handler = _unwrap(api.get)
handler = unwrap(api.get)
with app.test_request_context("/"):
result = handler(api, _make_account(), snippet=SimpleNamespace(id="snippet-1"), node_id="llm-1")
@@ -158,7 +153,7 @@ def test_node_variable_collection_delete_deletes_node_variables(app: Flask, monk
monkeypatch.setattr(module.db, "session", db_session)
api = module.SnippetNodeVariableCollectionApi()
handler = _unwrap(api.delete)
handler = unwrap(api.delete)
with app.test_request_context("/", method="DELETE"):
response = handler(api, _make_account(), snippet=SimpleNamespace(id="snippet-1"), node_id="llm-1")
@@ -177,7 +172,7 @@ def test_variable_patch_returns_variable_when_no_changes(app: Flask, monkeypatch
monkeypatch.setattr(module, "WorkflowDraftVariableService", Mock(return_value=draft_var_service))
api = module.SnippetVariableApi()
handler = _unwrap(api.patch)
handler = unwrap(api.patch)
with app.test_request_context("/", method="PATCH", json={}):
result = handler(
@@ -202,7 +197,7 @@ def test_variable_delete_deletes_variable(app: Flask, monkeypatch: pytest.Monkey
monkeypatch.setattr(module, "WorkflowDraftVariableService", Mock(return_value=draft_var_service))
api = module.SnippetVariableApi()
handler = _unwrap(api.delete)
handler = unwrap(api.delete)
with app.test_request_context("/", method="DELETE"):
response = handler(api, _make_account(), snippet=SimpleNamespace(id="snippet-1"), variable_id="var-1")
@@ -230,7 +225,7 @@ def test_variable_reset_returns_no_content_when_reset_result_is_none(app: Flask,
)
api = module.SnippetVariableResetApi()
handler = _unwrap(api.put)
handler = unwrap(api.put)
with app.test_request_context("/", method="PUT"):
response = handler(api, _make_account(), snippet=SimpleNamespace(id="snippet-1"), variable_id="var-1")
@@ -260,7 +255,7 @@ def test_environment_variables_returns_workflow_environment_variables(app: Flask
)
api = module.SnippetEnvironmentVariableCollectionApi()
handler = _unwrap(api.get)
handler = unwrap(api.get)
with app.test_request_context("/"):
result = handler(api, _make_account(), snippet=SimpleNamespace(id="snippet-1"))
@@ -22,23 +22,17 @@ class TestSpecSchemaDefinitionsApi:
assert status == 200
assert resp == schema_definitions
def test_get_exception_returns_empty_list(self):
def test_get_exception_returns_empty_list(self, caplog):
api = spec_module.SpecSchemaDefinitionsApi()
method = unwrap(api.get)
with (
patch.object(
spec_module,
"SchemaManager",
side_effect=Exception("boom"),
),
patch.object(
spec_module.logger,
"exception",
) as log_exception,
with patch.object(
spec_module,
"SchemaManager",
side_effect=Exception("boom"),
):
resp, status = method(api)
assert status == 200
assert resp == []
log_exception.assert_called_once()
assert "boom" in caplog.text
@@ -201,10 +201,10 @@ class TestPaginationMapping:
},
]
assert response["pagination"] == {
"total_count": 5,
"total_count": 4,
"per_page": 2,
"current_page": 1,
"total_pages": 3,
"total_pages": 2,
}
mock_list.assert_not_called()
@@ -1,4 +1,5 @@
import inspect
import logging
from io import BytesIO
from unittest.mock import MagicMock, patch
@@ -151,7 +152,9 @@ class TestTenantListApi:
get_plan_bulk_mock.assert_called_once_with(["t1", "t2"])
get_features_mock.assert_called_once_with("t2", exclude_vector_space=True)
def test_get_saas_path_falls_back_to_legacy_feature_path_on_bulk_error(self, app: Flask):
def test_get_saas_path_falls_back_to_legacy_feature_path_on_bulk_error(
self, app: Flask, caplog: pytest.LogCaptureFixture
):
"""Test fallback to FeatureService when bulk billing returns empty result.
BillingService.get_plan_bulk catches exceptions internally and returns empty dict,
@@ -170,6 +173,7 @@ class TestTenantListApi:
with (
app.test_request_context("/workspaces"),
caplog.at_level(logging.WARNING, logger="controllers.console.workspace.workspace"),
patch(
"controllers.console.workspace.workspace.TenantService.get_workspaces_for_account",
return_value=[(tenant1, make_membership()), (tenant2, make_membership())],
@@ -185,7 +189,6 @@ class TestTenantListApi:
"controllers.console.workspace.workspace.FeatureService.get_features",
return_value=features,
) as get_features_mock,
patch("controllers.console.workspace.workspace.logger.warning") as logger_warning_mock,
):
result, status = method(api, "t2", user)
@@ -194,7 +197,7 @@ class TestTenantListApi:
assert result["workspaces"][1]["plan"] == CloudPlan.TEAM
get_plan_bulk_mock.assert_called_once_with(["t1", "t2"])
assert get_features_mock.call_count == 2
logger_warning_mock.assert_called_once()
assert "get_plan_bulk returned empty result, falling back to legacy feature path" in caplog.messages
def test_get_billing_disabled_community_path(self, app: Flask):
api = TenantListApi()
@@ -365,7 +368,7 @@ class TestTenantApi:
with pytest.raises(Unauthorized):
method(api, user)
def test_post_info_path(self, app: Flask):
def test_post_info_path(self, app: Flask, caplog: pytest.LogCaptureFixture):
api = TenantApi()
method = inspect.unwrap(api.post)
@@ -374,15 +377,15 @@ class TestTenantApi:
with (
app.test_request_context("/info"),
caplog.at_level(logging.WARNING, logger="controllers.console.workspace.workspace"),
patch(
"controllers.console.workspace.workspace.WorkspaceService.get_tenant_info",
return_value={"id": "t1"},
),
patch("controllers.console.workspace.workspace.logger.warning") as warn_mock,
):
result, status = method(api, user)
warn_mock.assert_called_once()
assert "Deprecated URL /info was used." in caplog.messages
assert status == 200
@@ -321,3 +321,56 @@ def test_guard_no_external_identity_when_subject_email_absent(app):
view()
assert received["data"].external_identity is None
# --- auth-failure mapping (no raw 500 leak) ---
def test_guard_expired_token_raises_session_expired_401(app):
from controllers.openapi._errors import OpenApiErrorCode, SessionExpired
from libs.oauth_bearer import TokenExpiredError
router = _make_router()
with app.test_request_context("/test", headers={"Authorization": "Bearer tok"}):
with (
patch("controllers.openapi.auth.pipeline.extract_bearer", return_value="tok"),
patch("controllers.openapi.auth.pipeline.get_authenticator") as mock_auth,
patch("controllers.openapi.auth.pipeline.current_edition", return_value=Edition.CE),
):
mock_auth.return_value.authenticate.side_effect = TokenExpiredError("token_expired")
@router.guard(scope=Scope.FULL)
def view(*, auth_data):
pass
with pytest.raises(SessionExpired) as exc:
view()
assert exc.value.code == 401
assert exc.value.error_code == OpenApiErrorCode.TOKEN_EXPIRED
def test_guard_invalid_token_raises_unified_401_not_500(app):
from controllers.openapi._errors import InvalidBearer, OpenApiErrorCode
from libs.oauth_bearer import InvalidBearerError
router = _make_router()
with app.test_request_context("/test", headers={"Authorization": "Bearer tok"}):
with (
patch("controllers.openapi.auth.pipeline.extract_bearer", return_value="tok"),
patch("controllers.openapi.auth.pipeline.get_authenticator") as mock_auth,
patch("controllers.openapi.auth.pipeline.current_edition", return_value=Edition.CE),
):
mock_auth.return_value.authenticate.side_effect = InvalidBearerError("invalid_bearer")
@router.guard(scope=Scope.FULL)
def view(*, auth_data):
pass
with pytest.raises(InvalidBearer) as exc:
view()
assert exc.value.code == 401
assert exc.value.error_code == OpenApiErrorCode.UNAUTHORIZED
@@ -3,13 +3,36 @@
from __future__ import annotations
import sys
from types import SimpleNamespace
import uuid
from unittest.mock import Mock
import pytest
from flask import Flask
from controllers.openapi._models import AppRunRequest
from models import Account
from models.model import App, AppMode
_TEST_APP_ID = str(uuid.uuid4())
_TEST_TENANT_ID = str(uuid.uuid4())
_TEST_ACCOUNT_ID = str(uuid.uuid4())
def _make_app() -> App:
app = App()
app.id = _TEST_APP_ID
app.tenant_id = _TEST_TENANT_ID
app.name = "Streaming app"
app.mode = AppMode.CHAT
app.enable_site = False
app.enable_api = True
return app
def _make_account() -> Account:
account = Account(name="OpenAPI caller", email="caller@example.com")
account.id = _TEST_ACCOUNT_ID
return account
def test_app_run_request_has_no_response_mode_field():
@@ -40,15 +63,19 @@ def test_run_chat_always_calls_generate_with_streaming_true(
from controllers.openapi.app_run import _run_chat
generate_mock = Mock(return_value=iter([]))
class GenerateService:
generate = generate_mock
monkeypatch.setattr(
sys.modules["controllers.openapi.app_run"],
"AppGenerateService",
SimpleNamespace(generate=generate_mock),
GenerateService,
)
with app.test_request_context("/openapi/v1/apps/app-1/run", method="POST"):
with app.test_request_context(f"/openapi/v1/apps/{_TEST_APP_ID}/run", method="POST"):
_run_chat(
SimpleNamespace(id="app-1", tenant_id="t-1"),
SimpleNamespace(id="acct-1"),
_make_app(),
_make_account(),
AppRunRequest(inputs={}, query="hello"),
)
_, kwargs = generate_mock.call_args
@@ -80,11 +107,11 @@ def test_stop_task_calls_queue_manager_and_graph_engine(app: Flask, bypass_pipel
auth_data = AuthData.model_construct(
token_type=TokenType.OAUTH_ACCOUNT,
account_id=uuid.uuid4(),
account_id=uuid.UUID(_TEST_ACCOUNT_ID),
token_hash="test",
scopes=frozenset({Scope.FULL}),
app=SimpleNamespace(id="app-1", tenant_id="t-1"),
caller=SimpleNamespace(id="acct-1"),
app=_make_app(),
caller=_make_account(),
caller_kind="account",
)
@@ -5,6 +5,7 @@ view function decorated with @accepts/@returns, driven inside a request context.
"""
from functools import wraps
from typing import Any, cast
import pytest
from pydantic import BaseModel, ConfigDict, Field
@@ -100,7 +101,7 @@ def test_accepts_validation_error_is_sanitized_and_structured(app):
with pytest.raises(UnprocessableEntity) as exc_info:
view()
data = exc_info.value.data
data = cast(dict[str, Any], cast(Any, exc_info.value).data)
assert data["message"] == "Request validation failed"
assert isinstance(data["errors"], list)
assert data["errors"]
@@ -33,6 +33,7 @@ from controllers.openapi._errors import (
OpenApiErrorCode,
OpenApiErrorFormatter,
RecipientSurfaceMismatch,
SessionExpired,
)
from controllers.service_api.app.error import (
AppUnavailableError,
@@ -353,3 +354,20 @@ class TestErrorCodeEnumRegistration:
schema = model.__schema__
assert schema["type"] == "string"
assert set(schema["enum"]) == {member.value for member in OpenApiErrorCode}
class TestSessionExpired:
def test_session_expired_emits_token_expired_401_with_hint(self):
fmt = OpenApiErrorFormatter()
e = SessionExpired()
data = {"code": "unauthorized", "message": e.description, "status": 401}
wire = fmt.finalize(e, data, 401)
assert wire["code"] == OpenApiErrorCode.TOKEN_EXPIRED
assert wire["status"] == 401
assert wire["hint"]
def test_session_expired_code_is_401(self):
assert SessionExpired.code == 401
assert SessionExpired.error_code == OpenApiErrorCode.TOKEN_EXPIRED
@@ -1,4 +1,5 @@
import base64
import logging
from types import SimpleNamespace
from unittest.mock import MagicMock, patch
@@ -16,6 +17,13 @@ def encode_code(code: str) -> str:
return base64.b64encode(code.encode("utf-8")).decode()
def assert_login_failure_logged(caplog: pytest.LogCaptureFixture, email: str, reason: LoginFailureReason) -> None:
records = [record for record in caplog.records if record.name == "controllers.web.login"]
assert len(records) == 1
assert records[0].args[0] == email
assert records[0].args[1] == reason
@pytest.fixture
def app():
flask_app = Flask(__name__)
@@ -114,10 +122,10 @@ class TestLoginApi:
"controllers.web.login.WebAppAuthService.authenticate",
side_effect=services.errors.account.AccountLoginError(),
)
def test_login_banned_account(self, mock_auth: MagicMock, app: Flask) -> None:
def test_login_banned_account(self, mock_auth: MagicMock, app: Flask, caplog: pytest.LogCaptureFixture) -> None:
from controllers.console.error import AccountBannedError
with patch("controllers.web.login.logger.warning") as mock_log_warning:
with caplog.at_level(logging.WARNING, logger="controllers.web.login"):
with app.test_request_context(
"/web/login",
method="POST",
@@ -126,18 +134,16 @@ class TestLoginApi:
with pytest.raises(AccountBannedError):
LoginApi().post()
assert mock_log_warning.call_count == 1
assert mock_log_warning.call_args.args[1] == "user@example.com"
assert mock_log_warning.call_args.args[2] == LoginFailureReason.ACCOUNT_BANNED
assert_login_failure_logged(caplog, "user@example.com", LoginFailureReason.ACCOUNT_BANNED)
@patch(
"controllers.web.login.WebAppAuthService.authenticate",
side_effect=services.errors.account.AccountPasswordError(),
)
def test_login_wrong_password(self, mock_auth: MagicMock, app: Flask) -> None:
def test_login_wrong_password(self, mock_auth: MagicMock, app: Flask, caplog: pytest.LogCaptureFixture) -> None:
from controllers.console.auth.error import AuthenticationFailedError
with patch("controllers.web.login.logger.warning") as mock_log_warning:
with caplog.at_level(logging.WARNING, logger="controllers.web.login"):
with app.test_request_context(
"/web/login",
method="POST",
@@ -146,18 +152,16 @@ class TestLoginApi:
with pytest.raises(AuthenticationFailedError):
LoginApi().post()
assert mock_log_warning.call_count == 1
assert mock_log_warning.call_args.args[1] == "user@example.com"
assert mock_log_warning.call_args.args[2] == LoginFailureReason.INVALID_CREDENTIALS
assert_login_failure_logged(caplog, "user@example.com", LoginFailureReason.INVALID_CREDENTIALS)
@patch(
"controllers.web.login.WebAppAuthService.authenticate",
side_effect=services.errors.account.AccountNotFoundError(),
)
def test_login_account_not_found(self, mock_auth: MagicMock, app: Flask) -> None:
def test_login_account_not_found(self, mock_auth: MagicMock, app: Flask, caplog: pytest.LogCaptureFixture) -> None:
from controllers.console.auth.error import AuthenticationFailedError
with patch("controllers.web.login.logger.warning") as mock_log_warning:
with caplog.at_level(logging.WARNING, logger="controllers.web.login"):
with app.test_request_context(
"/web/login",
method="POST",
@@ -166,13 +170,13 @@ class TestLoginApi:
with pytest.raises(AuthenticationFailedError):
LoginApi().post()
assert mock_log_warning.call_count == 1
assert mock_log_warning.call_args.args[1] == "missing@example.com"
assert mock_log_warning.call_args.args[2] == LoginFailureReason.ACCOUNT_NOT_FOUND
assert_login_failure_logged(caplog, "missing@example.com", LoginFailureReason.ACCOUNT_NOT_FOUND)
@patch("controllers.web.login.WebAppAuthService.get_email_code_login_data", return_value=None)
def test_email_code_login_logs_invalid_token(self, mock_get_token_data: MagicMock, app: Flask) -> None:
with patch("controllers.web.login.logger.warning") as mock_log_warning:
def test_email_code_login_logs_invalid_token(
self, mock_get_token_data: MagicMock, app: Flask, caplog: pytest.LogCaptureFixture
) -> None:
with caplog.at_level(logging.WARNING, logger="controllers.web.login"):
with app.test_request_context(
"/web/email-code-login/validity",
method="POST",
@@ -182,9 +186,7 @@ class TestLoginApi:
EmailCodeLoginApi().post()
mock_get_token_data.assert_called_once_with("token-123")
assert mock_log_warning.call_count == 1
assert mock_log_warning.call_args.args[1] == "user@example.com"
assert mock_log_warning.call_args.args[2] == LoginFailureReason.INVALID_EMAIL_CODE_TOKEN
assert_login_failure_logged(caplog, "user@example.com", LoginFailureReason.INVALID_EMAIL_CODE_TOKEN)
@patch("controllers.web.login.WebAppAuthService.revoke_email_code_login_token")
@patch(
@@ -201,10 +203,11 @@ class TestLoginApi:
mock_get_user: MagicMock,
mock_revoke_token: MagicMock,
app: Flask,
caplog: pytest.LogCaptureFixture,
) -> None:
from controllers.console.error import AccountBannedError
with patch("controllers.web.login.logger.warning") as mock_log_warning:
with caplog.at_level(logging.WARNING, logger="controllers.web.login"):
with app.test_request_context(
"/web/email-code-login/validity",
method="POST",
@@ -215,9 +218,7 @@ class TestLoginApi:
mock_get_token_data.assert_called_once_with("token-123")
mock_revoke_token.assert_called_once_with("token-123")
assert mock_log_warning.call_count == 1
assert mock_log_warning.call_args.args[1] == "user@example.com"
assert mock_log_warning.call_args.args[2] == LoginFailureReason.ACCOUNT_BANNED
assert_login_failure_logged(caplog, "user@example.com", LoginFailureReason.ACCOUNT_BANNED)
class TestLoginStatusApi:
@@ -1,5 +1,6 @@
from __future__ import annotations
import logging
from contextlib import contextmanager
from types import SimpleNamespace
from unittest.mock import MagicMock
@@ -961,7 +962,9 @@ class TestAdvancedChatAppGeneratorInternals:
stream=False,
)
def test_handle_response_re_raises_value_error(self, monkeypatch: pytest.MonkeyPatch):
def test_handle_response_re_raises_value_error(
self, monkeypatch: pytest.MonkeyPatch, caplog: pytest.LogCaptureFixture
):
generator = AdvancedChatAppGenerator()
generator._dialogue_count = 1
app_config = self._build_app_config()
@@ -986,29 +989,28 @@ class TestAdvancedChatAppGeneratorInternals:
def process(self):
raise ValueError("other error")
logger_exception = MagicMock()
monkeypatch.setattr("core.app.apps.advanced_chat.app_generator.logger.exception", logger_exception)
monkeypatch.setattr("core.app.apps.advanced_chat.app_generator.AdvancedChatAppGenerateTaskPipeline", _Pipeline)
with pytest.raises(ValueError, match="other error"):
generator._handle_advanced_chat_response(
application_generate_entity=application_generate_entity,
workflow=WorkflowSnapshot(id="wf", tenant_id="tenant", features_dict={}),
queue_manager=SimpleNamespace(),
conversation=ConversationSnapshot(id="conv", mode=AppMode.ADVANCED_CHAT),
message=MessageSnapshot(
id="msg",
query="hello",
created_at=naive_utc_now(),
status=MessageStatus.NORMAL,
answer="",
),
user=SimpleNamespace(),
draft_var_saver_factory=lambda **kwargs: None,
stream=False,
)
with caplog.at_level(logging.ERROR, logger="core.app.apps.advanced_chat.app_generator"):
with pytest.raises(ValueError, match="other error"):
generator._handle_advanced_chat_response(
application_generate_entity=application_generate_entity,
workflow=WorkflowSnapshot(id="wf", tenant_id="tenant", features_dict={}),
queue_manager=SimpleNamespace(),
conversation=ConversationSnapshot(id="conv", mode=AppMode.ADVANCED_CHAT),
message=MessageSnapshot(
id="msg",
query="hello",
created_at=naive_utc_now(),
status=MessageStatus.NORMAL,
answer="",
),
user=SimpleNamespace(),
draft_var_saver_factory=lambda **kwargs: None,
stream=False,
)
logger_exception.assert_called_once()
assert "Failed to process generate task pipeline, conversation_id: conv" in caplog.messages
def test_generate_worker_handles_invoke_auth_error(self, monkeypatch: pytest.MonkeyPatch):
generator = AdvancedChatAppGenerator()
@@ -69,7 +69,7 @@ class TestGenerateSuccess:
def test_runtime_session_snapshot_id_is_stable_for_debugger_only(self):
assert (
AgentAppGenerator._runtime_session_snapshot_id(invoke_from=InvokeFrom.DEBUGGER, snapshot_id="snap-1")
== "snap-1"
is None
)
assert (
AgentAppGenerator._runtime_session_snapshot_id(invoke_from=InvokeFrom.WEB_APP, snapshot_id="snap-1")
@@ -111,12 +111,7 @@ class TestGenerateSuccess:
assert result == {"result": "ok"}
thread_obj.start.assert_called_once()
generator._resolve_agent.assert_called_once_with(
app_model,
invoke_from=InvokeFrom.WEB_APP,
draft_type=None,
user=user,
)
generator._resolve_agent.assert_called_once_with(app_model)
def test_generate_loads_existing_conversation(self, generator: AgentAppGenerator, mocker: MockerFixture):
app_model = mocker.MagicMock(id="app1", tenant_id="tenant", mode="agent")
@@ -169,19 +169,12 @@ class TestAgentAppRuntimeRequestBuilder:
"model": "gpt-4o-mini",
},
"knowledge": {
"sets": [
{
"id": "support",
"name": "Support KB",
"datasets": [{"id": "dataset-1"}, {"id": "dataset-2"}],
"query": {"mode": "generated_query"},
"retrieval": {
"mode": "multiple",
"top_k": 3,
"score_threshold": None,
},
}
],
"datasets": [{"id": "dataset-1"}, {"id": "dataset-2"}],
"query_config": {
"top_k": 3,
"score_threshold": 0.5,
"score_threshold_enabled": False,
},
},
}
)
@@ -196,12 +189,10 @@ class TestAgentAppRuntimeRequestBuilder:
assert knowledge.type == "dify.knowledge_base"
assert knowledge.deps == {"execution_context": "execution_context"}
dumped_config = knowledge.config.model_dump(mode="json", by_alias=True)
knowledge_set = dumped_config["sets"][0]
assert [dataset["id"] for dataset in knowledge_set["datasets"]] == ["dataset-1", "dataset-2"]
assert knowledge_set["query"] == {"mode": "generated_query", "value": None}
assert knowledge_set["retrieval"]["mode"] == "multiple"
assert knowledge_set["retrieval"]["top_k"] == 3
assert knowledge_set["retrieval"]["score_threshold"] == 0.0
assert dumped_config["dataset_ids"] == ["dataset-1", "dataset-2"]
assert dumped_config["retrieval"]["mode"] == "multiple"
assert dumped_config["retrieval"]["top_k"] == 3
assert dumped_config["retrieval"]["score_threshold"] == 0.0
def test_build_raises_when_model_missing(self):
builder = AgentAppRuntimeRequestBuilder(
@@ -1,4 +1,5 @@
import contextlib
import logging
import pytest
from pydantic import ValidationError
@@ -274,7 +275,9 @@ class TestAgentChatAppGeneratorWorker:
assert queue_manager.publish_error.called
def test_generate_worker_logs_value_error_when_debug(self, generator, mocker: MockerFixture):
def test_generate_worker_logs_value_error_when_debug(
self, generator, mocker: MockerFixture, caplog: pytest.LogCaptureFixture
):
queue_manager = mocker.MagicMock()
generator._get_conversation = mocker.MagicMock(return_value=mocker.MagicMock())
generator._get_message = mocker.MagicMock(return_value=mocker.MagicMock())
@@ -285,15 +288,15 @@ class TestAgentChatAppGeneratorWorker:
mocker.patch("core.app.apps.agent_chat.app_generator.db.session.close")
mocker.patch("core.app.apps.agent_chat.app_generator.dify_config", new=mocker.MagicMock(DEBUG=True))
logger = mocker.patch("core.app.apps.agent_chat.app_generator.logger")
generator._generate_worker(
flask_app=mocker.MagicMock(),
context=mocker.MagicMock(),
application_generate_entity=mocker.MagicMock(),
queue_manager=queue_manager,
conversation_id="conv",
message_id="msg",
)
with caplog.at_level(logging.ERROR, logger="core.app.apps.agent_chat.app_generator"):
generator._generate_worker(
flask_app=mocker.MagicMock(),
context=mocker.MagicMock(),
application_generate_entity=mocker.MagicMock(),
queue_manager=queue_manager,
conversation_id="conv",
message_id="msg",
)
logger.exception.assert_called_once()
assert "Error when generating" in caplog.messages
@@ -1,5 +1,6 @@
from __future__ import annotations
import logging
from types import SimpleNamespace
from unittest.mock import MagicMock
@@ -263,11 +264,11 @@ class TestAppRunner:
files=[],
)
def test_handle_invoke_result_stream_routes_chunks_and_builds_message(self, monkeypatch: pytest.MonkeyPatch):
def test_handle_invoke_result_stream_routes_chunks_and_builds_message(
self, monkeypatch: pytest.MonkeyPatch, caplog: pytest.LogCaptureFixture
):
runner = AppRunner()
queue = _QueueRecorder()
warning_logger = MagicMock()
monkeypatch.setattr("core.app.apps.base_app_runner._logger.warning", warning_logger)
image_content = ImagePromptMessageContent(
url="https://example.com/image.png", format="png", mime_type="image/png"
@@ -290,23 +291,24 @@ class TestAppRunner:
),
)
runner._handle_invoke_result(
invoke_result=_stream(),
queue_manager=queue,
stream=True,
agent=False,
)
with caplog.at_level(logging.WARNING, logger="core.app.apps.base_app_runner"):
runner._handle_invoke_result(
invoke_result=_stream(),
queue_manager=queue,
stream=True,
agent=False,
)
assert isinstance(queue.events[0], QueueLLMChunkEvent)
assert isinstance(queue.events[-1], QueueMessageEndEvent)
assert queue.events[-1].llm_result.message.content == "abc"
warning_logger.assert_called_once()
assert "Received multimodal output but missing required parameters" in caplog.messages
def test_handle_invoke_result_stream_agent_mode_handles_multimodal_errors(self, monkeypatch: pytest.MonkeyPatch):
def test_handle_invoke_result_stream_agent_mode_handles_multimodal_errors(
self, monkeypatch: pytest.MonkeyPatch, caplog: pytest.LogCaptureFixture
):
runner = AppRunner()
queue = _QueueRecorder()
exception_logger = MagicMock()
monkeypatch.setattr("core.app.apps.base_app_runner._logger.exception", exception_logger)
monkeypatch.setattr(
runner,
@@ -335,19 +337,20 @@ class TestAppRunner:
),
)
runner._handle_invoke_result_stream(
invoke_result=_stream(),
queue_manager=queue,
agent=True,
message_id="message-id",
user_id="user-id",
tenant_id="tenant-id",
)
with caplog.at_level(logging.ERROR, logger="core.app.apps.base_app_runner"):
runner._handle_invoke_result_stream(
invoke_result=_stream(),
queue_manager=queue,
agent=True,
message_id="message-id",
user_id="user-id",
tenant_id="tenant-id",
)
assert isinstance(queue.events[0], QueueAgentMessageEvent)
assert isinstance(queue.events[-1], QueueMessageEndEvent)
assert queue.events[-1].llm_result.usage == usage
exception_logger.assert_called_once()
assert "Failed to handle multimodal image output" in caplog.messages
def test_handle_invoke_result_stream_closes_generator_when_stopped(self):
runner = AppRunner()
@@ -1,5 +1,6 @@
from __future__ import annotations
import logging
from contextlib import contextmanager
from types import SimpleNamespace
from unittest.mock import MagicMock
@@ -639,7 +640,9 @@ class TestWorkflowGenerateTaskPipeline:
assert sleep_spy
assert any(isinstance(item, MessageAudioEndStreamResponse) for item in responses)
def test_wrapper_process_stream_response_handles_audio_exception(self, monkeypatch: pytest.MonkeyPatch):
def test_wrapper_process_stream_response_handles_audio_exception(
self, monkeypatch: pytest.MonkeyPatch, caplog: pytest.LogCaptureFixture
):
pipeline = _make_pipeline()
pipeline._workflow_features_dict = {
"text_to_speech": {"enabled": True, "autoPlay": "enabled", "voice": "v", "language": "en"}
@@ -659,20 +662,16 @@ class TestWorkflowGenerateTaskPipeline:
def publish(self, message):
_ = message
logger_exception = []
monkeypatch.setattr("core.app.apps.workflow.generate_task_pipeline.time.time", lambda: 0.0)
monkeypatch.setattr(
"core.app.apps.workflow.generate_task_pipeline.logger.exception",
lambda *args, **kwargs: logger_exception.append((args, kwargs)),
)
monkeypatch.setattr(
"core.app.apps.workflow.generate_task_pipeline.AppGeneratorTTSPublisher",
_Publisher,
)
responses = list(pipeline._wrapper_process_stream_response())
with caplog.at_level(logging.ERROR, logger="core.app.apps.workflow.generate_task_pipeline"):
responses = list(pipeline._wrapper_process_stream_response())
assert logger_exception
assert "Fails to get audio trunk, task_id: task" in caplog.messages
assert any(isinstance(item, MessageAudioEndStreamResponse) for item in responses)
def test_database_session_rolls_back_on_error(self, monkeypatch: pytest.MonkeyPatch):
@@ -2042,7 +2042,9 @@ def test_get_custom_provider_models_skips_schema_models_with_mismatched_type() -
assert all(model.model != "embed-model" for model in models)
def test_get_custom_provider_models_skips_custom_models_on_schema_error_or_none() -> None:
def test_get_custom_provider_models_skips_custom_models_on_schema_error_or_none(
caplog: pytest.LogCaptureFixture,
) -> None:
configuration = _build_provider_configuration()
configuration.custom_configuration.models = [
CustomModelConfiguration(model="error-custom", model_type=ModelType.LLM, credentials={"k": "v"}),
@@ -2064,7 +2066,7 @@ def test_get_custom_provider_models_skips_custom_models_on_schema_error_or_none(
return None
return _build_ai_model(model)
with patch("core.entities.provider_configuration.logger.warning") as mock_warning:
with caplog.at_level(logging.WARNING, logger="core.entities.provider_configuration"):
with patch.object(ProviderConfiguration, "get_model_schema", side_effect=_schema):
models = configuration._get_custom_provider_models(
model_types=[ModelType.LLM],
@@ -2072,6 +2074,6 @@ def test_get_custom_provider_models_skips_custom_models_on_schema_error_or_none(
model_setting_map={},
)
assert mock_warning.call_count == 1
assert "get custom model schema failed, boom" in caplog.messages
assert any(model.model == "ok-custom" for model in models)
assert all(model.model != "none-custom" for model in models)
@@ -512,55 +512,12 @@ def test_build_maps_agent_soul_knowledge_to_knowledge_layer_config():
"model": "gpt-test",
},
"knowledge": {
"sets": [
{
"id": "support",
"name": "Support KB",
"description": "Support content",
"datasets": [{"id": "dataset-1"}, {"id": "dataset-2"}],
"query": {"mode": "generated_query"},
"retrieval": {
"mode": "multiple",
"top_k": 6,
"score_threshold": 0.4,
"reranking_model": {"provider": "cohere", "model": "rerank-v3"},
"weights": {"weight_type": "weighted_score", "vector_setting": {"vector_weight": 0.7}},
},
"metadata_filtering": {
"mode": "manual",
"conditions": {
"logical_operator": "and",
"conditions": [
{"name": "category", "comparison_operator": "contains", "value": "auth"}
],
},
},
},
{
"id": "release",
"name": "Release Notes",
"datasets": [{"id": "dataset-3"}],
"query": {"mode": "user_query", "value": "release notes"},
"retrieval": {
"mode": "single",
"model": {
"provider": "openai",
"name": "gpt-4o-mini",
"mode": "chat",
"completion_params": {"temperature": 0.2},
},
},
"metadata_filtering": {
"mode": "automatic",
"model_config": {
"provider": "openai",
"name": "gpt-4o-mini",
"mode": "chat",
"completion_params": {},
},
},
},
],
"datasets": [{"id": "dataset-1"}, {"id": " "}, {"id": "dataset-2"}],
"query_config": {
"top_k": 6,
"score_threshold": 0.4,
"score_threshold_enabled": True,
},
},
}
),
@@ -574,75 +531,25 @@ def test_build_maps_agent_soul_knowledge_to_knowledge_layer_config():
knowledge_layer = layers["knowledge"]
assert knowledge_layer["type"] == "dify.knowledge_base"
assert knowledge_layer["deps"] == {"execution_context": DIFY_EXECUTION_CONTEXT_LAYER_ID}
assert knowledge_layer["config"]["sets"] == [
{
"id": "support",
"name": "Support KB",
"description": "Support content",
"datasets": [
{"id": "dataset-1", "name": None, "description": None},
{"id": "dataset-2", "name": None, "description": None},
],
"query": {"mode": "generated_query", "value": None},
"retrieval": {
"mode": "multiple",
"top_k": 6,
"score_threshold": 0.4,
"reranking_mode": "reranking_model",
"reranking_enable": True,
"reranking_model": {"provider": "cohere", "model": "rerank-v3"},
"weights": {"weight_type": "weighted_score", "vector_setting": {"vector_weight": 0.7}},
"model": None,
},
"metadata_filtering": {
"mode": "manual",
"metadata_model_config": None,
"conditions": {
"logical_operator": "and",
"conditions": [
{"name": "category", "comparison_operator": "contains", "value": "auth"}
],
},
},
assert knowledge_layer["config"] == {
"dataset_ids": ["dataset-1", "dataset-2"],
"retrieval": {
"mode": "multiple",
"top_k": 6,
"score_threshold": 0.4,
"reranking_mode": "reranking_model",
"reranking_enable": True,
"reranking_model": None,
"weights": None,
"model": None,
},
{
"id": "release",
"name": "Release Notes",
"description": None,
"datasets": [{"id": "dataset-3", "name": None, "description": None}],
"query": {"mode": "user_query", "value": "release notes"},
"retrieval": {
"mode": "single",
"top_k": None,
"score_threshold": 0.0,
"reranking_mode": "reranking_model",
"reranking_enable": True,
"reranking_model": None,
"weights": None,
"model": {
"provider": "openai",
"name": "gpt-4o-mini",
"mode": "chat",
"completion_params": {"temperature": 0.2},
},
},
"metadata_filtering": {
"mode": "automatic",
"metadata_model_config": {
"provider": "openai",
"name": "gpt-4o-mini",
"mode": "chat",
"completion_params": {},
},
"conditions": None,
},
},
]
assert knowledge_layer["config"]["max_result_content_chars"] == 2000
assert knowledge_layer["config"]["max_observation_chars"] == 12000
"metadata_filtering": {"mode": "disabled", "metadata_model_config": None, "conditions": None},
"max_result_content_chars": 2000,
"max_observation_chars": 12000,
}
def test_build_knowledge_layer_maps_disabled_score_threshold_to_zero():
def test_build_knowledge_layer_uses_stable_default_top_k_when_query_config_omits_it():
context = _context()
snapshot = AgentConfigSnapshot(
id="snapshot-1",
@@ -658,19 +565,8 @@ def test_build_knowledge_layer_maps_disabled_score_threshold_to_zero():
"model": "gpt-test",
},
"knowledge": {
"sets": [
{
"id": "support",
"name": "Support KB",
"datasets": [{"id": "dataset-1"}],
"query": {"mode": "generated_query"},
"retrieval": {
"mode": "multiple",
"top_k": 4,
"score_threshold": None,
},
}
],
"datasets": [{"id": "dataset-1"}],
"query_config": {},
},
}
),
@@ -681,10 +577,10 @@ def test_build_knowledge_layer_maps_disabled_score_threshold_to_zero():
dumped = result.request.model_dump(mode="json")
knowledge_layer = next(layer for layer in dumped["composition"]["layers"] if layer["name"] == "knowledge")
assert knowledge_layer["config"]["sets"][0]["retrieval"]["score_threshold"] == 0.0
assert knowledge_layer["config"]["retrieval"]["top_k"] == 4
def test_build_skips_knowledge_layer_when_agent_soul_has_no_sets():
def test_build_skips_knowledge_layer_when_agent_soul_has_no_valid_dataset_ids():
context = _context()
snapshot = AgentConfigSnapshot(
id="snapshot-1",
@@ -699,7 +595,9 @@ def test_build_skips_knowledge_layer_when_agent_soul_has_no_sets():
"model_provider": "openai",
"model": "gpt-test",
},
"knowledge": {"sets": []},
"knowledge": {
"datasets": [{"id": " "}, {}],
},
}
),
)
@@ -942,29 +840,46 @@ def _soul_with_drive_skill() -> AgentSoulConfig:
"and [§file:files%2Fsample.pdf:sample.pdf§]."
)
},
files={
"skills": [
{
"id": "tender-analyzer",
"name": "Tender Analyzer",
"description": "Parses RFPs.",
"path": "tender-analyzer",
"skill_md_key": "tender-analyzer/SKILL.md",
"full_archive_key": "tender-analyzer/.DIFY-SKILL-FULL.zip",
}
],
"files": [{"id": "files/sample.pdf", "name": "sample.pdf", "drive_key": "files/sample.pdf"}],
},
model=AgentSoulModelConfig(plugin_id="langgenius/openai", model_provider="openai", model="gpt-test"),
)
def _mock_drive_catalog(monkeypatch: pytest.MonkeyPatch) -> None:
return None
monkeypatch.setattr(
"core.workflow.nodes.agent_v2.runtime_request_builder.AgentDriveService.list_skills",
lambda self, *, tenant_id, agent_id: [
{
"path": "tender-analyzer",
"skill_md_key": "tender-analyzer/SKILL.md",
"archive_key": "tender-analyzer/.DIFY-SKILL-FULL.zip",
"name": "Tender Analyzer",
"description": "Parses RFPs.",
"size": 123,
"mime_type": "text/markdown",
"hash": "hash-1",
"created_at": 1,
}
],
)
monkeypatch.setattr(
"core.workflow.nodes.agent_v2.runtime_request_builder.AgentDriveService.manifest",
lambda self, *, tenant_id, agent_id, prefix="", include_download_url=False: [
{"key": "tender-analyzer/SKILL.md", "is_skill": True},
{"key": "tender-analyzer/.DIFY-SKILL-FULL.zip", "is_skill": False},
{"key": "files/sample.pdf", "is_skill": False},
],
)
def _mock_empty_drive_catalog(monkeypatch: pytest.MonkeyPatch) -> None:
return None
monkeypatch.setattr(
"core.workflow.nodes.agent_v2.runtime_request_builder.AgentDriveService.list_skills",
lambda self, *, tenant_id, agent_id: [],
)
monkeypatch.setattr(
"core.workflow.nodes.agent_v2.runtime_request_builder.AgentDriveService.manifest",
lambda self, *, tenant_id, agent_id, prefix="", include_download_url=False: [],
)
def test_build_drive_layer_config_catalogs_drive_skills_and_mentions(monkeypatch: pytest.MonkeyPatch):
@@ -1090,6 +1005,14 @@ def test_workflow_runtime_missing_drive_mentions_fall_back_to_label_then_decoded
monkeypatch.setattr(
"core.workflow.nodes.agent_v2.runtime_request_builder.dify_config.AGENT_DRIVE_MANIFEST_ENABLED", True
)
monkeypatch.setattr(
"core.workflow.nodes.agent_v2.runtime_request_builder.AgentDriveService.list_skills",
lambda self, *, tenant_id, agent_id: [],
)
monkeypatch.setattr(
"core.workflow.nodes.agent_v2.runtime_request_builder.AgentDriveService.manifest",
lambda self, *, tenant_id, agent_id, prefix="", include_download_url=False: [],
)
context = _context()
context.snapshot.config_snapshot = AgentSoulConfig(
prompt={
@@ -1098,7 +1021,6 @@ def test_workflow_runtime_missing_drive_mentions_fall_back_to_label_then_decoded
"and [§file:files%2Fno-label.txt§]."
)
},
files={"files": [{"id": "files/no-label.txt", "name": "no-label.txt", "drive_key": "files/no-label.txt"}]},
model=AgentSoulModelConfig(plugin_id="langgenius/openai", model_provider="openai", model="gpt-test"),
)
@@ -1172,15 +1094,7 @@ def test_feature_manifest_marks_knowledge_supported_without_warning_when_configu
soul = AgentSoulConfig.model_validate(
{
"knowledge": {
"sets": [
{
"id": "product",
"name": "Product Docs",
"datasets": [{"id": "dataset-1", "name": "Product Docs"}],
"query": {"mode": "generated_query"},
"retrieval": {"mode": "multiple", "top_k": 4},
}
],
"datasets": [{"id": "dataset-1", "name": "Product Docs"}],
}
}
)
@@ -1192,13 +1106,13 @@ def test_feature_manifest_marks_knowledge_supported_without_warning_when_configu
assert all("knowledge" not in w["section"] for w in manifest["unsupported_runtime_warnings"])
def test_feature_manifest_treats_empty_knowledge_sets_as_not_configured():
def test_feature_manifest_treats_blank_knowledge_dataset_ids_as_not_configured():
from core.workflow.nodes.agent_v2.runtime_feature_manifest import build_runtime_feature_manifest
soul = AgentSoulConfig.model_validate(
{
"knowledge": {
"sets": [],
"datasets": [{"id": " "}, {}],
}
}
)
@@ -55,33 +55,6 @@ def _snapshot() -> AgentConfigSnapshot:
)
def _snapshot_with_knowledge_dataset(dataset_id: str) -> AgentConfigSnapshot:
return AgentConfigSnapshot(
id="snapshot-1",
tenant_id="tenant-1",
agent_id="agent-1",
version=1,
config_snapshot=AgentSoulConfig(
model=AgentSoulModelConfig(
plugin_id="langgenius/openai",
model_provider="openai",
model="gpt-test",
),
knowledge={
"sets": [
{
"id": "support",
"name": "Support KB",
"datasets": [{"id": dataset_id}],
"query": {"mode": "generated_query"},
"retrieval": {"mode": "multiple", "top_k": 4},
}
]
},
),
)
def _graph(edges: list[dict]) -> dict:
return {
"nodes": [
@@ -542,35 +515,6 @@ def test_publish_validation_rejects_missing_file_ref():
)
def test_publish_validation_rejects_missing_or_out_of_scope_knowledge_datasets(
monkeypatch: pytest.MonkeyPatch,
):
dataset_id = "550e8400-e29b-41d4-a716-446655440000"
node_job = WorkflowNodeJobConfig.model_validate({})
snapshot = _snapshot_with_knowledge_dataset(dataset_id)
session = Mock()
session.scalar.side_effect = [_binding(node_job), _agent(), snapshot]
captured = {}
def fake_get_datasets_by_ids(ids, tenant_id):
captured["ids"] = ids
captured["tenant_id"] = tenant_id
return [], 0
import services.dataset_service as dataset_service_module
monkeypatch.setattr(dataset_service_module.DatasetService, "get_datasets_by_ids", fake_get_datasets_by_ids)
with pytest.raises(WorkflowAgentNodeValidationError, match=dataset_id):
WorkflowAgentNodeValidator.validate_published_workflow(
session=session,
workflow=_workflow(_graph([{"source": "start", "target": "agent-node"}])),
)
assert captured == {"ids": [dataset_id], "tenant_id": "tenant-1"}
def test_publish_validation_accepts_tool_node_agentic_manual_mode():
session = Mock()
@@ -21,6 +21,7 @@ from core.ops.entities.trace_entity import (
WorkflowNodeTraceInfo,
WorkflowTraceInfo,
)
from enterprise.telemetry.enterprise_trace import EnterpriseOtelTrace
from enterprise.telemetry.entities import (
EnterpriseTelemetryCounter,
EnterpriseTelemetryEvent,
@@ -297,43 +298,43 @@ def test_init_succeeds_with_valid_exporter(mock_exporter):
class TestSafePayloadValue:
def test_string_passthrough(self, trace_handler):
def test_string_passthrough(self, trace_handler: EnterpriseOtelTrace):
assert trace_handler._safe_payload_value("hello") == "hello"
def test_dict_passthrough(self, trace_handler):
def test_dict_passthrough(self, trace_handler: EnterpriseOtelTrace):
d = {"key": "val"}
assert trace_handler._safe_payload_value(d) == d
def test_list_passthrough(self, trace_handler):
def test_list_passthrough(self, trace_handler: EnterpriseOtelTrace):
lst = [1, 2, 3]
assert trace_handler._safe_payload_value(lst) == lst
def test_none_returns_none(self, trace_handler):
def test_none_returns_none(self, trace_handler: EnterpriseOtelTrace):
assert trace_handler._safe_payload_value(None) is None
def test_int_returns_none(self, trace_handler):
def test_int_returns_none(self, trace_handler: EnterpriseOtelTrace):
assert trace_handler._safe_payload_value(42) is None
def test_bool_returns_none(self, trace_handler):
def test_bool_returns_none(self, trace_handler: EnterpriseOtelTrace):
assert trace_handler._safe_payload_value(True) is None
class TestMaybeJson:
def test_none_returns_none(self, trace_handler):
def test_none_returns_none(self, trace_handler: EnterpriseOtelTrace):
assert trace_handler._maybe_json(None) is None
def test_string_passthrough(self, trace_handler):
def test_string_passthrough(self, trace_handler: EnterpriseOtelTrace):
assert trace_handler._maybe_json("hello") == "hello"
def test_dict_serialised(self, trace_handler):
def test_dict_serialised(self, trace_handler: EnterpriseOtelTrace):
result = trace_handler._maybe_json({"a": 1})
assert result == json.dumps({"a": 1})
def test_list_serialised(self, trace_handler):
def test_list_serialised(self, trace_handler: EnterpriseOtelTrace):
result = trace_handler._maybe_json([1, 2])
assert result == "[1, 2]"
def test_non_serialisable_falls_back_to_str(self, trace_handler):
def test_non_serialisable_falls_back_to_str(self, trace_handler: EnterpriseOtelTrace):
class Unserializable:
def __repr__(self):
return "Unserializable()"
@@ -344,22 +345,22 @@ class TestMaybeJson:
class TestContentOrRef:
def test_returns_content_when_include_content_true(self, trace_handler, mock_exporter):
def test_returns_content_when_include_content_true(self, trace_handler: EnterpriseOtelTrace, mock_exporter):
mock_exporter.include_content = True
result = trace_handler._content_or_ref("actual content", "ref:x=1")
assert result == "actual content"
def test_returns_ref_when_include_content_false(self, trace_handler, mock_exporter):
def test_returns_ref_when_include_content_false(self, trace_handler: EnterpriseOtelTrace, mock_exporter):
mock_exporter.include_content = False
result = trace_handler._content_or_ref("actual content", "ref:x=1")
assert result == "ref:x=1"
def test_dict_serialised_when_include_content_true(self, trace_handler, mock_exporter):
def test_dict_serialised_when_include_content_true(self, trace_handler: EnterpriseOtelTrace, mock_exporter):
mock_exporter.include_content = True
result = trace_handler._content_or_ref({"key": "val"}, "ref:x=1")
assert result == json.dumps({"key": "val"})
def test_none_returns_none_when_include_content_true(self, trace_handler, mock_exporter):
def test_none_returns_none_when_include_content_true(self, trace_handler: EnterpriseOtelTrace, mock_exporter):
mock_exporter.include_content = True
result = trace_handler._content_or_ref(None, "ref:x=1")
assert result is None
@@ -371,67 +372,67 @@ class TestContentOrRef:
class TestTraceDispatcher:
def test_dispatches_workflow_trace(self, trace_handler):
def test_dispatches_workflow_trace(self, trace_handler: EnterpriseOtelTrace):
with patch.object(trace_handler, "_workflow_trace") as mock_method:
info = make_workflow_info()
trace_handler.trace(info)
mock_method.assert_called_once_with(info)
def test_dispatches_message_trace(self, trace_handler):
def test_dispatches_message_trace(self, trace_handler: EnterpriseOtelTrace):
with patch.object(trace_handler, "_message_trace") as mock_method:
info = make_message_info()
trace_handler.trace(info)
mock_method.assert_called_once_with(info)
def test_dispatches_tool_trace(self, trace_handler):
def test_dispatches_tool_trace(self, trace_handler: EnterpriseOtelTrace):
with patch.object(trace_handler, "_tool_trace") as mock_method:
info = make_tool_info()
trace_handler.trace(info)
mock_method.assert_called_once_with(info)
def test_dispatches_draft_node_execution_trace(self, trace_handler):
def test_dispatches_draft_node_execution_trace(self, trace_handler: EnterpriseOtelTrace):
with patch.object(trace_handler, "_draft_node_execution_trace") as mock_method:
info = make_draft_node_info()
trace_handler.trace(info)
mock_method.assert_called_once_with(info)
def test_dispatches_node_execution_trace(self, trace_handler):
def test_dispatches_node_execution_trace(self, trace_handler: EnterpriseOtelTrace):
with patch.object(trace_handler, "_node_execution_trace") as mock_method:
info = make_node_info()
trace_handler.trace(info)
mock_method.assert_called_once_with(info)
def test_dispatches_moderation_trace(self, trace_handler):
def test_dispatches_moderation_trace(self, trace_handler: EnterpriseOtelTrace):
with patch.object(trace_handler, "_moderation_trace") as mock_method:
info = make_moderation_info()
trace_handler.trace(info)
mock_method.assert_called_once_with(info)
def test_dispatches_suggested_question_trace(self, trace_handler):
def test_dispatches_suggested_question_trace(self, trace_handler: EnterpriseOtelTrace):
with patch.object(trace_handler, "_suggested_question_trace") as mock_method:
info = make_suggested_question_info()
trace_handler.trace(info)
mock_method.assert_called_once_with(info)
def test_dispatches_dataset_retrieval_trace(self, trace_handler):
def test_dispatches_dataset_retrieval_trace(self, trace_handler: EnterpriseOtelTrace):
with patch.object(trace_handler, "_dataset_retrieval_trace") as mock_method:
info = make_dataset_retrieval_info()
trace_handler.trace(info)
mock_method.assert_called_once_with(info)
def test_dispatches_generate_name_trace(self, trace_handler):
def test_dispatches_generate_name_trace(self, trace_handler: EnterpriseOtelTrace):
with patch.object(trace_handler, "_generate_name_trace") as mock_method:
info = make_generate_name_info()
trace_handler.trace(info)
mock_method.assert_called_once_with(info)
def test_dispatches_prompt_generation_trace(self, trace_handler):
def test_dispatches_prompt_generation_trace(self, trace_handler: EnterpriseOtelTrace):
with patch.object(trace_handler, "_prompt_generation_trace") as mock_method:
info = make_prompt_generation_info()
trace_handler.trace(info)
mock_method.assert_called_once_with(info)
def test_draft_node_dispatched_before_node(self, trace_handler):
def test_draft_node_dispatched_before_node(self, trace_handler: EnterpriseOtelTrace):
"""DraftNodeExecutionTrace is a subclass of WorkflowNodeTraceInfo;
it must be dispatched to _draft_node_execution_trace, not _node_execution_trace."""
with (
@@ -450,7 +451,7 @@ class TestTraceDispatcher:
class TestWorkflowTrace:
def test_emits_correct_span_attributes(self, trace_handler, mock_exporter):
def test_emits_correct_span_attributes(self, trace_handler: EnterpriseOtelTrace, mock_exporter):
with patch("enterprise.telemetry.enterprise_trace.emit_telemetry_log") as mock_log:
info = make_workflow_info()
trace_handler._workflow_trace(info)
@@ -465,7 +466,7 @@ class TestWorkflowTrace:
assert attrs["dify.workflow.status"] == "succeeded"
assert attrs["gen_ai.usage.total_tokens"] == 100
def test_span_timing_passed_correctly(self, trace_handler, mock_exporter):
def test_span_timing_passed_correctly(self, trace_handler: EnterpriseOtelTrace, mock_exporter):
with patch("enterprise.telemetry.enterprise_trace.emit_telemetry_log"):
info = make_workflow_info()
trace_handler._workflow_trace(info)
@@ -474,7 +475,7 @@ class TestWorkflowTrace:
assert span_call[1]["start_time"] == _T0
assert span_call[1]["end_time"] == _T1
def test_emits_companion_log_with_event_name(self, trace_handler, mock_exporter):
def test_emits_companion_log_with_event_name(self, trace_handler: EnterpriseOtelTrace, mock_exporter):
with patch("enterprise.telemetry.enterprise_trace.emit_telemetry_log") as mock_log:
trace_handler._workflow_trace(make_workflow_info())
@@ -482,7 +483,7 @@ class TestWorkflowTrace:
assert mock_log.call_args[1]["event_name"] == EnterpriseTelemetryEvent.WORKFLOW_RUN
assert mock_log.call_args[1]["tenant_id"] == "tenant-abc"
def test_companion_log_includes_content_when_enabled(self, trace_handler, mock_exporter):
def test_companion_log_includes_content_when_enabled(self, trace_handler: EnterpriseOtelTrace, mock_exporter):
mock_exporter.include_content = True
with patch("enterprise.telemetry.enterprise_trace.emit_telemetry_log") as mock_log:
trace_handler._workflow_trace(make_workflow_info())
@@ -491,7 +492,7 @@ class TestWorkflowTrace:
assert log_attrs["dify.workflow.inputs"] == json.dumps({"query": "hello"})
assert log_attrs["dify.workflow.outputs"] == json.dumps({"answer": "world"})
def test_companion_log_uses_ref_when_content_disabled(self, trace_handler, mock_exporter):
def test_companion_log_uses_ref_when_content_disabled(self, trace_handler: EnterpriseOtelTrace, mock_exporter):
mock_exporter.include_content = False
with patch("enterprise.telemetry.enterprise_trace.emit_telemetry_log") as mock_log:
trace_handler._workflow_trace(make_workflow_info())
@@ -500,7 +501,7 @@ class TestWorkflowTrace:
assert log_attrs["dify.workflow.inputs"].startswith("ref:workflow_run_id=")
assert log_attrs["dify.workflow.outputs"].startswith("ref:workflow_run_id=")
def test_increments_token_counter(self, trace_handler, mock_exporter):
def test_increments_token_counter(self, trace_handler: EnterpriseOtelTrace, mock_exporter):
with patch("enterprise.telemetry.enterprise_trace.emit_telemetry_log"):
trace_handler._workflow_trace(make_workflow_info())
@@ -510,7 +511,7 @@ class TestWorkflowTrace:
assert len(token_calls) == 1
assert token_calls[0][0][1] == 100
def test_increments_input_and_output_token_counters(self, trace_handler, mock_exporter):
def test_increments_input_and_output_token_counters(self, trace_handler: EnterpriseOtelTrace, mock_exporter):
with patch("enterprise.telemetry.enterprise_trace.emit_telemetry_log"):
trace_handler._workflow_trace(make_workflow_info())
@@ -519,7 +520,7 @@ class TestWorkflowTrace:
assert EnterpriseTelemetryCounter.INPUT_TOKENS in counter_names
assert EnterpriseTelemetryCounter.OUTPUT_TOKENS in counter_names
def test_no_input_token_counter_when_prompt_tokens_zero(self, trace_handler, mock_exporter):
def test_no_input_token_counter_when_prompt_tokens_zero(self, trace_handler: EnterpriseOtelTrace, mock_exporter):
with patch("enterprise.telemetry.enterprise_trace.emit_telemetry_log"):
info = make_workflow_info(prompt_tokens=0)
trace_handler._workflow_trace(info)
@@ -528,7 +529,7 @@ class TestWorkflowTrace:
counter_names = [c[0][0] for c in all_calls]
assert EnterpriseTelemetryCounter.INPUT_TOKENS not in counter_names
def test_records_workflow_duration_histogram(self, trace_handler, mock_exporter):
def test_records_workflow_duration_histogram(self, trace_handler: EnterpriseOtelTrace, mock_exporter):
with patch("enterprise.telemetry.enterprise_trace.emit_telemetry_log"):
trace_handler._workflow_trace(make_workflow_info())
@@ -537,7 +538,9 @@ class TestWorkflowTrace:
assert hist_call[0][0] == EnterpriseTelemetryHistogram.WORKFLOW_DURATION
assert hist_call[0][1] == pytest.approx(5.0)
def test_duration_falls_back_to_elapsed_time_when_timestamps_missing(self, trace_handler, mock_exporter):
def test_duration_falls_back_to_elapsed_time_when_timestamps_missing(
self, trace_handler: EnterpriseOtelTrace, mock_exporter
):
with patch("enterprise.telemetry.enterprise_trace.emit_telemetry_log"):
info = make_workflow_info(start_time=None, end_time=None, workflow_run_elapsed_time=7.3)
trace_handler._workflow_trace(info)
@@ -545,7 +548,7 @@ class TestWorkflowTrace:
hist_call = mock_exporter.record_histogram.call_args
assert hist_call[0][1] == pytest.approx(7.3)
def test_duration_defaults_to_zero_when_no_timing(self, trace_handler, mock_exporter):
def test_duration_defaults_to_zero_when_no_timing(self, trace_handler: EnterpriseOtelTrace, mock_exporter):
with patch("enterprise.telemetry.enterprise_trace.emit_telemetry_log"):
info = make_workflow_info(start_time=None, end_time=None, workflow_run_elapsed_time=0)
trace_handler._workflow_trace(info)
@@ -553,7 +556,7 @@ class TestWorkflowTrace:
hist_call = mock_exporter.record_histogram.call_args
assert hist_call[0][1] == pytest.approx(0.0)
def test_error_path_increments_error_counter(self, trace_handler, mock_exporter):
def test_error_path_increments_error_counter(self, trace_handler: EnterpriseOtelTrace, mock_exporter):
with patch("enterprise.telemetry.enterprise_trace.emit_telemetry_log"):
info = make_workflow_info(error="Something went wrong", workflow_run_status="failed")
trace_handler._workflow_trace(info)
@@ -563,7 +566,7 @@ class TestWorkflowTrace:
]
assert len(error_calls) == 1
def test_no_error_counter_on_success(self, trace_handler, mock_exporter):
def test_no_error_counter_on_success(self, trace_handler: EnterpriseOtelTrace, mock_exporter):
with patch("enterprise.telemetry.enterprise_trace.emit_telemetry_log"):
trace_handler._workflow_trace(make_workflow_info())
@@ -572,7 +575,7 @@ class TestWorkflowTrace:
]
assert len(error_calls) == 0
def test_parent_trace_context_injected_into_span_attrs(self, trace_handler, mock_exporter):
def test_parent_trace_context_injected_into_span_attrs(self, trace_handler: EnterpriseOtelTrace, mock_exporter):
with patch("enterprise.telemetry.enterprise_trace.emit_telemetry_log"):
info = make_workflow_info(
metadata={
@@ -601,14 +604,14 @@ class TestWorkflowTrace:
class TestNodeExecutionTrace:
def test_emits_span_with_node_execution_span_name(self, trace_handler, mock_exporter):
def test_emits_span_with_node_execution_span_name(self, trace_handler: EnterpriseOtelTrace, mock_exporter):
with patch("enterprise.telemetry.enterprise_trace.emit_telemetry_log"):
trace_handler._node_execution_trace(make_node_info())
span_call = mock_exporter.export_span.call_args
assert span_call[0][0] == EnterpriseTelemetrySpan.NODE_EXECUTION
def test_span_contains_core_node_attributes(self, trace_handler, mock_exporter):
def test_span_contains_core_node_attributes(self, trace_handler: EnterpriseOtelTrace, mock_exporter):
with patch("enterprise.telemetry.enterprise_trace.emit_telemetry_log"):
trace_handler._node_execution_trace(make_node_info())
@@ -620,7 +623,7 @@ class TestNodeExecutionTrace:
assert attrs["gen_ai.request.model"] == "gpt-4"
assert attrs["gen_ai.provider.name"] == "openai"
def test_increments_token_counters_when_tokens_present(self, trace_handler, mock_exporter):
def test_increments_token_counters_when_tokens_present(self, trace_handler: EnterpriseOtelTrace, mock_exporter):
with patch("enterprise.telemetry.enterprise_trace.emit_telemetry_log"):
trace_handler._node_execution_trace(make_node_info())
@@ -629,7 +632,7 @@ class TestNodeExecutionTrace:
assert EnterpriseTelemetryCounter.INPUT_TOKENS in counter_names
assert EnterpriseTelemetryCounter.OUTPUT_TOKENS in counter_names
def test_no_token_counters_when_total_tokens_zero(self, trace_handler, mock_exporter):
def test_no_token_counters_when_total_tokens_zero(self, trace_handler: EnterpriseOtelTrace, mock_exporter):
with patch("enterprise.telemetry.enterprise_trace.emit_telemetry_log"):
trace_handler._node_execution_trace(make_node_info(total_tokens=0))
@@ -637,7 +640,7 @@ class TestNodeExecutionTrace:
assert EnterpriseTelemetryCounter.TOKENS not in counter_names
assert EnterpriseTelemetryCounter.INPUT_TOKENS not in counter_names
def test_records_node_duration_histogram(self, trace_handler, mock_exporter):
def test_records_node_duration_histogram(self, trace_handler: EnterpriseOtelTrace, mock_exporter):
with patch("enterprise.telemetry.enterprise_trace.emit_telemetry_log"):
trace_handler._node_execution_trace(make_node_info())
@@ -645,7 +648,7 @@ class TestNodeExecutionTrace:
assert hist_call[0][0] == EnterpriseTelemetryHistogram.NODE_DURATION
assert hist_call[0][1] == pytest.approx(2.5)
def test_error_path_increments_error_counter(self, trace_handler, mock_exporter):
def test_error_path_increments_error_counter(self, trace_handler: EnterpriseOtelTrace, mock_exporter):
with patch("enterprise.telemetry.enterprise_trace.emit_telemetry_log"):
trace_handler._node_execution_trace(make_node_info(error="Node failed", status="failed"))
@@ -654,14 +657,16 @@ class TestNodeExecutionTrace:
]
assert len(error_calls) == 1
def test_emits_companion_log_with_span_name_as_event(self, trace_handler, mock_exporter):
def test_emits_companion_log_with_span_name_as_event(self, trace_handler: EnterpriseOtelTrace, mock_exporter):
with patch("enterprise.telemetry.enterprise_trace.emit_telemetry_log") as mock_log:
trace_handler._node_execution_trace(make_node_info())
mock_log.assert_called_once()
assert mock_log.call_args[1]["event_name"] == EnterpriseTelemetrySpan.NODE_EXECUTION.value
def test_plugin_name_added_to_duration_labels_for_tool_node(self, trace_handler, mock_exporter):
def test_plugin_name_added_to_duration_labels_for_tool_node(
self, trace_handler: EnterpriseOtelTrace, mock_exporter
):
with patch("enterprise.telemetry.enterprise_trace.emit_telemetry_log"):
info = make_node_info(
node_type="tool",
@@ -677,7 +682,7 @@ class TestNodeExecutionTrace:
duration_labels = hist_call[0][2]
assert duration_labels.get("plugin_name") == "my-plugin"
def test_plugin_name_not_added_for_non_tool_node(self, trace_handler, mock_exporter):
def test_plugin_name_not_added_for_non_tool_node(self, trace_handler: EnterpriseOtelTrace, mock_exporter):
with patch("enterprise.telemetry.enterprise_trace.emit_telemetry_log"):
info = make_node_info(
node_type="llm",
@@ -693,7 +698,9 @@ class TestNodeExecutionTrace:
duration_labels = hist_call[0][2]
assert "plugin_name" not in duration_labels
def test_companion_log_inputs_use_ref_when_content_disabled(self, trace_handler, mock_exporter):
def test_companion_log_inputs_use_ref_when_content_disabled(
self, trace_handler: EnterpriseOtelTrace, mock_exporter
):
mock_exporter.include_content = False
with patch("enterprise.telemetry.enterprise_trace.emit_telemetry_log") as mock_log:
trace_handler._node_execution_trace(
@@ -711,14 +718,14 @@ class TestNodeExecutionTrace:
class TestDraftNodeExecutionTrace:
def test_uses_draft_span_name(self, trace_handler, mock_exporter):
def test_uses_draft_span_name(self, trace_handler: EnterpriseOtelTrace, mock_exporter):
with patch("enterprise.telemetry.enterprise_trace.emit_telemetry_log"):
trace_handler._draft_node_execution_trace(make_draft_node_info())
span_call = mock_exporter.export_span.call_args
assert span_call[0][0] == EnterpriseTelemetrySpan.DRAFT_NODE_EXECUTION
def test_correlation_id_is_node_execution_id(self, trace_handler, mock_exporter):
def test_correlation_id_is_node_execution_id(self, trace_handler: EnterpriseOtelTrace, mock_exporter):
with patch("enterprise.telemetry.enterprise_trace.emit_telemetry_log"):
info = make_draft_node_info()
trace_handler._draft_node_execution_trace(info)
@@ -726,7 +733,7 @@ class TestDraftNodeExecutionTrace:
span_call = mock_exporter.export_span.call_args
assert span_call[1]["correlation_id"] == "ne-draft-001"
def test_trace_correlation_override_is_workflow_run_id(self, trace_handler, mock_exporter):
def test_trace_correlation_override_is_workflow_run_id(self, trace_handler: EnterpriseOtelTrace, mock_exporter):
with patch("enterprise.telemetry.enterprise_trace.emit_telemetry_log"):
info = make_draft_node_info()
trace_handler._draft_node_execution_trace(info)
@@ -734,7 +741,7 @@ class TestDraftNodeExecutionTrace:
span_call = mock_exporter.export_span.call_args
assert span_call[1]["trace_correlation_override"] == "run-draft-001"
def test_companion_log_uses_draft_span_name(self, trace_handler, mock_exporter):
def test_companion_log_uses_draft_span_name(self, trace_handler: EnterpriseOtelTrace, mock_exporter):
with patch("enterprise.telemetry.enterprise_trace.emit_telemetry_log") as mock_log:
trace_handler._draft_node_execution_trace(make_draft_node_info())
@@ -747,34 +754,36 @@ class TestDraftNodeExecutionTrace:
class TestMessageTrace:
def test_emits_event_with_correct_name(self, trace_handler, mock_exporter):
def test_emits_event_with_correct_name(self, trace_handler: EnterpriseOtelTrace, mock_exporter):
with patch("enterprise.telemetry.enterprise_trace.emit_metric_only_event") as mock_emit:
trace_handler._message_trace(make_message_info())
mock_emit.assert_called_once()
assert mock_emit.call_args[1]["event_name"] == EnterpriseTelemetryEvent.MESSAGE_RUN
def test_emits_correct_tenant_and_user(self, trace_handler, mock_exporter):
def test_emits_correct_tenant_and_user(self, trace_handler: EnterpriseOtelTrace, mock_exporter):
with patch("enterprise.telemetry.enterprise_trace.emit_metric_only_event") as mock_emit:
trace_handler._message_trace(make_message_info())
assert mock_emit.call_args[1]["tenant_id"] == "tenant-abc"
def test_duration_computed_from_timestamps(self, trace_handler, mock_exporter):
def test_duration_computed_from_timestamps(self, trace_handler: EnterpriseOtelTrace, mock_exporter):
with patch("enterprise.telemetry.enterprise_trace.emit_metric_only_event") as mock_emit:
trace_handler._message_trace(make_message_info())
attrs = mock_emit.call_args[1]["attributes"]
assert attrs["dify.message.duration"] == pytest.approx(5.0)
def test_no_duration_when_timestamps_missing(self, trace_handler, mock_exporter):
def test_no_duration_when_timestamps_missing(self, trace_handler: EnterpriseOtelTrace, mock_exporter):
with patch("enterprise.telemetry.enterprise_trace.emit_metric_only_event") as mock_emit:
trace_handler._message_trace(make_message_info(start_time=None, end_time=None))
attrs = mock_emit.call_args[1]["attributes"]
assert "dify.message.duration" not in attrs
def test_records_duration_histogram_when_timestamps_present(self, trace_handler, mock_exporter):
def test_records_duration_histogram_when_timestamps_present(
self, trace_handler: EnterpriseOtelTrace, mock_exporter
):
with patch("enterprise.telemetry.enterprise_trace.emit_metric_only_event"):
trace_handler._message_trace(make_message_info())
@@ -786,14 +795,14 @@ class TestMessageTrace:
assert len(hist_calls) == 1
assert hist_calls[0][0][1] == pytest.approx(5.0)
def test_no_duration_histogram_when_timestamps_missing(self, trace_handler, mock_exporter):
def test_no_duration_histogram_when_timestamps_missing(self, trace_handler: EnterpriseOtelTrace, mock_exporter):
with patch("enterprise.telemetry.enterprise_trace.emit_metric_only_event"):
trace_handler._message_trace(make_message_info(start_time=None, end_time=None))
hist_names = [c[0][0] for c in mock_exporter.record_histogram.call_args_list]
assert EnterpriseTelemetryHistogram.MESSAGE_DURATION not in hist_names
def test_records_ttft_histogram_when_present(self, trace_handler, mock_exporter):
def test_records_ttft_histogram_when_present(self, trace_handler: EnterpriseOtelTrace, mock_exporter):
with patch("enterprise.telemetry.enterprise_trace.emit_metric_only_event"):
trace_handler._message_trace(make_message_info(gen_ai_server_time_to_first_token=0.42))
@@ -805,14 +814,14 @@ class TestMessageTrace:
assert len(ttft_calls) == 1
assert ttft_calls[0][0][1] == pytest.approx(0.42)
def test_no_ttft_histogram_when_not_present(self, trace_handler, mock_exporter):
def test_no_ttft_histogram_when_not_present(self, trace_handler: EnterpriseOtelTrace, mock_exporter):
with patch("enterprise.telemetry.enterprise_trace.emit_metric_only_event"):
trace_handler._message_trace(make_message_info(gen_ai_server_time_to_first_token=None))
hist_names = [c[0][0] for c in mock_exporter.record_histogram.call_args_list]
assert EnterpriseTelemetryHistogram.MESSAGE_TTFT not in hist_names
def test_increments_token_counters(self, trace_handler, mock_exporter):
def test_increments_token_counters(self, trace_handler: EnterpriseOtelTrace, mock_exporter):
with patch("enterprise.telemetry.enterprise_trace.emit_metric_only_event"):
trace_handler._message_trace(make_message_info())
@@ -821,7 +830,7 @@ class TestMessageTrace:
assert EnterpriseTelemetryCounter.INPUT_TOKENS in counter_names
assert EnterpriseTelemetryCounter.OUTPUT_TOKENS in counter_names
def test_error_path_increments_error_counter(self, trace_handler, mock_exporter):
def test_error_path_increments_error_counter(self, trace_handler: EnterpriseOtelTrace, mock_exporter):
with patch("enterprise.telemetry.enterprise_trace.emit_metric_only_event"):
trace_handler._message_trace(make_message_info(error="LLM failed"))
@@ -830,7 +839,7 @@ class TestMessageTrace:
]
assert len(error_calls) == 1
def test_inputs_and_outputs_gated_by_include_content(self, trace_handler, mock_exporter):
def test_inputs_and_outputs_gated_by_include_content(self, trace_handler: EnterpriseOtelTrace, mock_exporter):
mock_exporter.include_content = False
with patch("enterprise.telemetry.enterprise_trace.emit_metric_only_event") as mock_emit:
trace_handler._message_trace(make_message_info())
@@ -846,27 +855,27 @@ class TestMessageTrace:
class TestToolTrace:
def test_emits_event_with_correct_name(self, trace_handler, mock_exporter):
def test_emits_event_with_correct_name(self, trace_handler: EnterpriseOtelTrace, mock_exporter):
with patch("enterprise.telemetry.enterprise_trace.emit_metric_only_event") as mock_emit:
trace_handler._tool_trace(make_tool_info())
assert mock_emit.call_args[1]["event_name"] == EnterpriseTelemetryEvent.TOOL_EXECUTION
def test_status_is_succeeded_on_success(self, trace_handler, mock_exporter):
def test_status_is_succeeded_on_success(self, trace_handler: EnterpriseOtelTrace, mock_exporter):
with patch("enterprise.telemetry.enterprise_trace.emit_metric_only_event") as mock_emit:
trace_handler._tool_trace(make_tool_info())
attrs = mock_emit.call_args[1]["attributes"]
assert attrs["dify.tool.status"] == "succeeded"
def test_status_is_failed_on_error(self, trace_handler, mock_exporter):
def test_status_is_failed_on_error(self, trace_handler: EnterpriseOtelTrace, mock_exporter):
with patch("enterprise.telemetry.enterprise_trace.emit_metric_only_event") as mock_emit:
trace_handler._tool_trace(make_tool_info(error="Tool error"))
attrs = mock_emit.call_args[1]["attributes"]
assert attrs["dify.tool.status"] == "failed"
def test_records_tool_duration_histogram(self, trace_handler, mock_exporter):
def test_records_tool_duration_histogram(self, trace_handler: EnterpriseOtelTrace, mock_exporter):
with patch("enterprise.telemetry.enterprise_trace.emit_metric_only_event"):
trace_handler._tool_trace(make_tool_info())
@@ -874,7 +883,7 @@ class TestToolTrace:
assert hist_call[0][0] == EnterpriseTelemetryHistogram.TOOL_DURATION
assert hist_call[0][1] == pytest.approx(1.5)
def test_error_increments_error_counter(self, trace_handler, mock_exporter):
def test_error_increments_error_counter(self, trace_handler: EnterpriseOtelTrace, mock_exporter):
with patch("enterprise.telemetry.enterprise_trace.emit_metric_only_event"):
trace_handler._tool_trace(make_tool_info(error="Tool crashed"))
@@ -883,7 +892,7 @@ class TestToolTrace:
]
assert len(error_calls) == 1
def test_inputs_and_outputs_gated_by_include_content(self, trace_handler, mock_exporter):
def test_inputs_and_outputs_gated_by_include_content(self, trace_handler: EnterpriseOtelTrace, mock_exporter):
mock_exporter.include_content = False
with patch("enterprise.telemetry.enterprise_trace.emit_metric_only_event") as mock_emit:
trace_handler._tool_trace(make_tool_info())
@@ -892,7 +901,7 @@ class TestToolTrace:
assert attrs["dify.tool.inputs"].startswith("ref:message_id=")
assert attrs["dify.tool.outputs"].startswith("ref:message_id=")
def test_inputs_present_when_include_content_true(self, trace_handler, mock_exporter):
def test_inputs_present_when_include_content_true(self, trace_handler: EnterpriseOtelTrace, mock_exporter):
mock_exporter.include_content = True
with patch("enterprise.telemetry.enterprise_trace.emit_metric_only_event") as mock_emit:
trace_handler._tool_trace(make_tool_info())
@@ -901,7 +910,7 @@ class TestToolTrace:
assert attrs["dify.tool.inputs"] == json.dumps({"query": "test"})
assert attrs["dify.tool.outputs"] == "search results"
def test_increments_requests_counter(self, trace_handler, mock_exporter):
def test_increments_requests_counter(self, trace_handler: EnterpriseOtelTrace, mock_exporter):
with patch("enterprise.telemetry.enterprise_trace.emit_metric_only_event"):
trace_handler._tool_trace(make_tool_info())
@@ -918,27 +927,27 @@ class TestToolTrace:
class TestModerationTrace:
def test_emits_event_with_correct_name(self, trace_handler, mock_exporter):
def test_emits_event_with_correct_name(self, trace_handler: EnterpriseOtelTrace, mock_exporter):
with patch("enterprise.telemetry.enterprise_trace.emit_metric_only_event") as mock_emit:
trace_handler._moderation_trace(make_moderation_info())
assert mock_emit.call_args[1]["event_name"] == EnterpriseTelemetryEvent.MODERATION_CHECK
def test_flagged_true_sets_attribute(self, trace_handler, mock_exporter):
def test_flagged_true_sets_attribute(self, trace_handler: EnterpriseOtelTrace, mock_exporter):
with patch("enterprise.telemetry.enterprise_trace.emit_metric_only_event") as mock_emit:
trace_handler._moderation_trace(make_moderation_info(flagged=True))
attrs = mock_emit.call_args[1]["attributes"]
assert attrs["dify.moderation.flagged"] is True
def test_flagged_false_sets_attribute(self, trace_handler, mock_exporter):
def test_flagged_false_sets_attribute(self, trace_handler: EnterpriseOtelTrace, mock_exporter):
with patch("enterprise.telemetry.enterprise_trace.emit_metric_only_event") as mock_emit:
trace_handler._moderation_trace(make_moderation_info(flagged=False))
attrs = mock_emit.call_args[1]["attributes"]
assert attrs["dify.moderation.flagged"] is False
def test_query_gated_by_include_content(self, trace_handler, mock_exporter):
def test_query_gated_by_include_content(self, trace_handler: EnterpriseOtelTrace, mock_exporter):
mock_exporter.include_content = False
with patch("enterprise.telemetry.enterprise_trace.emit_metric_only_event") as mock_emit:
trace_handler._moderation_trace(make_moderation_info())
@@ -946,7 +955,7 @@ class TestModerationTrace:
attrs = mock_emit.call_args[1]["attributes"]
assert attrs["dify.moderation.query"].startswith("ref:message_id=")
def test_query_present_when_include_content_true(self, trace_handler, mock_exporter):
def test_query_present_when_include_content_true(self, trace_handler: EnterpriseOtelTrace, mock_exporter):
mock_exporter.include_content = True
with patch("enterprise.telemetry.enterprise_trace.emit_metric_only_event") as mock_emit:
trace_handler._moderation_trace(make_moderation_info())
@@ -954,7 +963,7 @@ class TestModerationTrace:
attrs = mock_emit.call_args[1]["attributes"]
assert attrs["dify.moderation.query"] == "is this ok?"
def test_increments_requests_counter(self, trace_handler, mock_exporter):
def test_increments_requests_counter(self, trace_handler: EnterpriseOtelTrace, mock_exporter):
with patch("enterprise.telemetry.enterprise_trace.emit_metric_only_event"):
trace_handler._moderation_trace(make_moderation_info())
@@ -971,48 +980,48 @@ class TestModerationTrace:
class TestSuggestedQuestionTrace:
def test_emits_event_with_correct_name(self, trace_handler, mock_exporter):
def test_emits_event_with_correct_name(self, trace_handler: EnterpriseOtelTrace, mock_exporter):
with patch("enterprise.telemetry.enterprise_trace.emit_metric_only_event") as mock_emit:
trace_handler._suggested_question_trace(make_suggested_question_info())
assert mock_emit.call_args[1]["event_name"] == EnterpriseTelemetryEvent.SUGGESTED_QUESTION_GENERATION
def test_duration_computed_from_timestamps(self, trace_handler, mock_exporter):
def test_duration_computed_from_timestamps(self, trace_handler: EnterpriseOtelTrace, mock_exporter):
with patch("enterprise.telemetry.enterprise_trace.emit_metric_only_event") as mock_emit:
trace_handler._suggested_question_trace(make_suggested_question_info())
attrs = mock_emit.call_args[1]["attributes"]
assert attrs["dify.suggested_question.duration"] == pytest.approx(5.0)
def test_duration_is_none_when_timestamps_missing(self, trace_handler, mock_exporter):
def test_duration_is_none_when_timestamps_missing(self, trace_handler: EnterpriseOtelTrace, mock_exporter):
with patch("enterprise.telemetry.enterprise_trace.emit_metric_only_event") as mock_emit:
trace_handler._suggested_question_trace(make_suggested_question_info(start_time=None, end_time=None))
attrs = mock_emit.call_args[1]["attributes"]
assert attrs["dify.suggested_question.duration"] is None
def test_status_is_failed_when_error_present(self, trace_handler, mock_exporter):
def test_status_is_failed_when_error_present(self, trace_handler: EnterpriseOtelTrace, mock_exporter):
with patch("enterprise.telemetry.enterprise_trace.emit_metric_only_event") as mock_emit:
trace_handler._suggested_question_trace(make_suggested_question_info(error="Generation failed"))
attrs = mock_emit.call_args[1]["attributes"]
assert attrs["dify.suggested_question.status"] == "failed"
def test_status_falls_back_to_succeeded_when_no_error(self, trace_handler, mock_exporter):
def test_status_falls_back_to_succeeded_when_no_error(self, trace_handler: EnterpriseOtelTrace, mock_exporter):
with patch("enterprise.telemetry.enterprise_trace.emit_metric_only_event") as mock_emit:
trace_handler._suggested_question_trace(make_suggested_question_info(status=None, error=None))
attrs = mock_emit.call_args[1]["attributes"]
assert attrs["dify.suggested_question.status"] == "succeeded"
def test_question_count_attribute(self, trace_handler, mock_exporter):
def test_question_count_attribute(self, trace_handler: EnterpriseOtelTrace, mock_exporter):
with patch("enterprise.telemetry.enterprise_trace.emit_metric_only_event") as mock_emit:
trace_handler._suggested_question_trace(make_suggested_question_info())
attrs = mock_emit.call_args[1]["attributes"]
assert attrs["dify.suggested_question.count"] == 2
def test_questions_gated_by_include_content(self, trace_handler, mock_exporter):
def test_questions_gated_by_include_content(self, trace_handler: EnterpriseOtelTrace, mock_exporter):
mock_exporter.include_content = False
with patch("enterprise.telemetry.enterprise_trace.emit_metric_only_event") as mock_emit:
trace_handler._suggested_question_trace(make_suggested_question_info())
@@ -1020,7 +1029,7 @@ class TestSuggestedQuestionTrace:
attrs = mock_emit.call_args[1]["attributes"]
assert attrs["dify.suggested_question.questions"].startswith("ref:message_id=")
def test_increments_requests_counter(self, trace_handler, mock_exporter):
def test_increments_requests_counter(self, trace_handler: EnterpriseOtelTrace, mock_exporter):
with patch("enterprise.telemetry.enterprise_trace.emit_metric_only_event"):
trace_handler._suggested_question_trace(make_suggested_question_info())
@@ -1037,48 +1046,48 @@ class TestSuggestedQuestionTrace:
class TestDatasetRetrievalTrace:
def test_emits_event_with_correct_name(self, trace_handler, mock_exporter):
def test_emits_event_with_correct_name(self, trace_handler: EnterpriseOtelTrace, mock_exporter):
with patch("enterprise.telemetry.enterprise_trace.emit_metric_only_event") as mock_emit:
trace_handler._dataset_retrieval_trace(make_dataset_retrieval_info())
assert mock_emit.call_args[1]["event_name"] == EnterpriseTelemetryEvent.DATASET_RETRIEVAL
def test_document_count_attribute(self, trace_handler, mock_exporter):
def test_document_count_attribute(self, trace_handler: EnterpriseOtelTrace, mock_exporter):
with patch("enterprise.telemetry.enterprise_trace.emit_metric_only_event") as mock_emit:
trace_handler._dataset_retrieval_trace(make_dataset_retrieval_info())
attrs = mock_emit.call_args[1]["attributes"]
assert attrs["dify.retrieval.document_count"] == 1
def test_dataset_ids_extracted(self, trace_handler, mock_exporter):
def test_dataset_ids_extracted(self, trace_handler: EnterpriseOtelTrace, mock_exporter):
with patch("enterprise.telemetry.enterprise_trace.emit_metric_only_event") as mock_emit:
trace_handler._dataset_retrieval_trace(make_dataset_retrieval_info())
attrs = mock_emit.call_args[1]["attributes"]
assert "ds-001" in attrs["dify.dataset.id"]
def test_empty_documents_has_zero_count(self, trace_handler, mock_exporter):
def test_empty_documents_has_zero_count(self, trace_handler: EnterpriseOtelTrace, mock_exporter):
with patch("enterprise.telemetry.enterprise_trace.emit_metric_only_event") as mock_emit:
trace_handler._dataset_retrieval_trace(make_dataset_retrieval_info(documents=[]))
attrs = mock_emit.call_args[1]["attributes"]
assert attrs["dify.retrieval.document_count"] == 0
def test_status_succeeded_when_no_error(self, trace_handler, mock_exporter):
def test_status_succeeded_when_no_error(self, trace_handler: EnterpriseOtelTrace, mock_exporter):
with patch("enterprise.telemetry.enterprise_trace.emit_metric_only_event") as mock_emit:
trace_handler._dataset_retrieval_trace(make_dataset_retrieval_info())
attrs = mock_emit.call_args[1]["attributes"]
assert attrs["dify.retrieval.status"] == "succeeded"
def test_status_failed_when_error_present(self, trace_handler, mock_exporter):
def test_status_failed_when_error_present(self, trace_handler: EnterpriseOtelTrace, mock_exporter):
with patch("enterprise.telemetry.enterprise_trace.emit_metric_only_event") as mock_emit:
trace_handler._dataset_retrieval_trace(make_dataset_retrieval_info(error="DB error"))
attrs = mock_emit.call_args[1]["attributes"]
assert attrs["dify.retrieval.status"] == "failed"
def test_embedding_model_attributes_set_when_present(self, trace_handler, mock_exporter):
def test_embedding_model_attributes_set_when_present(self, trace_handler: EnterpriseOtelTrace, mock_exporter):
with patch("enterprise.telemetry.enterprise_trace.emit_metric_only_event") as mock_emit:
trace_handler._dataset_retrieval_trace(make_dataset_retrieval_info())
@@ -1086,7 +1095,7 @@ class TestDatasetRetrievalTrace:
assert "dify.dataset.embedding_providers" in attrs
assert "dify.dataset.embedding_models" in attrs
def test_no_embedding_model_attributes_when_not_provided(self, trace_handler, mock_exporter):
def test_no_embedding_model_attributes_when_not_provided(self, trace_handler: EnterpriseOtelTrace, mock_exporter):
with patch("enterprise.telemetry.enterprise_trace.emit_metric_only_event") as mock_emit:
trace_handler._dataset_retrieval_trace(
make_dataset_retrieval_info(metadata={"app_id": "app-001", "tenant_id": "tenant-abc"})
@@ -1096,7 +1105,7 @@ class TestDatasetRetrievalTrace:
assert "dify.dataset.embedding_providers" not in attrs
assert "dify.dataset.embedding_models" not in attrs
def test_rerank_attributes_set_when_present(self, trace_handler, mock_exporter):
def test_rerank_attributes_set_when_present(self, trace_handler: EnterpriseOtelTrace, mock_exporter):
with patch("enterprise.telemetry.enterprise_trace.emit_metric_only_event") as mock_emit:
trace_handler._dataset_retrieval_trace(
make_dataset_retrieval_info(
@@ -1113,7 +1122,7 @@ class TestDatasetRetrievalTrace:
assert attrs["dify.retrieval.rerank_provider"] == "cohere"
assert attrs["dify.retrieval.rerank_model"] == "rerank-english"
def test_no_rerank_attributes_when_not_present(self, trace_handler, mock_exporter):
def test_no_rerank_attributes_when_not_present(self, trace_handler: EnterpriseOtelTrace, mock_exporter):
with patch("enterprise.telemetry.enterprise_trace.emit_metric_only_event") as mock_emit:
trace_handler._dataset_retrieval_trace(
make_dataset_retrieval_info(metadata={"app_id": "app-001", "tenant_id": "tenant-abc"})
@@ -1123,7 +1132,7 @@ class TestDatasetRetrievalTrace:
assert "dify.retrieval.rerank_provider" not in attrs
assert "dify.retrieval.rerank_model" not in attrs
def test_dataset_retrieval_counter_incremented_per_dataset(self, trace_handler, mock_exporter):
def test_dataset_retrieval_counter_incremented_per_dataset(self, trace_handler: EnterpriseOtelTrace, mock_exporter):
with patch("enterprise.telemetry.enterprise_trace.emit_metric_only_event"):
trace_handler._dataset_retrieval_trace(make_dataset_retrieval_info())
@@ -1135,7 +1144,7 @@ class TestDatasetRetrievalTrace:
assert len(ds_calls) == 1
assert ds_calls[0][0][2]["dataset_id"] == "ds-001"
def test_no_dataset_retrieval_counter_when_no_documents(self, trace_handler, mock_exporter):
def test_no_dataset_retrieval_counter_when_no_documents(self, trace_handler: EnterpriseOtelTrace, mock_exporter):
with patch("enterprise.telemetry.enterprise_trace.emit_metric_only_event"):
trace_handler._dataset_retrieval_trace(make_dataset_retrieval_info(documents=[]))
@@ -1146,7 +1155,7 @@ class TestDatasetRetrievalTrace:
]
assert len(ds_calls) == 0
def test_query_gated_by_include_content(self, trace_handler, mock_exporter):
def test_query_gated_by_include_content(self, trace_handler: EnterpriseOtelTrace, mock_exporter):
mock_exporter.include_content = False
with patch("enterprise.telemetry.enterprise_trace.emit_metric_only_event") as mock_emit:
trace_handler._dataset_retrieval_trace(make_dataset_retrieval_info())
@@ -1161,34 +1170,34 @@ class TestDatasetRetrievalTrace:
class TestGenerateNameTrace:
def test_emits_event_with_correct_name(self, trace_handler, mock_exporter):
def test_emits_event_with_correct_name(self, trace_handler: EnterpriseOtelTrace, mock_exporter):
with patch("enterprise.telemetry.enterprise_trace.emit_metric_only_event") as mock_emit:
trace_handler._generate_name_trace(make_generate_name_info())
assert mock_emit.call_args[1]["event_name"] == EnterpriseTelemetryEvent.GENERATE_NAME_EXECUTION
def test_duration_computed_from_timestamps(self, trace_handler, mock_exporter):
def test_duration_computed_from_timestamps(self, trace_handler: EnterpriseOtelTrace, mock_exporter):
with patch("enterprise.telemetry.enterprise_trace.emit_metric_only_event") as mock_emit:
trace_handler._generate_name_trace(make_generate_name_info())
attrs = mock_emit.call_args[1]["attributes"]
assert attrs["dify.generate_name.duration"] == pytest.approx(5.0)
def test_no_duration_when_timestamps_missing(self, trace_handler, mock_exporter):
def test_no_duration_when_timestamps_missing(self, trace_handler: EnterpriseOtelTrace, mock_exporter):
with patch("enterprise.telemetry.enterprise_trace.emit_metric_only_event") as mock_emit:
trace_handler._generate_name_trace(make_generate_name_info(start_time=None, end_time=None))
attrs = mock_emit.call_args[1]["attributes"]
assert attrs["dify.generate_name.duration"] is None
def test_status_succeeded_on_success(self, trace_handler, mock_exporter):
def test_status_succeeded_on_success(self, trace_handler: EnterpriseOtelTrace, mock_exporter):
with patch("enterprise.telemetry.enterprise_trace.emit_metric_only_event") as mock_emit:
trace_handler._generate_name_trace(make_generate_name_info())
attrs = mock_emit.call_args[1]["attributes"]
assert attrs["dify.generate_name.status"] == "succeeded"
def test_status_failed_when_metadata_has_error(self, trace_handler, mock_exporter):
def test_status_failed_when_metadata_has_error(self, trace_handler: EnterpriseOtelTrace, mock_exporter):
with patch("enterprise.telemetry.enterprise_trace.emit_metric_only_event") as mock_emit:
trace_handler._generate_name_trace(
make_generate_name_info(
@@ -1203,7 +1212,7 @@ class TestGenerateNameTrace:
attrs = mock_emit.call_args[1]["attributes"]
assert attrs["dify.generate_name.status"] == "failed"
def test_inputs_and_outputs_gated_by_include_content(self, trace_handler, mock_exporter):
def test_inputs_and_outputs_gated_by_include_content(self, trace_handler: EnterpriseOtelTrace, mock_exporter):
mock_exporter.include_content = False
with patch("enterprise.telemetry.enterprise_trace.emit_metric_only_event") as mock_emit:
trace_handler._generate_name_trace(make_generate_name_info())
@@ -1212,7 +1221,7 @@ class TestGenerateNameTrace:
assert attrs["dify.generate_name.inputs"].startswith("ref:conversation_id=")
assert attrs["dify.generate_name.outputs"].startswith("ref:conversation_id=")
def test_increments_requests_counter(self, trace_handler, mock_exporter):
def test_increments_requests_counter(self, trace_handler: EnterpriseOtelTrace, mock_exporter):
with patch("enterprise.telemetry.enterprise_trace.emit_metric_only_event"):
trace_handler._generate_name_trace(make_generate_name_info())
@@ -1229,27 +1238,27 @@ class TestGenerateNameTrace:
class TestPromptGenerationTrace:
def test_emits_event_with_correct_name(self, trace_handler, mock_exporter):
def test_emits_event_with_correct_name(self, trace_handler: EnterpriseOtelTrace, mock_exporter):
with patch("enterprise.telemetry.enterprise_trace.emit_metric_only_event") as mock_emit:
trace_handler._prompt_generation_trace(make_prompt_generation_info())
assert mock_emit.call_args[1]["event_name"] == EnterpriseTelemetryEvent.PROMPT_GENERATION_EXECUTION
def test_status_succeeded_on_success(self, trace_handler, mock_exporter):
def test_status_succeeded_on_success(self, trace_handler: EnterpriseOtelTrace, mock_exporter):
with patch("enterprise.telemetry.enterprise_trace.emit_metric_only_event") as mock_emit:
trace_handler._prompt_generation_trace(make_prompt_generation_info())
attrs = mock_emit.call_args[1]["attributes"]
assert attrs["dify.prompt_generation.status"] == "succeeded"
def test_status_failed_when_error_present(self, trace_handler, mock_exporter):
def test_status_failed_when_error_present(self, trace_handler: EnterpriseOtelTrace, mock_exporter):
with patch("enterprise.telemetry.enterprise_trace.emit_metric_only_event") as mock_emit:
trace_handler._prompt_generation_trace(make_prompt_generation_info(error="Generation error"))
attrs = mock_emit.call_args[1]["attributes"]
assert attrs["dify.prompt_generation.status"] == "failed"
def test_token_counters_incremented(self, trace_handler, mock_exporter):
def test_token_counters_incremented(self, trace_handler: EnterpriseOtelTrace, mock_exporter):
with patch("enterprise.telemetry.enterprise_trace.emit_metric_only_event"):
trace_handler._prompt_generation_trace(make_prompt_generation_info())
@@ -1258,7 +1267,7 @@ class TestPromptGenerationTrace:
assert EnterpriseTelemetryCounter.INPUT_TOKENS in counter_names
assert EnterpriseTelemetryCounter.OUTPUT_TOKENS in counter_names
def test_records_duration_histogram(self, trace_handler, mock_exporter):
def test_records_duration_histogram(self, trace_handler: EnterpriseOtelTrace, mock_exporter):
with patch("enterprise.telemetry.enterprise_trace.emit_metric_only_event"):
trace_handler._prompt_generation_trace(make_prompt_generation_info())
@@ -1270,7 +1279,7 @@ class TestPromptGenerationTrace:
assert len(hist_calls) == 1
assert hist_calls[0][0][1] == pytest.approx(3.2)
def test_total_price_attribute_set_when_present(self, trace_handler, mock_exporter):
def test_total_price_attribute_set_when_present(self, trace_handler: EnterpriseOtelTrace, mock_exporter):
with patch("enterprise.telemetry.enterprise_trace.emit_metric_only_event") as mock_emit:
trace_handler._prompt_generation_trace(make_prompt_generation_info(total_price=0.05, currency="USD"))
@@ -1278,14 +1287,14 @@ class TestPromptGenerationTrace:
assert attrs["dify.prompt_generation.total_price"] == pytest.approx(0.05)
assert attrs["dify.prompt_generation.currency"] == "USD"
def test_no_total_price_attribute_when_none(self, trace_handler, mock_exporter):
def test_no_total_price_attribute_when_none(self, trace_handler: EnterpriseOtelTrace, mock_exporter):
with patch("enterprise.telemetry.enterprise_trace.emit_metric_only_event") as mock_emit:
trace_handler._prompt_generation_trace(make_prompt_generation_info(total_price=None))
attrs = mock_emit.call_args[1]["attributes"]
assert "dify.prompt_generation.total_price" not in attrs
def test_error_increments_error_counter(self, trace_handler, mock_exporter):
def test_error_increments_error_counter(self, trace_handler: EnterpriseOtelTrace, mock_exporter):
with patch("enterprise.telemetry.enterprise_trace.emit_metric_only_event"):
trace_handler._prompt_generation_trace(make_prompt_generation_info(error="Prompt failed"))
@@ -1294,7 +1303,7 @@ class TestPromptGenerationTrace:
]
assert len(error_calls) == 1
def test_no_error_counter_on_success(self, trace_handler, mock_exporter):
def test_no_error_counter_on_success(self, trace_handler: EnterpriseOtelTrace, mock_exporter):
with patch("enterprise.telemetry.enterprise_trace.emit_metric_only_event"):
trace_handler._prompt_generation_trace(make_prompt_generation_info())
@@ -1303,7 +1312,7 @@ class TestPromptGenerationTrace:
]
assert len(error_calls) == 0
def test_instruction_gated_by_include_content(self, trace_handler, mock_exporter):
def test_instruction_gated_by_include_content(self, trace_handler: EnterpriseOtelTrace, mock_exporter):
mock_exporter.include_content = False
with patch("enterprise.telemetry.enterprise_trace.emit_metric_only_event") as mock_emit:
trace_handler._prompt_generation_trace(make_prompt_generation_info())
@@ -1311,7 +1320,7 @@ class TestPromptGenerationTrace:
attrs = mock_emit.call_args[1]["attributes"]
assert attrs["dify.prompt_generation.instruction"].startswith("ref:trace_id=")
def test_operation_type_label_used_in_token_counters(self, trace_handler, mock_exporter):
def test_operation_type_label_used_in_token_counters(self, trace_handler: EnterpriseOtelTrace, mock_exporter):
with patch("enterprise.telemetry.enterprise_trace.emit_metric_only_event"):
trace_handler._prompt_generation_trace(make_prompt_generation_info(operation_type="code_generate"))
@@ -1321,7 +1330,7 @@ class TestPromptGenerationTrace:
assert len(token_calls) == 1
assert token_calls[0][0][2]["operation_type"] == "code_generate"
def test_emits_correct_tenant_id(self, trace_handler, mock_exporter):
def test_emits_correct_tenant_id(self, trace_handler: EnterpriseOtelTrace, mock_exporter):
with patch("enterprise.telemetry.enterprise_trace.emit_metric_only_event") as mock_emit:
trace_handler._prompt_generation_trace(make_prompt_generation_info())
@@ -0,0 +1,87 @@
"""Resolver-level expiry signalling.
An expired token must be distinguishable from an unknown/revoked one: the
resolver raises ``TokenExpiredError`` for expiry and returns ``None`` for
everything else. The signal survives the negative-cache window via a distinct
``expired`` marker so a retry inside ``NEGATIVE_TTL`` still reports expiry.
"""
from __future__ import annotations
from datetime import UTC, datetime, timedelta
from unittest.mock import MagicMock
import pytest
from libs.oauth_bearer import (
OAuthAccessTokenResolver,
TokenExpiredError,
)
def _row(expires_at: datetime):
row = MagicMock()
row.id = "11111111-1111-1111-1111-111111111111"
row.account_id = "22222222-2222-2222-2222-222222222222"
row.prefix = "dfoa_"
row.subject_email = None
row.subject_issuer = None
row.client_id = None
row.expires_at = expires_at
return row
def _resolver(redis: MagicMock, db_row=None) -> OAuthAccessTokenResolver:
session = MagicMock()
session.query.return_value.filter.return_value.one_or_none.return_value = db_row
session.execute.return_value.rowcount = 1
return OAuthAccessTokenResolver(session_factory=lambda: session, redis_client=redis)
def test_resolve_raises_token_expired_for_expired_db_row():
redis = MagicMock()
redis.get.return_value = None # cache miss -> DB path
past = datetime.now(UTC) - timedelta(minutes=1)
resolver = _resolver(redis, db_row=_row(past))
with pytest.raises(TokenExpiredError):
resolver.for_account().resolve("expiredhash")
def test_resolve_raises_token_expired_for_expired_cache_marker():
redis = MagicMock()
redis.get.return_value = b"expired" # negative-cache replay
resolver = _resolver(redis, db_row=None)
with pytest.raises(TokenExpiredError):
resolver.for_account().resolve("expiredhash")
def test_resolve_returns_none_for_invalid_cache_marker():
redis = MagicMock()
redis.get.return_value = b"invalid"
resolver = _resolver(redis, db_row=None)
assert resolver.for_account().resolve("revokedhash") is None
def test_resolve_returns_none_for_unknown_token():
redis = MagicMock()
redis.get.return_value = None # cache miss
resolver = _resolver(redis, db_row=None) # no DB row
assert resolver.for_account().resolve("unknownhash") is None
def test_hard_expire_caches_expired_marker_not_invalid():
redis = MagicMock()
redis.get.return_value = None
past = datetime.now(UTC) - timedelta(minutes=1)
resolver = _resolver(redis, db_row=_row(past))
with pytest.raises(TokenExpiredError):
resolver.for_account().resolve("expiredhash")
setex_values = [call.args[2] for call in redis.setex.call_args_list]
assert "expired" in setex_values
assert "invalid" not in setex_values

Some files were not shown because too many files have changed in this diff Show More