Compare commits

..
9977 changed files with 451537 additions and 540237 deletions
@@ -5,7 +5,7 @@ description: Write, update, or review Dify end-to-end tests under `e2e/` that us
# Dify E2E Cucumber + Playwright
Use this skill for Dify's repository-level E2E suite in `e2e/`. Use [`e2e/AGENTS.md`](../../../e2e/AGENTS.md) as the canonical package guide for local architecture and conventions, then read any feature-scoped `AGENTS.md` that owns the target area. Apply Playwright/Cucumber best practices only where they fit the current suite.
Use this skill for Dify's repository-level E2E suite in `e2e/`. Use [`e2e/AGENTS.md`](../../../e2e/AGENTS.md) as the canonical guide for local architecture and conventions, then apply Playwright/Cucumber best practices only where they fit the current suite.
## Scope
@@ -25,18 +25,17 @@ Use this skill for Dify's repository-level E2E suite in `e2e/`. Use [`e2e/AGENTS
4. Read [`references/cucumber-best-practices.md`](references/cucumber-best-practices.md) only when scenario wording, step granularity, tags, or expression design are involved.
5. Re-check official Playwright or Cucumber docs with the available documentation tools before introducing a new framework pattern.
Keep this skill focused on Cucumber, Playwright, and package-level E2E guidance. Put feature-specific conventions in the owning feature's `AGENTS.md` instead of adding them here.
## Local Rules
- `e2e/` uses Cucumber for scenarios and Playwright as the browser layer.
- `DifyWorld` is the per-scenario context object. Type `this` as `DifyWorld` and use `async function`, not arrow functions.
- Keep glue organized by capability under `e2e/features/step-definitions/`; use `common/` only for broadly reusable steps.
- Treat `e2e/AGENTS.md`, `features/support/hooks.ts`, and the Cucumber configuration as the owners of current session and tag semantics. Verify them when behavior depends on session state instead of copying a tag inventory into this skill.
- Browser session behavior comes from `features/support/hooks.ts`:
- default: authenticated session with shared storage state
- `@unauthenticated`: clean browser context
- `@authenticated`: readability/selective-run tag only unless implementation changes
- `@fresh`: only for `e2e:full*` flows
- Do not import Playwright Test runner patterns that bypass the current Cucumber + `DifyWorld` architecture unless the task is explicitly about changing that architecture.
- Perform the behavior under test through Playwright. APIs are allowed for setup, seed preparation, persistence polling, and cleanup, but ordinary Console JSON and representable multipart operations must use the scenario- or process-owned generated oRPC client with request and response validation enabled. Keep the setup/cleanup API identity independent from an unauthenticated or logged-out behavior browser.
- Consume generated operations directly. Do not add one-to-one API wrappers, handwritten endpoint URLs, response DTO casts, duplicate schemas, global mutable clients, or TanStack Query caching in Cucumber. Keep helpers only for real fixture construction, multi-operation orchestration, invariants, polling, derived test views, or protocol adapters.
- Keep SSE, binary, redirect-only, external-service, and readiness exceptions centralized under their protocol owner. A contract mismatch must fail and be fixed at the backend schema owner followed by regeneration; never weaken validation to make E2E pass.
## Workflow
@@ -44,7 +43,6 @@ Keep this skill focused on Cucumber, Playwright, and package-level E2E guidance.
- Inspect the target feature area.
- Reuse an existing step when wording and behavior already match.
- Add a new step only for a genuinely new user action or assertion.
- Before adding several similar steps, scan the target capability for an existing domain noun that can be parameterized without hiding behavior.
- Keep edits close to the current capability folder unless the step is broadly reusable.
2. Write behavior-first scenarios.
- Describe user-observable behavior, not DOM mechanics.
@@ -53,19 +51,15 @@ Keep this skill focused on Cucumber, Playwright, and package-level E2E guidance.
3. Write step definitions in the local style.
- Keep one step to one user-visible action or one assertion.
- Prefer Cucumber Expressions such as `{string}` and `{int}`.
- Use a bounded regex only when the accepted values are a small explicit domain set and Cucumber Expressions would make the Gherkin less natural.
- Do not create one-off steps for each case variant when the same domain action or outcome applies to named surfaces, modes, or resources.
- Scope locators to stable containers when the page has repeated elements.
- Avoid page-object layers or extra helper abstractions unless repeated complexity clearly justifies them.
4. Use Playwright in the local style.
- Prefer user-facing locators: `getByRole`, `getByLabel`, `getByPlaceholder`, `getByText`, then `getByTestId` for explicit contracts.
- Use web-first `expect(...)` assertions.
- Do not use `waitForTimeout`, manual polling, or raw visibility checks when a locator action or retrying assertion already expresses the behavior.
- Use `expect.poll` for API persistence, backend eventual consistency, captured browser events, or other non-DOM state; prefer locator assertions for DOM readiness and visible UI state.
- If a product element has real user-facing semantics but no accessible name, prefer fixing that accessible contract over adding a test id.
5. Validate narrowly.
- Run the narrowest tagged scenario or flow that exercises the change.
- Run the package-required static checks documented in `e2e/AGENTS.md`.
- Run `pnpm -C e2e check`.
- Broaden verification only when the change affects hooks, tags, setup, or shared step semantics.
## Review Checklist
@@ -76,8 +70,6 @@ Keep this skill focused on Cucumber, Playwright, and package-level E2E guidance.
- Are locators user-facing and assertions web-first?
- Does the change introduce hidden coupling across scenarios, tags, or instance state?
- Does it document or implement behavior that differs from the real hooks or configuration?
- Does setup/cleanup use the generated client directly, with any remaining helper owning more than a one-to-one endpoint forward?
- Is every raw HTTP call a documented protocol or infrastructure exception rather than an ordinary Console operation?
Lead findings with correctness, flake risk, and architecture drift.
@@ -39,7 +39,6 @@ Prefer reuse when:
- the user action is genuinely the same
- the expected outcome is genuinely the same
- the wording stays natural across features
- the parameter is a real product domain value such as a named surface, mode, resource, or status
Write a new step when:
@@ -47,8 +46,6 @@ Write a new step when:
- reusing the old wording would make the scenario misleading
- a supposedly generic step would become an implementation-detail wrapper
Do not optimize for a low step count by making vague steps. Optimize for a small set of truthful, domain-owned steps.
### 4. Prefer Cucumber Expressions
Use Cucumber Expressions for parameters unless regex is clearly necessary.
@@ -62,8 +59,6 @@ Common examples:
Keep expressions readable. If a step needs complicated parsing logic, first ask whether the scenario wording should be simpler.
Use regex for a bounded natural-language alternative only when it keeps Gherkin readable, for example `/(Web app|Backend service API)/`. Avoid broad regexes that accept unowned language.
### 5. Keep step definitions thin and meaningful
Step definitions are glue between Gherkin and automation, not a second abstraction language.
@@ -41,7 +41,6 @@ Also remember:
- repeated content usually needs scoping to a stable container
- exact text matching is often too brittle when role/name or label already exists
- `getByTestId` is acceptable when semantics are weak but the contract is intentional
- when a real UI region, card, status, or icon lacks an accessible name, prefer adding that semantic contract in product code before falling back to `getByTestId`
### 3. Use web-first assertions
@@ -63,8 +62,6 @@ Avoid:
If a condition genuinely needs custom retry logic, use Playwright's polling/assertion tools deliberately and keep that choice local and explicit.
Use `expect.poll` for non-DOM truth such as API state, backend eventual consistency, generated resources, or captured browser events. For DOM state, use locator assertions so Playwright can apply actionability and web-first retry semantics.
### 4. Let actions wait for actionability
Locator actions already wait for the element to be actionable. Do not preface every click/fill with extra timing logic unless the action needs a specific visible/ready assertion for clarity.
@@ -28,12 +28,9 @@ Flag:
- Missing `package.json#exports` entry for a new primitive.
- Internal package imports using workspace subpaths instead of relative paths.
- Exported props using internal-only types that consumers cannot import from the component subpath.
- Canonical primitive boundaries or their associated public types using a redundant `Root` suffix when no higher-level convenience component exists in the same subpath.
Consumers use subpath exports such as `@langgenius/dify-ui/button`.
Canonical boundaries use the primitive name and matching public types (`Select` / `SelectProps`). Keep `Root` only to distinguish a low-level anatomy root from a higher-level convenience component (`CheckboxRoot` / `Checkbox`); implementation aliases should still show their Base UI source (`BaseSelect.Root.Props`).
## Props And State
Flag:
@@ -43,22 +40,18 @@ Flag:
- JavaScript conditional class logic for visual states that the Dify UI/Base UI primitive already exposes through `data-*` attributes or CSS variables.
- Controlled props added when uncontrolled DOM state or CSS variables would be enough.
- Thin wrappers that rename Base UI parts without adding semantics.
- Generic Base UI selection primitives wrapped without preserving their value generics, such as `Select.Root<Value, Multiple>`, `RadioGroup<Value>`, or `Radio.Root<Value>`.
- Shared select/radio option components that type selected values as `string` while callers pass enums, unions, booleans, numbers, objects, or nullable placeholder values.
Prefer Base UI/Dify UI data attributes and CSS variables for visual state: `data-open`, `data-checked`, `data-disabled`, `data-highlighted`, `data-popup-open`, `group-data-*`, `peer-data-*`, `has-[:focus-visible]`, and primitive CSS variables such as anchor width or transform origin. Use JS conditional classes for product/business state that the primitive does not expose.
For non-string `Select` and `RadioGroup` values, prefer explicit domain generics at the root and at child value carriers. JSX children do not inherit the parent generic, so `RadioGroup<PromptMode>` should compose with `Radio<PromptMode>`, `RadioItem<PromptMode>`, or option values from a typed collection. For `Select`, prefer the Base UI `items` collection pattern for typed value-to-label rendering, and flag string coercion helpers used only to recover display labels.
## Forms
Flag:
- Form-like UI using unrelated `Input` and `Button` pieces without a submit boundary.
- Text-like fields not composed through `Field`, `FieldLabel`, and `FieldControl` when using Dify UI form semantics.
- Text-like fields not composed through `FieldRoot`, `FieldLabel`, and `FieldControl` when using Dify UI form semantics.
- Select fields using `FieldLabel` instead of `SelectLabel`.
- Slider fields using a generic label instead of `SliderLabel`.
- Checkbox/radio groups missing `Fieldset` and `FieldsetLegend`.
- Checkbox/radio groups missing `FieldsetRoot` and `FieldsetLegend`.
- Field errors or descriptions rendered without `FieldDescription` / `FieldError` relationships.
`Form` is the submit boundary. Dify UI form primitives are not a form state-management framework; business validation and schema-driven behavior belong in `web/`.
@@ -1,36 +1,72 @@
# Testing Review Rules
`web/docs/test.md` is the canonical frontend testing policy. Use this file only to translate that policy into review findings.
Use these rules when reviewing test files, testability of changed code, or risky frontend changes that should have tests.
## Request Missing Tests When Risk Justifies Them
## Missing Coverage
Flag missing coverage when a change alters a reachable contract such as:
Flag missing tests when the change affects:
- User interaction, navigation, form submission, validation, or permissions.
- Query or mutation behavior, URL state, persistence, or one-shot signals.
- Loading, error, empty, and recovery states that users can encounter.
- Accessibility-critical labels, keyboard flow, focus, disabled state, or overlay behavior.
- A regression-prone business rule or bug fix that can be reproduced through a public boundary.
- User-visible behavior, navigation, form submission, validation, permissions, or loading/error/empty states.
- Query/mutation cache behavior.
- Accessibility-critical behavior such as labels, keyboard flow, focus, disabled state, or popup reachability.
- URL state parsing/serialization.
- Storage persistence or one-shot signals.
- Regression-prone workflow or generated contract migration paths.
Do not request tests for mechanical changes, pass-through wrappers, implementation details, or visual-only styling unless they affect behavior. Low coverage alone is not a finding.
Do not request tests for purely mechanical renames or styling-only changes unless the styling affects layout, focus, or interaction.
## Flag Low-Value or Fragile Tests
## Selectors
Flag tests that:
Flag:
- Assert internal state, refs, hook usage, effect dependencies, private DOM structure, or cosmetic classes.
- Exist only to render a component, exercise a prop, or cover generic invalid inputs without a product scenario.
- Mock away the behavior under review or use mocks that do not match the public contract.
- Add production `data-testid` attributes where semantic markup would work.
- Use fake timers without timer behavior, leave async work unawaited, or leak shared state.
- Duplicate a contract already protected at a more useful owner boundary.
- `getByTestId` used where role, label, text, placeholder, landmark, or scoped dialog/menu queries are available.
- Production `data-testid` added only to satisfy tests.
- Assertions against decorative icons rather than the named control.
- Tests that cannot find controls semantically but leave broken markup unchanged.
## Review the Test Boundary
Prefer `getByRole` with accessible name, then `getByLabelText`, `getByPlaceholderText`, `getByText`, and `within(...)`.
- Prefer semantic queries and accessible names.
- Prefer real feature components when integration semantics matter.
- Allow intentional child or provider mocks when setup would dominate the test and that boundary is covered independently.
- Do not accept semantically inaccurate mocks of Dify UI or legacy base primitives.
- Require a real-browser or visual verification plan when `happy-dom` cannot represent the risk.
## Mocking
Treat test quality, determinism, and regression value as the review criteria. Do not use test count or coverage percentage as a proxy for quality.
Flag:
- Mocking `@langgenius/dify-ui/*` primitives.
- Mocking `@/app/components/base/*` components when the real component is practical.
- Mocking sibling or child components in the same directory for integration behavior.
- Mocks that do not match the real component's conditional rendering.
- Module-level mock state not reset in `beforeEach`.
- `vi.clearAllMocks()` in `afterEach` instead of `beforeEach`.
Use real project components for integration behavior. Mock APIs, `next/navigation`, browser shims, or complex providers only when setup would dominate the test.
## Behavior
Flag:
- Tests inspecting implementation details instead of user-observable behavior.
- Assertions that hardcode brittle copy when pattern matching or semantic roles would express behavior better.
- Fake timers used without real timing behavior.
- Async assertions missing `await`, `findBy*`, or `waitFor`.
- Test data missing required fields because inline partial objects bypass real types.
Use typed factory functions with complete defaults and partial overrides.
## URL State
For `nuqs` or query-state hooks, flag tests that:
- Mock URL state when URL synchronization is the behavior under review.
- Do not test parser serialize/parse round trips for custom parsers.
- Do not assert default-clearing behavior when defaults should be removed from the URL.
Prefer shared `NuqsTestingAdapter` helpers when available.
## Organization
Flag:
- Component/hook/util tests outside sibling `__tests__/` directories.
- Directory-level reviews that test only `index.tsx` while other files in scope contain behavior.
- Large test files with repeated setup that should use local builders.
When a component is very complex, prefer a refactor finding before asking for exhaustive tests.
+315 -19
View File
@@ -1,35 +1,331 @@
---
name: frontend-testing
description: Write, update, or review Dify frontend tests using Vitest and Testing Library. Trigger for frontend specs, test coverage requests, regressions, testability, or testing strategy under web/ or packages/dify-ui/.
description: Generate Vitest + React Testing Library tests for Dify frontend components, hooks, and utilities. Triggers on testing, spec files, coverage, Vitest, RTL, unit tests, integration tests, or write/review test requests.
---
# Dify Frontend Testing
# Dify Frontend Testing Skill
Use this skill for Vitest work under `web/` and `packages/dify-ui/`. Do not use it for Python tests or Cucumber/Playwright tests under `e2e/`.
This skill enables Codex to generate high-quality, comprehensive frontend tests for the Dify project following established conventions and best practices.
## Required Source
> **⚠️ Authoritative Source**: This skill is derived from `web/docs/test.md`. Use Vitest mock/timer APIs (`vi.*`).
Before writing, changing, or reviewing frontend tests, read `web/docs/test.md` completely. It is the single source of truth. This skill provides an execution checklist and must not redefine or extend that policy.
## When to Apply This Skill
## Workflow
Apply this skill when the user:
1. Read the source, its behavior owner, nearby specs, and relevant public dependencies.
1. Apply the canonical guide to decide whether a test is needed and choose its boundary.
1. For a behavior change or bug fix, write or identify the failing scenario first when practical.
1. Implement one coherent scenario at a time and run the focused spec before expanding scope.
1. Finish with the affected suite and relevant repository checks.
1. Report what behavior was verified and any risk that still requires browser, visual, or end-to-end validation.
- Asks to **write tests** for a component, hook, or utility
- Asks to **review existing tests** for completeness
- Mentions **Vitest**, **React Testing Library**, **RTL**, or **spec files**
- Requests **test coverage** improvement
- Uses `pnpm analyze-component` output as context
- Mentions **testing**, **unit tests**, or **integration tests** for frontend code
- Wants to understand **testing patterns** in the Dify codebase
When reviewing existing tests, recommend deleting low-value tests as readily as adding missing behavior coverage.
**Do NOT apply** when:
Run focused tests from the owning workspace:
- User is asking about backend/API tests (Python/pytest)
- User is asking about E2E tests (Cucumber + Playwright under `e2e/`)
- User is only asking conceptual questions without code context
## Quick Reference
### Key Commands
Run these commands from `web/`. From the repository root, prefix them with `pnpm -C web`.
```bash
# web/
vp test run path/to/spec-or-directory
# Run all tests
pnpm test
# packages/dify-ui/
vp test run --project unit src/path/to/spec
# Watch mode
pnpm test --watch
# Run specific file
pnpm test path/to/file.spec.tsx
# Generate coverage report
pnpm test --coverage
# Analyze component complexity
pnpm analyze-component <path>
# Review existing test
pnpm analyze-component <path> --review
```
Run Dify UI Storybook tests with `vp test --project storybook --run`. Run broader checks only after the focused behavior passes.
### File Naming
- Test files: `ComponentName.spec.tsx` inside a same-level `__tests__/` directory
- Placement rule: Component, hook, and utility tests must live in a sibling `__tests__/` folder at the same level as the source under test. For example, `foo/index.tsx` maps to `foo/__tests__/index.spec.tsx`, and `foo/bar.ts` maps to `foo/__tests__/bar.spec.ts`.
- Integration tests: `web/__tests__/` directory
## Test Structure Template
```typescript
import { render, screen, fireEvent, waitFor } from '@testing-library/react'
import Component from './index'
// ✅ Import real project components (DO NOT mock these)
// import Loading from '@/app/components/base/loading'
// import { ChildComponent } from './child-component'
// ✅ Mock external dependencies only
vi.mock('@/service/api')
vi.mock('next/navigation', () => ({
useRouter: () => ({ push: vi.fn() }),
usePathname: () => '/test',
}))
// ✅ Zustand stores: Use real stores (auto-mocked globally)
// Set test state with: useAppStore.setState({ ... })
// Shared state for mocks (if needed)
let mockSharedState = false
describe('ComponentName', () => {
beforeEach(() => {
vi.clearAllMocks() // ✅ Reset mocks BEFORE each test
mockSharedState = false // ✅ Reset shared state
})
// Rendering tests (REQUIRED)
describe('Rendering', () => {
it('should render without crashing', () => {
// Arrange
const props = { title: 'Test' }
// Act
render(<Component {...props} />)
// Assert
expect(screen.getByText('Test')).toBeInTheDocument()
})
})
// Props tests (REQUIRED)
describe('Props', () => {
it('should apply custom className', () => {
render(<Component className="custom" />)
expect(screen.getByRole('button')).toHaveClass('custom')
})
})
// User Interactions
describe('User Interactions', () => {
it('should handle click events', () => {
const handleClick = vi.fn()
render(<Component onClick={handleClick} />)
fireEvent.click(screen.getByRole('button'))
expect(handleClick).toHaveBeenCalledTimes(1)
})
})
// Edge Cases (REQUIRED)
describe('Edge Cases', () => {
it('should handle null data', () => {
render(<Component data={null} />)
expect(screen.getByText(/no data/i)).toBeInTheDocument()
})
it('should handle empty array', () => {
render(<Component items={[]} />)
expect(screen.getByText(/empty/i)).toBeInTheDocument()
})
})
})
```
## Testing Workflow (CRITICAL)
### ⚠️ Incremental Approach Required
**NEVER generate all test files at once.** For complex components or multi-file directories:
1. **Analyze & Plan**: List all files, order by complexity (simple → complex)
1. **Process ONE at a time**: Write test → Run test → Fix if needed → Next
1. **Verify before proceeding**: Do NOT continue to next file until current passes
```
For each file:
┌────────────────────────────────────────┐
│ 1. Write test │
│ 2. Run: pnpm test <file>.spec.tsx │
│ 3. PASS? → Mark complete, next file │
│ FAIL? → Fix first, then continue │
└────────────────────────────────────────┘
```
### Complexity-Based Order
Process in this order for multi-file testing:
1. 🟢 Utility functions (simplest)
1. 🟢 Custom hooks
1. 🟡 Simple components (presentational)
1. 🟡 Medium components (state, effects)
1. 🔴 Complex components (API, routing)
1. 🔴 Integration tests (index files - last)
### When to Refactor First
- **Complexity > 50**: Break into smaller pieces before testing
- **500+ lines**: Consider splitting before testing
- **Many dependencies**: Extract logic into hooks first
> 📖 See `references/workflow.md` for complete workflow details and todo list format.
## Testing Strategy
### Path-Level Testing (Directory Testing)
When assigned to test a directory/path, test **ALL content** within that path:
- Test all components, hooks, utilities in the directory (not just `index` file)
- Use incremental approach: one file at a time, verify each before proceeding
- Goal: 100% coverage of ALL files in the directory
### Integration Testing First
**Prefer integration testing** when writing tests for a directory:
-**Import real project components** directly (including base components and siblings)
-**Only mock**: API services (`@/service/*`), `next/navigation`, complex context providers
-**DO NOT mock** base components (`@/app/components/base/*`) or dify-ui primitives (`@langgenius/dify-ui/*`)
-**DO NOT mock** sibling/child components in the same directory
> See [Test Structure Template](#test-structure-template) for correct import/mock patterns.
### `nuqs` Query State Testing (Required for URL State Hooks)
When a component or hook uses `useQueryState` / `useQueryStates`:
- ✅ Use `NuqsTestingAdapter` (prefer shared helpers in `web/test/nuqs-testing.tsx`)
- ✅ Assert URL synchronization via `onUrlUpdate` (`searchParams`, `options.history`)
- ✅ For custom parsers (`createParser`), keep `parse` and `serialize` bijective and add round-trip edge cases (`%2F`, `%25`, spaces, legacy encoded values)
- ✅ Verify default-clearing behavior (default values should be removed from URL when applicable)
- ⚠️ Only mock `nuqs` directly when URL behavior is explicitly out of scope for the test
## Core Principles
### 1. AAA Pattern (Arrange-Act-Assert)
Every test should clearly separate:
- **Arrange**: Setup test data and render component
- **Act**: Perform user actions
- **Assert**: Verify expected outcomes
### 2. Black-Box Testing
- Test observable behavior, not implementation details
- Use semantic queries (`getByRole` with accessible `name`, `getByLabelText`, `getByPlaceholderText`, `getByText`, and scoped `within(...)`)
- Treat `getByTestId` as a last resort. If a control cannot be found by role/name, label, landmark, or dialog scope, fix the component accessibility first instead of adding or relying on `data-testid`.
- Remove production `data-testid` attributes when semantic selectors can cover the behavior. Keep them only for non-visual mocked boundaries, editor/browser shims such as Monaco, canvas/chart output, or third-party widgets with no accessible DOM in the test environment.
- Do not assert decorative icons by test id. Assert the named control that contains them, or mark decorative icons `aria-hidden`.
- Avoid testing internal state directly
- **Prefer pattern matching over hardcoded strings** in assertions:
```typescript
// ❌ Avoid: hardcoded text assertions
expect(screen.getByText('Loading...')).toBeInTheDocument()
// ✅ Better: role-based queries
expect(screen.getByRole('status')).toBeInTheDocument()
// ✅ Better: pattern matching
expect(screen.getByText(/loading/i)).toBeInTheDocument()
```
### 3. Single Behavior Per Test
Each test verifies ONE user-observable behavior:
```typescript
// ✅ Good: One behavior
it('should disable button when loading', () => {
render(<Button loading />)
expect(screen.getByRole('button')).toBeDisabled()
})
// ❌ Bad: Multiple behaviors
it('should handle loading state', () => {
render(<Button loading />)
expect(screen.getByRole('button')).toBeDisabled()
expect(screen.getByText('Loading...')).toBeInTheDocument()
expect(screen.getByRole('button')).toHaveClass('loading')
})
```
### 4. Semantic Naming
Use `should <behavior> when <condition>`:
```typescript
it('should show error message when validation fails')
it('should call onSubmit when form is valid')
it('should disable input when isReadOnly is true')
```
## Required Test Scenarios
### Always Required (All Components)
1. **Rendering**: Component renders without crashing
1. **Props**: Required props, optional props, default values
1. **Edge Cases**: null, undefined, empty values, boundary conditions
### Conditional (When Present)
| Feature | Test Focus |
|---------|-----------|
| `useState` | Initial state, transitions, cleanup |
| `useEffect` | Execution, dependencies, cleanup |
| Event handlers | All onClick, onChange, onSubmit, keyboard |
| API calls | Loading, success, error states |
| Routing | Navigation, params, query strings |
| `useCallback`/`useMemo` | Referential equality |
| Context | Provider values, consumer behavior |
| Forms | Validation, submission, error display |
## Coverage Goals (Per File)
For each test file generated, aim for:
-**100%** function coverage
-**100%** statement coverage
-**>95%** branch coverage
-**>95%** line coverage
> **Note**: For multi-file directories, process one file at a time with full coverage each. See `references/workflow.md`.
## Detailed Guides
For more detailed information, refer to:
- `references/workflow.md` - **Incremental testing workflow** (MUST READ for multi-file testing)
- `references/mocking.md` - Mock patterns, Zustand store testing, and best practices
- `references/async-testing.md` - Async operations and API calls
- `references/domain-components.md` - Workflow, Dataset, Configuration testing
- `references/common-patterns.md` - Frequently used testing patterns
- `references/checklist.md` - Test generation checklist and validation steps
## Authoritative References
### Primary Specification (MUST follow)
- **`web/docs/test.md`** - The canonical testing specification. This skill is derived from this document.
### Reference Examples in Codebase
- `web/utils/classnames.spec.ts` - Utility function tests
- `web/app/components/base/radio/__tests__/index.spec.tsx` - Component tests
- `web/__mocks__/provider-context.ts` - Mock factory example
### Project Configuration
- `web/vite.config.ts` - Vite/Vitest configuration
- `web/vitest.setup.ts` - Test environment setup
- `web/scripts/analyze-component.js` - Component analysis tool
- Modules are not mocked automatically. Global mocks live in `web/vitest.setup.ts` (for example `react-i18next`, `next/image`); mock other modules like `ky` or `mime` locally in test files.
@@ -0,0 +1,293 @@
/**
* Test Template for React Components
*
* WHY THIS STRUCTURE?
* - Organized sections make tests easy to navigate and maintain
* - Mocks at top ensure consistent test isolation
* - Factory functions reduce duplication and improve readability
* - describe blocks group related scenarios for better debugging
*
* INSTRUCTIONS:
* 1. Replace `ComponentName` with your component name
* 2. Update import path
* 3. Add/remove test sections based on component features (use analyze-component)
* 4. Follow AAA pattern: Arrange → Act → Assert
*
* RUN FIRST: pnpm analyze-component <path> to identify required test scenarios
*/
import { render, screen, fireEvent, waitFor } from '@testing-library/react'
import userEvent from '@testing-library/user-event'
// import ComponentName from './index'
// ============================================================================
// Mocks
// ============================================================================
// WHY: Mocks must be hoisted to top of file (Vitest requirement).
// They run BEFORE imports, so keep them before component imports.
// i18n (automatically mocked)
// WHY: Global mock in web/vitest.setup.ts is auto-loaded by Vitest setup
// The global mock provides: useTranslation, Trans, useMixedTranslation, useGetLanguage
// No explicit mock needed for most tests
//
// Override only if custom translations are required:
// import { createReactI18nextMock } from '@/test/i18n-mock'
// vi.mock('react-i18next', () => createReactI18nextMock({
// 'my.custom.key': 'Custom Translation',
// 'button.save': 'Save',
// }))
// Router (if component uses useRouter, usePathname, useSearchParams)
// WHY: Isolates tests from Next.js routing, enables testing navigation behavior
// const mockPush = vi.fn()
// vi.mock('@/next/navigation', () => ({
// useRouter: () => ({ push: mockPush }),
// usePathname: () => '/test-path',
// }))
// API services (if component fetches data)
// WHY: Prevents real network calls, enables testing all states (loading/success/error)
// vi.mock('@/service/api')
// import * as api from '@/service/api'
// const mockedApi = vi.mocked(api)
// Shared mock state (for portal/dropdown components)
// WHY: Portal components like PortalToFollowElem need shared state between
// parent and child mocks to correctly simulate open/close behavior
// let mockOpenState = false
// ============================================================================
// Test Data Factories
// ============================================================================
// WHY FACTORIES?
// - Avoid hard-coded test data scattered across tests
// - Easy to create variations with overrides
// - Type-safe when using actual types from source
// - Single source of truth for default test values
// const createMockProps = (overrides = {}) => ({
// // Default props that make component render successfully
// ...overrides,
// })
// const createMockItem = (overrides = {}) => ({
// id: 'item-1',
// name: 'Test Item',
// ...overrides,
// })
// ============================================================================
// Test Helpers
// ============================================================================
// const renderComponent = (props = {}) => {
// return render(<ComponentName {...createMockProps(props)} />)
// }
// ============================================================================
// Tests
// ============================================================================
describe('ComponentName', () => {
// WHY beforeEach with clearAllMocks?
// - Ensures each test starts with clean slate
// - Prevents mock call history from leaking between tests
// - MUST be beforeEach (not afterEach) to reset BEFORE assertions like toHaveBeenCalledTimes
beforeEach(() => {
vi.clearAllMocks()
// Reset shared mock state if used (CRITICAL for portal/dropdown tests)
// mockOpenState = false
})
// --------------------------------------------------------------------------
// Rendering Tests (REQUIRED - Every component MUST have these)
// --------------------------------------------------------------------------
// WHY: Catches import errors, missing providers, and basic render issues
describe('Rendering', () => {
it('should render without crashing', () => {
// Arrange - Setup data and mocks
// const props = createMockProps()
// Act - Render the component
// render(<ComponentName {...props} />)
// Assert - Verify expected output
// Prefer getByRole for accessibility; it's what users "see"
// expect(screen.getByRole('...')).toBeInTheDocument()
})
it('should render with default props', () => {
// WHY: Verifies component works without optional props
// render(<ComponentName />)
// expect(screen.getByText('...')).toBeInTheDocument()
})
})
// --------------------------------------------------------------------------
// Props Tests (REQUIRED - Every component MUST test prop behavior)
// --------------------------------------------------------------------------
// WHY: Props are the component's API contract. Test them thoroughly.
describe('Props', () => {
it('should apply custom className', () => {
// WHY: Common pattern in Dify - components should merge custom classes
// render(<ComponentName className="custom-class" />)
// expect(screen.getByTestId('component')).toHaveClass('custom-class')
})
it('should use default values for optional props', () => {
// WHY: Verifies TypeScript defaults work at runtime
// render(<ComponentName />)
// expect(screen.getByRole('...')).toHaveAttribute('...', 'default-value')
})
})
// --------------------------------------------------------------------------
// User Interactions (if component has event handlers - on*, handle*)
// --------------------------------------------------------------------------
// WHY: Event handlers are core functionality. Test from user's perspective.
describe('User Interactions', () => {
it('should call onClick when clicked', async () => {
// WHY userEvent over fireEvent?
// - userEvent simulates real user behavior (focus, hover, then click)
// - fireEvent is lower-level, doesn't trigger all browser events
// const user = userEvent.setup()
// const handleClick = vi.fn()
// render(<ComponentName onClick={handleClick} />)
//
// await user.click(screen.getByRole('button'))
//
// expect(handleClick).toHaveBeenCalledTimes(1)
})
it('should call onChange when value changes', async () => {
// const user = userEvent.setup()
// const handleChange = vi.fn()
// render(<ComponentName onChange={handleChange} />)
//
// await user.type(screen.getByRole('textbox'), 'new value')
//
// expect(handleChange).toHaveBeenCalled()
})
})
// --------------------------------------------------------------------------
// State Management (if component uses useState/useReducer)
// --------------------------------------------------------------------------
// WHY: Test state through observable UI changes, not internal state values
describe('State Management', () => {
it('should update state on interaction', async () => {
// WHY test via UI, not state?
// - State is implementation detail; UI is what users see
// - If UI works correctly, state must be correct
// const user = userEvent.setup()
// render(<ComponentName />)
//
// // Initial state - verify what user sees
// expect(screen.getByText('Initial')).toBeInTheDocument()
//
// // Trigger state change via user action
// await user.click(screen.getByRole('button'))
//
// // New state - verify UI updated
// expect(screen.getByText('Updated')).toBeInTheDocument()
})
})
// --------------------------------------------------------------------------
// Async Operations (if component fetches data - useQuery, fetch)
// --------------------------------------------------------------------------
// WHY: Async operations have 3 states users experience: loading, success, error
describe('Async Operations', () => {
it('should show loading state', () => {
// WHY never-resolving promise?
// - Keeps component in loading state for assertion
// - Alternative: use fake timers
// mockedApi.fetchData.mockImplementation(() => new Promise(() => {}))
// render(<ComponentName />)
//
// expect(screen.getByText(/loading/i)).toBeInTheDocument()
})
it('should show data on success', async () => {
// WHY waitFor?
// - Component updates asynchronously after fetch resolves
// - waitFor retries assertion until it passes or times out
// mockedApi.fetchData.mockResolvedValue({ items: ['Item 1'] })
// render(<ComponentName />)
//
// await waitFor(() => {
// expect(screen.getByText('Item 1')).toBeInTheDocument()
// })
})
it('should show error on failure', async () => {
// mockedApi.fetchData.mockRejectedValue(new Error('Network error'))
// render(<ComponentName />)
//
// await waitFor(() => {
// expect(screen.getByText(/error/i)).toBeInTheDocument()
// })
})
})
// --------------------------------------------------------------------------
// Edge Cases (REQUIRED - Every component MUST handle edge cases)
// --------------------------------------------------------------------------
// WHY: Real-world data is messy. Components must handle:
// - Null/undefined from API failures or optional fields
// - Empty arrays/strings from user clearing data
// - Boundary values (0, MAX_INT, special characters)
describe('Edge Cases', () => {
it('should handle null value', () => {
// WHY test null specifically?
// - API might return null for missing data
// - Prevents "Cannot read property of null" in production
// render(<ComponentName value={null} />)
// expect(screen.getByText(/no data/i)).toBeInTheDocument()
})
it('should handle undefined value', () => {
// WHY test undefined separately from null?
// - TypeScript treats them differently
// - Optional props are undefined, not null
// render(<ComponentName value={undefined} />)
// expect(screen.getByText(/no data/i)).toBeInTheDocument()
})
it('should handle empty array', () => {
// WHY: Empty state often needs special UI (e.g., "No items yet")
// render(<ComponentName items={[]} />)
// expect(screen.getByText(/empty/i)).toBeInTheDocument()
})
it('should handle empty string', () => {
// WHY: Empty strings are truthy in JS but visually empty
// render(<ComponentName text="" />)
// expect(screen.getByText(/placeholder/i)).toBeInTheDocument()
})
})
// --------------------------------------------------------------------------
// Accessibility (optional but recommended for Dify's enterprise users)
// --------------------------------------------------------------------------
// WHY: Dify has enterprise customers who may require accessibility compliance
describe('Accessibility', () => {
it('should have accessible name', () => {
// WHY getByRole with name?
// - Tests that screen readers can identify the element
// - Enforces proper labeling practices
// render(<ComponentName label="Test Label" />)
// expect(screen.getByRole('button', { name: /test label/i })).toBeInTheDocument()
})
it('should support keyboard navigation', async () => {
// WHY: Some users can't use a mouse
// const user = userEvent.setup()
// render(<ComponentName />)
//
// await user.tab()
// expect(screen.getByRole('button')).toHaveFocus()
})
})
})
@@ -0,0 +1,207 @@
/**
* Test Template for Custom Hooks
*
* Instructions:
* 1. Replace `useHookName` with your hook name
* 2. Update import path
* 3. Add/remove test sections based on hook features
*/
import { renderHook, act, waitFor } from '@testing-library/react'
// import { useHookName } from './use-hook-name'
// ============================================================================
// Mocks
// ============================================================================
// API services (if hook fetches data)
// vi.mock('@/service/api')
// import * as api from '@/service/api'
// const mockedApi = vi.mocked(api)
// ============================================================================
// Test Helpers
// ============================================================================
// Wrapper for hooks that need context
// const createWrapper = (contextValue = {}) => {
// return ({ children }: { children: React.ReactNode }) => (
// <SomeContext.Provider value={contextValue}>
// {children}
// </SomeContext.Provider>
// )
// }
// ============================================================================
// Tests
// ============================================================================
describe('useHookName', () => {
beforeEach(() => {
vi.clearAllMocks()
})
// --------------------------------------------------------------------------
// Initial State
// --------------------------------------------------------------------------
describe('Initial State', () => {
it('should return initial state', () => {
// const { result } = renderHook(() => useHookName())
//
// expect(result.current.value).toBe(initialValue)
// expect(result.current.isLoading).toBe(false)
})
it('should accept initial value from props', () => {
// const { result } = renderHook(() => useHookName({ initialValue: 'custom' }))
//
// expect(result.current.value).toBe('custom')
})
})
// --------------------------------------------------------------------------
// State Updates
// --------------------------------------------------------------------------
describe('State Updates', () => {
it('should update value when setValue is called', () => {
// const { result } = renderHook(() => useHookName())
//
// act(() => {
// result.current.setValue('new value')
// })
//
// expect(result.current.value).toBe('new value')
})
it('should reset to initial value', () => {
// const { result } = renderHook(() => useHookName({ initialValue: 'initial' }))
//
// act(() => {
// result.current.setValue('changed')
// })
// expect(result.current.value).toBe('changed')
//
// act(() => {
// result.current.reset()
// })
// expect(result.current.value).toBe('initial')
})
})
// --------------------------------------------------------------------------
// Async Operations
// --------------------------------------------------------------------------
describe('Async Operations', () => {
it('should fetch data on mount', async () => {
// mockedApi.fetchData.mockResolvedValue({ data: 'test' })
//
// const { result } = renderHook(() => useHookName())
//
// // Initially loading
// expect(result.current.isLoading).toBe(true)
//
// // Wait for data
// await waitFor(() => {
// expect(result.current.isLoading).toBe(false)
// })
//
// expect(result.current.data).toEqual({ data: 'test' })
})
it('should handle fetch error', async () => {
// mockedApi.fetchData.mockRejectedValue(new Error('Network error'))
//
// const { result } = renderHook(() => useHookName())
//
// await waitFor(() => {
// expect(result.current.error).toBeTruthy()
// })
//
// expect(result.current.error?.message).toBe('Network error')
})
it('should refetch when dependency changes', async () => {
// mockedApi.fetchData.mockResolvedValue({ data: 'test' })
//
// const { result, rerender } = renderHook(
// ({ id }) => useHookName(id),
// { initialProps: { id: '1' } }
// )
//
// await waitFor(() => {
// expect(mockedApi.fetchData).toHaveBeenCalledWith('1')
// })
//
// rerender({ id: '2' })
//
// await waitFor(() => {
// expect(mockedApi.fetchData).toHaveBeenCalledWith('2')
// })
})
})
// --------------------------------------------------------------------------
// Side Effects
// --------------------------------------------------------------------------
describe('Side Effects', () => {
it('should call callback when value changes', () => {
// const callback = vi.fn()
// const { result } = renderHook(() => useHookName({ onChange: callback }))
//
// act(() => {
// result.current.setValue('new value')
// })
//
// expect(callback).toHaveBeenCalledWith('new value')
})
it('should cleanup on unmount', () => {
// const cleanup = vi.fn()
// vi.spyOn(window, 'addEventListener')
// vi.spyOn(window, 'removeEventListener')
//
// const { unmount } = renderHook(() => useHookName())
//
// expect(window.addEventListener).toHaveBeenCalled()
//
// unmount()
//
// expect(window.removeEventListener).toHaveBeenCalled()
})
})
// --------------------------------------------------------------------------
// Edge Cases
// --------------------------------------------------------------------------
describe('Edge Cases', () => {
it('should handle null input', () => {
// const { result } = renderHook(() => useHookName(null))
//
// expect(result.current.value).toBeNull()
})
it('should handle rapid updates', () => {
// const { result } = renderHook(() => useHookName())
//
// act(() => {
// result.current.setValue('1')
// result.current.setValue('2')
// result.current.setValue('3')
// })
//
// expect(result.current.value).toBe('3')
})
})
// --------------------------------------------------------------------------
// With Context (if hook uses context)
// --------------------------------------------------------------------------
describe('With Context', () => {
it('should use context value', () => {
// const wrapper = createWrapper({ someValue: 'context-value' })
// const { result } = renderHook(() => useHookName(), { wrapper })
//
// expect(result.current.contextValue).toBe('context-value')
})
})
})
@@ -0,0 +1,154 @@
/**
* Test Template for Utility Functions
*
* Instructions:
* 1. Replace `utilityFunction` with your function name
* 2. Update import path
* 3. Use test.each for data-driven tests
*/
// import { utilityFunction } from './utility'
// ============================================================================
// Tests
// ============================================================================
describe('utilityFunction', () => {
// --------------------------------------------------------------------------
// Basic Functionality
// --------------------------------------------------------------------------
describe('Basic Functionality', () => {
it('should return expected result for valid input', () => {
// expect(utilityFunction('input')).toBe('expected-output')
})
it('should handle multiple arguments', () => {
// expect(utilityFunction('a', 'b', 'c')).toBe('abc')
})
})
// --------------------------------------------------------------------------
// Data-Driven Tests
// --------------------------------------------------------------------------
describe('Input/Output Mapping', () => {
test.each([
// [input, expected]
['input1', 'output1'],
['input2', 'output2'],
['input3', 'output3'],
])('should return %s for input %s', (input, expected) => {
// expect(utilityFunction(input)).toBe(expected)
})
})
// --------------------------------------------------------------------------
// Edge Cases
// --------------------------------------------------------------------------
describe('Edge Cases', () => {
it('should handle empty string', () => {
// expect(utilityFunction('')).toBe('')
})
it('should handle null', () => {
// expect(utilityFunction(null)).toBe(null)
// or
// expect(() => utilityFunction(null)).toThrow()
})
it('should handle undefined', () => {
// expect(utilityFunction(undefined)).toBe(undefined)
// or
// expect(() => utilityFunction(undefined)).toThrow()
})
it('should handle empty array', () => {
// expect(utilityFunction([])).toEqual([])
})
it('should handle empty object', () => {
// expect(utilityFunction({})).toEqual({})
})
})
// --------------------------------------------------------------------------
// Boundary Conditions
// --------------------------------------------------------------------------
describe('Boundary Conditions', () => {
it('should handle minimum value', () => {
// expect(utilityFunction(0)).toBe(0)
})
it('should handle maximum value', () => {
// expect(utilityFunction(Number.MAX_SAFE_INTEGER)).toBe(...)
})
it('should handle negative numbers', () => {
// expect(utilityFunction(-1)).toBe(...)
})
})
// --------------------------------------------------------------------------
// Type Coercion (if applicable)
// --------------------------------------------------------------------------
describe('Type Handling', () => {
it('should handle numeric string', () => {
// expect(utilityFunction('123')).toBe(123)
})
it('should handle boolean', () => {
// expect(utilityFunction(true)).toBe(...)
})
})
// --------------------------------------------------------------------------
// Error Cases
// --------------------------------------------------------------------------
describe('Error Handling', () => {
it('should throw for invalid input', () => {
// expect(() => utilityFunction('invalid')).toThrow('Error message')
})
it('should throw with specific error type', () => {
// expect(() => utilityFunction('invalid')).toThrow(ValidationError)
})
})
// --------------------------------------------------------------------------
// Complex Objects (if applicable)
// --------------------------------------------------------------------------
describe('Object Handling', () => {
it('should preserve object structure', () => {
// const input = { a: 1, b: 2 }
// expect(utilityFunction(input)).toEqual({ a: 1, b: 2 })
})
it('should handle nested objects', () => {
// const input = { nested: { deep: 'value' } }
// expect(utilityFunction(input)).toEqual({ nested: { deep: 'transformed' } })
})
it('should not mutate input', () => {
// const input = { a: 1 }
// const inputCopy = { ...input }
// utilityFunction(input)
// expect(input).toEqual(inputCopy)
})
})
// --------------------------------------------------------------------------
// Array Handling (if applicable)
// --------------------------------------------------------------------------
describe('Array Handling', () => {
it('should process all elements', () => {
// expect(utilityFunction([1, 2, 3])).toEqual([2, 4, 6])
})
it('should handle single element array', () => {
// expect(utilityFunction([1])).toEqual([2])
})
it('should preserve order', () => {
// expect(utilityFunction(['c', 'a', 'b'])).toEqual(['c', 'a', 'b'])
})
})
})
@@ -0,0 +1,345 @@
# Async Testing Guide
## Core Async Patterns
### 1. waitFor - Wait for Condition
```typescript
import { render, screen, waitFor } from '@testing-library/react'
it('should load and display data', async () => {
render(<DataComponent />)
// Wait for element to appear
await waitFor(() => {
expect(screen.getByText('Loaded Data')).toBeInTheDocument()
})
})
it('should hide loading spinner after load', async () => {
render(<DataComponent />)
// Wait for element to disappear
await waitFor(() => {
expect(screen.queryByText('Loading...')).not.toBeInTheDocument()
})
})
```
### 2. findBy\* - Async Queries
```typescript
it('should show user name after fetch', async () => {
render(<UserProfile />)
// findBy returns a promise, auto-waits up to 1000ms
const userName = await screen.findByText('John Doe')
expect(userName).toBeInTheDocument()
// findByRole with options
const button = await screen.findByRole('button', { name: /submit/i })
expect(button).toBeEnabled()
})
```
### 3. userEvent for Async Interactions
```typescript
import userEvent from '@testing-library/user-event'
it('should submit form', async () => {
const user = userEvent.setup()
const onSubmit = vi.fn()
render(<Form onSubmit={onSubmit} />)
// userEvent methods are async
await user.type(screen.getByLabelText('Email'), 'test@example.com')
await user.click(screen.getByRole('button', { name: /submit/i }))
await waitFor(() => {
expect(onSubmit).toHaveBeenCalledWith({ email: 'test@example.com' })
})
})
```
## Fake Timers
### When to Use Fake Timers
- Testing components with `setTimeout`/`setInterval`
- Testing debounce/throttle behavior
- Testing animations or delayed transitions
- Testing polling or retry logic
### Basic Fake Timer Setup
```typescript
describe('Debounced Search', () => {
beforeEach(() => {
vi.useFakeTimers()
})
afterEach(() => {
vi.useRealTimers()
})
it('should debounce search input', async () => {
const onSearch = vi.fn()
render(<SearchInput onSearch={onSearch} debounceMs={300} />)
// Type in the input
fireEvent.change(screen.getByRole('textbox'), { target: { value: 'query' } })
// Search not called immediately
expect(onSearch).not.toHaveBeenCalled()
// Advance timers
vi.advanceTimersByTime(300)
// Now search is called
expect(onSearch).toHaveBeenCalledWith('query')
})
})
```
### Fake Timers with Async Code
```typescript
it('should retry on failure', async () => {
vi.useFakeTimers()
const fetchData = vi.fn()
.mockRejectedValueOnce(new Error('Network error'))
.mockResolvedValueOnce({ data: 'success' })
render(<RetryComponent fetchData={fetchData} retryDelayMs={1000} />)
// First call fails
await waitFor(() => {
expect(fetchData).toHaveBeenCalledTimes(1)
})
// Advance timer for retry
vi.advanceTimersByTime(1000)
// Second call succeeds
await waitFor(() => {
expect(fetchData).toHaveBeenCalledTimes(2)
expect(screen.getByText('success')).toBeInTheDocument()
})
vi.useRealTimers()
})
```
### Common Fake Timer Utilities
```typescript
// Run all pending timers
vi.runAllTimers()
// Run only pending timers (not new ones created during execution)
vi.runOnlyPendingTimers()
// Advance by specific time
vi.advanceTimersByTime(1000)
// Get current fake time
Date.now()
// Clear all timers
vi.clearAllTimers()
```
## API Testing Patterns
### Loading → Success → Error States
```typescript
describe('DataFetcher', () => {
beforeEach(() => {
vi.clearAllMocks()
})
it('should show loading state', () => {
mockedApi.fetchData.mockImplementation(() => new Promise(() => {})) // Never resolves
render(<DataFetcher />)
expect(screen.getByTestId('loading-spinner')).toBeInTheDocument()
})
it('should show data on success', async () => {
mockedApi.fetchData.mockResolvedValue({ items: ['Item 1', 'Item 2'] })
render(<DataFetcher />)
// Use findBy* for multiple async elements (better error messages than waitFor with multiple assertions)
const item1 = await screen.findByText('Item 1')
const item2 = await screen.findByText('Item 2')
expect(item1).toBeInTheDocument()
expect(item2).toBeInTheDocument()
expect(screen.queryByTestId('loading-spinner')).not.toBeInTheDocument()
})
it('should show error on failure', async () => {
mockedApi.fetchData.mockRejectedValue(new Error('Failed to fetch'))
render(<DataFetcher />)
await waitFor(() => {
expect(screen.getByText(/failed to fetch/i)).toBeInTheDocument()
})
})
it('should retry on error', async () => {
mockedApi.fetchData.mockRejectedValue(new Error('Network error'))
render(<DataFetcher />)
await waitFor(() => {
expect(screen.getByRole('button', { name: /retry/i })).toBeInTheDocument()
})
mockedApi.fetchData.mockResolvedValue({ items: ['Item 1'] })
fireEvent.click(screen.getByRole('button', { name: /retry/i }))
await waitFor(() => {
expect(screen.getByText('Item 1')).toBeInTheDocument()
})
})
})
```
### Testing Mutations
```typescript
it('should submit form and show success', async () => {
const user = userEvent.setup()
mockedApi.createItem.mockResolvedValue({ id: '1', name: 'New Item' })
render(<CreateItemForm />)
await user.type(screen.getByLabelText('Name'), 'New Item')
await user.click(screen.getByRole('button', { name: /create/i }))
// Button should be disabled during submission
expect(screen.getByRole('button', { name: /creating/i })).toBeDisabled()
await waitFor(() => {
expect(screen.getByText(/created successfully/i)).toBeInTheDocument()
})
expect(mockedApi.createItem).toHaveBeenCalledWith({ name: 'New Item' })
})
```
## useEffect Testing
### Testing Effect Execution
```typescript
it('should fetch data on mount', async () => {
const fetchData = vi.fn().mockResolvedValue({ data: 'test' })
render(<ComponentWithEffect fetchData={fetchData} />)
await waitFor(() => {
expect(fetchData).toHaveBeenCalledTimes(1)
})
})
```
### Testing Effect Dependencies
```typescript
it('should refetch when id changes', async () => {
const fetchData = vi.fn().mockResolvedValue({ data: 'test' })
const { rerender } = render(<ComponentWithEffect id="1" fetchData={fetchData} />)
await waitFor(() => {
expect(fetchData).toHaveBeenCalledWith('1')
})
rerender(<ComponentWithEffect id="2" fetchData={fetchData} />)
await waitFor(() => {
expect(fetchData).toHaveBeenCalledWith('2')
expect(fetchData).toHaveBeenCalledTimes(2)
})
})
```
### Testing Effect Cleanup
```typescript
it('should cleanup subscription on unmount', () => {
const subscribe = vi.fn()
const unsubscribe = vi.fn()
subscribe.mockReturnValue(unsubscribe)
const { unmount } = render(<SubscriptionComponent subscribe={subscribe} />)
expect(subscribe).toHaveBeenCalledTimes(1)
unmount()
expect(unsubscribe).toHaveBeenCalledTimes(1)
})
```
## Common Async Pitfalls
### ❌ Don't: Forget to await
```typescript
// Bad - test may pass even if assertion fails
it('should load data', () => {
render(<Component />)
waitFor(() => {
expect(screen.getByText('Data')).toBeInTheDocument()
})
})
// Good - properly awaited
it('should load data', async () => {
render(<Component />)
await waitFor(() => {
expect(screen.getByText('Data')).toBeInTheDocument()
})
})
```
### ❌ Don't: Use multiple assertions in single waitFor
```typescript
// Bad - if first assertion fails, won't know about second
await waitFor(() => {
expect(screen.getByText('Title')).toBeInTheDocument()
expect(screen.getByText('Description')).toBeInTheDocument()
})
// Good - separate waitFor or use findBy
const title = await screen.findByText('Title')
const description = await screen.findByText('Description')
expect(title).toBeInTheDocument()
expect(description).toBeInTheDocument()
```
### ❌ Don't: Mix fake timers with real async
```typescript
// Bad - fake timers don't work well with real Promises
vi.useFakeTimers()
await waitFor(() => {
expect(screen.getByText('Data')).toBeInTheDocument()
}) // May timeout!
// Good - use runAllTimers or advanceTimersByTime
vi.useFakeTimers()
render(<Component />)
vi.runAllTimers()
expect(screen.getByText('Data')).toBeInTheDocument()
```
@@ -0,0 +1,208 @@
# Test Generation Checklist
Use this checklist when generating or reviewing tests for Dify frontend components.
## Pre-Generation
- [ ] Read the component source code completely
- [ ] Identify component type (component, hook, utility, page)
- [ ] Run `pnpm analyze-component <path>` if available
- [ ] Note complexity score and features detected
- [ ] Check for existing tests in the same directory
- [ ] **Identify ALL files in the directory** that need testing (not just index)
## Testing Strategy
### ⚠️ Incremental Workflow (CRITICAL for Multi-File)
- [ ] **NEVER generate all tests at once** - process one file at a time
- [ ] Order files by complexity: utilities → hooks → simple → complex → integration
- [ ] Create a todo list to track progress before starting
- [ ] For EACH file: write → run test → verify pass → then next
- [ ] **DO NOT proceed** to next file until current one passes
### Path-Level Coverage
- [ ] **Test ALL files** in the assigned directory/path
- [ ] List all components, hooks, utilities that need coverage
- [ ] Decide: single spec file (integration) or multiple spec files (unit)
### Complexity Assessment
- [ ] Run `pnpm analyze-component <path>` for complexity score
- [ ] **Complexity > 50**: Consider refactoring before testing
- [ ] **500+ lines**: Consider splitting before testing
- [ ] **30-50 complexity**: Use multiple describe blocks, organized structure
### Integration vs Mocking
- [ ] **DO NOT mock base components or dify-ui primitives** (base `Loading`, `Input`, `Badge`; dify-ui `Button`, `Tooltip`, `Dialog`, etc.)
- [ ] Import real project components instead of mocking
- [ ] Only mock: API calls, complex context providers, third-party libs with side effects
- [ ] Prefer integration testing when using single spec file
## Required Test Sections
### All Components MUST Have
- [ ] **Rendering tests** - Component renders without crashing
- [ ] **Props tests** - Required props, optional props, default values
- [ ] **Edge cases** - null, undefined, empty values, boundaries
### Conditional Sections (Add When Feature Present)
| Feature | Add Tests For |
|---------|---------------|
| `useState` | Initial state, transitions, cleanup |
| `useEffect` | Execution, dependencies, cleanup |
| Event handlers | onClick, onChange, onSubmit, keyboard |
| API calls | Loading, success, error states |
| Routing | Navigation, params, query strings |
| `useCallback`/`useMemo` | Referential equality |
| Context | Provider values, consumer behavior |
| Forms | Validation, submission, error display |
## Code Quality Checklist
### Structure
- [ ] Uses `describe` blocks to group related tests
- [ ] Test names follow `should <behavior> when <condition>` pattern
- [ ] AAA pattern (Arrange-Act-Assert) is clear
- [ ] Comments explain complex test scenarios
### Mocks
- [ ] **DO NOT mock base components or dify-ui primitives** (`@/app/components/base/*` or `@langgenius/dify-ui/*`)
- [ ] `vi.clearAllMocks()` in `beforeEach` (not `afterEach`)
- [ ] Shared mock state reset in `beforeEach`
- [ ] i18n uses global mock (auto-loaded in `web/vitest.setup.ts`); only override locally for custom translations
- [ ] Router mocks match actual Next.js API
- [ ] Mocks reflect actual component conditional behavior
- [ ] Only mock: API services, complex context providers, third-party libs
- [ ] For `nuqs` URL-state tests, wrap with `NuqsTestingAdapter` (prefer `web/test/nuqs-testing.tsx`)
- [ ] For `nuqs` URL-state tests, assert `onUrlUpdate` payload (`searchParams`, `options.history`)
- [ ] If custom `nuqs` parser exists, add round-trip tests for encoded edge cases (`%2F`, `%25`, spaces, legacy encoded values)
### Queries
- [ ] Prefer semantic queries (`getByRole`, `getByLabelText`)
- [ ] Use `queryBy*` for absence assertions
- [ ] Use `findBy*` for async elements
- [ ] `getByTestId` only as last resort
### Async
- [ ] All async tests use `async/await`
- [ ] `waitFor` wraps async assertions
- [ ] Fake timers properly setup/teardown
- [ ] No floating promises
### TypeScript
- [ ] No `any` types without justification
- [ ] Mock data uses actual types from source
- [ ] Factory functions have proper return types
## Coverage Goals (Per File)
For the current file being tested:
- [ ] 100% function coverage
- [ ] 100% statement coverage
- [ ] >95% branch coverage
- [ ] >95% line coverage
## Post-Generation (Per File)
**Run these checks after EACH test file, not just at the end:**
- [ ] Run `pnpm test path/to/file.spec.tsx` - **MUST PASS before next file**
- [ ] Fix any failures immediately
- [ ] Mark file as complete in todo list
- [ ] Only then proceed to next file
### After All Files Complete
- [ ] Run full directory test: `pnpm test path/to/directory/`
- [ ] Check coverage report: `pnpm test:coverage`
- [ ] Run `pnpm lint:fix` on all test files
- [ ] Run `pnpm type-check`
## Common Issues to Watch
### False Positives
```typescript
// ❌ Mock doesn't match actual behavior
vi.mock('./Component', () => () => <div>Mocked</div>)
// ✅ Mock matches actual conditional logic
vi.mock('./Component', () => ({ isOpen }: any) =>
isOpen ? <div>Content</div> : null
)
```
### State Leakage
```typescript
// ❌ Shared state not reset
let mockState = false
vi.mock('./useHook', () => () => mockState)
// ✅ Reset in beforeEach
beforeEach(() => {
mockState = false
})
```
### Async Race Conditions
```typescript
// ❌ Not awaited
it('loads data', () => {
render(<Component />)
expect(screen.getByText('Data')).toBeInTheDocument()
})
// ✅ Properly awaited
it('loads data', async () => {
render(<Component />)
await waitFor(() => {
expect(screen.getByText('Data')).toBeInTheDocument()
})
})
```
### Missing Edge Cases
Always test these scenarios:
- `null` / `undefined` inputs
- Empty strings / arrays / objects
- Boundary values (0, -1, MAX_INT)
- Error states
- Loading states
- Disabled states
## Quick Commands
```bash
# Run specific test
pnpm test path/to/file.spec.tsx
# Run with coverage
pnpm test:coverage path/to/file.spec.tsx
# Watch mode
pnpm test:watch path/to/file.spec.tsx
# Update snapshots (use sparingly)
pnpm test -u path/to/file.spec.tsx
# Analyze component
pnpm analyze-component path/to/component.tsx
# Review existing test
pnpm analyze-component path/to/component.tsx --review
```
@@ -0,0 +1,449 @@
# Common Testing Patterns
## Query Priority
Use queries in this order (most to least preferred):
```typescript
// 1. getByRole - Most recommended (accessibility)
screen.getByRole('button', { name: /submit/i })
screen.getByRole('textbox', { name: /email/i })
screen.getByRole('heading', { level: 1 })
// 2. getByLabelText - Form fields
screen.getByLabelText('Email address')
screen.getByLabelText(/password/i)
// 3. getByPlaceholderText - When no label
screen.getByPlaceholderText('Search...')
// 4. getByText - Non-interactive elements
screen.getByText('Welcome to Dify')
screen.getByText(/loading/i)
// 5. getByDisplayValue - Current input value
screen.getByDisplayValue('current value')
// 6. getByAltText - Images
screen.getByAltText('Company logo')
// 7. getByTitle - Tooltip elements
screen.getByTitle('Close')
// 8. getByTestId - Last resort only!
screen.getByTestId('custom-element')
```
## Event Handling Patterns
### Click Events
```typescript
// Basic click
fireEvent.click(screen.getByRole('button'))
// With userEvent (preferred for realistic interaction)
const user = userEvent.setup()
await user.click(screen.getByRole('button'))
// Double click
await user.dblClick(screen.getByRole('button'))
// Right click
await user.pointer({ keys: '[MouseRight]', target: screen.getByRole('button') })
```
### Form Input
```typescript
const user = userEvent.setup()
// Type in input
await user.type(screen.getByRole('textbox'), 'Hello World')
// Clear and type
await user.clear(screen.getByRole('textbox'))
await user.type(screen.getByRole('textbox'), 'New value')
// Select option
await user.selectOptions(screen.getByRole('combobox'), 'option-value')
// Check checkbox
await user.click(screen.getByRole('checkbox'))
// Upload file
const file = new File(['content'], 'test.pdf', { type: 'application/pdf' })
await user.upload(screen.getByLabelText(/upload/i), file)
```
### Keyboard Events
```typescript
const user = userEvent.setup()
// Press Enter
await user.keyboard('{Enter}')
// Press Escape
await user.keyboard('{Escape}')
// Keyboard shortcut
await user.keyboard('{Control>}a{/Control}') // Ctrl+A
// Tab navigation
await user.tab()
// Arrow keys
await user.keyboard('{ArrowDown}')
await user.keyboard('{ArrowUp}')
```
## Component State Testing
### Testing State Transitions
```typescript
describe('Counter', () => {
it('should increment count', async () => {
const user = userEvent.setup()
render(<Counter initialCount={0} />)
// Initial state
expect(screen.getByText('Count: 0')).toBeInTheDocument()
// Trigger transition
await user.click(screen.getByRole('button', { name: /increment/i }))
// New state
expect(screen.getByText('Count: 1')).toBeInTheDocument()
})
})
```
### Testing Controlled Components
```typescript
describe('ControlledInput', () => {
it('should call onChange with new value', async () => {
const user = userEvent.setup()
const handleChange = vi.fn()
render(<ControlledInput value="" onChange={handleChange} />)
await user.type(screen.getByRole('textbox'), 'a')
expect(handleChange).toHaveBeenCalledWith('a')
})
it('should display controlled value', () => {
render(<ControlledInput value="controlled" onChange={vi.fn()} />)
expect(screen.getByRole('textbox')).toHaveValue('controlled')
})
})
```
## Conditional Rendering Testing
```typescript
describe('ConditionalComponent', () => {
it('should show loading state', () => {
render(<DataDisplay isLoading={true} data={null} />)
expect(screen.getByText(/loading/i)).toBeInTheDocument()
expect(screen.queryByTestId('data-content')).not.toBeInTheDocument()
})
it('should show error state', () => {
render(<DataDisplay isLoading={false} data={null} error="Failed to load" />)
expect(screen.getByText(/failed to load/i)).toBeInTheDocument()
})
it('should show data when loaded', () => {
render(<DataDisplay isLoading={false} data={{ name: 'Test' }} />)
expect(screen.getByText('Test')).toBeInTheDocument()
})
it('should show empty state when no data', () => {
render(<DataDisplay isLoading={false} data={[]} />)
expect(screen.getByText(/no data/i)).toBeInTheDocument()
})
})
```
## List Rendering Testing
```typescript
describe('ItemList', () => {
const items = [
{ id: '1', name: 'Item 1' },
{ id: '2', name: 'Item 2' },
{ id: '3', name: 'Item 3' },
]
it('should render all items', () => {
render(<ItemList items={items} />)
expect(screen.getAllByRole('listitem')).toHaveLength(3)
items.forEach(item => {
expect(screen.getByText(item.name)).toBeInTheDocument()
})
})
it('should handle item selection', async () => {
const user = userEvent.setup()
const onSelect = vi.fn()
render(<ItemList items={items} onSelect={onSelect} />)
await user.click(screen.getByText('Item 2'))
expect(onSelect).toHaveBeenCalledWith(items[1])
})
it('should handle empty list', () => {
render(<ItemList items={[]} />)
expect(screen.getByText(/no items/i)).toBeInTheDocument()
})
})
```
## Modal/Dialog Testing
```typescript
describe('Modal', () => {
it('should not render when closed', () => {
render(<Modal isOpen={false} onClose={vi.fn()} />)
expect(screen.queryByRole('dialog')).not.toBeInTheDocument()
})
it('should render when open', () => {
render(<Modal isOpen={true} onClose={vi.fn()} />)
expect(screen.getByRole('dialog')).toBeInTheDocument()
})
it('should call onClose when clicking overlay', async () => {
const user = userEvent.setup()
const handleClose = vi.fn()
render(<Modal isOpen={true} onClose={handleClose} />)
await user.click(screen.getByTestId('modal-overlay'))
expect(handleClose).toHaveBeenCalled()
})
it('should call onClose when pressing Escape', async () => {
const user = userEvent.setup()
const handleClose = vi.fn()
render(<Modal isOpen={true} onClose={handleClose} />)
await user.keyboard('{Escape}')
expect(handleClose).toHaveBeenCalled()
})
it('should trap focus inside modal', async () => {
const user = userEvent.setup()
render(
<Modal isOpen={true} onClose={vi.fn()}>
<button>First</button>
<button>Second</button>
</Modal>
)
// Focus should cycle within modal
await user.tab()
expect(screen.getByText('First')).toHaveFocus()
await user.tab()
expect(screen.getByText('Second')).toHaveFocus()
await user.tab()
expect(screen.getByText('First')).toHaveFocus() // Cycles back
})
})
```
## Form Testing
```typescript
describe('LoginForm', () => {
it('should submit valid form', async () => {
const user = userEvent.setup()
const onSubmit = vi.fn()
render(<LoginForm onSubmit={onSubmit} />)
await user.type(screen.getByLabelText(/email/i), 'test@example.com')
await user.type(screen.getByLabelText(/password/i), 'password123')
await user.click(screen.getByRole('button', { name: /sign in/i }))
expect(onSubmit).toHaveBeenCalledWith({
email: 'test@example.com',
password: 'password123',
})
})
it('should show validation errors', async () => {
const user = userEvent.setup()
render(<LoginForm onSubmit={vi.fn()} />)
// Submit empty form
await user.click(screen.getByRole('button', { name: /sign in/i }))
expect(screen.getByText(/email is required/i)).toBeInTheDocument()
expect(screen.getByText(/password is required/i)).toBeInTheDocument()
})
it('should validate email format', async () => {
const user = userEvent.setup()
render(<LoginForm onSubmit={vi.fn()} />)
await user.type(screen.getByLabelText(/email/i), 'invalid-email')
await user.click(screen.getByRole('button', { name: /sign in/i }))
expect(screen.getByText(/invalid email/i)).toBeInTheDocument()
})
it('should disable submit button while submitting', async () => {
const user = userEvent.setup()
const onSubmit = vi.fn(() => new Promise(resolve => setTimeout(resolve, 100)))
render(<LoginForm onSubmit={onSubmit} />)
await user.type(screen.getByLabelText(/email/i), 'test@example.com')
await user.type(screen.getByLabelText(/password/i), 'password123')
await user.click(screen.getByRole('button', { name: /sign in/i }))
expect(screen.getByRole('button', { name: /signing in/i })).toBeDisabled()
await waitFor(() => {
expect(screen.getByRole('button', { name: /sign in/i })).toBeEnabled()
})
})
})
```
## Data-Driven Tests with test.each
```typescript
describe('StatusBadge', () => {
test.each([
['success', 'bg-green-500'],
['warning', 'bg-yellow-500'],
['error', 'bg-red-500'],
['info', 'bg-blue-500'],
])('should apply correct class for %s status', (status, expectedClass) => {
render(<StatusBadge status={status} />)
expect(screen.getByTestId('status-badge')).toHaveClass(expectedClass)
})
test.each([
{ input: null, expected: 'Unknown' },
{ input: undefined, expected: 'Unknown' },
{ input: '', expected: 'Unknown' },
{ input: 'invalid', expected: 'Unknown' },
])('should show "Unknown" for invalid input: $input', ({ input, expected }) => {
render(<StatusBadge status={input} />)
expect(screen.getByText(expected)).toBeInTheDocument()
})
})
```
## Debugging Tips
```typescript
// Print entire DOM
screen.debug()
// Print specific element
screen.debug(screen.getByRole('button'))
// Log testing playground URL
screen.logTestingPlaygroundURL()
// Pretty print DOM
import { prettyDOM } from '@testing-library/react'
console.log(prettyDOM(screen.getByRole('dialog')))
// Check available roles
import { getRoles } from '@testing-library/react'
console.log(getRoles(container))
```
## Common Mistakes to Avoid
### ❌ Don't Use Implementation Details
```typescript
// Bad - testing implementation
expect(component.state.isOpen).toBe(true)
expect(wrapper.find('.internal-class').length).toBe(1)
// Good - testing behavior
expect(screen.getByRole('dialog')).toBeInTheDocument()
```
### ❌ Don't Forget Cleanup
```typescript
// Bad - may leak state between tests
it('test 1', () => {
render(<Component />)
})
// Good - cleanup is automatic with RTL, but reset mocks
beforeEach(() => {
vi.clearAllMocks()
})
```
### ❌ Don't Use Exact String Matching (Prefer Black-Box Assertions)
```typescript
// ❌ Bad - hardcoded strings are brittle
expect(screen.getByText('Submit Form')).toBeInTheDocument()
expect(screen.getByText('Loading...')).toBeInTheDocument()
// ✅ Good - role-based queries (most semantic)
expect(screen.getByRole('button', { name: /submit/i })).toBeInTheDocument()
expect(screen.getByRole('status')).toBeInTheDocument()
// ✅ Good - pattern matching (flexible)
expect(screen.getByText(/submit/i)).toBeInTheDocument()
expect(screen.getByText(/loading/i)).toBeInTheDocument()
// ✅ Good - test behavior, not exact UI text
expect(screen.getByRole('button')).toBeDisabled()
expect(screen.getByRole('alert')).toBeInTheDocument()
```
**Why prefer black-box assertions?**
- Text content may change (i18n, copy updates)
- Role-based queries test accessibility
- Pattern matching is resilient to minor changes
- Tests focus on behavior, not implementation details
### ❌ Don't Assert on Absence Without Query
```typescript
// Bad - throws if not found
expect(screen.getByText('Error')).not.toBeInTheDocument() // Error!
// Good - use queryBy for absence assertions
expect(screen.queryByText('Error')).not.toBeInTheDocument()
```
@@ -0,0 +1,523 @@
# Domain-Specific Component Testing
This guide covers testing patterns for Dify's domain-specific components.
## Workflow Components (`workflow/`)
Workflow components handle node configuration, data flow, and graph operations.
### Key Test Areas
1. **Node Configuration**
1. **Data Validation**
1. **Variable Passing**
1. **Edge Connections**
1. **Error Handling**
### Example: Node Configuration Panel
```typescript
import { render, screen, fireEvent, waitFor } from '@testing-library/react'
import userEvent from '@testing-library/user-event'
import NodeConfigPanel from './node-config-panel'
import { createMockNode, createMockWorkflowContext } from '@/__mocks__/workflow'
// Mock workflow context
vi.mock('@/app/components/workflow/hooks', () => ({
useWorkflowStore: () => mockWorkflowStore,
useNodesInteractions: () => mockNodesInteractions,
}))
let mockWorkflowStore = {
nodes: [],
edges: [],
updateNode: vi.fn(),
}
let mockNodesInteractions = {
handleNodeSelect: vi.fn(),
handleNodeDelete: vi.fn(),
}
describe('NodeConfigPanel', () => {
beforeEach(() => {
vi.clearAllMocks()
mockWorkflowStore = {
nodes: [],
edges: [],
updateNode: vi.fn(),
}
})
describe('Node Configuration', () => {
it('should render node type selector', () => {
const node = createMockNode({ type: 'llm' })
render(<NodeConfigPanel node={node} />)
expect(screen.getByLabelText(/model/i)).toBeInTheDocument()
})
it('should update node config on change', async () => {
const user = userEvent.setup()
const node = createMockNode({ type: 'llm' })
render(<NodeConfigPanel node={node} />)
await user.selectOptions(screen.getByLabelText(/model/i), 'gpt-4')
expect(mockWorkflowStore.updateNode).toHaveBeenCalledWith(
node.id,
expect.objectContaining({ model: 'gpt-4' })
)
})
})
describe('Data Validation', () => {
it('should show error for invalid input', async () => {
const user = userEvent.setup()
const node = createMockNode({ type: 'code' })
render(<NodeConfigPanel node={node} />)
// Enter invalid code
const codeInput = screen.getByLabelText(/code/i)
await user.clear(codeInput)
await user.type(codeInput, 'invalid syntax {{{')
await waitFor(() => {
expect(screen.getByText(/syntax error/i)).toBeInTheDocument()
})
})
it('should validate required fields', async () => {
const node = createMockNode({ type: 'http', data: { url: '' } })
render(<NodeConfigPanel node={node} />)
fireEvent.click(screen.getByRole('button', { name: /save/i }))
await waitFor(() => {
expect(screen.getByText(/url is required/i)).toBeInTheDocument()
})
})
})
describe('Variable Passing', () => {
it('should display available variables from upstream nodes', () => {
const upstreamNode = createMockNode({
id: 'node-1',
type: 'start',
data: { outputs: [{ name: 'user_input', type: 'string' }] },
})
const currentNode = createMockNode({
id: 'node-2',
type: 'llm',
})
mockWorkflowStore.nodes = [upstreamNode, currentNode]
mockWorkflowStore.edges = [{ source: 'node-1', target: 'node-2' }]
render(<NodeConfigPanel node={currentNode} />)
// Variable selector should show upstream variables
fireEvent.click(screen.getByRole('button', { name: /add variable/i }))
expect(screen.getByText('user_input')).toBeInTheDocument()
})
it('should insert variable into prompt template', async () => {
const user = userEvent.setup()
const node = createMockNode({ type: 'llm' })
render(<NodeConfigPanel node={node} />)
// Click variable button
await user.click(screen.getByRole('button', { name: /insert variable/i }))
await user.click(screen.getByText('user_input'))
const promptInput = screen.getByLabelText(/prompt/i)
expect(promptInput).toHaveValue(expect.stringContaining('{{user_input}}'))
})
})
})
```
## Dataset Components (`dataset/`)
Dataset components handle file uploads, data display, and search/filter operations.
### Key Test Areas
1. **File Upload**
1. **File Type Validation**
1. **Pagination**
1. **Search & Filtering**
1. **Data Format Handling**
### Example: Document Uploader
```typescript
import { render, screen, fireEvent, waitFor } from '@testing-library/react'
import userEvent from '@testing-library/user-event'
import DocumentUploader from './document-uploader'
vi.mock('@/service/datasets', () => ({
uploadDocument: vi.fn(),
parseDocument: vi.fn(),
}))
import * as datasetService from '@/service/datasets'
const mockedService = vi.mocked(datasetService)
describe('DocumentUploader', () => {
beforeEach(() => {
vi.clearAllMocks()
})
describe('File Upload', () => {
it('should accept valid file types', async () => {
const user = userEvent.setup()
const onUpload = vi.fn()
mockedService.uploadDocument.mockResolvedValue({ id: 'doc-1' })
render(<DocumentUploader onUpload={onUpload} />)
const file = new File(['content'], 'test.pdf', { type: 'application/pdf' })
const input = screen.getByLabelText(/upload/i)
await user.upload(input, file)
await waitFor(() => {
expect(mockedService.uploadDocument).toHaveBeenCalledWith(
expect.any(FormData)
)
})
})
it('should reject invalid file types', async () => {
const user = userEvent.setup()
render(<DocumentUploader />)
const file = new File(['content'], 'test.exe', { type: 'application/x-msdownload' })
const input = screen.getByLabelText(/upload/i)
await user.upload(input, file)
expect(screen.getByText(/unsupported file type/i)).toBeInTheDocument()
expect(mockedService.uploadDocument).not.toHaveBeenCalled()
})
it('should show upload progress', async () => {
const user = userEvent.setup()
// Mock upload with progress
mockedService.uploadDocument.mockImplementation(() => {
return new Promise((resolve) => {
setTimeout(() => resolve({ id: 'doc-1' }), 100)
})
})
render(<DocumentUploader />)
const file = new File(['content'], 'test.pdf', { type: 'application/pdf' })
await user.upload(screen.getByLabelText(/upload/i), file)
expect(screen.getByRole('progressbar')).toBeInTheDocument()
await waitFor(() => {
expect(screen.queryByRole('progressbar')).not.toBeInTheDocument()
})
})
})
describe('Error Handling', () => {
it('should handle upload failure', async () => {
const user = userEvent.setup()
mockedService.uploadDocument.mockRejectedValue(new Error('Upload failed'))
render(<DocumentUploader />)
const file = new File(['content'], 'test.pdf', { type: 'application/pdf' })
await user.upload(screen.getByLabelText(/upload/i), file)
await waitFor(() => {
expect(screen.getByText(/upload failed/i)).toBeInTheDocument()
})
})
it('should allow retry after failure', async () => {
const user = userEvent.setup()
mockedService.uploadDocument
.mockRejectedValueOnce(new Error('Network error'))
.mockResolvedValueOnce({ id: 'doc-1' })
render(<DocumentUploader />)
const file = new File(['content'], 'test.pdf', { type: 'application/pdf' })
await user.upload(screen.getByLabelText(/upload/i), file)
await waitFor(() => {
expect(screen.getByRole('button', { name: /retry/i })).toBeInTheDocument()
})
await user.click(screen.getByRole('button', { name: /retry/i }))
await waitFor(() => {
expect(screen.getByText(/uploaded successfully/i)).toBeInTheDocument()
})
})
})
})
```
### Example: Document List with Pagination
```typescript
describe('DocumentList', () => {
describe('Pagination', () => {
it('should load first page on mount', async () => {
mockedService.getDocuments.mockResolvedValue({
data: [{ id: '1', name: 'Doc 1' }],
total: 50,
page: 1,
pageSize: 10,
})
render(<DocumentList datasetId="ds-1" />)
await waitFor(() => {
expect(screen.getByText('Doc 1')).toBeInTheDocument()
})
expect(mockedService.getDocuments).toHaveBeenCalledWith('ds-1', { page: 1 })
})
it('should navigate to next page', async () => {
const user = userEvent.setup()
mockedService.getDocuments.mockResolvedValue({
data: [{ id: '1', name: 'Doc 1' }],
total: 50,
page: 1,
pageSize: 10,
})
render(<DocumentList datasetId="ds-1" />)
await waitFor(() => {
expect(screen.getByText('Doc 1')).toBeInTheDocument()
})
mockedService.getDocuments.mockResolvedValue({
data: [{ id: '11', name: 'Doc 11' }],
total: 50,
page: 2,
pageSize: 10,
})
await user.click(screen.getByRole('button', { name: /next/i }))
await waitFor(() => {
expect(screen.getByText('Doc 11')).toBeInTheDocument()
})
})
})
describe('Search & Filtering', () => {
it('should filter by search query', async () => {
const user = userEvent.setup()
vi.useFakeTimers()
render(<DocumentList datasetId="ds-1" />)
await user.type(screen.getByPlaceholderText(/search/i), 'test query')
// Debounce
vi.advanceTimersByTime(300)
await waitFor(() => {
expect(mockedService.getDocuments).toHaveBeenCalledWith(
'ds-1',
expect.objectContaining({ search: 'test query' })
)
})
vi.useRealTimers()
})
})
})
```
## Configuration Components (`app/configuration/`, `config/`)
Configuration components handle forms, validation, and data persistence.
### Key Test Areas
1. **Form Validation**
1. **Save/Reset**
1. **Required vs Optional Fields**
1. **Configuration Persistence**
1. **Error Feedback**
### Example: App Configuration Form
```typescript
import { render, screen, fireEvent, waitFor } from '@testing-library/react'
import userEvent from '@testing-library/user-event'
import AppConfigForm from './app-config-form'
vi.mock('@/service/apps', () => ({
updateAppConfig: vi.fn(),
getAppConfig: vi.fn(),
}))
import * as appService from '@/service/apps'
const mockedService = vi.mocked(appService)
describe('AppConfigForm', () => {
const defaultConfig = {
name: 'My App',
description: '',
icon: 'default',
openingStatement: '',
}
beforeEach(() => {
vi.clearAllMocks()
mockedService.getAppConfig.mockResolvedValue(defaultConfig)
})
describe('Form Validation', () => {
it('should require app name', async () => {
const user = userEvent.setup()
render(<AppConfigForm appId="app-1" />)
await waitFor(() => {
expect(screen.getByLabelText(/name/i)).toHaveValue('My App')
})
// Clear name field
await user.clear(screen.getByLabelText(/name/i))
await user.click(screen.getByRole('button', { name: /save/i }))
expect(screen.getByText(/name is required/i)).toBeInTheDocument()
expect(mockedService.updateAppConfig).not.toHaveBeenCalled()
})
it('should validate name length', async () => {
const user = userEvent.setup()
render(<AppConfigForm appId="app-1" />)
await waitFor(() => {
expect(screen.getByLabelText(/name/i)).toBeInTheDocument()
})
// Enter very long name
await user.clear(screen.getByLabelText(/name/i))
await user.type(screen.getByLabelText(/name/i), 'a'.repeat(101))
expect(screen.getByText(/name must be less than 100 characters/i)).toBeInTheDocument()
})
it('should allow empty optional fields', async () => {
const user = userEvent.setup()
mockedService.updateAppConfig.mockResolvedValue({ success: true })
render(<AppConfigForm appId="app-1" />)
await waitFor(() => {
expect(screen.getByLabelText(/name/i)).toHaveValue('My App')
})
// Leave description empty (optional)
await user.click(screen.getByRole('button', { name: /save/i }))
await waitFor(() => {
expect(mockedService.updateAppConfig).toHaveBeenCalled()
})
})
})
describe('Save/Reset Functionality', () => {
it('should save configuration', async () => {
const user = userEvent.setup()
mockedService.updateAppConfig.mockResolvedValue({ success: true })
render(<AppConfigForm appId="app-1" />)
await waitFor(() => {
expect(screen.getByLabelText(/name/i)).toHaveValue('My App')
})
await user.clear(screen.getByLabelText(/name/i))
await user.type(screen.getByLabelText(/name/i), 'Updated App')
await user.click(screen.getByRole('button', { name: /save/i }))
await waitFor(() => {
expect(mockedService.updateAppConfig).toHaveBeenCalledWith(
'app-1',
expect.objectContaining({ name: 'Updated App' })
)
})
expect(screen.getByText(/saved successfully/i)).toBeInTheDocument()
})
it('should reset to default values', async () => {
const user = userEvent.setup()
render(<AppConfigForm appId="app-1" />)
await waitFor(() => {
expect(screen.getByLabelText(/name/i)).toHaveValue('My App')
})
// Make changes
await user.clear(screen.getByLabelText(/name/i))
await user.type(screen.getByLabelText(/name/i), 'Changed Name')
// Reset
await user.click(screen.getByRole('button', { name: /reset/i }))
expect(screen.getByLabelText(/name/i)).toHaveValue('My App')
})
it('should show unsaved changes warning', async () => {
const user = userEvent.setup()
render(<AppConfigForm appId="app-1" />)
await waitFor(() => {
expect(screen.getByLabelText(/name/i)).toHaveValue('My App')
})
// Make changes
await user.type(screen.getByLabelText(/name/i), ' Updated')
expect(screen.getByText(/unsaved changes/i)).toBeInTheDocument()
})
})
describe('Error Handling', () => {
it('should show error on save failure', async () => {
const user = userEvent.setup()
mockedService.updateAppConfig.mockRejectedValue(new Error('Server error'))
render(<AppConfigForm appId="app-1" />)
await waitFor(() => {
expect(screen.getByLabelText(/name/i)).toHaveValue('My App')
})
await user.click(screen.getByRole('button', { name: /save/i }))
await waitFor(() => {
expect(screen.getByText(/failed to save/i)).toBeInTheDocument()
})
})
})
})
```
@@ -0,0 +1,535 @@
# Mocking Guide for Dify Frontend Tests
## ⚠️ Important: What NOT to Mock
### DO NOT Mock Base Components or dify-ui Primitives
**Never mock components from `@/app/components/base/` or from `@langgenius/dify-ui/*`** such as:
- Legacy base (`@/app/components/base/*`): `Loading`, `Spinner`, `Input`, `Badge`, `Tag`
- dify-ui primitives (`@langgenius/dify-ui/*`): `Button`, `Tooltip`, `Dialog`, `Popover`, `DropdownMenu`, `ContextMenu`, `Select`, `AlertDialog`, `Toast`
**Why?**
- These components have their own dedicated tests
- Mocking them creates false positives (tests pass but real integration fails)
- Using real components tests actual integration behavior
```typescript
// ❌ WRONG: Don't mock base components or dify-ui primitives
vi.mock('@/app/components/base/loading', () => () => <div>Loading</div>)
vi.mock('@langgenius/dify-ui/button', () => ({ Button: ({ children }: any) => <button>{children}</button> }))
// ✅ CORRECT: Import and use the real components
import Loading from '@/app/components/base/loading'
import { Button } from '@langgenius/dify-ui/button'
// They will render normally in tests
```
### What TO Mock
Only mock these categories:
1. **API services** (`@/service/*`) - Network calls
1. **Complex context providers** - When setup is too difficult
1. **Third-party libraries with side effects** - `next/navigation`, external SDKs
1. **i18n** - Always mock to return keys
### Zustand Stores - DO NOT Mock Manually
**Zustand is globally mocked** in `web/vitest.setup.ts`. Use real stores with `setState()`:
```typescript
// ✅ CORRECT: Use real store, set test state
import { useAppStore } from '@/app/components/app/store'
useAppStore.setState({ appDetail: { id: 'test', name: 'Test' } })
render(<MyComponent />)
// ❌ WRONG: Don't mock the store module
vi.mock('@/app/components/app/store', () => ({ ... }))
```
See [Zustand Store Testing](#zustand-store-testing) section for full details.
## Mock Placement
| Location | Purpose |
|----------|---------|
| `web/vitest.setup.ts` | Global mocks shared by all tests (`react-i18next`, `zustand`, clipboard, FloatingPortal, Monaco, localStorage`) |
| `web/__mocks__/zustand.ts` | Zustand mock implementation (auto-resets stores after each test) |
| `web/__mocks__/` | Reusable mock factories shared across multiple test files |
| Test file | Test-specific mocks, inline with `vi.mock()` |
Modules are not mocked automatically. Use `vi.mock` in test files, or add global mocks in `web/vitest.setup.ts`.
**Note**: Zustand is special - it's globally mocked but you should NOT mock store modules manually. See [Zustand Store Testing](#zustand-store-testing).
## Essential Mocks
### 1. i18n (Auto-loaded via Global Mock)
A global mock is defined in `web/vitest.setup.ts` and is auto-loaded by Vitest setup.
The global mock provides:
- `useTranslation` - returns translation keys with namespace prefix
- `Trans` component - renders i18nKey and components
- `useMixedTranslation` (from `@/app/components/plugins/marketplace/hooks`)
- `useGetLanguage` (from `@/context/i18n`) - returns `'en-US'`
**Default behavior**: Most tests should use the global mock (no local override needed).
**For custom translations**: Use the helper function from `@/test/i18n-mock`:
```typescript
import { createReactI18nextMock } from '@/test/i18n-mock'
vi.mock('react-i18next', () => createReactI18nextMock({
'my.custom.key': 'Custom translation',
'button.save': 'Save',
}))
```
**Avoid**: Manually defining `useTranslation` mocks that just return the key - the global mock already does this.
### 2. Next.js Router
```typescript
const mockPush = vi.fn()
const mockReplace = vi.fn()
vi.mock('next/navigation', () => ({
useRouter: () => ({
push: mockPush,
replace: mockReplace,
back: vi.fn(),
prefetch: vi.fn(),
}),
usePathname: () => '/current-path',
useSearchParams: () => new URLSearchParams('?key=value'),
}))
describe('Component', () => {
beforeEach(() => {
vi.clearAllMocks()
})
it('should navigate on click', () => {
render(<Component />)
fireEvent.click(screen.getByRole('button'))
expect(mockPush).toHaveBeenCalledWith('/expected-path')
})
})
```
### 2.1 `nuqs` Query State (Preferred: Testing Adapter)
For tests that validate URL query behavior, use `NuqsTestingAdapter` instead of mocking `nuqs` directly.
```typescript
import { renderHookWithNuqs } from '@/test/nuqs-testing'
it('should sync query to URL with push history', async () => {
const { result, onUrlUpdate } = renderHookWithNuqs(() => useMyQueryState(), {
searchParams: '?page=1',
})
act(() => {
result.current.setQuery({ page: 2 })
})
await waitFor(() => expect(onUrlUpdate).toHaveBeenCalled())
const update = onUrlUpdate.mock.calls[onUrlUpdate.mock.calls.length - 1][0]
expect(update.options.history).toBe('push')
expect(update.searchParams.get('page')).toBe('2')
})
```
Use direct `vi.mock('nuqs')` only when URL synchronization is intentionally out of scope.
### 3. Portal Components (with Shared State)
```typescript
// ⚠️ Important: Use shared state for components that depend on each other
let mockPortalOpenState = false
vi.mock('@/app/components/base/portal-to-follow-elem', () => ({
PortalToFollowElem: ({ children, open, ...props }: any) => {
mockPortalOpenState = open || false // Update shared state
return <div data-testid="portal" data-open={open}>{children}</div>
},
PortalToFollowElemContent: ({ children }: any) => {
// ✅ Matches actual: returns null when portal is closed
if (!mockPortalOpenState) return null
return <div data-testid="portal-content">{children}</div>
},
PortalToFollowElemTrigger: ({ children }: any) => (
<div data-testid="portal-trigger">{children}</div>
),
}))
describe('Component', () => {
beforeEach(() => {
vi.clearAllMocks()
mockPortalOpenState = false // ✅ Reset shared state
})
})
```
### 4. API Service Mocks
```typescript
import * as api from '@/service/api'
vi.mock('@/service/api')
const mockedApi = vi.mocked(api)
describe('Component', () => {
beforeEach(() => {
vi.clearAllMocks()
// Setup default mock implementation
mockedApi.fetchData.mockResolvedValue({ data: [] })
})
it('should show data on success', async () => {
mockedApi.fetchData.mockResolvedValue({ data: [{ id: 1 }] })
render(<Component />)
await waitFor(() => {
expect(screen.getByText('1')).toBeInTheDocument()
})
})
it('should show error on failure', async () => {
mockedApi.fetchData.mockRejectedValue(new Error('Network error'))
render(<Component />)
await waitFor(() => {
expect(screen.getByText(/error/i)).toBeInTheDocument()
})
})
})
```
### 5. HTTP and `fetch` Mocking
```typescript
describe('GithubComponent', () => {
beforeEach(() => {
vi.clearAllMocks()
})
it('should display repo info', async () => {
vi.mocked(globalThis.fetch).mockResolvedValueOnce(
new Response(JSON.stringify({ name: 'dify', stars: 1000 }), {
status: 200,
headers: { 'Content-Type': 'application/json' },
}),
)
render(<GithubComponent />)
await waitFor(() => {
expect(screen.getByText('dify')).toBeInTheDocument()
})
})
it('should handle API error', async () => {
vi.mocked(globalThis.fetch).mockResolvedValueOnce(
new Response(JSON.stringify({ message: 'Server error' }), {
status: 500,
headers: { 'Content-Type': 'application/json' },
}),
)
render(<GithubComponent />)
await waitFor(() => {
expect(screen.getByText(/error/i)).toBeInTheDocument()
})
})
})
```
Prefer mocking `@/service/*` modules or spying on `global.fetch` / `ky` clients with deterministic responses. Do not introduce an HTTP interception dependency such as `nock` or MSW unless it is already declared in the workspace or adding it is part of the task.
### 6. Context Providers
```typescript
import { ProviderContext } from '@/context/provider-context'
import { createMockProviderContextValue, createMockPlan } from '@/__mocks__/provider-context'
describe('Component with Context', () => {
it('should render for free plan', () => {
const mockContext = createMockPlan('sandbox')
render(
<ProviderContext.Provider value={mockContext}>
<Component />
</ProviderContext.Provider>
)
expect(screen.getByText('Upgrade')).toBeInTheDocument()
})
it('should render for pro plan', () => {
const mockContext = createMockPlan('professional')
render(
<ProviderContext.Provider value={mockContext}>
<Component />
</ProviderContext.Provider>
)
expect(screen.queryByText('Upgrade')).not.toBeInTheDocument()
})
})
```
### 7. React Query
```typescript
import { QueryClient, QueryClientProvider } from '@tanstack/react-query'
const createTestQueryClient = () => new QueryClient({
defaultOptions: {
queries: { retry: false },
mutations: { retry: false },
},
})
const renderWithQueryClient = (ui: React.ReactElement) => {
const queryClient = createTestQueryClient()
return render(
<QueryClientProvider client={queryClient}>
{ui}
</QueryClientProvider>
)
}
```
## Mock Best Practices
### ✅ DO
1. **Use real base components and dify-ui primitives** - Import from `@/app/components/base/` or `@langgenius/dify-ui/*` directly
1. **Use real project components** - Prefer importing over mocking
1. **Use real Zustand stores** - Set test state via `store.setState()`
1. **Reset mocks in `beforeEach`**, not `afterEach`
1. **Match actual component behavior** in mocks (when mocking is necessary)
1. **Use factory functions** for complex mock data
1. **Import actual types** for type safety
1. **Reset shared mock state** in `beforeEach`
### ❌ DON'T
1. **Don't mock base components or dify-ui primitives** (`Loading`, `Input`, `Button`, `Tooltip`, `Dialog`, etc.)
1. **Don't mock Zustand store modules** - Use real stores with `setState()`
1. Don't mock components you can import directly
1. Don't create overly simplified mocks that miss conditional logic
1. Don't leave HTTP mocks or service mock state leaking between tests
1. Don't use `any` types in mocks without necessity
### Mock Decision Tree
```
Need to use a component in test?
├─ Is it from @/app/components/base/* or @langgenius/dify-ui/*?
│ └─ YES → Import real component, DO NOT mock
├─ Is it a project component?
│ └─ YES → Prefer importing real component
│ Only mock if setup is extremely complex
├─ Is it an API service (@/service/*)?
│ └─ YES → Mock it
├─ Is it a third-party lib with side effects?
│ └─ YES → Mock it (next/navigation, external SDKs)
├─ Is it a Zustand store?
│ └─ YES → DO NOT mock the module!
│ Use real store + setState() to set test state
│ (Global mock handles auto-reset)
└─ Is it i18n?
└─ YES → Uses shared mock (auto-loaded). Override only for custom translations
```
## Zustand Store Testing
### Global Zustand Mock (Auto-loaded)
Zustand is globally mocked in `web/vitest.setup.ts` following the [official Zustand testing guide](https://zustand.docs.pmnd.rs/guides/testing). The mock in `web/__mocks__/zustand.ts` provides:
- Real store behavior with `getState()`, `setState()`, `subscribe()` methods
- Automatic store reset after each test via `afterEach`
- Proper test isolation between tests
### ✅ Recommended: Use Real Stores (Official Best Practice)
**DO NOT mock store modules manually.** Import and use the real store, then use `setState()` to set test state:
```typescript
// ✅ CORRECT: Use real store with setState
import { useAppStore } from '@/app/components/app/store'
describe('MyComponent', () => {
it('should render app details', () => {
// Arrange: Set test state via setState
useAppStore.setState({
appDetail: {
id: 'test-app',
name: 'Test App',
mode: 'chat',
},
})
// Act
render(<MyComponent />)
// Assert
expect(screen.getByText('Test App')).toBeInTheDocument()
// Can also verify store state directly
expect(useAppStore.getState().appDetail?.name).toBe('Test App')
})
// No cleanup needed - global mock auto-resets after each test
})
```
### ❌ Avoid: Manual Store Module Mocking
Manual mocking conflicts with the global Zustand mock and loses store functionality:
```typescript
// ❌ WRONG: Don't mock the store module
vi.mock('@/app/components/app/store', () => ({
useStore: (selector) => mockSelector(selector), // Missing getState, setState!
}))
// ❌ WRONG: This conflicts with global zustand mock
vi.mock('@/app/components/workflow/store', () => ({
useWorkflowStore: vi.fn(() => mockState),
}))
```
**Problems with manual mocking:**
1. Loses `getState()`, `setState()`, `subscribe()` methods
1. Conflicts with global Zustand mock behavior
1. Requires manual maintenance of store API
1. Tests don't reflect actual store behavior
### When Manual Store Mocking is Necessary
In rare cases where the store has complex initialization or side effects, you can mock it, but ensure you provide the full store API:
```typescript
// If you MUST mock (rare), include full store API
const mockStore = {
appDetail: { id: 'test', name: 'Test' },
setAppDetail: vi.fn(),
}
vi.mock('@/app/components/app/store', () => ({
useStore: Object.assign(
(selector: (state: typeof mockStore) => unknown) => selector(mockStore),
{
getState: () => mockStore,
setState: vi.fn(),
subscribe: vi.fn(),
},
),
}))
```
### Store Testing Decision Tree
```
Need to test a component using Zustand store?
├─ Can you use the real store?
│ └─ YES → Use real store + setState (RECOMMENDED)
│ useAppStore.setState({ ... })
├─ Does the store have complex initialization/side effects?
│ └─ YES → Consider mocking, but include full API
│ (getState, setState, subscribe)
└─ Are you testing the store itself (not a component)?
└─ YES → Test store directly with getState/setState
const store = useMyStore
store.setState({ count: 0 })
store.getState().increment()
expect(store.getState().count).toBe(1)
```
### Example: Testing Store Actions
```typescript
import { useCounterStore } from '@/stores/counter'
describe('Counter Store', () => {
it('should increment count', () => {
// Initial state (auto-reset by global mock)
expect(useCounterStore.getState().count).toBe(0)
// Call action
useCounterStore.getState().increment()
// Verify state change
expect(useCounterStore.getState().count).toBe(1)
})
it('should reset to initial state', () => {
// Set some state
useCounterStore.setState({ count: 100 })
expect(useCounterStore.getState().count).toBe(100)
// After this test, global mock will reset to initial state
})
})
```
## Factory Function Pattern
```typescript
// __mocks__/data-factories.ts
import type { User, Project } from '@/types'
export const createMockUser = (overrides: Partial<User> = {}): User => ({
id: 'user-1',
name: 'Test User',
email: 'test@example.com',
role: 'member',
createdAt: new Date().toISOString(),
...overrides,
})
export const createMockProject = (overrides: Partial<Project> = {}): Project => ({
id: 'project-1',
name: 'Test Project',
description: 'A test project',
owner: createMockUser(),
members: [],
createdAt: new Date().toISOString(),
...overrides,
})
// Usage in tests
it('should display project owner', () => {
const project = createMockProject({
owner: createMockUser({ name: 'John Doe' }),
})
render(<ProjectCard project={project} />)
expect(screen.getByText('John Doe')).toBeInTheDocument()
})
```
@@ -0,0 +1,269 @@
# Testing Workflow Guide
This guide defines the workflow for generating tests, especially for complex components or directories with multiple files.
## Scope Clarification
This guide addresses **multi-file workflow** (how to process multiple test files). For coverage requirements within a single test file, see `web/docs/test.md` § Coverage Goals.
| Scope | Rule |
|-------|------|
| **Single file** | Complete coverage in one generation (100% function, >95% branch) |
| **Multi-file directory** | Process one file at a time, verify each before proceeding |
## ⚠️ Critical Rule: Incremental Approach for Multi-File Testing
When testing a **directory with multiple files**, **NEVER generate all test files at once.** Use an incremental, verify-as-you-go approach.
### Why Incremental?
| Batch Approach (❌) | Incremental Approach (✅) |
|---------------------|---------------------------|
| Generate 5+ tests at once | Generate 1 test at a time |
| Run tests only at the end | Run test immediately after each file |
| Multiple failures compound | Single point of failure, easy to debug |
| Hard to identify root cause | Clear cause-effect relationship |
| Mock issues affect many files | Mock issues caught early |
| Messy git history | Clean, atomic commits possible |
## Single File Workflow
When testing a **single component, hook, or utility**:
```
1. Read source code completely
2. Run `pnpm analyze-component <path>` (if available)
3. Check complexity score and features detected
4. Write the test file
5. Run test: `pnpm test <file>.spec.tsx`
6. Fix any failures
7. Verify coverage meets goals (100% function, >95% branch)
```
## Directory/Multi-File Workflow (MUST FOLLOW)
When testing a **directory or multiple files**, follow this strict workflow:
### Step 1: Analyze and Plan
1. **List all files** that need tests in the directory
1. **Categorize by complexity**:
- 🟢 **Simple**: Utility functions, simple hooks, presentational components
- 🟡 **Medium**: Components with state, effects, or event handlers
- 🔴 **Complex**: Components with API calls, routing, or many dependencies
1. **Order by dependency**: Test dependencies before dependents
1. **Create a todo list** to track progress
### Step 2: Determine Processing Order
Process files in this recommended order:
```
1. Utility functions (simplest, no React)
2. Custom hooks (isolated logic)
3. Simple presentational components (few/no props)
4. Medium complexity components (state, effects)
5. Complex components (API, routing, many deps)
6. Container/index components (integration tests - last)
```
**Rationale**:
- Simpler files help establish mock patterns
- Hooks used by components should be tested first
- Integration tests (index files) depend on child components working
### Step 3: Process Each File Incrementally
**For EACH file in the ordered list:**
```
┌─────────────────────────────────────────────┐
│ 1. Write test file │
│ 2. Run: pnpm test <file>.spec.tsx │
│ 3. If FAIL → Fix immediately, re-run │
│ 4. If PASS → Mark complete in todo list │
│ 5. ONLY THEN proceed to next file │
└─────────────────────────────────────────────┘
```
**DO NOT proceed to the next file until the current one passes.**
### Step 4: Final Verification
After all individual tests pass:
```bash
# Run all tests in the directory together
pnpm test path/to/directory/
# Check coverage
pnpm test:coverage path/to/directory/
```
## Component Complexity Guidelines
Use `pnpm analyze-component <path>` to assess complexity before testing.
### 🔴 Very Complex Components (Complexity > 50)
**Consider refactoring BEFORE testing:**
- Break component into smaller, testable pieces
- Extract complex logic into custom hooks
- Separate container and presentational layers
**If testing as-is:**
- Use integration tests for complex workflows
- Use `test.each()` for data-driven testing
- Multiple `describe` blocks for organization
- Consider testing major sections separately
### 🟡 Medium Complexity (Complexity 30-50)
- Group related tests in `describe` blocks
- Test integration scenarios between internal parts
- Focus on state transitions and side effects
- Use helper functions to reduce test complexity
### 🟢 Simple Components (Complexity < 30)
- Standard test structure
- Focus on props, rendering, and edge cases
- Usually straightforward to test
### 📏 Large Files (500+ lines)
Regardless of complexity score:
- **Strongly consider refactoring** before testing
- If testing as-is, test major sections separately
- Create helper functions for test setup
- May need multiple test files
## Todo List Format
When testing multiple files, use a todo list like this:
```
Testing: path/to/directory/
Ordered by complexity (simple → complex):
☐ utils/helper.ts [utility, simple]
☐ hooks/use-custom-hook.ts [hook, simple]
☐ empty-state.tsx [component, simple]
☐ item-card.tsx [component, medium]
☐ list.tsx [component, complex]
☐ index.tsx [integration]
Progress: 0/6 complete
```
Update status as you complete each:
- ☐ → ⏳ (in progress)
- ⏳ → ✅ (complete and verified)
- ⏳ → ❌ (blocked, needs attention)
## When to Stop and Verify
**Always run tests after:**
- Completing a test file
- Making changes to fix a failure
- Modifying shared mocks
- Updating test utilities or helpers
**Signs you should pause:**
- More than 2 consecutive test failures
- Mock-related errors appearing
- Unclear why a test is failing
- Test passing but coverage unexpectedly low
## Common Pitfalls to Avoid
### ❌ Don't: Generate Everything First
```
# BAD: Writing all files then testing
Write component-a.spec.tsx
Write component-b.spec.tsx
Write component-c.spec.tsx
Write component-d.spec.tsx
Run pnpm test ← Multiple failures, hard to debug
```
### ✅ Do: Verify Each Step
```
# GOOD: Incremental with verification
Write component-a.spec.tsx
Run pnpm test component-a.spec.tsx ✅
Write component-b.spec.tsx
Run pnpm test component-b.spec.tsx ✅
...continue...
```
### ❌ Don't: Skip Verification for "Simple" Components
Even simple components can have:
- Import errors
- Missing mock setup
- Incorrect assumptions about props
**Always verify, regardless of perceived simplicity.**
### ❌ Don't: Continue When Tests Fail
Failing tests compound:
- A mock issue in file A affects files B, C, D
- Fixing A later requires revisiting all dependent tests
- Time wasted on debugging cascading failures
**Fix failures immediately before proceeding.**
## Integration with Codex's Todo Feature
When using Codex for multi-file testing:
1. **Create a todo list** before starting
1. **Process one file at a time**
1. **Verify each test passes** before asking for the next
1. **Mark todos complete** as you progress
Example prompt:
```
Test all components in `path/to/directory/`.
First, analyze the directory and create a todo list ordered by complexity.
Then, process ONE file at a time, waiting for my confirmation that tests pass
before proceeding to the next.
```
## Summary Checklist
Before starting multi-file testing:
- [ ] Listed all files needing tests
- [ ] Ordered by complexity (simple → complex)
- [ ] Created todo list for tracking
- [ ] Understand dependencies between files
During testing:
- [ ] Processing ONE file at a time
- [ ] Running tests after EACH file
- [ ] Fixing failures BEFORE proceeding
- [ ] Updating todo list progress
After completion:
- [ ] All individual tests pass
- [ ] Full directory test run passes
- [ ] Coverage goals met
- [ ] Todo list shows all complete
+92 -108
View File
@@ -1,144 +1,128 @@
---
name: how-to-write-component
description: Use when writing, refactoring, or reviewing React/TypeScript components in Dify web, especially decisions about component ownership, props/types, URL/query state, Jotai state, async state, generated API contracts, queries/mutations, overlays, effects, navigation, performance, and empty states.
description: React/TypeScript component style guide. Use when writing, refactoring, or reviewing React components, especially around abstraction choices, props typing, state boundaries, shared local state with Jotai atoms, API types, query/mutation contracts, navigation, memoization, wrappers, and empty-state handling.
---
# How To Write A Component
Use this as the component decision guide for Dify web. Existing code is reference material, not automatic precedent; if touched code violates these rules, adapt it and fix equivalent patterns in the same feature branch.
## First Decisions
| Question | Default | Promote or extract only when |
| --- | --- | --- |
| Where should code live? | Keep it local to the feature workflow, route, or owner. | Multiple verticals need the same stable primitive. |
| How should route/tab folders be named? | Match the current route segment, tab name, or user-visible surface. | Keep a historical or broader parent only when it still owns multiple surfaces. |
| Who owns state, data, and handlers? | The lowest component that uses them. | A parent coordinates shared loading, errors, empty UI, selection, submission, navigation, or one consistent snapshot. |
| Should this become Jotai state? | Keep synchronous UI/form state in component or DOM state. | Siblings need one source of truth, the value drives atoms, or scoped workflow state must survive hidden/unmounted steps. |
| Should URL state enter Jotai? | Let Next.js route params and `nuqs` own URL state and updates. | Query atoms or shared derived atoms need a read-only bridge hydrated at the route/surface boundary. |
| Should this query/mutation become an atom? | Use TanStack Query hooks at the lowest owner. | It reads atom state, feeds derived atoms, or participates in shared Jotai workflow orchestration. |
| Should this be a helper/wrapper? | Prefer direct readable code at the use site. | The name captures a stable domain rule or the wrapper owns real behavior, validation, state, error handling, or semantics. |
| Where should a hotkey live? | Keep a single-owner hotkey constant in its component. | Multiple production files share one command, or the feature owns a real command registry with shared metadata and behavior. |
| Is an Effect needed? | No. Derive during render or handle the user action in the event handler. | It synchronizes with an external system such as browser APIs, subscriptions, timers, analytics, or imperative DOM/non-React widgets. |
Use this as the decision guide for React/TypeScript component structure. Existing code is reference material, not automatic precedent; when it conflicts with these rules, adapt the approach instead of reproducing the violation.
## Core Defaults
- Search before adding UI, hooks, helpers, query utilities, or styling patterns. Reuse existing base components, feature components, hooks, utilities, and design styles when they fit.
- Follow Dify's CSS-first Tailwind v4 contract from `packages/dify-ui/README.md` and `packages/dify-ui/AGENTS.md`. Prefer design-system tokens, utilities, and radius mappings over generic Tailwind choices.
- Preserve visible keyboard focus states on the final focusable element. Prefer styled `@langgenius/dify-ui/*` controls when available, because components such as `Button` and form/control primitives carry the standard Dify UI `focus-visible` styling. Do not assume every Dify UI export provides visual focus styles: headless anatomy parts and direct Base UI re-exports such as dialog/popover/tooltip/drawer triggers usually only provide behavior and semantics. When using native `button` / `a`, custom trigger `render` props, clickable rows, icon buttons, menu-like items, or direct trigger parts, verify the rendered focusable element has a visible focus state. If it does not, add the standard Dify UI focus style: `outline-hidden focus-visible:ring-2 focus-visible:ring-state-accent-solid`. Do not hide outlines without an equivalent visible `focus-visible` indicator. Component-specific focus styles should follow an existing styled primitive pattern or a concrete design constraint, not a new ad hoc style.
- Group feature code by workflow, route, or ownership area with route-aligned names: components, hooks, local types, query helpers, atoms, constants, tests, and small utilities should live near the code that changes with them.
- For each feature module, keep a module-local `README.md` as a boundary note. Start with the module name, a brief one-sentence description, then split dependencies into `Internal Modules` and `External Modules` sections; keep both sections and write `None.` when one category is empty. `Internal Modules` lists modules inside the same overall feature using paths from that feature root, such as `shared/domain/runtime-status`; `External Modules` lists project modules outside the feature using paths from the web root without a `web/` prefix, such as `app/components/base/skeleton`. Omit npm packages, workspace package dependencies, and whitelisted plumbing modules. Do not copy caller-relative import paths into the README.
- Module README whitelist: `@/service/client`, `@/next/*`.
- Keep source/default selection, validation, dirty checks, and payload shaping close to the workflow that owns submit behavior. Do not hide flow-specific priority order, fallback behavior, or submit semantics in generic utilities.
- Prefer direct conditionals for small branch-specific decisions, especially form source selection and request payload assembly.
- Loading states for page sections, cards, lists, tables, forms, and drawers should be skeletons scoped to the content being loaded. Use spinners only for small inline busy indicators.
- Search before adding UI, hooks, helpers, or styling patterns. Reuse existing base components, feature components, hooks, utilities, and design styles when they fit.
- Group code by feature workflow, route, or ownership area: components, hooks, local types, query helpers, atoms, constants, and small utilities should live near the code that changes with them.
- Promote code to shared only when multiple verticals need the same stable primitive. Otherwise keep it local and compose shared primitives inside the owning feature.
- Prefer local code and purpose-named helpers over catch-all utility modules; do not group workflow-specific defaults, validation, payload shaping, or metadata merging in a generic utils file just because they share a DTO.
- Keep source/default selection, validation, and payload shaping close to the workflow that owns the behavior. Do not extract a shared helper just because two flows read the same DTO when their priority order, fallback behavior, or submit semantics differ.
- Prefer direct, readable conditionals at the use site for small branch-specific decisions, especially form source selection and request payload assembly. Extract only when the helper name captures a stable domain rule and removes repeated complexity without hiding flow-specific behavior.
- When fixing an invalid pattern, scan the touched feature or branch for equivalent patterns and fix them together.
- Follow Dify's CSS-first Tailwind v4 contract from `packages/dify-ui/README.md` and `packages/dify-ui/AGENTS.md`. Prefer design-system tokens, utilities, and radius mappings over generic Tailwind guidance.
## Layout And Ownership
## Feature Workflow Layout
- State-heavy wizards, drawers, modals, and secondary workflows can be a small feature surface: an entry file, one feature-local state file when Jotai is actually needed, and shallow `ui/` owners that match real visual regions.
- The entry file handles route integration, provider wiring, close behavior, and surface mounting. The composition owner handles high-level workflow branching. The closest visual owner handles section branching.
- When a page or tab maps to a route segment, name its feature folder after that route/tab surface instead of a stale parent grouping. Remove misleading intermediate folders when only one surface remains.
- When a tab folder grows into several independent sections or action areas, split the first level by product/visual owners. Keep the root for the entry component and cross-owner state, colocate tests with the owner folder, and put truly shared local UI under a specifically named `components/` file.
- Repeated TanStack query calls in sibling components are acceptable when each component independently consumes the data; TanStack Query deduplicates and shares cache.
- Pass stable domain identity across boundaries. Do not forward derived presentation state when the receiver can derive it from its own data source.
- A component that owns a visual surface should also own data access, loading, empty, and error states for content rendered inside it unless a parent truly coordinates that state.
- Avoid prop drilling. One pass-through layer is acceptable; repeated forwarding means ownership should move down or into feature-scoped Jotai UI state. Keep server/cache state in Query and API flow.
- Do not replace prop drilling with one large view-model hook threaded through section props. Move each hook, query, derived value, and handler to the concrete section that consumes it.
- Keep callbacks in a parent only for workflow coordination such as form submission, shared selection, batch behavior, or navigation. Otherwise let the child, menu, or row own the action.
- State-heavy wizards, drawers, modals, and secondary workflows work best as a small feature surface with route/entry files, a single feature-local state file, and feature-local UI.
- Keep `ui/` shallow with owner files that map to the workflow's real composition boundaries and major visual regions.
- Owner files contain the section components, field components, skeletons, and one-off helper components that belong to their visual region.
- Folders represent groups of related files with a shared owner and a stable reason to change together.
- The entry file handles route integration, provider wiring, close behavior, and feature surface mounting. The composition owner handles high-level workflow branching, and the closest visual owner handles section branching.
## Feature-Scoped Jotai
## Ownership
- A Jotai-backed feature has one feature-local state file for shared primitive atoms, query atoms, derived atoms, write-only actions, mutation atoms, submission orchestration, provider exports, and optional scope configuration.
- Keep component-owned synchronous UI state local even inside Jotai features: dialog open flags, menus/popovers, confirmations, field drafts, and selected local options usually belong in component state.
- Use uncontrolled `@langgenius/dify-ui/form` and `@langgenius/dify-ui/field` controls for edit/create forms whose fields are read only at submit time. Initialize query-backed defaults with `defaultValue` and keyed remounts.
- Promote form state to atoms only when another component must react to in-progress values, a draft must survive unmount/remount in the scoped workflow, or multiple steps share the same editable draft before submit.
- Treat `useParams`, route args, and `nuqs` query state as framework-owned state. When atom logic needs those values, hydrate primitive atoms at the route or surface boundary, such as with `useHydrateAtoms(..., { dangerouslyForceHydrate: true })`; keep URL updates in the route/query-state APIs instead of write atoms.
- Within a route-owned feature, choose one source for route identity. If route params are bridged into feature atoms, use that bridge consistently for route-derived queries and actions instead of also threading the same route id through page, tab, and section props.
- For async work tied to atom state, use `atomWithQuery` or `atomWithMutation`; write atoms should update only the inputs that drive those atoms. This applies to pure frontend async work as well as network requests, so do not hand-roll loading/error/in-flight state with `useState` or `useRef` for atom-orchestrated async behavior. For component-owned remote work, use `useQuery` or `useMutation` directly.
- `jotai-tanstack-query` query atoms do not support TanStack Query tracked properties. A component that reads `useAtomValue(queryAtom)` subscribes to the whole query result, even if it only accesses `data`, `isLoading`, or `isError`. Export field-specific derived atoms and have components read the exact fields they render; use `selectAtom(queryAtom, result => result.field)` for query-result fields so unchanged selections do not notify subscribers. Keep direct `useAtomValue(queryAtom)` only when the component or hook genuinely needs the full observer result.
- Row-local async state belongs to the row owner unless it participates in a shared Jotai workflow or needs atom-scoped reset semantics.
- Leave query and mutation atoms unscoped so they keep shared QueryClient cache and invalidation behavior. Scope resettable primitives and explicit hydration tuples; scope a derived atom only when every dependency should be private to that surface.
- For scoped primitives that are always hydrated by `ScopeProvider`, prefer `atomWithLazy<T>(() => { throw new Error(...) })` when consumers should see a non-null type.
- Order state files by dependency graph: types/constants, primitives, query atoms, query-data derived atoms, business/readiness derived atoms, write actions, mutation atoms, submission orchestration, provider exports.
- Name derived atoms as business facts and write atoms as user or workflow commands. Components should read or write the exact atom they need with `useAtomValue` or `useSetAtom`.
- Menu/dialog `open` state usually stays local, but a scoped atom is acceptable when a composed menu plus secondary surface would otherwise pass confusing `open`/`onClose` props through unrelated layers. Scope that primitive with the surface instance so reset behavior stays local.
- Keep independent dialog lifecycles separate. Avoid one discriminated "current action dialog" atom when dialogs have separate open state, loading guards, or reset behavior.
- Put local state, queries, mutations, handlers, and derived UI data in the lowest component that uses them. Extract a purpose-built owner component only when the logic has no natural home.
- Repeated TanStack query calls in sibling components are acceptable when each component independently consumes the data. Do not hoist a query only because it is duplicated; TanStack Query handles deduplication and cache sharing.
- Hoist state, queries, or callbacks to a parent only when the parent consumes the data, coordinates shared loading/error/empty UI, needs one consistent snapshot, or owns a workflow spanning children.
- Pass stable domain identity across boundaries; avoid forwarding derived presentation state when the receiver can derive it from its own data source. A component that owns a visual surface should also own the data access, loading, empty, and error states for content rendered inside it unless a parent truly coordinates that state.
- Loading states for visual surfaces should use skeleton placeholders scoped to the content that is actually loading, with shape, density, and dimensions close to the final UI. Avoid generic loading text or centered spinners for page sections, cards, lists, tables, forms, and drawers; reserve spinners for small inline busy indicators such as an in-progress status icon.
- Avoid prop drilling. One pass-through layer is acceptable; repeated forwarding means ownership should move down or into feature-scoped Jotai UI state. Keep server/cache state in query and API data flow.
- Do not replace prop drilling with one top-level hook that returns a large view model and then thread that object through section props. Move each hook, query, derived value, and handler to the concrete section that consumes it, or use feature-scoped Jotai atoms for simple shared form/UI state when siblings need the same source of truth.
- When using feature-scoped Jotai state for a form, drawer, or other secondary surface, scope the store to that surface instance when stale cross-instance state is possible. Initialize stable config at the owning boundary, then let descendants read only the atoms or purpose-named hooks they actually need.
- Keep callbacks in a parent only for workflow coordination such as form submission, shared selection, batch behavior, or navigation. Otherwise let the child or row own its action.
- Prefer uncontrolled DOM state and CSS variables before adding controlled props.
## Feature-Scoped Jotai State
- A module's feature-local state lives in one state file for Jotai-backed features: primitive atoms, query atoms, derived atoms, write-only action atoms, mutation atoms, submission orchestration, provider exports, and optional scope configuration.
- Atom order in the state file follows the dependency graph: types/constants, editable primitives, query atoms, query-data derived atoms, readiness/business derived atoms, write actions, mutation atoms, submission orchestration, provider exports.
- Derived atom names read as business facts. Write atom names read as user or workflow commands.
- UI components read and write the exact atom they use with `useAtomValue` or `useSetAtom`. Repeated workflow semantics live in named derived atoms or write atoms.
- Non-query derived atoms return a narrow value with a clear domain name. Query atoms expose the TanStack Query result object so loading, error, fetch, and pagination state stay attached to the query contract.
- Write-only atoms own state transitions that update multiple primitives, reset dependent state, guard stale async work, or advance the workflow.
- `jotai-tanstack-query` atoms use the same QueryClient as the React Query provider. Query atoms belong in feature state when atoms are the feature's local state surface.
- Jotai scope is an optional instance-isolation tool for secondary surfaces with independent local state. Query atoms keep shared cache behavior through the shared QueryClient.
## Components, Props, And Types
- Type component signatures directly; do not use `FC` or `React.FC`.
- Prefer `function` for top-level components and module helpers. Use arrow functions for local callbacks, handlers, and lambda-style APIs.
- Prefer named exports. Use default exports only where the framework requires them, such as Next.js route files.
- Avoid barrel files that only re-export secondary owners. `index.tsx` is acceptable for a route/tab entry component; import header controls, switches, sections, and row owners from their concrete owner files.
- Type simple one-off props inline. Use a named `Props` type only when reused, exported, complex, or clearer.
- Use API-generated or API-returned types at component boundaries. Keep small UI conversion helpers and one-off UI extensions beside the component that needs them.
- Preserve domain value types for selection components. Do not widen enum, union, boolean, numeric, object, or nullable select/radio values to `string`; keep wrappers and option value carriers typed from their feature option collection.
- Avoid `common.tsx` buckets for shared UI. Use a feature-local `components/` folder with concrete filenames that describe the shared role.
- Do not create type aliases that only rename another type. Use aliases only for real UI concepts, refinements, or reusable local contracts.
- Name values by their domain role and backend API contract, especially persistent IDs and route params. Normalize framework or route params at the boundary.
- Put fallback and invariant checks in the lowest component that already handles that state. Do not extract helpers whose only behavior is hiding missing display data.
- Do not create type aliases that only rename another type. Use an alias only when it encodes a real UI concept, refinement, or reusable local contract.
- Name values by their domain role and backend API contract, and keep that name stable across the call chain, especially persistent IDs and route params. Normalize framework or route params at the boundary.
- Keep fallback and invariant checks at the lowest component that already handles that state; avoid defensive fallbacks that mask impossible states.
- Do not extract fallback helpers whose only behavior is hiding missing display data. The component that renders the surface owns the empty, disabled, hidden, or placeholder state.
## Keyboard Shortcuts
- Distinguish application commands from local keyboard semantics before choosing an API. Use `@tanstack/react-hotkeys` for application commands. Keep menu navigation, dialog Escape handling owned by a primitive, editor commands, and other widget-scoped ARIA interactions in their local component or primitive.
- Use `useHotkey` or `useHotkeys` for registered commands. For a command intentionally owned by an existing `onKeyDown`, use `matchesKeyboardEvent` instead of hand-written `metaKey` / `ctrlKey` parsing or a second global listener.
- Define a reusable string command with `satisfies Hotkey` and an object-form command with `satisfies RawHotkey`. Reserve `RegisterableHotkey` for API boundaries that intentionally accept either form. A one-time inline literal passed directly to TanStack is already type-checked; extract it when registration, display, metadata, or another production consumer needs the same source.
- Keep registered hotkeys distinct from held keys and display-only accelerators. Use `IndividualKey` with `useKeyHold` for held-key interactions, and use an explicitly named `displayKey` for local widget accelerators that are not registered `Hotkey` values.
- Keep registration and keycap/menu display derived from one canonical command. Do not maintain a hotkey string beside a separate `['Mod', ...]` display array.
- Keep a single-owner command constant in its owning component. Create a feature-local `hotkeys.ts` only when multiple production files consume the same command. Keep a dedicated definitions/registry module when a feature owns a real command system with IDs, metadata, alternate bindings, and centralized registration. Tests do not count as another production owner, and file-name uniformity alone is not a reason to extract.
- Make scope and availability explicit. Use `enabled` for business or surface lifecycle, `ignoreInputs` for whether input-like elements may trigger the command, and `target` when the command belongs to a concrete DOM subtree. Global application commands may use the document target; inline editors and composed overlays should prefer the actual editor or Base UI Popup ref when that owner is exposed.
- Put a scoped ref on the real behavior owner. Do not add a wrapper DOM element solely to obtain a hotkey target. If a shared overlay convenience component hides the Popup ref, either rely on its modal lifecycle/focus boundary when that is sufficient or design the primitive API separately; do not create a fake owner at the call site.
- Set `preventDefault` and `stopPropagation` according to the existing product behavior and browser interaction. Do not silently accept TanStack defaults when migrating from another listener if that changes typing, submission, or propagation semantics.
- Test observable command behavior, disabled/input/target scope, and the shared registration/display contract at the owning feature boundary. Prefer partial mocks that retain TanStack formatting and matching behavior when a registration boundary must be isolated.
## Generated API And Nullable Data
## Generated API Contracts
- Treat generated contracts as authoritative at API, query, mutation, cache, and service boundaries. For enterprise APIs, use `packages/contracts/generated/enterprise/*`.
- Do not hand-write DTO mirrors, widen generated fields/enums, or add parallel frontend enum/status layers unless they model product state not represented by the API.
- Use generated enum objects and union types directly in props, comparisons, status logic, and i18n keys. Presentation-only tone maps should be keyed by generated enums.
- Normalize or coerce only at real boundaries: user-entered forms, search, URL/query params, file names, DOM IDs, or legacy adapters.
- Do not coerce nullable or optional API strings to `''` in query, derived model, or payload-building code. Keep `null` or `undefined` until the final boundary requiring a string.
- Do not use `value || undefined` for mutation fields where `''` means "clear this value". Trim or normalize at the form boundary, then preserve intentional empty strings.
- Prefer nullable-tolerant render props for API-returned rows. Narrow only where a real value is required, such as mutation params, route hrefs, select values, query input, or required React keys.
- Build required values in the same branch that proves them, using `flatMap`, a local loop, or an early return. Avoid truthiness guards, `filter(Boolean)`, `filter(item => item.id)`, and `!` after filters.
- Use conditional spreads or explicit pushes for conditional array items instead of `undefined` placeholders followed by narrowing filters.
- Empty collection fallbacks are for not-yet-loaded query data or genuinely nullable collections at the owning render boundary, not for hiding required API fields.
- Do not hand-write local request/response/reply/page/cache-data types that mirror generated DTOs. Import or infer the generated type.
- Do not widen generated fields or enums for compatibility. Normalize legacy input at the boundary, then return the generated field type.
- Do not repair generated or API-returned contract fields in components unless the API contract or product requirement says they need normalization. Treat enums, statuses, and presence flags as exact contract values.
- Use generated enum objects and union types directly in props, comparisons, status logic, and i18n keys. Do not add local enum constants or parallel frontend enum/status layers unless they model real product state not represented by the API. Presentation-only tone maps should be keyed by the generated enum.
- Normalize or coerce only at a real boundary, such as user-entered forms, search, URL/query params, file names, DOM IDs, or legacy adapters. Preserve user-entered values when whitespace or formatting can be meaningful.
- Do not coerce nullable or optional API strings to `''` in query, derived model, or payload-building code. Keep `undefined` or `null` until the final boundary that requires a string.
- Local UI models are fine for presentation, form state, select options, or guarded required-field refinements. Name them as UI concepts, not generated DTO mirrors.
- Required-value refinements are allowed only after same-branch filtering or early return. Prefer nullable-tolerant props for render-only data.
- When a component needs a stricter shape than a generated DTO, refine once at the API/query-to-UI boundary into a purpose-named UI type instead of hiding missing fields with generic fallback or coercion helpers.
## Nullable API Data
- Prefer nullable-tolerant call boundaries. Pass API-returned types through for render-only rows, and let the component render fallback, disabled state, or nothing.
- Narrow only where a real value is required, such as mutation params, route hrefs, select values, or query input. Build that target model with `flatMap`, a local loop, or an early return so the required value is captured in the same branch.
- If design says a field is the display value, use that field. Only the final component should decide whether a nullable display value renders a placeholder, hides content, or disables an action.
- Do not wrap required arrays or fields in null-fallback helpers. Use empty collection fallbacks only for not-yet-loaded query data or genuinely nullable collections at the owning render boundary.
- Do not drop rows only to satisfy props or React keys; use a stable fallback key when possible.
- Use conditional spreads or explicit pushes for conditional array items instead of `undefined` placeholders followed by a narrowing filter.
- Avoid truthiness type guards, `filter(Boolean)`, `filter(item => item.id)`, and `!` after those filters.
- Use type guards only for meaningful domain or runtime validation, such as enum membership, object shape, or a reusable business invariant.
## Queries And Mutations
- Keep `web/contract/*` as the API shape source of truth and follow the `{ params, query?, body? }` input shape.
- Consume generated queries with `useQuery(consoleQuery.xxx.queryOptions(...))` or `useQuery(marketplaceQuery.xxx.queryOptions(...))`.
- If a generated query input comes from an atom, including a route-identity bridge atom, keep the query in `atomWithQuery`; do not unwrap the atom in a component just to call `useQuery`.
- Consume owner-local mutations with `useMutation(consoleQuery.xxx.mutationOptions(...))` or `useMutation(marketplaceQuery.xxx.mutationOptions(...))` when pending/error state is not consumed by feature atoms.
- In `atomWithQuery`, `atomWithInfiniteQuery`, and `atomWithMutation`, return generated `queryOptions()`, `infiniteOptions()`, or `mutationOptions()` directly. Pass `enabled`, `retry`, `placeholderData`, `select`, and pagination options into the generated call instead of spreading options into a hand-built object.
- For generated oRPC options with missing required input, branch the whole input with `input: condition ? validInput : skipToken` and `enabled: Boolean(condition)`. Never place `skipToken` inside a nested placeholder payload or coerce required IDs to `''`.
- When prefetch and render use the same request, extract local query options or a query-options atom so `prefetchQuery` and `useQuery`/`atomWithQuery` share the exact options.
- For custom query or mutation functions, wrap options with TanStack `queryOptions(...)` or `mutationOptions(...)`.
- Do not extract generated `queryOptions(...)` into a helper solely to share input construction; extract only when prefetch/render must share exact options or the helper owns real domain behavior.
- Avoid pass-through hooks and thin `web/service/use-*` wrappers that only rename generated options. Keep feature hooks for real orchestration, workflow state, or shared domain behavior.
- Put shared cache behavior in `createTanstackQueryUtils(...experimental_defaults...)`. Component or atom callbacks may handle local toasts, closing dialogs, and navigation, but should not replace shared invalidation or patch shared server state locally.
- For overlays that may open heavier secondary content, prefetch from the trigger/menu open event with `queryClient.prefetchQuery(queryOptions)` when `onOpenChange` is available. Do not mount hidden subscribers just to warm cache.
- Keep `web/contract/*` as the single source of truth for API shape; follow existing domain/router patterns and the `{ params, query?, body? }` input shape.
- Consume queries directly with `useQuery(consoleQuery.xxx.queryOptions(...))` or `useQuery(marketplaceQuery.xxx.queryOptions(...))`.
- Avoid pass-through hooks and thin `web/service/use-*` wrappers that only rename `queryOptions()` or `mutationOptions()`. Extract a small `queryOptions` helper only when repeated call-site options justify it.
- Keep feature hooks for real orchestration, workflow state, or shared domain behavior.
- For TanStack cache data, use generated or query-derived types; do not create local wrappers for `getQueryData` or `getQueriesData`.
- For generated oRPC `queryOptions()` / `infiniteOptions()`, do not pass `skipToken` as `input`; keep a valid placeholder input shape and use `enabled` to gate missing required params because the OpenAPI codec encodes input eagerly.
- Consume mutations directly with `useMutation(consoleQuery.xxx.mutationOptions(...))` or `useMutation(marketplaceQuery.xxx.mutationOptions(...))`; use oRPC clients as `mutationFn` only for custom flows.
- Put shared cache behavior in `createTanstackQueryUtils(...experimental_defaults...)`; components may add UI feedback callbacks, but should not own shared invalidation rules.
- Do not use deprecated `useInvalid` or `useReset`.
- Prefer `mutate(...)`; use `mutateAsync(...)` only when Promise semantics are required, and wrap awaited calls in `try/catch`.
## Boundaries And Overlays
## Component Boundaries
- Use the first level below a page or tab to organize independent page sections when it adds structure or the root folder becomes noisy. This layer is layout/semantic first, not automatically the data owner.
- Treat component names, semantic roles, and user- or design-marked visual regions as boundary constraints. Keep adjacent UI as a sibling owner or introduce a correctly named broader owner.
- Use the first level below a page or tab to organize independent page sections when it adds real structure. This layer is layout/semantic first, not automatically the data owner.
- Treat component names, semantic roles, and user- or design-marked visual regions as boundary constraints. Do not expand a child component's responsibility just because its data is useful nearby; keep adjacent UI as a sibling owner or introduce a correctly named broader owner.
- Split deeper components by the data and state each layer actually needs. Each component should access only necessary data, and ownership should stay at the lowest consumer.
- Keep cohesive forms, menu bodies, and one-off helpers local unless they need their own state, reuse, or semantic boundary.
- Separate hidden secondary surfaces from the trigger's main flow. For dialogs, dropdowns, popovers, and similar branches, extract a small local component when hidden content would obscure the parent.
- Preserve composability by separating behavior ownership from placement ownership: an action can own trigger/open/menu content while the caller owns slots, offsets, and alignment.
- When a dialog, dropdown, or popover accepts controlled `open`, mount it unconditionally unless unmounting is required for performance or reset semantics. Use keyed scope or local state reset instead of `{open && <Surface />}` wrappers.
- When opening a dialog from a menu item, keep the menu and dialog as sibling surfaces. Let the menu command open the dialog, and mount the dialog outside menu popup content.
- For dialogs and alert dialogs, keep the root responsible for `open` wiring and put query/mutation hooks inside the content component when work should mount only after the overlay opens.
- Prefer uncontrolled overlay roots when the library can own open state. Use `onOpenChange` for side effects and CSS/data selectors for open-state styling.
- Avoid wrapper DOM unless it provides layout, semantics, accessibility, state ownership, or library integration. Avoid shallow wrappers, hook-to-props adapters, layout-only render props, children pass-through wrappers, and prop renaming unless they add real behavior or a real boundary.
- Separate hidden secondary surfaces from the trigger's main flow. For dialogs, dropdowns, popovers, and similar branches, extract a small local component that owns the trigger, open state, and hidden content when it would obscure the parent flow.
- Preserve composability by separating behavior ownership from layout ownership. A dropdown action may own its trigger, open state, and menu content; the caller owns placement such as slots, offsets, and alignment.
- Avoid unnecessary DOM hierarchy. Do not add wrapper elements unless they provide layout, semantics, accessibility, state ownership, or integration with a library API; prefer fragments or styling an existing element when possible.
- Avoid shallow wrappers, hook-to-props adapter components, layout-only render-prop wrappers, and prop renaming unless the wrapper adds validation, orchestration, error handling, state ownership, or a real semantic boundary. If a component only calls a hook and forwards every returned field to one child, move the hook into that child or make the wrapper own a real surface.
## Effects, Navigation, And Performance
## You Might Not Need An Effect
- Use Effects only to synchronize with external systems such as browser APIs, non-React widgets, subscriptions, timers, analytics that must run because the component was shown, or imperative DOM integration.
- Do not use Effects to transform props or state for rendering. Calculate derived values during render, and use `useMemo` only when the calculation is actually expensive.
- Do not use Effects to handle user actions. Put action-specific logic in the event handler where the cause is known.
- Do not use Effects to copy one state value into another state value representing the same concept. Pick one source of truth and derive the rest during render.
- Do not reset or adjust state from props with an Effect. Prefer a `key` reset, storing a stable ID and deriving the selected object, or guarded same-component render-time adjustment when truly necessary.
- Prefer framework data APIs or TanStack Query for data fetching instead of writing request Effects in components.
- If an Effect still seems necessary, first name the external system it synchronizes with. If there is no external system, remove the Effect and restructure the state or event flow.
## Navigation And Performance
- Use Effects only to synchronize with external systems. Do not use Effects to transform props/state for rendering, handle user actions, copy state, reset state from props, or fetch data.
- For forms initialized from query data, prefer keyed remounts or surface-entry atom hydration over Effects that copy query data into form state.
- Prefer framework data APIs or TanStack Query for data fetching.
- Prefer `Link` for normal navigation. Use router APIs only for command-flow side effects such as mutation success, guarded redirects, or form submission.
- Before using `memo`, move changing state down to the smallest component that uses it. If state must wrap stable content, lift the stable content up and pass it as `children`.
- Before reaching for `memo`, first try moving changing state down to the smallest component that actually uses it so unrelated sibling trees stay untouched.
- If changing state must wrap other content, lift the unchanged content up and pass it as `children` so the stateful wrapper can update without React visiting that subtree.
- Avoid `memo`, `useMemo`, and `useCallback` unless there is a clear performance reason.
+45 -22
View File
@@ -6,8 +6,8 @@
* @crazywoola @laipz8200
# Lint bulk suppression baselines.
/oxlint-suppressions.json
# ESLint suppression file is maintained by autofix.ci pruning.
/eslint-suppressions.json
# CODEOWNERS file
/.github/CODEOWNERS @laipz8200 @crazywoola
@@ -26,15 +26,34 @@
/cli/ @GareArc
/.github/workflows/cli-tests.yml @GareArc
# E2E
/e2e/ @lyzno1
# Backend (default owner, more specific rules below will override)
/api/ @QuantumGhost
# Backend - MCP
/api/core/mcp/ @Nov1c444
/api/core/entities/mcp_provider.py @Nov1c444
/api/services/tools/mcp_tools_manage_service.py @Nov1c444
/api/controllers/mcp/ @Nov1c444
/api/controllers/console/app/mcp_server.py @Nov1c444
# Backend - Tests
/api/tests/ @laipz8200 @QuantumGhost
/api/tests/**/*mcp* @Nov1c444
# Backend - Workflow - Engine (Core graph execution engine)
/api/core/workflow/graph_engine/ @laipz8200 @QuantumGhost
/api/core/workflow/runtime/ @laipz8200 @QuantumGhost
/api/core/workflow/graph/ @laipz8200 @QuantumGhost
/api/core/workflow/graph_events/ @laipz8200 @QuantumGhost
/api/core/workflow/node_events/ @laipz8200 @QuantumGhost
# Backend - Workflow - Nodes (Agent, Iteration, Loop, LLM)
/api/core/workflow/nodes/agent/ @Nov1c444
/api/core/workflow/nodes/iteration/ @Nov1c444
/api/core/workflow/nodes/loop/ @Nov1c444
/api/core/workflow/nodes/llm/ @Nov1c444
# Backend - RAG (Retrieval Augmented Generation)
/api/core/rag/ @JohnJyong
/api/services/rag_pipeline/ @JohnJyong
@@ -88,6 +107,7 @@
/api/core/app/layers/trigger_post_layer.py @CourTeous33
/api/services/trigger/ @CourTeous33
/api/models/trigger.py @CourTeous33
/api/fields/workflow_trigger_fields.py @CourTeous33
/api/repositories/workflow_trigger_log_repository.py @CourTeous33
/api/repositories/sqlalchemy_workflow_trigger_log_repository.py @CourTeous33
/api/libs/schedule_utils.py @CourTeous33
@@ -112,11 +132,11 @@
/api/controllers/console/billing/ @hj24 @zyssyz123
# Backend - Enterprise
/api/configs/enterprise/ @GareArc
/api/services/enterprise/ @GareArc
/api/services/feature_service.py @GareArc
/api/controllers/console/feature.py @GareArc
/api/controllers/web/feature.py @GareArc
/api/configs/enterprise/ @GarfieldDai @GareArc
/api/services/enterprise/ @GarfieldDai @GareArc
/api/services/feature_service.py @GarfieldDai @GareArc
/api/controllers/console/feature.py @GarfieldDai @GareArc
/api/controllers/web/feature.py @GarfieldDai @GareArc
# Backend - Database Migrations
/api/migrations/ @snakevash @laipz8200 @MRZHUH
@@ -129,6 +149,7 @@
# Frontend - Platform and Features
/web/config/ @lyzno1
/web/contract/ @lyzno1
/web/env.ts @lyzno1
/web/features/ @lyzno1
/web/hooks/ @lyzno1
@@ -187,6 +208,7 @@
/web/app/components/rag-pipeline/store/ @iamjoel @zxhlyh
# Frontend - RAG - Documents List
/web/app/components/datasets/documents/list.tsx @iamjoel @WTW0313
/web/app/components/datasets/documents/create-from-pipeline/ @iamjoel @WTW0313
# Frontend - RAG - Segments List
@@ -205,22 +227,22 @@
/web/app/components/plugins/marketplace/ @iamjoel @Yessenia-d
# Frontend - Login and Registration
/web/app/signin/ @iamjoel
/web/app/signup/ @iamjoel
/web/app/reset-password/ @iamjoel
/web/app/install/ @iamjoel
/web/app/init/ @iamjoel
/web/app/forgot-password/ @iamjoel
/web/app/account/ @iamjoel
/web/app/signin/ @douxc @iamjoel
/web/app/signup/ @douxc @iamjoel
/web/app/reset-password/ @douxc @iamjoel
/web/app/install/ @douxc @iamjoel
/web/app/init/ @douxc @iamjoel
/web/app/forgot-password/ @douxc @iamjoel
/web/app/account/ @douxc @iamjoel
# Frontend - Service Authentication
/web/service/base.ts @iamjoel
/web/service/base.ts @douxc @iamjoel
# Frontend - WebApp Authentication and Access Control
/web/app/(shareLayout)/components/ @iamjoel
/web/app/(shareLayout)/webapp-signin/ @iamjoel
/web/app/(shareLayout)/webapp-reset-password/ @iamjoel
/web/app/components/app/app-access-control/ @iamjoel
/web/app/(shareLayout)/components/ @douxc @iamjoel
/web/app/(shareLayout)/webapp-signin/ @douxc @iamjoel
/web/app/(shareLayout)/webapp-reset-password/ @douxc @iamjoel
/web/app/components/app/app-access-control/ @douxc @iamjoel
# Frontend - Explore Page
/web/app/components/explore/ @CodingOnStar @iamjoel
@@ -239,6 +261,7 @@
/web/app/components/base/**/*.spec.tsx @hyoban @CodingOnStar
# Frontend - Utils and Hooks
/web/utils/classnames.ts @iamjoel @zxhlyh
/web/utils/time.ts @iamjoel @zxhlyh
/web/utils/format.ts @iamjoel @zxhlyh
/web/utils/clipboard.ts @iamjoel @zxhlyh
-6
View File
@@ -7,9 +7,3 @@ web:
- 'pnpm-lock.yaml'
- 'pnpm-workspace.yaml'
- '.nvmrc'
e2e:
- changed-files:
- any-glob-to-any-file:
- 'e2e/**'
- '.github/workflows/web-e2e.yml'
+1 -1
View File
@@ -45,7 +45,7 @@ while IFS= read -r commit_sha; do
)
if [[ -z "$source_sha" ]]; then
error "Commit $commit_sha ($subject) is missing cherry-pick provenance. $REMEDIATION_HINT If version differences prevent using git cherry-pick -x, manually add '(cherry picked from commit <sha>)' to the commit message."
error "Commit $commit_sha ($subject) is missing cherry-pick provenance. $REMEDIATION_HINT"
failed=1
continue
fi
+9 -6
View File
@@ -29,13 +29,13 @@ jobs:
steps:
- name: Checkout code
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
with:
fetch-depth: 0
persist-credentials: false
- name: Setup UV and Python
uses: astral-sh/setup-uv@11f9893b081a58869d3b5fccaea48c9e9e46f990 # v8.3.2
uses: astral-sh/setup-uv@fac544c07dec837d0ccb6301d7b5580bf5edae39 # v8.2.0
with:
enable-cache: true
python-version: ${{ matrix.python-version }}
@@ -47,6 +47,9 @@ jobs:
- name: Install dependencies
run: uv sync --project api --dev
- name: Run dify config tests
run: uv run --project api pytest api/tests/unit_tests/configs/test_env_consistency.py
- name: Run Unit Tests
run: |
uv run --project api pytest \
@@ -88,13 +91,13 @@ jobs:
steps:
- name: Checkout code
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
with:
fetch-depth: 0
persist-credentials: false
- name: Setup UV and Python
uses: astral-sh/setup-uv@11f9893b081a58869d3b5fccaea48c9e9e46f990 # v8.3.2
uses: astral-sh/setup-uv@fac544c07dec837d0ccb6301d7b5580bf5edae39 # v8.2.0
with:
enable-cache: true
python-version: ${{ matrix.python-version }}
@@ -139,13 +142,13 @@ jobs:
steps:
- name: Checkout code
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
with:
fetch-depth: 0
persist-credentials: false
- name: Setup UV and Python
uses: astral-sh/setup-uv@11f9893b081a58869d3b5fccaea48c9e9e46f990 # v8.3.2
uses: astral-sh/setup-uv@fac544c07dec837d0ccb6301d7b5580bf5edae39 # v8.2.0
with:
enable-cache: true
python-version: "3.12"
+9 -35
View File
@@ -13,14 +13,14 @@ permissions:
jobs:
autofix:
if: github.repository == 'langgenius/dify'
runs-on: depot-ubuntu-24.04-4
runs-on: depot-ubuntu-24.04
steps:
- name: Complete merge group check
if: github.event_name == 'merge_group'
run: echo "autofix.ci updates pull request branches, not merge group refs."
- if: github.event_name != 'merge_group'
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
- name: Check Docker Compose inputs
if: github.event_name != 'merge_group'
@@ -39,22 +39,11 @@ jobs:
with:
files: |
web/**
cli/**
e2e/**
packages/**
sdks/nodejs-client/**
package.json
pnpm-lock.yaml
pnpm-workspace.yaml
.nvmrc
vite.config.ts
lint.config.ts
eslint.config.mjs
oxlint-suppressions.json
eslint-suppressions.json
.vscode/**
.github/workflows/autofix.yml
.github/workflows/style.yml
- name: Check api inputs
if: github.event_name != 'merge_group'
id: api-changes
@@ -62,18 +51,6 @@ jobs:
with:
files: |
api/**
- name: Check frontend contract inputs
if: github.event_name != 'merge_group'
id: frontend-contract-changes
uses: tj-actions/changed-files@9426d40962ed5378910ee2e21d5f8c6fcbf2dd96 # v47.0.6
with:
files: |
api/**
packages/contracts/openapi-ts.api.config.ts
packages/contracts/package.json
packages/contracts/openapi/**
pnpm-lock.yaml
pnpm-workspace.yaml
- name: Check dify-agent inputs
if: github.event_name != 'merge_group'
id: dify-agent-changes
@@ -84,12 +61,12 @@ jobs:
dify-agent/pyproject.toml
dify-agent/uv.lock
- if: github.event_name != 'merge_group'
uses: actions/setup-python@ece7cb06caefa5fff74198d8649806c4678c61a1 # v6.3.0
uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
with:
python-version: "3.11"
- if: github.event_name != 'merge_group'
uses: astral-sh/setup-uv@11f9893b081a58869d3b5fccaea48c9e9e46f990 # v8.3.2
uses: astral-sh/setup-uv@fac544c07dec837d0ccb6301d7b5580bf5edae39 # v8.2.0
- name: Generate Docker Compose
if: github.event_name != 'merge_group' && steps.docker-compose-changes.outputs.any_changed == 'true'
@@ -166,16 +143,13 @@ jobs:
uv run dev/generate_swagger_markdown_docs.py --swagger-dir ../packages/contracts/openapi --markdown-dir openapi/markdown --keep-swagger-json
- name: Generate frontend contracts
if: github.event_name != 'merge_group' && steps.frontend-contract-changes.outputs.any_changed == 'true'
run: pnpm --dir packages/contracts gen-api-contract
if: github.event_name != 'merge_group' && steps.api-changes.outputs.any_changed == 'true'
run: pnpm --dir packages/contracts gen-api-contract-from-openapi
- name: ESLint fallback autofix
- name: ESLint autofix
if: github.event_name != 'merge_group' && steps.web-changes.outputs.any_changed == 'true'
run: pnpm lint:eslint:fix --quiet || true
- name: Vite+ static autofix
if: github.event_name != 'merge_group' && (steps.web-changes.outputs.any_changed == 'true' || steps.frontend-contract-changes.outputs.any_changed == 'true')
run: vp check --fix || true
run: |
vp exec eslint --concurrency=2 --prune-suppressions --quiet || true
- if: github.event_name != 'merge_group'
uses: autofix-ci/action@c5b2d67aa2274e7b5a18224e8171550871fc7e4a # v1.3.4
+8 -29
View File
@@ -8,6 +8,8 @@ on:
- "build/**"
- "release/e-*"
- "hotfix/**"
- "feat/hitl-backend"
- "feat/rbac"
tags:
- "*"
@@ -21,7 +23,6 @@ env:
DIFY_WEB_IMAGE_NAME: ${{ vars.DIFY_WEB_IMAGE_NAME || 'langgenius/dify-web' }}
DIFY_API_IMAGE_NAME: ${{ vars.DIFY_API_IMAGE_NAME || 'langgenius/dify-api' }}
DIFY_AGENT_IMAGE_NAME: ${{ vars.DIFY_AGENT_IMAGE_NAME || 'langgenius/dify-agent-backend' }}
DIFY_AGENT_LOCAL_SANDBOX_IMAGE_NAME: ${{ vars.DIFY_AGENT_LOCAL_SANDBOX_IMAGE_NAME || 'langgenius/dify-agent-local-sandbox' }}
jobs:
build:
@@ -75,20 +76,6 @@ jobs:
file: "dify-agent/Dockerfile"
platform: linux/arm64
runs_on: depot-ubuntu-24.04-4
- service_name: "build-agent-local-sandbox-amd64"
image_name_env: "DIFY_AGENT_LOCAL_SANDBOX_IMAGE_NAME"
artifact_context: "local-sandbox"
build_context: "{{defaultContext}}:dify-agent-runtime"
file: "docker/Dockerfile"
platform: linux/amd64
runs_on: depot-ubuntu-24.04-4
- service_name: "build-agent-local-sandbox-arm64"
image_name_env: "DIFY_AGENT_LOCAL_SANDBOX_IMAGE_NAME"
artifact_context: "local-sandbox"
build_context: "{{defaultContext}}:dify-agent-runtime"
file: "docker/Dockerfile"
platform: linux/arm64
runs_on: depot-ubuntu-24.04-4
steps:
- name: Prepare
@@ -97,7 +84,7 @@ jobs:
echo "PLATFORM_PAIR=${platform//\//-}" >> $GITHUB_ENV
- name: Login to Docker Hub
uses: docker/login-action@af1e73f918a031802d376d3c8bbc3fe56130a9b0 # v4.4.0
uses: docker/login-action@650006c6eb7dba73a995cc03b0b2d7f5ca915bee # v4.2.0
with:
username: ${{ env.DOCKERHUB_USER }}
password: ${{ env.DOCKERHUB_TOKEN }}
@@ -107,7 +94,7 @@ jobs:
- name: Extract metadata for Docker
id: meta
uses: docker/metadata-action@dc802804100637a589fabce1cb79ff13a1411302 # v6.2.0
uses: docker/metadata-action@80c7e94dd9b9319bd5eb7a0e0fe9291e23a2a2e9 # v6.1.0
with:
images: ${{ env[matrix.image_name_env] }}
@@ -120,7 +107,6 @@ jobs:
file: ${{ matrix.file }}
platforms: ${{ matrix.platform }}
build-args: COMMIT_SHA=${{ fromJSON(steps.meta.outputs.json).labels['org.opencontainers.image.revision'] }}
build-contexts: ${{ matrix.build_contexts }}
labels: ${{ steps.meta.outputs.labels }}
outputs: type=image,name=${{ env[matrix.image_name_env] }},push-by-digest=true,name-canonical=true,push=true
@@ -155,20 +141,16 @@ jobs:
- service_name: "validate-agent-amd64"
build_context: "{{defaultContext}}"
file: "dify-agent/Dockerfile"
- service_name: "validate-agent-local-sandbox-amd64"
build_context: "{{defaultContext}}:dify-agent-runtime"
file: "docker/Dockerfile"
steps:
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@bb05f3f5519dd87d3ba754cc423b652a5edd6d2c # v4.2.0
uses: docker/setup-buildx-action@d7f5e7f509e45cec5c76c4d5afdd7de93d0b3df5 # v4.1.0
- name: Validate Docker image
uses: docker/build-push-action@53b7df96c91f9c12dcc8a07bcb9ccacbed38856a # v7.3.0
uses: docker/build-push-action@f9f3042f7e2789586610d6e8b85c8f03e5195baf # v7.2.0
with:
push: false
context: ${{ matrix.build_context }}
file: ${{ matrix.file }}
build-contexts: ${{ matrix.build_contexts }}
platforms: linux/amd64
create-manifest:
@@ -187,9 +169,6 @@ jobs:
- service_name: "merge-agent-images"
image_name_env: "DIFY_AGENT_IMAGE_NAME"
context: "agent"
- service_name: "merge-agent-local-sandbox-images"
image_name_env: "DIFY_AGENT_LOCAL_SANDBOX_IMAGE_NAME"
context: "local-sandbox"
steps:
- name: Download digests
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
@@ -199,14 +178,14 @@ jobs:
merge-multiple: true
- name: Login to Docker Hub
uses: docker/login-action@af1e73f918a031802d376d3c8bbc3fe56130a9b0 # v4.4.0
uses: docker/login-action@650006c6eb7dba73a995cc03b0b2d7f5ca915bee # v4.2.0
with:
username: ${{ env.DOCKERHUB_USER }}
password: ${{ env.DOCKERHUB_TOKEN }}
- name: Extract metadata for Docker
id: meta
uses: docker/metadata-action@dc802804100637a589fabce1cb79ff13a1411302 # v6.2.0
uses: docker/metadata-action@80c7e94dd9b9319bd5eb7a0e0fe9291e23a2a2e9 # v6.1.0
with:
images: ${{ env[matrix.image_name_env] }}
tags: |
+6 -6
View File
@@ -79,7 +79,7 @@ jobs:
ws2_app_id: ${{ steps.out.outputs.DIFY_E2E_WS2_APP_ID }}
steps:
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v4
- uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v4
with:
ref: ${{ inputs.cli_ref || github.ref }}
persist-credentials: false
@@ -123,7 +123,7 @@ jobs:
shell: bash
steps:
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v4
- uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v4
with:
ref: ${{ inputs.cli_ref || github.ref }}
persist-credentials: false
@@ -170,7 +170,7 @@ jobs:
shell: bash
steps:
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v4
- uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v4
with:
ref: ${{ inputs.cli_ref || github.ref }}
persist-credentials: false
@@ -233,7 +233,7 @@ jobs:
shell: bash
steps:
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v4
- uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v4
with:
ref: ${{ inputs.cli_ref || github.ref }}
persist-credentials: false
@@ -295,7 +295,7 @@ jobs:
shell: bash
steps:
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v4
- uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v4
with:
ref: ${{ inputs.cli_ref || github.ref }}
persist-credentials: false
@@ -351,7 +351,7 @@ jobs:
shell: bash
steps:
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v4
- uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v4
with:
ref: ${{ inputs.cli_ref || github.ref }}
persist-credentials: false
+1 -1
View File
@@ -23,7 +23,7 @@ jobs:
working-directory: ./cli
steps:
- name: Checkout
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
with:
persist-credentials: false
fetch-depth: 0
+2 -2
View File
@@ -35,7 +35,7 @@ jobs:
dify_tag: ${{ steps.resolve.outputs.dify_tag }}
steps:
- name: Checkout
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
with:
persist-credentials: false
@@ -98,7 +98,7 @@ jobs:
DIFY_TAG: ${{ needs.validate.outputs.dify_tag }}
steps:
- name: Checkout
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
with:
persist-credentials: false
fetch-depth: 1
+1 -1
View File
@@ -24,7 +24,7 @@ jobs:
shell: bash
steps:
- name: Checkout cli ref
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
with:
ref: ${{ inputs.cli_ref || github.ref }}
persist-credentials: false
+2 -2
View File
@@ -30,7 +30,7 @@ jobs:
steps:
- name: Checkout code
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
with:
persist-credentials: false
@@ -46,7 +46,7 @@ jobs:
if: matrix.os == 'depot-ubuntu-24.04'
run: scripts/release-validate-manifest.sh
- name: CI pipeline (tree, coverage, build)
- name: CI pipeline (typecheck, lint, coverage, build)
run: pnpm run ci
- name: Report coverage
+4 -4
View File
@@ -13,13 +13,13 @@ jobs:
steps:
- name: Checkout code
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
with:
fetch-depth: 0
persist-credentials: false
- name: Setup UV and Python
uses: astral-sh/setup-uv@11f9893b081a58869d3b5fccaea48c9e9e46f990 # v8.3.2
uses: astral-sh/setup-uv@fac544c07dec837d0ccb6301d7b5580bf5edae39 # v8.2.0
with:
enable-cache: true
python-version: "3.12"
@@ -63,13 +63,13 @@ jobs:
steps:
- name: Checkout code
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
with:
fetch-depth: 0
persist-credentials: false
- name: Setup UV and Python
uses: astral-sh/setup-uv@11f9893b081a58869d3b5fccaea48c9e9e46f990 # v8.3.2
uses: astral-sh/setup-uv@fac544c07dec837d0ccb6301d7b5580bf5edae39 # v8.2.0
with:
enable-cache: true
python-version: "3.12"
+25
View File
@@ -0,0 +1,25 @@
name: Deploy HITL
on:
workflow_run:
workflows: ["Build and Push API & Web"]
branches:
- "build/feat/hitl"
types:
- completed
jobs:
deploy:
runs-on: depot-ubuntu-24.04
if: |
github.event.workflow_run.conclusion == 'success' &&
github.event.workflow_run.head_branch == 'build/feat/hitl'
steps:
- name: Deploy to server
uses: appleboy/ssh-action@0ff4204d59e8e51228ff73bce53f80d53301dee2 # v1.2.5
with:
host: ${{ secrets.HITL_SSH_HOST }}
username: ${{ secrets.SSH_USER }}
key: ${{ secrets.SSH_PRIVATE_KEY }}
script: |
${{ vars.SSH_SCRIPT || secrets.SSH_SCRIPT }}
+2 -22
View File
@@ -13,11 +13,6 @@ on:
- dify-agent/README.md
- dify-agent/src/**
- web/Dockerfile
- dify-agent-runtime/docker/Dockerfile
- dify-agent-runtime/go.mod
- dify-agent-runtime/go.sum
- dify-agent-runtime/cmd/**
- dify-agent-runtime/internal/**
concurrency:
group: docker-build-${{ github.head_ref || github.run_id }}
@@ -53,16 +48,6 @@ jobs:
runs_on: depot-ubuntu-24.04-4
context: "{{defaultContext}}"
file: "web/Dockerfile"
- service_name: "local-sandbox-amd64"
platform: linux/amd64
runs_on: depot-ubuntu-24.04-4
context: "{{defaultContext}}:dify-agent-runtime"
file: "docker/Dockerfile"
- service_name: "local-sandbox-arm64"
platform: linux/arm64
runs_on: depot-ubuntu-24.04-4
context: "{{defaultContext}}:dify-agent-runtime"
file: "docker/Dockerfile"
steps:
- name: Set up Depot CLI
uses: depot/setup-action@15c09a5f77a0840ad4bce955686522a257853461 # v1.7.1
@@ -74,7 +59,6 @@ jobs:
push: false
context: ${{ matrix.context }}
file: ${{ matrix.file }}
build-contexts: ${{ matrix.build_contexts }}
platforms: ${{ matrix.platform }}
build-docker-fork:
@@ -91,18 +75,14 @@ jobs:
- service_name: "web-amd64"
context: "{{defaultContext}}"
file: "web/Dockerfile"
- service_name: "local-sandbox-amd64"
context: "{{defaultContext}}:dify-agent-runtime"
file: "docker/Dockerfile"
steps:
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@bb05f3f5519dd87d3ba754cc423b652a5edd6d2c # v4.2.0
uses: docker/setup-buildx-action@d7f5e7f509e45cec5c76c4d5afdd7de93d0b3df5 # v4.1.0
- name: Build Docker Image
uses: docker/build-push-action@53b7df96c91f9c12dcc8a07bcb9ccacbed38856a # v7.3.0
uses: docker/build-push-action@f9f3042f7e2789586610d6e8b85c8f03e5195baf # v7.2.0
with:
push: false
context: ${{ matrix.context }}
file: ${{ matrix.file }}
build-contexts: ${{ matrix.build_contexts }}
platforms: linux/amd64
+1 -1
View File
@@ -24,7 +24,7 @@ jobs:
name: Require cherry-pick provenance
runs-on: depot-ubuntu-24.04
steps:
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
- uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
with:
fetch-depth: 0
+1 -1
View File
@@ -9,6 +9,6 @@ jobs:
pull-requests: write
runs-on: depot-ubuntu-24.04
steps:
- uses: actions/labeler@b8dd2d9be0f68b860e7dae5dae7d772984eacd6d # v6.2.0
- uses: actions/labeler@f27b608878404679385c85cfa523b85ccb86e213 # v6.1.0
with:
sync-labels: true
+7 -83
View File
@@ -6,6 +6,8 @@ on:
merge_group:
branches: ["main"]
types: [checks_requested]
push:
branches: ["main"]
permissions:
actions: write
@@ -27,7 +29,7 @@ jobs:
steps:
- id: skip_check
continue-on-error: true
uses: fkirc/skip-duplicate-actions@b974a9395958c231af965b70070979a577efa578 # v5.3.2
uses: fkirc/skip-duplicate-actions@f75f66ce1886f00957d99748a42c724f4330bdcf # v5.3.1
with:
cancel_others: 'true'
concurrent_skipping: same_content_newer
@@ -45,23 +47,14 @@ jobs:
web-changed: ${{ steps.changes.outputs.web }}
vdb-changed: ${{ steps.changes.outputs.vdb }}
migration-changed: ${{ steps.changes.outputs.migration }}
sandbox-runtime-changed: ${{ steps.changes.outputs.sandbox-runtime }}
steps:
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
- uses: dorny/paths-filter@7b450fff21473bca461d4b92ce414b9d0420d706 # v4.0.2
- uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
- uses: dorny/paths-filter@fbd0ab8f3e69293af611ebaee6363fc25e6d187d # v4.0.1
id: changes
with:
filters: |
api:
- 'api/**'
- 'scripts/ast_grep_guard.py'
- 'scripts/check_no_new_getattr.py'
- 'scripts/check_no_new_controller_sqlalchemy.py'
- 'scripts/lint_controller_sqlalchemy.py'
- 'scripts/ast_grep_rules/no_new_getattr.yml'
- 'scripts/ast_grep_rules/no_new_controller_sqlalchemy.yml'
- '.github/workflows/style.yml'
- '.github/workflows/main-ci.yml'
- '.github/workflows/api-tests.yml'
- 'docker/.env.example'
- 'docker/envs/middleware.env.example'
@@ -72,15 +65,15 @@ jobs:
- 'docker/volumes/sandbox/conf/**'
cli:
- 'cli/**'
- 'packages/contracts/**'
- 'packages/tsconfig/**'
- 'package.json'
- 'pnpm-lock.yaml'
- 'pnpm-workspace.yaml'
- 'lint.config.ts'
- 'eslint.config.mjs'
- '.npmrc'
- '.nvmrc'
- '.github/workflows/cli-tests.yml'
- '.github/workflows/cli-docker-build.yml'
- '.github/actions/setup-web/**'
web:
- 'web/**'
@@ -105,7 +98,6 @@ jobs:
- 'docker/docker-compose.middleware.yaml'
- 'docker/envs/middleware.env.example'
- '.github/workflows/web-e2e.yml'
- '.github/workflows/main-ci.yml'
- '.github/actions/setup-web/**'
vdb:
- 'api/core/rag/datasource/**'
@@ -132,9 +124,6 @@ jobs:
- 'docker/volumes/**'
- 'api/uv.lock'
- 'api/pyproject.toml'
sandbox-runtime:
- 'dify-agent-runtime/**'
- '.github/workflows/sandbox-runtime-tests.yml'
migration:
- 'api/migrations/**'
- 'api/.env.example'
@@ -335,9 +324,6 @@ jobs:
- check-changes
if: needs.pre_job.outputs.should_skip != 'true' && needs.check-changes.outputs.e2e-changed == 'true'
uses: ./.github/workflows/web-e2e.yml
with:
run-external-runtime: false
secrets: inherit
web-e2e-skip:
name: Skip Web Full-Stack E2E
@@ -395,8 +381,6 @@ jobs:
needs: pre_job
if: needs.pre_job.outputs.should_skip != 'true'
uses: ./.github/workflows/style.yml
with:
base-rev: ${{ github.event.pull_request.base.sha || github.event.merge_group.base_sha }}
vdb-tests-run:
name: Run VDB Tests
@@ -515,63 +499,3 @@ jobs:
echo "DB migration tests were not required, but the skip job finished with result: $SKIP_RESULT" >&2
exit 1
sandbox-runtime-tests-run:
name: Run Sandbox Runtime Tests
needs:
- pre_job
- check-changes
if: needs.pre_job.outputs.should_skip != 'true' && needs.check-changes.outputs.sandbox-runtime-changed == 'true'
uses: ./.github/workflows/sandbox-runtime-tests.yml
secrets: inherit
sandbox-runtime-tests-skip:
name: Skip Sandbox Runtime Tests
needs:
- pre_job
- check-changes
if: needs.pre_job.outputs.should_skip != 'true' && needs.check-changes.outputs.sandbox-runtime-changed != 'true'
runs-on: depot-ubuntu-24.04
steps:
- name: Report skipped sandbox runtime tests
run: echo "No sandbox-runtime-related changes detected; skipping sandbox runtime tests."
sandbox-runtime-tests:
name: Sandbox Runtime Tests
if: ${{ always() }}
needs:
- pre_job
- check-changes
- sandbox-runtime-tests-run
- sandbox-runtime-tests-skip
runs-on: depot-ubuntu-24.04
steps:
- name: Finalize Sandbox Runtime Tests status
env:
SHOULD_SKIP_WORKFLOW: ${{ needs.pre_job.outputs.should_skip }}
TESTS_CHANGED: ${{ needs.check-changes.outputs.sandbox-runtime-changed }}
RUN_RESULT: ${{ needs.sandbox-runtime-tests-run.result }}
SKIP_RESULT: ${{ needs.sandbox-runtime-tests-skip.result }}
run: |
if [[ "$SHOULD_SKIP_WORKFLOW" == 'true' ]]; then
echo "Sandbox runtime tests were skipped because this workflow run duplicated a successful or newer run."
exit 0
fi
if [[ "$TESTS_CHANGED" == 'true' ]]; then
if [[ "$RUN_RESULT" == 'success' ]]; then
echo "Sandbox runtime tests ran successfully."
exit 0
fi
echo "Sandbox runtime tests were required but finished with result: $RUN_RESULT" >&2
exit 1
fi
if [[ "$SKIP_RESULT" == 'success' ]]; then
echo "Sandbox runtime tests were skipped because no sandbox-runtime-related files changed."
exit 0
fi
echo "Sandbox runtime tests were not required, but the skip job finished with result: $SKIP_RESULT" >&2
exit 1
-94
View File
@@ -1,94 +0,0 @@
name: Post-Merge Checks
on:
push:
branches: ["main"]
permissions:
contents: read
concurrency:
group: post-merge-${{ github.sha }}
cancel-in-progress: false
jobs:
check-changes:
name: Check Changed Files
runs-on: depot-ubuntu-24.04
outputs:
external-e2e-changed: ${{ steps.changes.outputs.external_e2e }}
steps:
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
- uses: dorny/paths-filter@7b450fff21473bca461d4b92ce414b9d0420d706 # v4.0.2
id: changes
with:
filters: |
external_e2e:
- 'e2e/features/agent-v2/**'
- 'e2e/features/step-definitions/agent-v2/**'
- 'e2e/features/step-definitions/common/**'
- 'e2e/features/support/**'
- 'e2e/fixtures/auth.ts'
- 'e2e/fixtures/test-materials/**'
- 'e2e/scripts/**'
- 'e2e/support/**'
- 'e2e/cucumber.config.ts'
- 'e2e/package.json'
- 'e2e/test-env.ts'
- 'e2e/tsconfig.json'
- 'e2e/tsx-register.js'
- 'package.json'
- 'pnpm-lock.yaml'
- '.nvmrc'
- '.github/workflows/post-merge.yml'
- '.github/workflows/web-e2e.yml'
- '.github/actions/setup-web/**'
- 'docker/docker-compose.middleware.yaml'
- 'docker/envs/middleware.env.example'
- 'dify-agent/**'
- 'dify-agent-runtime/**'
- 'api/pyproject.toml'
- 'api/uv.lock'
- 'api/tests/integration_tests/.env.example'
- 'api/clients/agent_backend/**'
- 'api/core/app/apps/agent_app/**'
- 'api/core/workflow/nodes/agent_v2/**'
- 'api/controllers/console/agent/**'
- 'api/services/agent/**'
- 'api/core/plugin/**'
- 'api/services/plugin/**'
- 'api/core/tools/**'
- 'api/services/tools/**'
- 'packages/contracts/package.json'
- 'packages/contracts/generated/api/console/agent/**'
- 'packages/contracts/generated/api/console/apps/**'
- 'packages/contracts/generated/api/console/datasets/**'
- 'packages/contracts/generated/api/console/orpc.gen.ts'
- 'packages/contracts/generated/api/console/workspaces/**'
- 'packages/contracts/generated/api/service/**'
- 'web/features/agent-v2/**'
- 'web/app/(commonLayout)/agents/**'
- 'web/app/(commonLayout)/@detailSidebar/agents/**'
- 'web/app/(commonLayout)/roster/**'
- 'web/app/components/base/chat/chat/**'
- 'web/app/components/base/voice-input/**'
- 'web/app/components/workflow/nodes/agent-v2/**'
- 'web/i18n/en-US/agent-v-2.json'
- 'web/i18n/en-US/common.json'
- 'web/service/base.ts'
- 'web/service/client.ts'
- 'web/service/console-link.ts'
- 'web/service/console-openapi-url.ts'
- 'web/service/console-router-loader.ts'
- 'web/service/share.ts'
- 'web/package.json'
- 'pnpm-workspace.yaml'
external-e2e:
name: External Runtime E2E
needs: check-changes
if: needs.check-changes.outputs.external-e2e-changed == 'true'
uses: ./.github/workflows/web-e2e.yml
with:
run-external-runtime: true
secrets: inherit
+2 -2
View File
@@ -17,12 +17,12 @@ jobs:
pull-requests: write
steps:
- name: Checkout PR branch
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
with:
fetch-depth: 0
- name: Setup Python & UV
uses: astral-sh/setup-uv@11f9893b081a58869d3b5fccaea48c9e9e46f990 # v8.3.2
uses: astral-sh/setup-uv@fac544c07dec837d0ccb6301d7b5580bf5edae39 # v8.2.0
with:
enable-cache: true
@@ -21,10 +21,10 @@ jobs:
if: ${{ github.event.workflow_run.conclusion == 'success' && github.event.workflow_run.pull_requests[0].head.repo.full_name != github.repository }}
steps:
- name: Checkout default branch (trusted code)
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
- name: Setup Python & UV
uses: astral-sh/setup-uv@11f9893b081a58869d3b5fccaea48c9e9e46f990 # v8.3.2
uses: astral-sh/setup-uv@fac544c07dec837d0ccb6301d7b5580bf5edae39 # v8.2.0
with:
enable-cache: true
+2 -2
View File
@@ -17,12 +17,12 @@ jobs:
pull-requests: write
steps:
- name: Checkout PR branch
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
with:
fetch-depth: 0
- name: Setup Python & UV
uses: astral-sh/setup-uv@11f9893b081a58869d3b5fccaea48c9e9e46f990 # v8.3.2
uses: astral-sh/setup-uv@fac544c07dec837d0ccb6301d7b5580bf5edae39 # v8.2.0
with:
enable-cache: true
@@ -1,98 +0,0 @@
name: Sandbox Runtime Tests
on:
workflow_call:
permissions:
contents: read
concurrency:
group: sandbox-runtime-tests-${{ github.head_ref || github.run_id }}
cancel-in-progress: true
jobs:
sandbox-runtime-unit:
name: Sandbox Runtime Unit Tests
runs-on: depot-ubuntu-24.04
defaults:
run:
shell: bash
working-directory: dify-agent-runtime
steps:
- name: Checkout code
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
with:
fetch-depth: 0
persist-credentials: false
- name: Setup Go
uses: actions/setup-go@b7ad1dad31e06c5925ef5d2fc7ad053ef454303e # v7.0.0
with:
go-version-file: dify-agent-runtime/go.mod
cache-dependency-path: dify-agent-runtime/go.sum
- name: Run unit tests
run: go test -race -count=1 ./...
sandbox-runtime-lint:
name: Sandbox Runtime Lint
runs-on: depot-ubuntu-24.04
defaults:
run:
shell: bash
working-directory: dify-agent-runtime
steps:
- name: Checkout code
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
with:
fetch-depth: 0
persist-credentials: false
- name: Setup Go
uses: actions/setup-go@b7ad1dad31e06c5925ef5d2fc7ad053ef454303e # v7.0.0
with:
go-version-file: dify-agent-runtime/go.mod
cache-dependency-path: dify-agent-runtime/go.sum
- name: Run golangci-lint
uses: golangci/golangci-lint-action@ba0d7d2ec06a0ea1cb5fa41b2e4a3ab91d21278a # v6.5.0
with:
working-directory: dify-agent-runtime
version: latest
sandbox-runtime-integration:
name: Sandbox Runtime Integration Tests
runs-on: depot-ubuntu-24.04
defaults:
run:
shell: bash
working-directory: dify-agent-runtime
steps:
- name: Checkout code
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
with:
fetch-depth: 0
persist-credentials: false
- name: Setup Go
uses: actions/setup-go@b7ad1dad31e06c5925ef5d2fc7ad053ef454303e # v7.0.0
with:
go-version-file: dify-agent-runtime/go.mod
cache-dependency-path: dify-agent-runtime/go.sum
- name: Build and start runtime
run: make integration-up
- name: Run integration tests
run: make integration-test
- name: Dump container logs on failure
if: failure()
run: make integration-logs || true
- name: Stop runtime container
if: always()
run: make integration-down
+1 -1
View File
@@ -18,7 +18,7 @@ jobs:
pull-requests: write
steps:
- uses: actions/stale@1e223db275d687790206a7acac4d1a11bd6fe629 # v10.4.0
- uses: actions/stale@eb5cf3af3ac0a1aa4c9c45633dd1ae542a27a899 # v10.3.0
with:
days-before-issue-stale: 15
days-before-issue-close: 3
+30 -45
View File
@@ -2,10 +2,6 @@ name: Style check
on:
workflow_call:
inputs:
base-rev:
required: true
type: string
concurrency:
group: style-${{ github.head_ref || github.run_id }}
@@ -23,10 +19,9 @@ jobs:
steps:
- name: Checkout code
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
with:
persist-credentials: false
fetch-depth: 0
- name: Check changed files
id: changed-files
@@ -34,18 +29,11 @@ jobs:
with:
files: |
api/**
scripts/ast_grep_guard.py
scripts/check_no_new_getattr.py
scripts/check_no_new_controller_sqlalchemy.py
scripts/lint_controller_sqlalchemy.py
scripts/ast_grep_rules/no_new_getattr.yml
scripts/ast_grep_rules/no_new_controller_sqlalchemy.yml
.github/workflows/style.yml
.github/workflows/main-ci.yml
- name: Setup UV and Python
if: steps.changed-files.outputs.any_changed == 'true'
uses: astral-sh/setup-uv@11f9893b081a58869d3b5fccaea48c9e9e46f990 # v8.3.2
uses: astral-sh/setup-uv@fac544c07dec837d0ccb6301d7b5580bf5edae39 # v8.2.0
with:
enable-cache: false
python-version: "3.12"
@@ -63,18 +51,8 @@ jobs:
if: steps.changed-files.outputs.any_changed == 'true'
run: uv run --project api --dev python api/dev/lint_response_contracts.py --fail-on-mismatch
- name: Run No New Getattr Guard
if: steps.changed-files.outputs.any_changed == 'true'
run: uv run --project api python scripts/check_no_new_getattr.py --base-rev "${{ inputs.base-rev }}"
- name: Run No New Controller SQLAlchemy Guard
if: steps.changed-files.outputs.any_changed == 'true'
run: uv run --project api python scripts/check_no_new_controller_sqlalchemy.py --base-rev "${{ inputs.base-rev }}"
- name: Run Type Checks
if: steps.changed-files.outputs.any_changed == 'true'
env:
PYREFLY_OUTPUT_FORMAT: github
run: make type-check-core
- name: Dotenv check
@@ -93,7 +71,7 @@ jobs:
steps:
- name: Checkout code
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
with:
persist-credentials: false
@@ -127,24 +105,16 @@ jobs:
if: steps.changed-files.outputs.any_changed == 'true'
run: vp run knip
- name: Web dead code check production
if: steps.changed-files.outputs.any_changed == 'true'
run: vp run knip:production
- name: Web production unused declarations check
if: steps.changed-files.outputs.any_changed == 'true'
run: vp run knip:production-unused-check
ts-common-style:
name: TS Common
runs-on: depot-ubuntu-24.04-4
runs-on: depot-ubuntu-24.04
permissions:
checks: write
pull-requests: read
steps:
- name: Checkout code
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
with:
persist-credentials: false
@@ -162,13 +132,7 @@ jobs:
pnpm-lock.yaml
pnpm-workspace.yaml
.nvmrc
vite.config.ts
lint.config.ts
eslint.config.mjs
oxlint-suppressions.json
eslint-suppressions.json
.vscode/**
.github/workflows/autofix.yml
.github/workflows/style.yml
.github/actions/setup-web/**
@@ -176,9 +140,30 @@ jobs:
if: steps.changed-files.outputs.any_changed == 'true'
uses: ./.github/actions/setup-web
- name: Static check
- name: Restore ESLint cache
if: steps.changed-files.outputs.any_changed == 'true'
run: pnpm -w check
id: eslint-cache-restore
uses: actions/cache/restore@27d5ce7f107fe9357f9df03efb73ab90386fccae # v5.0.5
with:
path: .eslintcache
key: ${{ runner.os }}-eslint-${{ hashFiles('pnpm-lock.yaml', 'eslint.config.mjs', 'web/eslint.config.mjs', 'web/eslint.constants.mjs', 'web/plugins/eslint/**') }}-${{ github.sha }}
restore-keys: |
${{ runner.os }}-eslint-${{ hashFiles('pnpm-lock.yaml', 'eslint.config.mjs', 'web/eslint.config.mjs', 'web/eslint.constants.mjs', 'web/plugins/eslint/**') }}-
- name: Style check
if: steps.changed-files.outputs.any_changed == 'true'
run: vp run lint:ci
- name: Type check
if: steps.changed-files.outputs.any_changed == 'true'
run: vp run type-check
- name: Save ESLint cache
if: steps.changed-files.outputs.any_changed == 'true' && success() && steps.eslint-cache-restore.outputs.cache-hit != 'true'
uses: actions/cache/save@27d5ce7f107fe9357f9df03efb73ab90386fccae # v5.0.5
with:
path: .eslintcache
key: ${{ steps.eslint-cache-restore.outputs.cache-primary-key }}
superlinter:
name: SuperLinter
@@ -186,7 +171,7 @@ jobs:
steps:
- name: Checkout code
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
with:
fetch-depth: 0
persist-credentials: false
@@ -204,7 +189,7 @@ jobs:
.editorconfig
- name: Super-linter
uses: super-linter/super-linter/slim@4ce20838b8ab83717e78138c5b3a1407148e0918 # v8.7.0
uses: super-linter/super-linter/slim@9e863354e3ff62e0727d37183162c4a88873df41 # v8.6.0
if: steps.changed-files.outputs.any_changed == 'true'
env:
BASH_SEVERITY: warning
+2 -2
View File
@@ -24,12 +24,12 @@ jobs:
working-directory: sdks/nodejs-client
steps:
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
- uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
with:
persist-credentials: false
- name: Use Node.js
uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0
with:
node-version: 22
cache: ''
+4 -4
View File
@@ -40,7 +40,7 @@ jobs:
steps:
- name: Checkout repository
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
with:
fetch-depth: 0
token: ${{ secrets.GITHUB_TOKEN }}
@@ -158,7 +158,7 @@ jobs:
- name: Run Claude Code for Translation Sync
if: steps.context.outputs.CHANGED_FILES != ''
uses: anthropics/claude-code-action@af0559ee4f514d1ef21826982bed13f7edc3c35e # v1.0.178
uses: anthropics/claude-code-action@806af32823ef69c8ef357086c573a902af641307 # v1.0.151
with:
anthropic_api_key: ${{ secrets.ANTHROPIC_API_KEY }}
github_token: ${{ secrets.GITHUB_TOKEN }}
@@ -223,7 +223,7 @@ jobs:
- Match the existing terminology and register used by each locale.
- Prefer one Edit per file when stable, but prioritize correctness over batching.
6. Verify only the edited files.
- Run `vp fmt <repo-relative edited paths such as web/i18n/...>`
- Run `vp run dify-web#lint:fix --quiet -- <relative edited i18n file paths under web/>`
- Run `vp run dify-web#i18n:check ${{ steps.context.outputs.FILE_ARGS }} ${{ steps.context.outputs.LANG_ARGS }}`
- If verification fails, fix the remaining problems before continuing.
7. Stop after the scoped locale files are updated and verification passes.
@@ -322,7 +322,7 @@ jobs:
'## Verification',
'',
`- \`vp run dify-web#i18n:check --file ${process.env.FILES_IN_SCOPE} --lang ${process.env.TARGET_LANGS}\``,
`- \`vp fmt <repo-relative edited paths such as web/i18n/...>\``,
`- \`vp run dify-web#lint:fix --quiet -- <edited i18n files under web/>\``,
'',
'## Notes',
'',
+1 -1
View File
@@ -21,7 +21,7 @@ jobs:
steps:
- name: Checkout repository
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
with:
fetch-depth: 0
+2 -2
View File
@@ -24,7 +24,7 @@ jobs:
steps:
- name: Checkout code
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
with:
persist-credentials: false
@@ -36,7 +36,7 @@ jobs:
remove_tool_cache: true
- name: Setup UV and Python
uses: astral-sh/setup-uv@11f9893b081a58869d3b5fccaea48c9e9e46f990 # v8.3.2
uses: astral-sh/setup-uv@fac544c07dec837d0ccb6301d7b5580bf5edae39 # v8.2.0
with:
enable-cache: true
python-version: ${{ matrix.python-version }}
+2 -2
View File
@@ -21,7 +21,7 @@ jobs:
steps:
- name: Checkout code
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
with:
persist-credentials: false
@@ -33,7 +33,7 @@ jobs:
remove_tool_cache: true
- name: Setup UV and Python
uses: astral-sh/setup-uv@11f9893b081a58869d3b5fccaea48c9e9e46f990 # v8.3.2
uses: astral-sh/setup-uv@fac544c07dec837d0ccb6301d7b5580bf5edae39 # v8.2.0
with:
enable-cache: true
python-version: ${{ matrix.python-version }}
+5 -125
View File
@@ -2,11 +2,6 @@ name: Web Full-Stack E2E
on:
workflow_call:
inputs:
run-external-runtime:
description: Run only the prepared and external runtime suite instead of the core suites.
required: true
type: boolean
permissions:
contents: read
@@ -19,14 +14,13 @@ jobs:
test:
name: Web Full-Stack E2E
runs-on: depot-ubuntu-24.04-4
timeout-minutes: 120
defaults:
run:
shell: bash
steps:
- name: Checkout code
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
with:
persist-credentials: false
@@ -34,28 +28,20 @@ jobs:
uses: ./.github/actions/setup-web
- name: Setup UV and Python
uses: astral-sh/setup-uv@11f9893b081a58869d3b5fccaea48c9e9e46f990 # v8.3.2
uses: astral-sh/setup-uv@fac544c07dec837d0ccb6301d7b5580bf5edae39 # v8.2.0
with:
enable-cache: true
python-version: "3.12"
cache-dependency-glob: |
api/uv.lock
dify-agent/uv.lock
cache-dependency-glob: api/uv.lock
- name: Install API dependencies
run: uv sync --project api --dev
- name: Run E2E support unit tests
if: ${{ !inputs.run-external-runtime }}
working-directory: ./e2e
run: vp run test:unit
- name: Install Playwright browser
working-directory: ./e2e
run: vp run e2e:install
- name: Run isolated source-api and built-web Cucumber E2E tests
if: ${{ !inputs.run-external-runtime }}
working-directory: ./e2e
env:
E2E_ADMIN_EMAIL: e2e-admin@example.com
@@ -65,105 +51,12 @@ jobs:
E2E_INIT_PASSWORD: E2eInit12345
run: vp run e2e:full
- name: Preserve Chromium E2E report and logs
if: ${{ !cancelled() && !inputs.run-external-runtime }}
run: |
if [[ -d e2e/cucumber-report ]]; then
mv e2e/cucumber-report e2e/cucumber-report-non-external
fi
if [[ -d e2e/.logs ]]; then
mv e2e/.logs e2e/.logs-non-external
fi
- name: Run WebKit keyboard and browser smoke tests
if: ${{ !inputs.run-external-runtime }}
working-directory: ./e2e
env:
E2E_ADMIN_EMAIL: e2e-admin@example.com
E2E_ADMIN_NAME: E2E Admin
E2E_ADMIN_PASSWORD: E2eAdmin12345
E2E_BROWSER: webkit
E2E_INIT_PASSWORD: E2eInit12345
run: |
teardown_webkit_smoke() {
local run_status=$?
trap - EXIT
if ! vp run e2e:middleware:down; then
echo "::error title=E2E teardown failed::WebKit smoke middleware did not shut down cleanly."
if [[ "$run_status" -eq 0 ]]; then
run_status=1
fi
fi
exit "$run_status"
}
trap teardown_webkit_smoke EXIT
vp run e2e:middleware:up
vp run e2e -- --tags '@browser-smoke'
- name: Preserve WebKit E2E report and logs
if: ${{ !cancelled() && !inputs.run-external-runtime }}
run: |
if [[ -d e2e/cucumber-report ]]; then
mv e2e/cucumber-report e2e/cucumber-report-webkit
fi
if [[ -d e2e/.logs ]]; then
mv e2e/.logs e2e/.logs-webkit
fi
- name: Run prepared and external runtime E2E tests
if: ${{ inputs.run-external-runtime }}
working-directory: ./e2e
env:
E2E_ADMIN_EMAIL: e2e-admin@example.com
E2E_ADMIN_NAME: E2E Admin
E2E_ADMIN_PASSWORD: E2eAdmin12345
E2E_AGENT_DECISION_MODEL_NAME: ${{ vars.E2E_AGENT_DECISION_MODEL_NAME || 'gpt-5.5' }}
E2E_AGENT_DECISION_MODEL_PROVIDER: ${{ vars.E2E_AGENT_DECISION_MODEL_PROVIDER || 'openai' }}
E2E_AGENT_DECISION_MODEL_TYPE: ${{ vars.E2E_AGENT_DECISION_MODEL_TYPE || 'llm' }}
E2E_FORCE_WEB_BUILD: "1"
E2E_INIT_PASSWORD: E2eInit12345
E2E_MARKETPLACE_API_URL: ${{ vars.E2E_MARKETPLACE_API_URL }}
E2E_MARKETPLACE_PLUGIN_IDS: ${{ vars.E2E_MARKETPLACE_PLUGIN_IDS }}
E2E_MODEL_PROVIDER_CREDENTIALS_JSON: ${{ secrets.E2E_MODEL_PROVIDER_CREDENTIALS_JSON }}
E2E_SPEECH_TO_TEXT_MODEL_NAME: ${{ vars.E2E_SPEECH_TO_TEXT_MODEL_NAME || 'gpt-4o-mini-transcribe' }}
E2E_SPEECH_TO_TEXT_MODEL_PROVIDER: ${{ vars.E2E_SPEECH_TO_TEXT_MODEL_PROVIDER || 'openai' }}
E2E_START_AGENT_BACKEND: ${{ vars.E2E_START_AGENT_BACKEND || '1' }}
E2E_STABLE_MODEL_NAME: ${{ vars.E2E_STABLE_MODEL_NAME || 'gpt-5-nano' }}
E2E_STABLE_MODEL_PROVIDER: ${{ vars.E2E_STABLE_MODEL_PROVIDER || 'openai' }}
E2E_STABLE_MODEL_TYPE: ${{ vars.E2E_STABLE_MODEL_TYPE || 'llm' }}
run: |
if [[ -z "${E2E_MODEL_PROVIDER_CREDENTIALS_JSON}" ]]; then
echo "E2E_MODEL_PROVIDER_CREDENTIALS_JSON is required for external runtime E2E." >&2
exit 1
fi
teardown_external_runtime() {
local run_status=$?
trap - EXIT
if ! vp run e2e:middleware:down; then
echo "::error title=E2E teardown failed::External runtime middleware did not shut down cleanly."
if [[ "$run_status" -eq 0 ]]; then
run_status=1
fi
fi
exit "$run_status"
}
trap teardown_external_runtime EXIT
vp run e2e:middleware:up
vp run e2e:post-merge:prepare
vp run e2e:post-merge
- name: Upload Cucumber report
if: ${{ !cancelled() }}
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: cucumber-report
path: |
e2e/cucumber-report
e2e/cucumber-report-non-external
e2e/cucumber-report-webkit
path: e2e/cucumber-report
retention-days: 7
- name: Upload E2E logs
@@ -171,18 +64,5 @@ jobs:
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: e2e-logs
path: |
e2e/.logs/*.log
e2e/.logs-non-external/*.log
e2e/.logs-webkit/*.log
include-hidden-files: true
retention-days: 7
- name: Upload E2E seed report
if: ${{ !cancelled() && inputs.run-external-runtime }}
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: e2e-seed-report
path: e2e/seed-report
if-no-files-found: ignore
path: e2e/.logs
retention-days: 7
+6 -4
View File
@@ -17,6 +17,8 @@ jobs:
test:
name: Web Tests (${{ matrix.shardIndex }}/${{ matrix.shardTotal }})
runs-on: depot-ubuntu-24.04-4
env:
VITEST_COVERAGE_SCOPE: app-components
strategy:
fail-fast: false
matrix:
@@ -29,7 +31,7 @@ jobs:
steps:
- name: Checkout code
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
with:
persist-credentials: false
@@ -62,7 +64,7 @@ jobs:
steps:
- name: Checkout code
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
with:
persist-credentials: false
@@ -100,7 +102,7 @@ jobs:
steps:
- name: Checkout code
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
with:
persist-credentials: false
@@ -132,7 +134,7 @@ jobs:
steps:
- name: Checkout code
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
with:
persist-credentials: false
+2 -5
View File
@@ -214,7 +214,6 @@ sdks/python-client/dify_client.egg-info
api/.vscode
# vscode Code History Extension
.history
.eslintcache
.idea/
@@ -259,9 +258,7 @@ scripts/stress-test/reports/
# Code Agent Folder
.qoder/*
.context/
.eslintcache
# Vitest local reports
web/.vitest-reports/
# dify-agent-runtime
dify-agent-runtime/bin/
dify-agent-runtime/.integration-state
+18 -4
View File
@@ -5,9 +5,7 @@
"name": "Python: API (gevent)",
"type": "debugpy",
"request": "launch",
"module": "gevent.monkey",
"args": ["--module", "app"],
"gevent": true,
"program": "${workspaceFolder}/api/app.py",
"jinja": true,
"justMyCode": true,
"cwd": "${workspaceFolder}/api",
@@ -35,6 +33,22 @@
"justMyCode": false,
"cwd": "${workspaceFolder}/api",
"python": "${workspaceFolder}/api/.venv/bin/python"
}
},
{
"name": "Next.js: debug full stack",
"type": "node",
"request": "launch",
"program": "${workspaceFolder}/web/node_modules/next/dist/bin/next",
"runtimeArgs": ["--inspect"],
"skipFiles": ["<node_internals>/**"],
"serverReadyAction": {
"action": "debugWithChrome",
"killOnServerStop": true,
"pattern": "- Local:.+(https?://.+)",
"uriFormat": "%s",
"webRoot": "${workspaceFolder}/web"
},
"cwd": "${workspaceFolder}/web"
}
]
}
+48 -31
View File
@@ -1,37 +1,54 @@
{
"cucumber.features": ["e2e/features/**/*.feature"],
"cucumber.glue": ["e2e/features/**/*.ts"],
"cucumber.features": [
"e2e/features/**/*.feature",
],
"cucumber.glue": [
"e2e/features/**/*.ts",
],
"tailwindCSS.experimental.configFile": "web/app/styles/globals.css",
// Format
"[javascript]": { "editor.defaultFormatter": "oxc.oxc-vscode" },
"[javascriptreact]": { "editor.defaultFormatter": "oxc.oxc-vscode" },
"[typescript]": { "editor.defaultFormatter": "oxc.oxc-vscode" },
"[typescriptreact]": { "editor.defaultFormatter": "oxc.oxc-vscode" },
"[json]": { "editor.defaultFormatter": "oxc.oxc-vscode" },
"[jsonc]": { "editor.defaultFormatter": "oxc.oxc-vscode" },
"[markdown]": { "editor.defaultFormatter": "oxc.oxc-vscode" },
"[mdx]": { "editor.defaultFormatter": "oxc.oxc-vscode" },
"[yaml]": { "editor.defaultFormatter": "oxc.oxc-vscode" },
"[toml]": { "editor.defaultFormatter": "oxc.oxc-vscode" },
"[css]": { "editor.defaultFormatter": "oxc.oxc-vscode" },
"[scss]": { "editor.defaultFormatter": "oxc.oxc-vscode" },
"[less]": { "editor.defaultFormatter": "oxc.oxc-vscode" },
"[html]": { "editor.defaultFormatter": "oxc.oxc-vscode" },
"[vue]": { "editor.defaultFormatter": "oxc.oxc-vscode" },
"[svelte]": { "editor.defaultFormatter": "oxc.oxc-vscode" },
"[graphql]": { "editor.defaultFormatter": "oxc.oxc-vscode" },
"[handlebars]": { "editor.defaultFormatter": "oxc.oxc-vscode" },
"editor.formatOnSave": true,
"editor.formatOnSaveMode": "file",
"oxc.fmt.configPath": "./vite.config.ts",
// Lint fix
"eslint.useFlatConfig": true,
"eslint.validate": ["json", "jsonc", "markdown", "yaml", "toml"],
// Auto fix
"editor.codeActionsOnSave": {
"source.fixAll.oxc": "explicit",
"source.fixAll.eslint": "explicit"
}
"source.fixAll.eslint": "explicit",
},
// Silent the stylistic rules in your IDE, but still auto fix them
"eslint.rules.customizations": [
{ "rule": "style/*", "severity": "off", "fixable": true },
{ "rule": "format/*", "severity": "off", "fixable": true },
{ "rule": "*-indent", "severity": "off", "fixable": true },
{ "rule": "*-spacing", "severity": "off", "fixable": true },
{ "rule": "*-spaces", "severity": "off", "fixable": true },
{ "rule": "*-order", "severity": "off", "fixable": true },
{ "rule": "*-dangle", "severity": "off", "fixable": true },
{ "rule": "*-newline", "severity": "off", "fixable": true },
{ "rule": "*quotes", "severity": "off", "fixable": true },
{ "rule": "*semi", "severity": "off", "fixable": true }
],
// Enable eslint for all supported languages
"eslint.validate": [
"javascript",
"javascriptreact",
"typescript",
"typescriptreact",
"vue",
"html",
"markdown",
"json",
"jsonc",
"yaml",
"toml",
"xml",
"gql",
"graphql",
"astro",
"svelte",
"css",
"less",
"scss",
"pcss",
"postcss"
]
}
+1 -1
View File
@@ -31,7 +31,7 @@ The codebase is split into:
## Language Style
- **Python**: Keep type hints on functions and attributes, and implement relevant special methods (e.g., `__repr__`, `__str__`). Prefer `TypedDict` over `dict` or `Mapping` for type safety and better code documentation.
- **TypeScript**: Use the strict config, run `pnpm check` for formatting, Oxlint, ESLint non-code checks, and type checking, and avoid `any` types.
- **TypeScript**: Use the strict config, rely on ESLint (`pnpm lint:fix` preferred) plus `pnpm type-check`, and avoid `any` types.
## General Practices
+2 -2
View File
@@ -66,7 +66,7 @@ How we prioritize:
1. Fork the repository
1. Before you draft a PR, please create an issue to discuss the changes you want to make
1. Create a new branch for your changes
1. Add or update tests when the change affects observable behavior or carries meaningful regression risk
1. Please add tests for your changes accordingly
1. Ensure your code passes the existing tests
1. Please link the issue in the PR description, `fixes #<issue_number>`
1. Get merged!
@@ -77,7 +77,7 @@ How we prioritize:
For setting up the frontend service, please refer to our comprehensive [guide](https://github.com/langgenius/dify/blob/main/web/README.md) in the `web/README.md` file. This document provides detailed instructions to help you set up the frontend environment properly.
**Testing**: Add focused tests when a change affects observable behavior or carries meaningful regression risk. See [web/docs/test.md](https://github.com/langgenius/dify/blob/main/web/docs/test.md) for the canonical frontend testing guidelines.
**Testing**: All React components must have comprehensive test coverage. See [web/docs/test.md](https://github.com/langgenius/dify/blob/main/web/docs/test.md) for the canonical frontend testing guidelines and follow every requirement described there.
#### Backend
+3 -20
View File
@@ -2,7 +2,6 @@
DOCKER_REGISTRY=langgenius
WEB_IMAGE=$(DOCKER_REGISTRY)/dify-web
API_IMAGE=$(DOCKER_REGISTRY)/dify-api
SANDBOX_RUNTIME_IMAGE=$(DOCKER_REGISTRY)/dify-agent-local-sandbox
VERSION=latest
DOCKER_DIR=docker
DOCKER_MIDDLEWARE_ENV=$(DOCKER_DIR)/middleware.env
@@ -107,7 +106,6 @@ test:
echo "Target: $(TARGET_TESTS)"; \
uv run --project api --dev pytest $(TARGET_TESTS); \
else \
set -e; \
echo "Running backend unit tests"; \
uv run --project api --dev pytest -p no:benchmark --timeout "$${PYTEST_TIMEOUT:-20}" -n auto \
api/tests/unit_tests \
@@ -125,7 +123,6 @@ test-all:
echo "Target: $(TARGET_TESTS)"; \
uv run --project api --dev pytest $(TARGET_TESTS); \
else \
set -e; \
echo "Running backend unit tests"; \
uv run --project api --dev pytest -p no:benchmark --timeout "$${PYTEST_TIMEOUT:-20}" -n auto \
api/tests/unit_tests \
@@ -163,13 +160,6 @@ build-api:
docker build -t $(API_IMAGE):$(VERSION) -f api/Dockerfile .
@echo "API Docker image built successfully: $(API_IMAGE):$(VERSION)"
build-sandbox-runtime:
@echo "Building sandbox runtime Docker image: $(SANDBOX_RUNTIME_IMAGE):$(VERSION)..."
docker build -t $(SANDBOX_RUNTIME_IMAGE):$(VERSION) \
-f dify-agent-runtime/docker/Dockerfile \
dify-agent-runtime
@echo "Sandbox runtime Docker image built successfully: $(SANDBOX_RUNTIME_IMAGE):$(VERSION)"
# Push Docker images
push-web:
@echo "Pushing web Docker image: $(WEB_IMAGE):$(VERSION)..."
@@ -181,20 +171,14 @@ push-api:
docker push $(API_IMAGE):$(VERSION)
@echo "API Docker image pushed successfully: $(API_IMAGE):$(VERSION)"
push-sandbox-runtime:
@echo "Pushing sandbox runtime Docker image: $(SANDBOX_RUNTIME_IMAGE):$(VERSION)..."
docker push $(SANDBOX_RUNTIME_IMAGE):$(VERSION)
@echo "Sandbox runtime Docker image pushed successfully: $(SANDBOX_RUNTIME_IMAGE):$(VERSION)"
# Build all images
build-all: build-web build-api build-sandbox-runtime
build-all: build-web build-api
# Push all images
push-all: push-web push-api push-sandbox-runtime
push-all: push-web push-api
build-push-api: build-api push-api
build-push-web: build-web push-web
build-push-sandbox-runtime: build-sandbox-runtime push-sandbox-runtime
# Build and push all images
build-push-all: build-all push-all
@@ -222,10 +206,9 @@ help:
@echo "Docker Build Targets:"
@echo " make build-web - Build web Docker image"
@echo " make build-api - Build API Docker image"
@echo " make build-sandbox-runtime - Build sandbox runtime Docker image"
@echo " make build-all - Build all Docker images"
@echo " make push-all - Push all Docker images"
@echo " make build-push-all - Build and push all Docker images"
# Phony targets
.PHONY: build-web build-api build-sandbox-runtime push-web push-api push-sandbox-runtime build-all push-all build-push-all build-push-sandbox-runtime dev-setup prepare-docker prepare-web prepare-api dev-clean help format check lint api-contract-lint type-check test test-all
.PHONY: build-web build-api push-web push-api build-all push-all build-push-all dev-setup prepare-docker prepare-web prepare-api dev-clean help format check lint api-contract-lint type-check test test-all
+7 -12
View File
@@ -15,7 +15,7 @@
<a href="https://discord.gg/FngNHpbcY7" target="_blank">
<img src="https://img.shields.io/discord/1082486657678311454?logo=discord&labelColor=%20%235462eb&logoColor=%20%23f5f5f5&color=%20%235462eb"
alt="chat on Discord"></a>
<a href="https://reddit.com/r/difyai" target="_blank">
<a href="https://reddit.com/r/difyai" target="_blank">
<img src="https://img.shields.io/reddit/subreddit-subscribers/difyai?style=plastic&logo=reddit&label=r%2Fdifyai&labelColor=white"
alt="join Reddit"></a>
<a href="https://twitter.com/intent/follow?screen_name=dify_ai" target="_blank">
@@ -71,7 +71,7 @@ Dify is an open-source LLM app development platform. Its intuitive interface com
<br/>
The easiest way to start the Dify server is through [Docker Compose](docker/docker-compose.yaml). Before running Dify with the following commands, make sure that [Docker](https://docs.docker.com/get-docker/) and Docker Compose v2.24.0 or later are installed on your machine:
The easiest way to start the Dify server is through [Docker Compose](docker/docker-compose.yaml). Before running Dify with the following commands, make sure that [Docker](https://docs.docker.com/get-docker/) and [Docker Compose](https://docs.docker.com/compose/install/) are installed on your machine:
```bash
cd dify
@@ -116,7 +116,7 @@ All of Dify's offerings come with corresponding APIs, so you could effortlessly
## Using Dify
- **Cloud <br/>**
We host a [Dify Cloud](https://dify.ai) service for anyone to try with zero setup. It provides all the capabilities of the self-deployed version, and includes 200 free GPT-4 calls in the sandbox plan. If you run into issues with Dify Cloud, [contact our Cloud support team](mailto:cloud@dify.ai?subject=%5BGitHub%5DDify%20Cloud%20Support).
We host a [Dify Cloud](https://dify.ai) service for anyone to try with zero setup. It provides all the capabilities of the self-deployed version, and includes 200 free GPT-4 calls in the sandbox plan.
- **Self-hosting Dify Community Edition<br/>**
Quickly get Dify running in your environment with this [starter guide](#quick-start).
@@ -125,6 +125,8 @@ All of Dify's offerings come with corresponding APIs, so you could effortlessly
- **Dify for enterprise / organizations<br/>**
We provide additional enterprise-centric features. [Send us an email](mailto:business@dify.ai?subject=%5BGitHub%5DBusiness%20License%20Inquiry) to discuss your enterprise needs. <br/>
> For startups and small businesses using AWS, check out [Dify Premium on AWS Marketplace](https://aws.amazon.com/marketplace/pp/prodview-t22mebxzwjhu6) and deploy it to your own AWS VPC with one click. It's an affordable AMI offering with the option to create apps with custom logo and branding.
## Staying ahead
Star Dify on GitHub and be instantly notified of new releases.
@@ -207,16 +209,9 @@ At the same time, please consider supporting Dify by sharing it on social media
<img src="https://contrib.rocks/image?repo=langgenius/dify" />
</a>
## Star History
## Star history
<!-- GitHub token name: star-history -->
<a href="https://www.star-history.com/?repos=langgenius%2Fdify&type=date&legend=top-left">
<picture>
<source media="(prefers-color-scheme: dark)" srcset="https://api.star-history.com/chart?repos=langgenius/dify&type=date&theme=dark&legend=top-left&sealed_token=p-SWD-UXZEDc5a2d0EMfgMmyCwVyRlSof0Qox68v7k4PvQPKRlCx0jDIlNztw7mbA6DEn96R50DojO9pCi5LQUlDBAoIhRswt8-GuC8K3rZQ3naJUXbuHqR_oItIW2F0NNM-7npevzw5SXp7L5mwixgqJncAvAGzuGq5zmhQfiDRSW_Jd6y8TQCZMJbt" />
<source media="(prefers-color-scheme: light)" srcset="https://api.star-history.com/chart?repos=langgenius/dify&type=date&legend=top-left&sealed_token=p-SWD-UXZEDc5a2d0EMfgMmyCwVyRlSof0Qox68v7k4PvQPKRlCx0jDIlNztw7mbA6DEn96R50DojO9pCi5LQUlDBAoIhRswt8-GuC8K3rZQ3naJUXbuHqR_oItIW2F0NNM-7npevzw5SXp7L5mwixgqJncAvAGzuGq5zmhQfiDRSW_Jd6y8TQCZMJbt" />
<img alt="Star History Chart" src="https://api.star-history.com/chart?repos=langgenius/dify&type=date&legend=top-left&sealed_token=p-SWD-UXZEDc5a2d0EMfgMmyCwVyRlSof0Qox68v7k4PvQPKRlCx0jDIlNztw7mbA6DEn96R50DojO9pCi5LQUlDBAoIhRswt8-GuC8K3rZQ3naJUXbuHqR_oItIW2F0NNM-7npevzw5SXp7L5mwixgqJncAvAGzuGq5zmhQfiDRSW_Jd6y8TQCZMJbt" />
</picture>
</a>
[![Star History Chart](https://api.star-history.com/svg?repos=langgenius/dify&type=Date)](https://star-history.com/#langgenius/dify&Date)
## Security disclosure
+1 -29
View File
@@ -36,9 +36,6 @@ FILES_ACCESS_TIMEOUT=300
# Collaboration mode toggle
ENABLE_COLLABORATION_MODE=true
# Learn app feature toggle
ENABLE_LEARN_APP=true
# Access token expiration time in minutes
ACCESS_TOKEN_EXPIRE_MINUTES=60
@@ -86,10 +83,6 @@ REDIS_RETRY_BACKOFF_CAP=10.0
REDIS_SOCKET_TIMEOUT=5.0
REDIS_SOCKET_CONNECT_TIMEOUT=5.0
REDIS_HEALTH_CHECK_INTERVAL=30
REDIS_KEEPALIVE_IDLE=30
REDIS_KEEPALIVE_INTERVAL=10
REDIS_KEEPALIVE_COUNT=10
REDIS_KEEPALIVE=true
# celery configuration
CELERY_BROKER_URL=redis://:difyai123456@localhost:${REDIS_PORT}/1
@@ -561,8 +554,6 @@ WORKFLOW_MAX_EXECUTION_STEPS=500
WORKFLOW_MAX_EXECUTION_TIME=1200
WORKFLOW_CALL_MAX_DEPTH=5
MAX_VARIABLE_SIZE=204800
# Maximum concurrent node-builder LLM calls per workflow generation request
WORKFLOW_GENERATOR_NODE_BUILDER_MAX_WORKERS=6
# GraphEngine Worker Pool Configuration
# Minimum number of workers per GraphEngine instance (default: 1)
@@ -667,28 +658,8 @@ PLUGIN_REMOTE_INSTALL_HOST=localhost
PLUGIN_MAX_PACKAGE_SIZE=15728640
PLUGIN_MODEL_SCHEMA_CACHE_TTL=3600
PLUGIN_MODEL_PROVIDERS_CACHE_TTL=86400
# Comma-separated marketplace plugin IDs whose latest versions are installed for newly registered users.
# Example: langgenius/openai,langgenius/gemini
NEW_USER_DEFAULT_PLUGIN_IDS=
# Comma-separated model_type:provider:model entries assigned after default plugins finish installing.
# Example: llm:langgenius/openai/openai:gpt-4o-mini,text-embedding:langgenius/openai/openai:text-embedding-3-small
NEW_USER_DEFAULT_MODELS=
INNER_API_KEY_FOR_PLUGIN=QaHbTe77CtuXmsfyhR7+vRjI/+XbV1AaFy691iy+kGDv2Jvy0/eAh8Y1
# Dify Agent backend
AGENT_BACKEND_BASE_URL=http://localhost:5050
AGENT_BACKEND_STREAM_READ_TIMEOUT_SECONDS=30
AGENT_BACKEND_STREAM_MAX_RECONNECTS=3
AGENT_BACKEND_RUN_TIMEOUT_SECONDS=1200
# KnowledgeFS (Dataset 2.0)
KNOWLEDGE_FS_ENABLED=false
KNOWLEDGE_FS_BASE_URL=
# Shared with KnowledgeFS; use at least 32 random characters.
KNOWLEDGE_FS_JWT_SECRET=
KNOWLEDGE_FS_SSE_READ_TIMEOUT_SECONDS=300
KNOWLEDGE_FS_TIMEOUT_SECONDS=10
# Marketplace configuration
MARKETPLACE_ENABLED=true
MARKETPLACE_API_URL=https://marketplace.dify.ai
@@ -797,6 +768,7 @@ EVENT_BUS_REDIS_CHANNEL_TYPE=pubsub
# Whether to use Redis cluster mode while use redis as event bus.
# It's highly recommended to enable this for large deployments.
EVENT_BUS_REDIS_USE_CLUSTERS=false
EVENT_BUS_LISTENER_JOIN_TIMEOUT_MS=2000
# Whether to Enable human input timeout check task
ENABLE_HUMAN_INPUT_TIMEOUT_TASK=true
-12
View File
@@ -109,18 +109,6 @@ class Example:
- Reuse existing helpers in `core/`, `services/`, and `libs/` before creating new abstractions.
- Optimise for observability: deterministic control flow, clear logging, actionable errors.
### Owner-Bound Resource References
- Resolve and validate the outer owner before binding a nested resource ID.
- For stable single-parent chains, use immutable nested `NamedTuple` refs.
- Root refs carry tenant plus root ID; child refs carry the parent ref.
- In production, construct refs through the domain ref service.
- Python allowing direct construction does not grant authorization.
- Scope every consuming query with complete owner predicates; refs are not security tokens.
- Keep polymorphic owners flat until explicit nominal owner types exist.
- Do not add generic ref bases or compatibility fields only for uniformity.
- Reconstruct internal refs from validated database state after payload or async boundaries.
### Logging & Errors
- Never use `print`; use a module-level logger:
-19
View File
@@ -1,24 +1,5 @@
from __future__ import annotations
# ``python -m app`` (docker DEBUG=true, or IDE debugging) serves through the
# gevent pywsgi server at the bottom of this file, so the stdlib must be
# monkey-patched BEFORE any other import pulls in sockets or locks. Without
# this, every request runs as a greenlet on one OS thread while blocking
# calls (LLM invokes, ``Future.result`` waits, DB I/O) pin that thread — the
# whole process freezes until the call returns. Gunicorn and Celery apply
# their own patching (see gunicorn.conf.py / celery_entrypoint.py), and
# ``flask run`` uses real Werkzeug threads, so both skip this branch.
if __name__ == "__main__":
from gevent import monkey
monkey.patch_all()
import psycogreen.gevent as psycogreen_gevent
from grpc.experimental import gevent as grpc_gevent
grpc_gevent.init_gevent()
psycogreen_gevent.patch_psycopg()
import logging
import sys
from typing import TYPE_CHECKING, cast
-14
View File
@@ -18,7 +18,6 @@ from clients.agent_backend.errors import (
AgentBackendValidationError,
)
from clients.agent_backend.event_adapter import (
AgentBackendAgentMessageDeltaInternalEvent,
AgentBackendDeferredToolCallInternalEvent,
AgentBackendInternalEvent,
AgentBackendInternalEventType,
@@ -33,8 +32,6 @@ from clients.agent_backend.factory import create_agent_backend_run_client
from clients.agent_backend.fake_client import FakeAgentBackendRunClient, FakeAgentBackendScenario
from clients.agent_backend.request_builder import (
AGENT_SOUL_PROMPT_LAYER_ID,
DIFY_CONFIG_LAYER_ID,
DIFY_CORE_TOOLS_LAYER_ID,
DIFY_EXECUTION_CONTEXT_LAYER_ID,
DIFY_KNOWLEDGE_BASE_LAYER_ID,
DIFY_PLUGIN_TOOLS_LAYER_ID,
@@ -47,23 +44,15 @@ from clients.agent_backend.request_builder import (
AgentBackendWorkflowNodeRunInput,
redact_for_agent_backend_log,
)
from clients.agent_backend.session_cleanup import (
AgentBackendSessionCleanupPayload,
AgentBackendSessionCleanupResult,
cleanup_agent_backend_session,
)
__all__ = [
"AGENT_SOUL_PROMPT_LAYER_ID",
"DIFY_CONFIG_LAYER_ID",
"DIFY_CORE_TOOLS_LAYER_ID",
"DIFY_EXECUTION_CONTEXT_LAYER_ID",
"DIFY_KNOWLEDGE_BASE_LAYER_ID",
"DIFY_PLUGIN_TOOLS_LAYER_ID",
"WORKFLOW_NODE_JOB_PROMPT_LAYER_ID",
"WORKFLOW_USER_PROMPT_LAYER_ID",
"AgentBackendAgentAppRunInput",
"AgentBackendAgentMessageDeltaInternalEvent",
"AgentBackendDeferredToolCallInternalEvent",
"AgentBackendError",
"AgentBackendHTTPError",
@@ -80,8 +69,6 @@ __all__ = [
"AgentBackendRunRequestBuilder",
"AgentBackendRunStartedInternalEvent",
"AgentBackendRunSucceededInternalEvent",
"AgentBackendSessionCleanupPayload",
"AgentBackendSessionCleanupResult",
"AgentBackendStreamError",
"AgentBackendStreamInternalEvent",
"AgentBackendTransportError",
@@ -91,7 +78,6 @@ __all__ = [
"FakeAgentBackendRunClient",
"FakeAgentBackendScenario",
"RuntimeLayerSpec",
"cleanup_agent_backend_session",
"create_agent_backend_run_client",
"extract_runtime_layer_specs",
"redact_for_agent_backend_log",
+6 -40
View File
@@ -8,7 +8,7 @@ creating another wire contract.
from __future__ import annotations
from collections.abc import Callable, Iterator
from collections.abc import Iterator
from typing import Protocol
from dify_agent.client import (
@@ -45,13 +45,7 @@ class AgentBackendRunClient(Protocol):
def cancel_run(self, run_id: str, request: CancelRunRequest | None = None) -> CancelRunResponse:
"""Request explicit cancellation for one Agent backend run."""
def stream_events(
self,
run_id: str,
*,
after: str | None = None,
should_stop: Callable[[], bool] | None = None,
) -> Iterator[RunEvent]:
def stream_events(self, run_id: str, *, after: str | None = None) -> Iterator[RunEvent]:
"""Yield public ``dify-agent`` run events in stream order."""
def wait_run(self, run_id: str, *, timeout_seconds: float | None = None) -> RunStatusResponse:
@@ -67,15 +61,7 @@ class _DifyAgentSyncClient(Protocol):
def cancel_run_sync(self, run_id: str, request: CancelRunRequest | None = None) -> CancelRunResponse:
"""Cancel one run synchronously."""
def stream_events_sync(
self,
run_id: str,
*,
after: str | None = None,
max_reconnects: int | None = None,
timeout_seconds: float | None = None,
should_stop: Callable[[], bool] | None = None,
) -> Iterator[RunEvent]:
def stream_events_sync(self, run_id: str, *, after: str | None = None) -> Iterator[RunEvent]:
"""Stream run events synchronously."""
def wait_run_sync(self, run_id: str, *, timeout_seconds: float | None = None) -> RunStatusResponse:
@@ -87,16 +73,8 @@ class DifyAgentBackendRunClient:
client: _DifyAgentSyncClient
def __init__(
self,
client: _DifyAgentSyncClient,
*,
stream_max_reconnects: int = 3,
stream_timeout_seconds: float = 1200,
) -> None:
def __init__(self, client: _DifyAgentSyncClient) -> None:
self.client = client
self._stream_max_reconnects = stream_max_reconnects
self._stream_timeout_seconds = stream_timeout_seconds
def create_run(self, request: CreateRunRequest) -> CreateRunResponse:
"""Create one run through ``POST /runs`` and normalize client exceptions."""
@@ -112,22 +90,10 @@ class DifyAgentBackendRunClient:
except Exception as exc:
raise _normalize_dify_agent_error(exc) from exc
def stream_events(
self,
run_id: str,
*,
after: str | None = None,
should_stop: Callable[[], bool] | None = None,
) -> Iterator[RunEvent]:
def stream_events(self, run_id: str, *, after: str | None = None) -> Iterator[RunEvent]:
"""Stream run events from ``/events/sse`` with the wrapped client's reconnect policy."""
try:
yield from self.client.stream_events_sync(
run_id,
after=after,
max_reconnects=self._stream_max_reconnects,
timeout_seconds=self._stream_timeout_seconds,
should_stop=should_stop,
)
yield from self.client.stream_events_sync(run_id, after=after)
except Exception as exc:
raise _normalize_dify_agent_error(exc) from exc
+2 -15
View File
@@ -54,21 +54,8 @@ class AgentBackendRunFailedError(AgentBackendError):
run_id: str
detail: Any
reason: str | None
source_event_id: str | None
def __init__(
self,
run_id: str,
detail: Any,
*,
message: str | None = None,
reason: str | None = None,
source_event_id: str | None = None,
) -> None:
def __init__(self, run_id: str, detail: Any) -> None:
self.run_id = run_id
self.detail = detail
self.reason = reason
self.source_event_id = source_event_id
display_message = message or f"Agent backend run failed: {run_id}"
super().__init__(f"{display_message} (agent_run_id={run_id})")
super().__init__(f"Agent backend run failed: {run_id}")
@@ -5,9 +5,6 @@ The adapter does not define a new cross-service event contract. It consumes
workflow Agent Node maps to Graphon/AppQueue events. Deferred external tool calls
remain Dify Agent ``run_succeeded`` payloads on the wire; API code turns them
into an internal event so workflow pause/session handling stays local to API.
Agent-message deltas are exposed as annotations on ``PydanticAIStreamRunEvent``
so API code does not have to parse Pydantic AI stream-event internals to
preserve streaming. The terminal answer remains the ``run_succeeded`` output.
"""
from __future__ import annotations
@@ -35,7 +32,6 @@ class AgentBackendInternalEventType(StrEnum):
RUN_STARTED = "run_started"
STREAM_EVENT = "stream_event"
AGENT_MESSAGE_DELTA = "agent_message_delta"
DEFERRED_TOOL_CALL = "deferred_tool_call"
RUN_SUCCEEDED = "run_succeeded"
RUN_FAILED = "run_failed"
@@ -65,20 +61,12 @@ class AgentBackendStreamInternalEvent(AgentBackendInternalEventBase):
data: JsonValue
class AgentBackendAgentMessageDeltaInternalEvent(AgentBackendInternalEventBase):
"""API-internal agent-message delta emitted independently from raw stream events."""
type: Literal[AgentBackendInternalEventType.AGENT_MESSAGE_DELTA] = AgentBackendInternalEventType.AGENT_MESSAGE_DELTA
delta: str
class AgentBackendRunSucceededInternalEvent(AgentBackendInternalEventBase):
"""API-internal terminal success event carrying final output and session state."""
type: Literal[AgentBackendInternalEventType.RUN_SUCCEEDED] = AgentBackendInternalEventType.RUN_SUCCEEDED
output: JsonValue
session_snapshot: CompositorSessionSnapshot
usage: dict[str, JsonValue] | None = None
class AgentBackendDeferredToolCallInternalEvent(AgentBackendInternalEventBase):
@@ -88,7 +76,6 @@ class AgentBackendDeferredToolCallInternalEvent(AgentBackendInternalEventBase):
deferred_tool_call: DeferredToolCallPayload
message: str | None = None
session_snapshot: CompositorSessionSnapshot
usage: dict[str, JsonValue] | None = None
class AgentBackendRunFailedInternalEvent(AgentBackendInternalEventBase):
@@ -110,7 +97,6 @@ class AgentBackendRunCancelledInternalEvent(AgentBackendInternalEventBase):
type AgentBackendInternalEvent = Annotated[
AgentBackendRunStartedInternalEvent
| AgentBackendStreamInternalEvent
| AgentBackendAgentMessageDeltaInternalEvent
| AgentBackendDeferredToolCallInternalEvent
| AgentBackendRunSucceededInternalEvent
| AgentBackendRunFailedInternalEvent
@@ -133,14 +119,6 @@ class AgentBackendRunEventAdapter:
)
]
case PydanticAIStreamRunEvent():
if event.agent_message_delta:
return [
AgentBackendAgentMessageDeltaInternalEvent(
run_id=event.run_id,
source_event_id=event.id,
delta=event.agent_message_delta,
)
]
data = cast(JsonValue, _EVENT_DATA_ADAPTER.dump_python(event.data, mode="json"))
event_kind = data.get("event_kind") if isinstance(data, dict) else None
return [
@@ -162,7 +140,6 @@ class AgentBackendRunEventAdapter:
deferred_tool_call=event.data.deferred_tool_call,
message=_deferred_tool_call_message(event.data.deferred_tool_call),
session_snapshot=event.data.session_snapshot,
usage=_agent_run_usage(event.data.usage),
)
]
return [
@@ -171,7 +148,6 @@ class AgentBackendRunEventAdapter:
source_event_id=event.id,
output=event.data.output,
session_snapshot=event.data.session_snapshot,
usage=_agent_run_usage(event.data.usage),
)
]
case RunFailedEvent():
@@ -208,13 +184,3 @@ def _deferred_tool_call_message(payload: DeferredToolCallPayload) -> str:
return title
return f"Agent backend requested external input via deferred tool '{payload.tool_name}'."
def _agent_run_usage(usage: object | None) -> dict[str, JsonValue] | None:
"""Return JSON-safe usage metadata from optional Agent backend usage."""
if usage is None:
return None
dumped = _EVENT_DATA_ADAPTER.dump_python(usage, mode="json")
if not isinstance(dumped, dict):
return None
return cast(dict[str, JsonValue], dumped)
+1 -8
View File
@@ -13,17 +13,10 @@ def create_agent_backend_run_client(
base_url: str | None = None,
use_fake: bool = False,
fake_scenario: str | FakeAgentBackendScenario = FakeAgentBackendScenario.SUCCESS,
stream_read_timeout_seconds: float = 30,
stream_max_reconnects: int = 3,
stream_run_timeout_seconds: float = 1200,
) -> AgentBackendRunClient:
"""Create the API-side run client without hiding the ``dify-agent`` protocol."""
if use_fake:
return FakeAgentBackendRunClient(scenario=FakeAgentBackendScenario(fake_scenario))
if base_url is None:
raise ValueError("base_url is required when creating a real Agent backend client")
return DifyAgentBackendRunClient(
Client(base_url=base_url, stream_timeout=stream_read_timeout_seconds),
stream_max_reconnects=stream_max_reconnects,
stream_timeout_seconds=stream_run_timeout_seconds,
)
return DifyAgentBackendRunClient(Client(base_url=base_url))
+2 -10
View File
@@ -7,7 +7,7 @@ separate ``agent-backend.v1`` event stream.
from __future__ import annotations
from collections.abc import Callable, Iterator
from collections.abc import Iterator
from datetime import UTC, datetime
from enum import StrEnum
@@ -69,17 +69,9 @@ class FakeAgentBackendRunClient:
del request
return CancelRunResponse(run_id=run_id, status="cancelled")
def stream_events(
self,
run_id: str,
*,
after: str | None = None,
should_stop: Callable[[], bool] | None = None,
) -> Iterator[RunEvent]:
def stream_events(self, run_id: str, *, after: str | None = None) -> Iterator[RunEvent]:
"""Yield the deterministic public ``RunEvent`` sequence for ``run_id``."""
for event in self._events(run_id):
if should_stop is not None and should_stop():
return
if after is not None and event.id is not None and event.id <= after:
continue
yield event
+56 -207
View File
@@ -11,9 +11,8 @@ composition-driven.
from __future__ import annotations
import re
from collections.abc import Mapping
from typing import ClassVar, Literal
from typing import ClassVar
from agenton.compositor import CompositorSessionSnapshot
from agenton.compositor.schemas import LayerSessionSnapshot
@@ -21,8 +20,6 @@ from agenton.layers import ExitIntent
from agenton_collections.layers.plain import PLAIN_PROMPT_LAYER_TYPE_ID, PromptLayerConfig
from agenton_collections.layers.pydantic_ai import PYDANTIC_AI_HISTORY_LAYER_TYPE_ID
from dify_agent.layers.ask_human import DIFY_ASK_HUMAN_LAYER_TYPE_ID, DifyAskHumanLayerConfig
from dify_agent.layers.config import DIFY_CONFIG_LAYER_TYPE_ID, DifyConfigLayerConfig
from dify_agent.layers.dify_core_tools import DIFY_CORE_TOOLS_LAYER_TYPE_ID, DifyCoreToolsLayerConfig
from dify_agent.layers.dify_plugin import (
DIFY_PLUGIN_LLM_LAYER_TYPE_ID,
DIFY_PLUGIN_TOOLS_LAYER_TYPE_ID,
@@ -47,6 +44,7 @@ from dify_agent.protocol import (
LayerExitSignals,
RunComposition,
RunLayerSpec,
RunPurpose,
RuntimeLayerSpec,
)
from pydantic import BaseModel, ConfigDict, Field, JsonValue, field_validator
@@ -56,14 +54,11 @@ WORKFLOW_NODE_JOB_PROMPT_LAYER_ID = "workflow_node_job_prompt"
WORKFLOW_USER_PROMPT_LAYER_ID = "workflow_user_prompt"
AGENT_APP_USER_PROMPT_LAYER_ID = "agent_app_user_prompt"
DIFY_EXECUTION_CONTEXT_LAYER_ID = "execution_context"
DIFY_CONFIG_LAYER_ID = "config"
DIFY_DRIVE_LAYER_ID = "drive"
DIFY_PLUGIN_TOOLS_LAYER_ID = "tools"
DIFY_CORE_TOOLS_LAYER_ID = "core_tools"
DIFY_KNOWLEDGE_BASE_LAYER_ID = "knowledge"
DIFY_ASK_HUMAN_LAYER_ID = "ask_human"
DIFY_SHELL_LAYER_ID = "shell"
type AgentConfigVersionKind = Literal["snapshot", "draft", "build_draft"]
def _filter_snapshot_to_specs(
@@ -83,81 +78,6 @@ def _filter_snapshot_to_specs(
return CompositorSessionSnapshot(schema_version=snapshot.schema_version, layers=filtered_layers)
def _shell_layer_deps() -> dict[str, str]:
return {"execution_context": DIFY_EXECUTION_CONTEXT_LAYER_ID}
def _drive_layer_deps() -> dict[str, str]:
return {"shell": DIFY_SHELL_LAYER_ID}
def _config_layer_deps() -> dict[str, str]:
return {"shell": DIFY_SHELL_LAYER_ID}
def _shell_config_with_drive_ref(
shell_config: DifyShellLayerConfig | None,
drive_config: DifyDriveLayerConfig | None,
) -> DifyShellLayerConfig:
config = shell_config or DifyShellLayerConfig()
if drive_config is None:
return config
return config.model_copy(update={"agent_stub_drive_ref": drive_config.drive_ref})
def _markdown_backtick_fence(text: str) -> str:
"""Choose a fence that will not terminate inside the prompt body."""
longest_backtick_run = max((len(match.group(0)) for match in re.finditer(r"`+", text)), default=0)
return "`" * max(3, longest_backtick_run + 1)
_BUILD_DRAFT_AGENT_SOUL_PROMPT = """You are running in build mode.
Objective:
- Improve this agent's working environment, configuration, tools, files, notes,
and context so it can handle the intended task well.
Guidance:
- Treat the intended task as context for setup work, validation, and configuration decisions.
- Perform concrete investigative or setup steps when they help improve or verify the agent configuration.
- Use the installed `dify-agent` CLI when you need to inspect or persist Agent configuration."""
def _wrap_build_draft_agent_soul_prompt(prompt: str | None) -> str:
"""Reframe build-draft Agent Soul prompts as preparation work for a future run."""
prompt_body = (prompt or "").strip()
if not prompt_body:
return _BUILD_DRAFT_AGENT_SOUL_PROMPT + "\n\nIntended task for later normal runs:\nNo task prompt was provided."
fence = _markdown_backtick_fence(prompt_body)
return (
_BUILD_DRAFT_AGENT_SOUL_PROMPT
+ f"\n\nIntended task for later normal runs:\n{fence}text\n{prompt_body}\n{fence}"
)
def _agent_soul_prompt_for_layer(
prompt: str | None,
*,
config_version_kind: AgentConfigVersionKind,
) -> str | None:
"""Preserve normal snapshot/draft prompts and only wrap build-draft prompts.
The API-side layer adapter is the product boundary where Agent Soul text
becomes the model-facing system-prompt layer. ``snapshot`` and normal
``draft`` runs pass through the original effective prompt unchanged, while
``build_draft`` always emits a setup prompt. When an original prompt is
present, it is reframed as future-run context and embedded in a fenced
block; when it is blank, the setup instruction is still kept.
"""
if config_version_kind != "build_draft":
if prompt is None:
return None
if not prompt.strip():
return None
return prompt
return _wrap_build_draft_agent_soul_prompt(prompt)
class AgentBackendModelConfig(BaseModel):
"""API-side model/plugin selection before it is converted to Dify Agent layers."""
@@ -217,15 +137,13 @@ class AgentBackendWorkflowNodeRunInput(BaseModel):
workflow_node_job_prompt: str
user_prompt: str
agent_soul_prompt: str | None = None
agent_config_version_kind: AgentConfigVersionKind = "snapshot"
purpose: RunPurpose = "workflow_node"
idempotency_key: str | None = None
output: AgentBackendOutputConfig | None = None
tools: DifyPluginToolsLayerConfig | None = None
core_tools: DifyCoreToolsLayerConfig | None = None
knowledge: DifyKnowledgeBaseLayerConfig | None = None
config_layer_config: DifyConfigLayerConfig | None = None
# Drive Skills & Files declaration (dify.drive) — an index the agent pulls
# through the back proxy, never inline content.
# through the back proxy, never inline content; see AGENT_DRIVE_MANIFEST_ENABLED.
drive_config: DifyDriveLayerConfig | None = None
# Human-in-the-loop ask_human deferred tool (dify.ask_human). Present only when
# the Agent Soul configures human involvement; a deferred call ends the run and
@@ -266,15 +184,13 @@ class AgentBackendAgentAppRunInput(BaseModel):
execution_context: DifyExecutionContextLayerConfig
user_prompt: str
agent_soul_prompt: str | None = None
agent_config_version_kind: AgentConfigVersionKind = "snapshot"
purpose: RunPurpose = "agent_app"
idempotency_key: str | None = None
output: AgentBackendOutputConfig | None = None
tools: DifyPluginToolsLayerConfig | None = None
core_tools: DifyCoreToolsLayerConfig | None = None
knowledge: DifyKnowledgeBaseLayerConfig | None = None
config_layer_config: DifyConfigLayerConfig | None = None
# Drive Skills & Files declaration (dify.drive) — an index the agent pulls
# through the back proxy, never inline content.
# through the back proxy, never inline content; see AGENT_DRIVE_MANIFEST_ENABLED.
drive_config: DifyDriveLayerConfig | None = None
# Human-in-the-loop ask_human deferred tool (dify.ask_human). Present only when
# the Agent Soul configures human involvement (ENG-635).
@@ -308,24 +224,18 @@ class AgentBackendRunRequestBuilder:
"""Build an Agent App conversation-turn run request.
Layer graph: optional Agent Soul system prompt → user prompt →
execution context → optional shell / config / drive / history
(multi-turn) → LLM → optional plugin-direct tools / core-routed tools /
knowledge search / ask_human / structured output. Mirrors the
workflow-node layer ordering minus the workflow-job / previous-node
prompt.
execution context → optional history (multi-turn) → LLM → optional
plugin tools / knowledge search → optional structured output. Mirrors the workflow-node
layer ordering minus the workflow-job / previous-node prompt.
"""
layers: list[RunLayerSpec] = []
agent_soul_prompt = _agent_soul_prompt_for_layer(
run_input.agent_soul_prompt,
config_version_kind=run_input.agent_config_version_kind,
)
if agent_soul_prompt:
if run_input.agent_soul_prompt:
layers.append(
RunLayerSpec(
name=AGENT_SOUL_PROMPT_LAYER_ID,
type=PLAIN_PROMPT_LAYER_TYPE_ID,
metadata={**run_input.metadata, "origin": "agent_soul"},
config=PromptLayerConfig(prefix=agent_soul_prompt),
config=PromptLayerConfig(prefix=run_input.agent_soul_prompt),
)
)
@@ -346,42 +256,13 @@ class AgentBackendRunRequestBuilder:
]
)
include_shell = (
run_input.include_shell or run_input.config_layer_config is not None or run_input.drive_config is not None
)
if include_shell:
# Sandboxed bash workspace (dify.shell). It enters before config/drive
# so eager pulls materialize content in the same filesystem used by
# model commands.
layers.append(
RunLayerSpec(
name=DIFY_SHELL_LAYER_ID,
type=DIFY_SHELL_LAYER_TYPE_ID,
deps=_shell_layer_deps(),
metadata=run_input.metadata,
config=_shell_config_with_drive_ref(run_input.shell_config, run_input.drive_config),
)
)
if run_input.config_layer_config is not None:
layers.append(
RunLayerSpec(
name=DIFY_CONFIG_LAYER_ID,
type=DIFY_CONFIG_LAYER_TYPE_ID,
deps=_config_layer_deps(),
metadata=run_input.metadata,
config=run_input.config_layer_config,
)
)
if run_input.drive_config is not None:
# Drive Skills & Files declaration (dify.drive): the catalog plus
# prompt-mentioned entries eagerly pulled through the shell layer.
# Drive Skills & Files declaration (dify.drive): a config-only index;
# the agent pulls listed entries through the back proxy by drive_ref.
layers.append(
RunLayerSpec(
name=DIFY_DRIVE_LAYER_ID,
type=DIFY_DRIVE_LAYER_TYPE_ID,
deps=_drive_layer_deps(),
metadata=run_input.metadata,
config=run_input.drive_config,
)
@@ -413,31 +294,17 @@ class AgentBackendRunRequestBuilder:
)
if run_input.tools is not None and run_input.tools.tools:
plugin_tool_deps = {"execution_context": DIFY_EXECUTION_CONTEXT_LAYER_ID}
if include_shell:
plugin_tool_deps["shell"] = DIFY_SHELL_LAYER_ID
layers.append(
RunLayerSpec(
name=DIFY_PLUGIN_TOOLS_LAYER_ID,
type=DIFY_PLUGIN_TOOLS_LAYER_TYPE_ID,
deps=plugin_tool_deps,
deps={"execution_context": DIFY_EXECUTION_CONTEXT_LAYER_ID},
metadata=run_input.metadata,
config=run_input.tools,
)
)
if run_input.core_tools is not None and run_input.core_tools.tools:
layers.append(
RunLayerSpec(
name=DIFY_CORE_TOOLS_LAYER_ID,
type=DIFY_CORE_TOOLS_LAYER_TYPE_ID,
deps={"execution_context": DIFY_EXECUTION_CONTEXT_LAYER_ID},
metadata=run_input.metadata,
config=run_input.core_tools,
)
)
if run_input.knowledge is not None and run_input.knowledge.sets:
if run_input.knowledge is not None and run_input.knowledge.dataset_ids:
layers.append(
RunLayerSpec(
name=DIFY_KNOWLEDGE_BASE_LAYER_ID,
@@ -461,6 +328,20 @@ class AgentBackendRunRequestBuilder:
)
)
if run_input.include_shell:
# Sandboxed bash workspace (dify.shell). Depends on execution_context so
# the agent server can mint per-command Agent Stub env (back proxy);
# shellctl connection itself is server-injected.
layers.append(
RunLayerSpec(
name=DIFY_SHELL_LAYER_ID,
type=DIFY_SHELL_LAYER_TYPE_ID,
deps={"execution_context": DIFY_EXECUTION_CONTEXT_LAYER_ID},
metadata=run_input.metadata,
config=run_input.shell_config or DifyShellLayerConfig(),
)
)
if run_input.output is not None:
layers.append(
RunLayerSpec(
@@ -477,6 +358,7 @@ class AgentBackendRunRequestBuilder:
return CreateRunRequest(
composition=RunComposition(layers=layers),
purpose=run_input.purpose,
idempotency_key=run_input.idempotency_key,
metadata=run_input.metadata,
session_snapshot=run_input.session_snapshot,
@@ -501,8 +383,7 @@ class AgentBackendRunRequestBuilder:
non-plugin layer graph that produced the snapshot. Plugin layers
(``dify.plugin.llm``, ``dify.plugin.tools``) are excluded from both the
composition and the snapshot before submission because their configs
may carry credentials or runtime-only declarations that are not
persisted between runs.
require credentials that are not persisted between runs.
"""
if not runtime_layer_specs:
raise ValueError(
@@ -524,6 +405,7 @@ class AgentBackendRunRequestBuilder:
filtered_snapshot = _filter_snapshot_to_specs(session_snapshot, runtime_layer_specs)
return CreateRunRequest(
composition=RunComposition(layers=layers),
purpose="workflow_node",
idempotency_key=idempotency_key,
metadata=request_metadata,
session_snapshot=filtered_snapshot,
@@ -534,22 +416,17 @@ class AgentBackendRunRequestBuilder:
"""Build a workflow Agent Node run request without defining another wire schema.
Layer graph mirrors the workflow surface: prompts → execution context →
optional shell / config / drive / history → LLM → optional
plugin-direct tools / core-routed tools / knowledge search /
ask_human / structured output.
optional drive/history → LLM → optional plugin tools / knowledge search
→ optional auxiliary layers such as ask_human, shell, and structured output.
"""
layers: list[RunLayerSpec] = []
agent_soul_prompt = _agent_soul_prompt_for_layer(
run_input.agent_soul_prompt,
config_version_kind=run_input.agent_config_version_kind,
)
if agent_soul_prompt:
if run_input.agent_soul_prompt:
layers.append(
RunLayerSpec(
name=AGENT_SOUL_PROMPT_LAYER_ID,
type=PLAIN_PROMPT_LAYER_TYPE_ID,
metadata={**run_input.metadata, "origin": "agent_soul"},
config=PromptLayerConfig(prefix=agent_soul_prompt),
config=PromptLayerConfig(prefix=run_input.agent_soul_prompt),
)
)
@@ -559,7 +436,7 @@ class AgentBackendRunRequestBuilder:
name=WORKFLOW_NODE_JOB_PROMPT_LAYER_ID,
type=PLAIN_PROMPT_LAYER_TYPE_ID,
metadata={**run_input.metadata, "origin": "workflow_node_job"},
config=PromptLayerConfig(user=run_input.workflow_node_job_prompt),
config=PromptLayerConfig(prefix=run_input.workflow_node_job_prompt),
),
RunLayerSpec(
name=WORKFLOW_USER_PROMPT_LAYER_ID,
@@ -576,42 +453,13 @@ class AgentBackendRunRequestBuilder:
]
)
include_shell = (
run_input.include_shell or run_input.config_layer_config is not None or run_input.drive_config is not None
)
if include_shell:
# Sandboxed bash workspace (dify.shell). It enters before drive so
# drive can materialize mentioned targets with `dify-agent drive pull`
# in the same shell-visible filesystem used by model commands.
layers.append(
RunLayerSpec(
name=DIFY_SHELL_LAYER_ID,
type=DIFY_SHELL_LAYER_TYPE_ID,
deps=_shell_layer_deps(),
metadata=run_input.metadata,
config=_shell_config_with_drive_ref(run_input.shell_config, run_input.drive_config),
)
)
if run_input.config_layer_config is not None:
layers.append(
RunLayerSpec(
name=DIFY_CONFIG_LAYER_ID,
type=DIFY_CONFIG_LAYER_TYPE_ID,
deps=_config_layer_deps(),
metadata=run_input.metadata,
config=run_input.config_layer_config,
)
)
if run_input.drive_config is not None:
# Drive Skills & Files declaration (dify.drive): the catalog plus
# prompt-mentioned entries eagerly pulled through the shell layer.
# Drive Skills & Files declaration (dify.drive): a config-only index;
# the agent pulls listed entries through the back proxy by drive_ref.
layers.append(
RunLayerSpec(
name=DIFY_DRIVE_LAYER_ID,
type=DIFY_DRIVE_LAYER_TYPE_ID,
deps=_drive_layer_deps(),
metadata=run_input.metadata,
config=run_input.drive_config,
)
@@ -645,31 +493,17 @@ class AgentBackendRunRequestBuilder:
)
if run_input.tools is not None and run_input.tools.tools:
plugin_tool_deps = {"execution_context": DIFY_EXECUTION_CONTEXT_LAYER_ID}
if include_shell:
plugin_tool_deps["shell"] = DIFY_SHELL_LAYER_ID
layers.append(
RunLayerSpec(
name=DIFY_PLUGIN_TOOLS_LAYER_ID,
type=DIFY_PLUGIN_TOOLS_LAYER_TYPE_ID,
deps=plugin_tool_deps,
deps={"execution_context": DIFY_EXECUTION_CONTEXT_LAYER_ID},
metadata=run_input.metadata,
config=run_input.tools,
)
)
if run_input.core_tools is not None and run_input.core_tools.tools:
layers.append(
RunLayerSpec(
name=DIFY_CORE_TOOLS_LAYER_ID,
type=DIFY_CORE_TOOLS_LAYER_TYPE_ID,
deps={"execution_context": DIFY_EXECUTION_CONTEXT_LAYER_ID},
metadata=run_input.metadata,
config=run_input.core_tools,
)
)
if run_input.knowledge is not None and run_input.knowledge.sets:
if run_input.knowledge is not None and run_input.knowledge.dataset_ids:
layers.append(
RunLayerSpec(
name=DIFY_KNOWLEDGE_BASE_LAYER_ID,
@@ -693,6 +527,20 @@ class AgentBackendRunRequestBuilder:
)
)
if run_input.include_shell:
# Sandboxed bash workspace (dify.shell). Depends on execution_context so
# the agent server can mint per-command Agent Stub env (back proxy);
# shellctl connection itself is server-injected.
layers.append(
RunLayerSpec(
name=DIFY_SHELL_LAYER_ID,
type=DIFY_SHELL_LAYER_TYPE_ID,
deps={"execution_context": DIFY_EXECUTION_CONTEXT_LAYER_ID},
metadata=run_input.metadata,
config=run_input.shell_config or DifyShellLayerConfig(),
)
)
if run_input.output is not None:
layers.append(
RunLayerSpec(
@@ -709,6 +557,7 @@ class AgentBackendRunRequestBuilder:
return CreateRunRequest(
composition=RunComposition(layers=layers),
purpose=run_input.purpose,
idempotency_key=run_input.idempotency_key,
metadata=run_input.metadata,
session_snapshot=run_input.session_snapshot,
@@ -1,100 +0,0 @@
"""Shared API-side helper for Agent backend lifecycle-only session cleanup.
Product code owns local row retirement and background-task dispatch. This module
only adapts persisted cleanup inputs into the public ``dify-agent`` run
protocol, performs the synchronous ``create_run + wait_run`` loop used by Celery
workers, and reports whether the backend cleanup succeeded, was skipped, or
failed.
"""
from __future__ import annotations
from dataclasses import dataclass
from typing import ClassVar, Literal
from agenton.compositor import CompositorSessionSnapshot
from dify_agent.protocol import RuntimeLayerSpec
from pydantic import BaseModel, ConfigDict, Field, JsonValue
from clients.agent_backend.client import AgentBackendRunClient
from clients.agent_backend.errors import AgentBackendError
from clients.agent_backend.request_builder import AgentBackendRunRequestBuilder
class AgentBackendSessionCleanupPayload(BaseModel):
"""Serialized cleanup inputs preserved across API and Celery boundaries."""
session_snapshot: CompositorSessionSnapshot | None = None
runtime_layer_specs: list[RuntimeLayerSpec] = Field(default_factory=list)
idempotency_key: str | None = None
metadata: dict[str, JsonValue] = Field(default_factory=dict)
timeout_seconds: float = 30.0
model_config: ClassVar[ConfigDict] = ConfigDict(extra="forbid")
@dataclass(frozen=True, slots=True)
class AgentBackendSessionCleanupResult:
"""Terminal outcome of one backend cleanup attempt."""
status: Literal["succeeded", "skipped", "failed"]
reason: str | None = None
cleanup_run_id: str | None = None
@classmethod
def succeeded(cls, cleanup_run_id: str) -> AgentBackendSessionCleanupResult:
return cls(status="succeeded", cleanup_run_id=cleanup_run_id)
@classmethod
def skipped(cls, reason: str) -> AgentBackendSessionCleanupResult:
return cls(status="skipped", reason=reason)
@classmethod
def failed(cls, reason: str, cleanup_run_id: str | None = None) -> AgentBackendSessionCleanupResult:
return cls(status="failed", reason=reason, cleanup_run_id=cleanup_run_id)
def cleanup_agent_backend_session(
*,
payload: AgentBackendSessionCleanupPayload,
client: AgentBackendRunClient | None,
request_builder: AgentBackendRunRequestBuilder | None = None,
) -> AgentBackendSessionCleanupResult:
"""Run lifecycle-only cleanup against the Agent backend and report status."""
if client is None:
return AgentBackendSessionCleanupResult.skipped("no_agent_backend_client")
if payload.session_snapshot is None:
return AgentBackendSessionCleanupResult.skipped("missing_session_snapshot")
if not payload.runtime_layer_specs:
return AgentBackendSessionCleanupResult.skipped("missing_runtime_layer_specs")
builder = request_builder or AgentBackendRunRequestBuilder()
request = builder.build_cleanup_request(
session_snapshot=payload.session_snapshot,
runtime_layer_specs=payload.runtime_layer_specs,
idempotency_key=payload.idempotency_key,
metadata=payload.metadata,
)
try:
response = client.create_run(request)
except AgentBackendError as exc:
return AgentBackendSessionCleanupResult.failed(str(exc))
try:
status_response = client.wait_run(response.run_id, timeout_seconds=payload.timeout_seconds)
except AgentBackendError as exc:
return AgentBackendSessionCleanupResult.failed(str(exc), cleanup_run_id=response.run_id)
if status_response.status != "succeeded":
reason = status_response.error or f"cleanup run ended with status {status_response.status}"
return AgentBackendSessionCleanupResult.failed(reason, cleanup_run_id=response.run_id)
return AgentBackendSessionCleanupResult.succeeded(response.run_id)
__all__ = [
"AgentBackendSessionCleanupPayload",
"AgentBackendSessionCleanupResult",
"cleanup_agent_backend_session",
]
+1 -6
View File
@@ -22,11 +22,9 @@ from .plugin import (
setup_system_trigger_oauth_client,
transform_datasource_credentials,
)
from .rbac import migrate_dataset_permissions_to_rbac, migrate_member_roles_to_rbac
from .rbac import migrate_member_roles_to_rbac
from .retention import (
archive_workflow_runs,
archive_workflow_runs_plan,
backfill_workflow_run_archive_bundles,
clean_expired_messages,
clean_workflow_runs,
cleanup_orphaned_draft_variables,
@@ -53,9 +51,7 @@ from .vector import (
__all__ = [
"add_qdrant_index",
"archive_workflow_runs",
"archive_workflow_runs_plan",
"backfill_plugin_auto_upgrade",
"backfill_workflow_run_archive_bundles",
"clean_expired_messages",
"clean_workflow_runs",
"cleanup_orphaned_draft_variables",
@@ -78,7 +74,6 @@ __all__ = [
"legacy_model_types",
"migrate_annotation_vector_database",
"migrate_data_for_plugin",
"migrate_dataset_permissions_to_rbac",
"migrate_knowledge_vector_database",
"migrate_member_roles_to_rbac",
"migrate_oss",
+3 -4
View File
@@ -25,7 +25,7 @@ def reset_password(email, new_password, password_confirm):
return
normalized_email = email.strip().lower()
account = AccountService.get_account_by_email_with_case_fallback(email.strip(), session=db.session())
account = AccountService.get_account_by_email_with_case_fallback(email.strip())
if not account:
click.echo(click.style(f"Account not found for email: {email}", fg="red"))
@@ -67,7 +67,7 @@ def reset_email(email, new_email, email_confirm):
return
normalized_new_email = new_email.strip().lower()
account = AccountService.get_account_by_email_with_case_fallback(email.strip(), session=db.session())
account = AccountService.get_account_by_email_with_case_fallback(email.strip())
if not account:
click.echo(click.style(f"Account not found for email: {email}", fg="red"))
@@ -133,9 +133,8 @@ def create_tenant(email: str, language: str | None = None, name: str | None = No
password=new_password,
language=language,
create_workspace_required=False,
session=db.session(),
)
TenantService.create_owner_tenant_if_not_exist(account, name, session=db.session())
TenantService.create_owner_tenant_if_not_exist(account, name)
click.echo(
click.style(
-2
View File
@@ -7,7 +7,6 @@ from typing import cast
import click
from commands.rbac import migrate_dataset_permissions_to_rbac
from extensions.ext_database import db
from graphon.model_runtime.entities.model_entities import ModelType
from services.legacy_model_type_migration import (
@@ -178,4 +177,3 @@ def legacy_model_types(
data_migrate.add_command(legacy_model_types)
data_migrate.add_command(migrate_dataset_permissions_to_rbac)
+16 -24
View File
@@ -9,9 +9,7 @@ from uuid import UUID
import click
import sqlalchemy as sa
import yaml
from sqlalchemy.orm import Session
from core.db.session_factory import session_factory
from extensions.ext_database import db
from models import Tenant
from models.model import App
@@ -108,8 +106,7 @@ def export_migration_data(input_file: str | None, output_file: str | None, overw
assert output_file is not None
raw_config = _load_json_object(input_file, "Export config")
selection = ExportConfigParser().parse(raw_config)
with session_factory.create_session() as session:
result = MigrationExportService().export(selection, session=session)
result = MigrationExportService().export(selection)
MigrationPackageService().save_package(result.package, output_file, overwrite=overwrite)
click.echo(click.style(f"Output written to {output_file}", fg="green"))
_render_report(result.report_items, context=_with_output_path(result.report_context, output_file))
@@ -156,21 +153,19 @@ def import_migration_data(
_require_options(("--input", input_file))
assert input_file is not None
package = MigrationPackageService().load_package(input_file)
with session_factory.create_session() as session:
result = MigrationImportService().import_package(
ImportRequest(
package=package,
cli_target_tenant=target_tenant,
operator_email=operator_email,
options_override=_build_options_override(
package.metadata.import_options,
id_strategy=id_strategy,
conflict_strategy=conflict_strategy,
create_app_api_token_on_import=create_app_api_token_on_import,
),
result = MigrationImportService().import_package(
ImportRequest(
package=package,
cli_target_tenant=target_tenant,
operator_email=operator_email,
options_override=_build_options_override(
package.metadata.import_options,
id_strategy=id_strategy,
conflict_strategy=conflict_strategy,
create_app_api_token_on_import=create_app_api_token_on_import,
),
session=session,
)
)
_render_report(result.report_items, context=result.report_context)
except MigrationDataError as exc:
raise click.ClickException(str(exc)) from exc
@@ -218,9 +213,7 @@ def migration_data_wizard() -> None:
default=True,
show_default=False,
)
auto_tools = _discover_auto_tools(
[app for app in apps if app.id in set(app_ids)], include_referenced_tools, session=db.session()
)
auto_tools = _discover_auto_tools([app for app in apps if app.id in set(app_ids)], include_referenced_tools)
auto_tools = _resolve_auto_tool_names(tenant.id, auto_tools)
_print_auto_tools(auto_tools)
additional_tools = _prompt_additional_tools(tenant.id, auto_tools)
@@ -255,8 +248,7 @@ def migration_data_wizard() -> None:
conflict_strategy=conflict_strategy,
output_file=output_file,
)
with session_factory.create_session() as session:
result = MigrationExportService().export(selection, session=session)
result = MigrationExportService().export(selection)
MigrationPackageService().save_package(result.package, output_file, overwrite=overwrite)
click.echo(click.style(f"Output written to {output_file}", fg="green"))
_print_wizard_step("Report")
@@ -397,13 +389,13 @@ def _prompt_import_options() -> tuple[bool, bool, str, str]:
return include_secrets, create_tokens, id_strategy, conflict_strategy
def _discover_auto_tools(apps: list[App], include_referenced_tools: bool, *, session: Session) -> WizardToolMap:
def _discover_auto_tools(apps: list[App], include_referenced_tools: bool) -> WizardToolMap:
auto_tools: WizardToolMap = {"api_tools": {}, "workflow_tools": {}, "mcp_tools": {}}
if not include_referenced_tools:
return auto_tools
discovery_service = DependencyDiscoveryService()
for app in apps:
dsl_content = AppDslService.export_dsl(app_model=app, session=session, include_secret=False)
dsl_content = AppDslService.export_dsl(app_model=app, include_secret=False)
raw_dsl = yaml.safe_load(dsl_content) if dsl_content else {}
dsl = raw_dsl if isinstance(raw_dsl, dict) else {}
for dependency in discovery_service.discover_from_dsl(dsl):
+20 -19
View File
@@ -9,7 +9,6 @@ from sqlalchemy import delete, func, select
from sqlalchemy.engine import CursorResult
from configs import dify_config
from core.db.session_factory import session_factory
from core.helper import encrypter
from core.plugin.entities.plugin_daemon import CredentialType
from core.plugin.impl.plugin import PluginInstaller
@@ -17,7 +16,7 @@ from core.plugin.plugin_service import PluginService
from core.tools.utils.system_encryption import encrypt_system_params
from extensions.ext_database import db
from models import Tenant
from models.account import TenantPluginAutoUpgradeCategory, TenantPluginAutoUpgradeStrategy
from models.account import TenantPluginAutoUpgradeStrategy
from models.oauth import DatasourceOauthParamConfig, DatasourceProvider
from models.provider_ids import DatasourceProviderID, ToolProviderID
from models.source import DataSourceApiKeyAuthBinding, DataSourceOauthBinding
@@ -189,13 +188,13 @@ def transform_datasource_credentials(environment: str):
firecrawl_plugin_id = "langgenius/firecrawl_datasource"
jina_plugin_id = "langgenius/jina_datasource"
if environment == "online":
notion_package_identifier = plugin_migration._fetch_latest_package_identifier(notion_plugin_id)
firecrawl_package_identifier = plugin_migration._fetch_latest_package_identifier(firecrawl_plugin_id)
jina_package_identifier = plugin_migration._fetch_latest_package_identifier(jina_plugin_id)
notion_plugin_unique_identifier = plugin_migration._fetch_plugin_unique_identifier(notion_plugin_id)
firecrawl_plugin_unique_identifier = plugin_migration._fetch_plugin_unique_identifier(firecrawl_plugin_id)
jina_plugin_unique_identifier = plugin_migration._fetch_plugin_unique_identifier(jina_plugin_id)
else:
notion_package_identifier = None
firecrawl_package_identifier = None
jina_package_identifier = None
notion_plugin_unique_identifier = None
firecrawl_plugin_unique_identifier = None
jina_plugin_unique_identifier = None
oauth_credential_type = CredentialType.OAUTH2
api_key_credential_type = CredentialType.API_KEY
@@ -220,9 +219,9 @@ def transform_datasource_credentials(environment: str):
installed_plugins = installer_manager.list_plugins(tenant_id)
installed_plugins_ids = [plugin.plugin_id for plugin in installed_plugins]
if notion_plugin_id not in installed_plugins_ids:
if notion_package_identifier:
if notion_plugin_unique_identifier:
# install notion plugin
PluginService.install_from_marketplace_pkg(tenant_id, [notion_package_identifier])
PluginService.install_from_marketplace_pkg(tenant_id, [notion_plugin_unique_identifier])
auth_count = 0
for notion_tenant_credential in notion_tenant_credentials:
auth_count += 1
@@ -280,9 +279,9 @@ def transform_datasource_credentials(environment: str):
installed_plugins = installer_manager.list_plugins(tenant_id)
installed_plugins_ids = [plugin.plugin_id for plugin in installed_plugins]
if firecrawl_plugin_id not in installed_plugins_ids:
if firecrawl_package_identifier:
if firecrawl_plugin_unique_identifier:
# install firecrawl plugin
PluginService.install_from_marketplace_pkg(tenant_id, [firecrawl_package_identifier])
PluginService.install_from_marketplace_pkg(tenant_id, [firecrawl_plugin_unique_identifier])
auth_count = 0
for firecrawl_tenant_credential in firecrawl_tenant_credentials:
@@ -344,10 +343,10 @@ def transform_datasource_credentials(environment: str):
installed_plugins = installer_manager.list_plugins(tenant_id)
installed_plugins_ids = [plugin.plugin_id for plugin in installed_plugins]
if jina_plugin_id not in installed_plugins_ids:
if jina_package_identifier:
if jina_plugin_unique_identifier:
# install jina plugin
logger.debug("Installing Jina plugin %s", jina_package_identifier)
PluginService.install_from_marketplace_pkg(tenant_id, [jina_package_identifier])
logger.debug("Installing Jina plugin %s", jina_plugin_unique_identifier)
PluginService.install_from_marketplace_pkg(tenant_id, [jina_plugin_unique_identifier])
auth_count = 0
for jina_tenant_credential in jina_tenant_credentials:
@@ -407,7 +406,7 @@ def migrate_data_for_plugin():
def _candidate_auto_upgrade_strategy_tenant_ids_stmt(limit: int | None = None):
category_count = len(TenantPluginAutoUpgradeCategory)
category_count = len(TenantPluginAutoUpgradeStrategy.PluginCategory)
stmt = (
select(TenantPluginAutoUpgradeStrategy.tenant_id)
.group_by(TenantPluginAutoUpgradeStrategy.tenant_id)
@@ -473,7 +472,6 @@ def backfill_plugin_auto_upgrade(
try:
result = PluginAutoUpgradeService.backfill_strategy_categories(
current_tenant_id,
session=db.session(),
)
except Exception as e:
failed_count += 1
@@ -579,6 +577,9 @@ def install_rag_pipeline_plugins(input_file, output_file, workers):
"""
click.echo(click.style("Installing rag pipeline plugins", fg="yellow"))
plugin_migration = PluginMigration()
with session_factory.create_session() as session:
plugin_migration.install_rag_pipeline_plugins(input_file, output_file, workers, session=session)
plugin_migration.install_rag_pipeline_plugins(
input_file,
output_file,
workers,
)
click.echo(click.style("Installing rag pipeline plugins successfully", fg="green"))
+64 -463
View File
@@ -1,56 +1,11 @@
from __future__ import annotations
import json
from collections.abc import Iterator
from concurrent.futures import ThreadPoolExecutor, as_completed
import click
from sqlalchemy import select
from sqlalchemy.orm import Session
from configs import dify_config
from core.db.session_factory import session_factory
from core.rbac import RBACResourceWhitelistScope
from models import Dataset, DatasetPermission, DatasetPermissionEnum, TenantAccountJoin, TenantAccountRole
from services.enterprise.rbac_service import ListOption, RBACService, ReplaceMemberBindings, ReplaceUserAccessPolicies
_RBAC_DEFAULT_ACCESS_POLICY_ID = "default"
_LEGACY_ROLE_TO_BUILTIN_TAG = {
TenantAccountRole.OWNER.value: "owner",
TenantAccountRole.ADMIN.value: "admin",
TenantAccountRole.EDITOR.value: "editor",
TenantAccountRole.NORMAL.value: "normal",
TenantAccountRole.DATASET_OPERATOR.value: "dataset_operator",
}
def _resolve_builtin_role_ids(tenant_id: str, operator_account_id: str) -> dict[str, str]:
"""Resolve every legacy workspace role to the current tenant's builtin RBAC role id.
The migration replays the old `TenantAccountJoin.role` values onto the
RBAC member-role binding API. Builtin RBAC roles are tenant-scoped and
identified by runtime ids, so the command must look them up per tenant.
"""
roles = RBACService.Roles.list(
tenant_id=tenant_id,
account_id=operator_account_id,
options=ListOption(page_number=1, results_per_page=100),
).data
role_id_by_tag = {
role.role_tag: role.id
for role in roles
if role.is_builtin and role.category == "global_system_default" and role.role_tag
}
resolved: dict[str, str] = {}
for legacy_role, expected_builtin_tag in _LEGACY_ROLE_TO_BUILTIN_TAG.items():
role_id = role_id_by_tag.get(expected_builtin_tag)
if expected_builtin_tag == "dataset_operator" and not dify_config.DATASET_OPERATOR_ENABLED:
continue
if not role_id:
raise ValueError(f"Builtin RBAC role not found for tenant={tenant_id}, legacy_role={legacy_role}")
resolved[legacy_role] = role_id
return resolved
from models import TenantAccountJoin, TenantAccountRole
from services.enterprise.rbac_service import ListOption, RBACService
def _resolve_builtin_role_id(tenant_id: str, operator_account_id: str, legacy_role: str) -> str:
@@ -60,105 +15,26 @@ def _resolve_builtin_role_id(tenant_id: str, operator_account_id: str, legacy_ro
RBAC member-role binding API. Builtin RBAC roles are tenant-scoped and
identified by runtime ids, so the command must look them up per tenant.
"""
if legacy_role not in _LEGACY_ROLE_TO_BUILTIN_TAG:
expected_builtin_tag = {
TenantAccountRole.OWNER.value: "owner",
TenantAccountRole.ADMIN.value: "admin",
TenantAccountRole.EDITOR.value: "editor",
TenantAccountRole.NORMAL.value: "normal",
TenantAccountRole.DATASET_OPERATOR.value: "dataset_operator",
}.get(legacy_role)
if not expected_builtin_tag:
raise ValueError(f"Unsupported legacy workspace role: {legacy_role}")
return _resolve_builtin_role_ids(tenant_id, operator_account_id)[legacy_role]
def _iter_tenant_member_batches(
tenant_id: str | None,
*,
db_batch_size: int,
api_batch_size: int,
) -> Iterator[tuple[str, str, list[tuple[str, str]]]]:
"""Yield legacy member roles in tenant-scoped API-sized batches.
Rows are projected to primitive values and streamed from the database, so
the command never materializes every TenantAccountJoin ORM object. The
iterator only keeps one tenant's API-sized batches in memory while it
finds that tenant's owner account.
"""
with session_factory.create_session() as session:
stmt = (
select(TenantAccountJoin.tenant_id, TenantAccountJoin.account_id, TenantAccountJoin.role)
.order_by(TenantAccountJoin.tenant_id.asc(), TenantAccountJoin.id.asc())
.execution_options(yield_per=db_batch_size)
)
if tenant_id:
stmt = stmt.where(TenantAccountJoin.tenant_id == tenant_id)
current_tenant_id: str | None = None
owner_account_id: str | None = None
batches: list[list[tuple[str, str]]] = []
batch: list[tuple[str, str]] = []
def flush_current_tenant() -> Iterator[tuple[str, str, list[tuple[str, str]]]]:
if current_tenant_id is None:
return
if batch:
batches.append(batch.copy())
if not owner_account_id:
raise ValueError(f"Workspace owner not found for tenant={current_tenant_id}")
for item in batches:
yield current_tenant_id, owner_account_id, item
for row in session.execute(stmt):
workspace_id = str(row.tenant_id)
if current_tenant_id is not None and workspace_id != current_tenant_id:
yield from flush_current_tenant()
owner_account_id = None
batches = []
batch = []
current_tenant_id = workspace_id
account_id = str(row.account_id)
role = str(row.role)
if role == TenantAccountRole.OWNER.value:
owner_account_id = account_id
batch.append((account_id, role))
if len(batch) >= api_batch_size:
batches.append(batch)
batch = []
yield from flush_current_tenant()
def _member_already_has_role(current_roles_by_account_id: dict[str, set[str]], account_id: str, role_id: str) -> bool:
return current_roles_by_account_id.get(account_id) == {role_id}
def _replace_member_role(
tenant_id: str,
operator_account_id: str,
member_account_id: str,
role_id: str,
*,
session: Session,
) -> str:
RBACService.MemberRoles.replace(
roles = RBACService.Roles.list(
tenant_id=tenant_id,
account_id=operator_account_id,
member_account_id=member_account_id,
role_ids=[role_id],
session=session,
)
return member_account_id
options=ListOption(page_number=1, results_per_page=100),
).data
for role in roles:
if role.is_builtin and role.category == "global_system_default" and role.role_tag == expected_builtin_tag:
return str(role.id)
def _replace_member_role_with_new_session(
tenant_id: str,
operator_account_id: str,
member_account_id: str,
role_id: str,
) -> str:
with session_factory.create_session() as session:
return _replace_member_role(
tenant_id=tenant_id,
operator_account_id=operator_account_id,
member_account_id=member_account_id,
role_id=role_id,
session=session,
)
raise ValueError(f"Builtin RBAC role not found for tenant={tenant_id}, legacy_role={legacy_role}")
@click.command(
@@ -166,16 +42,7 @@ def _replace_member_role_with_new_session(
)
@click.option("--tenant-id", help="Only migrate a single workspace.")
@click.option("--dry-run", is_flag=True, default=False, help="Preview the migration without writing RBAC bindings.")
@click.option("--db-batch-size", default=5000, show_default=True, help="Rows fetched per database batch.")
@click.option("--api-batch-size", default=200, show_default=True, help="Members checked per RBAC batch_get call.")
@click.option("--workers", default=1, show_default=True, help="Concurrent member role replace calls per tenant batch.")
def migrate_member_roles_to_rbac(
tenant_id: str | None,
dry_run: bool,
db_batch_size: int,
api_batch_size: int,
workers: int,
) -> None:
def migrate_member_roles_to_rbac(tenant_id: str | None, dry_run: bool) -> None:
"""Backfill RBAC member-role bindings from legacy `TenantAccountJoin.role` data.
This is an offline migration command for workspaces that already have
@@ -183,329 +50,63 @@ def migrate_member_roles_to_rbac(
member-role binding store.
"""
click.echo(click.style("Starting RBAC member-role migration.", fg="green"))
if workers < 1:
raise click.BadParameter("workers must be >= 1", param_hint="--workers")
tenant_count = 0
scanned_count = 0
skipped_count = 0
migrated_count = 0
current_tenant_id: str | None = None
role_ids_by_legacy_role: dict[str, str] = {}
with session_factory.create_session() as session:
stmt = select(TenantAccountJoin).order_by(TenantAccountJoin.tenant_id.asc(), TenantAccountJoin.id.asc())
if tenant_id:
stmt = stmt.where(TenantAccountJoin.tenant_id == tenant_id)
for workspace_id, owner_account_id, batch in _iter_tenant_member_batches(
tenant_id,
db_batch_size=db_batch_size,
api_batch_size=api_batch_size,
):
scanned_count += len(batch)
if workspace_id != current_tenant_id:
tenant_count += 1
current_tenant_id = workspace_id
role_ids_by_legacy_role = _resolve_builtin_role_ids(workspace_id, owner_account_id)
click.echo(f"tenant={workspace_id}")
joins = list(session.scalars(stmt).all())
current_roles_by_account_id: dict[str, set[str]] = {}
if not dry_run:
current_roles = RBACService.MemberRoles.batch_get(
tenant_id=workspace_id,
account_id=owner_account_id,
member_account_ids=[account_id for account_id, _ in batch],
)
current_roles_by_account_id = {item.account_id: {role.id for role in item.roles} for item in current_roles}
replace_jobs: list[tuple[str, str]] = []
for member_account_id, legacy_role in batch:
resolved_role_id = role_ids_by_legacy_role.get(legacy_role)
if not resolved_role_id:
raise ValueError(f"Unsupported legacy workspace role: {legacy_role}")
if dry_run:
click.echo(
f"tenant={workspace_id} member={member_account_id} "
f"legacy_role={legacy_role} -> rbac_role_id={resolved_role_id}"
)
continue
if _member_already_has_role(current_roles_by_account_id, member_account_id, resolved_role_id):
skipped_count += 1
continue
replace_jobs.append((member_account_id, resolved_role_id))
if replace_jobs:
if workers == 1:
with session_factory.create_session() as session:
for member_account_id, resolved_role_id in replace_jobs:
_replace_member_role(
workspace_id,
owner_account_id,
member_account_id,
resolved_role_id,
session=session,
)
migrated_count += 1
else:
with ThreadPoolExecutor(max_workers=workers) as executor:
futures = [
executor.submit(
_replace_member_role_with_new_session,
workspace_id,
owner_account_id,
member_account_id,
resolved_role_id,
)
for member_account_id, resolved_role_id in replace_jobs
]
for future in as_completed(futures):
future.result()
migrated_count += 1
if scanned_count % 10000 == 0:
click.echo(
f"progress scanned={scanned_count} migrated={migrated_count} skipped={skipped_count}",
err=True,
)
if scanned_count == 0:
if not joins:
click.echo(click.style("No workspace members found for migration.", fg="yellow"))
return
if dry_run:
click.echo(
click.style(
f"Dry run completed. Scanned {scanned_count} members across {tenant_count} tenants. "
"No RBAC bindings were written.",
fg="yellow",
owner_account_by_tenant: dict[str, str] = {}
resolved_role_ids: dict[tuple[str, str], str] = {}
migrated_count = 0
for join in joins:
workspace_id = str(join.tenant_id)
member_account_id = str(join.account_id)
legacy_role = str(join.role)
if workspace_id not in owner_account_by_tenant:
owner_join = next(
(
item
for item in joins
if str(item.tenant_id) == workspace_id and str(item.role) == TenantAccountRole.OWNER.value
),
None,
)
)
else:
if not owner_join:
raise ValueError(f"Workspace owner not found for tenant={workspace_id}")
owner_account_by_tenant[workspace_id] = str(owner_join.account_id)
operator_account_id = owner_account_by_tenant[workspace_id]
cache_key = (workspace_id, legacy_role)
if cache_key not in resolved_role_ids:
resolved_role_ids[cache_key] = _resolve_builtin_role_id(workspace_id, operator_account_id, legacy_role)
resolved_role_id = resolved_role_ids[cache_key]
click.echo(
click.style(
f"RBAC member-role migration completed. Scanned {scanned_count} members across {tenant_count} tenants, "
f"migrated {migrated_count}, skipped {skipped_count} already up-to-date.",
fg="green",
)
f"tenant={workspace_id} member={member_account_id} "
f"legacy_role={legacy_role} -> rbac_role_id={resolved_role_id}"
)
if dry_run:
continue
def _dataset_permission_enum(permission: DatasetPermissionEnum | str | None) -> DatasetPermissionEnum:
if permission is None:
return DatasetPermissionEnum.ONLY_ME
try:
return DatasetPermissionEnum(permission)
except ValueError as exc:
raise ValueError(f"Unsupported legacy dataset permission: {permission}") from exc
def _rbac_dataset_scope_for_legacy_permission(permission: DatasetPermissionEnum) -> RBACResourceWhitelistScope:
if permission is DatasetPermissionEnum.ALL_TEAM:
return RBACResourceWhitelistScope.ALL
if permission in {DatasetPermissionEnum.ONLY_ME, DatasetPermissionEnum.PARTIAL_TEAM}:
return RBACResourceWhitelistScope.SPECIFIC
raise ValueError(f"Unsupported legacy dataset permission: {permission}")
def _emit_dataset_permission_migration_event(payload: dict[str, object]) -> None:
click.echo(json.dumps(payload, sort_keys=True))
@click.command(
"rbac-migrate-dataset-permissions",
help=(
"Migrate legacy dataset permission scopes and partial members into RBAC dataset access bindings. "
"Side effect: replacing each dataset whitelist clears existing per-user policy bindings; "
"the command then recreates legacy partial-member default bindings."
),
)
@click.option("--tenant-id", help="Only migrate datasets in a single workspace.")
@click.option("--dataset-id", help="Only migrate a single dataset.")
@click.option("--batch-size", default=500, show_default=True, type=click.IntRange(min=1))
@click.option(
"--dry-run/--apply",
default=True,
show_default=True,
help="Preview the migration without writing RBAC bindings. Use --apply to write changes.",
)
def migrate_dataset_permissions_to_rbac(
tenant_id: str | None,
dataset_id: str | None,
batch_size: int,
dry_run: bool,
) -> None:
"""Backfill RBAC dataset access config from legacy `Dataset.permission`.
Legacy mapping:
- all_team_members -> RBAC dataset whitelist scope "all"
- partial_members -> RBAC dataset whitelist scope "specific" plus each partial member gets the
virtual default policy
- only_me -> RBAC dataset whitelist scope "specific" with no member policy bindings
The command replaces each dataset's RBAC whitelist scope first. RBAC clears
existing per-user policy bindings during that replace, then this command
recreates the legacy partial-member default bindings. Re-running it is
therefore idempotent for a dataset's current legacy configuration.
"""
click.echo(click.style("Starting RBAC dataset permission migration.", fg="green"))
scanned_count = 0
scope_migrated_count = 0
user_policy_migrated_count = 0
partial_dataset_count = 0
last_dataset_id: str | None = None
while True:
with session_factory.create_session() as session:
stmt = (
select(Dataset.id, Dataset.tenant_id, Dataset.permission, Dataset.created_by)
.order_by(Dataset.id.asc())
.limit(batch_size)
)
if tenant_id:
stmt = stmt.where(Dataset.tenant_id == tenant_id)
if dataset_id:
stmt = stmt.where(Dataset.id == dataset_id)
if last_dataset_id:
stmt = stmt.where(Dataset.id > last_dataset_id)
dataset_rows = list(session.execute(stmt).all())
if not dataset_rows:
break
dataset_ids = [str(row.id) for row in dataset_rows]
partial_members_by_dataset_id: dict[str, list[str]] = {item: [] for item in dataset_ids}
permission_rows = session.execute(
select(DatasetPermission.dataset_id, DatasetPermission.account_id).where(
DatasetPermission.dataset_id.in_(dataset_ids)
)
).all()
for row in permission_rows:
partial_members_by_dataset_id[str(row.dataset_id)].append(str(row.account_id))
for dataset in dataset_rows:
workspace_id = str(dataset.tenant_id)
current_dataset_id = str(dataset.id)
operator_account_id = str(dataset.created_by)
permission_value = _dataset_permission_enum(dataset.permission)
scope = _rbac_dataset_scope_for_legacy_permission(permission_value)
partial_member_ids = sorted(set(partial_members_by_dataset_id[current_dataset_id]))
should_bind_partial_members = permission_value is DatasetPermissionEnum.PARTIAL_TEAM
click.echo(
f"tenant={workspace_id} dataset={current_dataset_id} "
f"operator={operator_account_id} "
f"legacy_permission={permission_value} -> rbac_scope={scope} "
f"partial_members={len(partial_member_ids) if should_bind_partial_members else 0}"
)
scanned_count += 1
replace_whitelist_payload = ReplaceMemberBindings(scope=scope)
if dry_run:
_emit_dataset_permission_migration_event(
{
"event": "dataset_permission_migration_proposed_change",
"action": "replace_whitelist",
"dry_run": True,
"tenant_id": workspace_id,
"dataset_id": current_dataset_id,
"operator_account_id": operator_account_id,
"before": {
"legacy_dataset_permission": permission_value.value,
"legacy_partial_member_ids": partial_member_ids if should_bind_partial_members else [],
},
"after": {
"rbac_whitelist_scope": scope.value,
},
"call": {
"method": "RBACService.DatasetAccess.replace_whitelist",
"kwargs": {
"tenant_id": workspace_id,
"account_id": operator_account_id,
"dataset_id": current_dataset_id,
"payload": replace_whitelist_payload.model_dump(mode="json"),
},
},
}
)
if not dry_run:
RBACService.DatasetAccess.replace_whitelist(
tenant_id=workspace_id,
account_id=operator_account_id,
dataset_id=current_dataset_id,
payload=replace_whitelist_payload,
)
scope_migrated_count += 1
if should_bind_partial_members:
partial_dataset_count += 1
for member_account_id in partial_member_ids:
replace_user_access_policies_payload = ReplaceUserAccessPolicies(
access_policy_ids=[_RBAC_DEFAULT_ACCESS_POLICY_ID],
)
if dry_run:
_emit_dataset_permission_migration_event(
{
"event": "dataset_permission_migration_proposed_change",
"action": "replace_user_access_policies",
"dry_run": True,
"tenant_id": workspace_id,
"dataset_id": current_dataset_id,
"operator_account_id": operator_account_id,
"target_account_id": member_account_id,
"before": {
"legacy_dataset_permission": permission_value.value,
"legacy_partial_member_id": member_account_id,
},
"after": {
"rbac_user_access_policy_ids": [_RBAC_DEFAULT_ACCESS_POLICY_ID],
},
"call": {
"method": "RBACService.DatasetAccess.replace_user_access_policies",
"kwargs": {
"tenant_id": workspace_id,
"account_id": operator_account_id,
"dataset_id": current_dataset_id,
"target_account_id": member_account_id,
"payload": replace_user_access_policies_payload.model_dump(
mode="json", exclude_unset=True
),
},
},
}
)
continue
RBACService.DatasetAccess.replace_user_access_policies(
tenant_id=workspace_id,
account_id=operator_account_id,
dataset_id=current_dataset_id,
target_account_id=member_account_id,
payload=replace_user_access_policies_payload,
)
user_policy_migrated_count += 1
last_dataset_id = dataset_ids[-1]
if dataset_id:
break
if scanned_count == 0:
click.echo(click.style("No datasets found for migration.", fg="yellow"))
return
RBACService.MemberRoles.replace(
tenant_id=workspace_id,
account_id=operator_account_id,
member_account_id=member_account_id,
role_ids=[resolved_role_id],
)
migrated_count += 1
if dry_run:
click.echo(
click.style(
f"Dry run completed. Scanned {scanned_count} datasets; "
f"{partial_dataset_count} partial-member datasets would be migrated.",
fg="yellow",
)
)
click.echo(click.style("Dry run completed. No RBAC bindings were written.", fg="yellow"))
else:
click.echo(
click.style(
"RBAC dataset permission migration completed. "
f"Scanned {scanned_count} datasets, migrated {scope_migrated_count} scopes, "
f"wrote {user_policy_migrated_count} user default-policy bindings.",
fg="green",
)
)
click.echo(click.style(f"RBAC member-role migration completed. Migrated {migrated_count} members.", fg="green"))
+125 -918
View File
File diff suppressed because it is too large Load Diff
+4 -7
View File
@@ -188,26 +188,23 @@ where sites.id is null limit 1000"""
if app_id in failed_app_ids:
continue
session = db.session()
try:
app = session.scalar(select(App).where(App.id == app_id))
app = db.session.scalar(select(App).where(App.id == app_id))
if not app:
logger.info("App %s not found", app_id)
continue
tenant = session.get(Tenant, app.tenant_id)
tenant = app.tenant
if tenant:
accounts = tenant.get_accounts(session=session)
accounts = tenant.get_accounts()
if not accounts:
logger.info("Fix failed for app %s", app.id)
continue
account = accounts[0]
logger.info("Fixing missing site for app %s", app.id)
app_was_created.send(app, account=account, session=session)
session.commit()
app_was_created.send(app, account=account)
except Exception:
session.rollback()
failed_app_ids.append(app_id)
click.echo(click.style(f"Failed to fix missing site for app {app_id}", fg="red"))
logger.exception("Failed to fix app related site missing issue, app_id: %s", app_id)
+6 -13
View File
@@ -1,11 +1,10 @@
import json
from typing import cast
import click
from flask import current_app
from sqlalchemy import select
from sqlalchemy.exc import SQLAlchemyError
from sqlalchemy.orm import Session, sessionmaker
from sqlalchemy.orm import sessionmaker
from configs import dify_config
from core.rag.datasource.vdb.vector_factory import Vector
@@ -14,7 +13,6 @@ from core.rag.index_processor.constant.built_in_field import BuiltInField
from core.rag.index_processor.constant.index_type import IndexStructureType, IndexTechniqueType
from core.rag.models.document import ChildDocument, Document
from extensions.ext_database import db
from libs.pagination import paginate_query
from models.dataset import Dataset, DatasetCollectionBinding, DatasetMetadata, DatasetMetadataBinding, DocumentSegment
from models.dataset import Document as DatasetDocument
from models.enums import DatasetMetadataType, IndexingStatus, SegmentStatus
@@ -102,8 +100,7 @@ def migrate_annotation_vector_database():
)
documents.append(document)
with Session(db.engine) as session:
vector = Vector(dataset, attributes=["doc_id", "annotation_id", "app_id"], session=session)
vector = Vector(dataset, attributes=["doc_id", "annotation_id", "app_id"])
click.echo(f"Migrating annotations for app: {app.id}.")
try:
@@ -178,7 +175,6 @@ def migrate_knowledge_vector_database():
VectorType.OCEANBASE,
}
page = 1
db_session = db.session()
while True:
try:
stmt = (
@@ -187,7 +183,7 @@ def migrate_knowledge_vector_database():
.order_by(Dataset.created_at.desc())
)
datasets = paginate_query(stmt, page=page, per_page=50, max_per_page=50, session=db_session)
datasets = db.paginate(select=stmt, page=page, per_page=50, max_per_page=50, error_out=False)
if not datasets.items:
break
except SQLAlchemyError:
@@ -230,8 +226,7 @@ def migrate_knowledge_vector_database():
index_struct_dict = {"type": vector_type, "vector_store": {"class_prefix": collection_name}}
dataset.index_struct = json.dumps(index_struct_dict)
with Session(db.engine) as session:
vector = Vector(dataset, session=session)
vector = Vector(dataset)
click.echo(f"Migrating dataset {dataset.id}.")
try:
@@ -278,7 +273,7 @@ def migrate_knowledge_vector_database():
},
)
if dataset_document.doc_form == IndexStructureType.PARENT_CHILD_INDEX:
child_chunks = segment.get_child_chunks(session=db_session)
child_chunks = segment.get_child_chunks()
if child_chunks:
child_documents = []
for child_chunk in child_chunks:
@@ -414,9 +409,7 @@ def old_metadata_migration():
.where(DatasetDocument.doc_metadata.is_not(None))
.order_by(DatasetDocument.created_at.desc())
)
documents = paginate_query(
stmt, page=page, per_page=50, max_per_page=50, session=cast(Session, db.session())
)
documents = db.paginate(select=stmt, page=page, per_page=50, max_per_page=50, error_out=False)
except SQLAlchemyError:
raise
if not documents:
+1 -13
View File
@@ -14,15 +14,9 @@ class EnterpriseFeatureConfig(BaseSettings):
default=False,
)
WEBAPP_PUBLIC_ACCESS_ENABLED: bool = Field(
description="Whether admins are allowed to set a webapp's access mode to public (anyone with the link, "
"no auth). Disable in security-sensitive on-prem deployments.",
default=True,
)
CAN_REPLACE_LOGO: bool = Field(
description="Allow customization of the enterprise logo.",
default=False,
default=True,
)
ENTERPRISE_REQUEST_TIMEOUT: int = Field(
@@ -40,12 +34,6 @@ class EnterpriseFeatureConfig(BaseSettings):
default=False,
)
ENTERPRISE_RBAC_REQUEST_TIMEOUT: int = Field(
ge=1,
description="Maximum timeout in seconds for inner RBAC requests.",
default=30,
)
class EnterpriseTelemetryConfig(BaseSettings):
"""
-2
View File
@@ -1,6 +1,5 @@
from configs.extra.agent_backend_config import AgentBackendConfig
from configs.extra.archive_config import ArchiveStorageConfig
from configs.extra.knowledge_fs_config import KnowledgeFSConfig
from configs.extra.notion_config import NotionConfig
from configs.extra.sentry_config import SentryConfig
@@ -9,7 +8,6 @@ class ExtraServiceConfig(
# place the configs in alphabet order
AgentBackendConfig,
ArchiveStorageConfig,
KnowledgeFSConfig,
NotionConfig,
SentryConfig,
):
+11 -23
View File
@@ -1,4 +1,4 @@
from pydantic import Field, NonNegativeFloat, NonNegativeInt, PositiveFloat
from pydantic import Field
from pydantic_settings import BaseSettings
@@ -22,34 +22,22 @@ class AgentBackendConfig(BaseSettings):
default="success",
)
AGENT_BACKEND_STREAM_READ_TIMEOUT_SECONDS: PositiveFloat = Field(
description="Read timeout for one Agent backend SSE connection.",
default=30,
)
AGENT_BACKEND_STREAM_MAX_RECONNECTS: NonNegativeInt = Field(
description="Maximum Agent backend SSE reconnects before failing the run.",
default=3,
)
AGENT_BACKEND_RUN_TIMEOUT_SECONDS: PositiveFloat = Field(
description="Total deadline for one Agent backend run event stream.",
default=1200,
)
AGENT_SHELL_ENABLED: bool = Field(
description=(
"Inject the dify.shell layer (sandboxed bash workspace) into Agent runs. "
"Requires the agent backend to be wired with a shellctl entrypoint before "
"shell-using Agent runs are executed."
"Requires the agent backend to be wired with a shellctl entrypoint; keep it "
"off until shellctl is deployed, otherwise every agent run that includes the "
"shell layer will fail."
),
default=True,
default=False,
)
AGENT_APP_TEXT_DELTA_DEBOUNCE_SECONDS: NonNegativeFloat = Field(
AGENT_DRIVE_MANIFEST_ENABLED: bool = Field(
description=(
"Buffer Agent App assistant text deltas for up to this many seconds before "
"publishing SSE chunks. Set to 0 to publish each delta immediately."
"Inject the dify.drive layer (Skills & Files drive manifest declaration) "
"into Agent runs. The declaration is an index only — the agent backend "
"pulls the actual SKILL.md / files through the back proxy. Keep it off "
"until the agent backend registers the dify.drive layer type."
),
default=0.5,
default=False,
)
-64
View File
@@ -1,64 +0,0 @@
"""Configuration for the optional KnowledgeFS Console bridge."""
from urllib.parse import urlsplit
from pydantic import Field, PositiveFloat, SecretStr, field_validator, model_validator
from pydantic_settings import BaseSettings
class KnowledgeFSConfig(BaseSettings):
"""Server-only settings for the KnowledgeFS production connection."""
KNOWLEDGE_FS_ENABLED: bool = Field(
default=False,
description="Enable the private KnowledgeFS Console bridge.",
)
KNOWLEDGE_FS_BASE_URL: str | None = Field(default=None, description="KnowledgeFS gateway base URL.")
KNOWLEDGE_FS_JWT_SECRET: SecretStr | None = Field(
default=None,
min_length=32,
description="Shared secret used to sign short-lived KnowledgeFS service JWTs.",
)
KNOWLEDGE_FS_SSE_READ_TIMEOUT_SECONDS: PositiveFloat = Field(default=300.0, le=3600.0, allow_inf_nan=False)
KNOWLEDGE_FS_TIMEOUT_SECONDS: PositiveFloat = Field(default=10.0, le=60.0, allow_inf_nan=False)
@field_validator(
"KNOWLEDGE_FS_BASE_URL",
"KNOWLEDGE_FS_JWT_SECRET",
mode="before",
)
@classmethod
def normalize_optional_string(cls, value: object) -> object:
if isinstance(value, SecretStr):
normalized = value.get_secret_value().strip()
return SecretStr(normalized) if normalized else None
if isinstance(value, str):
normalized = value.strip()
return normalized or None
return value
@field_validator("KNOWLEDGE_FS_BASE_URL")
@classmethod
def validate_base_url(cls, value: str | None) -> str | None:
if value is None:
return None
parsed = urlsplit(value)
if parsed.scheme not in {"http", "https"} or not parsed.netloc:
raise ValueError("KNOWLEDGE_FS_BASE_URL must be an absolute HTTP(S) URL")
try:
_ = parsed.port
except ValueError as exc:
raise ValueError("KNOWLEDGE_FS_BASE_URL must include a valid port") from exc
if parsed.username or parsed.password or parsed.query or parsed.fragment:
raise ValueError("KNOWLEDGE_FS_BASE_URL must not include credentials, query, or fragment")
return value.rstrip("/")
@model_validator(mode="after")
def validate_enabled_connection(self) -> "KnowledgeFSConfig":
if not self.KNOWLEDGE_FS_ENABLED:
return self
if bool(self.KNOWLEDGE_FS_BASE_URL) != bool(self.KNOWLEDGE_FS_JWT_SECRET):
raise ValueError("KNOWLEDGE_FS_BASE_URL and KNOWLEDGE_FS_JWT_SECRET must be configured together")
if not self.KNOWLEDGE_FS_BASE_URL:
raise ValueError("KnowledgeFS connection settings are required when the integration is enabled")
return self
+2 -95
View File
@@ -1,4 +1,4 @@
from datetime import datetime, timedelta
from datetime import timedelta
from enum import StrEnum
from typing import Literal
@@ -11,7 +11,6 @@ from pydantic import (
PositiveFloat,
PositiveInt,
computed_field,
field_validator,
)
from pydantic_settings import BaseSettings
@@ -276,63 +275,6 @@ class PluginConfig(BaseSettings):
default=50 * 1024 * 1024,
)
NEW_USER_DEFAULT_PLUGIN_IDS: str = Field(
description="Comma-separated marketplace plugin IDs whose latest versions are installed for new users",
default="",
)
@field_validator("PLUGIN_REMOTE_INSTALL_PORT", mode="before")
@classmethod
def _reject_host_port_shaped_plugin_remote_install_port(cls, v):
"""Reject ``host:port``-shaped values with an actionable hint.
``EXPOSE_PLUGIN_DEBUGGING_PORT`` is overloaded: it feeds both the
plugin_daemon ``ports:`` mapping (where ``127.0.0.1:5003`` is valid
compose syntax) and this integer app setting advertised in the console.
Without this guard a loopback bind spec crashloops the api container
with an opaque ``int_parsing`` traceback. See issue #39323.
"""
if isinstance(v, str) and ":" in v.strip():
raise ValueError(
"PLUGIN_REMOTE_INSTALL_PORT must be a bare port number, got "
f"{v!r}. A 'host:port' value usually means "
"EXPOSE_PLUGIN_DEBUGGING_PORT was set to a compose publish spec "
"like '127.0.0.1:5003'; bind loopback via a "
"docker-compose.override.yaml instead of overloading this var."
)
return v
@property
def NEW_USER_DEFAULT_PLUGIN_ID_LIST(self) -> list[str]:
return [item.strip() for item in self.NEW_USER_DEFAULT_PLUGIN_IDS.split(",") if item.strip()]
NEW_USER_DEFAULT_MODELS: str = Field(
description=("Comma-separated default models for new users in 'model_type:provider:model' format"),
default="",
)
@property
def NEW_USER_DEFAULT_MODEL_LIST(self) -> list[tuple[str, str, str]]:
default_models: list[tuple[str, str, str]] = []
configured_model_types: set[str] = set()
for item in self.NEW_USER_DEFAULT_MODELS.split(","):
if not item.strip():
continue
parts = tuple(part.strip() for part in item.split(":", 2))
if len(parts) != 3 or not all(parts):
raise ValueError("NEW_USER_DEFAULT_MODELS entries must use 'model_type:provider:model' format")
model_type, provider, model = parts
if model_type in configured_model_types:
raise ValueError(f"NEW_USER_DEFAULT_MODELS contains duplicate model type: {model_type}")
configured_model_types.add(model_type)
default_models.append((model_type, provider, model))
return default_models
class MarketplaceConfig(BaseSettings):
"""
@@ -421,10 +363,7 @@ class FileAccessConfig(BaseSettings):
INTERNAL_FILES_URL: str = Field(
description="Internal base URL for file access within Docker network,"
" used for plugin daemon and internal service communication."
" Explicit INTERNAL_FILES_URL takes precedence; otherwise SERVER_CONSOLE_API_URL is used,"
" then FILES_URL.",
validation_alias=AliasChoices("INTERNAL_FILES_URL", "SERVER_CONSOLE_API_URL"),
alias_priority=1,
" Falls back to FILES_URL if not specified.",
default="",
)
@@ -842,11 +781,6 @@ class WorkflowConfig(BaseSettings):
default=500,
)
WORKFLOW_GENERATOR_NODE_BUILDER_MAX_WORKERS: PositiveInt = Field(
description="Maximum concurrent node-builder LLM calls per workflow generation request",
default=6,
)
WORKFLOW_MAX_EXECUTION_TIME: PositiveInt = Field(
description="Maximum execution time in seconds for a single workflow",
default=1200,
@@ -1139,12 +1073,6 @@ class MailConfig(BaseSettings):
default=None,
)
class HomepageConfig(BaseSettings):
"""
Configuration for homepage feature toggles exposed through system features.
"""
ENABLE_TRIAL_APP: bool = Field(
description="Enable trial app",
default=False,
@@ -1155,21 +1083,6 @@ class HomepageConfig(BaseSettings):
default=False,
)
ENABLE_LEARN_APP: bool = Field(
description="Enable Learn App",
default=True,
)
ENABLE_STEP_BY_STEP_TOUR: bool = Field(
description="Enable account-level Step-by-step Tour eligibility checks",
default=False,
)
STEP_BY_STEP_TOUR_ROLLOUT_STARTED_AT: datetime | None = Field(
description="UTC timestamp after which newly initialized accounts are eligible for Step-by-step Tour",
default=None,
)
class RagEtlConfig(BaseSettings):
"""
@@ -1497,11 +1410,6 @@ class LoginConfig(BaseSettings):
class AccountConfig(BaseSettings):
ENABLE_CHANGE_EMAIL: bool = Field(
description="whether users can change their email address",
default=True,
)
ACCOUNT_DELETION_TOKEN_EXPIRY_MINUTES: PositiveInt = Field(
description="Duration in minutes for which a account deletion token remains valid",
default=5,
@@ -1581,7 +1489,6 @@ class FeatureConfig(
EndpointConfig,
FileAccessConfig,
FileUploadConfig,
HomepageConfig,
HttpConfig,
InnerAPIConfig,
IndexingConfig,
-5
View File
@@ -153,11 +153,6 @@ class RedisConfig(BaseSettings):
default=30,
)
REDIS_KEEPALIVE: bool = Field(default=False, description="Keepalive for Redis connections")
REDIS_KEEPALIVE_IDLE: PositiveInt = Field(default=30, description="redis keepalive idle timeout")
REDIS_KEEPALIVE_INTERVAL: PositiveInt = Field(default=10, description="redis keepalive interval")
REDIS_KEEPALIVE_COUNT: PositiveInt = Field(default=10, description="redis keepalive count")
@field_validator("REDIS_MAX_CONNECTIONS", mode="before")
@classmethod
def _empty_string_to_none_for_max_conns(cls, v):
+19
View File
@@ -2,6 +2,7 @@ from typing import Literal, Protocol, cast
from urllib.parse import quote_plus, urlunparse
from pydantic import AliasChoices, Field
from pydantic.types import NonNegativeInt
from pydantic_settings import BaseSettings
@@ -70,6 +71,24 @@ class RedisPubSubConfig(BaseSettings):
default=600,
)
PUBSUB_LISTENER_JOIN_TIMEOUT_MS: NonNegativeInt = Field(
validation_alias=AliasChoices("EVENT_BUS_LISTENER_JOIN_TIMEOUT_MS", "PUBSUB_LISTENER_JOIN_TIMEOUT_MS"),
description=(
"Maximum time (milliseconds) that ``Subscription.close()`` waits for its listener thread to "
"finish before returning. Bounds the tail latency between a terminal event being delivered to "
"an SSE client and the response stream actually closing.\n\n"
"The listener thread blocks on a polling read (XREAD BLOCK for streams, get_message timeout "
"for pubsub/sharded) with a fixed 1s window, so close() naturally has to wait up to ~1s for "
"the thread to notice the subscription was closed. Setting this lower (e.g. 100) lets close() "
"return promptly while the daemon listener thread cleans itself up on the next poll "
"boundary - safe because the listener holds no critical state and exits within one poll "
"window. Setting it higher (e.g. 5000) gives the listener more grace before close() gives up "
"and logs a warning. Default 2000ms preserves the pre-change behaviour.\n\n"
"Also accepts ENV: EVENT_BUS_LISTENER_JOIN_TIMEOUT_MS."
),
default=2000,
)
def _build_default_pubsub_url(self) -> str:
defaults = _redis_defaults(self)
if not defaults.REDIS_HOST or not defaults.REDIS_PORT:
-16
View File
@@ -6,17 +6,6 @@ class PyProjectConfig(BaseModel):
version: str = Field(description="Dify version", default="")
class DifyToolConfig(BaseModel):
min_difyctl_version: str = Field(
description="Oldest difyctl version served on /openapi/v1",
default="0.0.0",
)
class ToolConfig(BaseModel):
dify: DifyToolConfig = Field(default=DifyToolConfig())
class PyProjectTomlConfig(BaseSettings):
"""
configs in api/pyproject.toml
@@ -26,8 +15,3 @@ class PyProjectTomlConfig(BaseSettings):
description="configs in the project section of pyproject.toml",
default=PyProjectConfig(),
)
tool: ToolConfig = Field(
description="configs in the [tool.*] section of pyproject.toml",
default=ToolConfig(),
)
+1 -1
View File
@@ -1 +1 @@
CURRENT_APP_DSL_VERSION = "0.7.0"
CURRENT_APP_DSL_VERSION = "0.6.0"
File diff suppressed because one or more lines are too long
@@ -1,61 +0,0 @@
from typing import Any
from sqlalchemy import select
from sqlalchemy.orm import Session
from core.app.apps.agent_app.app_feature_projection import merge_agent_app_features
from core.app.apps.agent_app.app_variable_projection import agent_app_variables_to_user_input_form
from core.app.apps.agent_app.errors import AgentAppGeneratorError, AgentAppNotPublishedError
from models.agent import Agent, AgentConfigSnapshot, AgentStatus
from models.agent_config_entities import AgentSoulConfig
from models.model import App, load_annotation_reply_config
def get_published_agent_app_feature_dict_and_user_input_form(
app_model: App,
*,
session: Session,
) -> tuple[dict[str, Any], list[dict[str, Any]]]:
"""Return public Agent App parameters backed by the published Agent Soul."""
app_model_config = app_model.app_model_config_with_session(session=session)
agent_id = app_model.bound_agent_id
if not agent_id:
raise AgentAppGeneratorError("Agent App has no bound Agent")
agent = session.scalar(
select(Agent)
.where(
Agent.tenant_id == app_model.tenant_id,
Agent.id == agent_id,
Agent.status == AgentStatus.ACTIVE,
)
.limit(1)
)
if agent is None:
raise AgentAppGeneratorError("Agent App has no bound Agent")
# active_config_is_published means the draft has no unpublished edits; the public app
# can still read parameters from the active snapshot while a newer draft is pending.
if not agent.active_config_snapshot_id:
raise AgentAppNotPublishedError("Agent has not been published")
snapshot = session.scalar(
select(AgentConfigSnapshot)
.where(
AgentConfigSnapshot.tenant_id == app_model.tenant_id,
AgentConfigSnapshot.agent_id == agent.id,
AgentConfigSnapshot.id == agent.active_config_snapshot_id,
)
.limit(1)
)
if snapshot is None:
raise AgentAppGeneratorError("Agent published version not found")
agent_soul = AgentSoulConfig.model_validate(snapshot.config_snapshot_dict)
annotation_reply = load_annotation_reply_config(session, app_model.id) if app_model_config else None
features_dict = merge_agent_app_features(
agent_soul=agent_soul,
app_model_config=app_model_config,
annotation_reply=annotation_reply,
)
return features_dict, agent_app_variables_to_user_input_form(agent_soul.app_variables)
-110
View File
@@ -1,110 +0,0 @@
from __future__ import annotations
from collections.abc import Sequence
from dataclasses import dataclass
from typing import TYPE_CHECKING
from sqlalchemy.orm import Session
from services.enterprise import rbac_service as enterprise_rbac_service
if TYPE_CHECKING:
from services.app_service import AppListBaseParams
from services.enterprise.rbac_service import MyPermissionsResponse
# Permission keys (dot-notation, from MyPermissionsResponse) that grant
# list/preview access to an app. Keep this the single source of truth for both
# the console and OpenAPI app-list endpoints.
APP_LIST_PERMISSION_KEYS: frozenset[str] = frozenset({"app.preview", "app.acl.preview", "app.full_access"})
# Workspace permission key that lets a caller see apps they maintain even when
# those apps are not in their preview whitelist.
_MANAGE_OWN_APPS_PERMISSION_KEY = "app.create_and_management"
def has_app_list_permission(permission_keys: Sequence[str]) -> bool:
"""Return True if any of ``permission_keys`` grants app list/preview access."""
return any(permission_key in APP_LIST_PERMISSION_KEYS for permission_key in permission_keys)
@dataclass(frozen=True)
class AppAccessFilter:
"""Resolved RBAC visibility for app list/read endpoints.
``accessible_app_ids`` of ``None`` means the caller can see every app in the
workspace (unrestricted). Otherwise it is the exact set of app ids the
caller may preview; combined with ``can_manage_own_apps`` it also covers
apps the caller maintains.
"""
accessible_app_ids: set[str] | None
can_manage_own_apps: bool
@classmethod
def unrestricted(cls) -> AppAccessFilter:
"""Filter that imposes no restriction (RBAC disabled / not applicable)."""
return cls(accessible_app_ids=None, can_manage_own_apps=False)
def is_app_accessible(self, app_id: str, maintainer: str | None, account_id: str) -> bool:
"""Whether a single app is visible to the caller under this filter.
Mirrors the service-layer query gate: an app is visible when the filter
is unrestricted, the app id is whitelisted, or the caller maintains it
and holds ``app.create_and_management``.
"""
if self.accessible_app_ids is None:
return True
if app_id in self.accessible_app_ids:
return True
return self.can_manage_own_apps and maintainer is not None and maintainer == account_id
def apply_to_params(self, params: AppListBaseParams) -> None:
if self.accessible_app_ids is None:
return
params.accessible_app_ids = sorted(self.accessible_app_ids)
params.include_own_apps = self.can_manage_own_apps
def resolve_app_access_filter(
tenant_id: str,
account_id: str,
*,
session: Session,
permissions: MyPermissionsResponse | None = None,
) -> AppAccessFilter:
"""Compute the RBAC app-access filter for ``account_id`` in ``tenant_id``.
Pass ``permissions`` when the caller has already fetched the snapshot (the
console controller reuses it for per-app permission keys) to avoid a second
inner-API round trip; otherwise it is fetched here.
"""
if permissions is None:
permissions = enterprise_rbac_service.RBACService.MyPermissions.get(tenant_id, account_id, session=session)
whitelist_scope = enterprise_rbac_service.RBACService.AppAccess.whitelist_resources(tenant_id, account_id)
can_manage_own_apps = _MANAGE_OWN_APPS_PERMISSION_KEY in permissions.workspace.permission_keys
has_default_preview = has_app_list_permission(permissions.app.default_permission_keys) or has_app_list_permission(
permissions.workspace.permission_keys
)
permission_app_ids: set[str] | None = None
if not has_default_preview:
# Collect apps the caller can preview via per-app permission overrides.
permission_app_ids = {
override.resource_id
for override in permissions.app.overrides
if has_app_list_permission(override.permission_keys)
}
accessible_app_ids: set[str] | None
if getattr(whitelist_scope, "unrestricted", False):
accessible_app_ids = permission_app_ids
else:
accessible_app_ids = set(whitelist_scope.resource_ids)
if permission_app_ids is not None:
accessible_app_ids |= permission_app_ids
elif has_default_preview:
# Default preview overrides the whitelist restriction.
accessible_app_ids = None
return AppAccessFilter(accessible_app_ids=accessible_app_ids, can_manage_own_apps=can_manage_own_apps)
+1 -7
View File
@@ -1,6 +1,6 @@
from __future__ import annotations
from typing import Any, Literal
from typing import Any
from pydantic import BaseModel, ConfigDict, Field, RootModel, computed_field
@@ -52,11 +52,6 @@ class AudioTranscriptResponse(ResponseModel):
text: str
class ValidationResultResponse(ResponseModel):
result: Literal["success", "error"]
error: str | None = None
class SimpleResultMessageResponse(ResponseModel):
result: str
message: str
@@ -188,7 +183,6 @@ class Site(BaseModel):
description: str | None = None
copyright: str | None = None
privacy_policy: str | None = None
input_placeholder: str | None = None
custom_disclaimer: str | None = None
default_language: str
show_workflow_steps: bool
-65
View File
@@ -1,65 +0,0 @@
"""Controller session decorators.
`with_session` is an HTTP controller helper: it opens one SQLAlchemy session
for a Resource handler and injects it as the first argument after `self`.
Write handlers commit on success and roll back on failure. They use a regular
Session context so existing services may commit an intermediate unit and keep
using the same Session through SQLAlchemy's autobegin behavior. Pure read
handlers may opt out with `write=False`.
"""
from collections.abc import Callable
from functools import wraps
from typing import Concatenate, overload
from sqlalchemy.orm import Session
from core.db.session_factory import session_factory
@overload
def with_session[T, **P, R](
view: Callable[Concatenate[T, Session, P], R],
*,
write: bool = True,
) -> Callable[Concatenate[T, P], R]: ...
@overload
def with_session[T, **P, R](
view: None = None,
*,
write: bool = True,
) -> Callable[[Callable[Concatenate[T, Session, P], R]], Callable[Concatenate[T, P], R]]: ...
def with_session[T, **P, R](
view: Callable[Concatenate[T, Session, P], R] | None = None,
*,
write: bool = True,
) -> (
Callable[Concatenate[T, P], R] | Callable[[Callable[Concatenate[T, Session, P], R]], Callable[Concatenate[T, P], R]]
):
"""Inject a request-scoped session and finalize write handlers."""
def decorator(view: Callable[Concatenate[T, Session, P], R]) -> Callable[Concatenate[T, P], R]:
@wraps(view)
def wrapper(self: T, *args: P.args, **kwargs: P.kwargs) -> R:
if write:
with session_factory.create_session() as session:
try:
result = view(self, session, *args, **kwargs)
session.commit()
return result
except Exception:
session.rollback() # noqa: no-new-controller-sqlalchemy decorator owns transaction rollback
raise
with session_factory.create_session() as session:
return view(self, session, *args, **kwargs)
return wrapper
if view is None:
return decorator
return decorator(view)
+71 -72
View File
@@ -1,5 +1,26 @@
"""Shared decorator utilities for Dify controller layers.
This module provides decorators that are not tied to any single API group (e.g.
console, inner, service). Currently it exposes the RBAC permission gate, which
can be applied to any blueprint.
Key exports
-----------
``rbac_permission_required`` decorator that enforces enterprise RBAC access
control. When ``RBAC_ENABLED`` is ``False`` it is a no-op.
``RBACPermission``, ``RBACResourceScope`` re-exported from ``core.rbac`` so
callers only need a single import site.
Private helpers
---------------
``_extract_resource_id``, ``_is_resource_owned_by_current_user`` kept module-
private but accessible via the module namespace for unit-test patching.
"""
from collections.abc import Callable
from functools import wraps
from typing import TYPE_CHECKING
from sqlalchemy import select
from werkzeug.exceptions import Forbidden, NotFound
@@ -10,61 +31,40 @@ from extensions.ext_database import db
from libs.login import current_account_with_tenant
from models.dataset import Dataset
from models.model import App
from services.agent.roster_service import AgentRosterService
from services.enterprise.rbac_service import RBACService
__all__ = ["RBACPermission", "RBACResourceScope", "enforce_rbac_access", "rbac_permission_required"]
if TYPE_CHECKING:
from controllers.openapi.auth.data import AuthData
__all__ = ["RBACPermission", "RBACResourceScope", "rbac_permission_required"]
def enforce_rbac_access(
*,
tenant_id: str,
account_id: str,
def openapi_rbac_permission_required[**P, R](
resource_type: RBACResourceScope,
scene: RBACPermission,
*,
resource_required: bool = True,
path_args: dict[str, object] | None = None,
) -> None:
"""Enforce enterprise RBAC for an explicit account/tenant pair.
) -> Callable[[Callable[P, R]], Callable[P, R]]:
"""RBAC guard for OpenAPI endpoints that may be called by either an Account or an EndUser."""
inner = rbac_permission_required(resource_type, scene, resource_required=resource_required)
This is the flask-login-independent core of the RBAC gate so it can run
inside request-handling layers that resolve the caller themselves (e.g. the
openapi auth pipeline, which has the account on ``AuthData`` before
flask-login is mounted).
def decorator(view: Callable[P, R]) -> Callable[P, R]:
guarded = inner(view)
No-op when ``RBAC_ENABLED`` is ``False``. For resource-scoped checks the
resource ID is taken from ``path_args`` merged with ``request.view_args``;
resource ownership short-circuits the check. Raises ``Forbidden`` when
access is denied. For workspace-level checks pass ``resource_required=False``
so the RBAC request omits ``resource_id``.
@wraps(view)
def decorated(*args: P.args, **kwargs: P.kwargs) -> R:
auth_data: "AuthData | None" = kwargs.get("auth_data")
if not auth_data:
raise Forbidden() # openapi auth pipeline is required
if auth_data.caller_kind == "end_user":
# end_user is handled by openapi scope control
return view(*args, **kwargs)
return guarded(*args, **kwargs)
Args:
tenant_id: The tenant the access is evaluated against.
account_id: The account requesting access.
resource_type: The :class:`RBACResourceScope` member (app/dataset/workspace).
scene: The :class:`RBACPermission` permission point, e.g. ``RBACPermission.APP_DELETE``.
resource_required: Whether a concrete resource ID is required.
path_args: Extra path arguments to merge with ``request.view_args``.
"""
if not dify_config.RBAC_ENABLED:
return
check_resource_type = None if resource_type == RBACResourceScope.WORKSPACE else resource_type
resource_id = None
if resource_required and check_resource_type:
resource_id = _extract_resource_id(resource_type, tenant_id, path_args)
if _is_resource_owned_by_current_user(tenant_id, account_id, resource_type, resource_id):
return
allowed = RBACService.CheckAccess.check(
tenant_id,
account_id,
scene=scene,
resource_type=check_resource_type,
resource_id=resource_id,
)
if not allowed:
raise Forbidden()
return decorated
return decorator
def rbac_permission_required[**P, R](
resource_type: RBACResourceScope,
@@ -72,12 +72,14 @@ def rbac_permission_required[**P, R](
*,
resource_required: bool = True,
) -> Callable[[Callable[P, R]], Callable[P, R]]:
"""Check enterprise RBAC permissions for the current flask-login user.
"""Check enterprise RBAC permissions for the current user.
When ``RBAC_ENABLED`` is ``False`` the decorator is a no-op and the
request passes through unchanged. When enabled it resolves the current
account/tenant and delegates to :func:`enforce_rbac_access`, raising
``Forbidden`` if access is denied.
request passes through unchanged. When enabled it extracts the resource ID
from ``request.view_args`` for resource-scoped checks, calls the RBAC
service ``check-access`` endpoint, and raises ``Forbidden`` if the access
is denied. For workspace-level checks, set ``resource_required=False`` so
the RBAC request omits ``resource_id``.
Args:
resource_type: The :class:`RBACResourceScope` member (app/dataset/workspace).
@@ -92,14 +94,23 @@ def rbac_permission_required[**P, R](
return view(*args, **kwargs)
current_user, current_tenant_id = current_account_with_tenant()
enforce_rbac_access(
tenant_id=current_tenant_id,
account_id=current_user.id,
resource_type=resource_type,
check_resource_type = None if resource_type == RBACResourceScope.WORKSPACE else resource_type
resource_id = None
if resource_required and check_resource_type:
resource_id = _extract_resource_id(resource_type, kwargs)
if _is_resource_owned_by_current_user(current_tenant_id, current_user.id, resource_type, resource_id):
return view(*args, **kwargs)
allowed = RBACService.CheckAccess.check(
current_tenant_id,
current_user.id,
scene=scene,
resource_required=resource_required,
path_args=kwargs,
resource_type=check_resource_type,
resource_id=resource_id,
)
if not allowed:
raise Forbidden()
return view(*args, **kwargs)
return decorated
@@ -132,14 +143,11 @@ def _is_resource_owned_by_current_user(
return False
def _extract_resource_id(
resource_type: RBACResourceScope, tenant_id: str, path_args: dict[str, object] | None = None
) -> str:
def _extract_resource_id(resource_type: RBACResourceScope, path_args: dict[str, object] | None = None) -> str:
"""Extract the resource ID from matched path arguments.
Some legacy route classes use neutral names such as ``resource_id`` for
app/dataset resources, and Agent routes carry ``agent_id``, which is
resolved to the App backing that Agent.
app/dataset resources, and Agent App routes use ``agent_id`` as the app id.
Dataset endpoints behind a rag-pipeline route contain ``pipeline_id``
instead of ``dataset_id``. In that case we look up the associated
``Dataset`` row via ``Dataset.pipeline_id``.
@@ -150,19 +158,10 @@ def _extract_resource_id(
matched_args = {**view_args, **(path_args or {})}
if resource_type == RBACResourceScope.APP:
app_id = matched_args.get("app_id")
if app_id:
return str(app_id)
agent_id = matched_args.get("agent_id")
if agent_id:
authz_app_id = AgentRosterService(db.session).peek_authz_app_id(tenant_id=tenant_id, agent_id=str(agent_id))
return authz_app_id or str(agent_id)
resource_id = matched_args.get("resource_id")
if resource_id:
return str(resource_id)
raise ValueError("Missing app_id in request path")
app_id = matched_args.get("app_id") or matched_args.get("agent_id") or matched_args.get("resource_id")
if not app_id:
raise ValueError("Missing app_id in request path")
return str(app_id)
if resource_type == RBACResourceScope.DATASET:
dataset_id = matched_args.get("dataset_id") or matched_args.get("resource_id")
-8
View File
@@ -38,14 +38,11 @@ from . import (
feature,
human_input_form,
init_validate,
knowledge_fs_proxy,
notification,
onboarding,
ping,
setup,
spec,
version,
workflow_run_archive,
)
from .agent import composer as agent_composer
from .agent import roster as agent_roster
@@ -57,7 +54,6 @@ from .app import (
agent_app_access,
agent_app_feature,
agent_app_sandbox,
agent_config_inspector,
agent_drive_inspector,
annotation,
app,
@@ -161,7 +157,6 @@ __all__ = [
"agent_app_feature",
"agent_app_sandbox",
"agent_composer",
"agent_config_inspector",
"agent_drive_inspector",
"agent_providers",
"agent_roster",
@@ -197,7 +192,6 @@ __all__ = [
"human_input_form",
"init_validate",
"installed_app",
"knowledge_fs_proxy",
"load_balancing_config",
"login",
"mcp_server",
@@ -210,7 +204,6 @@ __all__ = [
"notification",
"oauth",
"oauth_server",
"onboarding",
"ops_trace",
"parameter",
"ping",
@@ -243,7 +236,6 @@ __all__ = [
"workflow_draft_variable",
"workflow_node_output_inspector",
"workflow_run",
"workflow_run_archive",
"workflow_statistic",
"workflow_trigger",
"workspace",
+4 -15
View File
@@ -1,21 +1,10 @@
from uuid import UUID
from sqlalchemy.orm import Session
from extensions.ext_database import db
from models.model import App
from services.agent.roster_service import AgentRosterService
def resolve_agent_app_model(*, session: Session, tenant_id: str, agent_id: UUID) -> App:
"""Resolve a roster Agent's public Agent App."""
return AgentRosterService(session).get_agent_app_model(tenant_id=tenant_id, agent_id=str(agent_id))
def resolve_agent_runtime_app_model(*, session: Session, tenant_id: str, agent_id: UUID) -> App:
"""Resolve the App that backs an Agent runtime surface.
This accepts both roster Agent Apps and workflow-only inline Agents with a
hidden backing App.
"""
return AgentRosterService(session).get_agent_runtime_app_model(tenant_id=tenant_id, agent_id=str(agent_id))
def resolve_agent_app_model(*, tenant_id: str, agent_id: UUID) -> App:
"""Resolve the hidden Agent App backing an Agent Console resource."""
return AgentRosterService(db.session).get_agent_app_model(tenant_id=tenant_id, agent_id=str(agent_id))
+36 -315
View File
@@ -1,13 +1,10 @@
from uuid import UUID
from flask import request
from flask_restx import Resource
from sqlalchemy.orm import Session
from werkzeug.exceptions import NotFound
from controllers.common.schema import query_params_from_model, register_response_schema_models, register_schema_models
from controllers.common.session import with_session
from controllers.common.schema import register_response_schema_models, register_schema_models
from controllers.console import console_ns
from controllers.console.agent.app_helpers import resolve_agent_app_model
from controllers.console.app.wraps import get_app_model
from controllers.console.wraps import (
RBACPermission,
@@ -31,16 +28,9 @@ from libs.login import login_required
from models.model import App, AppMode
from services.agent.composer_service import AgentComposerService
from services.agent.composer_validator import ComposerConfigValidator
from services.entities.agent_entities import (
ComposerSavePayload,
WorkflowAgentComposerQuery,
WorkflowComposerCopyFromRosterPayload,
)
from services.snippet_service import SnippetService
from services.entities.agent_entities import ComposerSavePayload
register_schema_models(
console_ns, ComposerSavePayload, WorkflowAgentComposerQuery, WorkflowComposerCopyFromRosterPayload
)
register_schema_models(console_ns, ComposerSavePayload)
register_response_schema_models(
console_ns,
AgentAppComposerResponse,
@@ -51,30 +41,27 @@ register_response_schema_models(
)
def _resolve_agent_app_id(*, tenant_id: str, agent_id: UUID) -> str:
return resolve_agent_app_model(tenant_id=tenant_id, agent_id=agent_id).id
@console_ns.route("/apps/<uuid:app_id>/workflows/draft/nodes/<string:node_id>/agent-composer")
class WorkflowAgentComposerApi(Resource):
@console_ns.response(
200, "Workflow agent composer state", console_ns.models[WorkflowAgentComposerResponse.__name__]
)
@console_ns.doc(params=query_params_from_model(WorkflowAgentComposerQuery))
@setup_required
@login_required
@account_initialization_required
@with_current_user_id
@with_current_tenant_id
@with_session
@get_app_model(mode=[AppMode.WORKFLOW, AppMode.ADVANCED_CHAT])
def get(self, session: Session, tenant_id: str, account_id: str, app_model: App, node_id: str):
query = WorkflowAgentComposerQuery.model_validate(request.args.to_dict(flat=True))
@with_current_tenant_id
def get(self, tenant_id: str, app_model: App, node_id: str):
return dump_response(
WorkflowAgentComposerResponse,
AgentComposerService.load_workflow_composer(
session=session,
tenant_id=tenant_id,
app_id=app_model.id,
node_id=node_id,
account_id=account_id,
snapshot_id=query.snapshot_id,
),
)
@@ -87,16 +74,14 @@ class WorkflowAgentComposerApi(Resource):
@account_initialization_required
@edit_permission_required
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_EDIT)
@get_app_model(mode=[AppMode.WORKFLOW, AppMode.ADVANCED_CHAT])
@with_current_user_id
@with_current_tenant_id
@with_session
@get_app_model(mode=[AppMode.WORKFLOW, AppMode.ADVANCED_CHAT])
def put(self, session: Session, tenant_id: str, account_id: str, app_model: App, node_id: str):
def put(self, tenant_id: str, account_id: str, app_model: App, node_id: str):
payload = ComposerSavePayload.model_validate(console_ns.payload or {})
return dump_response(
WorkflowAgentComposerResponse,
AgentComposerService.save_workflow_composer(
session=session,
tenant_id=tenant_id,
app_id=app_model.id,
node_id=node_id,
@@ -106,40 +91,6 @@ class WorkflowAgentComposerApi(Resource):
)
@console_ns.route("/apps/<uuid:app_id>/workflows/draft/nodes/<string:node_id>/agent-composer/copy-from-roster")
class WorkflowAgentComposerCopyFromRosterApi(Resource):
@console_ns.expect(console_ns.models[WorkflowComposerCopyFromRosterPayload.__name__])
@console_ns.response(
200,
"Workflow roster agent copied to inline agent",
console_ns.models[WorkflowAgentComposerResponse.__name__],
)
@setup_required
@login_required
@account_initialization_required
@edit_permission_required
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_EDIT)
@with_current_user_id
@with_current_tenant_id
@with_session
@get_app_model(mode=[AppMode.WORKFLOW, AppMode.ADVANCED_CHAT])
def post(self, session: Session, tenant_id: str, account_id: str, app_model: App, node_id: str):
payload = WorkflowComposerCopyFromRosterPayload.model_validate(console_ns.payload or {})
return dump_response(
WorkflowAgentComposerResponse,
AgentComposerService.copy_workflow_composer_from_roster(
session=session,
tenant_id=tenant_id,
app_id=app_model.id,
node_id=node_id,
account_id=account_id,
source_agent_id=payload.source_agent_id,
source_snapshot_id=payload.source_snapshot_id,
idempotency_key=payload.idempotency_key,
),
)
@console_ns.route("/apps/<uuid:app_id>/workflows/draft/nodes/<string:node_id>/agent-composer/validate")
class WorkflowAgentComposerValidateApi(Resource):
@console_ns.expect(console_ns.models[ComposerSavePayload.__name__])
@@ -149,21 +100,16 @@ class WorkflowAgentComposerValidateApi(Resource):
@setup_required
@login_required
@account_initialization_required
@with_current_tenant_id
@with_session(write=False)
@get_app_model(mode=[AppMode.WORKFLOW, AppMode.ADVANCED_CHAT])
def post(self, session: Session, tenant_id: str, app_model: App, node_id: str):
@with_current_tenant_id
def post(self, tenant_id: str, app_model: App, node_id: str):
payload = ComposerSavePayload.model_validate(console_ns.payload or {})
ComposerConfigValidator.validate_publish_payload(payload)
AgentComposerService.validate_knowledge_datasets(
session=session, tenant_id=tenant_id, agent_soul=payload.agent_soul
)
ComposerConfigValidator.validate_save_payload(payload)
findings = AgentComposerService.collect_validation_findings(
session=session,
tenant_id=tenant_id,
payload=payload,
agent_id=AgentComposerService.resolve_workflow_node_agent_id(
session=session, tenant_id=tenant_id, app_id=app_model.id, node_id=node_id
tenant_id=tenant_id, app_id=app_model.id, node_id=node_id
),
)
return dump_response(AgentComposerValidateResponse, {"result": "success", "errors": [], **findings})
@@ -177,15 +123,13 @@ class WorkflowAgentComposerCandidatesApi(Resource):
@setup_required
@login_required
@account_initialization_required
@get_app_model(mode=[AppMode.WORKFLOW, AppMode.ADVANCED_CHAT])
@with_current_user_id
@with_current_tenant_id
@with_session(write=False)
@get_app_model(mode=[AppMode.WORKFLOW, AppMode.ADVANCED_CHAT])
def get(self, session: Session, tenant_id: str, current_user_id: str, app_model: App, node_id: str):
def get(self, tenant_id: str, current_user_id: str, app_model: App, node_id: str):
return dump_response(
AgentComposerCandidatesResponse,
AgentComposerService.get_workflow_candidates(
session=session,
tenant_id=tenant_id,
app_id=app_model.id,
node_id=node_id,
@@ -201,10 +145,9 @@ class WorkflowAgentComposerImpactApi(Resource):
@setup_required
@login_required
@account_initialization_required
@with_current_tenant_id
@with_session(write=False)
@get_app_model(mode=[AppMode.WORKFLOW, AppMode.ADVANCED_CHAT])
def post(self, session: Session, tenant_id: str, app_model: App, node_id: str):
@with_current_tenant_id
def post(self, tenant_id: str, app_model: App, node_id: str):
payload = ComposerSavePayload.model_validate(console_ns.payload or {})
current_snapshot_id = payload.binding.current_snapshot_id if payload.binding else None
if not current_snapshot_id:
@@ -213,9 +156,7 @@ class WorkflowAgentComposerImpactApi(Resource):
)
return dump_response(
AgentComposerImpactResponse,
AgentComposerService.calculate_impact(
session=session, tenant_id=tenant_id, current_snapshot_id=current_snapshot_id
),
AgentComposerService.calculate_impact(tenant_id=tenant_id, current_snapshot_id=current_snapshot_id),
)
@@ -230,17 +171,14 @@ class WorkflowAgentComposerSaveToRosterApi(Resource):
@account_initialization_required
@edit_permission_required
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_EDIT)
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.AGENT_MANAGE, resource_required=False)
@get_app_model(mode=[AppMode.WORKFLOW, AppMode.ADVANCED_CHAT])
@with_current_user_id
@with_current_tenant_id
@with_session
@get_app_model(mode=[AppMode.WORKFLOW, AppMode.ADVANCED_CHAT])
def post(self, session: Session, tenant_id: str, account_id: str, app_model: App, node_id: str):
def post(self, tenant_id: str, account_id: str, app_model: App, node_id: str):
payload = ComposerSavePayload.model_validate(console_ns.payload or {})
return dump_response(
WorkflowAgentComposerResponse,
AgentComposerService.save_workflow_composer(
session=session,
tenant_id=tenant_id,
app_id=app_model.id,
node_id=node_id,
@@ -250,215 +188,6 @@ class WorkflowAgentComposerSaveToRosterApi(Resource):
)
def _require_snippet_app_id(*, session: Session, tenant_id: str, snippet_id: UUID) -> str:
snippet = SnippetService(session=session).get_snippet_by_id(
snippet_id=str(snippet_id),
tenant_id=tenant_id,
)
if snippet is None:
raise NotFound("Snippet not found")
return snippet.id
@console_ns.route("/snippets/<uuid:snippet_id>/workflows/draft/nodes/<string:node_id>/agent-composer")
class SnippetAgentComposerApi(Resource):
@console_ns.response(200, "Snippet agent composer state", console_ns.models[WorkflowAgentComposerResponse.__name__])
@console_ns.doc(params=query_params_from_model(WorkflowAgentComposerQuery))
@setup_required
@login_required
@account_initialization_required
@with_current_user_id
@with_current_tenant_id
@with_session
def get(self, session: Session, tenant_id: str, account_id: str, snippet_id: UUID, node_id: str):
query = WorkflowAgentComposerQuery.model_validate(request.args.to_dict(flat=True))
return dump_response(
WorkflowAgentComposerResponse,
AgentComposerService.load_workflow_composer(
session=session,
tenant_id=tenant_id,
app_id=_require_snippet_app_id(session=session, tenant_id=tenant_id, snippet_id=snippet_id),
node_id=node_id,
account_id=account_id,
snapshot_id=query.snapshot_id,
),
)
@console_ns.expect(console_ns.models[ComposerSavePayload.__name__])
@console_ns.response(200, "Snippet agent composer saved", console_ns.models[WorkflowAgentComposerResponse.__name__])
@setup_required
@login_required
@account_initialization_required
@edit_permission_required
@rbac_permission_required(
RBACResourceScope.WORKSPACE, RBACPermission.SNIPPETS_CREATE_AND_MODIFY, resource_required=False
)
@with_current_user_id
@with_current_tenant_id
@with_session
def put(self, session: Session, tenant_id: str, account_id: str, snippet_id: UUID, node_id: str):
payload = ComposerSavePayload.model_validate(console_ns.payload or {})
return dump_response(
WorkflowAgentComposerResponse,
AgentComposerService.save_workflow_composer(
session=session,
tenant_id=tenant_id,
app_id=_require_snippet_app_id(session=session, tenant_id=tenant_id, snippet_id=snippet_id),
node_id=node_id,
account_id=account_id,
payload=payload,
),
)
@console_ns.route("/snippets/<uuid:snippet_id>/workflows/draft/nodes/<string:node_id>/agent-composer/copy-from-roster")
class SnippetAgentComposerCopyFromRosterApi(Resource):
@console_ns.expect(console_ns.models[WorkflowComposerCopyFromRosterPayload.__name__])
@console_ns.response(
200, "Roster agent copied into snippet", console_ns.models[WorkflowAgentComposerResponse.__name__]
)
@setup_required
@login_required
@account_initialization_required
@edit_permission_required
@rbac_permission_required(
RBACResourceScope.WORKSPACE, RBACPermission.SNIPPETS_CREATE_AND_MODIFY, resource_required=False
)
@with_current_user_id
@with_current_tenant_id
@with_session
def post(self, session: Session, tenant_id: str, account_id: str, snippet_id: UUID, node_id: str):
payload = WorkflowComposerCopyFromRosterPayload.model_validate(console_ns.payload or {})
return dump_response(
WorkflowAgentComposerResponse,
AgentComposerService.copy_workflow_composer_from_roster(
session=session,
tenant_id=tenant_id,
app_id=_require_snippet_app_id(session=session, tenant_id=tenant_id, snippet_id=snippet_id),
node_id=node_id,
account_id=account_id,
source_agent_id=payload.source_agent_id,
source_snapshot_id=payload.source_snapshot_id,
idempotency_key=payload.idempotency_key,
),
)
@console_ns.route("/snippets/<uuid:snippet_id>/workflows/draft/nodes/<string:node_id>/agent-composer/validate")
class SnippetAgentComposerValidateApi(Resource):
@console_ns.expect(console_ns.models[ComposerSavePayload.__name__])
@console_ns.response(
200, "Snippet agent composer validation", console_ns.models[AgentComposerValidateResponse.__name__]
)
@setup_required
@login_required
@account_initialization_required
@with_current_tenant_id
@with_session(write=False)
def post(self, session: Session, tenant_id: str, snippet_id: UUID, node_id: str):
app_id = _require_snippet_app_id(session=session, tenant_id=tenant_id, snippet_id=snippet_id)
payload = ComposerSavePayload.model_validate(console_ns.payload or {})
ComposerConfigValidator.validate_publish_payload(payload)
AgentComposerService.validate_knowledge_datasets(
session=session, tenant_id=tenant_id, agent_soul=payload.agent_soul
)
findings = AgentComposerService.collect_validation_findings(
session=session,
tenant_id=tenant_id,
payload=payload,
agent_id=AgentComposerService.resolve_workflow_node_agent_id(
session=session,
tenant_id=tenant_id,
app_id=app_id,
node_id=node_id,
),
)
return dump_response(AgentComposerValidateResponse, {"result": "success", "errors": [], **findings})
@console_ns.route("/snippets/<uuid:snippet_id>/workflows/draft/nodes/<string:node_id>/agent-composer/candidates")
class SnippetAgentComposerCandidatesApi(Resource):
@console_ns.response(
200, "Snippet agent composer candidates", console_ns.models[AgentComposerCandidatesResponse.__name__]
)
@setup_required
@login_required
@account_initialization_required
@with_current_user_id
@with_current_tenant_id
@with_session(write=False)
def get(self, session: Session, tenant_id: str, current_user_id: str, snippet_id: UUID, node_id: str):
return dump_response(
AgentComposerCandidatesResponse,
AgentComposerService.get_workflow_candidates(
session=session,
tenant_id=tenant_id,
app_id=_require_snippet_app_id(session=session, tenant_id=tenant_id, snippet_id=snippet_id),
node_id=node_id,
user_id=current_user_id,
),
)
@console_ns.route("/snippets/<uuid:snippet_id>/workflows/draft/nodes/<string:node_id>/agent-composer/impact")
class SnippetAgentComposerImpactApi(Resource):
@console_ns.expect(console_ns.models[ComposerSavePayload.__name__])
@console_ns.response(200, "Snippet agent composer impact", console_ns.models[AgentComposerImpactResponse.__name__])
@setup_required
@login_required
@account_initialization_required
@with_current_tenant_id
@with_session(write=False)
def post(self, session: Session, tenant_id: str, snippet_id: UUID, node_id: str):
_require_snippet_app_id(session=session, tenant_id=tenant_id, snippet_id=snippet_id)
payload = ComposerSavePayload.model_validate(console_ns.payload or {})
current_snapshot_id = payload.binding.current_snapshot_id if payload.binding else None
if not current_snapshot_id:
return dump_response(
AgentComposerImpactResponse, {"current_snapshot_id": None, "workflow_node_count": 0, "bindings": []}
)
return dump_response(
AgentComposerImpactResponse,
AgentComposerService.calculate_impact(
session=session,
tenant_id=tenant_id,
current_snapshot_id=current_snapshot_id,
),
)
@console_ns.route("/snippets/<uuid:snippet_id>/workflows/draft/nodes/<string:node_id>/agent-composer/save-to-roster")
class SnippetAgentComposerSaveToRosterApi(Resource):
@console_ns.expect(console_ns.models[ComposerSavePayload.__name__])
@console_ns.response(
200, "Snippet agent saved to roster", console_ns.models[WorkflowAgentComposerResponse.__name__]
)
@setup_required
@login_required
@account_initialization_required
@edit_permission_required
@rbac_permission_required(
RBACResourceScope.WORKSPACE, RBACPermission.SNIPPETS_CREATE_AND_MODIFY, resource_required=False
)
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.AGENT_MANAGE, resource_required=False)
@with_current_user_id
@with_current_tenant_id
@with_session
def post(self, session: Session, tenant_id: str, account_id: str, snippet_id: UUID, node_id: str):
payload = ComposerSavePayload.model_validate(console_ns.payload or {})
return dump_response(
WorkflowAgentComposerResponse,
AgentComposerService.save_workflow_composer(
session=session,
tenant_id=tenant_id,
app_id=_require_snippet_app_id(session=session, tenant_id=tenant_id, snippet_id=snippet_id),
node_id=node_id,
account_id=account_id,
payload=payload,
),
)
@console_ns.route("/agent/<uuid:agent_id>/composer")
class AgentComposerApi(Resource):
@console_ns.response(200, "Agent app composer state", console_ns.models[AgentAppComposerResponse.__name__])
@@ -466,11 +195,11 @@ class AgentComposerApi(Resource):
@login_required
@account_initialization_required
@with_current_tenant_id
@with_session
def get(self, session: Session, tenant_id: str, agent_id: UUID):
def get(self, tenant_id: str, agent_id: UUID):
app_id = _resolve_agent_app_id(tenant_id=tenant_id, agent_id=agent_id)
return dump_response(
AgentAppComposerResponse,
AgentComposerService.load_agent_composer(session=session, tenant_id=tenant_id, agent_id=str(agent_id)),
AgentComposerService.load_agent_app_composer(tenant_id=tenant_id, app_id=app_id),
)
@console_ns.expect(console_ns.models[ComposerSavePayload.__name__])
@@ -480,18 +209,16 @@ class AgentComposerApi(Resource):
@account_initialization_required
@edit_permission_required
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_EDIT)
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.AGENT_MANAGE, resource_required=False)
@with_current_user_id
@with_current_tenant_id
@with_session
def put(self, session: Session, tenant_id: str, account_id: str, agent_id: UUID):
def put(self, tenant_id: str, account_id: str, agent_id: UUID):
app_id = _resolve_agent_app_id(tenant_id=tenant_id, agent_id=agent_id)
payload = ComposerSavePayload.model_validate(console_ns.payload or {})
return dump_response(
AgentAppComposerResponse,
AgentComposerService.save_agent_composer(
session=session,
AgentComposerService.save_agent_app_composer(
tenant_id=tenant_id,
agent_id=str(agent_id),
app_id=app_id,
account_id=account_id,
payload=payload,
),
@@ -508,16 +235,11 @@ class AgentComposerValidateApi(Resource):
@login_required
@account_initialization_required
@with_current_tenant_id
@with_session
def post(self, session: Session, tenant_id: str, agent_id: UUID):
AgentComposerService.load_agent_composer(session=session, tenant_id=tenant_id, agent_id=str(agent_id))
def post(self, tenant_id: str, agent_id: UUID):
_resolve_agent_app_id(tenant_id=tenant_id, agent_id=agent_id)
payload = ComposerSavePayload.model_validate(console_ns.payload or {})
ComposerConfigValidator.validate_publish_payload(payload)
AgentComposerService.validate_knowledge_datasets(
session=session, tenant_id=tenant_id, agent_soul=payload.agent_soul
)
ComposerConfigValidator.validate_save_payload(payload)
findings = AgentComposerService.collect_validation_findings(
session=session,
tenant_id=tenant_id,
payload=payload,
agent_id=str(agent_id),
@@ -535,14 +257,13 @@ class AgentComposerCandidatesApi(Resource):
@account_initialization_required
@with_current_user_id
@with_current_tenant_id
@with_session(write=False)
def get(self, session: Session, tenant_id: str, current_user_id: str, agent_id: UUID):
def get(self, tenant_id: str, current_user_id: str, agent_id: UUID):
app_id = _resolve_agent_app_id(tenant_id=tenant_id, agent_id=agent_id)
return dump_response(
AgentComposerCandidatesResponse,
AgentComposerService.get_agent_app_candidates(
session=session,
tenant_id=tenant_id,
agent_id=str(agent_id),
app_id=app_id,
user_id=current_user_id,
),
)
File diff suppressed because it is too large Load Diff
+38 -97
View File
@@ -6,13 +6,12 @@ from flask_restx import Resource
from flask_restx._http import HTTPStatus
from pydantic import field_validator
from sqlalchemy import delete, func, select
from sqlalchemy.orm import Session
from sqlalchemy.orm import sessionmaker
from werkzeug.exceptions import Forbidden
from configs import dify_config
from controllers.common.schema import register_response_schema_models
from controllers.common.session import with_session
from controllers.console.app.wraps import agent_manage_required_for_agent_app
from extensions.ext_database import db
from fields.base import ResponseModel
from libs.helper import dump_response, to_timestamp
from libs.login import login_required
@@ -55,10 +54,11 @@ class ApiKeyList(ResponseModel):
register_response_schema_models(console_ns, ApiKeyItem, ApiKeyList)
def _get_resource(resource_id, tenant_id, resource_model, *, session: Session):
resource = session.execute(
select(resource_model).filter_by(id=resource_id, tenant_id=tenant_id)
).scalar_one_or_none()
def _get_resource(resource_id, tenant_id, resource_model):
with sessionmaker(db.engine).begin() as session:
resource = session.execute(
select(resource_model).filter_by(id=resource_id, tenant_id=tenant_id)
).scalar_one_or_none()
if resource is None:
flask_restx.abort(HTTPStatus.NOT_FOUND, message=f"{resource_model.__name__} not found.")
@@ -75,18 +75,14 @@ class BaseApiKeyListResource(Resource):
token_prefix: str | None = None
max_keys = 10
@with_session(write=False)
def get(self, session: Session, resource_id: str, current_tenant_id: str) -> dict[str, object]:
return dump_response(
ApiKeyList,
self._get_api_key_list(resource_id, current_tenant_id, session=session),
)
def get(self, resource_id: str, current_tenant_id: str) -> dict[str, object]:
return dump_response(ApiKeyList, self._get_api_key_list(resource_id, current_tenant_id))
def _get_api_key_list(self, resource_id: str, current_tenant_id: str, *, session: Session) -> ApiKeyList:
def _get_api_key_list(self, resource_id: str, current_tenant_id: str) -> ApiKeyList:
assert self.resource_id_field is not None, "resource_id_field must be set"
_get_resource(resource_id, current_tenant_id, self.resource_model, session=session)
keys = session.scalars(
_get_resource(resource_id, current_tenant_id, self.resource_model)
keys = db.session.scalars(
select(ApiToken).where(
ApiToken.type == self.resource_type, getattr(ApiToken, self.resource_id_field) == resource_id
)
@@ -94,18 +90,14 @@ class BaseApiKeyListResource(Resource):
return ApiKeyList.model_validate({"data": keys}, from_attributes=True)
@edit_permission_required
@with_session
def post(self, session: Session, resource_id: str, current_tenant_id: str) -> tuple[dict[str, object], int]:
return dump_response(
ApiKeyItem,
self._create_api_key(resource_id, current_tenant_id, session=session),
), 201
def post(self, resource_id: str, current_tenant_id: str) -> tuple[dict[str, object], int]:
return dump_response(ApiKeyItem, self._create_api_key(resource_id, current_tenant_id)), 201
def _create_api_key(self, resource_id: str, current_tenant_id: str, *, session: Session) -> ApiToken:
def _create_api_key(self, resource_id: str, current_tenant_id: str) -> ApiToken:
assert self.resource_id_field is not None, "resource_id_field must be set"
_get_resource(resource_id, current_tenant_id, self.resource_model, session=session)
_get_resource(resource_id, current_tenant_id, self.resource_model)
current_key_count: int = (
session.scalar(
db.session.scalar(
select(func.count(ApiToken.id)).where(
ApiToken.type == self.resource_type, getattr(ApiToken, self.resource_id_field) == resource_id
)
@@ -120,15 +112,15 @@ class BaseApiKeyListResource(Resource):
custom="max_keys_exceeded",
)
key = ApiToken.generate_api_key(self.token_prefix or "", 24, session=session)
key = ApiToken.generate_api_key(self.token_prefix or "", 24)
assert self.resource_type is not None, "resource_type must be set"
api_token = ApiToken()
setattr(api_token, self.resource_id_field, resource_id)
api_token.tenant_id = current_tenant_id
api_token.token = key
api_token.type = self.resource_type
session.add(api_token)
session.commit()
db.session.add(api_token)
db.session.commit()
return api_token
@@ -139,16 +131,10 @@ class BaseApiKeyResource(Resource):
resource_model: type | None = None
resource_id_field: str | None = None
@with_session
def delete(
self,
session: Session,
resource_id: str,
api_key_id: str,
current_tenant_id: str,
current_user: Account,
self, resource_id: str, api_key_id: str, current_tenant_id: str, current_user: Account
) -> tuple[str, int]:
self._delete_api_key(resource_id, api_key_id, current_tenant_id, current_user, session=session)
self._delete_api_key(resource_id, api_key_id, current_tenant_id, current_user)
return "", 204
def _delete_api_key(
@@ -157,16 +143,14 @@ class BaseApiKeyResource(Resource):
api_key_id: str,
current_tenant_id: str,
current_user: Account,
*,
session: Session,
) -> None:
assert self.resource_id_field is not None, "resource_id_field must be set"
_get_resource(resource_id, current_tenant_id, self.resource_model, session=session)
_get_resource(resource_id, current_tenant_id, self.resource_model)
if not dify_config.RBAC_ENABLED and not current_user.is_admin_or_owner:
raise Forbidden()
key = session.scalar(
key = db.session.scalar(
select(ApiToken)
.where(
getattr(ApiToken, self.resource_id_field) == resource_id,
@@ -184,8 +168,8 @@ class BaseApiKeyResource(Resource):
assert key is not None # nosec - for type checker only
ApiTokenCache.delete(key.token, key.type)
session.execute(delete(ApiToken).where(ApiToken.id == api_key_id))
session.commit()
db.session.execute(delete(ApiToken).where(ApiToken.id == api_key_id))
db.session.commit()
@console_ns.route("/apps/<uuid:resource_id>/api-keys")
@@ -195,14 +179,9 @@ class AppApiKeyListResource(BaseApiKeyListResource):
@console_ns.doc(params={"resource_id": "App ID"})
@console_ns.response(200, "API keys retrieved successfully", console_ns.models[ApiKeyList.__name__])
@with_current_tenant_id
@agent_manage_required_for_agent_app
@with_session(write=False)
def get(self, session: Session, current_tenant_id: str, resource_id: UUID) -> dict[str, object]:
def get(self, current_tenant_id: str, resource_id: UUID) -> dict[str, object]:
"""Get all API keys for an app"""
return dump_response(
ApiKeyList,
self._get_api_key_list(str(resource_id), current_tenant_id, session=session),
)
return dump_response(ApiKeyList, self._get_api_key_list(str(resource_id), current_tenant_id))
@console_ns.doc("create_app_api_key")
@console_ns.doc(description="Create a new API key for an app")
@@ -212,14 +191,9 @@ class AppApiKeyListResource(BaseApiKeyListResource):
@with_current_tenant_id
@edit_permission_required
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_RELEASE_AND_VERSION)
@agent_manage_required_for_agent_app
@with_session
def post(self, session: Session, current_tenant_id: str, resource_id: UUID) -> tuple[dict[str, object], int]:
def post(self, current_tenant_id: str, resource_id: UUID) -> tuple[dict[str, object], int]:
"""Create a new API key for an app"""
return dump_response(
ApiKeyItem,
self._create_api_key(str(resource_id), current_tenant_id, session=session),
), 201
return dump_response(ApiKeyItem, self._create_api_key(str(resource_id), current_tenant_id)), 201
resource_type = ApiTokenType.APP
resource_model = App
@@ -236,24 +210,11 @@ class AppApiKeyResource(BaseApiKeyResource):
@with_current_user
@with_current_tenant_id
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_RELEASE_AND_VERSION)
@agent_manage_required_for_agent_app
@with_session
def delete(
self,
session: Session,
current_tenant_id: str,
current_user: Account,
resource_id: UUID,
api_key_id: UUID,
self, current_tenant_id: str, current_user: Account, resource_id: UUID, api_key_id: UUID
) -> tuple[str, int]:
"""Delete an API key for an app"""
self._delete_api_key(
str(resource_id),
str(api_key_id),
current_tenant_id,
current_user,
session=session,
)
self._delete_api_key(str(resource_id), str(api_key_id), current_tenant_id, current_user)
return "", 204
resource_type = ApiTokenType.APP
@@ -268,13 +229,9 @@ class DatasetApiKeyListResource(BaseApiKeyListResource):
@console_ns.doc(params={"resource_id": "Dataset ID"})
@console_ns.response(200, "API keys retrieved successfully", console_ns.models[ApiKeyList.__name__])
@with_current_tenant_id
@with_session(write=False)
def get(self, session: Session, current_tenant_id: str, resource_id: UUID) -> dict[str, object]:
def get(self, current_tenant_id: str, resource_id: UUID) -> dict[str, object]:
"""Get all API keys for a dataset"""
return dump_response(
ApiKeyList,
self._get_api_key_list(str(resource_id), current_tenant_id, session=session),
)
return dump_response(ApiKeyList, self._get_api_key_list(str(resource_id), current_tenant_id))
@console_ns.doc("create_dataset_api_key")
@console_ns.doc(description="Create a new API key for a dataset")
@@ -284,13 +241,9 @@ class DatasetApiKeyListResource(BaseApiKeyListResource):
@with_current_tenant_id
@edit_permission_required
@rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_API_KEY_MANAGE)
@with_session
def post(self, session: Session, current_tenant_id: str, resource_id: UUID) -> tuple[dict[str, object], int]:
def post(self, current_tenant_id: str, resource_id: UUID) -> tuple[dict[str, object], int]:
"""Create a new API key for a dataset"""
return dump_response(
ApiKeyItem,
self._create_api_key(str(resource_id), current_tenant_id, session=session),
), 201
return dump_response(ApiKeyItem, self._create_api_key(str(resource_id), current_tenant_id)), 201
resource_type = ApiTokenType.DATASET
resource_model = Dataset
@@ -307,23 +260,11 @@ class DatasetApiKeyResource(BaseApiKeyResource):
@with_current_user
@with_current_tenant_id
@rbac_permission_required(RBACResourceScope.DATASET, RBACPermission.DATASET_API_KEY_MANAGE)
@with_session
def delete(
self,
session: Session,
current_tenant_id: str,
current_user: Account,
resource_id: UUID,
api_key_id: UUID,
self, current_tenant_id: str, current_user: Account, resource_id: UUID, api_key_id: UUID
) -> tuple[str, int]:
"""Delete an API key for a dataset"""
self._delete_api_key(
str(resource_id),
str(api_key_id),
current_tenant_id,
current_user,
session=session,
)
self._delete_api_key(str(resource_id), str(api_key_id), current_tenant_id, current_user)
return "", 204
resource_type = ApiTokenType.DATASET
+106 -105
View File
@@ -1,3 +1,4 @@
import logging
from typing import Any
from uuid import UUID
@@ -5,7 +6,6 @@ from flask import request
from flask_restx import Resource
from pydantic import BaseModel, Field, field_validator
from sqlalchemy import select
from sqlalchemy.orm import Session
from controllers.common.schema import (
query_params_from_model,
@@ -13,9 +13,8 @@ from controllers.common.schema import (
register_response_schema_models,
register_schema_models,
)
from controllers.common.session import with_session
from controllers.console import console_ns
from controllers.console.agent.app_helpers import resolve_agent_runtime_app_model
from controllers.console.agent.app_helpers import resolve_agent_app_model
from controllers.console.app.wraps import get_app_model
from controllers.console.wraps import (
RBACPermission,
@@ -26,10 +25,12 @@ from controllers.console.wraps import (
with_current_tenant_id,
with_current_user,
)
from extensions.ext_database import db
from fields.base import ResponseModel
from libs.helper import uuid_value
from libs.login import login_required
from models import Account
from models.agent_config_entities import AgentFileRefConfig, AgentSkillRefConfig
from models.model import App, AppMode, UploadFile
from services.agent.composer_service import AgentComposerService
from services.agent.skill_package_service import SkillManifest, SkillPackageError
@@ -48,6 +49,8 @@ from services.agent_drive_service import (
)
from services.agent_service import AgentService
logger = logging.getLogger(__name__)
_WORKFLOW_AGENT_DRIVE_APP_MODES = [AppMode.WORKFLOW, AppMode.ADVANCED_CHAT]
_AGENT_SKILL_UPLOAD_PARAMS = {
"file": {
@@ -127,16 +130,8 @@ class AgentLogResponse(ResponseModel):
files: list[Any] = Field(default_factory=list)
class AgentUploadedSkillResponse(ResponseModel):
name: str
description: str
path: str
skill_md_key: str
archive_key: str | None = None
class AgentSkillUploadResponse(ResponseModel):
skill: AgentUploadedSkillResponse
skill: AgentSkillRefConfig
manifest: SkillManifest
@@ -150,11 +145,13 @@ class AgentDriveFileResponse(ResponseModel):
class AgentDriveFileCommitResponse(ResponseModel):
file: AgentDriveFileResponse
config_version_id: str | None = None
class AgentDriveDeleteResponse(ResponseModel):
result: str
removed_keys: list[str] = Field(default_factory=list)
config_version_id: str | None = None
register_schema_models(console_ns, AgentLogQuery, AgentDriveFilePayload, AgentDriveDeleteFileByAgentQuery)
@@ -164,29 +161,28 @@ register_response_schema_models(
AgentDriveFileCommitResponse,
AgentDriveFileResponse,
AgentLogResponse,
AgentUploadedSkillResponse,
AgentSkillUploadResponse,
SkillToolInferenceResult,
)
def _resolve_agent_id(session: Session, app_model: App, node_id: str | None) -> str | None:
def _resolve_agent_id(app_model: App, node_id: str | None) -> str | None:
if node_id and app_model.mode != AppMode.AGENT:
return AgentComposerService.resolve_workflow_node_agent_id(
session=session, tenant_id=app_model.tenant_id, app_id=app_model.id, node_id=node_id
tenant_id=app_model.tenant_id, app_id=app_model.id, node_id=node_id
)
return app_model.bound_agent_id_with_session(session=session)
return app_model.bound_agent_id
def _agent_not_bound() -> tuple[dict[str, str], int]:
return {"code": "agent_not_bound", "message": "no agent is bound for this app/node"}, 400
def _upload_skill_for_app(*, session: Session, current_user: Account, app_model: App):
def _upload_skill_for_app(*, current_user: Account, app_model: App):
"""Upload one skill package and commit its normalized files into the agent drive."""
query = query_params_from_request(AgentDriveMutationQuery)
agent_id = _resolve_agent_id(session, app_model, query.node_id)
agent_id = _resolve_agent_id(app_model, query.node_id)
if not agent_id:
return _agent_not_bound()
if "file" not in request.files:
@@ -203,22 +199,21 @@ def _upload_skill_for_app(*, session: Session, current_user: Account, app_model:
tenant_id=app_model.tenant_id,
user_id=current_user.id,
agent_id=agent_id,
session=session,
)
except (SkillPackageError, AgentDriveError) as exc:
return {"code": exc.code, "message": exc.message}, exc.status_code
return result, 201
def _commit_drive_file_for_app(*, session: Session, current_user: Account, app_model: App, allow_node_id: bool = True):
def _commit_drive_file_for_app(*, current_user: Account, app_model: App, allow_node_id: bool = True):
query = query_params_from_request(AgentDriveMutationQuery)
node_id = query.node_id if allow_node_id else None
agent_id = _resolve_agent_id(session, app_model, node_id)
agent_id = _resolve_agent_id(app_model, node_id)
if not agent_id:
return _agent_not_bound()
payload = AgentDriveFilePayload.model_validate(console_ns.payload or {})
upload_file = session.scalar(
upload_file = db.session.scalar(
select(UploadFile).where(
UploadFile.id == payload.upload_file_id,
UploadFile.tenant_id == app_model.tenant_id,
@@ -242,12 +237,29 @@ def _commit_drive_file_for_app(*, session: Session, current_user: Account, app_m
value_owned_by_drive=True,
)
],
session=session,
)
except AgentDriveError as exc:
return {"code": exc.code, "message": exc.message}, exc.status_code
row = committed[0]
file_ref = AgentFileRefConfig.model_validate(
{
"id": row["key"],
"name": upload_file.name,
"file_id": upload_file.id,
"drive_key": row["key"],
"type": row.get("mime_type"),
"size": row.get("size"),
}
)
config_version_id = AgentComposerService.add_drive_file_ref(
tenant_id=app_model.tenant_id,
agent_id=agent_id,
account_id=current_user.id,
file_ref=file_ref,
app_id=app_model.id,
node_id=node_id,
)
return {
"file": {
"name": upload_file.name,
@@ -256,13 +268,14 @@ def _commit_drive_file_for_app(*, session: Session, current_user: Account, app_m
"size": row.get("size"),
"mime_type": row.get("mime_type"),
},
"config_version_id": config_version_id,
}, 201
def _delete_drive_file_for_app(*, session: Session, current_user: Account, app_model: App, allow_node_id: bool = True):
def _delete_drive_file_for_app(*, current_user: Account, app_model: App, allow_node_id: bool = True):
query = query_params_from_request(AgentDriveDeleteFileQuery)
node_id = query.node_id if allow_node_id else None
agent_id = _resolve_agent_id(session, app_model, node_id)
agent_id = _resolve_agent_id(app_model, node_id)
if not agent_id:
return _agent_not_bound()
try:
@@ -270,59 +283,62 @@ def _delete_drive_file_for_app(*, session: Session, current_user: Account, app_m
except AgentDriveError as exc:
return {"code": exc.code, "message": exc.message}, exc.status_code
config_version_id = AgentComposerService.remove_drive_refs(
tenant_id=app_model.tenant_id,
agent_id=agent_id,
account_id=current_user.id,
file_key=key,
app_id=app_model.id,
node_id=node_id,
)
removed_keys: list[str] = []
try:
result = AgentDriveService().commit(
tenant_id=app_model.tenant_id,
user_id=current_user.id,
agent_id=agent_id,
items=[DriveCommitItem(key=key, file_ref=None)],
session=session,
)
removed_keys = AgentDriveService().delete(tenant_id=app_model.tenant_id, agent_id=agent_id, key=key)
except AgentDriveError as exc:
return {"code": exc.code, "message": exc.message}, exc.status_code
removed_keys = [item["key"] for item in result if item.get("removed")]
return {"result": "success", "removed_keys": removed_keys}
except Exception:
# Soul-first ordering: the ref is already gone; orphan KV rows are
# harmless and an idempotent DELETE retry cleans them.
logger.exception("agent drive delete failed for key %s (soul already updated)", key)
return {"result": "success", "removed_keys": removed_keys, "config_version_id": config_version_id}
def _delete_skill_for_app(
*, session: Session, current_user: Account, app_model: App, slug: str, allow_node_id: bool = True
):
def _delete_skill_for_app(*, current_user: Account, app_model: App, slug: str, allow_node_id: bool = True):
query = query_params_from_request(AgentDriveMutationQuery)
node_id = query.node_id if allow_node_id else None
agent_id = _resolve_agent_id(session, app_model, node_id)
agent_id = _resolve_agent_id(app_model, node_id)
if not agent_id:
return _agent_not_bound()
if "/" in slug or not slug.strip():
return {"code": "drive_key_invalid", "message": "skill slug must be a single path segment"}, 400
config_version_id = AgentComposerService.remove_drive_refs(
tenant_id=app_model.tenant_id,
agent_id=agent_id,
account_id=current_user.id,
skill_slug=slug,
app_id=app_model.id,
node_id=node_id,
)
removed_keys: list[str] = []
try:
result = AgentDriveService().commit(
tenant_id=app_model.tenant_id,
user_id=current_user.id,
agent_id=agent_id,
items=[
DriveCommitItem(key=f"{slug}/SKILL.md", file_ref=None),
DriveCommitItem(key=f"{slug}/.DIFY-SKILL-FULL.zip", file_ref=None),
],
session=session,
)
removed_keys = AgentDriveService().delete(tenant_id=app_model.tenant_id, agent_id=agent_id, prefix=f"{slug}/")
except AgentDriveError as exc:
return {"code": exc.code, "message": exc.message}, exc.status_code
removed_keys = [item["key"] for item in result if item.get("removed")]
return {"result": "success", "removed_keys": removed_keys}
except Exception:
logger.exception("agent drive delete failed for skill %s (soul already updated)", slug)
return {"result": "success", "removed_keys": removed_keys, "config_version_id": config_version_id}
def _infer_skill_tools_for_app(*, session: Session, app_model: App, slug: str):
def _infer_skill_tools_for_app(*, app_model: App, slug: str):
query = query_params_from_request(AgentDriveMutationQuery)
agent_id = _resolve_agent_id(session, app_model, query.node_id)
agent_id = _resolve_agent_id(app_model, query.node_id)
if not agent_id:
return _agent_not_bound()
if "/" in slug or not slug.strip():
return {"code": "drive_key_invalid", "message": "skill slug must be a single path segment"}, 400
try:
return SkillToolInferenceService().infer(
tenant_id=app_model.tenant_id, agent_id=agent_id, slug=slug, session=session
)
return SkillToolInferenceService().infer(tenant_id=app_model.tenant_id, agent_id=agent_id, slug=slug)
except SkillToolInferenceError as exc:
return {"code": exc.code, "message": exc.message}, exc.status_code
@@ -339,13 +355,12 @@ class AgentLogApi(Resource):
@login_required
@account_initialization_required
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_VIEW_LAYOUT)
@with_session(write=False)
@get_app_model(mode=[AppMode.AGENT_CHAT])
def get(self, session: Session, app_model: App):
def get(self, app_model: App):
"""Get agent logs"""
args = AgentLogQuery.model_validate(request.args.to_dict(flat=True))
return AgentService.get_agent_logs(app_model, args.conversation_id, args.message_id, session)
return AgentService.get_agent_logs(app_model, args.conversation_id, args.message_id)
@console_ns.route("/agent/<uuid:agent_id>/skills/upload")
@@ -360,10 +375,9 @@ class AgentSkillUploadByAgentApi(Resource):
@account_initialization_required
@with_current_user
@with_current_tenant_id
@with_session
def post(self, session: Session, tenant_id: str, current_user: Account, agent_id: UUID):
app_model = resolve_agent_runtime_app_model(session=session, tenant_id=tenant_id, agent_id=agent_id)
return _upload_skill_for_app(session=session, current_user=current_user, app_model=app_model)
def post(self, tenant_id: str, current_user: Account, agent_id: UUID):
app_model = resolve_agent_app_model(tenant_id=tenant_id, agent_id=agent_id)
return _upload_skill_for_app(current_user=current_user, app_model=app_model)
@console_ns.route("/apps/<uuid:app_id>/agent/skills/upload")
@@ -383,12 +397,11 @@ class AgentSkillUploadApi(Resource):
@setup_required
@login_required
@account_initialization_required
@with_current_user
@with_session
@get_app_model(mode=_WORKFLOW_AGENT_DRIVE_APP_MODES)
def post(self, session: Session, current_user: Account, app_model: App):
@with_current_user
def post(self, current_user: Account, app_model: App):
"""Upload a Skill, validate it, and commit drive-backed skill files."""
return _upload_skill_for_app(session=session, current_user=current_user, app_model=app_model)
return _upload_skill_for_app(current_user=current_user, app_model=app_model)
@console_ns.route("/agent/<uuid:agent_id>/files")
@@ -405,12 +418,9 @@ class AgentDriveFilesByAgentApi(Resource):
@account_initialization_required
@with_current_user
@with_current_tenant_id
@with_session
def post(self, session: Session, tenant_id: str, current_user: Account, agent_id: UUID):
app_model = resolve_agent_runtime_app_model(session=session, tenant_id=tenant_id, agent_id=agent_id)
return _commit_drive_file_for_app(
session=session, current_user=current_user, app_model=app_model, allow_node_id=False
)
def post(self, tenant_id: str, current_user: Account, agent_id: UUID):
app_model = resolve_agent_app_model(tenant_id=tenant_id, agent_id=agent_id)
return _commit_drive_file_for_app(current_user=current_user, app_model=app_model, allow_node_id=False)
@console_ns.doc("delete_agent_drive_file_by_agent")
@console_ns.doc(description="Delete one Agent App drive file by key")
@@ -421,12 +431,9 @@ class AgentDriveFilesByAgentApi(Resource):
@account_initialization_required
@with_current_user
@with_current_tenant_id
@with_session
def delete(self, session: Session, tenant_id: str, current_user: Account, agent_id: UUID):
app_model = resolve_agent_runtime_app_model(session=session, tenant_id=tenant_id, agent_id=agent_id)
return _delete_drive_file_for_app(
session=session, current_user=current_user, app_model=app_model, allow_node_id=False
)
def delete(self, tenant_id: str, current_user: Account, agent_id: UUID):
app_model = resolve_agent_app_model(tenant_id=tenant_id, agent_id=agent_id)
return _delete_drive_file_for_app(current_user=current_user, app_model=app_model, allow_node_id=False)
@console_ns.route("/apps/<uuid:app_id>/agent/files")
@@ -441,25 +448,23 @@ class AgentDriveFilesApi(Resource):
@setup_required
@login_required
@account_initialization_required
@with_current_user
@with_session
@get_app_model(mode=_WORKFLOW_AGENT_DRIVE_APP_MODES)
def post(self, session: Session, current_user: Account, app_model: App):
@with_current_user
def post(self, current_user: Account, app_model: App):
"""ADD FILE: commit one uploaded file into the bound agent's drive."""
return _commit_drive_file_for_app(session=session, current_user=current_user, app_model=app_model)
return _commit_drive_file_for_app(current_user=current_user, app_model=app_model)
@console_ns.doc("delete_agent_drive_file")
@console_ns.doc(description="Delete one drive file by key via drive commit-null semantics")
@console_ns.doc(description="Delete one drive file by key; soul ref first, then the KV row (ENG-625 D5)")
@console_ns.doc(params={"app_id": "Application ID", **query_params_from_model(AgentDriveDeleteFileQuery)})
@console_ns.response(200, "File removed", console_ns.models[AgentDriveDeleteResponse.__name__])
@setup_required
@login_required
@account_initialization_required
@with_current_user
@with_session
@get_app_model(mode=_WORKFLOW_AGENT_DRIVE_APP_MODES)
def delete(self, session: Session, current_user: Account, app_model: App):
return _delete_drive_file_for_app(session=session, current_user=current_user, app_model=app_model)
@with_current_user
def delete(self, current_user: Account, app_model: App):
return _delete_drive_file_for_app(current_user=current_user, app_model=app_model)
@console_ns.route("/agent/<uuid:agent_id>/skills/<string:slug>")
@@ -473,18 +478,17 @@ class AgentSkillByAgentApi(Resource):
@account_initialization_required
@with_current_user
@with_current_tenant_id
@with_session
def delete(self, session: Session, tenant_id: str, current_user: Account, agent_id: UUID, slug: str):
app_model = resolve_agent_runtime_app_model(session=session, tenant_id=tenant_id, agent_id=agent_id)
return _delete_skill_for_app(
session=session, current_user=current_user, app_model=app_model, slug=slug, allow_node_id=False
)
def delete(self, tenant_id: str, current_user: Account, agent_id: UUID, slug: str):
app_model = resolve_agent_app_model(tenant_id=tenant_id, agent_id=agent_id)
return _delete_skill_for_app(current_user=current_user, app_model=app_model, slug=slug, allow_node_id=False)
@console_ns.route("/apps/<uuid:app_id>/agent/skills/<string:slug>")
class AgentSkillApi(Resource):
@console_ns.doc("delete_agent_skill")
@console_ns.doc(description="Delete a standardized skill by removing its known drive keys via commit-null")
@console_ns.doc(
description="Delete a standardized skill: soul ref first, then the <slug>/ drive prefix (ENG-625 D5)"
)
@console_ns.doc(
params={
"app_id": "Application ID",
@@ -496,11 +500,10 @@ class AgentSkillApi(Resource):
@setup_required
@login_required
@account_initialization_required
@with_current_user
@with_session
@get_app_model(mode=_WORKFLOW_AGENT_DRIVE_APP_MODES)
def delete(self, session: Session, current_user: Account, app_model: App, slug: str):
return _delete_skill_for_app(session=session, current_user=current_user, app_model=app_model, slug=slug)
@with_current_user
def delete(self, current_user: Account, app_model: App, slug: str):
return _delete_skill_for_app(current_user=current_user, app_model=app_model, slug=slug)
@console_ns.route("/agent/<uuid:agent_id>/skills/<string:slug>/infer-tools")
@@ -517,10 +520,9 @@ class AgentSkillInferToolsByAgentApi(Resource):
@login_required
@account_initialization_required
@with_current_tenant_id
@with_session(write=False)
def post(self, session: Session, tenant_id: str, agent_id: UUID, slug: str):
app_model = resolve_agent_runtime_app_model(session=session, tenant_id=tenant_id, agent_id=agent_id)
return _infer_skill_tools_for_app(session=session, app_model=app_model, slug=slug)
def post(self, tenant_id: str, agent_id: UUID, slug: str):
app_model = resolve_agent_app_model(tenant_id=tenant_id, agent_id=agent_id)
return _infer_skill_tools_for_app(app_model=app_model, slug=slug)
@console_ns.route("/apps/<uuid:app_id>/agent/skills/<string:slug>/infer-tools")
@@ -544,8 +546,7 @@ class AgentSkillInferToolsApi(Resource):
@setup_required
@login_required
@account_initialization_required
@with_session(write=False)
@get_app_model(mode=_WORKFLOW_AGENT_DRIVE_APP_MODES)
def post(self, session: Session, app_model: App, slug: str):
def post(self, app_model: App, slug: str):
"""Suggest CLI tools/env for a skill. Saving still goes through composer validation."""
return _infer_skill_tools_for_app(session=session, app_model=app_model, slug=slug)
return _infer_skill_tools_for_app(app_model=app_model, slug=slug)
@@ -9,13 +9,12 @@ from uuid import UUID
from flask_restx import Resource
from pydantic import Field
from sqlalchemy.orm import Session
from controllers.common.schema import register_response_schema_models
from controllers.common.session import with_session
from controllers.console import console_ns
from controllers.console.agent.app_helpers import resolve_agent_app_model
from controllers.console.wraps import account_initialization_required, setup_required, with_current_tenant_id
from extensions.ext_database import db
from fields.base import ResponseModel
from libs.login import login_required
from services.agent.roster_service import AgentRosterService
@@ -56,10 +55,9 @@ class AgentAppReferencingWorkflowsResource(Resource):
@login_required
@account_initialization_required
@with_current_tenant_id
@with_session(write=False)
def get(self, session: Session, tenant_id: str, agent_id: UUID):
app_model = resolve_agent_app_model(session=session, tenant_id=tenant_id, agent_id=agent_id)
workflows = AgentRosterService(session).list_workflows_referencing_app_agent(
def get(self, tenant_id: str, agent_id: UUID):
app_model = resolve_agent_app_model(tenant_id=tenant_id, agent_id=agent_id)
workflows = AgentRosterService(db.session).list_workflows_referencing_app_agent(
tenant_id=tenant_id, app_id=app_model.id
)
return AgentReferencingWorkflowsResponse(
@@ -13,13 +13,11 @@ from uuid import UUID
from flask_restx import Resource
from pydantic import BaseModel, Field
from sqlalchemy.orm import Session
from controllers.common.fields import SimpleResultResponse
from controllers.common.schema import register_response_schema_models, register_schema_models
from controllers.common.session import with_session
from controllers.console import console_ns
from controllers.console.agent.app_helpers import resolve_agent_runtime_app_model
from controllers.console.agent.app_helpers import resolve_agent_app_model
from controllers.console.wraps import (
RBACPermission,
RBACResourceScope,
@@ -31,6 +29,8 @@ from controllers.console.wraps import (
with_current_user,
)
from events.app_event import app_model_config_was_updated
from extensions.ext_database import db
from libs.helper import dump_response
from libs.login import login_required
from models import Account
from models.agent_config_entities import (
@@ -86,23 +86,14 @@ class AgentAppFeatureConfigResource(Resource):
@account_initialization_required
@with_current_user
@with_current_tenant_id
@with_session
def post(self, session: Session, tenant_id: str, current_user: Account, agent_id: UUID):
app_model = resolve_agent_runtime_app_model(session=session, tenant_id=tenant_id, agent_id=agent_id)
def post(self, tenant_id: str, current_user: Account, agent_id: UUID):
app_model = resolve_agent_app_model(tenant_id=tenant_id, agent_id=agent_id)
args = AgentAppFeaturesPayload.model_validate(console_ns.payload or {})
new_app_model_config = AgentAppFeatureConfigService.update_features(
app_model=app_model,
account=current_user,
config=args.model_dump(exclude_none=True),
session=session,
app_model=app_model, account=current_user, config=args.model_dump(exclude_none=True), session=db.session
)
app_model_config_was_updated.send(
app_model,
app_model_config=new_app_model_config,
session=session,
)
session.commit()
app_model_config_was_updated.send(app_model, app_model_config=new_app_model_config)
return SimpleResultResponse(result="success").model_dump(mode="json")
return dump_response(SimpleResultResponse, {"result": "success"})
@@ -14,7 +14,6 @@ from dify_agent.client import DifyAgentClientError, DifyAgentHTTPError, DifyAgen
from flask import request
from flask_restx import Resource
from pydantic import BaseModel, Field
from sqlalchemy.orm import Session
from controllers.common.schema import (
query_params_from_model,
@@ -22,12 +21,10 @@ from controllers.common.schema import (
register_response_schema_models,
register_schema_models,
)
from controllers.common.session import with_session
from controllers.console import console_ns
from controllers.console.agent.app_helpers import resolve_agent_runtime_app_model
from controllers.console.agent.app_helpers import resolve_agent_app_model
from controllers.console.app.wraps import get_app_model
from controllers.console.wraps import account_initialization_required, setup_required, with_current_tenant_id
from extensions.ext_database import db
from fields.base import ResponseModel
from libs.login import login_required
from models.model import App, AppMode
@@ -47,10 +44,6 @@ class AgentSandboxListQuery(BaseModel):
path: str = Field(default=".", description="Directory path relative to the sandbox workspace")
class AgentSandboxInfoQuery(BaseModel):
conversation_id: str = Field(min_length=1, description="Agent App conversation ID")
class AgentSandboxFileQuery(BaseModel):
conversation_id: str = Field(min_length=1, description="Agent App conversation ID")
path: str = Field(min_length=1, description="File path relative to the sandbox workspace")
@@ -98,11 +91,6 @@ class SandboxListResponse(ResponseModel):
truncated: bool = False
class SandboxInfoResponse(ResponseModel):
session_id: str
workspace_cwd: str
class SandboxReadResponse(ResponseModel):
path: str
size: int | None = None
@@ -111,8 +99,14 @@ class SandboxReadResponse(ResponseModel):
text: str | None = None
class SandboxToolFileResponse(ResponseModel):
transfer_method: Literal["tool_file"] = "tool_file"
reference: str
class SandboxUploadResponse(ResponseModel):
url: str
path: str
file: SandboxToolFileResponse
register_schema_models(
@@ -120,13 +114,7 @@ register_schema_models(
AgentSandboxUploadPayload,
WorkflowAgentSandboxUploadPayload,
)
register_response_schema_models(
console_ns,
SandboxInfoResponse,
SandboxListResponse,
SandboxReadResponse,
SandboxUploadResponse,
)
register_response_schema_models(console_ns, SandboxListResponse, SandboxReadResponse, SandboxUploadResponse)
def _handle(exc: Exception) -> tuple[dict[str, object], int]:
@@ -145,31 +133,6 @@ def _handle(exc: Exception) -> tuple[dict[str, object], int]:
raise exc
@console_ns.route("/agent/<uuid:agent_id>/sandbox")
class AgentAppSandboxInfoResource(Resource):
@console_ns.doc("get_agent_app_sandbox_info")
@console_ns.doc(description="Get basic information for an Agent App conversation sandbox")
@console_ns.doc(params={"agent_id": "Agent ID", **query_params_from_model(AgentSandboxInfoQuery)})
@console_ns.response(200, "Sandbox information returned", console_ns.models[SandboxInfoResponse.__name__])
@setup_required
@login_required
@account_initialization_required
@with_current_tenant_id
@with_session(write=False)
def get(self, session: Session, tenant_id: str, agent_id: UUID):
app_model = resolve_agent_runtime_app_model(session=session, tenant_id=tenant_id, agent_id=agent_id)
query = query_params_from_request(AgentSandboxInfoQuery)
try:
result = AgentAppSandboxService().get_info(
tenant_id=tenant_id,
app_id=app_model.id,
conversation_id=query.conversation_id,
)
except Exception as exc:
return _handle(exc)
return result.model_dump()
@console_ns.route("/agent/<uuid:agent_id>/sandbox/files")
class AgentAppSandboxListResource(Resource):
@console_ns.doc("list_agent_app_sandbox_files")
@@ -180,9 +143,8 @@ class AgentAppSandboxListResource(Resource):
@login_required
@account_initialization_required
@with_current_tenant_id
@with_session(write=False)
def get(self, session: Session, tenant_id: str, agent_id: UUID):
app_model = resolve_agent_runtime_app_model(session=session, tenant_id=tenant_id, agent_id=agent_id)
def get(self, tenant_id: str, agent_id: UUID):
app_model = resolve_agent_app_model(tenant_id=tenant_id, agent_id=agent_id)
query = query_params_from_request(AgentSandboxListQuery)
try:
result = AgentAppSandboxService().list_files(
@@ -206,9 +168,8 @@ class AgentAppSandboxReadResource(Resource):
@login_required
@account_initialization_required
@with_current_tenant_id
@with_session(write=False)
def get(self, session: Session, tenant_id: str, agent_id: UUID):
app_model = resolve_agent_runtime_app_model(session=session, tenant_id=tenant_id, agent_id=agent_id)
def get(self, tenant_id: str, agent_id: UUID):
app_model = resolve_agent_app_model(tenant_id=tenant_id, agent_id=agent_id)
query = query_params_from_request(AgentSandboxFileQuery)
try:
result = AgentAppSandboxService().read_file(
@@ -225,16 +186,15 @@ class AgentAppSandboxReadResource(Resource):
@console_ns.route("/agent/<uuid:agent_id>/sandbox/files/upload")
class AgentAppSandboxUploadResource(Resource):
@console_ns.doc("upload_agent_app_sandbox_file")
@console_ns.doc(description="Upload one Agent App sandbox file and return a signed download URL")
@console_ns.doc(description="Upload one Agent App sandbox file as a Dify ToolFile mapping")
@console_ns.expect(console_ns.models[AgentSandboxUploadPayload.__name__])
@console_ns.response(200, "Uploaded", console_ns.models[SandboxUploadResponse.__name__])
@setup_required
@login_required
@account_initialization_required
@with_current_tenant_id
@with_session(write=False)
def post(self, session: Session, tenant_id: str, agent_id: UUID):
app_model = resolve_agent_runtime_app_model(session=session, tenant_id=tenant_id, agent_id=agent_id)
def post(self, tenant_id: str, agent_id: UUID):
app_model = resolve_agent_app_model(tenant_id=tenant_id, agent_id=agent_id)
payload = AgentSandboxUploadPayload.model_validate(request.get_json(silent=True) or {})
try:
result = AgentAppSandboxService().upload_file(
@@ -276,7 +236,6 @@ class WorkflowAgentSandboxListResource(Resource):
node_id=node_id,
node_execution_id=query.node_execution_id,
path=query.path,
session=db.session(),
)
except Exception as exc:
return _handle(exc)
@@ -313,7 +272,6 @@ class WorkflowAgentSandboxReadResource(Resource):
node_id=node_id,
node_execution_id=query.node_execution_id,
path=query.path,
session=db.session(),
)
except Exception as exc:
return _handle(exc)
@@ -325,7 +283,7 @@ class WorkflowAgentSandboxReadResource(Resource):
)
class WorkflowAgentSandboxUploadResource(Resource):
@console_ns.doc("upload_workflow_agent_sandbox_file")
@console_ns.doc(description="Upload one workflow Agent sandbox file and return a signed download URL")
@console_ns.doc(description="Upload one workflow Agent sandbox file as a Dify ToolFile mapping")
@console_ns.expect(console_ns.models[WorkflowAgentSandboxUploadPayload.__name__])
@console_ns.response(200, "Uploaded", console_ns.models[SandboxUploadResponse.__name__])
@setup_required
@@ -343,7 +301,6 @@ class WorkflowAgentSandboxUploadResource(Resource):
node_id=node_id,
node_execution_id=payload.node_execution_id,
path=payload.path,
session=db.session(),
)
except Exception as exc:
return _handle(exc)
File diff suppressed because it is too large Load Diff

Some files were not shown because too many files have changed in this diff Show More