Compare commits
10
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
b7cb4927f5 | ||
|
|
7d33794853 | ||
|
|
996148697e | ||
|
|
af852a8176 | ||
|
|
7ae49bacba | ||
|
|
e81a5ba3a8 | ||
|
|
f0df9ee016 | ||
|
|
0531ffaeba | ||
|
|
230faff872 | ||
|
|
b078c75ccf |
@@ -28,6 +28,7 @@ Follow these steps when using this skill:
|
||||
3. Compose the final output strictly follow the **Required Output Format**.
|
||||
|
||||
Notes when using this skill:
|
||||
|
||||
- Always include actionable fixes or suggestions (including possible code snippets).
|
||||
- Use best-effort `File:Line` references when a file path and line numbers are available; otherwise, use the most specific identifier you can.
|
||||
|
||||
@@ -43,6 +44,7 @@ Notes when using this skill:
|
||||
### 1. Security Review
|
||||
|
||||
Check for:
|
||||
|
||||
- SQL injection vulnerabilities
|
||||
- Server-Side Request Forgery (SSRF)
|
||||
- Command injection
|
||||
@@ -54,6 +56,7 @@ Check for:
|
||||
### 2. Performance Review
|
||||
|
||||
Check for:
|
||||
|
||||
- N+1 queries
|
||||
- Missing database indexes
|
||||
- Memory leaks
|
||||
@@ -63,6 +66,7 @@ Check for:
|
||||
### 3. Code Quality Review
|
||||
|
||||
Check for:
|
||||
|
||||
- Code forward compatibility
|
||||
- Code duplication (DRY violations)
|
||||
- Functions doing too much (SRP violations)
|
||||
@@ -75,6 +79,7 @@ Check for:
|
||||
### 4. Testing Review
|
||||
|
||||
Check for:
|
||||
|
||||
- Missing test coverage for new code
|
||||
- Tests that don't test behavior
|
||||
- Flaky test patterns
|
||||
@@ -108,6 +113,7 @@ FilePath: <path> line <line>
|
||||
2. <code example> (optional, omit if not applicable)
|
||||
|
||||
---
|
||||
|
||||
... (repeat for each critical issue) ...
|
||||
|
||||
Found <Y> suggestions for improvement:
|
||||
@@ -129,11 +135,13 @@ FilePath: <path> line <line>
|
||||
2. <code example> (optional, omit if not applicable)
|
||||
|
||||
---
|
||||
|
||||
... (repeat for each suggestion) ...
|
||||
|
||||
Found <Z> optional nits:
|
||||
|
||||
## 🟢 Nits (Optional)
|
||||
|
||||
### 1. <brief description of the nit>
|
||||
|
||||
FilePath: <path> line <line>
|
||||
@@ -148,6 +156,7 @@ FilePath: <path> line <line>
|
||||
- <minor suggestions>
|
||||
|
||||
---
|
||||
|
||||
... (repeat for each nits) ...
|
||||
|
||||
## ✅ What's Good
|
||||
@@ -164,5 +173,6 @@ FilePath: <path> line <line>
|
||||
|
||||
```markdown
|
||||
## Code Review Summary
|
||||
|
||||
✅ No issues found.
|
||||
```
|
||||
```
|
||||
|
||||
@@ -1,11 +1,13 @@
|
||||
# Rule Catalog — Architecture
|
||||
|
||||
## Scope
|
||||
|
||||
- Covers: controller/service/core-domain/libs/model layering, dependency direction, responsibility placement, observability-friendly flow.
|
||||
|
||||
## Rules
|
||||
|
||||
### Keep business logic out of controllers
|
||||
|
||||
- Category: maintainability
|
||||
- Severity: critical
|
||||
- Description: Controllers should parse input, call services, and return serialized responses. Business decisions inside controllers make behavior hard to reuse and test.
|
||||
@@ -33,12 +35,14 @@
|
||||
```
|
||||
|
||||
### Preserve layer dependency direction
|
||||
|
||||
- Category: best practices
|
||||
- Severity: critical
|
||||
- Description: Controllers may depend on services, and services may depend on core/domain abstractions. Reversing this direction (for example, core importing controller/web modules) creates cycles and leaks transport concerns into domain code.
|
||||
- Suggested fix: Extract shared contracts into core/domain or service-level modules and make upper layers depend on lower, not the reverse.
|
||||
- Example:
|
||||
- Bad:
|
||||
|
||||
```python
|
||||
# core/policy/publish_policy.py
|
||||
from controllers.console.app import request_context
|
||||
@@ -46,7 +50,9 @@
|
||||
def can_publish() -> bool:
|
||||
return request_context.current_user.is_admin
|
||||
```
|
||||
|
||||
- Good:
|
||||
|
||||
```python
|
||||
# core/policy/publish_policy.py
|
||||
def can_publish(role: str) -> bool:
|
||||
@@ -57,6 +63,7 @@
|
||||
```
|
||||
|
||||
### Keep libs business-agnostic
|
||||
|
||||
- Category: maintainability
|
||||
- Severity: critical
|
||||
- Description: Modules under `api/libs/` should remain reusable, business-agnostic building blocks. They must not encode product/domain-specific rules, workflow orchestration, or business decisions.
|
||||
@@ -65,6 +72,7 @@
|
||||
- Keep `libs` dependencies clean: avoid importing service/controller/domain-specific modules into `api/libs/`.
|
||||
- Example:
|
||||
- Bad:
|
||||
|
||||
```python
|
||||
# api/libs/conversation_filter.py
|
||||
from services.conversation_service import ConversationService
|
||||
@@ -76,7 +84,9 @@
|
||||
return conversation.idle_days > 90
|
||||
return conversation.idle_days > 30
|
||||
```
|
||||
|
||||
- Good:
|
||||
|
||||
```python
|
||||
# api/libs/datetime_utils.py (business-agnostic helper)
|
||||
def older_than_days(idle_days: int, threshold_days: int) -> bool:
|
||||
@@ -88,4 +98,4 @@
|
||||
def should_archive_conversation(conversation, tenant_id: str) -> bool:
|
||||
threshold_days = 90 if has_paid_plan(tenant_id) else 30
|
||||
return older_than_days(conversation.idle_days, threshold_days)
|
||||
```
|
||||
```
|
||||
|
||||
@@ -1,12 +1,14 @@
|
||||
# Rule Catalog — DB Schema Design
|
||||
|
||||
## Scope
|
||||
|
||||
- Covers: model/base inheritance, schema boundaries in model properties, tenant-aware schema design, index redundancy checks, dialect portability in models, and cross-database compatibility in migrations.
|
||||
- Does NOT cover: session lifecycle, transaction boundaries, and query execution patterns (handled by `sqlalchemy-rule.md`).
|
||||
|
||||
## Rules
|
||||
|
||||
### Do not query other tables inside `@property`
|
||||
|
||||
- Category: [maintainability, performance]
|
||||
- Severity: critical
|
||||
- Description: A model `@property` must not open sessions or query other tables. This hides dependencies across models, tightly couples schema objects to data access, and can cause N+1 query explosions when iterating collections.
|
||||
@@ -16,6 +18,7 @@
|
||||
- For list/batch reads, fetch required related data explicitly (join/preload/bulk query) before rendering derived values.
|
||||
- Example:
|
||||
- Bad:
|
||||
|
||||
```python
|
||||
class Conversation(TypeBase):
|
||||
__tablename__ = "conversations"
|
||||
@@ -26,7 +29,9 @@
|
||||
app = session.execute(select(App).where(App.id == self.app_id)).scalar_one()
|
||||
return app.name
|
||||
```
|
||||
|
||||
- Good:
|
||||
|
||||
```python
|
||||
class Conversation(TypeBase):
|
||||
__tablename__ = "conversations"
|
||||
@@ -40,6 +45,7 @@
|
||||
```
|
||||
|
||||
### Prefer including `tenant_id` in model definitions
|
||||
|
||||
- Category: maintainability
|
||||
- Severity: suggestion
|
||||
- Description: In multi-tenant domains, include `tenant_id` in schema definitions whenever the entity belongs to tenant-owned data. This improves data isolation safety and keeps future partitioning/sharding strategies practical as data volume grows.
|
||||
@@ -49,6 +55,7 @@
|
||||
- Exception: if a table is explicitly designed as non-tenant-scoped global metadata, document that design decision clearly.
|
||||
- Example:
|
||||
- Bad:
|
||||
|
||||
```python
|
||||
from sqlalchemy.orm import Mapped
|
||||
|
||||
@@ -57,7 +64,9 @@
|
||||
id: Mapped[str] = mapped_column(StringUUID, primary_key=True)
|
||||
name: Mapped[str] = mapped_column(sa.String(255), nullable=False)
|
||||
```
|
||||
|
||||
- Good:
|
||||
|
||||
```python
|
||||
from sqlalchemy.orm import Mapped
|
||||
|
||||
@@ -69,6 +78,7 @@
|
||||
```
|
||||
|
||||
### Detect and avoid duplicate/redundant indexes
|
||||
|
||||
- Category: performance
|
||||
- Severity: suggestion
|
||||
- Description: Review index definitions for leftmost-prefix redundancy. For example, index `(a, b, c)` can safely cover most lookups for `(a, b)`. Keeping both may increase write overhead and can mislead the optimizer into suboptimal execution plans.
|
||||
@@ -93,6 +103,7 @@
|
||||
```
|
||||
|
||||
### Avoid PostgreSQL-only dialect usage in models; wrap in `models.types`
|
||||
|
||||
- Category: maintainability
|
||||
- Severity: critical
|
||||
- Description: Model/schema definitions should avoid PostgreSQL-only constructs directly in business models. When database-specific behavior is required, encapsulate it in `api/models/types.py` using both PostgreSQL and MySQL dialect implementations, then consume that abstraction from model code.
|
||||
@@ -101,6 +112,7 @@
|
||||
- Add or extend wrappers in `models.types` (for example, `AdjustedJSON`, `LongText`, `BinaryData`) to normalize behavior across PostgreSQL and MySQL.
|
||||
- Example:
|
||||
- Bad:
|
||||
|
||||
```python
|
||||
from sqlalchemy.dialects.postgresql import JSONB
|
||||
from sqlalchemy.orm import Mapped
|
||||
@@ -109,7 +121,9 @@
|
||||
__tablename__ = "tool_configs"
|
||||
config: Mapped[dict] = mapped_column(JSONB, nullable=False)
|
||||
```
|
||||
|
||||
- Good:
|
||||
|
||||
```python
|
||||
from sqlalchemy.orm import Mapped
|
||||
|
||||
@@ -121,6 +135,7 @@
|
||||
```
|
||||
|
||||
### Guard migration incompatibilities with dialect checks and shared types
|
||||
|
||||
- Category: maintainability
|
||||
- Severity: critical
|
||||
- Description: Migration scripts under `api/migrations/versions/` must account for PostgreSQL/MySQL incompatibilities explicitly. For dialect-sensitive DDL or defaults, branch on the active dialect (for example, `conn.dialect.name == "postgresql"`), and prefer reusable compatibility abstractions from `models.types` where applicable.
|
||||
@@ -142,6 +157,7 @@
|
||||
)
|
||||
```
|
||||
- Good:
|
||||
|
||||
```python
|
||||
def _is_pg(conn) -> bool:
|
||||
return conn.dialect.name == "postgresql"
|
||||
|
||||
@@ -1,17 +1,19 @@
|
||||
# Rule Catalog - Repositories Abstraction
|
||||
|
||||
## Scope
|
||||
|
||||
- Covers: when to reuse existing repository abstractions, when to introduce new repositories, and how to preserve dependency direction between service/core and infrastructure implementations.
|
||||
- Does NOT cover: SQLAlchemy session lifecycle and query-shape specifics (handled by `sqlalchemy-rule.md`), and table schema/migration design (handled by `db-schema-rule.md`).
|
||||
|
||||
## Rules
|
||||
|
||||
### Introduce repositories abstraction
|
||||
|
||||
- Category: maintainability
|
||||
- Severity: suggestion
|
||||
- Description: If a table/model already has a repository abstraction, all reads/writes/queries for that table should use the existing repository. If no repository exists, introduce one only when complexity justifies it, such as large/high-volume tables, repeated complex query logic, or likely storage-strategy variation.
|
||||
- Suggested fix:
|
||||
- First check `api/repositories`, `api/core/repositories`, and `api/extensions/*/repositories/` to verify whether the table/model already has a repository abstraction. If it exists, route all operations through it and add missing repository methods instead of bypassing it with ad-hoc SQLAlchemy access.
|
||||
- First check `api/repositories`, `api/core/repositories`, and `api/extensions/*/repositories/` to verify whether the table/model already has a repository abstraction. If it exists, route all operations through it and add missing repository methods instead of bypassing it with ad-hoc SQLAlchemy access.
|
||||
- If no repository exists, add one only when complexity warrants it (for example, repeated complex queries, large data domains, or multiple storage strategies), while preserving dependency direction (service/core depends on abstraction; infra provides implementation).
|
||||
- Example:
|
||||
- Bad:
|
||||
@@ -26,6 +28,7 @@
|
||||
self.session.commit()
|
||||
```
|
||||
- Good:
|
||||
|
||||
```python
|
||||
# Case A: Existing repository must be reused for all table operations.
|
||||
class AppService:
|
||||
@@ -37,6 +40,7 @@
|
||||
# If the query is missing, extend the existing abstraction.
|
||||
active_apps = self.app_repo.list_active_for_tenant(tenant_id=tenant_id)
|
||||
```
|
||||
|
||||
- Bad:
|
||||
```python
|
||||
# No repository exists, but large-domain query logic is scattered in service code.
|
||||
@@ -46,6 +50,7 @@
|
||||
# many filters/joins/pagination variants duplicated across services
|
||||
```
|
||||
- Good:
|
||||
|
||||
```python
|
||||
# Case B: Introduce repository for large/complex domains or storage variation.
|
||||
class ConversationRepository(Protocol):
|
||||
|
||||
@@ -1,12 +1,14 @@
|
||||
# Rule Catalog — SQLAlchemy Patterns
|
||||
|
||||
## Scope
|
||||
|
||||
- Covers: SQLAlchemy session and transaction lifecycle, query construction, tenant scoping, raw SQL boundaries, and write-path concurrency safeguards.
|
||||
- Does NOT cover: table/model schema and migration design details (handled by `db-schema-rule.md`).
|
||||
|
||||
## Rules
|
||||
|
||||
### Use Session context manager with explicit transaction control behavior
|
||||
|
||||
- Category: best practices
|
||||
- Severity: critical
|
||||
- Description: Session and transaction lifecycle must be explicit and bounded on write paths. Missing commits can silently drop intended updates, while ad-hoc or long-lived transactions increase contention, lock duration, and deadlock risk.
|
||||
@@ -16,6 +18,7 @@
|
||||
- Keep transaction windows short: avoid network I/O, heavy computation, or unrelated work inside the transaction.
|
||||
- Example:
|
||||
- Bad:
|
||||
|
||||
```python
|
||||
# Missing commit: write may never be persisted.
|
||||
with Session(db.engine, expire_on_commit=False) as session:
|
||||
@@ -28,7 +31,9 @@
|
||||
run.status = "cancelled"
|
||||
call_external_api()
|
||||
```
|
||||
|
||||
- Good:
|
||||
|
||||
```python
|
||||
# Option 1: explicit commit.
|
||||
with Session(db.engine, expire_on_commit=False) as session:
|
||||
@@ -46,6 +51,7 @@
|
||||
```
|
||||
|
||||
### Enforce tenant_id scoping on shared-resource queries
|
||||
|
||||
- Category: security
|
||||
- Severity: critical
|
||||
- Description: Reads and writes against shared tables must be scoped by `tenant_id` to prevent cross-tenant data leakage or corruption.
|
||||
@@ -66,6 +72,7 @@
|
||||
```
|
||||
|
||||
### Prefer SQLAlchemy expressions over raw SQL by default
|
||||
|
||||
- Category: maintainability
|
||||
- Severity: suggestion
|
||||
- Description: Raw SQL should be exceptional. ORM/Core expressions are easier to evolve, safer to compose, and more consistent with the codebase.
|
||||
@@ -88,6 +95,7 @@
|
||||
```
|
||||
|
||||
### Protect write paths with concurrency safeguards
|
||||
|
||||
- Category: quality
|
||||
- Severity: critical
|
||||
- Description: Multi-writer paths without explicit concurrency control can silently overwrite data. Choose the safeguard based on contention level, lock scope, and throughput cost instead of defaulting to one strategy.
|
||||
@@ -104,6 +112,7 @@
|
||||
session.commit() # silently overwrites concurrent updates
|
||||
```
|
||||
- Good:
|
||||
|
||||
```python
|
||||
# 1) Optimistic lock (low contention, retry on conflict)
|
||||
result = session.execute(
|
||||
@@ -136,4 +145,4 @@
|
||||
).scalar_one()
|
||||
run.status = "cancelled"
|
||||
session.commit()
|
||||
```
|
||||
```
|
||||
|
||||
@@ -0,0 +1,449 @@
|
||||
---
|
||||
name: component-refactoring
|
||||
description: Refactor high-complexity React components in Dify frontend. Use when `pnpm analyze-component --json` shows complexity > 50 or lineCount > 300, when the user asks for code splitting, hook extraction, or complexity reduction, or when `pnpm analyze-component` warns to refactor before testing; avoid for simple/well-structured components, third-party wrappers, or when the user explicitly wants testing without refactoring.
|
||||
---
|
||||
|
||||
# Dify Component Refactoring Skill
|
||||
|
||||
Refactor high-complexity React components in the Dify frontend codebase with the patterns and workflow below.
|
||||
|
||||
> **Complexity Threshold**: Components with complexity > 50 (measured by `pnpm analyze-component`) should be refactored before testing.
|
||||
|
||||
## Quick Reference
|
||||
|
||||
### Commands (run from `web/`)
|
||||
|
||||
Use paths relative to `web/` (e.g., `app/components/...`).
|
||||
Use `refactor-component` for refactoring prompts and `analyze-component` for testing prompts and metrics.
|
||||
|
||||
```bash
|
||||
cd web
|
||||
|
||||
# Generate refactoring prompt
|
||||
pnpm refactor-component <path>
|
||||
|
||||
# Output refactoring analysis as JSON
|
||||
pnpm refactor-component <path> --json
|
||||
|
||||
# Generate testing prompt (after refactoring)
|
||||
pnpm analyze-component <path>
|
||||
|
||||
# Output testing analysis as JSON
|
||||
pnpm analyze-component <path> --json
|
||||
```
|
||||
|
||||
### Complexity Analysis
|
||||
|
||||
```bash
|
||||
# Analyze component complexity
|
||||
pnpm analyze-component <path> --json
|
||||
|
||||
# Key metrics to check:
|
||||
# - complexity: normalized score 0-100 (target < 50)
|
||||
# - maxComplexity: highest single function complexity
|
||||
# - lineCount: total lines (target < 300)
|
||||
```
|
||||
|
||||
### Complexity Score Interpretation
|
||||
|
||||
| Score | Level | Action |
|
||||
| ------ | --------------- | --------------------------- |
|
||||
| 0-25 | 🟢 Simple | Ready for testing |
|
||||
| 26-50 | 🟡 Medium | Consider minor refactoring |
|
||||
| 51-75 | 🟠 Complex | **Refactor before testing** |
|
||||
| 76-100 | 🔴 Very Complex | **Must refactor** |
|
||||
|
||||
## Core Refactoring Patterns
|
||||
|
||||
### Pattern 1: Extract Custom Hooks
|
||||
|
||||
**When**: Component has complex state management, multiple `useState`/`useEffect`, or business logic mixed with UI.
|
||||
|
||||
**Dify Convention**: Place hooks in a `hooks/` subdirectory or alongside the component as `use-<feature>.ts`.
|
||||
|
||||
```typescript
|
||||
// ❌ Before: Complex state logic in component
|
||||
function Configuration() {
|
||||
const [modelConfig, setModelConfig] = useState<ModelConfig>(...)
|
||||
const [datasetConfigs, setDatasetConfigs] = useState<DatasetConfigs>(...)
|
||||
const [completionParams, setCompletionParams] = useState<FormValue>({})
|
||||
|
||||
// 50+ lines of state management logic...
|
||||
|
||||
return <div>...</div>
|
||||
}
|
||||
|
||||
// ✅ After: Extract to custom hook
|
||||
// hooks/use-model-config.ts
|
||||
export const useModelConfig = (appId: string) => {
|
||||
const [modelConfig, setModelConfig] = useState<ModelConfig>(...)
|
||||
const [completionParams, setCompletionParams] = useState<FormValue>({})
|
||||
|
||||
// Related state management logic here
|
||||
|
||||
return { modelConfig, setModelConfig, completionParams, setCompletionParams }
|
||||
}
|
||||
|
||||
// Component becomes cleaner
|
||||
function Configuration() {
|
||||
const { modelConfig, setModelConfig } = useModelConfig(appId)
|
||||
return <div>...</div>
|
||||
}
|
||||
```
|
||||
|
||||
**Dify Examples**:
|
||||
|
||||
- `web/app/components/app/configuration/hooks/use-advanced-prompt-config.ts`
|
||||
- `web/app/components/app/configuration/debug/hooks.tsx`
|
||||
- `web/app/components/workflow/hooks/use-workflow.ts`
|
||||
|
||||
### Pattern 2: Extract Sub-Components
|
||||
|
||||
**When**: Single component has multiple UI sections, conditional rendering blocks, or repeated patterns.
|
||||
|
||||
**Dify Convention**: Place sub-components in subdirectories or as separate files in the same directory.
|
||||
|
||||
```typescript
|
||||
// ❌ Before: Monolithic JSX with multiple sections
|
||||
const AppInfo = () => {
|
||||
return (
|
||||
<div>
|
||||
{/* 100 lines of header UI */}
|
||||
{/* 100 lines of operations UI */}
|
||||
{/* 100 lines of modals */}
|
||||
</div>
|
||||
)
|
||||
}
|
||||
|
||||
// ✅ After: Split into focused components
|
||||
// app-info/
|
||||
// ├── index.tsx (orchestration only)
|
||||
// ├── app-header.tsx (header UI)
|
||||
// ├── app-operations.tsx (operations UI)
|
||||
// └── app-modals.tsx (modal management)
|
||||
|
||||
const AppInfo = () => {
|
||||
const { showModal, setShowModal } = useAppInfoModals()
|
||||
|
||||
return (
|
||||
<div>
|
||||
<AppHeader appDetail={appDetail} />
|
||||
<AppOperations onAction={handleAction} />
|
||||
<AppModals show={showModal} onClose={() => setShowModal(null)} />
|
||||
</div>
|
||||
)
|
||||
}
|
||||
```
|
||||
|
||||
**Dify Examples**:
|
||||
|
||||
- `web/app/components/app/configuration/` directory structure
|
||||
- `web/app/components/workflow/nodes/` per-node organization
|
||||
|
||||
### Pattern 3: Simplify Conditional Logic
|
||||
|
||||
**When**: Deep nesting (> 3 levels), complex ternaries, or multiple `if/else` chains.
|
||||
|
||||
```typescript
|
||||
// ❌ Before: Deeply nested conditionals
|
||||
const Template = useMemo(() => {
|
||||
if (appDetail?.mode === AppModeEnum.CHAT) {
|
||||
switch (locale) {
|
||||
case LanguagesSupported[1]:
|
||||
return <TemplateChatZh />
|
||||
case LanguagesSupported[7]:
|
||||
return <TemplateChatJa />
|
||||
default:
|
||||
return <TemplateChatEn />
|
||||
}
|
||||
}
|
||||
if (appDetail?.mode === AppModeEnum.ADVANCED_CHAT) {
|
||||
// Another 15 lines...
|
||||
}
|
||||
// More conditions...
|
||||
}, [appDetail, locale])
|
||||
|
||||
// ✅ After: Use lookup tables + early returns
|
||||
const TEMPLATE_MAP = {
|
||||
[AppModeEnum.CHAT]: {
|
||||
[LanguagesSupported[1]]: TemplateChatZh,
|
||||
[LanguagesSupported[7]]: TemplateChatJa,
|
||||
default: TemplateChatEn,
|
||||
},
|
||||
[AppModeEnum.ADVANCED_CHAT]: {
|
||||
[LanguagesSupported[1]]: TemplateAdvancedChatZh,
|
||||
// ...
|
||||
},
|
||||
}
|
||||
|
||||
const Template = useMemo(() => {
|
||||
const modeTemplates = TEMPLATE_MAP[appDetail?.mode]
|
||||
if (!modeTemplates) return null
|
||||
|
||||
const TemplateComponent = modeTemplates[locale] || modeTemplates.default
|
||||
return <TemplateComponent appDetail={appDetail} />
|
||||
}, [appDetail, locale])
|
||||
```
|
||||
|
||||
### Pattern 4: Extract API/Data Logic
|
||||
|
||||
**When**: Component directly handles API calls, data transformation, or complex async operations.
|
||||
|
||||
**Dify Convention**:
|
||||
|
||||
- This skill is for component decomposition, not query/mutation design.
|
||||
- Do not introduce deprecated `useInvalid` / `useReset`.
|
||||
- Do not add thin passthrough `useQuery` wrappers during refactoring; only extract a custom hook when it truly orchestrates multiple queries/mutations or shared derived state.
|
||||
|
||||
**Dify Examples**:
|
||||
|
||||
- `web/service/use-workflow.ts`
|
||||
- `web/service/use-common.ts`
|
||||
- `web/service/knowledge/use-dataset.ts`
|
||||
- `web/service/knowledge/use-document.ts`
|
||||
|
||||
### Pattern 5: Extract Modal/Dialog Management
|
||||
|
||||
**When**: Component manages multiple modals with complex open/close states.
|
||||
|
||||
**Dify Convention**: Modals should be extracted with their state management.
|
||||
|
||||
```typescript
|
||||
// ❌ Before: Multiple modal states in component
|
||||
const AppInfo = () => {
|
||||
const [showEditModal, setShowEditModal] = useState(false)
|
||||
const [showDuplicateModal, setShowDuplicateModal] = useState(false)
|
||||
const [showConfirmDelete, setShowConfirmDelete] = useState(false)
|
||||
const [showSwitchModal, setShowSwitchModal] = useState(false)
|
||||
const [showImportDSLModal, setShowImportDSLModal] = useState(false)
|
||||
// 5+ more modal states...
|
||||
}
|
||||
|
||||
// ✅ After: Extract to modal management hook
|
||||
type ModalType = 'edit' | 'duplicate' | 'delete' | 'switch' | 'import' | null
|
||||
|
||||
const useAppInfoModals = () => {
|
||||
const [activeModal, setActiveModal] = useState<ModalType>(null)
|
||||
|
||||
const openModal = useCallback((type: ModalType) => setActiveModal(type), [])
|
||||
const closeModal = useCallback(() => setActiveModal(null), [])
|
||||
|
||||
return {
|
||||
activeModal,
|
||||
openModal,
|
||||
closeModal,
|
||||
isOpen: (type: ModalType) => activeModal === type,
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Pattern 6: Extract Form Logic
|
||||
|
||||
**When**: Complex form validation, submission handling, or field transformation.
|
||||
|
||||
**Dify Convention**: Use `@tanstack/react-form` patterns from `web/app/components/base/form/`.
|
||||
|
||||
```typescript
|
||||
// ✅ Use existing form infrastructure
|
||||
import { useAppForm } from '@/app/components/base/form'
|
||||
|
||||
const ConfigForm = () => {
|
||||
const form = useAppForm({
|
||||
defaultValues: { name: '', description: '' },
|
||||
onSubmit: handleSubmit,
|
||||
})
|
||||
|
||||
return <form.Provider>...</form.Provider>
|
||||
}
|
||||
```
|
||||
|
||||
## Dify-Specific Refactoring Guidelines
|
||||
|
||||
### 1. Context Provider Extraction
|
||||
|
||||
**When**: Component provides complex context values with multiple states.
|
||||
|
||||
```typescript
|
||||
// ❌ Before: Large context value object
|
||||
const value = {
|
||||
appId, isAPIKeySet, isTrailFinished, mode, modelModeType,
|
||||
promptMode, isAdvancedMode, isAgent, isOpenAI, isFunctionCall,
|
||||
// 50+ more properties...
|
||||
}
|
||||
return <ConfigContext.Provider value={value}>...</ConfigContext.Provider>
|
||||
|
||||
// ✅ After: Split into domain-specific contexts
|
||||
<ModelConfigProvider value={modelConfigValue}>
|
||||
<DatasetConfigProvider value={datasetConfigValue}>
|
||||
<UIConfigProvider value={uiConfigValue}>
|
||||
{children}
|
||||
</UIConfigProvider>
|
||||
</DatasetConfigProvider>
|
||||
</ModelConfigProvider>
|
||||
```
|
||||
|
||||
**Dify Reference**: `web/context/` directory structure
|
||||
|
||||
### 2. Workflow Node Components
|
||||
|
||||
**When**: Refactoring workflow node components (`web/app/components/workflow/nodes/`).
|
||||
|
||||
**Conventions**:
|
||||
|
||||
- Keep node logic in `use-interactions.ts`
|
||||
- Extract panel UI to separate files
|
||||
- Use `_base` components for common patterns
|
||||
|
||||
```
|
||||
nodes/<node-type>/
|
||||
├── index.tsx # Node registration
|
||||
├── node.tsx # Node visual component
|
||||
├── panel.tsx # Configuration panel
|
||||
├── use-interactions.ts # Node-specific hooks
|
||||
└── types.ts # Type definitions
|
||||
```
|
||||
|
||||
### 3. Configuration Components
|
||||
|
||||
**When**: Refactoring app configuration components.
|
||||
|
||||
**Conventions**:
|
||||
|
||||
- Separate config sections into subdirectories
|
||||
- Use existing patterns from `web/app/components/app/configuration/`
|
||||
- Keep feature toggles in dedicated components
|
||||
|
||||
### 4. Tool/Plugin Components
|
||||
|
||||
**When**: Refactoring tool-related components (`web/app/components/tools/`).
|
||||
|
||||
**Conventions**:
|
||||
|
||||
- Follow existing modal patterns
|
||||
- Use service hooks from `web/service/use-tools.ts`
|
||||
- Keep provider-specific logic isolated
|
||||
|
||||
## Refactoring Workflow
|
||||
|
||||
### Step 1: Generate Refactoring Prompt
|
||||
|
||||
```bash
|
||||
pnpm refactor-component <path>
|
||||
```
|
||||
|
||||
This command will:
|
||||
|
||||
- Analyze component complexity and features
|
||||
- Identify specific refactoring actions needed
|
||||
- Generate a prompt for AI assistant (auto-copied to clipboard on macOS)
|
||||
- Provide detailed requirements based on detected patterns
|
||||
|
||||
### Step 2: Analyze Details
|
||||
|
||||
```bash
|
||||
pnpm analyze-component <path> --json
|
||||
```
|
||||
|
||||
Identify:
|
||||
|
||||
- Total complexity score
|
||||
- Max function complexity
|
||||
- Line count
|
||||
- Features detected (state, effects, API, etc.)
|
||||
|
||||
### Step 3: Plan
|
||||
|
||||
Create a refactoring plan based on detected features:
|
||||
|
||||
| Detected Feature | Refactoring Action |
|
||||
| ------------------------------------- | -------------------------- |
|
||||
| `hasState: true` + `hasEffects: true` | Extract custom hook |
|
||||
| `hasAPI: true` | Extract data/service hook |
|
||||
| `hasEvents: true` (many) | Extract event handlers |
|
||||
| `lineCount > 300` | Split into sub-components |
|
||||
| `maxComplexity > 50` | Simplify conditional logic |
|
||||
|
||||
### Step 4: Execute Incrementally
|
||||
|
||||
1. **Extract one piece at a time**
|
||||
2. **Run lint, type-check, and tests after each extraction**
|
||||
3. **Verify functionality before next step**
|
||||
|
||||
```
|
||||
For each extraction:
|
||||
┌────────────────────────────────────────┐
|
||||
│ 1. Extract code │
|
||||
│ 2. Run: pnpm lint:fix │
|
||||
│ 3. Run: pnpm type-check │
|
||||
│ 4. Run: pnpm test │
|
||||
│ 5. Test functionality manually │
|
||||
│ 6. PASS? → Next extraction │
|
||||
│ FAIL? → Fix before continuing │
|
||||
└────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
### Step 5: Verify
|
||||
|
||||
After refactoring:
|
||||
|
||||
```bash
|
||||
# Re-run refactor command to verify improvements
|
||||
pnpm refactor-component <path>
|
||||
|
||||
# If complexity < 25 and lines < 200, you'll see:
|
||||
# ✅ COMPONENT IS WELL-STRUCTURED
|
||||
|
||||
# For detailed metrics:
|
||||
pnpm analyze-component <path> --json
|
||||
|
||||
# Target metrics:
|
||||
# - complexity < 50
|
||||
# - lineCount < 300
|
||||
# - maxComplexity < 30
|
||||
```
|
||||
|
||||
## Common Mistakes to Avoid
|
||||
|
||||
### ❌ Over-Engineering
|
||||
|
||||
```typescript
|
||||
// ❌ Too many tiny hooks
|
||||
const useButtonText = () => useState('Click')
|
||||
const useButtonDisabled = () => useState(false)
|
||||
const useButtonLoading = () => useState(false)
|
||||
|
||||
// ✅ Cohesive hook with related state
|
||||
const useButtonState = () => {
|
||||
const [text, setText] = useState('Click')
|
||||
const [disabled, setDisabled] = useState(false)
|
||||
const [loading, setLoading] = useState(false)
|
||||
return { text, setText, disabled, setDisabled, loading, setLoading }
|
||||
}
|
||||
```
|
||||
|
||||
### ❌ Breaking Existing Patterns
|
||||
|
||||
- Follow existing directory structures
|
||||
- Maintain naming conventions
|
||||
- Preserve export patterns for compatibility
|
||||
|
||||
### ❌ Premature Abstraction
|
||||
|
||||
- Only extract when there's clear complexity benefit
|
||||
- Don't create abstractions for single-use code
|
||||
- Keep refactored code in the same domain area
|
||||
|
||||
## References
|
||||
|
||||
### Dify Codebase Examples
|
||||
|
||||
- **Hook extraction**: `web/app/components/app/configuration/hooks/`
|
||||
- **Component splitting**: `web/app/components/app/configuration/`
|
||||
- **Service hooks**: `web/service/use-*.ts`
|
||||
- **Workflow patterns**: `web/app/components/workflow/hooks/`
|
||||
- **Form patterns**: `web/app/components/base/form/`
|
||||
|
||||
### Related Skills
|
||||
|
||||
- `frontend-testing` - For testing refactored components
|
||||
- `web/docs/test.md` - Testing specification
|
||||
@@ -0,0 +1,496 @@
|
||||
# Complexity Reduction Patterns
|
||||
|
||||
This document provides patterns for reducing cognitive complexity in Dify React components.
|
||||
|
||||
## Understanding Complexity
|
||||
|
||||
### SonarJS Cognitive Complexity
|
||||
|
||||
The `pnpm analyze-component` tool uses SonarJS cognitive complexity metrics:
|
||||
|
||||
- **Total Complexity**: Sum of all functions' complexity in the file
|
||||
- **Max Complexity**: Highest single function complexity
|
||||
|
||||
### What Increases Complexity
|
||||
|
||||
| Pattern | Complexity Impact |
|
||||
| ------------------- | -------------------- | -------- | --------------- |
|
||||
| `if/else` | +1 per branch |
|
||||
| Nested conditions | +1 per nesting level |
|
||||
| `switch/case` | +1 per case |
|
||||
| `for/while/do` | +1 per loop |
|
||||
| `&&`/` | | ` chains | +1 per operator |
|
||||
| Nested callbacks | +1 per nesting level |
|
||||
| `try/catch` | +1 per catch |
|
||||
| Ternary expressions | +1 per nesting |
|
||||
|
||||
## Pattern 1: Replace Conditionals with Lookup Tables
|
||||
|
||||
**Before** (complexity: ~15):
|
||||
|
||||
```typescript
|
||||
const Template = useMemo(() => {
|
||||
if (appDetail?.mode === AppModeEnum.CHAT) {
|
||||
switch (locale) {
|
||||
case LanguagesSupported[1]:
|
||||
return <TemplateChatZh appDetail={appDetail} />
|
||||
case LanguagesSupported[7]:
|
||||
return <TemplateChatJa appDetail={appDetail} />
|
||||
default:
|
||||
return <TemplateChatEn appDetail={appDetail} />
|
||||
}
|
||||
}
|
||||
if (appDetail?.mode === AppModeEnum.ADVANCED_CHAT) {
|
||||
switch (locale) {
|
||||
case LanguagesSupported[1]:
|
||||
return <TemplateAdvancedChatZh appDetail={appDetail} />
|
||||
case LanguagesSupported[7]:
|
||||
return <TemplateAdvancedChatJa appDetail={appDetail} />
|
||||
default:
|
||||
return <TemplateAdvancedChatEn appDetail={appDetail} />
|
||||
}
|
||||
}
|
||||
if (appDetail?.mode === AppModeEnum.WORKFLOW) {
|
||||
// Similar pattern...
|
||||
}
|
||||
return null
|
||||
}, [appDetail, locale])
|
||||
```
|
||||
|
||||
**After** (complexity: ~3):
|
||||
|
||||
```typescript
|
||||
import type { ComponentType } from 'react'
|
||||
|
||||
// Define lookup table outside component
|
||||
const TEMPLATE_MAP: Record<AppModeEnum, Record<string, ComponentType<TemplateProps>>> = {
|
||||
[AppModeEnum.CHAT]: {
|
||||
[LanguagesSupported[1]]: TemplateChatZh,
|
||||
[LanguagesSupported[7]]: TemplateChatJa,
|
||||
default: TemplateChatEn,
|
||||
},
|
||||
[AppModeEnum.ADVANCED_CHAT]: {
|
||||
[LanguagesSupported[1]]: TemplateAdvancedChatZh,
|
||||
[LanguagesSupported[7]]: TemplateAdvancedChatJa,
|
||||
default: TemplateAdvancedChatEn,
|
||||
},
|
||||
[AppModeEnum.WORKFLOW]: {
|
||||
[LanguagesSupported[1]]: TemplateWorkflowZh,
|
||||
[LanguagesSupported[7]]: TemplateWorkflowJa,
|
||||
default: TemplateWorkflowEn,
|
||||
},
|
||||
// ...
|
||||
}
|
||||
|
||||
// Clean component logic
|
||||
const Template = useMemo(() => {
|
||||
if (!appDetail?.mode) return null
|
||||
|
||||
const templates = TEMPLATE_MAP[appDetail.mode]
|
||||
if (!templates) return null
|
||||
|
||||
const TemplateComponent = templates[locale] ?? templates.default
|
||||
return <TemplateComponent appDetail={appDetail} />
|
||||
}, [appDetail, locale])
|
||||
```
|
||||
|
||||
## Pattern 2: Use Early Returns
|
||||
|
||||
**Before** (complexity: ~10):
|
||||
|
||||
```typescript
|
||||
const handleSubmit = () => {
|
||||
if (isValid) {
|
||||
if (hasChanges) {
|
||||
if (isConnected) {
|
||||
submitData()
|
||||
} else {
|
||||
showConnectionError()
|
||||
}
|
||||
} else {
|
||||
showNoChangesMessage()
|
||||
}
|
||||
} else {
|
||||
showValidationError()
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**After** (complexity: ~4):
|
||||
|
||||
```typescript
|
||||
const handleSubmit = () => {
|
||||
if (!isValid) {
|
||||
showValidationError()
|
||||
return
|
||||
}
|
||||
|
||||
if (!hasChanges) {
|
||||
showNoChangesMessage()
|
||||
return
|
||||
}
|
||||
|
||||
if (!isConnected) {
|
||||
showConnectionError()
|
||||
return
|
||||
}
|
||||
|
||||
submitData()
|
||||
}
|
||||
```
|
||||
|
||||
## Pattern 3: Extract Complex Conditions
|
||||
|
||||
**Before** (complexity: high):
|
||||
|
||||
```typescript
|
||||
const canPublish = (() => {
|
||||
if (mode !== AppModeEnum.COMPLETION) {
|
||||
if (!isAdvancedMode) return true
|
||||
|
||||
if (modelModeType === ModelModeType.completion) {
|
||||
if (!hasSetBlockStatus.history || !hasSetBlockStatus.query) return false
|
||||
return true
|
||||
}
|
||||
return true
|
||||
}
|
||||
return !promptEmpty
|
||||
})()
|
||||
```
|
||||
|
||||
**After** (complexity: lower):
|
||||
|
||||
```typescript
|
||||
// Extract to named functions
|
||||
const canPublishInCompletionMode = () => !promptEmpty
|
||||
|
||||
const canPublishInChatMode = () => {
|
||||
if (!isAdvancedMode) return true
|
||||
if (modelModeType !== ModelModeType.completion) return true
|
||||
return hasSetBlockStatus.history && hasSetBlockStatus.query
|
||||
}
|
||||
|
||||
// Clean main logic
|
||||
const canPublish =
|
||||
mode === AppModeEnum.COMPLETION ? canPublishInCompletionMode() : canPublishInChatMode()
|
||||
```
|
||||
|
||||
## Pattern 4: Replace Chained Ternaries
|
||||
|
||||
**Before** (complexity: ~5):
|
||||
|
||||
```typescript
|
||||
const statusText = serverActivated
|
||||
? t('status.running')
|
||||
: serverPublished
|
||||
? t('status.inactive')
|
||||
: appUnpublished
|
||||
? t('status.unpublished')
|
||||
: t('status.notConfigured')
|
||||
```
|
||||
|
||||
**After** (complexity: ~2):
|
||||
|
||||
```typescript
|
||||
const getStatusText = () => {
|
||||
if (serverActivated) return t('status.running')
|
||||
if (serverPublished) return t('status.inactive')
|
||||
if (appUnpublished) return t('status.unpublished')
|
||||
return t('status.notConfigured')
|
||||
}
|
||||
|
||||
const statusText = getStatusText()
|
||||
```
|
||||
|
||||
Or use lookup:
|
||||
|
||||
```typescript
|
||||
const STATUS_TEXT_MAP = {
|
||||
running: 'status.running',
|
||||
inactive: 'status.inactive',
|
||||
unpublished: 'status.unpublished',
|
||||
notConfigured: 'status.notConfigured',
|
||||
} as const
|
||||
|
||||
const getStatusKey = (): keyof typeof STATUS_TEXT_MAP => {
|
||||
if (serverActivated) return 'running'
|
||||
if (serverPublished) return 'inactive'
|
||||
if (appUnpublished) return 'unpublished'
|
||||
return 'notConfigured'
|
||||
}
|
||||
|
||||
const statusText = t(STATUS_TEXT_MAP[getStatusKey()])
|
||||
```
|
||||
|
||||
## Pattern 5: Flatten Nested Loops
|
||||
|
||||
**Before** (complexity: high):
|
||||
|
||||
```typescript
|
||||
const processData = (items: Item[]) => {
|
||||
const results: ProcessedItem[] = []
|
||||
|
||||
for (const item of items) {
|
||||
if (item.isValid) {
|
||||
for (const child of item.children) {
|
||||
if (child.isActive) {
|
||||
for (const prop of child.properties) {
|
||||
if (prop.value !== null) {
|
||||
results.push({
|
||||
itemId: item.id,
|
||||
childId: child.id,
|
||||
propValue: prop.value,
|
||||
})
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return results
|
||||
}
|
||||
```
|
||||
|
||||
**After** (complexity: lower):
|
||||
|
||||
```typescript
|
||||
// Use functional approach
|
||||
const processData = (items: Item[]) => {
|
||||
return items
|
||||
.filter((item) => item.isValid)
|
||||
.flatMap((item) =>
|
||||
item.children
|
||||
.filter((child) => child.isActive)
|
||||
.flatMap((child) =>
|
||||
child.properties
|
||||
.filter((prop) => prop.value !== null)
|
||||
.map((prop) => ({
|
||||
itemId: item.id,
|
||||
childId: child.id,
|
||||
propValue: prop.value,
|
||||
})),
|
||||
),
|
||||
)
|
||||
}
|
||||
```
|
||||
|
||||
## Pattern 6: Extract Event Handler Logic
|
||||
|
||||
**Before** (complexity: high in component):
|
||||
|
||||
```typescript
|
||||
const Component = () => {
|
||||
const handleSelect = (data: DataSet[]) => {
|
||||
if (isEqual(data.map(item => item.id), dataSets.map(item => item.id))) {
|
||||
hideSelectDataSet()
|
||||
return
|
||||
}
|
||||
|
||||
formattingChangedDispatcher()
|
||||
let newDatasets = data
|
||||
if (data.find(item => !item.name)) {
|
||||
const newSelected = produce(data, (draft) => {
|
||||
data.forEach((item, index) => {
|
||||
if (!item.name) {
|
||||
const newItem = dataSets.find(i => i.id === item.id)
|
||||
if (newItem)
|
||||
draft[index] = newItem
|
||||
}
|
||||
})
|
||||
})
|
||||
setDataSets(newSelected)
|
||||
newDatasets = newSelected
|
||||
}
|
||||
else {
|
||||
setDataSets(data)
|
||||
}
|
||||
hideSelectDataSet()
|
||||
|
||||
// 40 more lines of logic...
|
||||
}
|
||||
|
||||
return <div>...</div>
|
||||
}
|
||||
```
|
||||
|
||||
**After** (complexity: lower):
|
||||
|
||||
```typescript
|
||||
// Extract to hook or utility
|
||||
const useDatasetSelection = (dataSets: DataSet[], setDataSets: SetState<DataSet[]>) => {
|
||||
const normalizeSelection = (data: DataSet[]) => {
|
||||
const hasUnloadedItem = data.some(item => !item.name)
|
||||
if (!hasUnloadedItem) return data
|
||||
|
||||
return produce(data, (draft) => {
|
||||
data.forEach((item, index) => {
|
||||
if (!item.name) {
|
||||
const existing = dataSets.find(i => i.id === item.id)
|
||||
if (existing) draft[index] = existing
|
||||
}
|
||||
})
|
||||
})
|
||||
}
|
||||
|
||||
const hasSelectionChanged = (newData: DataSet[]) => {
|
||||
return !isEqual(
|
||||
newData.map(item => item.id),
|
||||
dataSets.map(item => item.id)
|
||||
)
|
||||
}
|
||||
|
||||
return { normalizeSelection, hasSelectionChanged }
|
||||
}
|
||||
|
||||
// Component becomes cleaner
|
||||
const Component = () => {
|
||||
const { normalizeSelection, hasSelectionChanged } = useDatasetSelection(dataSets, setDataSets)
|
||||
|
||||
const handleSelect = (data: DataSet[]) => {
|
||||
if (!hasSelectionChanged(data)) {
|
||||
hideSelectDataSet()
|
||||
return
|
||||
}
|
||||
|
||||
formattingChangedDispatcher()
|
||||
const normalized = normalizeSelection(data)
|
||||
setDataSets(normalized)
|
||||
hideSelectDataSet()
|
||||
}
|
||||
|
||||
return <div>...</div>
|
||||
}
|
||||
```
|
||||
|
||||
## Pattern 7: Reduce Boolean Logic Complexity
|
||||
|
||||
**Before** (complexity: ~8):
|
||||
|
||||
```typescript
|
||||
const toggleDisabled =
|
||||
hasInsufficientPermissions ||
|
||||
appUnpublished ||
|
||||
missingStartNode ||
|
||||
triggerModeDisabled ||
|
||||
(isAdvancedApp && !currentWorkflow?.graph) ||
|
||||
(isBasicApp && !basicAppConfig.updated_at)
|
||||
```
|
||||
|
||||
**After** (complexity: ~3):
|
||||
|
||||
```typescript
|
||||
// Extract meaningful boolean functions
|
||||
const isAppReady = () => {
|
||||
if (isAdvancedApp) return !!currentWorkflow?.graph
|
||||
return !!basicAppConfig.updated_at
|
||||
}
|
||||
|
||||
const hasRequiredPermissions = () => {
|
||||
return isCurrentWorkspaceEditor && !hasInsufficientPermissions
|
||||
}
|
||||
|
||||
const canToggle = () => {
|
||||
if (!hasRequiredPermissions()) return false
|
||||
if (!isAppReady()) return false
|
||||
if (missingStartNode) return false
|
||||
if (triggerModeDisabled) return false
|
||||
return true
|
||||
}
|
||||
|
||||
const toggleDisabled = !canToggle()
|
||||
```
|
||||
|
||||
## Pattern 8: Simplify useMemo/useCallback Dependencies
|
||||
|
||||
**Before** (complexity: multiple recalculations):
|
||||
|
||||
```typescript
|
||||
const payload = useMemo(() => {
|
||||
let parameters: Parameter[] = []
|
||||
let outputParameters: OutputParameter[] = []
|
||||
|
||||
if (!published) {
|
||||
parameters = (inputs || []).map((item) => ({
|
||||
name: item.variable,
|
||||
description: '',
|
||||
form: 'llm',
|
||||
required: item.required,
|
||||
type: item.type,
|
||||
}))
|
||||
outputParameters = (outputs || []).map((item) => ({
|
||||
name: item.variable,
|
||||
description: '',
|
||||
type: item.value_type,
|
||||
}))
|
||||
} else if (detail && detail.tool) {
|
||||
parameters = (inputs || []).map((item) => ({
|
||||
// Complex transformation...
|
||||
}))
|
||||
outputParameters = (outputs || []).map((item) => ({
|
||||
// Complex transformation...
|
||||
}))
|
||||
}
|
||||
|
||||
return {
|
||||
icon: detail?.icon || icon,
|
||||
label: detail?.label || name,
|
||||
// ...more fields
|
||||
}
|
||||
}, [detail, published, workflowAppId, icon, name, description, inputs, outputs])
|
||||
```
|
||||
|
||||
**After** (complexity: separated concerns):
|
||||
|
||||
```typescript
|
||||
// Separate transformations
|
||||
const useParameterTransform = (inputs: InputVar[], detail?: ToolDetail, published?: boolean) => {
|
||||
return useMemo(() => {
|
||||
if (!published) {
|
||||
return inputs.map((item) => ({
|
||||
name: item.variable,
|
||||
description: '',
|
||||
form: 'llm',
|
||||
required: item.required,
|
||||
type: item.type,
|
||||
}))
|
||||
}
|
||||
|
||||
if (!detail?.tool) return []
|
||||
|
||||
return inputs.map((item) => ({
|
||||
name: item.variable,
|
||||
required: item.required,
|
||||
type: item.type === 'paragraph' ? 'string' : item.type,
|
||||
description:
|
||||
detail.tool.parameters.find((p) => p.name === item.variable)?.llm_description || '',
|
||||
form: detail.tool.parameters.find((p) => p.name === item.variable)?.form || 'llm',
|
||||
}))
|
||||
}, [inputs, detail, published])
|
||||
}
|
||||
|
||||
// Component uses hook
|
||||
const parameters = useParameterTransform(inputs, detail, published)
|
||||
const outputParameters = useOutputTransform(outputs, detail, published)
|
||||
|
||||
const payload = useMemo(
|
||||
() => ({
|
||||
icon: detail?.icon || icon,
|
||||
label: detail?.label || name,
|
||||
parameters,
|
||||
outputParameters,
|
||||
// ...
|
||||
}),
|
||||
[detail, icon, name, parameters, outputParameters],
|
||||
)
|
||||
```
|
||||
|
||||
## Target Metrics After Refactoring
|
||||
|
||||
| Metric | Target |
|
||||
| ----------------------- | -------------- |
|
||||
| Total Complexity | < 50 |
|
||||
| Max Function Complexity | < 30 |
|
||||
| Function Length | < 30 lines |
|
||||
| Nesting Depth | ≤ 3 levels |
|
||||
| Conditional Chains | ≤ 3 conditions |
|
||||
@@ -0,0 +1,477 @@
|
||||
# Component Splitting Patterns
|
||||
|
||||
This document provides detailed guidance on splitting large components into smaller, focused components in Dify.
|
||||
|
||||
## When to Split Components
|
||||
|
||||
Split a component when you identify:
|
||||
|
||||
1. **Multiple UI sections** - Distinct visual areas with minimal coupling that can be composed independently
|
||||
1. **Conditional rendering blocks** - Large `{condition && <JSX />}` blocks
|
||||
1. **Repeated patterns** - Similar UI structures used multiple times
|
||||
1. **300+ lines** - Component exceeds manageable size
|
||||
1. **Modal clusters** - Multiple modals rendered in one component
|
||||
|
||||
## Splitting Strategies
|
||||
|
||||
### Strategy 1: Section-Based Splitting
|
||||
|
||||
Identify visual sections and extract each as a component.
|
||||
|
||||
```typescript
|
||||
// ❌ Before: Monolithic component (500+ lines)
|
||||
const ConfigurationPage = () => {
|
||||
return (
|
||||
<div>
|
||||
{/* Header Section - 50 lines */}
|
||||
<div className="header">
|
||||
<h1>{t('configuration.title')}</h1>
|
||||
<div className="actions">
|
||||
{isAdvancedMode && <Badge>Advanced</Badge>}
|
||||
<ModelParameterModal ... />
|
||||
<AppPublisher ... />
|
||||
</div>
|
||||
</div>
|
||||
|
||||
{/* Config Section - 200 lines */}
|
||||
<div className="config">
|
||||
<Config />
|
||||
</div>
|
||||
|
||||
{/* Debug Section - 150 lines */}
|
||||
<div className="debug">
|
||||
<Debug ... />
|
||||
</div>
|
||||
|
||||
{/* Modals Section - 100 lines */}
|
||||
{showSelectDataSet && <SelectDataSet ... />}
|
||||
{showHistoryModal && <EditHistoryModal ... />}
|
||||
{showUseGPT4Confirm && <Confirm ... />}
|
||||
</div>
|
||||
)
|
||||
}
|
||||
|
||||
// ✅ After: Split into focused components
|
||||
// configuration/
|
||||
// ├── index.tsx (orchestration)
|
||||
// ├── configuration-header.tsx
|
||||
// ├── configuration-content.tsx
|
||||
// ├── configuration-debug.tsx
|
||||
// └── configuration-modals.tsx
|
||||
|
||||
// configuration-header.tsx
|
||||
interface ConfigurationHeaderProps {
|
||||
isAdvancedMode: boolean
|
||||
onPublish: () => void
|
||||
}
|
||||
|
||||
function ConfigurationHeader({
|
||||
isAdvancedMode,
|
||||
onPublish,
|
||||
}: ConfigurationHeaderProps) {
|
||||
const { t } = useTranslation()
|
||||
|
||||
return (
|
||||
<div className="header">
|
||||
<h1>{t('configuration.title')}</h1>
|
||||
<div className="actions">
|
||||
{isAdvancedMode && <Badge>Advanced</Badge>}
|
||||
<ModelParameterModal ... />
|
||||
<AppPublisher onPublish={onPublish} />
|
||||
</div>
|
||||
</div>
|
||||
)
|
||||
}
|
||||
|
||||
// index.tsx (orchestration only)
|
||||
const ConfigurationPage = () => {
|
||||
const { modelConfig, setModelConfig } = useModelConfig()
|
||||
const { activeModal, openModal, closeModal } = useModalState()
|
||||
|
||||
return (
|
||||
<div>
|
||||
<ConfigurationHeader
|
||||
isAdvancedMode={isAdvancedMode}
|
||||
onPublish={handlePublish}
|
||||
/>
|
||||
<ConfigurationContent
|
||||
modelConfig={modelConfig}
|
||||
onConfigChange={setModelConfig}
|
||||
/>
|
||||
{!isMobile && (
|
||||
<ConfigurationDebug
|
||||
inputs={inputs}
|
||||
onSetting={handleSetting}
|
||||
/>
|
||||
)}
|
||||
<ConfigurationModals
|
||||
activeModal={activeModal}
|
||||
onClose={closeModal}
|
||||
/>
|
||||
</div>
|
||||
)
|
||||
}
|
||||
```
|
||||
|
||||
### Strategy 2: Conditional Block Extraction
|
||||
|
||||
Extract large conditional rendering blocks.
|
||||
|
||||
```typescript
|
||||
// ❌ Before: Large conditional blocks
|
||||
const AppInfo = () => {
|
||||
return (
|
||||
<div>
|
||||
{expand ? (
|
||||
<div className="expanded">
|
||||
{/* 100 lines of expanded view */}
|
||||
</div>
|
||||
) : (
|
||||
<div className="collapsed">
|
||||
{/* 50 lines of collapsed view */}
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
)
|
||||
}
|
||||
|
||||
// ✅ After: Separate view components
|
||||
function AppInfoExpanded({ appDetail, onAction }: AppInfoViewProps) {
|
||||
return (
|
||||
<div className="expanded">
|
||||
{/* Clean, focused expanded view */}
|
||||
</div>
|
||||
)
|
||||
}
|
||||
|
||||
function AppInfoCollapsed({ appDetail, onAction }: AppInfoViewProps) {
|
||||
return (
|
||||
<div className="collapsed">
|
||||
{/* Clean, focused collapsed view */}
|
||||
</div>
|
||||
)
|
||||
}
|
||||
|
||||
const AppInfo = () => {
|
||||
return (
|
||||
<div>
|
||||
{expand
|
||||
? <AppInfoExpanded appDetail={appDetail} onAction={handleAction} />
|
||||
: <AppInfoCollapsed appDetail={appDetail} onAction={handleAction} />
|
||||
}
|
||||
</div>
|
||||
)
|
||||
}
|
||||
```
|
||||
|
||||
### Strategy 3: Modal Extraction
|
||||
|
||||
Extract modals with their trigger logic.
|
||||
|
||||
```typescript
|
||||
// ❌ Before: Multiple modals in one component
|
||||
const AppInfo = () => {
|
||||
const [showEdit, setShowEdit] = useState(false)
|
||||
const [showDuplicate, setShowDuplicate] = useState(false)
|
||||
const [showDelete, setShowDelete] = useState(false)
|
||||
const [showSwitch, setShowSwitch] = useState(false)
|
||||
|
||||
const onEdit = async (data) => { /* 20 lines */ }
|
||||
const onDuplicate = async (data) => { /* 20 lines */ }
|
||||
const onDelete = async () => { /* 15 lines */ }
|
||||
|
||||
return (
|
||||
<div>
|
||||
{/* Main content */}
|
||||
|
||||
{showEdit && <EditModal onConfirm={onEdit} onClose={() => setShowEdit(false)} />}
|
||||
{showDuplicate && <DuplicateModal onConfirm={onDuplicate} onClose={() => setShowDuplicate(false)} />}
|
||||
{showDelete && <DeleteConfirm onConfirm={onDelete} onClose={() => setShowDelete(false)} />}
|
||||
{showSwitch && <SwitchModal ... />}
|
||||
</div>
|
||||
)
|
||||
}
|
||||
|
||||
// ✅ After: Modal manager component
|
||||
// app-info-modals.tsx
|
||||
type ModalType = 'edit' | 'duplicate' | 'delete' | 'switch' | null
|
||||
|
||||
interface AppInfoModalsProps {
|
||||
appDetail: AppDetail
|
||||
activeModal: ModalType
|
||||
onClose: () => void
|
||||
onSuccess: () => void
|
||||
}
|
||||
|
||||
function AppInfoModals({
|
||||
appDetail,
|
||||
activeModal,
|
||||
onClose,
|
||||
onSuccess,
|
||||
}: AppInfoModalsProps) {
|
||||
const handleEdit = async (data) => { /* logic */ }
|
||||
const handleDuplicate = async (data) => { /* logic */ }
|
||||
const handleDelete = async () => { /* logic */ }
|
||||
|
||||
return (
|
||||
<>
|
||||
{activeModal === 'edit' && (
|
||||
<EditModal
|
||||
appDetail={appDetail}
|
||||
onConfirm={handleEdit}
|
||||
onClose={onClose}
|
||||
/>
|
||||
)}
|
||||
{activeModal === 'duplicate' && (
|
||||
<DuplicateModal
|
||||
appDetail={appDetail}
|
||||
onConfirm={handleDuplicate}
|
||||
onClose={onClose}
|
||||
/>
|
||||
)}
|
||||
{activeModal === 'delete' && (
|
||||
<DeleteConfirm
|
||||
onConfirm={handleDelete}
|
||||
onClose={onClose}
|
||||
/>
|
||||
)}
|
||||
{activeModal === 'switch' && (
|
||||
<SwitchModal
|
||||
appDetail={appDetail}
|
||||
onClose={onClose}
|
||||
/>
|
||||
)}
|
||||
</>
|
||||
)
|
||||
}
|
||||
|
||||
// Parent component
|
||||
const AppInfo = () => {
|
||||
const { activeModal, openModal, closeModal } = useModalState()
|
||||
|
||||
return (
|
||||
<div>
|
||||
{/* Main content with openModal triggers */}
|
||||
<Button onClick={() => openModal('edit')}>Edit</Button>
|
||||
|
||||
<AppInfoModals
|
||||
appDetail={appDetail}
|
||||
activeModal={activeModal}
|
||||
onClose={closeModal}
|
||||
onSuccess={handleSuccess}
|
||||
/>
|
||||
</div>
|
||||
)
|
||||
}
|
||||
```
|
||||
|
||||
### Strategy 4: List Item Extraction
|
||||
|
||||
Extract repeated item rendering.
|
||||
|
||||
```typescript
|
||||
// ❌ Before: Inline item rendering
|
||||
const OperationsList = () => {
|
||||
return (
|
||||
<div>
|
||||
{operations.map(op => (
|
||||
<div key={op.id} className="operation-item">
|
||||
<span className="icon">{op.icon}</span>
|
||||
<span className="title">{op.title}</span>
|
||||
<span className="description">{op.description}</span>
|
||||
<button onClick={() => op.onClick()}>
|
||||
{op.actionLabel}
|
||||
</button>
|
||||
{op.badge && <Badge>{op.badge}</Badge>}
|
||||
{/* More complex rendering... */}
|
||||
</div>
|
||||
))}
|
||||
</div>
|
||||
)
|
||||
}
|
||||
|
||||
// ✅ After: Extracted item component
|
||||
interface OperationItemProps {
|
||||
operation: Operation
|
||||
onAction: (id: string) => void
|
||||
}
|
||||
|
||||
function OperationItem({ operation, onAction }: OperationItemProps) {
|
||||
return (
|
||||
<div className="operation-item">
|
||||
<span className="icon">{operation.icon}</span>
|
||||
<span className="title">{operation.title}</span>
|
||||
<span className="description">{operation.description}</span>
|
||||
<button onClick={() => onAction(operation.id)}>
|
||||
{operation.actionLabel}
|
||||
</button>
|
||||
{operation.badge && <Badge>{operation.badge}</Badge>}
|
||||
</div>
|
||||
)
|
||||
}
|
||||
|
||||
const OperationsList = () => {
|
||||
const handleAction = useCallback((id: string) => {
|
||||
const op = operations.find(o => o.id === id)
|
||||
op?.onClick()
|
||||
}, [operations])
|
||||
|
||||
return (
|
||||
<div>
|
||||
{operations.map(op => (
|
||||
<OperationItem
|
||||
key={op.id}
|
||||
operation={op}
|
||||
onAction={handleAction}
|
||||
/>
|
||||
))}
|
||||
</div>
|
||||
)
|
||||
}
|
||||
```
|
||||
|
||||
## Directory Structure Patterns
|
||||
|
||||
### Pattern A: Flat Structure (Simple Components)
|
||||
|
||||
For components with 2-3 sub-components:
|
||||
|
||||
```
|
||||
component-name/
|
||||
├── index.tsx # Main component
|
||||
├── sub-component-a.tsx
|
||||
├── sub-component-b.tsx
|
||||
└── types.ts # Shared types
|
||||
```
|
||||
|
||||
### Pattern B: Nested Structure (Complex Components)
|
||||
|
||||
For components with many sub-components:
|
||||
|
||||
```
|
||||
component-name/
|
||||
├── index.tsx # Main orchestration
|
||||
├── types.ts # Shared types
|
||||
├── hooks/
|
||||
│ ├── use-feature-a.ts
|
||||
│ └── use-feature-b.ts
|
||||
├── components/
|
||||
│ ├── header/
|
||||
│ │ └── index.tsx
|
||||
│ ├── content/
|
||||
│ │ └── index.tsx
|
||||
│ └── modals/
|
||||
│ └── index.tsx
|
||||
└── utils/
|
||||
└── helpers.ts
|
||||
```
|
||||
|
||||
### Pattern C: Feature-Based Structure (Dify Standard)
|
||||
|
||||
Following Dify's existing patterns:
|
||||
|
||||
```
|
||||
configuration/
|
||||
├── index.tsx # Main page component
|
||||
├── base/ # Base/shared components
|
||||
│ ├── feature-panel/
|
||||
│ ├── group-name/
|
||||
│ └── operation-btn/
|
||||
├── config/ # Config section
|
||||
│ ├── index.tsx
|
||||
│ ├── agent/
|
||||
│ └── automatic/
|
||||
├── dataset-config/ # Dataset section
|
||||
│ ├── index.tsx
|
||||
│ ├── card-item/
|
||||
│ └── params-config/
|
||||
├── debug/ # Debug section
|
||||
│ ├── index.tsx
|
||||
│ └── hooks.tsx
|
||||
└── hooks/ # Shared hooks
|
||||
└── use-advanced-prompt-config.ts
|
||||
```
|
||||
|
||||
## Props Design
|
||||
|
||||
### Minimal Props Principle
|
||||
|
||||
Pass only what's needed:
|
||||
|
||||
```typescript
|
||||
// ❌ Bad: Passing entire objects when only some fields needed
|
||||
<ConfigHeader appDetail={appDetail} modelConfig={modelConfig} />
|
||||
|
||||
// ✅ Good: Destructure to minimum required
|
||||
<ConfigHeader
|
||||
appName={appDetail.name}
|
||||
isAdvancedMode={modelConfig.isAdvanced}
|
||||
onPublish={handlePublish}
|
||||
/>
|
||||
```
|
||||
|
||||
### Callback Props Pattern
|
||||
|
||||
Use callbacks for child-to-parent communication:
|
||||
|
||||
```typescript
|
||||
// Parent
|
||||
const Parent = () => {
|
||||
const [value, setValue] = useState('')
|
||||
|
||||
return (
|
||||
<Child
|
||||
value={value}
|
||||
onChange={setValue}
|
||||
onSubmit={handleSubmit}
|
||||
/>
|
||||
)
|
||||
}
|
||||
|
||||
// Child
|
||||
interface ChildProps {
|
||||
value: string
|
||||
onChange: (value: string) => void
|
||||
onSubmit: () => void
|
||||
}
|
||||
|
||||
function Child({ value, onChange, onSubmit }: ChildProps) {
|
||||
return (
|
||||
<div>
|
||||
<input value={value} onChange={e => onChange(e.target.value)} />
|
||||
<button onClick={onSubmit}>Submit</button>
|
||||
</div>
|
||||
)
|
||||
}
|
||||
```
|
||||
|
||||
### Render Props for Flexibility
|
||||
|
||||
When sub-components need parent context:
|
||||
|
||||
```typescript
|
||||
interface ListProps<T> {
|
||||
items: T[]
|
||||
renderItem: (item: T, index: number) => React.ReactNode
|
||||
renderEmpty?: () => React.ReactNode
|
||||
}
|
||||
|
||||
function List<T>({ items, renderItem, renderEmpty }: ListProps<T>) {
|
||||
if (items.length === 0 && renderEmpty) {
|
||||
return <>{renderEmpty()}</>
|
||||
}
|
||||
|
||||
return (
|
||||
<div>
|
||||
{items.map((item, index) => renderItem(item, index))}
|
||||
</div>
|
||||
)
|
||||
}
|
||||
|
||||
// Usage
|
||||
<List
|
||||
items={operations}
|
||||
renderItem={(op, i) => <OperationItem key={i} operation={op} />}
|
||||
renderEmpty={() => <EmptyState message="No operations" />}
|
||||
/>
|
||||
```
|
||||
@@ -0,0 +1,290 @@
|
||||
# Hook Extraction Patterns
|
||||
|
||||
This document provides detailed guidance on extracting custom hooks from complex components in Dify.
|
||||
|
||||
## When to Extract Hooks
|
||||
|
||||
Extract a custom hook when you identify:
|
||||
|
||||
1. **Coupled state groups** - Multiple `useState` hooks that are always used together
|
||||
1. **Complex effects** - `useEffect` with multiple dependencies or cleanup logic
|
||||
1. **Business logic** - Data transformations, validations, or calculations
|
||||
1. **Reusable patterns** - Logic that appears in multiple components
|
||||
|
||||
## Extraction Process
|
||||
|
||||
### Step 1: Identify State Groups
|
||||
|
||||
Look for state variables that are logically related:
|
||||
|
||||
```typescript
|
||||
// ❌ These belong together - extract to hook
|
||||
const [modelConfig, setModelConfig] = useState<ModelConfig>(...)
|
||||
const [completionParams, setCompletionParams] = useState<FormValue>({})
|
||||
const [modelModeType, setModelModeType] = useState<ModelModeType>(...)
|
||||
|
||||
// These are model-related state that should be in useModelConfig()
|
||||
```
|
||||
|
||||
### Step 2: Identify Related Effects
|
||||
|
||||
Find effects that modify the grouped state:
|
||||
|
||||
```typescript
|
||||
// ❌ These effects belong with the state above
|
||||
useEffect(() => {
|
||||
if (hasFetchedDetail && !modelModeType) {
|
||||
const mode = currModel?.model_properties.mode
|
||||
if (mode) {
|
||||
const newModelConfig = produce(modelConfig, (draft) => {
|
||||
draft.mode = mode
|
||||
})
|
||||
setModelConfig(newModelConfig)
|
||||
}
|
||||
}
|
||||
}, [textGenerationModelList, hasFetchedDetail, modelModeType, currModel])
|
||||
```
|
||||
|
||||
### Step 3: Create the Hook
|
||||
|
||||
```typescript
|
||||
// hooks/use-model-config.ts
|
||||
import type { FormValue } from '@/app/components/header/account-setting/model-provider-page/declarations'
|
||||
import type { ModelConfig } from '@/models/debug'
|
||||
import { produce } from 'immer'
|
||||
import { useEffect, useState } from 'react'
|
||||
import { ModelModeType } from '@/types/app'
|
||||
|
||||
interface UseModelConfigParams {
|
||||
initialConfig?: Partial<ModelConfig>
|
||||
currModel?: { model_properties?: { mode?: ModelModeType } }
|
||||
hasFetchedDetail: boolean
|
||||
}
|
||||
|
||||
interface UseModelConfigReturn {
|
||||
modelConfig: ModelConfig
|
||||
setModelConfig: (config: ModelConfig) => void
|
||||
completionParams: FormValue
|
||||
setCompletionParams: (params: FormValue) => void
|
||||
modelModeType: ModelModeType
|
||||
}
|
||||
|
||||
export const useModelConfig = ({
|
||||
initialConfig,
|
||||
currModel,
|
||||
hasFetchedDetail,
|
||||
}: UseModelConfigParams): UseModelConfigReturn => {
|
||||
const [modelConfig, setModelConfig] = useState<ModelConfig>({
|
||||
provider: 'langgenius/openai/openai',
|
||||
model_id: 'gpt-3.5-turbo',
|
||||
mode: ModelModeType.unset,
|
||||
// ... default values
|
||||
...initialConfig,
|
||||
})
|
||||
|
||||
const [completionParams, setCompletionParams] = useState<FormValue>({})
|
||||
|
||||
const modelModeType = modelConfig.mode
|
||||
|
||||
// Fill old app data missing model mode
|
||||
useEffect(() => {
|
||||
if (hasFetchedDetail && !modelModeType) {
|
||||
const mode = currModel?.model_properties?.mode
|
||||
if (mode) {
|
||||
setModelConfig(
|
||||
produce(modelConfig, (draft) => {
|
||||
draft.mode = mode
|
||||
}),
|
||||
)
|
||||
}
|
||||
}
|
||||
}, [hasFetchedDetail, modelModeType, currModel])
|
||||
|
||||
return {
|
||||
modelConfig,
|
||||
setModelConfig,
|
||||
completionParams,
|
||||
setCompletionParams,
|
||||
modelModeType,
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Step 4: Update Component
|
||||
|
||||
```typescript
|
||||
// Before: 50+ lines of state management
|
||||
function Configuration() {
|
||||
const [modelConfig, setModelConfig] = useState<ModelConfig>(...)
|
||||
// ... lots of related state and effects
|
||||
}
|
||||
|
||||
// After: Clean component
|
||||
function Configuration() {
|
||||
const {
|
||||
modelConfig,
|
||||
setModelConfig,
|
||||
completionParams,
|
||||
setCompletionParams,
|
||||
modelModeType,
|
||||
} = useModelConfig({
|
||||
currModel,
|
||||
hasFetchedDetail,
|
||||
})
|
||||
|
||||
// Component now focuses on UI
|
||||
}
|
||||
```
|
||||
|
||||
## Naming Conventions
|
||||
|
||||
### Hook Names
|
||||
|
||||
- Use `use` prefix: `useModelConfig`, `useDatasetConfig`
|
||||
- Be specific: `useAdvancedPromptConfig` not `usePrompt`
|
||||
- Include domain: `useWorkflowVariables`, `useMCPServer`
|
||||
|
||||
### File Names
|
||||
|
||||
- Kebab-case: `use-model-config.ts`
|
||||
- Place in `hooks/` subdirectory when multiple hooks exist
|
||||
- Place alongside component for single-use hooks
|
||||
|
||||
### Return Type Names
|
||||
|
||||
- Suffix with `Return`: `UseModelConfigReturn`
|
||||
- Suffix params with `Params`: `UseModelConfigParams`
|
||||
|
||||
## Common Hook Patterns in Dify
|
||||
|
||||
### 1. Data Fetching / Mutation Hooks
|
||||
|
||||
When hook extraction touches query or mutation code, do not use this reference as the source of truth for data-layer patterns.
|
||||
|
||||
- Do not introduce deprecated `useInvalid` / `useReset`.
|
||||
- Do not extract thin passthrough `useQuery` hooks; only extract orchestration hooks.
|
||||
|
||||
### 2. Form State Hook
|
||||
|
||||
```typescript
|
||||
// Pattern: Form state + validation + submission
|
||||
export const useConfigForm = (initialValues: ConfigFormValues) => {
|
||||
const [values, setValues] = useState(initialValues)
|
||||
const [errors, setErrors] = useState<Record<string, string>>({})
|
||||
const [isSubmitting, setIsSubmitting] = useState(false)
|
||||
|
||||
const validate = useCallback(() => {
|
||||
const newErrors: Record<string, string> = {}
|
||||
if (!values.name) newErrors.name = 'Name is required'
|
||||
setErrors(newErrors)
|
||||
return Object.keys(newErrors).length === 0
|
||||
}, [values])
|
||||
|
||||
const handleChange = useCallback((field: string, value: any) => {
|
||||
setValues((prev) => ({ ...prev, [field]: value }))
|
||||
}, [])
|
||||
|
||||
const handleSubmit = useCallback(
|
||||
async (onSubmit: (values: ConfigFormValues) => Promise<void>) => {
|
||||
if (!validate()) return
|
||||
setIsSubmitting(true)
|
||||
try {
|
||||
await onSubmit(values)
|
||||
} finally {
|
||||
setIsSubmitting(false)
|
||||
}
|
||||
},
|
||||
[values, validate],
|
||||
)
|
||||
|
||||
return { values, errors, isSubmitting, handleChange, handleSubmit }
|
||||
}
|
||||
```
|
||||
|
||||
### 3. Modal State Hook
|
||||
|
||||
```typescript
|
||||
// Pattern: Multiple modal management
|
||||
type ModalType = 'edit' | 'delete' | 'duplicate' | null
|
||||
|
||||
export const useModalState = () => {
|
||||
const [activeModal, setActiveModal] = useState<ModalType>(null)
|
||||
const [modalData, setModalData] = useState<any>(null)
|
||||
|
||||
const openModal = useCallback((type: ModalType, data?: any) => {
|
||||
setActiveModal(type)
|
||||
setModalData(data)
|
||||
}, [])
|
||||
|
||||
const closeModal = useCallback(() => {
|
||||
setActiveModal(null)
|
||||
setModalData(null)
|
||||
}, [])
|
||||
|
||||
return {
|
||||
activeModal,
|
||||
modalData,
|
||||
openModal,
|
||||
closeModal,
|
||||
isOpen: useCallback((type: ModalType) => activeModal === type, [activeModal]),
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### 4. Toggle/Boolean Hook
|
||||
|
||||
```typescript
|
||||
// Pattern: Boolean state with convenience methods
|
||||
export const useToggle = (initialValue = false) => {
|
||||
const [value, setValue] = useState(initialValue)
|
||||
|
||||
const toggle = useCallback(() => setValue((v) => !v), [])
|
||||
const setTrue = useCallback(() => setValue(true), [])
|
||||
const setFalse = useCallback(() => setValue(false), [])
|
||||
|
||||
return [value, { toggle, setTrue, setFalse, set: setValue }] as const
|
||||
}
|
||||
|
||||
// Usage
|
||||
const [isExpanded, { toggle, setTrue: expand, setFalse: collapse }] = useToggle()
|
||||
```
|
||||
|
||||
## Testing Extracted Hooks
|
||||
|
||||
After extraction, test hooks in isolation:
|
||||
|
||||
```typescript
|
||||
// use-model-config.spec.ts
|
||||
import { renderHook, act } from '@testing-library/react'
|
||||
import { useModelConfig } from './use-model-config'
|
||||
|
||||
describe('useModelConfig', () => {
|
||||
it('should initialize with default values', () => {
|
||||
const { result } = renderHook(() =>
|
||||
useModelConfig({
|
||||
hasFetchedDetail: false,
|
||||
}),
|
||||
)
|
||||
|
||||
expect(result.current.modelConfig.provider).toBe('langgenius/openai/openai')
|
||||
expect(result.current.modelModeType).toBe(ModelModeType.unset)
|
||||
})
|
||||
|
||||
it('should update model config', () => {
|
||||
const { result } = renderHook(() =>
|
||||
useModelConfig({
|
||||
hasFetchedDetail: true,
|
||||
}),
|
||||
)
|
||||
|
||||
act(() => {
|
||||
result.current.setModelConfig({
|
||||
...result.current.modelConfig,
|
||||
model_id: 'gpt-4',
|
||||
})
|
||||
})
|
||||
|
||||
expect(result.current.modelConfig.model_id).toBe('gpt-4')
|
||||
})
|
||||
})
|
||||
```
|
||||
@@ -5,7 +5,7 @@ description: Write, update, or review Dify end-to-end tests under `e2e/` that us
|
||||
|
||||
# Dify E2E Cucumber + Playwright
|
||||
|
||||
Use this skill for Dify's repository-level E2E suite in `e2e/`. Use [`e2e/AGENTS.md`](../../../e2e/AGENTS.md) as the canonical package guide for local architecture and conventions, then read any feature-scoped `AGENTS.md` that owns the target area. Apply Playwright/Cucumber best practices only where they fit the current suite.
|
||||
Use this skill for Dify's repository-level E2E suite in `e2e/`. Use [`e2e/AGENTS.md`](../../../e2e/AGENTS.md) as the canonical guide for local architecture and conventions, then apply Playwright/Cucumber best practices only where they fit the current suite.
|
||||
|
||||
## Scope
|
||||
|
||||
@@ -25,8 +25,6 @@ Use this skill for Dify's repository-level E2E suite in `e2e/`. Use [`e2e/AGENTS
|
||||
4. Read [`references/cucumber-best-practices.md`](references/cucumber-best-practices.md) only when scenario wording, step granularity, tags, or expression design are involved.
|
||||
5. Re-check official Playwright or Cucumber docs with the available documentation tools before introducing a new framework pattern.
|
||||
|
||||
Keep this skill focused on Cucumber, Playwright, and package-level E2E guidance. Put feature-specific conventions in the owning feature's `AGENTS.md` instead of adding them here.
|
||||
|
||||
## Local Rules
|
||||
|
||||
- `e2e/` uses Cucumber for scenarios and Playwright as the browser layer.
|
||||
@@ -45,7 +43,6 @@ Keep this skill focused on Cucumber, Playwright, and package-level E2E guidance.
|
||||
- Inspect the target feature area.
|
||||
- Reuse an existing step when wording and behavior already match.
|
||||
- Add a new step only for a genuinely new user action or assertion.
|
||||
- Before adding several similar steps, scan the target capability for an existing domain noun that can be parameterized without hiding behavior.
|
||||
- Keep edits close to the current capability folder unless the step is broadly reusable.
|
||||
2. Write behavior-first scenarios.
|
||||
- Describe user-observable behavior, not DOM mechanics.
|
||||
@@ -54,19 +51,15 @@ Keep this skill focused on Cucumber, Playwright, and package-level E2E guidance.
|
||||
3. Write step definitions in the local style.
|
||||
- Keep one step to one user-visible action or one assertion.
|
||||
- Prefer Cucumber Expressions such as `{string}` and `{int}`.
|
||||
- Use a bounded regex only when the accepted values are a small explicit domain set and Cucumber Expressions would make the Gherkin less natural.
|
||||
- Do not create one-off steps for each case variant when the same domain action or outcome applies to named surfaces, modes, or resources.
|
||||
- Scope locators to stable containers when the page has repeated elements.
|
||||
- Avoid page-object layers or extra helper abstractions unless repeated complexity clearly justifies them.
|
||||
4. Use Playwright in the local style.
|
||||
- Prefer user-facing locators: `getByRole`, `getByLabel`, `getByPlaceholder`, `getByText`, then `getByTestId` for explicit contracts.
|
||||
- Use web-first `expect(...)` assertions.
|
||||
- Do not use `waitForTimeout`, manual polling, or raw visibility checks when a locator action or retrying assertion already expresses the behavior.
|
||||
- Use `expect.poll` for API persistence, backend eventual consistency, captured browser events, or other non-DOM state; prefer locator assertions for DOM readiness and visible UI state.
|
||||
- If a product element has real user-facing semantics but no accessible name, prefer fixing that accessible contract over adding a test id.
|
||||
5. Validate narrowly.
|
||||
- Run the narrowest tagged scenario or flow that exercises the change.
|
||||
- Run `vpr lint --fix --quiet` from the repository root and `pnpm -C e2e type-check`.
|
||||
- Run `pnpm -C e2e check`.
|
||||
- Broaden verification only when the change affects hooks, tags, setup, or shared step semantics.
|
||||
|
||||
## Review Checklist
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
interface:
|
||||
display_name: "E2E Cucumber + Playwright"
|
||||
short_description: "Write and review Dify E2E scenarios."
|
||||
default_prompt: "Use $e2e-cucumber-playwright to write or review a Dify E2E scenario under e2e/."
|
||||
display_name: 'E2E Cucumber + Playwright'
|
||||
short_description: 'Write and review Dify E2E scenarios.'
|
||||
default_prompt: 'Use $e2e-cucumber-playwright to write or review a Dify E2E scenario under e2e/.'
|
||||
|
||||
@@ -39,7 +39,6 @@ Prefer reuse when:
|
||||
- the user action is genuinely the same
|
||||
- the expected outcome is genuinely the same
|
||||
- the wording stays natural across features
|
||||
- the parameter is a real product domain value such as a named surface, mode, resource, or status
|
||||
|
||||
Write a new step when:
|
||||
|
||||
@@ -47,8 +46,6 @@ Write a new step when:
|
||||
- reusing the old wording would make the scenario misleading
|
||||
- a supposedly generic step would become an implementation-detail wrapper
|
||||
|
||||
Do not optimize for a low step count by making vague steps. Optimize for a small set of truthful, domain-owned steps.
|
||||
|
||||
### 4. Prefer Cucumber Expressions
|
||||
|
||||
Use Cucumber Expressions for parameters unless regex is clearly necessary.
|
||||
@@ -62,8 +59,6 @@ Common examples:
|
||||
|
||||
Keep expressions readable. If a step needs complicated parsing logic, first ask whether the scenario wording should be simpler.
|
||||
|
||||
Use regex for a bounded natural-language alternative only when it keeps Gherkin readable, for example `/(Web app|Backend service API)/`. Avoid broad regexes that accept unowned language.
|
||||
|
||||
### 5. Keep step definitions thin and meaningful
|
||||
|
||||
Step definitions are glue between Gherkin and automation, not a second abstraction language.
|
||||
|
||||
@@ -41,7 +41,6 @@ Also remember:
|
||||
- repeated content usually needs scoping to a stable container
|
||||
- exact text matching is often too brittle when role/name or label already exists
|
||||
- `getByTestId` is acceptable when semantics are weak but the contract is intentional
|
||||
- when a real UI region, card, status, or icon lacks an accessible name, prefer adding that semantic contract in product code before falling back to `getByTestId`
|
||||
|
||||
### 3. Use web-first assertions
|
||||
|
||||
@@ -63,8 +62,6 @@ Avoid:
|
||||
|
||||
If a condition genuinely needs custom retry logic, use Playwright's polling/assertion tools deliberately and keep that choice local and explicit.
|
||||
|
||||
Use `expect.poll` for non-DOM truth such as API state, backend eventual consistency, generated resources, or captured browser events. For DOM state, use locator assertions so Playwright can apply actionability and web-first retry semantics.
|
||||
|
||||
### 4. Let actions wait for actionability
|
||||
|
||||
Locator actions already wait for the element to be actionable. Do not preface every click/fill with extra timing logic unless the action needs a specific visible/ready assertion for clarity.
|
||||
|
||||
@@ -40,13 +40,9 @@ Flag:
|
||||
- JavaScript conditional class logic for visual states that the Dify UI/Base UI primitive already exposes through `data-*` attributes or CSS variables.
|
||||
- Controlled props added when uncontrolled DOM state or CSS variables would be enough.
|
||||
- Thin wrappers that rename Base UI parts without adding semantics.
|
||||
- Generic Base UI selection primitives wrapped without preserving their value generics, such as `Select.Root<Value, Multiple>`, `RadioGroup<Value>`, or `Radio.Root<Value>`.
|
||||
- Shared select/radio option components that type selected values as `string` while callers pass enums, unions, booleans, numbers, objects, or nullable placeholder values.
|
||||
|
||||
Prefer Base UI/Dify UI data attributes and CSS variables for visual state: `data-open`, `data-checked`, `data-disabled`, `data-highlighted`, `data-popup-open`, `group-data-*`, `peer-data-*`, `has-[:focus-visible]`, and primitive CSS variables such as anchor width or transform origin. Use JS conditional classes for product/business state that the primitive does not expose.
|
||||
|
||||
For non-string `Select` and `RadioGroup` values, prefer explicit domain generics at the root and at child value carriers. JSX children do not inherit the parent generic, so `RadioGroup<PromptMode>` should compose with `Radio<PromptMode>`, `RadioRoot<PromptMode>`, or option values from a typed collection. For `Select`, prefer the Base UI `items` collection pattern for typed value-to-label rendering, and flag string coercion helpers used only to recover display labels.
|
||||
|
||||
## Forms
|
||||
|
||||
Flag:
|
||||
|
||||
@@ -93,20 +93,20 @@ describe('ComponentName', () => {
|
||||
it('should render without crashing', () => {
|
||||
// Arrange
|
||||
const props = { title: 'Test' }
|
||||
|
||||
|
||||
// Act
|
||||
render(<Component {...props} />)
|
||||
|
||||
|
||||
// Assert
|
||||
expect(screen.getByText('Test')).toBeInTheDocument()
|
||||
})
|
||||
})
|
||||
|
||||
// Props tests (REQUIRED when props change observable behavior)
|
||||
// Props tests (REQUIRED)
|
||||
describe('Props', () => {
|
||||
it('should disable the action when disabled', () => {
|
||||
render(<Component disabled />)
|
||||
expect(screen.getByRole('button')).toBeDisabled()
|
||||
it('should apply custom className', () => {
|
||||
render(<Component className="custom" />)
|
||||
expect(screen.getByRole('button')).toHaveClass('custom')
|
||||
})
|
||||
})
|
||||
|
||||
@@ -115,9 +115,9 @@ describe('ComponentName', () => {
|
||||
it('should handle click events', () => {
|
||||
const handleClick = vi.fn()
|
||||
render(<Component onClick={handleClick} />)
|
||||
|
||||
|
||||
fireEvent.click(screen.getByRole('button'))
|
||||
|
||||
|
||||
expect(handleClick).toHaveBeenCalledTimes(1)
|
||||
})
|
||||
})
|
||||
@@ -220,7 +220,6 @@ Every test should clearly separate:
|
||||
### 2. Black-Box Testing
|
||||
|
||||
- Test observable behavior, not implementation details
|
||||
- Test product contracts, not cosmetic implementation. Do not add or expand unit tests only to lock pure style classes, spacing, colors, backgrounds, or layout micro-adjustments. Cover visual-only fixes with browser/manual verification, screenshots, or E2E/visual checks when risk justifies it. Add unit tests only when the change affects user-observable behavior, accessibility semantics, state, data flow, routing, or a stable component API contract.
|
||||
- Use semantic queries (`getByRole` with accessible `name`, `getByLabelText`, `getByPlaceholderText`, `getByText`, and scoped `within(...)`)
|
||||
- Treat `getByTestId` as a last resort. If a control cannot be found by role/name, label, landmark, or dialog scope, fix the component accessibility first instead of adding or relying on `data-testid`.
|
||||
- Remove production `data-testid` attributes when semantic selectors can cover the behavior. Keep them only for non-visual mocked boundaries, editor/browser shims such as Monaco, canvas/chart output, or third-party widgets with no accessible DOM in the test environment.
|
||||
@@ -274,21 +273,21 @@ it('should disable input when isReadOnly is true')
|
||||
### Always Required (All Components)
|
||||
|
||||
1. **Rendering**: Component renders without crashing
|
||||
1. **Props**: Required props, optional props, default values that change observable behavior. Do not test pass-through styling props such as `className` unless they are an explicit, stable component API whose absence would break a real integration contract.
|
||||
1. **Props**: Required props, optional props, default values
|
||||
1. **Edge Cases**: null, undefined, empty values, boundary conditions
|
||||
|
||||
### Conditional (When Present)
|
||||
|
||||
| Feature | Test Focus |
|
||||
|---------|-----------|
|
||||
| `useState` | Initial state, transitions, cleanup |
|
||||
| `useEffect` | Execution, dependencies, cleanup |
|
||||
| Event handlers | All onClick, onChange, onSubmit, keyboard |
|
||||
| API calls | Loading, success, error states |
|
||||
| Routing | Navigation, params, query strings |
|
||||
| `useCallback`/`useMemo` | Referential equality |
|
||||
| Context | Provider values, consumer behavior |
|
||||
| Forms | Validation, submission, error display |
|
||||
| Feature | Test Focus |
|
||||
| ----------------------- | ----------------------------------------- |
|
||||
| `useState` | Initial state, transitions, cleanup |
|
||||
| `useEffect` | Execution, dependencies, cleanup |
|
||||
| Event handlers | All onClick, onChange, onSubmit, keyboard |
|
||||
| API calls | Loading, success, error states |
|
||||
| Routing | Navigation, params, query strings |
|
||||
| `useCallback`/`useMemo` | Referential equality |
|
||||
| Context | Provider values, consumer behavior |
|
||||
| Forms | Validation, submission, error display |
|
||||
|
||||
## Coverage Goals (Per File)
|
||||
|
||||
|
||||
@@ -108,10 +108,8 @@ describe('ComponentName', () => {
|
||||
it('should render without crashing', () => {
|
||||
// Arrange - Setup data and mocks
|
||||
// const props = createMockProps()
|
||||
|
||||
// Act - Render the component
|
||||
// render(<ComponentName {...props} />)
|
||||
|
||||
// Assert - Verify expected output
|
||||
// Prefer getByRole for accessibility; it's what users "see"
|
||||
// expect(screen.getByRole('...')).toBeInTheDocument()
|
||||
|
||||
@@ -9,7 +9,7 @@ import { render, screen, waitFor } from '@testing-library/react'
|
||||
|
||||
it('should load and display data', async () => {
|
||||
render(<DataComponent />)
|
||||
|
||||
|
||||
// Wait for element to appear
|
||||
await waitFor(() => {
|
||||
expect(screen.getByText('Loaded Data')).toBeInTheDocument()
|
||||
@@ -18,7 +18,7 @@ it('should load and display data', async () => {
|
||||
|
||||
it('should hide loading spinner after load', async () => {
|
||||
render(<DataComponent />)
|
||||
|
||||
|
||||
// Wait for element to disappear
|
||||
await waitFor(() => {
|
||||
expect(screen.queryByText('Loading...')).not.toBeInTheDocument()
|
||||
@@ -31,11 +31,11 @@ it('should hide loading spinner after load', async () => {
|
||||
```typescript
|
||||
it('should show user name after fetch', async () => {
|
||||
render(<UserProfile />)
|
||||
|
||||
|
||||
// findBy returns a promise, auto-waits up to 1000ms
|
||||
const userName = await screen.findByText('John Doe')
|
||||
expect(userName).toBeInTheDocument()
|
||||
|
||||
|
||||
// findByRole with options
|
||||
const button = await screen.findByRole('button', { name: /submit/i })
|
||||
expect(button).toBeEnabled()
|
||||
@@ -50,13 +50,13 @@ import userEvent from '@testing-library/user-event'
|
||||
it('should submit form', async () => {
|
||||
const user = userEvent.setup()
|
||||
const onSubmit = vi.fn()
|
||||
|
||||
|
||||
render(<Form onSubmit={onSubmit} />)
|
||||
|
||||
|
||||
// userEvent methods are async
|
||||
await user.type(screen.getByLabelText('Email'), 'test@example.com')
|
||||
await user.click(screen.getByRole('button', { name: /submit/i }))
|
||||
|
||||
|
||||
await waitFor(() => {
|
||||
expect(onSubmit).toHaveBeenCalledWith({ email: 'test@example.com' })
|
||||
})
|
||||
@@ -87,16 +87,16 @@ describe('Debounced Search', () => {
|
||||
it('should debounce search input', async () => {
|
||||
const onSearch = vi.fn()
|
||||
render(<SearchInput onSearch={onSearch} debounceMs={300} />)
|
||||
|
||||
|
||||
// Type in the input
|
||||
fireEvent.change(screen.getByRole('textbox'), { target: { value: 'query' } })
|
||||
|
||||
|
||||
// Search not called immediately
|
||||
expect(onSearch).not.toHaveBeenCalled()
|
||||
|
||||
|
||||
// Advance timers
|
||||
vi.advanceTimersByTime(300)
|
||||
|
||||
|
||||
// Now search is called
|
||||
expect(onSearch).toHaveBeenCalledWith('query')
|
||||
})
|
||||
@@ -111,23 +111,23 @@ it('should retry on failure', async () => {
|
||||
const fetchData = vi.fn()
|
||||
.mockRejectedValueOnce(new Error('Network error'))
|
||||
.mockResolvedValueOnce({ data: 'success' })
|
||||
|
||||
|
||||
render(<RetryComponent fetchData={fetchData} retryDelayMs={1000} />)
|
||||
|
||||
|
||||
// First call fails
|
||||
await waitFor(() => {
|
||||
expect(fetchData).toHaveBeenCalledTimes(1)
|
||||
})
|
||||
|
||||
|
||||
// Advance timer for retry
|
||||
vi.advanceTimersByTime(1000)
|
||||
|
||||
|
||||
// Second call succeeds
|
||||
await waitFor(() => {
|
||||
expect(fetchData).toHaveBeenCalledTimes(2)
|
||||
expect(screen.getByText('success')).toBeInTheDocument()
|
||||
})
|
||||
|
||||
|
||||
vi.useRealTimers()
|
||||
})
|
||||
```
|
||||
@@ -163,31 +163,31 @@ describe('DataFetcher', () => {
|
||||
|
||||
it('should show loading state', () => {
|
||||
mockedApi.fetchData.mockImplementation(() => new Promise(() => {})) // Never resolves
|
||||
|
||||
|
||||
render(<DataFetcher />)
|
||||
|
||||
|
||||
expect(screen.getByTestId('loading-spinner')).toBeInTheDocument()
|
||||
})
|
||||
|
||||
it('should show data on success', async () => {
|
||||
mockedApi.fetchData.mockResolvedValue({ items: ['Item 1', 'Item 2'] })
|
||||
|
||||
|
||||
render(<DataFetcher />)
|
||||
|
||||
|
||||
// Use findBy* for multiple async elements (better error messages than waitFor with multiple assertions)
|
||||
const item1 = await screen.findByText('Item 1')
|
||||
const item2 = await screen.findByText('Item 2')
|
||||
expect(item1).toBeInTheDocument()
|
||||
expect(item2).toBeInTheDocument()
|
||||
|
||||
|
||||
expect(screen.queryByTestId('loading-spinner')).not.toBeInTheDocument()
|
||||
})
|
||||
|
||||
it('should show error on failure', async () => {
|
||||
mockedApi.fetchData.mockRejectedValue(new Error('Failed to fetch'))
|
||||
|
||||
|
||||
render(<DataFetcher />)
|
||||
|
||||
|
||||
await waitFor(() => {
|
||||
expect(screen.getByText(/failed to fetch/i)).toBeInTheDocument()
|
||||
})
|
||||
@@ -195,16 +195,16 @@ describe('DataFetcher', () => {
|
||||
|
||||
it('should retry on error', async () => {
|
||||
mockedApi.fetchData.mockRejectedValue(new Error('Network error'))
|
||||
|
||||
|
||||
render(<DataFetcher />)
|
||||
|
||||
|
||||
await waitFor(() => {
|
||||
expect(screen.getByRole('button', { name: /retry/i })).toBeInTheDocument()
|
||||
})
|
||||
|
||||
|
||||
mockedApi.fetchData.mockResolvedValue({ items: ['Item 1'] })
|
||||
fireEvent.click(screen.getByRole('button', { name: /retry/i }))
|
||||
|
||||
|
||||
await waitFor(() => {
|
||||
expect(screen.getByText('Item 1')).toBeInTheDocument()
|
||||
})
|
||||
@@ -218,19 +218,19 @@ describe('DataFetcher', () => {
|
||||
it('should submit form and show success', async () => {
|
||||
const user = userEvent.setup()
|
||||
mockedApi.createItem.mockResolvedValue({ id: '1', name: 'New Item' })
|
||||
|
||||
|
||||
render(<CreateItemForm />)
|
||||
|
||||
|
||||
await user.type(screen.getByLabelText('Name'), 'New Item')
|
||||
await user.click(screen.getByRole('button', { name: /create/i }))
|
||||
|
||||
|
||||
// Button should be disabled during submission
|
||||
expect(screen.getByRole('button', { name: /creating/i })).toBeDisabled()
|
||||
|
||||
|
||||
await waitFor(() => {
|
||||
expect(screen.getByText(/created successfully/i)).toBeInTheDocument()
|
||||
})
|
||||
|
||||
|
||||
expect(mockedApi.createItem).toHaveBeenCalledWith({ name: 'New Item' })
|
||||
})
|
||||
```
|
||||
@@ -242,9 +242,9 @@ it('should submit form and show success', async () => {
|
||||
```typescript
|
||||
it('should fetch data on mount', async () => {
|
||||
const fetchData = vi.fn().mockResolvedValue({ data: 'test' })
|
||||
|
||||
|
||||
render(<ComponentWithEffect fetchData={fetchData} />)
|
||||
|
||||
|
||||
await waitFor(() => {
|
||||
expect(fetchData).toHaveBeenCalledTimes(1)
|
||||
})
|
||||
@@ -256,15 +256,15 @@ it('should fetch data on mount', async () => {
|
||||
```typescript
|
||||
it('should refetch when id changes', async () => {
|
||||
const fetchData = vi.fn().mockResolvedValue({ data: 'test' })
|
||||
|
||||
|
||||
const { rerender } = render(<ComponentWithEffect id="1" fetchData={fetchData} />)
|
||||
|
||||
|
||||
await waitFor(() => {
|
||||
expect(fetchData).toHaveBeenCalledWith('1')
|
||||
})
|
||||
|
||||
|
||||
rerender(<ComponentWithEffect id="2" fetchData={fetchData} />)
|
||||
|
||||
|
||||
await waitFor(() => {
|
||||
expect(fetchData).toHaveBeenCalledWith('2')
|
||||
expect(fetchData).toHaveBeenCalledTimes(2)
|
||||
@@ -279,13 +279,13 @@ it('should cleanup subscription on unmount', () => {
|
||||
const subscribe = vi.fn()
|
||||
const unsubscribe = vi.fn()
|
||||
subscribe.mockReturnValue(unsubscribe)
|
||||
|
||||
|
||||
const { unmount } = render(<SubscriptionComponent subscribe={subscribe} />)
|
||||
|
||||
|
||||
expect(subscribe).toHaveBeenCalledTimes(1)
|
||||
|
||||
|
||||
unmount()
|
||||
|
||||
|
||||
expect(unsubscribe).toHaveBeenCalledTimes(1)
|
||||
})
|
||||
```
|
||||
|
||||
@@ -51,16 +51,16 @@ Use this checklist when generating or reviewing tests for Dify frontend componen
|
||||
|
||||
### Conditional Sections (Add When Feature Present)
|
||||
|
||||
| Feature | Add Tests For |
|
||||
|---------|---------------|
|
||||
| `useState` | Initial state, transitions, cleanup |
|
||||
| `useEffect` | Execution, dependencies, cleanup |
|
||||
| Event handlers | onClick, onChange, onSubmit, keyboard |
|
||||
| API calls | Loading, success, error states |
|
||||
| Routing | Navigation, params, query strings |
|
||||
| `useCallback`/`useMemo` | Referential equality |
|
||||
| Context | Provider values, consumer behavior |
|
||||
| Forms | Validation, submission, error display |
|
||||
| Feature | Add Tests For |
|
||||
| ----------------------- | ------------------------------------- |
|
||||
| `useState` | Initial state, transitions, cleanup |
|
||||
| `useEffect` | Execution, dependencies, cleanup |
|
||||
| Event handlers | onClick, onChange, onSubmit, keyboard |
|
||||
| API calls | Loading, success, error states |
|
||||
| Routing | Navigation, params, query strings |
|
||||
| `useCallback`/`useMemo` | Referential equality |
|
||||
| Context | Provider values, consumer behavior |
|
||||
| Forms | Validation, submission, error display |
|
||||
|
||||
## Code Quality Checklist
|
||||
|
||||
@@ -110,8 +110,8 @@ For the current file being tested:
|
||||
|
||||
- [ ] 100% function coverage
|
||||
- [ ] 100% statement coverage
|
||||
- [ ] >95% branch coverage
|
||||
- [ ] >95% line coverage
|
||||
- [ ] > 95% branch coverage
|
||||
- [ ] > 95% line coverage
|
||||
|
||||
## Post-Generation (Per File)
|
||||
|
||||
|
||||
@@ -107,13 +107,13 @@ describe('Counter', () => {
|
||||
it('should increment count', async () => {
|
||||
const user = userEvent.setup()
|
||||
render(<Counter initialCount={0} />)
|
||||
|
||||
|
||||
// Initial state
|
||||
expect(screen.getByText('Count: 0')).toBeInTheDocument()
|
||||
|
||||
|
||||
// Trigger transition
|
||||
await user.click(screen.getByRole('button', { name: /increment/i }))
|
||||
|
||||
|
||||
// New state
|
||||
expect(screen.getByText('Count: 1')).toBeInTheDocument()
|
||||
})
|
||||
@@ -127,17 +127,17 @@ describe('ControlledInput', () => {
|
||||
it('should call onChange with new value', async () => {
|
||||
const user = userEvent.setup()
|
||||
const handleChange = vi.fn()
|
||||
|
||||
|
||||
render(<ControlledInput value="" onChange={handleChange} />)
|
||||
|
||||
|
||||
await user.type(screen.getByRole('textbox'), 'a')
|
||||
|
||||
|
||||
expect(handleChange).toHaveBeenCalledWith('a')
|
||||
})
|
||||
|
||||
it('should display controlled value', () => {
|
||||
render(<ControlledInput value="controlled" onChange={vi.fn()} />)
|
||||
|
||||
|
||||
expect(screen.getByRole('textbox')).toHaveValue('controlled')
|
||||
})
|
||||
})
|
||||
@@ -149,26 +149,26 @@ describe('ControlledInput', () => {
|
||||
describe('ConditionalComponent', () => {
|
||||
it('should show loading state', () => {
|
||||
render(<DataDisplay isLoading={true} data={null} />)
|
||||
|
||||
|
||||
expect(screen.getByText(/loading/i)).toBeInTheDocument()
|
||||
expect(screen.queryByTestId('data-content')).not.toBeInTheDocument()
|
||||
})
|
||||
|
||||
it('should show error state', () => {
|
||||
render(<DataDisplay isLoading={false} data={null} error="Failed to load" />)
|
||||
|
||||
|
||||
expect(screen.getByText(/failed to load/i)).toBeInTheDocument()
|
||||
})
|
||||
|
||||
it('should show data when loaded', () => {
|
||||
render(<DataDisplay isLoading={false} data={{ name: 'Test' }} />)
|
||||
|
||||
|
||||
expect(screen.getByText('Test')).toBeInTheDocument()
|
||||
})
|
||||
|
||||
it('should show empty state when no data', () => {
|
||||
render(<DataDisplay isLoading={false} data={[]} />)
|
||||
|
||||
|
||||
expect(screen.getByText(/no data/i)).toBeInTheDocument()
|
||||
})
|
||||
})
|
||||
@@ -186,7 +186,7 @@ describe('ItemList', () => {
|
||||
|
||||
it('should render all items', () => {
|
||||
render(<ItemList items={items} />)
|
||||
|
||||
|
||||
expect(screen.getAllByRole('listitem')).toHaveLength(3)
|
||||
items.forEach(item => {
|
||||
expect(screen.getByText(item.name)).toBeInTheDocument()
|
||||
@@ -196,17 +196,17 @@ describe('ItemList', () => {
|
||||
it('should handle item selection', async () => {
|
||||
const user = userEvent.setup()
|
||||
const onSelect = vi.fn()
|
||||
|
||||
|
||||
render(<ItemList items={items} onSelect={onSelect} />)
|
||||
|
||||
|
||||
await user.click(screen.getByText('Item 2'))
|
||||
|
||||
|
||||
expect(onSelect).toHaveBeenCalledWith(items[1])
|
||||
})
|
||||
|
||||
it('should handle empty list', () => {
|
||||
render(<ItemList items={[]} />)
|
||||
|
||||
|
||||
expect(screen.getByText(/no items/i)).toBeInTheDocument()
|
||||
})
|
||||
})
|
||||
@@ -218,55 +218,55 @@ describe('ItemList', () => {
|
||||
describe('Modal', () => {
|
||||
it('should not render when closed', () => {
|
||||
render(<Modal isOpen={false} onClose={vi.fn()} />)
|
||||
|
||||
|
||||
expect(screen.queryByRole('dialog')).not.toBeInTheDocument()
|
||||
})
|
||||
|
||||
it('should render when open', () => {
|
||||
render(<Modal isOpen={true} onClose={vi.fn()} />)
|
||||
|
||||
|
||||
expect(screen.getByRole('dialog')).toBeInTheDocument()
|
||||
})
|
||||
|
||||
it('should call onClose when clicking overlay', async () => {
|
||||
const user = userEvent.setup()
|
||||
const handleClose = vi.fn()
|
||||
|
||||
|
||||
render(<Modal isOpen={true} onClose={handleClose} />)
|
||||
|
||||
|
||||
await user.click(screen.getByTestId('modal-overlay'))
|
||||
|
||||
|
||||
expect(handleClose).toHaveBeenCalled()
|
||||
})
|
||||
|
||||
it('should call onClose when pressing Escape', async () => {
|
||||
const user = userEvent.setup()
|
||||
const handleClose = vi.fn()
|
||||
|
||||
|
||||
render(<Modal isOpen={true} onClose={handleClose} />)
|
||||
|
||||
|
||||
await user.keyboard('{Escape}')
|
||||
|
||||
|
||||
expect(handleClose).toHaveBeenCalled()
|
||||
})
|
||||
|
||||
it('should trap focus inside modal', async () => {
|
||||
const user = userEvent.setup()
|
||||
|
||||
|
||||
render(
|
||||
<Modal isOpen={true} onClose={vi.fn()}>
|
||||
<button>First</button>
|
||||
<button>Second</button>
|
||||
</Modal>
|
||||
)
|
||||
|
||||
|
||||
// Focus should cycle within modal
|
||||
await user.tab()
|
||||
expect(screen.getByText('First')).toHaveFocus()
|
||||
|
||||
|
||||
await user.tab()
|
||||
expect(screen.getByText('Second')).toHaveFocus()
|
||||
|
||||
|
||||
await user.tab()
|
||||
expect(screen.getByText('First')).toHaveFocus() // Cycles back
|
||||
})
|
||||
@@ -280,13 +280,13 @@ describe('LoginForm', () => {
|
||||
it('should submit valid form', async () => {
|
||||
const user = userEvent.setup()
|
||||
const onSubmit = vi.fn()
|
||||
|
||||
|
||||
render(<LoginForm onSubmit={onSubmit} />)
|
||||
|
||||
|
||||
await user.type(screen.getByLabelText(/email/i), 'test@example.com')
|
||||
await user.type(screen.getByLabelText(/password/i), 'password123')
|
||||
await user.click(screen.getByRole('button', { name: /sign in/i }))
|
||||
|
||||
|
||||
expect(onSubmit).toHaveBeenCalledWith({
|
||||
email: 'test@example.com',
|
||||
password: 'password123',
|
||||
@@ -295,39 +295,39 @@ describe('LoginForm', () => {
|
||||
|
||||
it('should show validation errors', async () => {
|
||||
const user = userEvent.setup()
|
||||
|
||||
|
||||
render(<LoginForm onSubmit={vi.fn()} />)
|
||||
|
||||
|
||||
// Submit empty form
|
||||
await user.click(screen.getByRole('button', { name: /sign in/i }))
|
||||
|
||||
|
||||
expect(screen.getByText(/email is required/i)).toBeInTheDocument()
|
||||
expect(screen.getByText(/password is required/i)).toBeInTheDocument()
|
||||
})
|
||||
|
||||
it('should validate email format', async () => {
|
||||
const user = userEvent.setup()
|
||||
|
||||
|
||||
render(<LoginForm onSubmit={vi.fn()} />)
|
||||
|
||||
|
||||
await user.type(screen.getByLabelText(/email/i), 'invalid-email')
|
||||
await user.click(screen.getByRole('button', { name: /sign in/i }))
|
||||
|
||||
|
||||
expect(screen.getByText(/invalid email/i)).toBeInTheDocument()
|
||||
})
|
||||
|
||||
it('should disable submit button while submitting', async () => {
|
||||
const user = userEvent.setup()
|
||||
const onSubmit = vi.fn(() => new Promise(resolve => setTimeout(resolve, 100)))
|
||||
|
||||
|
||||
render(<LoginForm onSubmit={onSubmit} />)
|
||||
|
||||
|
||||
await user.type(screen.getByLabelText(/email/i), 'test@example.com')
|
||||
await user.type(screen.getByLabelText(/password/i), 'password123')
|
||||
await user.click(screen.getByRole('button', { name: /sign in/i }))
|
||||
|
||||
|
||||
expect(screen.getByRole('button', { name: /signing in/i })).toBeDisabled()
|
||||
|
||||
|
||||
await waitFor(() => {
|
||||
expect(screen.getByRole('button', { name: /sign in/i })).toBeEnabled()
|
||||
})
|
||||
@@ -346,7 +346,7 @@ describe('StatusBadge', () => {
|
||||
['info', 'bg-blue-500'],
|
||||
])('should apply correct class for %s status', (status, expectedClass) => {
|
||||
render(<StatusBadge status={status} />)
|
||||
|
||||
|
||||
expect(screen.getByTestId('status-badge')).toHaveClass(expectedClass)
|
||||
})
|
||||
|
||||
@@ -357,7 +357,7 @@ describe('StatusBadge', () => {
|
||||
{ input: 'invalid', expected: 'Unknown' },
|
||||
])('should show "Unknown" for invalid input: $input', ({ input, expected }) => {
|
||||
render(<StatusBadge status={input} />)
|
||||
|
||||
|
||||
expect(screen.getByText(expected)).toBeInTheDocument()
|
||||
})
|
||||
})
|
||||
|
||||
@@ -53,18 +53,18 @@ describe('NodeConfigPanel', () => {
|
||||
it('should render node type selector', () => {
|
||||
const node = createMockNode({ type: 'llm' })
|
||||
render(<NodeConfigPanel node={node} />)
|
||||
|
||||
|
||||
expect(screen.getByLabelText(/model/i)).toBeInTheDocument()
|
||||
})
|
||||
|
||||
it('should update node config on change', async () => {
|
||||
const user = userEvent.setup()
|
||||
const node = createMockNode({ type: 'llm' })
|
||||
|
||||
|
||||
render(<NodeConfigPanel node={node} />)
|
||||
|
||||
|
||||
await user.selectOptions(screen.getByLabelText(/model/i), 'gpt-4')
|
||||
|
||||
|
||||
expect(mockWorkflowStore.updateNode).toHaveBeenCalledWith(
|
||||
node.id,
|
||||
expect.objectContaining({ model: 'gpt-4' })
|
||||
@@ -76,14 +76,14 @@ describe('NodeConfigPanel', () => {
|
||||
it('should show error for invalid input', async () => {
|
||||
const user = userEvent.setup()
|
||||
const node = createMockNode({ type: 'code' })
|
||||
|
||||
|
||||
render(<NodeConfigPanel node={node} />)
|
||||
|
||||
|
||||
// Enter invalid code
|
||||
const codeInput = screen.getByLabelText(/code/i)
|
||||
await user.clear(codeInput)
|
||||
await user.type(codeInput, 'invalid syntax {{{')
|
||||
|
||||
|
||||
await waitFor(() => {
|
||||
expect(screen.getByText(/syntax error/i)).toBeInTheDocument()
|
||||
})
|
||||
@@ -91,11 +91,11 @@ describe('NodeConfigPanel', () => {
|
||||
|
||||
it('should validate required fields', async () => {
|
||||
const node = createMockNode({ type: 'http', data: { url: '' } })
|
||||
|
||||
|
||||
render(<NodeConfigPanel node={node} />)
|
||||
|
||||
|
||||
fireEvent.click(screen.getByRole('button', { name: /save/i }))
|
||||
|
||||
|
||||
await waitFor(() => {
|
||||
expect(screen.getByText(/url is required/i)).toBeInTheDocument()
|
||||
})
|
||||
@@ -113,28 +113,28 @@ describe('NodeConfigPanel', () => {
|
||||
id: 'node-2',
|
||||
type: 'llm',
|
||||
})
|
||||
|
||||
|
||||
mockWorkflowStore.nodes = [upstreamNode, currentNode]
|
||||
mockWorkflowStore.edges = [{ source: 'node-1', target: 'node-2' }]
|
||||
|
||||
|
||||
render(<NodeConfigPanel node={currentNode} />)
|
||||
|
||||
|
||||
// Variable selector should show upstream variables
|
||||
fireEvent.click(screen.getByRole('button', { name: /add variable/i }))
|
||||
|
||||
|
||||
expect(screen.getByText('user_input')).toBeInTheDocument()
|
||||
})
|
||||
|
||||
it('should insert variable into prompt template', async () => {
|
||||
const user = userEvent.setup()
|
||||
const node = createMockNode({ type: 'llm' })
|
||||
|
||||
|
||||
render(<NodeConfigPanel node={node} />)
|
||||
|
||||
|
||||
// Click variable button
|
||||
await user.click(screen.getByRole('button', { name: /insert variable/i }))
|
||||
await user.click(screen.getByText('user_input'))
|
||||
|
||||
|
||||
const promptInput = screen.getByLabelText(/prompt/i)
|
||||
expect(promptInput).toHaveValue(expect.stringContaining('{{user_input}}'))
|
||||
})
|
||||
@@ -179,14 +179,14 @@ describe('DocumentUploader', () => {
|
||||
const user = userEvent.setup()
|
||||
const onUpload = vi.fn()
|
||||
mockedService.uploadDocument.mockResolvedValue({ id: 'doc-1' })
|
||||
|
||||
|
||||
render(<DocumentUploader onUpload={onUpload} />)
|
||||
|
||||
|
||||
const file = new File(['content'], 'test.pdf', { type: 'application/pdf' })
|
||||
const input = screen.getByLabelText(/upload/i)
|
||||
|
||||
|
||||
await user.upload(input, file)
|
||||
|
||||
|
||||
await waitFor(() => {
|
||||
expect(mockedService.uploadDocument).toHaveBeenCalledWith(
|
||||
expect.any(FormData)
|
||||
@@ -196,35 +196,35 @@ describe('DocumentUploader', () => {
|
||||
|
||||
it('should reject invalid file types', async () => {
|
||||
const user = userEvent.setup()
|
||||
|
||||
|
||||
render(<DocumentUploader />)
|
||||
|
||||
|
||||
const file = new File(['content'], 'test.exe', { type: 'application/x-msdownload' })
|
||||
const input = screen.getByLabelText(/upload/i)
|
||||
|
||||
|
||||
await user.upload(input, file)
|
||||
|
||||
|
||||
expect(screen.getByText(/unsupported file type/i)).toBeInTheDocument()
|
||||
expect(mockedService.uploadDocument).not.toHaveBeenCalled()
|
||||
})
|
||||
|
||||
it('should show upload progress', async () => {
|
||||
const user = userEvent.setup()
|
||||
|
||||
|
||||
// Mock upload with progress
|
||||
mockedService.uploadDocument.mockImplementation(() => {
|
||||
return new Promise((resolve) => {
|
||||
setTimeout(() => resolve({ id: 'doc-1' }), 100)
|
||||
})
|
||||
})
|
||||
|
||||
|
||||
render(<DocumentUploader />)
|
||||
|
||||
|
||||
const file = new File(['content'], 'test.pdf', { type: 'application/pdf' })
|
||||
await user.upload(screen.getByLabelText(/upload/i), file)
|
||||
|
||||
|
||||
expect(screen.getByRole('progressbar')).toBeInTheDocument()
|
||||
|
||||
|
||||
await waitFor(() => {
|
||||
expect(screen.queryByRole('progressbar')).not.toBeInTheDocument()
|
||||
})
|
||||
@@ -235,12 +235,12 @@ describe('DocumentUploader', () => {
|
||||
it('should handle upload failure', async () => {
|
||||
const user = userEvent.setup()
|
||||
mockedService.uploadDocument.mockRejectedValue(new Error('Upload failed'))
|
||||
|
||||
|
||||
render(<DocumentUploader />)
|
||||
|
||||
|
||||
const file = new File(['content'], 'test.pdf', { type: 'application/pdf' })
|
||||
await user.upload(screen.getByLabelText(/upload/i), file)
|
||||
|
||||
|
||||
await waitFor(() => {
|
||||
expect(screen.getByText(/upload failed/i)).toBeInTheDocument()
|
||||
})
|
||||
@@ -251,18 +251,18 @@ describe('DocumentUploader', () => {
|
||||
mockedService.uploadDocument
|
||||
.mockRejectedValueOnce(new Error('Network error'))
|
||||
.mockResolvedValueOnce({ id: 'doc-1' })
|
||||
|
||||
|
||||
render(<DocumentUploader />)
|
||||
|
||||
|
||||
const file = new File(['content'], 'test.pdf', { type: 'application/pdf' })
|
||||
await user.upload(screen.getByLabelText(/upload/i), file)
|
||||
|
||||
|
||||
await waitFor(() => {
|
||||
expect(screen.getByRole('button', { name: /retry/i })).toBeInTheDocument()
|
||||
})
|
||||
|
||||
|
||||
await user.click(screen.getByRole('button', { name: /retry/i }))
|
||||
|
||||
|
||||
await waitFor(() => {
|
||||
expect(screen.getByText(/uploaded successfully/i)).toBeInTheDocument()
|
||||
})
|
||||
@@ -283,13 +283,13 @@ describe('DocumentList', () => {
|
||||
page: 1,
|
||||
pageSize: 10,
|
||||
})
|
||||
|
||||
|
||||
render(<DocumentList datasetId="ds-1" />)
|
||||
|
||||
|
||||
await waitFor(() => {
|
||||
expect(screen.getByText('Doc 1')).toBeInTheDocument()
|
||||
})
|
||||
|
||||
|
||||
expect(mockedService.getDocuments).toHaveBeenCalledWith('ds-1', { page: 1 })
|
||||
})
|
||||
|
||||
@@ -301,22 +301,22 @@ describe('DocumentList', () => {
|
||||
page: 1,
|
||||
pageSize: 10,
|
||||
})
|
||||
|
||||
|
||||
render(<DocumentList datasetId="ds-1" />)
|
||||
|
||||
|
||||
await waitFor(() => {
|
||||
expect(screen.getByText('Doc 1')).toBeInTheDocument()
|
||||
})
|
||||
|
||||
|
||||
mockedService.getDocuments.mockResolvedValue({
|
||||
data: [{ id: '11', name: 'Doc 11' }],
|
||||
total: 50,
|
||||
page: 2,
|
||||
pageSize: 10,
|
||||
})
|
||||
|
||||
|
||||
await user.click(screen.getByRole('button', { name: /next/i }))
|
||||
|
||||
|
||||
await waitFor(() => {
|
||||
expect(screen.getByText('Doc 11')).toBeInTheDocument()
|
||||
})
|
||||
@@ -327,21 +327,21 @@ describe('DocumentList', () => {
|
||||
it('should filter by search query', async () => {
|
||||
const user = userEvent.setup()
|
||||
vi.useFakeTimers()
|
||||
|
||||
|
||||
render(<DocumentList datasetId="ds-1" />)
|
||||
|
||||
|
||||
await user.type(screen.getByPlaceholderText(/search/i), 'test query')
|
||||
|
||||
|
||||
// Debounce
|
||||
vi.advanceTimersByTime(300)
|
||||
|
||||
|
||||
await waitFor(() => {
|
||||
expect(mockedService.getDocuments).toHaveBeenCalledWith(
|
||||
'ds-1',
|
||||
expect.objectContaining({ search: 'test query' })
|
||||
)
|
||||
})
|
||||
|
||||
|
||||
vi.useRealTimers()
|
||||
})
|
||||
})
|
||||
@@ -391,50 +391,50 @@ describe('AppConfigForm', () => {
|
||||
describe('Form Validation', () => {
|
||||
it('should require app name', async () => {
|
||||
const user = userEvent.setup()
|
||||
|
||||
|
||||
render(<AppConfigForm appId="app-1" />)
|
||||
|
||||
|
||||
await waitFor(() => {
|
||||
expect(screen.getByLabelText(/name/i)).toHaveValue('My App')
|
||||
})
|
||||
|
||||
|
||||
// Clear name field
|
||||
await user.clear(screen.getByLabelText(/name/i))
|
||||
await user.click(screen.getByRole('button', { name: /save/i }))
|
||||
|
||||
|
||||
expect(screen.getByText(/name is required/i)).toBeInTheDocument()
|
||||
expect(mockedService.updateAppConfig).not.toHaveBeenCalled()
|
||||
})
|
||||
|
||||
it('should validate name length', async () => {
|
||||
const user = userEvent.setup()
|
||||
|
||||
|
||||
render(<AppConfigForm appId="app-1" />)
|
||||
|
||||
|
||||
await waitFor(() => {
|
||||
expect(screen.getByLabelText(/name/i)).toBeInTheDocument()
|
||||
})
|
||||
|
||||
|
||||
// Enter very long name
|
||||
await user.clear(screen.getByLabelText(/name/i))
|
||||
await user.type(screen.getByLabelText(/name/i), 'a'.repeat(101))
|
||||
|
||||
|
||||
expect(screen.getByText(/name must be less than 100 characters/i)).toBeInTheDocument()
|
||||
})
|
||||
|
||||
it('should allow empty optional fields', async () => {
|
||||
const user = userEvent.setup()
|
||||
mockedService.updateAppConfig.mockResolvedValue({ success: true })
|
||||
|
||||
|
||||
render(<AppConfigForm appId="app-1" />)
|
||||
|
||||
|
||||
await waitFor(() => {
|
||||
expect(screen.getByLabelText(/name/i)).toHaveValue('My App')
|
||||
})
|
||||
|
||||
|
||||
// Leave description empty (optional)
|
||||
await user.click(screen.getByRole('button', { name: /save/i }))
|
||||
|
||||
|
||||
await waitFor(() => {
|
||||
expect(mockedService.updateAppConfig).toHaveBeenCalled()
|
||||
})
|
||||
@@ -445,58 +445,58 @@ describe('AppConfigForm', () => {
|
||||
it('should save configuration', async () => {
|
||||
const user = userEvent.setup()
|
||||
mockedService.updateAppConfig.mockResolvedValue({ success: true })
|
||||
|
||||
|
||||
render(<AppConfigForm appId="app-1" />)
|
||||
|
||||
|
||||
await waitFor(() => {
|
||||
expect(screen.getByLabelText(/name/i)).toHaveValue('My App')
|
||||
})
|
||||
|
||||
|
||||
await user.clear(screen.getByLabelText(/name/i))
|
||||
await user.type(screen.getByLabelText(/name/i), 'Updated App')
|
||||
await user.click(screen.getByRole('button', { name: /save/i }))
|
||||
|
||||
|
||||
await waitFor(() => {
|
||||
expect(mockedService.updateAppConfig).toHaveBeenCalledWith(
|
||||
'app-1',
|
||||
expect.objectContaining({ name: 'Updated App' })
|
||||
)
|
||||
})
|
||||
|
||||
|
||||
expect(screen.getByText(/saved successfully/i)).toBeInTheDocument()
|
||||
})
|
||||
|
||||
it('should reset to default values', async () => {
|
||||
const user = userEvent.setup()
|
||||
|
||||
|
||||
render(<AppConfigForm appId="app-1" />)
|
||||
|
||||
|
||||
await waitFor(() => {
|
||||
expect(screen.getByLabelText(/name/i)).toHaveValue('My App')
|
||||
})
|
||||
|
||||
|
||||
// Make changes
|
||||
await user.clear(screen.getByLabelText(/name/i))
|
||||
await user.type(screen.getByLabelText(/name/i), 'Changed Name')
|
||||
|
||||
|
||||
// Reset
|
||||
await user.click(screen.getByRole('button', { name: /reset/i }))
|
||||
|
||||
|
||||
expect(screen.getByLabelText(/name/i)).toHaveValue('My App')
|
||||
})
|
||||
|
||||
it('should show unsaved changes warning', async () => {
|
||||
const user = userEvent.setup()
|
||||
|
||||
|
||||
render(<AppConfigForm appId="app-1" />)
|
||||
|
||||
|
||||
await waitFor(() => {
|
||||
expect(screen.getByLabelText(/name/i)).toHaveValue('My App')
|
||||
})
|
||||
|
||||
|
||||
// Make changes
|
||||
await user.type(screen.getByLabelText(/name/i), ' Updated')
|
||||
|
||||
|
||||
expect(screen.getByText(/unsaved changes/i)).toBeInTheDocument()
|
||||
})
|
||||
})
|
||||
@@ -505,15 +505,15 @@ describe('AppConfigForm', () => {
|
||||
it('should show error on save failure', async () => {
|
||||
const user = userEvent.setup()
|
||||
mockedService.updateAppConfig.mockRejectedValue(new Error('Server error'))
|
||||
|
||||
|
||||
render(<AppConfigForm appId="app-1" />)
|
||||
|
||||
|
||||
await waitFor(() => {
|
||||
expect(screen.getByLabelText(/name/i)).toHaveValue('My App')
|
||||
})
|
||||
|
||||
|
||||
await user.click(screen.getByRole('button', { name: /save/i }))
|
||||
|
||||
|
||||
await waitFor(() => {
|
||||
expect(screen.getByText(/failed to save/i)).toBeInTheDocument()
|
||||
})
|
||||
|
||||
@@ -54,12 +54,12 @@ See [Zustand Store Testing](#zustand-store-testing) section for full details.
|
||||
|
||||
## Mock Placement
|
||||
|
||||
| Location | Purpose |
|
||||
|----------|---------|
|
||||
| `web/vitest.setup.ts` | Global mocks shared by all tests (`react-i18next`, `zustand`, clipboard, FloatingPortal, Monaco, localStorage`) |
|
||||
| `web/__mocks__/zustand.ts` | Zustand mock implementation (auto-resets stores after each test) |
|
||||
| `web/__mocks__/` | Reusable mock factories shared across multiple test files |
|
||||
| Test file | Test-specific mocks, inline with `vi.mock()` |
|
||||
| Location | Purpose |
|
||||
| -------------------------- | --------------------------------------------------------------------------------------------------------------- |
|
||||
| `web/vitest.setup.ts` | Global mocks shared by all tests (`react-i18next`, `zustand`, clipboard, FloatingPortal, Monaco, localStorage`) |
|
||||
| `web/__mocks__/zustand.ts` | Zustand mock implementation (auto-resets stores after each test) |
|
||||
| `web/__mocks__/` | Reusable mock factories shared across multiple test files |
|
||||
| Test file | Test-specific mocks, inline with `vi.mock()` |
|
||||
|
||||
Modules are not mocked automatically. Use `vi.mock` in test files, or add global mocks in `web/vitest.setup.ts`.
|
||||
|
||||
@@ -85,10 +85,12 @@ The global mock provides:
|
||||
```typescript
|
||||
import { createReactI18nextMock } from '@/test/i18n-mock'
|
||||
|
||||
vi.mock('react-i18next', () => createReactI18nextMock({
|
||||
'my.custom.key': 'Custom translation',
|
||||
'button.save': 'Save',
|
||||
}))
|
||||
vi.mock('react-i18next', () =>
|
||||
createReactI18nextMock({
|
||||
'my.custom.key': 'Custom translation',
|
||||
'button.save': 'Save',
|
||||
}),
|
||||
)
|
||||
```
|
||||
|
||||
**Avoid**: Manually defining `useTranslation` mocks that just return the key - the global mock already does this.
|
||||
@@ -189,16 +191,16 @@ const mockedApi = vi.mocked(api)
|
||||
describe('Component', () => {
|
||||
beforeEach(() => {
|
||||
vi.clearAllMocks()
|
||||
|
||||
|
||||
// Setup default mock implementation
|
||||
mockedApi.fetchData.mockResolvedValue({ data: [] })
|
||||
})
|
||||
|
||||
it('should show data on success', async () => {
|
||||
mockedApi.fetchData.mockResolvedValue({ data: [{ id: 1 }] })
|
||||
|
||||
|
||||
render(<Component />)
|
||||
|
||||
|
||||
await waitFor(() => {
|
||||
expect(screen.getByText('1')).toBeInTheDocument()
|
||||
})
|
||||
@@ -206,9 +208,9 @@ describe('Component', () => {
|
||||
|
||||
it('should show error on failure', async () => {
|
||||
mockedApi.fetchData.mockRejectedValue(new Error('Network error'))
|
||||
|
||||
|
||||
render(<Component />)
|
||||
|
||||
|
||||
await waitFor(() => {
|
||||
expect(screen.getByText(/error/i)).toBeInTheDocument()
|
||||
})
|
||||
@@ -231,9 +233,9 @@ describe('GithubComponent', () => {
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
}),
|
||||
)
|
||||
|
||||
|
||||
render(<GithubComponent />)
|
||||
|
||||
|
||||
await waitFor(() => {
|
||||
expect(screen.getByText('dify')).toBeInTheDocument()
|
||||
})
|
||||
@@ -246,9 +248,9 @@ describe('GithubComponent', () => {
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
}),
|
||||
)
|
||||
|
||||
|
||||
render(<GithubComponent />)
|
||||
|
||||
|
||||
await waitFor(() => {
|
||||
expect(screen.getByText(/error/i)).toBeInTheDocument()
|
||||
})
|
||||
@@ -267,25 +269,25 @@ import { createMockProviderContextValue, createMockPlan } from '@/__mocks__/prov
|
||||
describe('Component with Context', () => {
|
||||
it('should render for free plan', () => {
|
||||
const mockContext = createMockPlan('sandbox')
|
||||
|
||||
|
||||
render(
|
||||
<ProviderContext.Provider value={mockContext}>
|
||||
<Component />
|
||||
</ProviderContext.Provider>
|
||||
)
|
||||
|
||||
|
||||
expect(screen.getByText('Upgrade')).toBeInTheDocument()
|
||||
})
|
||||
|
||||
it('should render for pro plan', () => {
|
||||
const mockContext = createMockPlan('professional')
|
||||
|
||||
|
||||
render(
|
||||
<ProviderContext.Provider value={mockContext}>
|
||||
<Component />
|
||||
</ProviderContext.Provider>
|
||||
)
|
||||
|
||||
|
||||
expect(screen.queryByText('Upgrade')).not.toBeInTheDocument()
|
||||
})
|
||||
})
|
||||
@@ -411,7 +413,7 @@ Manual mocking conflicts with the global Zustand mock and loses store functional
|
||||
```typescript
|
||||
// ❌ WRONG: Don't mock the store module
|
||||
vi.mock('@/app/components/app/store', () => ({
|
||||
useStore: (selector) => mockSelector(selector), // Missing getState, setState!
|
||||
useStore: (selector) => mockSelector(selector), // Missing getState, setState!
|
||||
}))
|
||||
|
||||
// ❌ WRONG: This conflicts with global zustand mock
|
||||
@@ -439,14 +441,11 @@ const mockStore = {
|
||||
}
|
||||
|
||||
vi.mock('@/app/components/app/store', () => ({
|
||||
useStore: Object.assign(
|
||||
(selector: (state: typeof mockStore) => unknown) => selector(mockStore),
|
||||
{
|
||||
getState: () => mockStore,
|
||||
setState: vi.fn(),
|
||||
subscribe: vi.fn(),
|
||||
},
|
||||
),
|
||||
useStore: Object.assign((selector: (state: typeof mockStore) => unknown) => selector(mockStore), {
|
||||
getState: () => mockStore,
|
||||
setState: vi.fn(),
|
||||
subscribe: vi.fn(),
|
||||
}),
|
||||
}))
|
||||
```
|
||||
|
||||
@@ -528,7 +527,7 @@ it('should display project owner', () => {
|
||||
const project = createMockProject({
|
||||
owner: createMockUser({ name: 'John Doe' }),
|
||||
})
|
||||
|
||||
|
||||
render(<ProjectCard project={project} />)
|
||||
expect(screen.getByText('John Doe')).toBeInTheDocument()
|
||||
})
|
||||
|
||||
@@ -6,10 +6,10 @@ This guide defines the workflow for generating tests, especially for complex com
|
||||
|
||||
This guide addresses **multi-file workflow** (how to process multiple test files). For coverage requirements within a single test file, see `web/docs/test.md` § Coverage Goals.
|
||||
|
||||
| Scope | Rule |
|
||||
|-------|------|
|
||||
| **Single file** | Complete coverage in one generation (100% function, >95% branch) |
|
||||
| **Multi-file directory** | Process one file at a time, verify each before proceeding |
|
||||
| Scope | Rule |
|
||||
| ------------------------ | ---------------------------------------------------------------- |
|
||||
| **Single file** | Complete coverage in one generation (100% function, >95% branch) |
|
||||
| **Multi-file directory** | Process one file at a time, verify each before proceeding |
|
||||
|
||||
## ⚠️ Critical Rule: Incremental Approach for Multi-File Testing
|
||||
|
||||
@@ -17,14 +17,14 @@ When testing a **directory with multiple files**, **NEVER generate all test file
|
||||
|
||||
### Why Incremental?
|
||||
|
||||
| Batch Approach (❌) | Incremental Approach (✅) |
|
||||
|---------------------|---------------------------|
|
||||
| Generate 5+ tests at once | Generate 1 test at a time |
|
||||
| Run tests only at the end | Run test immediately after each file |
|
||||
| Multiple failures compound | Single point of failure, easy to debug |
|
||||
| Hard to identify root cause | Clear cause-effect relationship |
|
||||
| Mock issues affect many files | Mock issues caught early |
|
||||
| Messy git history | Clean, atomic commits possible |
|
||||
| Batch Approach (❌) | Incremental Approach (✅) |
|
||||
| ----------------------------- | -------------------------------------- |
|
||||
| Generate 5+ tests at once | Generate 1 test at a time |
|
||||
| Run tests only at the end | Run test immediately after each file |
|
||||
| Multiple failures compound | Single point of failure, easy to debug |
|
||||
| Hard to identify root cause | Clear cause-effect relationship |
|
||||
| Mock issues affect many files | Mock issues caught early |
|
||||
| Messy git history | Clean, atomic commits possible |
|
||||
|
||||
## Single File Workflow
|
||||
|
||||
@@ -190,7 +190,7 @@ Update status as you complete each:
|
||||
```
|
||||
# BAD: Writing all files then testing
|
||||
Write component-a.spec.tsx
|
||||
Write component-b.spec.tsx
|
||||
Write component-b.spec.tsx
|
||||
Write component-c.spec.tsx
|
||||
Write component-d.spec.tsx
|
||||
Run pnpm test ← Multiple failures, hard to debug
|
||||
|
||||
@@ -1,128 +1,71 @@
|
||||
---
|
||||
name: how-to-write-component
|
||||
description: Use when writing, refactoring, or reviewing React/TypeScript components in Dify web, especially decisions about component ownership, props/types, URL/query state, Jotai state, async state, generated API contracts, queries/mutations, overlays, effects, navigation, performance, and empty states.
|
||||
description: React/TypeScript component style guide. Use when writing, refactoring, or reviewing React components, especially around props typing, state boundaries, shared local state with Jotai atoms, API types, query/mutation contracts, navigation, memoization, wrappers, and empty-state handling.
|
||||
---
|
||||
|
||||
# How To Write A Component
|
||||
|
||||
Use this as the component decision guide for Dify web. Existing code is reference material, not automatic precedent; if touched code violates these rules, adapt it and fix equivalent patterns in the same feature branch.
|
||||
|
||||
## First Decisions
|
||||
|
||||
| Question | Default | Promote or extract only when |
|
||||
| --- | --- | --- |
|
||||
| Where should code live? | Keep it local to the feature workflow, route, or owner. | Multiple verticals need the same stable primitive. |
|
||||
| How should route/tab folders be named? | Match the current route segment, tab name, or user-visible surface. | Keep a historical or broader parent only when it still owns multiple surfaces. |
|
||||
| Who owns state, data, and handlers? | The lowest component that uses them. | A parent coordinates shared loading, errors, empty UI, selection, submission, navigation, or one consistent snapshot. |
|
||||
| Should this become Jotai state? | Keep synchronous UI/form state in component or DOM state. | Siblings need one source of truth, the value drives atoms, or scoped workflow state must survive hidden/unmounted steps. |
|
||||
| Should URL state enter Jotai? | Let Next.js route params and `nuqs` own URL state and updates. | Query atoms or shared derived atoms need a read-only bridge hydrated at the route/surface boundary. |
|
||||
| Should this query/mutation become an atom? | Use TanStack Query hooks at the lowest owner. | It reads atom state, feeds derived atoms, or participates in shared Jotai workflow orchestration. |
|
||||
| Should this be a helper/wrapper? | Prefer direct readable code at the use site. | The name captures a stable domain rule or the wrapper owns real behavior, validation, state, error handling, or semantics. |
|
||||
| Is an Effect needed? | No. Derive during render or handle the user action in the event handler. | It synchronizes with an external system such as browser APIs, subscriptions, timers, analytics, or imperative DOM/non-React widgets. |
|
||||
Use this as the decision guide for React/TypeScript component structure. Existing code is reference material, not automatic precedent; when it conflicts with these rules, adapt the approach instead of reproducing the violation.
|
||||
|
||||
## Core Defaults
|
||||
|
||||
- Search before adding UI, hooks, helpers, query utilities, or styling patterns. Reuse existing base components, feature components, hooks, utilities, and design styles when they fit.
|
||||
- Follow Dify's CSS-first Tailwind v4 contract from `packages/dify-ui/README.md` and `packages/dify-ui/AGENTS.md`. Prefer design-system tokens, utilities, and radius mappings over generic Tailwind choices.
|
||||
- Group feature code by workflow, route, or ownership area with route-aligned names: components, hooks, local types, query helpers, atoms, constants, tests, and small utilities should live near the code that changes with them.
|
||||
- For each feature module, keep a module-local `README.md` as a boundary note. Start with the module name, a brief one-sentence description, then split dependencies into `Internal Modules` and `External Modules` sections; keep both sections and write `None.` when one category is empty. `Internal Modules` lists modules inside the same overall feature using paths from that feature root, such as `shared/domain/runtime-status`; `External Modules` lists project modules outside the feature using paths from the web root without a `web/` prefix, such as `app/components/base/skeleton`. Omit npm packages, workspace package dependencies, and whitelisted plumbing modules. Do not copy caller-relative import paths into the README.
|
||||
- Module README whitelist: `@/service/client`, `@/next/*`.
|
||||
- Keep source/default selection, validation, dirty checks, and payload shaping close to the workflow that owns submit behavior. Do not hide flow-specific priority order, fallback behavior, or submit semantics in generic utilities.
|
||||
- Prefer direct conditionals for small branch-specific decisions, especially form source selection and request payload assembly.
|
||||
- Loading states for page sections, cards, lists, tables, forms, and drawers should be skeletons scoped to the content being loaded. Use spinners only for small inline busy indicators.
|
||||
- Search before adding UI, hooks, helpers, or styling patterns. Reuse existing base components, feature components, hooks, utilities, and design styles when they fit.
|
||||
- Group code by feature workflow, route, or ownership area: components, hooks, local types, query helpers, atoms, constants, and small utilities should live near the code that changes with them.
|
||||
- Promote code to shared only when multiple verticals need the same stable primitive. Otherwise keep it local and compose shared primitives inside the owning feature.
|
||||
- Follow Dify's CSS-first Tailwind v4 contract from `packages/dify-ui/README.md` and `packages/dify-ui/AGENTS.md`. Prefer design-system tokens, utilities, and radius mappings over generic Tailwind guidance.
|
||||
|
||||
## Layout And Ownership
|
||||
## Ownership
|
||||
|
||||
- State-heavy wizards, drawers, modals, and secondary workflows can be a small feature surface: an entry file, one feature-local state file when Jotai is actually needed, and shallow `ui/` owners that match real visual regions.
|
||||
- The entry file handles route integration, provider wiring, close behavior, and surface mounting. The composition owner handles high-level workflow branching. The closest visual owner handles section branching.
|
||||
- When a page or tab maps to a route segment, name its feature folder after that route/tab surface instead of a stale parent grouping. Remove misleading intermediate folders when only one surface remains.
|
||||
- When a tab folder grows into several independent sections or action areas, split the first level by product/visual owners. Keep the root for the entry component and cross-owner state, colocate tests with the owner folder, and put truly shared local UI under a specifically named `components/` file.
|
||||
- Repeated TanStack query calls in sibling components are acceptable when each component independently consumes the data; TanStack Query deduplicates and shares cache.
|
||||
- Pass stable domain identity across boundaries. Do not forward derived presentation state when the receiver can derive it from its own data source.
|
||||
- A component that owns a visual surface should also own data access, loading, empty, and error states for content rendered inside it unless a parent truly coordinates that state.
|
||||
- Avoid prop drilling. One pass-through layer is acceptable; repeated forwarding means ownership should move down or into feature-scoped Jotai UI state. Keep server/cache state in Query and API flow.
|
||||
- Do not replace prop drilling with one large view-model hook threaded through section props. Move each hook, query, derived value, and handler to the concrete section that consumes it.
|
||||
- Keep callbacks in a parent only for workflow coordination such as form submission, shared selection, batch behavior, or navigation. Otherwise let the child, menu, or row own the action.
|
||||
|
||||
## Feature-Scoped Jotai
|
||||
|
||||
- A Jotai-backed feature has one feature-local state file for shared primitive atoms, query atoms, derived atoms, write-only actions, mutation atoms, submission orchestration, provider exports, and optional scope configuration.
|
||||
- Keep component-owned synchronous UI state local even inside Jotai features: dialog open flags, menus/popovers, confirmations, field drafts, and selected local options usually belong in component state.
|
||||
- Use uncontrolled `@langgenius/dify-ui/form` and `@langgenius/dify-ui/field` controls for edit/create forms whose fields are read only at submit time. Initialize query-backed defaults with `defaultValue` and keyed remounts.
|
||||
- Promote form state to atoms only when another component must react to in-progress values, a draft must survive unmount/remount in the scoped workflow, or multiple steps share the same editable draft before submit.
|
||||
- Treat `useParams`, route args, and `nuqs` query state as framework-owned state. When atom logic needs those values, hydrate primitive atoms at the route or surface boundary, such as with `useHydrateAtoms(..., { dangerouslyForceHydrate: true })`; keep URL updates in the route/query-state APIs instead of write atoms.
|
||||
- Within a route-owned feature, choose one source for route identity. If route params are bridged into feature atoms, use that bridge consistently for route-derived queries and actions instead of also threading the same route id through page, tab, and section props.
|
||||
- For async work tied to atom state, use `atomWithQuery` or `atomWithMutation`; write atoms should update only the inputs that drive those atoms. This applies to pure frontend async work as well as network requests, so do not hand-roll loading/error/in-flight state with `useState` or `useRef` for atom-orchestrated async behavior. For component-owned remote work, use `useQuery` or `useMutation` directly.
|
||||
- Row-local async state belongs to the row owner unless it participates in a shared Jotai workflow or needs atom-scoped reset semantics.
|
||||
- Leave query and mutation atoms unscoped so they keep shared QueryClient cache and invalidation behavior. Scope resettable primitives and explicit hydration tuples; scope a derived atom only when every dependency should be private to that surface.
|
||||
- For scoped primitives that are always hydrated by `ScopeProvider`, prefer `atomWithLazy<T>(() => { throw new Error(...) })` when consumers should see a non-null type.
|
||||
- Order state files by dependency graph: types/constants, primitives, query atoms, query-data derived atoms, business/readiness derived atoms, write actions, mutation atoms, submission orchestration, provider exports.
|
||||
- Name derived atoms as business facts and write atoms as user or workflow commands. Components should read or write the exact atom they need with `useAtomValue` or `useSetAtom`.
|
||||
- Menu/dialog `open` state usually stays local, but a scoped atom is acceptable when a composed menu plus secondary surface would otherwise pass confusing `open`/`onClose` props through unrelated layers. Scope that primitive with the surface instance so reset behavior stays local.
|
||||
- Keep independent dialog lifecycles separate. Avoid one discriminated "current action dialog" atom when dialogs have separate open state, loading guards, or reset behavior.
|
||||
- Put local state, queries, mutations, handlers, and derived UI data in the lowest component that uses them. Extract a purpose-built owner component only when the logic has no natural home.
|
||||
- Repeated TanStack query calls in sibling components are acceptable when each component independently consumes the data. Do not hoist a query only because it is duplicated; TanStack Query handles deduplication and cache sharing.
|
||||
- Hoist state, queries, or callbacks to a parent only when the parent consumes the data, coordinates shared loading/error/empty UI, needs one consistent snapshot, or owns a workflow spanning children.
|
||||
- Avoid prop drilling. One pass-through layer is acceptable; repeated forwarding means ownership should move down or into feature-scoped Jotai UI state. Keep server/cache state in query and API data flow.
|
||||
- Keep callbacks in a parent only for workflow coordination such as form submission, shared selection, batch behavior, or navigation. Otherwise let the child or row own its action.
|
||||
- Prefer uncontrolled DOM state and CSS variables before adding controlled props.
|
||||
|
||||
## Components, Props, And Types
|
||||
|
||||
- Type component signatures directly; do not use `FC` or `React.FC`.
|
||||
- Prefer `function` for top-level components and module helpers. Use arrow functions for local callbacks, handlers, and lambda-style APIs.
|
||||
- Prefer named exports. Use default exports only where the framework requires them, such as Next.js route files.
|
||||
- Avoid barrel files that only re-export secondary owners. `index.tsx` is acceptable for a route/tab entry component; import header controls, switches, sections, and row owners from their concrete owner files.
|
||||
- Type simple one-off props inline. Use a named `Props` type only when reused, exported, complex, or clearer.
|
||||
- Use API-generated or API-returned types at component boundaries. Keep small UI conversion helpers and one-off UI extensions beside the component that needs them.
|
||||
- Preserve domain value types for selection components. Do not widen enum, union, boolean, numeric, object, or nullable select/radio values to `string`; keep wrappers and option value carriers typed from their feature option collection.
|
||||
- Avoid `common.tsx` buckets for shared UI. Use a feature-local `components/` folder with concrete filenames that describe the shared role.
|
||||
- Do not create type aliases that only rename another type. Use aliases only for real UI concepts, refinements, or reusable local contracts.
|
||||
- Name values by their domain role and backend API contract, especially persistent IDs and route params. Normalize framework or route params at the boundary.
|
||||
- Put fallback and invariant checks in the lowest component that already handles that state. Do not extract helpers whose only behavior is hiding missing display data.
|
||||
|
||||
## Generated API And Nullable Data
|
||||
|
||||
- Treat generated contracts as authoritative at API, query, mutation, cache, and service boundaries. For enterprise APIs, use `packages/contracts/generated/enterprise/*`.
|
||||
- Do not hand-write DTO mirrors, widen generated fields/enums, or add parallel frontend enum/status layers unless they model product state not represented by the API.
|
||||
- Use generated enum objects and union types directly in props, comparisons, status logic, and i18n keys. Presentation-only tone maps should be keyed by generated enums.
|
||||
- Normalize or coerce only at real boundaries: user-entered forms, search, URL/query params, file names, DOM IDs, or legacy adapters.
|
||||
- Do not coerce nullable or optional API strings to `''` in query, derived model, or payload-building code. Keep `null` or `undefined` until the final boundary requiring a string.
|
||||
- Do not use `value || undefined` for mutation fields where `''` means "clear this value". Trim or normalize at the form boundary, then preserve intentional empty strings.
|
||||
- Prefer nullable-tolerant render props for API-returned rows. Narrow only where a real value is required, such as mutation params, route hrefs, select values, query input, or required React keys.
|
||||
- Build required values in the same branch that proves them, using `flatMap`, a local loop, or an early return. Avoid truthiness guards, `filter(Boolean)`, `filter(item => item.id)`, and `!` after filters.
|
||||
- Use conditional spreads or explicit pushes for conditional array items instead of `undefined` placeholders followed by narrowing filters.
|
||||
- Empty collection fallbacks are for not-yet-loaded query data or genuinely nullable collections at the owning render boundary, not for hiding required API fields.
|
||||
- Use API-generated or API-returned types at component boundaries. Keep small UI conversion helpers beside the component that needs them.
|
||||
- Name values by their domain role and backend API contract, and keep that name stable across the call chain, especially IDs like `appInstanceId`. Normalize framework or route params at the boundary.
|
||||
- Keep fallback and invariant checks at the lowest component that already handles that state; callers should pass raw values through instead of duplicating checks.
|
||||
|
||||
## Queries And Mutations
|
||||
|
||||
- Keep `web/contract/*` as the API shape source of truth and follow the `{ params, query?, body? }` input shape.
|
||||
- Consume generated queries with `useQuery(consoleQuery.xxx.queryOptions(...))` or `useQuery(marketplaceQuery.xxx.queryOptions(...))`.
|
||||
- If a generated query input comes from an atom, including a route-identity bridge atom, keep the query in `atomWithQuery`; do not unwrap the atom in a component just to call `useQuery`.
|
||||
- Consume owner-local mutations with `useMutation(consoleQuery.xxx.mutationOptions(...))` or `useMutation(marketplaceQuery.xxx.mutationOptions(...))` when pending/error state is not consumed by feature atoms.
|
||||
- In `atomWithQuery`, `atomWithInfiniteQuery`, and `atomWithMutation`, return generated `queryOptions()`, `infiniteOptions()`, or `mutationOptions()` directly. Pass `enabled`, `retry`, `placeholderData`, `select`, and pagination options into the generated call instead of spreading options into a hand-built object.
|
||||
- For generated oRPC options with missing required input, branch the whole input with `input: condition ? validInput : skipToken` and `enabled: Boolean(condition)`. Never place `skipToken` inside a nested placeholder payload or coerce required IDs to `''`.
|
||||
- When prefetch and render use the same request, extract local query options or a query-options atom so `prefetchQuery` and `useQuery`/`atomWithQuery` share the exact options.
|
||||
- For custom query or mutation functions, wrap options with TanStack `queryOptions(...)` or `mutationOptions(...)`.
|
||||
- Do not extract generated `queryOptions(...)` into a helper solely to share input construction; extract only when prefetch/render must share exact options or the helper owns real domain behavior.
|
||||
- Avoid pass-through hooks and thin `web/service/use-*` wrappers that only rename generated options. Keep feature hooks for real orchestration, workflow state, or shared domain behavior.
|
||||
- Put shared cache behavior in `createTanstackQueryUtils(...experimental_defaults...)`. Component or atom callbacks may handle local toasts, closing dialogs, and navigation, but should not replace shared invalidation or patch shared server state locally.
|
||||
- For overlays that may open heavier secondary content, prefetch from the trigger/menu open event with `queryClient.prefetchQuery(queryOptions)` when `onOpenChange` is available. Do not mount hidden subscribers just to warm cache.
|
||||
- Keep `web/contract/*` as the single source of truth for API shape; follow existing domain/router patterns and the `{ params, query?, body? }` input shape.
|
||||
- Consume queries directly with `useQuery(consoleQuery.xxx.queryOptions(...))` or `useQuery(marketplaceQuery.xxx.queryOptions(...))`.
|
||||
- Avoid pass-through hooks and thin `web/service/use-*` wrappers that only rename `queryOptions()` or `mutationOptions()`. Extract a small `queryOptions` helper only when repeated call-site options justify it.
|
||||
- Keep feature hooks for real orchestration, workflow state, or shared domain behavior.
|
||||
- For missing required query input, use `input: skipToken`; use `enabled` only for extra business gating after the input is valid.
|
||||
- Consume mutations directly with `useMutation(consoleQuery.xxx.mutationOptions(...))` or `useMutation(marketplaceQuery.xxx.mutationOptions(...))`; use oRPC clients as `mutationFn` only for custom flows.
|
||||
- Put shared cache behavior in `createTanstackQueryUtils(...experimental_defaults...)`; components may add UI feedback callbacks, but should not own shared invalidation rules.
|
||||
- Do not use deprecated `useInvalid` or `useReset`.
|
||||
- Prefer `mutate(...)`; use `mutateAsync(...)` only when Promise semantics are required, and wrap awaited calls in `try/catch`.
|
||||
|
||||
## Boundaries And Overlays
|
||||
## Component Boundaries
|
||||
|
||||
- Use the first level below a page or tab to organize independent page sections when it adds structure or the root folder becomes noisy. This layer is layout/semantic first, not automatically the data owner.
|
||||
- Treat component names, semantic roles, and user- or design-marked visual regions as boundary constraints. Keep adjacent UI as a sibling owner or introduce a correctly named broader owner.
|
||||
- Use the first level below a page or tab to organize independent page sections when it adds real structure. This layer is layout/semantic first, not automatically the data owner.
|
||||
- Split deeper components by the data and state each layer actually needs. Each component should access only necessary data, and ownership should stay at the lowest consumer.
|
||||
- Keep cohesive forms, menu bodies, and one-off helpers local unless they need their own state, reuse, or semantic boundary.
|
||||
- Separate hidden secondary surfaces from the trigger's main flow. For dialogs, dropdowns, popovers, and similar branches, extract a small local component when hidden content would obscure the parent.
|
||||
- Preserve composability by separating behavior ownership from placement ownership: an action can own trigger/open/menu content while the caller owns slots, offsets, and alignment.
|
||||
- When a dialog, dropdown, or popover accepts controlled `open`, mount it unconditionally unless unmounting is required for performance or reset semantics. Use keyed scope or local state reset instead of `{open && <Surface />}` wrappers.
|
||||
- When opening a dialog from a menu item, keep the menu and dialog as sibling surfaces. Let the menu command open the dialog, and mount the dialog outside menu popup content.
|
||||
- For dialogs and alert dialogs, keep the root responsible for `open` wiring and put query/mutation hooks inside the content component when work should mount only after the overlay opens.
|
||||
- Prefer uncontrolled overlay roots when the library can own open state. Use `onOpenChange` for side effects and CSS/data selectors for open-state styling.
|
||||
- Avoid wrapper DOM unless it provides layout, semantics, accessibility, state ownership, or library integration. Avoid shallow wrappers, hook-to-props adapters, layout-only render props, children pass-through wrappers, and prop renaming unless they add real behavior or a real boundary.
|
||||
- Separate hidden secondary surfaces from the trigger's main flow. For dialogs, dropdowns, popovers, and similar branches, extract a small local component that owns the trigger, open state, and hidden content when it would obscure the parent flow.
|
||||
- Preserve composability by separating behavior ownership from layout ownership. A dropdown action may own its trigger, open state, and menu content; the caller owns placement such as slots, offsets, and alignment.
|
||||
- Avoid unnecessary DOM hierarchy. Do not add wrapper elements unless they provide layout, semantics, accessibility, state ownership, or integration with a library API; prefer fragments or styling an existing element when possible.
|
||||
- Avoid shallow wrappers and prop renaming unless the wrapper adds validation, orchestration, error handling, state ownership, or a real semantic boundary.
|
||||
|
||||
## Effects, Navigation, And Performance
|
||||
## You Might Not Need An Effect
|
||||
|
||||
- Use Effects only to synchronize with external systems such as browser APIs, non-React widgets, subscriptions, timers, analytics that must run because the component was shown, or imperative DOM integration.
|
||||
- Do not use Effects to transform props or state for rendering. Calculate derived values during render, and use `useMemo` only when the calculation is actually expensive.
|
||||
- Do not use Effects to handle user actions. Put action-specific logic in the event handler where the cause is known.
|
||||
- Do not use Effects to copy one state value into another state value representing the same concept. Pick one source of truth and derive the rest during render.
|
||||
- Do not reset or adjust state from props with an Effect. Prefer a `key` reset, storing a stable ID and deriving the selected object, or guarded same-component render-time adjustment when truly necessary.
|
||||
- Prefer framework data APIs or TanStack Query for data fetching instead of writing request Effects in components.
|
||||
- If an Effect still seems necessary, first name the external system it synchronizes with. If there is no external system, remove the Effect and restructure the state or event flow.
|
||||
|
||||
## Navigation And Performance
|
||||
|
||||
- Use Effects only to synchronize with external systems. Do not use Effects to transform props/state for rendering, handle user actions, copy state, reset state from props, or fetch data.
|
||||
- For forms initialized from query data, prefer keyed remounts or surface-entry atom hydration over Effects that copy query data into form state.
|
||||
- Prefer framework data APIs or TanStack Query for data fetching.
|
||||
- Prefer `Link` for normal navigation. Use router APIs only for command-flow side effects such as mutation success, guarded redirects, or form submission.
|
||||
- Before using `memo`, move changing state down to the smallest component that uses it. If state must wrap stable content, lift the stable content up and pass it as `children`.
|
||||
- Avoid `memo`, `useMemo`, and `useCallback` unless there is a clear performance reason.
|
||||
|
||||
@@ -1,49 +1,43 @@
|
||||
// For format details, see https://aka.ms/devcontainer.json. For config options, see the
|
||||
// README at: https://github.com/devcontainers/templates/tree/main/src/anaconda
|
||||
{
|
||||
"name": "Python 3.12",
|
||||
"build": {
|
||||
"context": "..",
|
||||
"dockerfile": "Dockerfile"
|
||||
},
|
||||
"mounts": [
|
||||
"source=dify-dev-tmp,target=/tmp,type=volume"
|
||||
],
|
||||
"features": {
|
||||
"ghcr.io/devcontainers/features/node:1": {
|
||||
"nodeGypDependencies": true,
|
||||
"version": "lts"
|
||||
},
|
||||
"ghcr.io/devcontainers-extra/features/npm-package:1": {
|
||||
"package": "typescript",
|
||||
"version": "latest"
|
||||
},
|
||||
"ghcr.io/devcontainers/features/docker-in-docker:2": {
|
||||
"moby": true,
|
||||
"azureDnsAutoDetection": true,
|
||||
"installDockerBuildx": true,
|
||||
"version": "latest",
|
||||
"dockerDashComposeVersion": "v2"
|
||||
}
|
||||
},
|
||||
"customizations": {
|
||||
"vscode": {
|
||||
"extensions": [
|
||||
"ms-python.pylint",
|
||||
"GitHub.copilot",
|
||||
"ms-python.python"
|
||||
]
|
||||
}
|
||||
},
|
||||
"postStartCommand": "./.devcontainer/post_start_command.sh",
|
||||
"postCreateCommand": "./.devcontainer/post_create_command.sh"
|
||||
// Features to add to the dev container. More info: https://containers.dev/features.
|
||||
// "features": {},
|
||||
// Use 'forwardPorts' to make a list of ports inside the container available locally.
|
||||
// "forwardPorts": [],
|
||||
// Use 'postCreateCommand' to run commands after the container is created.
|
||||
// "postCreateCommand": "python --version",
|
||||
// Configure tool-specific properties.
|
||||
// "customizations": {},
|
||||
// Uncomment to connect as root instead. More info: https://aka.ms/dev-containers-non-root.
|
||||
}
|
||||
"name": "Python 3.12",
|
||||
"build": {
|
||||
"context": "..",
|
||||
"dockerfile": "Dockerfile"
|
||||
},
|
||||
"mounts": ["source=dify-dev-tmp,target=/tmp,type=volume"],
|
||||
"features": {
|
||||
"ghcr.io/devcontainers/features/node:1": {
|
||||
"nodeGypDependencies": true,
|
||||
"version": "lts"
|
||||
},
|
||||
"ghcr.io/devcontainers-extra/features/npm-package:1": {
|
||||
"package": "typescript",
|
||||
"version": "latest"
|
||||
},
|
||||
"ghcr.io/devcontainers/features/docker-in-docker:2": {
|
||||
"moby": true,
|
||||
"azureDnsAutoDetection": true,
|
||||
"installDockerBuildx": true,
|
||||
"version": "latest",
|
||||
"dockerDashComposeVersion": "v2"
|
||||
}
|
||||
},
|
||||
"customizations": {
|
||||
"vscode": {
|
||||
"extensions": ["ms-python.pylint", "GitHub.copilot", "ms-python.python"]
|
||||
}
|
||||
},
|
||||
"postStartCommand": "./.devcontainer/post_start_command.sh",
|
||||
"postCreateCommand": "./.devcontainer/post_create_command.sh"
|
||||
// Features to add to the dev container. More info: https://containers.dev/features.
|
||||
// "features": {},
|
||||
// Use 'forwardPorts' to make a list of ports inside the container available locally.
|
||||
// "forwardPorts": [],
|
||||
// Use 'postCreateCommand' to run commands after the container is created.
|
||||
// "postCreateCommand": "python --version",
|
||||
// Configure tool-specific properties.
|
||||
// "customizations": {},
|
||||
// Uncomment to connect as root instead. More info: https://aka.ms/dev-containers-non-root.
|
||||
}
|
||||
|
||||
@@ -26,9 +26,6 @@
|
||||
/cli/ @GareArc
|
||||
/.github/workflows/cli-tests.yml @GareArc
|
||||
|
||||
# E2E
|
||||
/e2e/ @lyzno1
|
||||
|
||||
# Backend (default owner, more specific rules below will override)
|
||||
/api/ @QuantumGhost
|
||||
|
||||
|
||||
@@ -1,17 +1,17 @@
|
||||
title: "General Discussion"
|
||||
title: 'General Discussion'
|
||||
body:
|
||||
- type: checkboxes
|
||||
attributes:
|
||||
label: Self Checks
|
||||
description: "To make sure we get to you in time, please check the following :)"
|
||||
description: 'To make sure we get to you in time, please check the following :)'
|
||||
options:
|
||||
- label: I have searched for existing issues [search for existing issues](https://github.com/langgenius/dify/issues), including closed ones.
|
||||
required: true
|
||||
- label: I confirm that I am using English to submit this report (我已阅读并同意 [Language Policy](https://github.com/langgenius/dify/issues/1542)).
|
||||
required: true
|
||||
- label: "[FOR CHINESE USERS] 请务必使用英文提交 Issue,否则会被关闭。谢谢!:)"
|
||||
- label: '[FOR CHINESE USERS] 请务必使用英文提交 Issue,否则会被关闭。谢谢!:)'
|
||||
required: true
|
||||
- label: "Please do not modify this template :) and fill in all the required fields."
|
||||
- label: 'Please do not modify this template :) and fill in all the required fields.'
|
||||
required: true
|
||||
- type: textarea
|
||||
attributes:
|
||||
|
||||
@@ -1,17 +1,17 @@
|
||||
title: "Help"
|
||||
title: 'Help'
|
||||
body:
|
||||
- type: checkboxes
|
||||
attributes:
|
||||
label: Self Checks
|
||||
description: "To make sure we get to you in time, please check the following :)"
|
||||
description: 'To make sure we get to you in time, please check the following :)'
|
||||
options:
|
||||
- label: I have searched for existing issues [search for existing issues](https://github.com/langgenius/dify/issues), including closed ones.
|
||||
required: true
|
||||
- label: I confirm that I am using English to submit this report (我已阅读并同意 [Language Policy](https://github.com/langgenius/dify/issues/1542)).
|
||||
required: true
|
||||
- label: "[FOR CHINESE USERS] 请务必使用英文提交 Issue,否则会被关闭。谢谢!:)"
|
||||
- label: '[FOR CHINESE USERS] 请务必使用英文提交 Issue,否则会被关闭。谢谢!:)'
|
||||
required: true
|
||||
- label: "Please do not modify this template :) and fill in all the required fields."
|
||||
- label: 'Please do not modify this template :) and fill in all the required fields.'
|
||||
required: true
|
||||
- type: textarea
|
||||
attributes:
|
||||
|
||||
@@ -3,15 +3,15 @@ body:
|
||||
- type: checkboxes
|
||||
attributes:
|
||||
label: Self Checks
|
||||
description: "To make sure we get to you in time, please check the following :)"
|
||||
description: 'To make sure we get to you in time, please check the following :)'
|
||||
options:
|
||||
- label: I have searched for existing issues [search for existing issues](https://github.com/langgenius/dify/issues), including closed ones.
|
||||
required: true
|
||||
- label: I confirm that I am using English to submit this report (我已阅读并同意 [Language Policy](https://github.com/langgenius/dify/issues/1542)).
|
||||
required: true
|
||||
- label: "[FOR CHINESE USERS] 请务必使用英文提交 Issue,否则会被关闭。谢谢!:)"
|
||||
- label: '[FOR CHINESE USERS] 请务必使用英文提交 Issue,否则会被关闭。谢谢!:)'
|
||||
required: true
|
||||
- label: "Please do not modify this template :) and fill in all the required fields."
|
||||
- label: 'Please do not modify this template :) and fill in all the required fields.'
|
||||
required: true
|
||||
- type: textarea
|
||||
attributes:
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
name: "🕷️ Bug report"
|
||||
name: '🕷️ Bug report'
|
||||
description: Report errors or unexpected behavior
|
||||
labels:
|
||||
- bug
|
||||
@@ -6,7 +6,7 @@ body:
|
||||
- type: checkboxes
|
||||
attributes:
|
||||
label: Self Checks
|
||||
description: "To make sure we get to you in time, please check the following :)"
|
||||
description: 'To make sure we get to you in time, please check the following :)'
|
||||
options:
|
||||
- label: I have read the [Contributing Guide](https://github.com/langgenius/dify/blob/main/CONTRIBUTING.md) and [Language Policy](https://github.com/langgenius/dify/issues/1542).
|
||||
required: true
|
||||
@@ -18,7 +18,7 @@ body:
|
||||
required: true
|
||||
- label: 【中文用户 & Non English User】请使用英语提交,否则会被关闭 :)
|
||||
required: true
|
||||
- label: "Please do not modify this template :) and fill in all the required fields."
|
||||
- label: 'Please do not modify this template :) and fill in all the required fields.'
|
||||
required: true
|
||||
|
||||
- type: input
|
||||
|
||||
@@ -1,13 +1,13 @@
|
||||
blank_issues_enabled: false
|
||||
contact_links:
|
||||
- name: "\U0001F510 Security Vulnerabilities"
|
||||
url: "https://github.com/langgenius/dify/security/advisories/new"
|
||||
url: 'https://github.com/langgenius/dify/security/advisories/new'
|
||||
about: Report security vulnerabilities through GitHub Security Advisories to ensure responsible disclosure. 💡 Please do not report security vulnerabilities in public issues.
|
||||
- name: "\U0001F4A1 Model Providers & Plugins"
|
||||
url: "https://github.com/langgenius/dify-official-plugins/issues/new/choose"
|
||||
url: 'https://github.com/langgenius/dify-official-plugins/issues/new/choose'
|
||||
about: Report issues with official plugins or model providers, you will need to provide the plugin version and other relevant details.
|
||||
- name: "\U0001F4AC Documentation Issues"
|
||||
url: "https://github.com/langgenius/dify-docs/issues/new"
|
||||
url: 'https://github.com/langgenius/dify-docs/issues/new'
|
||||
about: Report issues with the documentation, such as typos, outdated information, or missing content. Please provide the specific section and details of the issue.
|
||||
- name: "\U0001F4E7 Discussions"
|
||||
url: https://github.com/langgenius/dify/discussions/categories/general
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
name: "⭐ Feature or enhancement request"
|
||||
name: '⭐ Feature or enhancement request'
|
||||
description: Propose something new.
|
||||
labels:
|
||||
- enhancement
|
||||
@@ -6,7 +6,7 @@ body:
|
||||
- type: checkboxes
|
||||
attributes:
|
||||
label: Self Checks
|
||||
description: "To make sure we get to you in time, please check the following :)"
|
||||
description: 'To make sure we get to you in time, please check the following :)'
|
||||
options:
|
||||
- label: I have read the [Contributing Guide](https://github.com/langgenius/dify/blob/main/CONTRIBUTING.md) and [Language Policy](https://github.com/langgenius/dify/issues/1542).
|
||||
required: true
|
||||
@@ -14,7 +14,7 @@ body:
|
||||
required: true
|
||||
- label: I confirm that I am using English to submit this report, otherwise it will be closed.
|
||||
required: true
|
||||
- label: "Please do not modify this template :) and fill in all the required fields."
|
||||
- label: 'Please do not modify this template :) and fill in all the required fields.'
|
||||
required: true
|
||||
- type: textarea
|
||||
attributes:
|
||||
|
||||
@@ -1,11 +1,11 @@
|
||||
name: "✨ Refactor or Chore"
|
||||
name: '✨ Refactor or Chore'
|
||||
description: Refactor existing code or perform maintenance chores to improve readability and reliability.
|
||||
title: "[Refactor/Chore] "
|
||||
title: '[Refactor/Chore] '
|
||||
body:
|
||||
- type: checkboxes
|
||||
attributes:
|
||||
label: Self Checks
|
||||
description: "To make sure we get to you in time, please check the following :)"
|
||||
description: 'To make sure we get to you in time, please check the following :)'
|
||||
options:
|
||||
- label: I have read the [Contributing Guide](https://github.com/langgenius/dify/blob/main/CONTRIBUTING.md) and [Language Policy](https://github.com/langgenius/dify/issues/1542).
|
||||
required: true
|
||||
@@ -17,26 +17,26 @@ body:
|
||||
required: true
|
||||
- label: 【中文用户 & Non English User】请使用英语提交,否则会被关闭 :)
|
||||
required: true
|
||||
- label: "Please do not modify this template :) and fill in all the required fields."
|
||||
- label: 'Please do not modify this template :) and fill in all the required fields.'
|
||||
required: true
|
||||
- type: textarea
|
||||
id: description
|
||||
attributes:
|
||||
label: Description
|
||||
placeholder: "Describe the refactor or chore you are proposing."
|
||||
placeholder: 'Describe the refactor or chore you are proposing.'
|
||||
validations:
|
||||
required: true
|
||||
- type: textarea
|
||||
id: motivation
|
||||
attributes:
|
||||
label: Motivation
|
||||
placeholder: "Explain why this refactor or chore is necessary."
|
||||
placeholder: 'Explain why this refactor or chore is necessary.'
|
||||
validations:
|
||||
required: false
|
||||
- type: textarea
|
||||
id: additional-context
|
||||
attributes:
|
||||
label: Additional Context
|
||||
placeholder: "Add any other context or screenshots about the request here."
|
||||
placeholder: 'Add any other context or screenshots about the request here.'
|
||||
validations:
|
||||
required: false
|
||||
|
||||
+164
-164
@@ -1,223 +1,223 @@
|
||||
version: 2
|
||||
|
||||
updates:
|
||||
- package-ecosystem: "uv"
|
||||
directory: "/api"
|
||||
- package-ecosystem: 'uv'
|
||||
directory: '/api'
|
||||
open-pull-requests-limit: 10
|
||||
schedule:
|
||||
interval: "weekly"
|
||||
interval: 'weekly'
|
||||
groups:
|
||||
flask:
|
||||
patterns:
|
||||
- "flask"
|
||||
- "flask-*"
|
||||
- "werkzeug"
|
||||
- "gunicorn"
|
||||
- 'flask'
|
||||
- 'flask-*'
|
||||
- 'werkzeug'
|
||||
- 'gunicorn'
|
||||
google:
|
||||
patterns:
|
||||
- "google-*"
|
||||
- "googleapis-*"
|
||||
- 'google-*'
|
||||
- 'googleapis-*'
|
||||
opentelemetry:
|
||||
patterns:
|
||||
- "opentelemetry-*"
|
||||
- 'opentelemetry-*'
|
||||
pydantic:
|
||||
patterns:
|
||||
- "pydantic"
|
||||
- "pydantic-*"
|
||||
- 'pydantic'
|
||||
- 'pydantic-*'
|
||||
llm:
|
||||
patterns:
|
||||
- "langfuse"
|
||||
- "langsmith"
|
||||
- "litellm"
|
||||
- "mlflow*"
|
||||
- "opik"
|
||||
- "weave*"
|
||||
- "arize*"
|
||||
- "tiktoken"
|
||||
- "transformers"
|
||||
- 'langfuse'
|
||||
- 'langsmith'
|
||||
- 'litellm'
|
||||
- 'mlflow*'
|
||||
- 'opik'
|
||||
- 'weave*'
|
||||
- 'arize*'
|
||||
- 'tiktoken'
|
||||
- 'transformers'
|
||||
database:
|
||||
patterns:
|
||||
- "sqlalchemy"
|
||||
- "psycopg2*"
|
||||
- "psycogreen"
|
||||
- "redis*"
|
||||
- "alembic*"
|
||||
- 'sqlalchemy'
|
||||
- 'psycopg2*'
|
||||
- 'psycogreen'
|
||||
- 'redis*'
|
||||
- 'alembic*'
|
||||
storage:
|
||||
patterns:
|
||||
- "boto3*"
|
||||
- "botocore*"
|
||||
- "azure-*"
|
||||
- "bce-*"
|
||||
- "cos-python-*"
|
||||
- "esdk-obs-*"
|
||||
- "google-cloud-storage"
|
||||
- "opendal"
|
||||
- "oss2"
|
||||
- "supabase*"
|
||||
- "tos*"
|
||||
- 'boto3*'
|
||||
- 'botocore*'
|
||||
- 'azure-*'
|
||||
- 'bce-*'
|
||||
- 'cos-python-*'
|
||||
- 'esdk-obs-*'
|
||||
- 'google-cloud-storage'
|
||||
- 'opendal'
|
||||
- 'oss2'
|
||||
- 'supabase*'
|
||||
- 'tos*'
|
||||
vdb:
|
||||
patterns:
|
||||
- "alibabacloud*"
|
||||
- "chromadb"
|
||||
- "clickhouse-*"
|
||||
- "clickzetta-*"
|
||||
- "couchbase"
|
||||
- "elasticsearch"
|
||||
- "opensearch-py"
|
||||
- "oracledb"
|
||||
- "pgvect*"
|
||||
- "pymilvus"
|
||||
- "pymochow"
|
||||
- "pyobvector"
|
||||
- "qdrant-client"
|
||||
- "intersystems-*"
|
||||
- "tablestore"
|
||||
- "tcvectordb"
|
||||
- "tidb-vector"
|
||||
- "upstash-*"
|
||||
- "volcengine-*"
|
||||
- "weaviate-*"
|
||||
- "xinference-*"
|
||||
- "mo-vector"
|
||||
- "mysql-connector-*"
|
||||
- 'alibabacloud*'
|
||||
- 'chromadb'
|
||||
- 'clickhouse-*'
|
||||
- 'clickzetta-*'
|
||||
- 'couchbase'
|
||||
- 'elasticsearch'
|
||||
- 'opensearch-py'
|
||||
- 'oracledb'
|
||||
- 'pgvect*'
|
||||
- 'pymilvus'
|
||||
- 'pymochow'
|
||||
- 'pyobvector'
|
||||
- 'qdrant-client'
|
||||
- 'intersystems-*'
|
||||
- 'tablestore'
|
||||
- 'tcvectordb'
|
||||
- 'tidb-vector'
|
||||
- 'upstash-*'
|
||||
- 'volcengine-*'
|
||||
- 'weaviate-*'
|
||||
- 'xinference-*'
|
||||
- 'mo-vector'
|
||||
- 'mysql-connector-*'
|
||||
dev:
|
||||
patterns:
|
||||
- "coverage"
|
||||
- "dotenv-linter"
|
||||
- "faker"
|
||||
- "lxml-stubs"
|
||||
- "basedpyright"
|
||||
- "ruff"
|
||||
- "pytest*"
|
||||
- "types-*"
|
||||
- "boto3-stubs"
|
||||
- "hypothesis"
|
||||
- "pandas-stubs"
|
||||
- "scipy-stubs"
|
||||
- "import-linter"
|
||||
- "celery-types"
|
||||
- "mypy*"
|
||||
- "pyrefly"
|
||||
- 'coverage'
|
||||
- 'dotenv-linter'
|
||||
- 'faker'
|
||||
- 'lxml-stubs'
|
||||
- 'basedpyright'
|
||||
- 'ruff'
|
||||
- 'pytest*'
|
||||
- 'types-*'
|
||||
- 'boto3-stubs'
|
||||
- 'hypothesis'
|
||||
- 'pandas-stubs'
|
||||
- 'scipy-stubs'
|
||||
- 'import-linter'
|
||||
- 'celery-types'
|
||||
- 'mypy*'
|
||||
- 'pyrefly'
|
||||
python-packages:
|
||||
patterns:
|
||||
- "*"
|
||||
- package-ecosystem: "github-actions"
|
||||
directory: "/"
|
||||
- '*'
|
||||
- package-ecosystem: 'github-actions'
|
||||
directory: '/'
|
||||
open-pull-requests-limit: 5
|
||||
schedule:
|
||||
interval: "weekly"
|
||||
interval: 'weekly'
|
||||
groups:
|
||||
github-actions-dependencies:
|
||||
patterns:
|
||||
- "*"
|
||||
- package-ecosystem: "uv"
|
||||
directory: "/api"
|
||||
target-branch: "lts/1.13.x"
|
||||
- '*'
|
||||
- package-ecosystem: 'uv'
|
||||
directory: '/api'
|
||||
target-branch: 'lts/1.13.x'
|
||||
open-pull-requests-limit: 10
|
||||
schedule:
|
||||
interval: "weekly"
|
||||
interval: 'weekly'
|
||||
groups:
|
||||
flask:
|
||||
patterns:
|
||||
- "flask"
|
||||
- "flask-*"
|
||||
- "werkzeug"
|
||||
- "gunicorn"
|
||||
- 'flask'
|
||||
- 'flask-*'
|
||||
- 'werkzeug'
|
||||
- 'gunicorn'
|
||||
google:
|
||||
patterns:
|
||||
- "google-*"
|
||||
- "googleapis-*"
|
||||
- 'google-*'
|
||||
- 'googleapis-*'
|
||||
opentelemetry:
|
||||
patterns:
|
||||
- "opentelemetry-*"
|
||||
- 'opentelemetry-*'
|
||||
pydantic:
|
||||
patterns:
|
||||
- "pydantic"
|
||||
- "pydantic-*"
|
||||
- 'pydantic'
|
||||
- 'pydantic-*'
|
||||
llm:
|
||||
patterns:
|
||||
- "langfuse"
|
||||
- "langsmith"
|
||||
- "litellm"
|
||||
- "mlflow*"
|
||||
- "opik"
|
||||
- "weave*"
|
||||
- "arize*"
|
||||
- "tiktoken"
|
||||
- "transformers"
|
||||
- 'langfuse'
|
||||
- 'langsmith'
|
||||
- 'litellm'
|
||||
- 'mlflow*'
|
||||
- 'opik'
|
||||
- 'weave*'
|
||||
- 'arize*'
|
||||
- 'tiktoken'
|
||||
- 'transformers'
|
||||
database:
|
||||
patterns:
|
||||
- "sqlalchemy"
|
||||
- "psycopg2*"
|
||||
- "psycogreen"
|
||||
- "redis*"
|
||||
- "alembic*"
|
||||
- 'sqlalchemy'
|
||||
- 'psycopg2*'
|
||||
- 'psycogreen'
|
||||
- 'redis*'
|
||||
- 'alembic*'
|
||||
storage:
|
||||
patterns:
|
||||
- "boto3*"
|
||||
- "botocore*"
|
||||
- "azure-*"
|
||||
- "bce-*"
|
||||
- "cos-python-*"
|
||||
- "esdk-obs-*"
|
||||
- "google-cloud-storage"
|
||||
- "opendal"
|
||||
- "oss2"
|
||||
- "supabase*"
|
||||
- "tos*"
|
||||
- 'boto3*'
|
||||
- 'botocore*'
|
||||
- 'azure-*'
|
||||
- 'bce-*'
|
||||
- 'cos-python-*'
|
||||
- 'esdk-obs-*'
|
||||
- 'google-cloud-storage'
|
||||
- 'opendal'
|
||||
- 'oss2'
|
||||
- 'supabase*'
|
||||
- 'tos*'
|
||||
vdb:
|
||||
patterns:
|
||||
- "alibabacloud*"
|
||||
- "chromadb"
|
||||
- "clickhouse-*"
|
||||
- "clickzetta-*"
|
||||
- "couchbase"
|
||||
- "elasticsearch"
|
||||
- "opensearch-py"
|
||||
- "oracledb"
|
||||
- "pgvect*"
|
||||
- "pymilvus"
|
||||
- "pymochow"
|
||||
- "pyobvector"
|
||||
- "qdrant-client"
|
||||
- "intersystems-*"
|
||||
- "tablestore"
|
||||
- "tcvectordb"
|
||||
- "tidb-vector"
|
||||
- "upstash-*"
|
||||
- "volcengine-*"
|
||||
- "weaviate-*"
|
||||
- "xinference-*"
|
||||
- "mo-vector"
|
||||
- "mysql-connector-*"
|
||||
- 'alibabacloud*'
|
||||
- 'chromadb'
|
||||
- 'clickhouse-*'
|
||||
- 'clickzetta-*'
|
||||
- 'couchbase'
|
||||
- 'elasticsearch'
|
||||
- 'opensearch-py'
|
||||
- 'oracledb'
|
||||
- 'pgvect*'
|
||||
- 'pymilvus'
|
||||
- 'pymochow'
|
||||
- 'pyobvector'
|
||||
- 'qdrant-client'
|
||||
- 'intersystems-*'
|
||||
- 'tablestore'
|
||||
- 'tcvectordb'
|
||||
- 'tidb-vector'
|
||||
- 'upstash-*'
|
||||
- 'volcengine-*'
|
||||
- 'weaviate-*'
|
||||
- 'xinference-*'
|
||||
- 'mo-vector'
|
||||
- 'mysql-connector-*'
|
||||
dev:
|
||||
patterns:
|
||||
- "coverage"
|
||||
- "dotenv-linter"
|
||||
- "faker"
|
||||
- "lxml-stubs"
|
||||
- "basedpyright"
|
||||
- "ruff"
|
||||
- "pytest*"
|
||||
- "types-*"
|
||||
- "boto3-stubs"
|
||||
- "hypothesis"
|
||||
- "pandas-stubs"
|
||||
- "scipy-stubs"
|
||||
- "import-linter"
|
||||
- "celery-types"
|
||||
- "mypy*"
|
||||
- "pyrefly"
|
||||
- 'coverage'
|
||||
- 'dotenv-linter'
|
||||
- 'faker'
|
||||
- 'lxml-stubs'
|
||||
- 'basedpyright'
|
||||
- 'ruff'
|
||||
- 'pytest*'
|
||||
- 'types-*'
|
||||
- 'boto3-stubs'
|
||||
- 'hypothesis'
|
||||
- 'pandas-stubs'
|
||||
- 'scipy-stubs'
|
||||
- 'import-linter'
|
||||
- 'celery-types'
|
||||
- 'mypy*'
|
||||
- 'pyrefly'
|
||||
python-packages:
|
||||
patterns:
|
||||
- "*"
|
||||
- package-ecosystem: "github-actions"
|
||||
directory: "/"
|
||||
target-branch: "lts/1.13.x"
|
||||
- '*'
|
||||
- package-ecosystem: 'github-actions'
|
||||
directory: '/'
|
||||
target-branch: 'lts/1.13.x'
|
||||
open-pull-requests-limit: 5
|
||||
schedule:
|
||||
interval: "weekly"
|
||||
interval: 'weekly'
|
||||
groups:
|
||||
github-actions-dependencies:
|
||||
patterns:
|
||||
- "*"
|
||||
- '*'
|
||||
|
||||
@@ -7,9 +7,3 @@ web:
|
||||
- 'pnpm-lock.yaml'
|
||||
- 'pnpm-workspace.yaml'
|
||||
- '.nvmrc'
|
||||
|
||||
e2e:
|
||||
- changed-files:
|
||||
- any-glob-to-any-file:
|
||||
- 'e2e/**'
|
||||
- '.github/workflows/web-e2e.yml'
|
||||
|
||||
@@ -1 +1 @@
|
||||
failure-threshold: "error"
|
||||
failure-threshold: 'error'
|
||||
|
||||
@@ -1,5 +1,4 @@
|
||||
---
|
||||
|
||||
extends: default
|
||||
|
||||
rules:
|
||||
|
||||
@@ -1,22 +1,22 @@
|
||||
{
|
||||
"Verbose": false,
|
||||
"Debug": false,
|
||||
"IgnoreDefaults": false,
|
||||
"SpacesAfterTabs": false,
|
||||
"NoColor": false,
|
||||
"Exclude": [
|
||||
"^web/public/vs/",
|
||||
"^web/public/pdf.worker.min.mjs$",
|
||||
"web/app/components/base/icons/src/vender/"
|
||||
],
|
||||
"AllowedContentTypes": [],
|
||||
"PassedFiles": [],
|
||||
"Disable": {
|
||||
"EndOfLine": false,
|
||||
"Indentation": false,
|
||||
"IndentSize": true,
|
||||
"InsertFinalNewline": false,
|
||||
"TrimTrailingWhitespace": false,
|
||||
"MaxLineLength": false
|
||||
}
|
||||
"Verbose": false,
|
||||
"Debug": false,
|
||||
"IgnoreDefaults": false,
|
||||
"SpacesAfterTabs": false,
|
||||
"NoColor": false,
|
||||
"Exclude": [
|
||||
"^web/public/vs/",
|
||||
"^web/public/pdf.worker.min.mjs$",
|
||||
"web/app/components/base/icons/src/vender/"
|
||||
],
|
||||
"AllowedContentTypes": [],
|
||||
"PassedFiles": [],
|
||||
"Disable": {
|
||||
"EndOfLine": false,
|
||||
"Indentation": false,
|
||||
"IndentSize": true,
|
||||
"InsertFinalNewline": false,
|
||||
"TrimTrailingWhitespace": false,
|
||||
"MaxLineLength": false
|
||||
}
|
||||
}
|
||||
|
||||
@@ -12,8 +12,8 @@
|
||||
## Screenshots
|
||||
|
||||
| Before | After |
|
||||
|--------|-------|
|
||||
| ... | ... |
|
||||
| ------ | ----- |
|
||||
| ... | ... |
|
||||
|
||||
## Checklist
|
||||
|
||||
|
||||
@@ -8,31 +8,31 @@ const headSha = process.env.HEAD_SHA || ''
|
||||
const files = (process.env.CHANGED_FILES || '').split(/\s+/).filter(Boolean)
|
||||
const outputPath = process.env.I18N_CHANGES_OUTPUT_PATH || '/tmp/i18n-changes.json'
|
||||
|
||||
const englishPath = fileStem => path.join(repoRoot, 'web', 'i18n', 'en-US', `${fileStem}.json`)
|
||||
const englishPath = (fileStem) => path.join(repoRoot, 'web', 'i18n', 'en-US', `${fileStem}.json`)
|
||||
|
||||
const readCurrentJson = (fileStem) => {
|
||||
const filePath = englishPath(fileStem)
|
||||
if (!fs.existsSync(filePath))
|
||||
return null
|
||||
if (!fs.existsSync(filePath)) return null
|
||||
|
||||
return JSON.parse(fs.readFileSync(filePath, 'utf8'))
|
||||
}
|
||||
|
||||
const readBaseJson = (fileStem) => {
|
||||
if (!baseSha)
|
||||
return null
|
||||
if (!baseSha) return null
|
||||
|
||||
try {
|
||||
const relativePath = `web/i18n/en-US/${fileStem}.json`
|
||||
const content = execFileSync('git', ['show', `${baseSha}:${relativePath}`], { encoding: 'utf8' })
|
||||
const content = execFileSync('git', ['show', `${baseSha}:${relativePath}`], {
|
||||
encoding: 'utf8',
|
||||
})
|
||||
return JSON.parse(content)
|
||||
}
|
||||
catch {
|
||||
} catch {
|
||||
return null
|
||||
}
|
||||
}
|
||||
|
||||
const compareJson = (beforeValue, afterValue) => JSON.stringify(beforeValue) === JSON.stringify(afterValue)
|
||||
const compareJson = (beforeValue, afterValue) =>
|
||||
JSON.stringify(beforeValue) === JSON.stringify(afterValue)
|
||||
|
||||
const changes = {}
|
||||
|
||||
@@ -59,8 +59,7 @@ for (const fileStem of files) {
|
||||
}
|
||||
|
||||
for (const key of Object.keys(beforeJson)) {
|
||||
if (!(key in afterJson))
|
||||
deleted.push(key)
|
||||
if (!(key in afterJson)) deleted.push(key)
|
||||
}
|
||||
|
||||
changes[fileStem] = {
|
||||
@@ -78,5 +77,5 @@ fs.writeFileSync(
|
||||
headSha,
|
||||
files,
|
||||
changes,
|
||||
})
|
||||
}),
|
||||
)
|
||||
|
||||
@@ -25,17 +25,17 @@ jobs:
|
||||
strategy:
|
||||
matrix:
|
||||
python-version:
|
||||
- "3.12"
|
||||
- '3.12'
|
||||
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
|
||||
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
||||
with:
|
||||
fetch-depth: 0
|
||||
persist-credentials: false
|
||||
|
||||
- name: Setup UV and Python
|
||||
uses: astral-sh/setup-uv@fac544c07dec837d0ccb6301d7b5580bf5edae39 # v8.2.0
|
||||
uses: astral-sh/setup-uv@08807647e7069bb48b6ef5acd8ec9567f424441b # v8.1.0
|
||||
with:
|
||||
enable-cache: true
|
||||
python-version: ${{ matrix.python-version }}
|
||||
@@ -87,17 +87,17 @@ jobs:
|
||||
strategy:
|
||||
matrix:
|
||||
python-version:
|
||||
- "3.12"
|
||||
- '3.12'
|
||||
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
|
||||
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
||||
with:
|
||||
fetch-depth: 0
|
||||
persist-credentials: false
|
||||
|
||||
- name: Setup UV and Python
|
||||
uses: astral-sh/setup-uv@fac544c07dec837d0ccb6301d7b5580bf5edae39 # v8.2.0
|
||||
uses: astral-sh/setup-uv@08807647e7069bb48b6ef5acd8ec9567f424441b # v8.1.0
|
||||
with:
|
||||
enable-cache: true
|
||||
python-version: ${{ matrix.python-version }}
|
||||
@@ -142,16 +142,16 @@ jobs:
|
||||
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
|
||||
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
||||
with:
|
||||
fetch-depth: 0
|
||||
persist-credentials: false
|
||||
|
||||
- name: Setup UV and Python
|
||||
uses: astral-sh/setup-uv@fac544c07dec837d0ccb6301d7b5580bf5edae39 # v8.2.0
|
||||
uses: astral-sh/setup-uv@08807647e7069bb48b6ef5acd8ec9567f424441b # v8.1.0
|
||||
with:
|
||||
enable-cache: true
|
||||
python-version: "3.12"
|
||||
python-version: '3.12'
|
||||
cache-dependency-glob: api/uv.lock
|
||||
|
||||
- name: Install dependencies
|
||||
@@ -195,7 +195,7 @@ jobs:
|
||||
|
||||
- name: Report coverage
|
||||
if: ${{ env.CODECOV_TOKEN != '' }}
|
||||
uses: codecov/codecov-action@fb8b3582c8e4def4969c97caa2f19720cb33a72f # v7.0.0
|
||||
uses: codecov/codecov-action@e79a6962e0d4c0c17b229090214935d2e33f8354 # v6.0.1
|
||||
with:
|
||||
files: ./coverage.xml
|
||||
disable_search: true
|
||||
|
||||
@@ -1,12 +1,12 @@
|
||||
name: autofix.ci
|
||||
on:
|
||||
pull_request:
|
||||
branches: ["main"]
|
||||
branches: ['main']
|
||||
merge_group:
|
||||
branches: ["main"]
|
||||
branches: ['main']
|
||||
types: [checks_requested]
|
||||
push:
|
||||
branches: ["main"]
|
||||
branches: ['main']
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
@@ -20,7 +20,7 @@ jobs:
|
||||
run: echo "autofix.ci updates pull request branches, not merge group refs."
|
||||
|
||||
- if: github.event_name != 'merge_group'
|
||||
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
|
||||
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
||||
|
||||
- name: Check Docker Compose inputs
|
||||
if: github.event_name != 'merge_group'
|
||||
@@ -44,6 +44,12 @@ jobs:
|
||||
pnpm-lock.yaml
|
||||
pnpm-workspace.yaml
|
||||
.nvmrc
|
||||
vite.config.ts
|
||||
eslint.config.mjs
|
||||
web/eslint.config.mjs
|
||||
.vscode/**
|
||||
.github/workflows/autofix.yml
|
||||
.github/workflows/style.yml
|
||||
- name: Check api inputs
|
||||
if: github.event_name != 'merge_group'
|
||||
id: api-changes
|
||||
@@ -51,18 +57,6 @@ jobs:
|
||||
with:
|
||||
files: |
|
||||
api/**
|
||||
- name: Check frontend contract inputs
|
||||
if: github.event_name != 'merge_group'
|
||||
id: frontend-contract-changes
|
||||
uses: tj-actions/changed-files@9426d40962ed5378910ee2e21d5f8c6fcbf2dd96 # v47.0.6
|
||||
with:
|
||||
files: |
|
||||
api/**
|
||||
packages/contracts/openapi-ts.api.config.ts
|
||||
packages/contracts/package.json
|
||||
packages/contracts/openapi/**
|
||||
pnpm-lock.yaml
|
||||
pnpm-workspace.yaml
|
||||
- name: Check dify-agent inputs
|
||||
if: github.event_name != 'merge_group'
|
||||
id: dify-agent-changes
|
||||
@@ -75,10 +69,10 @@ jobs:
|
||||
- if: github.event_name != 'merge_group'
|
||||
uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
|
||||
with:
|
||||
python-version: "3.11"
|
||||
python-version: '3.11'
|
||||
|
||||
- if: github.event_name != 'merge_group'
|
||||
uses: astral-sh/setup-uv@fac544c07dec837d0ccb6301d7b5580bf5edae39 # v8.2.0
|
||||
uses: astral-sh/setup-uv@08807647e7069bb48b6ef5acd8ec9567f424441b # v8.1.0
|
||||
|
||||
- name: Generate Docker Compose
|
||||
if: github.event_name != 'merge_group' && steps.docker-compose-changes.outputs.any_changed == 'true'
|
||||
@@ -155,8 +149,14 @@ jobs:
|
||||
uv run dev/generate_swagger_markdown_docs.py --swagger-dir ../packages/contracts/openapi --markdown-dir openapi/markdown --keep-swagger-json
|
||||
|
||||
- name: Generate frontend contracts
|
||||
if: github.event_name != 'merge_group' && steps.frontend-contract-changes.outputs.any_changed == 'true'
|
||||
run: pnpm --dir packages/contracts gen-api-contract
|
||||
if: github.event_name != 'merge_group' && steps.api-changes.outputs.any_changed == 'true'
|
||||
run: pnpm --dir packages/contracts gen-api-contract-from-openapi
|
||||
|
||||
- name: Format web files
|
||||
if: github.event_name != 'merge_group' && steps.web-changes.outputs.any_changed == 'true'
|
||||
env:
|
||||
CHANGED_FILES: ${{ steps.web-changes.outputs.all_changed_files }}
|
||||
run: vp fmt --no-error-on-unmatched-pattern $CHANGED_FILES
|
||||
|
||||
- name: ESLint autofix
|
||||
if: github.event_name != 'merge_group' && steps.web-changes.outputs.any_changed == 'true'
|
||||
|
||||
@@ -3,13 +3,14 @@ name: Build and Push API & Web
|
||||
on:
|
||||
push:
|
||||
branches:
|
||||
- "main"
|
||||
- "deploy/**"
|
||||
- "build/**"
|
||||
- "release/e-*"
|
||||
- "hotfix/**"
|
||||
- 'main'
|
||||
- 'deploy/**'
|
||||
- 'build/**'
|
||||
- 'release/e-*'
|
||||
- 'hotfix/**'
|
||||
- 'feat/hitl-backend'
|
||||
tags:
|
||||
- "*"
|
||||
- '*'
|
||||
|
||||
concurrency:
|
||||
group: build-push-${{ github.head_ref || github.run_id }}
|
||||
@@ -20,8 +21,6 @@ env:
|
||||
DOCKERHUB_TOKEN: ${{ secrets.DOCKERHUB_TOKEN }}
|
||||
DIFY_WEB_IMAGE_NAME: ${{ vars.DIFY_WEB_IMAGE_NAME || 'langgenius/dify-web' }}
|
||||
DIFY_API_IMAGE_NAME: ${{ vars.DIFY_API_IMAGE_NAME || 'langgenius/dify-api' }}
|
||||
DIFY_AGENT_IMAGE_NAME: ${{ vars.DIFY_AGENT_IMAGE_NAME || 'langgenius/dify-agent-backend' }}
|
||||
DIFY_AGENT_LOCAL_SANDBOX_IMAGE_NAME: ${{ vars.DIFY_AGENT_LOCAL_SANDBOX_IMAGE_NAME || 'langgenius/dify-agent-local-sandbox' }}
|
||||
|
||||
jobs:
|
||||
build:
|
||||
@@ -33,60 +32,32 @@ jobs:
|
||||
strategy:
|
||||
matrix:
|
||||
include:
|
||||
- service_name: "build-api-amd64"
|
||||
image_name_env: "DIFY_API_IMAGE_NAME"
|
||||
artifact_context: "api"
|
||||
build_context: "{{defaultContext}}"
|
||||
file: "api/Dockerfile"
|
||||
- service_name: 'build-api-amd64'
|
||||
image_name_env: 'DIFY_API_IMAGE_NAME'
|
||||
artifact_context: 'api'
|
||||
build_context: '{{defaultContext}}'
|
||||
file: 'api/Dockerfile'
|
||||
platform: linux/amd64
|
||||
runs_on: depot-ubuntu-24.04-4
|
||||
- service_name: "build-api-arm64"
|
||||
image_name_env: "DIFY_API_IMAGE_NAME"
|
||||
artifact_context: "api"
|
||||
build_context: "{{defaultContext}}"
|
||||
file: "api/Dockerfile"
|
||||
- service_name: 'build-api-arm64'
|
||||
image_name_env: 'DIFY_API_IMAGE_NAME'
|
||||
artifact_context: 'api'
|
||||
build_context: '{{defaultContext}}'
|
||||
file: 'api/Dockerfile'
|
||||
platform: linux/arm64
|
||||
runs_on: depot-ubuntu-24.04-4
|
||||
- service_name: "build-web-amd64"
|
||||
image_name_env: "DIFY_WEB_IMAGE_NAME"
|
||||
artifact_context: "web"
|
||||
build_context: "{{defaultContext}}"
|
||||
file: "web/Dockerfile"
|
||||
- service_name: 'build-web-amd64'
|
||||
image_name_env: 'DIFY_WEB_IMAGE_NAME'
|
||||
artifact_context: 'web'
|
||||
build_context: '{{defaultContext}}'
|
||||
file: 'web/Dockerfile'
|
||||
platform: linux/amd64
|
||||
runs_on: depot-ubuntu-24.04-4
|
||||
- service_name: "build-web-arm64"
|
||||
image_name_env: "DIFY_WEB_IMAGE_NAME"
|
||||
artifact_context: "web"
|
||||
build_context: "{{defaultContext}}"
|
||||
file: "web/Dockerfile"
|
||||
platform: linux/arm64
|
||||
runs_on: depot-ubuntu-24.04-4
|
||||
- service_name: "build-agent-amd64"
|
||||
image_name_env: "DIFY_AGENT_IMAGE_NAME"
|
||||
artifact_context: "agent"
|
||||
build_context: "{{defaultContext}}"
|
||||
file: "dify-agent/Dockerfile"
|
||||
platform: linux/amd64
|
||||
runs_on: depot-ubuntu-24.04-4
|
||||
- service_name: "build-agent-arm64"
|
||||
image_name_env: "DIFY_AGENT_IMAGE_NAME"
|
||||
artifact_context: "agent"
|
||||
build_context: "{{defaultContext}}"
|
||||
file: "dify-agent/Dockerfile"
|
||||
platform: linux/arm64
|
||||
runs_on: depot-ubuntu-24.04-4
|
||||
- service_name: "build-agent-local-sandbox-amd64"
|
||||
image_name_env: "DIFY_AGENT_LOCAL_SANDBOX_IMAGE_NAME"
|
||||
artifact_context: "local-sandbox"
|
||||
build_context: "{{defaultContext}}:dify-agent"
|
||||
file: "docker/local-sandbox/Dockerfile"
|
||||
platform: linux/amd64
|
||||
runs_on: depot-ubuntu-24.04-4
|
||||
- service_name: "build-agent-local-sandbox-arm64"
|
||||
image_name_env: "DIFY_AGENT_LOCAL_SANDBOX_IMAGE_NAME"
|
||||
artifact_context: "local-sandbox"
|
||||
build_context: "{{defaultContext}}:dify-agent"
|
||||
file: "docker/local-sandbox/Dockerfile"
|
||||
- service_name: 'build-web-arm64'
|
||||
image_name_env: 'DIFY_WEB_IMAGE_NAME'
|
||||
artifact_context: 'web'
|
||||
build_context: '{{defaultContext}}'
|
||||
file: 'web/Dockerfile'
|
||||
platform: linux/arm64
|
||||
runs_on: depot-ubuntu-24.04-4
|
||||
|
||||
@@ -97,7 +68,7 @@ jobs:
|
||||
echo "PLATFORM_PAIR=${platform//\//-}" >> $GITHUB_ENV
|
||||
|
||||
- name: Login to Docker Hub
|
||||
uses: docker/login-action@650006c6eb7dba73a995cc03b0b2d7f5ca915bee # v4.2.0
|
||||
uses: docker/login-action@4907a6ddec9925e35a0a9e82d7399ccc52663121 # v4.1.0
|
||||
with:
|
||||
username: ${{ env.DOCKERHUB_USER }}
|
||||
password: ${{ env.DOCKERHUB_TOKEN }}
|
||||
@@ -107,13 +78,13 @@ jobs:
|
||||
|
||||
- name: Extract metadata for Docker
|
||||
id: meta
|
||||
uses: docker/metadata-action@80c7e94dd9b9319bd5eb7a0e0fe9291e23a2a2e9 # v6.1.0
|
||||
uses: docker/metadata-action@030e881283bb7a6894de51c315a6bfe6a94e05cf # v6.0.0
|
||||
with:
|
||||
images: ${{ env[matrix.image_name_env] }}
|
||||
|
||||
- name: Build Docker image
|
||||
id: build
|
||||
uses: depot/build-push-action@98e78adca7817480b8185f474a400b451d74e287 # v1.18.0
|
||||
uses: depot/build-push-action@5f3b3c2e5a00f0093de47f657aeaefcedff27d18 # v1.17.0
|
||||
with:
|
||||
project: ${{ vars.DEPOT_PROJECT_ID }}
|
||||
context: ${{ matrix.build_context }}
|
||||
@@ -145,24 +116,18 @@ jobs:
|
||||
strategy:
|
||||
matrix:
|
||||
include:
|
||||
- service_name: "validate-api-amd64"
|
||||
build_context: "{{defaultContext}}"
|
||||
file: "api/Dockerfile"
|
||||
- service_name: "validate-web-amd64"
|
||||
build_context: "{{defaultContext}}"
|
||||
file: "web/Dockerfile"
|
||||
- service_name: "validate-agent-amd64"
|
||||
build_context: "{{defaultContext}}"
|
||||
file: "dify-agent/Dockerfile"
|
||||
- service_name: "validate-agent-local-sandbox-amd64"
|
||||
build_context: "{{defaultContext}}:dify-agent"
|
||||
file: "docker/local-sandbox/Dockerfile"
|
||||
- service_name: 'validate-api-amd64'
|
||||
build_context: '{{defaultContext}}'
|
||||
file: 'api/Dockerfile'
|
||||
- service_name: 'validate-web-amd64'
|
||||
build_context: '{{defaultContext}}'
|
||||
file: 'web/Dockerfile'
|
||||
steps:
|
||||
- name: Set up Docker Buildx
|
||||
uses: docker/setup-buildx-action@d7f5e7f509e45cec5c76c4d5afdd7de93d0b3df5 # v4.1.0
|
||||
uses: docker/setup-buildx-action@4d04d5d9486b7bd6fa91e7baf45bbb4f8b9deedd # v4.0.0
|
||||
|
||||
- name: Validate Docker image
|
||||
uses: docker/build-push-action@f9f3042f7e2789586610d6e8b85c8f03e5195baf # v7.2.0
|
||||
uses: docker/build-push-action@bcafcacb16a39f128d818304e6c9c0c18556b85f # v7.1.0
|
||||
with:
|
||||
push: false
|
||||
context: ${{ matrix.build_context }}
|
||||
@@ -176,18 +141,12 @@ jobs:
|
||||
strategy:
|
||||
matrix:
|
||||
include:
|
||||
- service_name: "merge-api-images"
|
||||
image_name_env: "DIFY_API_IMAGE_NAME"
|
||||
context: "api"
|
||||
- service_name: "merge-web-images"
|
||||
image_name_env: "DIFY_WEB_IMAGE_NAME"
|
||||
context: "web"
|
||||
- service_name: "merge-agent-images"
|
||||
image_name_env: "DIFY_AGENT_IMAGE_NAME"
|
||||
context: "agent"
|
||||
- service_name: "merge-agent-local-sandbox-images"
|
||||
image_name_env: "DIFY_AGENT_LOCAL_SANDBOX_IMAGE_NAME"
|
||||
context: "local-sandbox"
|
||||
- service_name: 'merge-api-images'
|
||||
image_name_env: 'DIFY_API_IMAGE_NAME'
|
||||
context: 'api'
|
||||
- service_name: 'merge-web-images'
|
||||
image_name_env: 'DIFY_WEB_IMAGE_NAME'
|
||||
context: 'web'
|
||||
steps:
|
||||
- name: Download digests
|
||||
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
|
||||
@@ -197,14 +156,14 @@ jobs:
|
||||
merge-multiple: true
|
||||
|
||||
- name: Login to Docker Hub
|
||||
uses: docker/login-action@650006c6eb7dba73a995cc03b0b2d7f5ca915bee # v4.2.0
|
||||
uses: docker/login-action@4907a6ddec9925e35a0a9e82d7399ccc52663121 # v4.1.0
|
||||
with:
|
||||
username: ${{ env.DOCKERHUB_USER }}
|
||||
password: ${{ env.DOCKERHUB_TOKEN }}
|
||||
|
||||
- name: Extract metadata for Docker
|
||||
id: meta
|
||||
uses: docker/metadata-action@80c7e94dd9b9319bd5eb7a0e0fe9291e23a2a2e9 # v6.1.0
|
||||
uses: docker/metadata-action@030e881283bb7a6894de51c315a6bfe6a94e05cf # v6.0.0
|
||||
with:
|
||||
images: ${{ env[matrix.image_name_env] }}
|
||||
tags: |
|
||||
|
||||
+122
-123
@@ -4,19 +4,19 @@ on:
|
||||
workflow_dispatch:
|
||||
inputs:
|
||||
cli_ref:
|
||||
description: "Git ref (default: current branch)"
|
||||
description: 'Git ref (default: current branch)'
|
||||
type: string
|
||||
required: false
|
||||
|
||||
edition:
|
||||
description: "Dify edition"
|
||||
description: 'Dify edition'
|
||||
type: choice
|
||||
required: false
|
||||
default: ee
|
||||
options: [ee, ce]
|
||||
|
||||
test_scope:
|
||||
description: "smoke = [P0] only / full = all cases"
|
||||
description: 'smoke = [P0] only / full = all cases'
|
||||
type: choice
|
||||
required: false
|
||||
default: full
|
||||
@@ -24,23 +24,23 @@ on:
|
||||
|
||||
# ── Suite on/off ────────────────────────────────────────────────────────
|
||||
suite_framework_output_error:
|
||||
description: "framework + output + error-handling suites"
|
||||
description: 'framework + output + error-handling suites'
|
||||
type: boolean
|
||||
default: true
|
||||
suite_discovery:
|
||||
description: "discovery suite (get app / describe app)"
|
||||
description: 'discovery suite (get app / describe app)'
|
||||
type: boolean
|
||||
default: true
|
||||
suite_run:
|
||||
description: "run suite (basic / streaming / conversation / file / hitl)"
|
||||
description: 'run suite (basic / streaming / conversation / file / hitl)'
|
||||
type: boolean
|
||||
default: true
|
||||
suite_auth:
|
||||
description: "auth suite (login / status / whoami / use / devices / logout)"
|
||||
description: 'auth suite (login / status / whoami / use / devices / logout)'
|
||||
type: boolean
|
||||
default: true
|
||||
suite_agent:
|
||||
description: "agent suite"
|
||||
description: 'agent suite'
|
||||
type: boolean
|
||||
default: true
|
||||
|
||||
@@ -51,35 +51,34 @@ permissions:
|
||||
# Each job reads DIFY_E2E_TOKEN + app IDs from the provision job outputs,
|
||||
# so global-setup skips minting and finds existing apps in < 10 s.
|
||||
env:
|
||||
DIFY_E2E_NO_KEYRING: "1" # Linux CI has no keychain; skip probe
|
||||
VITEST_RETRY: "2" # Retry flaky staging responses
|
||||
DIFY_E2E_NO_KEYRING: '1' # Linux CI has no keychain; skip probe
|
||||
VITEST_RETRY: '2' # Retry flaky staging responses
|
||||
|
||||
jobs:
|
||||
|
||||
# ════════════════════════════════════════════════════════════════════════════
|
||||
# 0. PROVISION — mint token + import DSL fixtures (runs once, outputs IDs)
|
||||
# ════════════════════════════════════════════════════════════════════════════
|
||||
# ════════════════════════════════════════════════════════════════════════════
|
||||
# 0. PROVISION — mint token + import DSL fixtures (runs once, outputs IDs)
|
||||
# ════════════════════════════════════════════════════════════════════════════
|
||||
provision:
|
||||
name: "Provision: mint token + DSL apps"
|
||||
name: 'Provision: mint token + DSL apps'
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 10
|
||||
outputs:
|
||||
token: ${{ steps.out.outputs.DIFY_E2E_TOKEN }}
|
||||
workspace_id: ${{ steps.out.outputs.DIFY_E2E_WORKSPACE_ID }}
|
||||
workspace_name: ${{ steps.out.outputs.DIFY_E2E_WORKSPACE_NAME }}
|
||||
ws2_id: ${{ steps.out.outputs.DIFY_E2E_WS2_ID }}
|
||||
chat_app_id: ${{ steps.out.outputs.DIFY_E2E_CHAT_APP_ID }}
|
||||
workflow_app_id: ${{ steps.out.outputs.DIFY_E2E_WORKFLOW_APP_ID }}
|
||||
file_app_id: ${{ steps.out.outputs.DIFY_E2E_FILE_APP_ID }}
|
||||
file_chat_app_id: ${{ steps.out.outputs.DIFY_E2E_FILE_CHAT_APP_ID }}
|
||||
hitl_app_id: ${{ steps.out.outputs.DIFY_E2E_HITL_APP_ID }}
|
||||
hitl_external_app_id: ${{ steps.out.outputs.DIFY_E2E_HITL_EXTERNAL_APP_ID }}
|
||||
hitl_single_action_app_id: ${{ steps.out.outputs.DIFY_E2E_HITL_SINGLE_ACTION_APP_ID }}
|
||||
hitl_multi_node_app_id: ${{ steps.out.outputs.DIFY_E2E_HITL_MULTI_NODE_APP_ID }}
|
||||
ws2_app_id: ${{ steps.out.outputs.DIFY_E2E_WS2_APP_ID }}
|
||||
token: ${{ steps.out.outputs.DIFY_E2E_TOKEN }}
|
||||
workspace_id: ${{ steps.out.outputs.DIFY_E2E_WORKSPACE_ID }}
|
||||
workspace_name: ${{ steps.out.outputs.DIFY_E2E_WORKSPACE_NAME }}
|
||||
ws2_id: ${{ steps.out.outputs.DIFY_E2E_WS2_ID }}
|
||||
chat_app_id: ${{ steps.out.outputs.DIFY_E2E_CHAT_APP_ID }}
|
||||
workflow_app_id: ${{ steps.out.outputs.DIFY_E2E_WORKFLOW_APP_ID }}
|
||||
file_app_id: ${{ steps.out.outputs.DIFY_E2E_FILE_APP_ID }}
|
||||
file_chat_app_id: ${{ steps.out.outputs.DIFY_E2E_FILE_CHAT_APP_ID }}
|
||||
hitl_app_id: ${{ steps.out.outputs.DIFY_E2E_HITL_APP_ID }}
|
||||
hitl_external_app_id: ${{ steps.out.outputs.DIFY_E2E_HITL_EXTERNAL_APP_ID }}
|
||||
hitl_single_action_app_id: ${{ steps.out.outputs.DIFY_E2E_HITL_SINGLE_ACTION_APP_ID }}
|
||||
hitl_multi_node_app_id: ${{ steps.out.outputs.DIFY_E2E_HITL_MULTI_NODE_APP_ID }}
|
||||
ws2_app_id: ${{ steps.out.outputs.DIFY_E2E_WS2_APP_ID }}
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v4
|
||||
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v4
|
||||
with:
|
||||
ref: ${{ inputs.cli_ref || github.ref }}
|
||||
persist-credentials: false
|
||||
@@ -88,7 +87,7 @@ jobs:
|
||||
with:
|
||||
bun-version: latest
|
||||
|
||||
- uses: pnpm/action-setup@0ebf47130e4866e96fce0953f49152a61190b271 # v6.0.9
|
||||
- uses: pnpm/action-setup@b906affcce14559ad1aafd4ab0e942779e9f58b1 # v4
|
||||
with:
|
||||
package_json_field: packageManager
|
||||
run_install: false
|
||||
@@ -101,18 +100,18 @@ jobs:
|
||||
id: out
|
||||
working-directory: cli
|
||||
env:
|
||||
DIFY_E2E_HOST: ${{ secrets.DIFY_E2E_HOST }}
|
||||
DIFY_E2E_EMAIL: ${{ secrets.DIFY_E2E_EMAIL }}
|
||||
DIFY_E2E_HOST: ${{ secrets.DIFY_E2E_HOST }}
|
||||
DIFY_E2E_EMAIL: ${{ secrets.DIFY_E2E_EMAIL }}
|
||||
DIFY_E2E_PASSWORD: ${{ secrets.DIFY_E2E_PASSWORD }}
|
||||
DIFY_E2E_TOKEN: ${{ secrets.DIFY_E2E_TOKEN }}
|
||||
DIFY_E2E_EDITION: ${{ inputs.edition || 'ee' }}
|
||||
DIFY_E2E_TOKEN: ${{ secrets.DIFY_E2E_TOKEN }}
|
||||
DIFY_E2E_EDITION: ${{ inputs.edition || 'ee' }}
|
||||
run: bun scripts/e2e-provision.ts
|
||||
|
||||
# ════════════════════════════════════════════════════════════════════════════
|
||||
# 1-B. framework + output + error-handling (parallel with run/discovery)
|
||||
# ════════════════════════════════════════════════════════════════════════════
|
||||
# ════════════════════════════════════════════════════════════════════════════
|
||||
# 1-B. framework + output + error-handling (parallel with run/discovery)
|
||||
# ════════════════════════════════════════════════════════════════════════════
|
||||
suite-framework-output-error:
|
||||
name: "Suite: framework + output + error-handling"
|
||||
name: 'Suite: framework + output + error-handling'
|
||||
if: ${{ inputs.suite_framework_output_error != 'false' }}
|
||||
needs: provision
|
||||
runs-on: ubuntu-latest
|
||||
@@ -123,7 +122,7 @@ jobs:
|
||||
shell: bash
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v4
|
||||
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v4
|
||||
with:
|
||||
ref: ${{ inputs.cli_ref || github.ref }}
|
||||
persist-credentials: false
|
||||
@@ -131,23 +130,23 @@ jobs:
|
||||
- uses: ./.github/actions/setup-web
|
||||
- uses: oven-sh/setup-bun@v2
|
||||
with: { bun-version: latest }
|
||||
- uses: pnpm/action-setup@0ebf47130e4866e96fce0953f49152a61190b271 # v6.0.9
|
||||
- uses: pnpm/action-setup@b906affcce14559ad1aafd4ab0e942779e9f58b1 # v4
|
||||
with: { package_json_field: packageManager, run_install: false }
|
||||
- run: pnpm install --frozen-lockfile
|
||||
- run: pnpm tree:gen
|
||||
|
||||
- name: Run framework + output + error-handling
|
||||
env:
|
||||
DIFY_E2E_HOST: ${{ secrets.DIFY_E2E_HOST }}
|
||||
DIFY_E2E_EMAIL: ${{ secrets.DIFY_E2E_EMAIL }}
|
||||
DIFY_E2E_PASSWORD: ${{ secrets.DIFY_E2E_PASSWORD }}
|
||||
DIFY_E2E_EDITION: ${{ inputs.edition || 'ee' }}
|
||||
DIFY_E2E_TOKEN: ${{ needs.provision.outputs.token }}
|
||||
DIFY_E2E_WORKSPACE_ID: ${{ needs.provision.outputs.workspace_id }}
|
||||
DIFY_E2E_HOST: ${{ secrets.DIFY_E2E_HOST }}
|
||||
DIFY_E2E_EMAIL: ${{ secrets.DIFY_E2E_EMAIL }}
|
||||
DIFY_E2E_PASSWORD: ${{ secrets.DIFY_E2E_PASSWORD }}
|
||||
DIFY_E2E_EDITION: ${{ inputs.edition || 'ee' }}
|
||||
DIFY_E2E_TOKEN: ${{ needs.provision.outputs.token }}
|
||||
DIFY_E2E_WORKSPACE_ID: ${{ needs.provision.outputs.workspace_id }}
|
||||
DIFY_E2E_WORKSPACE_NAME: ${{ needs.provision.outputs.workspace_name }}
|
||||
DIFY_E2E_CHAT_APP_ID: ${{ needs.provision.outputs.chat_app_id }}
|
||||
DIFY_E2E_CHAT_APP_ID: ${{ needs.provision.outputs.chat_app_id }}
|
||||
DIFY_E2E_WORKFLOW_APP_ID: ${{ needs.provision.outputs.workflow_app_id }}
|
||||
DIFY_E2E_INCLUDE: "test/e2e/suites/framework/**/*.e2e.ts,test/e2e/suites/output/**/*.e2e.ts,test/e2e/suites/error-handling/**/*.e2e.ts"
|
||||
DIFY_E2E_INCLUDE: 'test/e2e/suites/framework/**/*.e2e.ts,test/e2e/suites/output/**/*.e2e.ts,test/e2e/suites/error-handling/**/*.e2e.ts'
|
||||
run: |
|
||||
if [ "${{ inputs.test_scope }}" = "smoke" ]; then
|
||||
pnpm test:e2e -- -t "\[P0\]"
|
||||
@@ -155,11 +154,11 @@ jobs:
|
||||
pnpm test:e2e
|
||||
fi
|
||||
|
||||
# ════════════════════════════════════════════════════════════════════════════
|
||||
# 1-C. Discovery (parallel)
|
||||
# ════════════════════════════════════════════════════════════════════════════
|
||||
# ════════════════════════════════════════════════════════════════════════════
|
||||
# 1-C. Discovery (parallel)
|
||||
# ════════════════════════════════════════════════════════════════════════════
|
||||
suite-discovery:
|
||||
name: "Suite: discovery"
|
||||
name: 'Suite: discovery'
|
||||
if: ${{ inputs.suite_discovery != 'false' }}
|
||||
needs: provision
|
||||
runs-on: ubuntu-latest
|
||||
@@ -170,7 +169,7 @@ jobs:
|
||||
shell: bash
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v4
|
||||
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v4
|
||||
with:
|
||||
ref: ${{ inputs.cli_ref || github.ref }}
|
||||
persist-credentials: false
|
||||
@@ -178,24 +177,24 @@ jobs:
|
||||
- uses: ./.github/actions/setup-web
|
||||
- uses: oven-sh/setup-bun@v2
|
||||
with: { bun-version: latest }
|
||||
- uses: pnpm/action-setup@0ebf47130e4866e96fce0953f49152a61190b271 # v6.0.9
|
||||
- uses: pnpm/action-setup@b906affcce14559ad1aafd4ab0e942779e9f58b1 # v4
|
||||
with: { package_json_field: packageManager, run_install: false }
|
||||
- run: pnpm install --frozen-lockfile
|
||||
- run: pnpm tree:gen
|
||||
|
||||
- name: Run discovery suite
|
||||
env:
|
||||
DIFY_E2E_HOST: ${{ secrets.DIFY_E2E_HOST }}
|
||||
DIFY_E2E_EMAIL: ${{ secrets.DIFY_E2E_EMAIL }}
|
||||
DIFY_E2E_PASSWORD: ${{ secrets.DIFY_E2E_PASSWORD }}
|
||||
DIFY_E2E_EDITION: ${{ inputs.edition || 'ee' }}
|
||||
DIFY_E2E_TOKEN: ${{ needs.provision.outputs.token }}
|
||||
DIFY_E2E_WORKSPACE_ID: ${{ needs.provision.outputs.workspace_id }}
|
||||
DIFY_E2E_HOST: ${{ secrets.DIFY_E2E_HOST }}
|
||||
DIFY_E2E_EMAIL: ${{ secrets.DIFY_E2E_EMAIL }}
|
||||
DIFY_E2E_PASSWORD: ${{ secrets.DIFY_E2E_PASSWORD }}
|
||||
DIFY_E2E_EDITION: ${{ inputs.edition || 'ee' }}
|
||||
DIFY_E2E_TOKEN: ${{ needs.provision.outputs.token }}
|
||||
DIFY_E2E_WORKSPACE_ID: ${{ needs.provision.outputs.workspace_id }}
|
||||
DIFY_E2E_WORKSPACE_NAME: ${{ needs.provision.outputs.workspace_name }}
|
||||
DIFY_E2E_WS2_ID: ${{ needs.provision.outputs.ws2_id }}
|
||||
DIFY_E2E_CHAT_APP_ID: ${{ needs.provision.outputs.chat_app_id }}
|
||||
DIFY_E2E_WS2_ID: ${{ needs.provision.outputs.ws2_id }}
|
||||
DIFY_E2E_CHAT_APP_ID: ${{ needs.provision.outputs.chat_app_id }}
|
||||
DIFY_E2E_WORKFLOW_APP_ID: ${{ needs.provision.outputs.workflow_app_id }}
|
||||
DIFY_E2E_INCLUDE: "test/e2e/suites/discovery/**/*.e2e.ts"
|
||||
DIFY_E2E_INCLUDE: 'test/e2e/suites/discovery/**/*.e2e.ts'
|
||||
run: |
|
||||
if [ "${{ inputs.test_scope }}" = "smoke" ]; then
|
||||
pnpm test:e2e -- -t "\[P0\]"
|
||||
@@ -203,11 +202,11 @@ jobs:
|
||||
pnpm test:e2e
|
||||
fi
|
||||
|
||||
# ════════════════════════════════════════════════════════════════════════════
|
||||
# 1-D. Run suite — 5 files in matrix (parallel)
|
||||
# ════════════════════════════════════════════════════════════════════════════
|
||||
# ════════════════════════════════════════════════════════════════════════════
|
||||
# 1-D. Run suite — 5 files in matrix (parallel)
|
||||
# ════════════════════════════════════════════════════════════════════════════
|
||||
suite-run:
|
||||
name: "Suite: run / ${{ matrix.name }}"
|
||||
name: 'Suite: run / ${{ matrix.name }}'
|
||||
if: ${{ inputs.suite_run != 'false' }}
|
||||
needs: provision
|
||||
runs-on: ubuntu-latest
|
||||
@@ -233,7 +232,7 @@ jobs:
|
||||
shell: bash
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v4
|
||||
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v4
|
||||
with:
|
||||
ref: ${{ inputs.cli_ref || github.ref }}
|
||||
persist-credentials: false
|
||||
@@ -241,30 +240,30 @@ jobs:
|
||||
- uses: ./.github/actions/setup-web
|
||||
- uses: oven-sh/setup-bun@v2
|
||||
with: { bun-version: latest }
|
||||
- uses: pnpm/action-setup@0ebf47130e4866e96fce0953f49152a61190b271 # v6.0.9
|
||||
- uses: pnpm/action-setup@b906affcce14559ad1aafd4ab0e942779e9f58b1 # v4
|
||||
with: { package_json_field: packageManager, run_install: false }
|
||||
- run: pnpm install --frozen-lockfile
|
||||
- run: pnpm tree:gen
|
||||
|
||||
- name: "Run run/${{ matrix.name }}"
|
||||
- name: 'Run run/${{ matrix.name }}'
|
||||
env:
|
||||
DIFY_E2E_HOST: ${{ secrets.DIFY_E2E_HOST }}
|
||||
DIFY_E2E_EMAIL: ${{ secrets.DIFY_E2E_EMAIL }}
|
||||
DIFY_E2E_PASSWORD: ${{ secrets.DIFY_E2E_PASSWORD }}
|
||||
DIFY_E2E_EDITION: ${{ inputs.edition || 'ee' }}
|
||||
DIFY_E2E_SSO_TOKEN: ${{ secrets.DIFY_E2E_SSO_TOKEN }}
|
||||
DIFY_E2E_TOKEN: ${{ needs.provision.outputs.token }}
|
||||
DIFY_E2E_WORKSPACE_ID: ${{ needs.provision.outputs.workspace_id }}
|
||||
DIFY_E2E_WORKSPACE_NAME: ${{ needs.provision.outputs.workspace_name }}
|
||||
DIFY_E2E_CHAT_APP_ID: ${{ needs.provision.outputs.chat_app_id }}
|
||||
DIFY_E2E_WORKFLOW_APP_ID: ${{ needs.provision.outputs.workflow_app_id }}
|
||||
DIFY_E2E_FILE_APP_ID: ${{ needs.provision.outputs.file_app_id }}
|
||||
DIFY_E2E_FILE_CHAT_APP_ID: ${{ needs.provision.outputs.file_chat_app_id }}
|
||||
DIFY_E2E_HITL_APP_ID: ${{ needs.provision.outputs.hitl_app_id }}
|
||||
DIFY_E2E_HITL_EXTERNAL_APP_ID: ${{ needs.provision.outputs.hitl_external_app_id }}
|
||||
DIFY_E2E_HOST: ${{ secrets.DIFY_E2E_HOST }}
|
||||
DIFY_E2E_EMAIL: ${{ secrets.DIFY_E2E_EMAIL }}
|
||||
DIFY_E2E_PASSWORD: ${{ secrets.DIFY_E2E_PASSWORD }}
|
||||
DIFY_E2E_EDITION: ${{ inputs.edition || 'ee' }}
|
||||
DIFY_E2E_SSO_TOKEN: ${{ secrets.DIFY_E2E_SSO_TOKEN }}
|
||||
DIFY_E2E_TOKEN: ${{ needs.provision.outputs.token }}
|
||||
DIFY_E2E_WORKSPACE_ID: ${{ needs.provision.outputs.workspace_id }}
|
||||
DIFY_E2E_WORKSPACE_NAME: ${{ needs.provision.outputs.workspace_name }}
|
||||
DIFY_E2E_CHAT_APP_ID: ${{ needs.provision.outputs.chat_app_id }}
|
||||
DIFY_E2E_WORKFLOW_APP_ID: ${{ needs.provision.outputs.workflow_app_id }}
|
||||
DIFY_E2E_FILE_APP_ID: ${{ needs.provision.outputs.file_app_id }}
|
||||
DIFY_E2E_FILE_CHAT_APP_ID: ${{ needs.provision.outputs.file_chat_app_id }}
|
||||
DIFY_E2E_HITL_APP_ID: ${{ needs.provision.outputs.hitl_app_id }}
|
||||
DIFY_E2E_HITL_EXTERNAL_APP_ID: ${{ needs.provision.outputs.hitl_external_app_id }}
|
||||
DIFY_E2E_HITL_SINGLE_ACTION_APP_ID: ${{ needs.provision.outputs.hitl_single_action_app_id }}
|
||||
DIFY_E2E_HITL_MULTI_NODE_APP_ID: ${{ needs.provision.outputs.hitl_multi_node_app_id }}
|
||||
DIFY_E2E_INCLUDE: "test/e2e/suites/run/${{ matrix.file }}"
|
||||
DIFY_E2E_HITL_MULTI_NODE_APP_ID: ${{ needs.provision.outputs.hitl_multi_node_app_id }}
|
||||
DIFY_E2E_INCLUDE: 'test/e2e/suites/run/${{ matrix.file }}'
|
||||
run: |
|
||||
if [ "${{ inputs.test_scope }}" = "smoke" ]; then
|
||||
pnpm test:e2e -- -t "\[P0\]"
|
||||
@@ -274,17 +273,17 @@ jobs:
|
||||
|
||||
- name: Upload results on failure
|
||||
if: failure()
|
||||
uses: actions/upload-artifact@v7
|
||||
uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: e2e-run-${{ matrix.name }}-${{ github.run_id }}
|
||||
path: cli/test-results/
|
||||
retention-days: 3
|
||||
|
||||
# ════════════════════════════════════════════════════════════════════════════
|
||||
# 1-E. auth/login + status + whoami (parallel, read-only, safe)
|
||||
# ════════════════════════════════════════════════════════════════════════════
|
||||
# ════════════════════════════════════════════════════════════════════════════
|
||||
# 1-E. auth/login + status + whoami (parallel, read-only, safe)
|
||||
# ════════════════════════════════════════════════════════════════════════════
|
||||
suite-auth-safe:
|
||||
name: "Suite: auth (login / status / whoami)"
|
||||
name: 'Suite: auth (login / status / whoami)'
|
||||
if: ${{ inputs.suite_auth != 'false' }}
|
||||
needs: provision
|
||||
runs-on: ubuntu-latest
|
||||
@@ -295,7 +294,7 @@ jobs:
|
||||
shell: bash
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v4
|
||||
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v4
|
||||
with:
|
||||
ref: ${{ inputs.cli_ref || github.ref }}
|
||||
persist-credentials: false
|
||||
@@ -303,22 +302,22 @@ jobs:
|
||||
- uses: ./.github/actions/setup-web
|
||||
- uses: oven-sh/setup-bun@v2
|
||||
with: { bun-version: latest }
|
||||
- uses: pnpm/action-setup@0ebf47130e4866e96fce0953f49152a61190b271 # v6.0.9
|
||||
- uses: pnpm/action-setup@b906affcce14559ad1aafd4ab0e942779e9f58b1 # v4
|
||||
with: { package_json_field: packageManager, run_install: false }
|
||||
- run: pnpm install --frozen-lockfile
|
||||
- run: pnpm tree:gen
|
||||
|
||||
- name: Run auth/login + status + whoami
|
||||
env:
|
||||
DIFY_E2E_HOST: ${{ secrets.DIFY_E2E_HOST }}
|
||||
DIFY_E2E_EMAIL: ${{ secrets.DIFY_E2E_EMAIL }}
|
||||
DIFY_E2E_PASSWORD: ${{ secrets.DIFY_E2E_PASSWORD }}
|
||||
DIFY_E2E_EDITION: ${{ inputs.edition || 'ee' }}
|
||||
DIFY_E2E_TOKEN: ${{ needs.provision.outputs.token }}
|
||||
DIFY_E2E_WORKSPACE_ID: ${{ needs.provision.outputs.workspace_id }}
|
||||
DIFY_E2E_HOST: ${{ secrets.DIFY_E2E_HOST }}
|
||||
DIFY_E2E_EMAIL: ${{ secrets.DIFY_E2E_EMAIL }}
|
||||
DIFY_E2E_PASSWORD: ${{ secrets.DIFY_E2E_PASSWORD }}
|
||||
DIFY_E2E_EDITION: ${{ inputs.edition || 'ee' }}
|
||||
DIFY_E2E_TOKEN: ${{ needs.provision.outputs.token }}
|
||||
DIFY_E2E_WORKSPACE_ID: ${{ needs.provision.outputs.workspace_id }}
|
||||
DIFY_E2E_WORKSPACE_NAME: ${{ needs.provision.outputs.workspace_name }}
|
||||
DIFY_E2E_WS2_ID: ${{ needs.provision.outputs.ws2_id }}
|
||||
DIFY_E2E_INCLUDE: "test/e2e/suites/auth/login.e2e.ts,test/e2e/suites/auth/status.e2e.ts,test/e2e/suites/auth/whoami.e2e.ts"
|
||||
DIFY_E2E_WS2_ID: ${{ needs.provision.outputs.ws2_id }}
|
||||
DIFY_E2E_INCLUDE: 'test/e2e/suites/auth/login.e2e.ts,test/e2e/suites/auth/status.e2e.ts,test/e2e/suites/auth/whoami.e2e.ts'
|
||||
run: |
|
||||
if [ "${{ inputs.test_scope }}" = "smoke" ]; then
|
||||
pnpm test:e2e -- -t "\[P0\]"
|
||||
@@ -326,13 +325,13 @@ jobs:
|
||||
pnpm test:e2e
|
||||
fi
|
||||
|
||||
# ════════════════════════════════════════════════════════════════════════════
|
||||
# 2. DESTRUCTIVE — auth/use + devices + logout + agent (serial, runs LAST)
|
||||
# Must wait for ALL parallel suites to finish to avoid token revocation
|
||||
# invalidating other in-flight requests.
|
||||
# ════════════════════════════════════════════════════════════════════════════
|
||||
# ════════════════════════════════════════════════════════════════════════════
|
||||
# 2. DESTRUCTIVE — auth/use + devices + logout + agent (serial, runs LAST)
|
||||
# Must wait for ALL parallel suites to finish to avoid token revocation
|
||||
# invalidating other in-flight requests.
|
||||
# ════════════════════════════════════════════════════════════════════════════
|
||||
suite-last:
|
||||
name: "Suite: auth-use + devices + logout + agent (last, serial)"
|
||||
name: 'Suite: auth-use + devices + logout + agent (last, serial)'
|
||||
# Runs when auth is selected; also runs after all parallel jobs finish
|
||||
if: ${{ inputs.suite_auth != 'false' || inputs.suite_agent != 'false' }}
|
||||
needs:
|
||||
@@ -351,7 +350,7 @@ jobs:
|
||||
shell: bash
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v4
|
||||
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v4
|
||||
with:
|
||||
ref: ${{ inputs.cli_ref || github.ref }}
|
||||
persist-credentials: false
|
||||
@@ -359,27 +358,27 @@ jobs:
|
||||
- uses: ./.github/actions/setup-web
|
||||
- uses: oven-sh/setup-bun@v2
|
||||
with: { bun-version: latest }
|
||||
- uses: pnpm/action-setup@0ebf47130e4866e96fce0953f49152a61190b271 # v6.0.9
|
||||
- uses: pnpm/action-setup@b906affcce14559ad1aafd4ab0e942779e9f58b1 # v4
|
||||
with: { package_json_field: packageManager, run_install: false }
|
||||
- run: pnpm install --frozen-lockfile
|
||||
- run: pnpm tree:gen
|
||||
|
||||
- name: Run use / devices / logout / agent (serial)
|
||||
env:
|
||||
DIFY_E2E_HOST: ${{ secrets.DIFY_E2E_HOST }}
|
||||
DIFY_E2E_EMAIL: ${{ secrets.DIFY_E2E_EMAIL }}
|
||||
DIFY_E2E_PASSWORD: ${{ secrets.DIFY_E2E_PASSWORD }}
|
||||
DIFY_E2E_EDITION: ${{ inputs.edition || 'ee' }}
|
||||
DIFY_E2E_TOKEN: ${{ needs.provision.outputs.token }}
|
||||
DIFY_E2E_WORKSPACE_ID: ${{ needs.provision.outputs.workspace_id }}
|
||||
DIFY_E2E_WORKSPACE_NAME: ${{ needs.provision.outputs.workspace_name }}
|
||||
DIFY_E2E_WS2_ID: ${{ needs.provision.outputs.ws2_id }}
|
||||
DIFY_E2E_CHAT_APP_ID: ${{ needs.provision.outputs.chat_app_id }}
|
||||
DIFY_E2E_WORKFLOW_APP_ID: ${{ needs.provision.outputs.workflow_app_id }}
|
||||
DIFY_E2E_HITL_APP_ID: ${{ needs.provision.outputs.hitl_app_id }}
|
||||
DIFY_E2E_HITL_EXTERNAL_APP_ID: ${{ needs.provision.outputs.hitl_external_app_id }}
|
||||
DIFY_E2E_HOST: ${{ secrets.DIFY_E2E_HOST }}
|
||||
DIFY_E2E_EMAIL: ${{ secrets.DIFY_E2E_EMAIL }}
|
||||
DIFY_E2E_PASSWORD: ${{ secrets.DIFY_E2E_PASSWORD }}
|
||||
DIFY_E2E_EDITION: ${{ inputs.edition || 'ee' }}
|
||||
DIFY_E2E_TOKEN: ${{ needs.provision.outputs.token }}
|
||||
DIFY_E2E_WORKSPACE_ID: ${{ needs.provision.outputs.workspace_id }}
|
||||
DIFY_E2E_WORKSPACE_NAME: ${{ needs.provision.outputs.workspace_name }}
|
||||
DIFY_E2E_WS2_ID: ${{ needs.provision.outputs.ws2_id }}
|
||||
DIFY_E2E_CHAT_APP_ID: ${{ needs.provision.outputs.chat_app_id }}
|
||||
DIFY_E2E_WORKFLOW_APP_ID: ${{ needs.provision.outputs.workflow_app_id }}
|
||||
DIFY_E2E_HITL_APP_ID: ${{ needs.provision.outputs.hitl_app_id }}
|
||||
DIFY_E2E_HITL_EXTERNAL_APP_ID: ${{ needs.provision.outputs.hitl_external_app_id }}
|
||||
DIFY_E2E_HITL_SINGLE_ACTION_APP_ID: ${{ needs.provision.outputs.hitl_single_action_app_id }}
|
||||
DIFY_E2E_HITL_MULTI_NODE_APP_ID: ${{ needs.provision.outputs.hitl_multi_node_app_id }}
|
||||
DIFY_E2E_HITL_MULTI_NODE_APP_ID: ${{ needs.provision.outputs.hitl_multi_node_app_id }}
|
||||
run: |
|
||||
# Collect files in safe order: use → devices → logout (revokes last) → agent
|
||||
FILES=()
|
||||
@@ -408,7 +407,7 @@ jobs:
|
||||
|
||||
- name: Upload results on failure
|
||||
if: failure()
|
||||
uses: actions/upload-artifact@v7
|
||||
uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: e2e-last-${{ github.run_id }}
|
||||
path: cli/test-results/
|
||||
|
||||
@@ -1,74 +0,0 @@
|
||||
name: CLI Edge Publish
|
||||
|
||||
on:
|
||||
push:
|
||||
branches: [main]
|
||||
paths:
|
||||
- 'cli/**'
|
||||
- 'packages/contracts/generated/api/openapi/**'
|
||||
workflow_dispatch:
|
||||
|
||||
concurrency:
|
||||
group: difyctl-edge-publish
|
||||
cancel-in-progress: false
|
||||
|
||||
jobs:
|
||||
publish:
|
||||
name: build + publish edge to R2
|
||||
runs-on: ${{ github.repository == 'langgenius/dify' && 'depot-ubuntu-24.04' || 'ubuntu-24.04' }}
|
||||
if: vars.DIFYCTL_R2_BUCKET != ''
|
||||
defaults:
|
||||
run:
|
||||
shell: bash
|
||||
working-directory: ./cli
|
||||
steps:
|
||||
- name: Checkout
|
||||
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
|
||||
with:
|
||||
persist-credentials: false
|
||||
fetch-depth: 0
|
||||
|
||||
- name: Enable cross-arch native prebuilds
|
||||
working-directory: ./
|
||||
run: cat cli/scripts/cross-arch.pnpm.yaml >> pnpm-workspace.yaml
|
||||
|
||||
- name: Setup web environment
|
||||
uses: ./.github/actions/setup-web
|
||||
|
||||
- name: Setup Bun
|
||||
uses: oven-sh/setup-bun@4bc047ad259df6fc24a6c9b0f9a0cb08cf17fbe5 # v2.0.2
|
||||
with:
|
||||
bun-version-file: cli/.bun-version
|
||||
|
||||
- name: Compute edge version
|
||||
id: ver
|
||||
run: echo "version=$(node scripts/release-naming.mjs edge-version "$(git rev-parse --short HEAD)")" >> "$GITHUB_OUTPUT"
|
||||
|
||||
- name: Compile standalone binaries (all targets, all-or-nothing)
|
||||
run: |
|
||||
CLI_VERSION="${{ steps.ver.outputs.version }}" \
|
||||
DIFYCTL_CHANNEL=edge \
|
||||
DIFYCTL_COMMIT="$(git rev-parse HEAD)" \
|
||||
DIFYCTL_BUILD_DATE="$(git log -1 --format=%cI HEAD)" \
|
||||
pnpm build:bin
|
||||
|
||||
- name: Generate sha256 checksums
|
||||
run: CLI_VERSION="${{ steps.ver.outputs.version }}" scripts/release-write-checksums.sh
|
||||
|
||||
- name: Smoke the runner-arch binary
|
||||
run: ./dist/bin/difyctl-v${{ steps.ver.outputs.version }}-linux-x64 version
|
||||
|
||||
- name: Publish to R2
|
||||
env:
|
||||
AWS_ACCESS_KEY_ID: ${{ secrets.DIFYCTL_R2_ACCESS_KEY_ID }}
|
||||
AWS_SECRET_ACCESS_KEY: ${{ secrets.DIFYCTL_R2_SECRET_ACCESS_KEY }}
|
||||
AWS_DEFAULT_REGION: auto
|
||||
AWS_REQUEST_CHECKSUM_CALCULATION: WHEN_REQUIRED
|
||||
AWS_RESPONSE_CHECKSUM_VALIDATION: WHEN_REQUIRED
|
||||
DIFYCTL_R2_S3_ENDPOINT: ${{ vars.DIFYCTL_R2_S3_ENDPOINT }}
|
||||
DIFYCTL_R2_BUCKET: ${{ vars.DIFYCTL_R2_BUCKET }}
|
||||
DIFYCTL_R2_PUBLIC_BASE: ${{ vars.DIFYCTL_R2_PUBLIC_BASE }}
|
||||
DIFYCTL_COMMIT: ${{ github.sha }}
|
||||
run: |
|
||||
DIFYCTL_BUILD_DATE="$(git log -1 --format=%cI HEAD)" \
|
||||
scripts/release-r2-publish.sh edge "${{ steps.ver.outputs.version }}"
|
||||
@@ -35,7 +35,7 @@ jobs:
|
||||
dify_tag: ${{ steps.resolve.outputs.dify_tag }}
|
||||
steps:
|
||||
- name: Checkout
|
||||
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
|
||||
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
||||
with:
|
||||
persist-credentials: false
|
||||
|
||||
@@ -98,7 +98,7 @@ jobs:
|
||||
DIFY_TAG: ${{ needs.validate.outputs.dify_tag }}
|
||||
steps:
|
||||
- name: Checkout
|
||||
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
|
||||
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
||||
with:
|
||||
persist-credentials: false
|
||||
fetch-depth: 1
|
||||
|
||||
@@ -4,11 +4,11 @@ on:
|
||||
workflow_dispatch:
|
||||
inputs:
|
||||
dify_version:
|
||||
description: "Dify image tag to test against (e.g. 1.7.0)"
|
||||
description: 'Dify image tag to test against (e.g. 1.7.0)'
|
||||
type: string
|
||||
required: true
|
||||
cli_ref:
|
||||
description: "Git ref to build the cli from (default: current branch)"
|
||||
description: 'Git ref to build the cli from (default: current branch)'
|
||||
type: string
|
||||
required: false
|
||||
|
||||
@@ -24,7 +24,7 @@ jobs:
|
||||
shell: bash
|
||||
steps:
|
||||
- name: Checkout cli ref
|
||||
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
|
||||
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
||||
with:
|
||||
ref: ${{ inputs.cli_ref || github.ref }}
|
||||
persist-credentials: false
|
||||
|
||||
@@ -30,7 +30,7 @@ jobs:
|
||||
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
|
||||
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
||||
with:
|
||||
persist-credentials: false
|
||||
|
||||
@@ -51,7 +51,7 @@ jobs:
|
||||
|
||||
- name: Report coverage
|
||||
if: ${{ env.CODECOV_TOKEN != '' && matrix.os == 'depot-ubuntu-24.04' }}
|
||||
uses: codecov/codecov-action@fb8b3582c8e4def4969c97caa2f19720cb33a72f # v7.0.0
|
||||
uses: codecov/codecov-action@57e3a136b779b570ffcdbf80b3bdc90e7fab3de2 # v6.0.0
|
||||
with:
|
||||
directory: cli/coverage
|
||||
flags: cli
|
||||
|
||||
@@ -13,16 +13,16 @@ jobs:
|
||||
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
|
||||
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
||||
with:
|
||||
fetch-depth: 0
|
||||
persist-credentials: false
|
||||
|
||||
- name: Setup UV and Python
|
||||
uses: astral-sh/setup-uv@fac544c07dec837d0ccb6301d7b5580bf5edae39 # v8.2.0
|
||||
uses: astral-sh/setup-uv@08807647e7069bb48b6ef5acd8ec9567f424441b # v8.1.0
|
||||
with:
|
||||
enable-cache: true
|
||||
python-version: "3.12"
|
||||
python-version: '3.12'
|
||||
cache-dependency-glob: api/uv.lock
|
||||
|
||||
- name: Install dependencies
|
||||
@@ -40,7 +40,7 @@ jobs:
|
||||
cp envs/middleware.env.example middleware.env
|
||||
|
||||
- name: Set up Middlewares
|
||||
uses: hoverkraft-tech/compose-action@11beaa1c2dae4e8ed7b1665aa074723b6cecb0e4 # v3.0.0
|
||||
uses: hoverkraft-tech/compose-action@d2bee4f07e8ca410d6b196d00f90c12e7d48c33a # v2.6.0
|
||||
with:
|
||||
compose-file: |
|
||||
docker/docker-compose.middleware.yaml
|
||||
@@ -63,16 +63,16 @@ jobs:
|
||||
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
|
||||
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
||||
with:
|
||||
fetch-depth: 0
|
||||
persist-credentials: false
|
||||
|
||||
- name: Setup UV and Python
|
||||
uses: astral-sh/setup-uv@fac544c07dec837d0ccb6301d7b5580bf5edae39 # v8.2.0
|
||||
uses: astral-sh/setup-uv@08807647e7069bb48b6ef5acd8ec9567f424441b # v8.1.0
|
||||
with:
|
||||
enable-cache: true
|
||||
python-version: "3.12"
|
||||
python-version: '3.12'
|
||||
cache-dependency-glob: api/uv.lock
|
||||
|
||||
- name: Install dependencies
|
||||
@@ -94,7 +94,7 @@ jobs:
|
||||
sed -i 's/DB_USERNAME=postgres/DB_USERNAME=mysql/' middleware.env
|
||||
|
||||
- name: Set up Middlewares
|
||||
uses: hoverkraft-tech/compose-action@11beaa1c2dae4e8ed7b1665aa074723b6cecb0e4 # v3.0.0
|
||||
uses: hoverkraft-tech/compose-action@d2bee4f07e8ca410d6b196d00f90c12e7d48c33a # v2.6.0
|
||||
with:
|
||||
compose-file: |
|
||||
docker/docker-compose.middleware.yaml
|
||||
|
||||
@@ -2,9 +2,9 @@ name: Deploy Dev
|
||||
|
||||
on:
|
||||
workflow_run:
|
||||
workflows: ["Build and Push API & Web"]
|
||||
workflows: ['Build and Push API & Web']
|
||||
branches:
|
||||
- "deploy/dev"
|
||||
- 'deploy/dev'
|
||||
types:
|
||||
- completed
|
||||
|
||||
|
||||
@@ -5,9 +5,9 @@ permissions:
|
||||
|
||||
on:
|
||||
workflow_run:
|
||||
workflows: ["Build and Push API & Web"]
|
||||
workflows: ['Build and Push API & Web']
|
||||
branches:
|
||||
- "deploy/enterprise"
|
||||
- 'deploy/enterprise'
|
||||
types:
|
||||
- completed
|
||||
|
||||
|
||||
@@ -1,13 +1,10 @@
|
||||
name: Deploy Agent
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
name: Deploy HITL
|
||||
|
||||
on:
|
||||
workflow_run:
|
||||
workflows: ["Build and Push API & Web"]
|
||||
workflows: ['Build and Push API & Web']
|
||||
branches:
|
||||
- "deploy/agent"
|
||||
- 'build/feat/hitl'
|
||||
types:
|
||||
- completed
|
||||
|
||||
@@ -16,13 +13,13 @@ jobs:
|
||||
runs-on: depot-ubuntu-24.04
|
||||
if: |
|
||||
github.event.workflow_run.conclusion == 'success' &&
|
||||
github.event.workflow_run.head_branch == 'deploy/agent'
|
||||
github.event.workflow_run.head_branch == 'build/feat/hitl'
|
||||
steps:
|
||||
- name: Deploy to server
|
||||
uses: appleboy/ssh-action@0ff4204d59e8e51228ff73bce53f80d53301dee2 # v1.2.5
|
||||
with:
|
||||
host: ${{ secrets.AGENT_SSH_HOST }}
|
||||
host: ${{ secrets.HITL_SSH_HOST }}
|
||||
username: ${{ secrets.SSH_USER }}
|
||||
key: ${{ secrets.SSH_PRIVATE_KEY }}
|
||||
script: |
|
||||
${{ vars.SSH_SCRIPT_AGENT || secrets.SSH_SCRIPT_AGENT }}
|
||||
${{ vars.SSH_SCRIPT || secrets.SSH_SCRIPT }}
|
||||
@@ -5,9 +5,9 @@ permissions:
|
||||
|
||||
on:
|
||||
workflow_run:
|
||||
workflows: ["Build and Push API & Web"]
|
||||
workflows: ['Build and Push API & Web']
|
||||
branches:
|
||||
- "deploy/saas"
|
||||
- 'deploy/saas'
|
||||
types:
|
||||
- completed
|
||||
|
||||
|
||||
@@ -3,7 +3,7 @@ name: Build docker image
|
||||
on:
|
||||
pull_request:
|
||||
branches:
|
||||
- "main"
|
||||
- 'main'
|
||||
paths:
|
||||
- api/Dockerfile
|
||||
- api/Dockerfile.dockerignore
|
||||
@@ -28,32 +28,32 @@ jobs:
|
||||
strategy:
|
||||
matrix:
|
||||
include:
|
||||
- service_name: "api-amd64"
|
||||
- service_name: 'api-amd64'
|
||||
platform: linux/amd64
|
||||
runs_on: depot-ubuntu-24.04-4
|
||||
context: "{{defaultContext}}"
|
||||
file: "api/Dockerfile"
|
||||
- service_name: "api-arm64"
|
||||
context: '{{defaultContext}}'
|
||||
file: 'api/Dockerfile'
|
||||
- service_name: 'api-arm64'
|
||||
platform: linux/arm64
|
||||
runs_on: depot-ubuntu-24.04-4
|
||||
context: "{{defaultContext}}"
|
||||
file: "api/Dockerfile"
|
||||
- service_name: "web-amd64"
|
||||
context: '{{defaultContext}}'
|
||||
file: 'api/Dockerfile'
|
||||
- service_name: 'web-amd64'
|
||||
platform: linux/amd64
|
||||
runs_on: depot-ubuntu-24.04-4
|
||||
context: "{{defaultContext}}"
|
||||
file: "web/Dockerfile"
|
||||
- service_name: "web-arm64"
|
||||
context: '{{defaultContext}}'
|
||||
file: 'web/Dockerfile'
|
||||
- service_name: 'web-arm64'
|
||||
platform: linux/arm64
|
||||
runs_on: depot-ubuntu-24.04-4
|
||||
context: "{{defaultContext}}"
|
||||
file: "web/Dockerfile"
|
||||
context: '{{defaultContext}}'
|
||||
file: 'web/Dockerfile'
|
||||
steps:
|
||||
- name: Set up Depot CLI
|
||||
uses: depot/setup-action@15c09a5f77a0840ad4bce955686522a257853461 # v1.7.1
|
||||
|
||||
- name: Build Docker Image
|
||||
uses: depot/build-push-action@98e78adca7817480b8185f474a400b451d74e287 # v1.18.0
|
||||
uses: depot/build-push-action@5f3b3c2e5a00f0093de47f657aeaefcedff27d18 # v1.17.0
|
||||
with:
|
||||
project: ${{ vars.DEPOT_PROJECT_ID }}
|
||||
push: false
|
||||
@@ -69,18 +69,18 @@ jobs:
|
||||
strategy:
|
||||
matrix:
|
||||
include:
|
||||
- service_name: "api-amd64"
|
||||
context: "{{defaultContext}}"
|
||||
file: "api/Dockerfile"
|
||||
- service_name: "web-amd64"
|
||||
context: "{{defaultContext}}"
|
||||
file: "web/Dockerfile"
|
||||
- service_name: 'api-amd64'
|
||||
context: '{{defaultContext}}'
|
||||
file: 'api/Dockerfile'
|
||||
- service_name: 'web-amd64'
|
||||
context: '{{defaultContext}}'
|
||||
file: 'web/Dockerfile'
|
||||
steps:
|
||||
- name: Set up Docker Buildx
|
||||
uses: docker/setup-buildx-action@d7f5e7f509e45cec5c76c4d5afdd7de93d0b3df5 # v4.1.0
|
||||
uses: docker/setup-buildx-action@4d04d5d9486b7bd6fa91e7baf45bbb4f8b9deedd # v4.0.0
|
||||
|
||||
- name: Build Docker Image
|
||||
uses: docker/build-push-action@f9f3042f7e2789586610d6e8b85c8f03e5195baf # v7.2.0
|
||||
uses: docker/build-push-action@bcafcacb16a39f128d818304e6c9c0c18556b85f # v7.1.0
|
||||
with:
|
||||
push: false
|
||||
context: ${{ matrix.context }}
|
||||
|
||||
@@ -24,7 +24,7 @@ jobs:
|
||||
name: Require cherry-pick provenance
|
||||
runs-on: depot-ubuntu-24.04
|
||||
steps:
|
||||
- uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
|
||||
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
||||
with:
|
||||
fetch-depth: 0
|
||||
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
name: "Pull Request Labeler"
|
||||
name: 'Pull Request Labeler'
|
||||
on:
|
||||
pull_request_target:
|
||||
|
||||
|
||||
@@ -2,10 +2,12 @@ name: Main CI Pipeline
|
||||
|
||||
on:
|
||||
pull_request:
|
||||
branches: ["main"]
|
||||
branches: ['main']
|
||||
merge_group:
|
||||
branches: ["main"]
|
||||
branches: ['main']
|
||||
types: [checks_requested]
|
||||
push:
|
||||
branches: ['main']
|
||||
|
||||
permissions:
|
||||
actions: write
|
||||
@@ -46,7 +48,7 @@ jobs:
|
||||
vdb-changed: ${{ steps.changes.outputs.vdb }}
|
||||
migration-changed: ${{ steps.changes.outputs.migration }}
|
||||
steps:
|
||||
- uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
|
||||
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
||||
- uses: dorny/paths-filter@fbd0ab8f3e69293af611ebaee6363fc25e6d187d # v4.0.1
|
||||
id: changes
|
||||
with:
|
||||
@@ -322,7 +324,6 @@ jobs:
|
||||
- check-changes
|
||||
if: needs.pre_job.outputs.should_skip != 'true' && needs.check-changes.outputs.e2e-changed == 'true'
|
||||
uses: ./.github/workflows/web-e2e.yml
|
||||
secrets: inherit
|
||||
|
||||
web-e2e-skip:
|
||||
name: Skip Web Full-Stack E2E
|
||||
|
||||
@@ -1,54 +0,0 @@
|
||||
name: Post-Merge Checks
|
||||
|
||||
on:
|
||||
push:
|
||||
branches: ["main"]
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
concurrency:
|
||||
group: post-merge-${{ github.sha }}
|
||||
cancel-in-progress: false
|
||||
|
||||
jobs:
|
||||
check-changes:
|
||||
name: Check Changed Files
|
||||
runs-on: depot-ubuntu-24.04
|
||||
outputs:
|
||||
external-e2e-changed: ${{ steps.changes.outputs.external_e2e }}
|
||||
steps:
|
||||
- uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
|
||||
- uses: dorny/paths-filter@fbd0ab8f3e69293af611ebaee6363fc25e6d187d # v4.0.1
|
||||
id: changes
|
||||
with:
|
||||
filters: |
|
||||
external_e2e:
|
||||
- 'e2e/features/agent-v2/**'
|
||||
- 'e2e/scripts/**'
|
||||
- 'e2e/support/**'
|
||||
- '.github/workflows/post-merge.yml'
|
||||
- '.github/workflows/web-e2e.yml'
|
||||
- '.github/actions/setup-web/**'
|
||||
- 'dify-agent/**'
|
||||
- 'api/clients/agent_backend/**'
|
||||
- 'api/core/app/apps/agent_app/**'
|
||||
- 'api/core/workflow/nodes/agent_v2/**'
|
||||
- 'api/controllers/console/agent/**'
|
||||
- 'api/services/agent/**'
|
||||
- 'api/core/plugin/**'
|
||||
- 'api/services/plugin/**'
|
||||
- 'api/core/tools/**'
|
||||
- 'api/services/tools/**'
|
||||
- 'web/features/agent-v2/**'
|
||||
- 'web/app/(commonLayout)/roster/**'
|
||||
- 'web/app/components/workflow/nodes/agent-v2/**'
|
||||
|
||||
external-e2e:
|
||||
name: External Runtime E2E
|
||||
needs: check-changes
|
||||
if: needs.check-changes.outputs.external-e2e-changed == 'true'
|
||||
uses: ./.github/workflows/web-e2e.yml
|
||||
with:
|
||||
run-external-runtime: true
|
||||
secrets: inherit
|
||||
@@ -17,12 +17,12 @@ jobs:
|
||||
pull-requests: write
|
||||
steps:
|
||||
- name: Checkout PR branch
|
||||
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
|
||||
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
||||
with:
|
||||
fetch-depth: 0
|
||||
|
||||
- name: Setup Python & UV
|
||||
uses: astral-sh/setup-uv@fac544c07dec837d0ccb6301d7b5580bf5edae39 # v8.2.0
|
||||
uses: astral-sh/setup-uv@08807647e7069bb48b6ef5acd8ec9567f424441b # v8.1.0
|
||||
with:
|
||||
enable-cache: true
|
||||
|
||||
|
||||
@@ -21,10 +21,10 @@ jobs:
|
||||
if: ${{ github.event.workflow_run.conclusion == 'success' && github.event.workflow_run.pull_requests[0].head.repo.full_name != github.repository }}
|
||||
steps:
|
||||
- name: Checkout default branch (trusted code)
|
||||
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
|
||||
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
||||
|
||||
- name: Setup Python & UV
|
||||
uses: astral-sh/setup-uv@fac544c07dec837d0ccb6301d7b5580bf5edae39 # v8.2.0
|
||||
uses: astral-sh/setup-uv@08807647e7069bb48b6ef5acd8ec9567f424441b # v8.1.0
|
||||
with:
|
||||
enable-cache: true
|
||||
|
||||
|
||||
@@ -17,12 +17,12 @@ jobs:
|
||||
pull-requests: write
|
||||
steps:
|
||||
- name: Checkout PR branch
|
||||
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
|
||||
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
||||
with:
|
||||
fetch-depth: 0
|
||||
|
||||
- name: Setup Python & UV
|
||||
uses: astral-sh/setup-uv@fac544c07dec837d0ccb6301d7b5580bf5edae39 # v8.2.0
|
||||
uses: astral-sh/setup-uv@08807647e7069bb48b6ef5acd8ec9567f424441b # v8.1.0
|
||||
with:
|
||||
enable-cache: true
|
||||
|
||||
|
||||
@@ -8,7 +8,7 @@ on:
|
||||
- reopened
|
||||
- synchronize
|
||||
merge_group:
|
||||
branches: ["main"]
|
||||
branches: ['main']
|
||||
types: [checks_requested]
|
||||
|
||||
jobs:
|
||||
|
||||
@@ -11,20 +11,19 @@ on:
|
||||
|
||||
jobs:
|
||||
stale:
|
||||
|
||||
runs-on: depot-ubuntu-24.04
|
||||
permissions:
|
||||
issues: write
|
||||
pull-requests: write
|
||||
|
||||
steps:
|
||||
- uses: actions/stale@eb5cf3af3ac0a1aa4c9c45633dd1ae542a27a899 # v10.3.0
|
||||
- uses: actions/stale@b5d41d4e1d5dceea10e7104786b73624c18a190f # v10.2.0
|
||||
with:
|
||||
days-before-issue-stale: 15
|
||||
days-before-issue-close: 3
|
||||
repo-token: ${{ secrets.GITHUB_TOKEN }}
|
||||
stale-issue-message: "Closed due to inactivity. If you have any questions, you can reopen it."
|
||||
stale-pr-message: "Closed due to inactivity. If you have any questions, you can reopen it."
|
||||
stale-issue-message: 'Closed due to inactivity. If you have any questions, you can reopen it.'
|
||||
stale-pr-message: 'Closed due to inactivity. If you have any questions, you can reopen it.'
|
||||
stale-issue-label: 'no-issue-activity'
|
||||
stale-pr-label: 'no-pr-activity'
|
||||
any-of-labels: '🌚 invalid,🙋♂️ question,wont-fix,no-issue-activity,no-pr-activity,💪 enhancement,🤔 cant-reproduce,🙏 help wanted'
|
||||
|
||||
+15
-31
@@ -19,10 +19,9 @@ jobs:
|
||||
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
|
||||
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
||||
with:
|
||||
persist-credentials: false
|
||||
fetch-depth: 0
|
||||
|
||||
- name: Check changed files
|
||||
id: changed-files
|
||||
@@ -30,16 +29,14 @@ jobs:
|
||||
with:
|
||||
files: |
|
||||
api/**
|
||||
scripts/check_no_new_getattr.py
|
||||
scripts/ast_grep_rules/no_new_getattr.yml
|
||||
.github/workflows/style.yml
|
||||
|
||||
- name: Setup UV and Python
|
||||
if: steps.changed-files.outputs.any_changed == 'true'
|
||||
uses: astral-sh/setup-uv@fac544c07dec837d0ccb6301d7b5580bf5edae39 # v8.2.0
|
||||
uses: astral-sh/setup-uv@08807647e7069bb48b6ef5acd8ec9567f424441b # v8.1.0
|
||||
with:
|
||||
enable-cache: false
|
||||
python-version: "3.12"
|
||||
python-version: '3.12'
|
||||
cache-dependency-glob: api/uv.lock
|
||||
|
||||
- name: Install dependencies
|
||||
@@ -54,22 +51,8 @@ jobs:
|
||||
if: steps.changed-files.outputs.any_changed == 'true'
|
||||
run: uv run --project api --dev python api/dev/lint_response_contracts.py --fail-on-mismatch
|
||||
|
||||
- name: Fetch merge target ref for getattr guard
|
||||
if: steps.changed-files.outputs.any_changed == 'true'
|
||||
run: git fetch --no-tags --depth=1 origin +refs/heads/main:refs/remotes/origin/main
|
||||
|
||||
- name: Bind merge target branch for getattr guard
|
||||
if: steps.changed-files.outputs.any_changed == 'true'
|
||||
run: git show-ref --verify --quiet refs/heads/main || git branch main origin/main
|
||||
|
||||
- name: Run No New Getattr Guard
|
||||
if: steps.changed-files.outputs.any_changed == 'true'
|
||||
run: uv run --project api python scripts/check_no_new_getattr.py --mode ci --merge-target main
|
||||
|
||||
- name: Run Type Checks
|
||||
if: steps.changed-files.outputs.any_changed == 'true'
|
||||
env:
|
||||
PYREFLY_OUTPUT_FORMAT: github
|
||||
run: make type-check-core
|
||||
|
||||
- name: Dotenv check
|
||||
@@ -88,7 +71,7 @@ jobs:
|
||||
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
|
||||
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
||||
with:
|
||||
persist-credentials: false
|
||||
|
||||
@@ -122,14 +105,6 @@ jobs:
|
||||
if: steps.changed-files.outputs.any_changed == 'true'
|
||||
run: vp run knip
|
||||
|
||||
- name: Web dead code check production
|
||||
if: steps.changed-files.outputs.any_changed == 'true'
|
||||
run: vp run knip:production
|
||||
|
||||
- name: Web production unused declarations check
|
||||
if: steps.changed-files.outputs.any_changed == 'true'
|
||||
run: vp run knip:production-unused-check
|
||||
|
||||
ts-common-style:
|
||||
name: TS Common
|
||||
runs-on: depot-ubuntu-24.04
|
||||
@@ -139,7 +114,7 @@ jobs:
|
||||
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
|
||||
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
||||
with:
|
||||
persist-credentials: false
|
||||
|
||||
@@ -157,7 +132,10 @@ jobs:
|
||||
pnpm-lock.yaml
|
||||
pnpm-workspace.yaml
|
||||
.nvmrc
|
||||
vite.config.ts
|
||||
eslint.config.mjs
|
||||
.vscode/**
|
||||
.github/workflows/autofix.yml
|
||||
.github/workflows/style.yml
|
||||
.github/actions/setup-web/**
|
||||
|
||||
@@ -165,6 +143,12 @@ jobs:
|
||||
if: steps.changed-files.outputs.any_changed == 'true'
|
||||
uses: ./.github/actions/setup-web
|
||||
|
||||
- name: Format check
|
||||
if: steps.changed-files.outputs.any_changed == 'true'
|
||||
env:
|
||||
CHANGED_FILES: ${{ steps.changed-files.outputs.all_changed_files }}
|
||||
run: vp fmt --check --no-error-on-unmatched-pattern $CHANGED_FILES
|
||||
|
||||
- name: Restore ESLint cache
|
||||
if: steps.changed-files.outputs.any_changed == 'true'
|
||||
id: eslint-cache-restore
|
||||
@@ -196,7 +180,7 @@ jobs:
|
||||
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
|
||||
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
||||
with:
|
||||
fetch-depth: 0
|
||||
persist-credentials: false
|
||||
|
||||
@@ -24,7 +24,7 @@ jobs:
|
||||
working-directory: sdks/nodejs-client
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
|
||||
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
||||
with:
|
||||
persist-credentials: false
|
||||
|
||||
|
||||
@@ -40,7 +40,7 @@ jobs:
|
||||
|
||||
steps:
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
|
||||
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
||||
with:
|
||||
fetch-depth: 0
|
||||
token: ${{ secrets.GITHUB_TOKEN }}
|
||||
@@ -158,7 +158,7 @@ jobs:
|
||||
|
||||
- name: Run Claude Code for Translation Sync
|
||||
if: steps.context.outputs.CHANGED_FILES != ''
|
||||
uses: anthropics/claude-code-action@806af32823ef69c8ef357086c573a902af641307 # v1.0.151
|
||||
uses: anthropics/claude-code-action@1dc994ee7a008f0ecc866d9ac23ef036b7229f84 # v1.0.127
|
||||
with:
|
||||
anthropic_api_key: ${{ secrets.ANTHROPIC_API_KEY }}
|
||||
github_token: ${{ secrets.GITHUB_TOKEN }}
|
||||
|
||||
@@ -21,7 +21,7 @@ jobs:
|
||||
|
||||
steps:
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
|
||||
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
||||
with:
|
||||
fetch-depth: 0
|
||||
|
||||
|
||||
@@ -20,11 +20,11 @@ jobs:
|
||||
strategy:
|
||||
matrix:
|
||||
python-version:
|
||||
- "3.12"
|
||||
- '3.12'
|
||||
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
|
||||
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
||||
with:
|
||||
persist-credentials: false
|
||||
|
||||
@@ -36,7 +36,7 @@ jobs:
|
||||
remove_tool_cache: true
|
||||
|
||||
- name: Setup UV and Python
|
||||
uses: astral-sh/setup-uv@fac544c07dec837d0ccb6301d7b5580bf5edae39 # v8.2.0
|
||||
uses: astral-sh/setup-uv@08807647e7069bb48b6ef5acd8ec9567f424441b # v8.1.0
|
||||
with:
|
||||
enable-cache: true
|
||||
python-version: ${{ matrix.python-version }}
|
||||
@@ -48,16 +48,16 @@ jobs:
|
||||
- name: Install dependencies
|
||||
run: uv sync --project api --dev
|
||||
|
||||
# - name: Set up Vector Store (TiDB)
|
||||
# uses: hoverkraft-tech/compose-action@v2.0.2
|
||||
# with:
|
||||
# compose-file: docker/tidb/docker-compose.yaml
|
||||
# services: |
|
||||
# tidb
|
||||
# tiflash
|
||||
# - name: Set up Vector Store (TiDB)
|
||||
# uses: hoverkraft-tech/compose-action@v2.0.2
|
||||
# with:
|
||||
# compose-file: docker/tidb/docker-compose.yaml
|
||||
# services: |
|
||||
# tidb
|
||||
# tiflash
|
||||
|
||||
# - name: Check VDB Ready (TiDB)
|
||||
# run: uv run --project api python api/providers/vdb/tidb-vector/tests/integration_tests/check_tiflash_ready.py
|
||||
# - name: Check VDB Ready (TiDB)
|
||||
# run: uv run --project api python api/providers/vdb/tidb-vector/tests/integration_tests/check_tiflash_ready.py
|
||||
|
||||
- name: Test Vector Stores
|
||||
run: |
|
||||
|
||||
@@ -17,11 +17,11 @@ jobs:
|
||||
strategy:
|
||||
matrix:
|
||||
python-version:
|
||||
- "3.12"
|
||||
- '3.12'
|
||||
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
|
||||
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
||||
with:
|
||||
persist-credentials: false
|
||||
|
||||
@@ -33,7 +33,7 @@ jobs:
|
||||
remove_tool_cache: true
|
||||
|
||||
- name: Setup UV and Python
|
||||
uses: astral-sh/setup-uv@fac544c07dec837d0ccb6301d7b5580bf5edae39 # v8.2.0
|
||||
uses: astral-sh/setup-uv@08807647e7069bb48b6ef5acd8ec9567f424441b # v8.1.0
|
||||
with:
|
||||
enable-cache: true
|
||||
python-version: ${{ matrix.python-version }}
|
||||
@@ -45,16 +45,16 @@ jobs:
|
||||
- name: Install dependencies
|
||||
run: uv sync --project api --dev
|
||||
|
||||
# - name: Set up Vector Store (TiDB)
|
||||
# uses: hoverkraft-tech/compose-action@v2.0.2
|
||||
# with:
|
||||
# compose-file: docker/tidb/docker-compose.yaml
|
||||
# services: |
|
||||
# tidb
|
||||
# tiflash
|
||||
# - name: Set up Vector Store (TiDB)
|
||||
# uses: hoverkraft-tech/compose-action@v2.0.2
|
||||
# with:
|
||||
# compose-file: docker/tidb/docker-compose.yaml
|
||||
# services: |
|
||||
# tidb
|
||||
# tiflash
|
||||
|
||||
# - name: Check VDB Ready (TiDB)
|
||||
# run: uv run --project api python api/providers/vdb/tidb-vector/tests/integration_tests/check_tiflash_ready.py
|
||||
# - name: Check VDB Ready (TiDB)
|
||||
# run: uv run --project api python api/providers/vdb/tidb-vector/tests/integration_tests/check_tiflash_ready.py
|
||||
|
||||
- name: Test Vector Stores
|
||||
run: |
|
||||
|
||||
@@ -2,11 +2,6 @@ name: Web Full-Stack E2E
|
||||
|
||||
on:
|
||||
workflow_call:
|
||||
inputs:
|
||||
run-external-runtime:
|
||||
required: false
|
||||
type: boolean
|
||||
default: false
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
@@ -25,7 +20,7 @@ jobs:
|
||||
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
|
||||
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
||||
with:
|
||||
persist-credentials: false
|
||||
|
||||
@@ -33,13 +28,11 @@ jobs:
|
||||
uses: ./.github/actions/setup-web
|
||||
|
||||
- name: Setup UV and Python
|
||||
uses: astral-sh/setup-uv@fac544c07dec837d0ccb6301d7b5580bf5edae39 # v8.2.0
|
||||
uses: astral-sh/setup-uv@08807647e7069bb48b6ef5acd8ec9567f424441b # v8.1.0
|
||||
with:
|
||||
enable-cache: true
|
||||
python-version: "3.12"
|
||||
cache-dependency-glob: |
|
||||
api/uv.lock
|
||||
dify-agent/uv.lock
|
||||
python-version: '3.12'
|
||||
cache-dependency-glob: api/uv.lock
|
||||
|
||||
- name: Install API dependencies
|
||||
run: uv sync --project api --dev
|
||||
@@ -54,56 +47,16 @@ jobs:
|
||||
E2E_ADMIN_EMAIL: e2e-admin@example.com
|
||||
E2E_ADMIN_NAME: E2E Admin
|
||||
E2E_ADMIN_PASSWORD: E2eAdmin12345
|
||||
E2E_FORCE_WEB_BUILD: "1"
|
||||
E2E_FORCE_WEB_BUILD: '1'
|
||||
E2E_INIT_PASSWORD: E2eInit12345
|
||||
run: vp run e2e:full
|
||||
|
||||
- name: Run external runtime E2E tests
|
||||
if: ${{ inputs.run-external-runtime }}
|
||||
working-directory: ./e2e
|
||||
env:
|
||||
E2E_ADMIN_EMAIL: e2e-admin@example.com
|
||||
E2E_ADMIN_NAME: E2E Admin
|
||||
E2E_ADMIN_PASSWORD: E2eAdmin12345
|
||||
E2E_AGENT_DECISION_MODEL_NAME: ${{ vars.E2E_AGENT_DECISION_MODEL_NAME || 'gpt-5.5' }}
|
||||
E2E_AGENT_DECISION_MODEL_PROVIDER: ${{ vars.E2E_AGENT_DECISION_MODEL_PROVIDER || 'openai' }}
|
||||
E2E_AGENT_DECISION_MODEL_TYPE: ${{ vars.E2E_AGENT_DECISION_MODEL_TYPE || 'llm' }}
|
||||
E2E_EXTERNAL_RUNTIME_SEED_SPECS: ${{ vars.E2E_EXTERNAL_RUNTIME_SEED_SPECS }}
|
||||
E2E_EXTERNAL_RUNTIME_TAGS: ${{ vars.E2E_EXTERNAL_RUNTIME_TAGS }}
|
||||
E2E_FORCE_WEB_BUILD: "1"
|
||||
E2E_INIT_PASSWORD: E2eInit12345
|
||||
E2E_MARKETPLACE_API_URL: ${{ vars.E2E_MARKETPLACE_API_URL }}
|
||||
E2E_MARKETPLACE_PLUGIN_IDS: ${{ vars.E2E_MARKETPLACE_PLUGIN_IDS }}
|
||||
E2E_MARKETPLACE_PLUGIN_UNIQUE_IDENTIFIERS: ${{ vars.E2E_MARKETPLACE_PLUGIN_UNIQUE_IDENTIFIERS }}
|
||||
E2E_MODEL_PROVIDER_CREDENTIALS_JSON: ${{ secrets.E2E_MODEL_PROVIDER_CREDENTIALS_JSON }}
|
||||
E2E_START_AGENT_BACKEND: ${{ vars.E2E_START_AGENT_BACKEND || '1' }}
|
||||
E2E_STABLE_MODEL_NAME: ${{ vars.E2E_STABLE_MODEL_NAME || 'gpt-5-nano' }}
|
||||
E2E_STABLE_MODEL_PROVIDER: ${{ vars.E2E_STABLE_MODEL_PROVIDER || 'openai' }}
|
||||
E2E_STABLE_MODEL_TYPE: ${{ vars.E2E_STABLE_MODEL_TYPE || 'llm' }}
|
||||
run: |
|
||||
if [[ -z "${E2E_MODEL_PROVIDER_CREDENTIALS_JSON}" ]]; then
|
||||
echo "E2E_MODEL_PROVIDER_CREDENTIALS_JSON is required for external runtime E2E." >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
if [[ -d cucumber-report ]]; then
|
||||
rm -rf cucumber-report-non-external
|
||||
mv cucumber-report cucumber-report-non-external
|
||||
fi
|
||||
|
||||
trap 'vp run e2e:middleware:down' EXIT
|
||||
vp run e2e:middleware:up
|
||||
vp run e2e:external:prepare
|
||||
vp run e2e:external
|
||||
|
||||
- name: Upload Cucumber report
|
||||
if: ${{ !cancelled() }}
|
||||
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
|
||||
with:
|
||||
name: cucumber-report
|
||||
path: |
|
||||
e2e/cucumber-report
|
||||
e2e/cucumber-report-non-external
|
||||
path: e2e/cucumber-report
|
||||
retention-days: 7
|
||||
|
||||
- name: Upload E2E logs
|
||||
|
||||
@@ -31,7 +31,7 @@ jobs:
|
||||
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
|
||||
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
||||
with:
|
||||
persist-credentials: false
|
||||
|
||||
@@ -64,7 +64,7 @@ jobs:
|
||||
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
|
||||
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
||||
with:
|
||||
persist-credentials: false
|
||||
|
||||
@@ -83,7 +83,7 @@ jobs:
|
||||
|
||||
- name: Report coverage
|
||||
if: ${{ env.CODECOV_TOKEN != '' }}
|
||||
uses: codecov/codecov-action@fb8b3582c8e4def4969c97caa2f19720cb33a72f # v7.0.0
|
||||
uses: codecov/codecov-action@e79a6962e0d4c0c17b229090214935d2e33f8354 # v6.0.1
|
||||
with:
|
||||
directory: web/coverage
|
||||
flags: web
|
||||
@@ -102,7 +102,7 @@ jobs:
|
||||
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
|
||||
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
||||
with:
|
||||
persist-credentials: false
|
||||
|
||||
@@ -113,36 +113,13 @@ jobs:
|
||||
run: vp exec playwright install --with-deps chromium
|
||||
|
||||
- name: Run dify-ui tests
|
||||
run: vp test run --project unit --coverage --silent=passed-only
|
||||
run: vp test run --coverage --silent=passed-only
|
||||
|
||||
- name: Report coverage
|
||||
if: ${{ env.CODECOV_TOKEN != '' }}
|
||||
uses: codecov/codecov-action@fb8b3582c8e4def4969c97caa2f19720cb33a72f # v7.0.0
|
||||
uses: codecov/codecov-action@e79a6962e0d4c0c17b229090214935d2e33f8354 # v6.0.1
|
||||
with:
|
||||
directory: packages/dify-ui/coverage
|
||||
flags: dify-ui
|
||||
env:
|
||||
CODECOV_TOKEN: ${{ env.CODECOV_TOKEN }}
|
||||
|
||||
dify-ui-storybook-test:
|
||||
name: dify-ui Storybook Tests
|
||||
runs-on: depot-ubuntu-24.04-4
|
||||
defaults:
|
||||
run:
|
||||
shell: bash
|
||||
working-directory: ./packages/dify-ui
|
||||
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
|
||||
with:
|
||||
persist-credentials: false
|
||||
|
||||
- name: Setup web environment
|
||||
uses: ./.github/actions/setup-web
|
||||
|
||||
- name: Install Chromium for Browser Mode
|
||||
run: vp exec playwright install --with-deps chromium
|
||||
|
||||
- name: Run dify-ui Storybook tests
|
||||
run: vp run test:storybook
|
||||
|
||||
@@ -1,30 +0,0 @@
|
||||
# HITL timeout semantics implementation report
|
||||
|
||||
## What changed
|
||||
|
||||
- Updated `api/core/workflow/nodes/human_input/callback.py` so `DifyHITLCallback` now preserves Dify's timeout split at the boundary:
|
||||
- `HumanInputFormStatus.TIMEOUT` returns the graphon timeout branch via `Expired(selected_handle="__timeout__", ...)`.
|
||||
- `HumanInputFormStatus.EXPIRED` is treated as an invalid resume state and raises `AssertionError`.
|
||||
- `HumanInputFormStatus.WAITING` with a past global deadline is treated as an invalid resume state and raises `AssertionError`.
|
||||
- `HumanInputFormStatus.WAITING` with only the node-level deadline expired still returns the timeout branch.
|
||||
- Added `created_at` to `HumanInputFormEntity` and `_HumanInputFormEntityImpl` so the callback can compute the global deadline using Dify's shared `HUMAN_INPUT_GLOBAL_TIMEOUT_SECONDS` invariant.
|
||||
- Kept the submitted and pause flows unchanged.
|
||||
- Added focused unit coverage in `api/tests/unit_tests/core/workflow/test_human_input_callback.py` for:
|
||||
- node timeout branch
|
||||
- global expiration rejection
|
||||
- waiting-form past node deadline timeout
|
||||
- waiting-form past global deadline rejection
|
||||
|
||||
## Verification
|
||||
|
||||
- `uv run --project api pytest -o addopts='' api/tests/unit_tests/core/workflow/test_human_input_callback.py api/tests/unit_tests/core/workflow/nodes/human_input/test_human_input_form_filled_event.py -q`
|
||||
- `git diff --check`
|
||||
|
||||
## Result
|
||||
|
||||
- The focused test set is expected to pass with the new `created_at` boundary in place.
|
||||
- No unrelated files were modified.
|
||||
|
||||
## Concerns
|
||||
|
||||
- The callback now fails fast on invalid resume states by design. That is intentional, but any caller that previously relied on `EXPIRED` being mapped to the timeout branch will now see an assertion failure instead.
|
||||
Vendored
+9
-23
@@ -1,31 +1,17 @@
|
||||
{
|
||||
"cucumber.features": [
|
||||
"e2e/features/**/*.feature",
|
||||
],
|
||||
"cucumber.glue": [
|
||||
"e2e/features/**/*.ts",
|
||||
],
|
||||
"cucumber.features": ["e2e/features/**/*.feature"],
|
||||
"cucumber.glue": ["e2e/features/**/*.ts"],
|
||||
|
||||
"tailwindCSS.experimental.configFile": "web/app/styles/globals.css",
|
||||
|
||||
// Auto fix
|
||||
"editor.codeActionsOnSave": {
|
||||
"source.fixAll.eslint": "explicit",
|
||||
},
|
||||
// Format
|
||||
"editor.formatOnSave": true,
|
||||
"oxc.fmt.configPath": "./vite.config.ts",
|
||||
|
||||
// Silent the stylistic rules in your IDE, but still auto fix them
|
||||
"eslint.rules.customizations": [
|
||||
{ "rule": "style/*", "severity": "off", "fixable": true },
|
||||
{ "rule": "format/*", "severity": "off", "fixable": true },
|
||||
{ "rule": "*-indent", "severity": "off", "fixable": true },
|
||||
{ "rule": "*-spacing", "severity": "off", "fixable": true },
|
||||
{ "rule": "*-spaces", "severity": "off", "fixable": true },
|
||||
{ "rule": "*-order", "severity": "off", "fixable": true },
|
||||
{ "rule": "*-dangle", "severity": "off", "fixable": true },
|
||||
{ "rule": "*-newline", "severity": "off", "fixable": true },
|
||||
{ "rule": "*quotes", "severity": "off", "fixable": true },
|
||||
{ "rule": "*semi", "severity": "off", "fixable": true }
|
||||
],
|
||||
// Lint fix
|
||||
"editor.codeActionsOnSave": {
|
||||
"source.fixAll.eslint": "explicit"
|
||||
},
|
||||
|
||||
// Enable eslint for all supported languages
|
||||
"eslint.validate": [
|
||||
|
||||
@@ -31,7 +31,7 @@ The codebase is split into:
|
||||
## Language Style
|
||||
|
||||
- **Python**: Keep type hints on functions and attributes, and implement relevant special methods (e.g., `__repr__`, `__str__`). Prefer `TypedDict` over `dict` or `Mapping` for type safety and better code documentation.
|
||||
- **TypeScript**: Use the strict config, rely on ESLint (`pnpm lint:fix` preferred) plus `pnpm type-check`, and avoid `any` types.
|
||||
- **TypeScript**: Use the strict config, rely on `vp fmt` for formatting, ESLint for lint-only fixes, plus `pnpm type-check`, and avoid `any` types.
|
||||
|
||||
## General Practices
|
||||
|
||||
|
||||
+10
-10
@@ -34,11 +34,11 @@ Don't forget to link an existing issue or open a new issue in the PR's descripti
|
||||
|
||||
How we prioritize:
|
||||
|
||||
| Issue Type | Priority |
|
||||
| ------------------------------------------------------------ | --------------- |
|
||||
| Bugs in core functions (cloud service, cannot login, applications not working, security loopholes) | Critical |
|
||||
| Non-critical bugs, performance boosts | Medium Priority |
|
||||
| Minor fixes (typos, confusing but working UI) | Low Priority |
|
||||
| Issue Type | Priority |
|
||||
| -------------------------------------------------------------------------------------------------- | --------------- |
|
||||
| Bugs in core functions (cloud service, cannot login, applications not working, security loopholes) | Critical |
|
||||
| Non-critical bugs, performance boosts | Medium Priority |
|
||||
| Minor fixes (typos, confusing but working UI) | Low Priority |
|
||||
|
||||
### Feature requests
|
||||
|
||||
@@ -52,12 +52,12 @@ How we prioritize:
|
||||
|
||||
How we prioritize:
|
||||
|
||||
| Feature Type | Priority |
|
||||
| ------------------------------------------------------------ | --------------- |
|
||||
| High-Priority Features as being labeled by a team member | High Priority |
|
||||
| Feature Type | Priority |
|
||||
| --------------------------------------------------------------------------------------------------------------------------------- | --------------- |
|
||||
| High-Priority Features as being labeled by a team member | High Priority |
|
||||
| Popular feature requests from our [community feedback board](https://github.com/langgenius/dify/discussions/categories/feedbacks) | Medium Priority |
|
||||
| Non-core features and minor enhancements | Low Priority |
|
||||
| Valuable but not immediate | Future-Feature |
|
||||
| Non-core features and minor enhancements | Low Priority |
|
||||
| Valuable but not immediate | Future-Feature |
|
||||
|
||||
## Submitting your PR
|
||||
|
||||
|
||||
@@ -116,7 +116,7 @@ All of Dify's offerings come with corresponding APIs, so you could effortlessly
|
||||
## Using Dify
|
||||
|
||||
- **Cloud <br/>**
|
||||
We host a [Dify Cloud](https://dify.ai) service for anyone to try with zero setup. It provides all the capabilities of the self-deployed version, and includes 200 free GPT-4 calls in the sandbox plan. If you run into issues with Dify Cloud, [contact our Cloud support team](mailto:cloud@dify.ai?subject=%5BGitHub%5DDify%20Cloud%20Support).
|
||||
We host a [Dify Cloud](https://dify.ai) service for anyone to try with zero setup. It provides all the capabilities of the self-deployed version, and includes 200 free GPT-4 calls in the sandbox plan.
|
||||
|
||||
- **Self-hosting Dify Community Edition<br/>**
|
||||
Quickly get Dify running in your environment with this [starter guide](#quick-start).
|
||||
|
||||
+1
-11
@@ -36,9 +36,6 @@ FILES_ACCESS_TIMEOUT=300
|
||||
# Collaboration mode toggle
|
||||
ENABLE_COLLABORATION_MODE=true
|
||||
|
||||
# Learn app feature toggle
|
||||
ENABLE_LEARN_APP=true
|
||||
|
||||
# Access token expiration time in minutes
|
||||
ACCESS_TOKEN_EXPIRE_MINUTES=60
|
||||
|
||||
@@ -771,16 +768,9 @@ EVENT_BUS_REDIS_CHANNEL_TYPE=pubsub
|
||||
# Whether to use Redis cluster mode while use redis as event bus.
|
||||
# It's highly recommended to enable this for large deployments.
|
||||
EVENT_BUS_REDIS_USE_CLUSTERS=false
|
||||
EVENT_BUS_LISTENER_JOIN_TIMEOUT_MS=2000
|
||||
|
||||
# Whether to Enable human input timeout check task
|
||||
ENABLE_HUMAN_INPUT_TIMEOUT_TASK=true
|
||||
# Human input timeout check interval in minutes
|
||||
HUMAN_INPUT_TIMEOUT_TASK_INTERVAL=1
|
||||
|
||||
# Nacos remote settings source HTTP timeouts (seconds).
|
||||
# Bound how long requests to the Nacos endpoint wait before failing, so a slow or
|
||||
# unresponsive Nacos server cannot stall API startup or token refresh.
|
||||
# Read timeout for Nacos requests (default: 10.0)
|
||||
DIFY_ENV_NACOS_REQUEST_TIMEOUT=10.0
|
||||
# Connect timeout for Nacos requests (default: 3.0)
|
||||
DIFY_ENV_NACOS_CONNECT_TIMEOUT=3.0
|
||||
|
||||
@@ -32,10 +32,7 @@ from clients.agent_backend.factory import create_agent_backend_run_client
|
||||
from clients.agent_backend.fake_client import FakeAgentBackendRunClient, FakeAgentBackendScenario
|
||||
from clients.agent_backend.request_builder import (
|
||||
AGENT_SOUL_PROMPT_LAYER_ID,
|
||||
DIFY_CONFIG_LAYER_ID,
|
||||
DIFY_CORE_TOOLS_LAYER_ID,
|
||||
DIFY_EXECUTION_CONTEXT_LAYER_ID,
|
||||
DIFY_KNOWLEDGE_BASE_LAYER_ID,
|
||||
DIFY_PLUGIN_TOOLS_LAYER_ID,
|
||||
WORKFLOW_NODE_JOB_PROMPT_LAYER_ID,
|
||||
WORKFLOW_USER_PROMPT_LAYER_ID,
|
||||
@@ -49,10 +46,7 @@ from clients.agent_backend.request_builder import (
|
||||
|
||||
__all__ = [
|
||||
"AGENT_SOUL_PROMPT_LAYER_ID",
|
||||
"DIFY_CONFIG_LAYER_ID",
|
||||
"DIFY_CORE_TOOLS_LAYER_ID",
|
||||
"DIFY_EXECUTION_CONTEXT_LAYER_ID",
|
||||
"DIFY_KNOWLEDGE_BASE_LAYER_ID",
|
||||
"DIFY_PLUGIN_TOOLS_LAYER_ID",
|
||||
"WORKFLOW_NODE_JOB_PROMPT_LAYER_ID",
|
||||
"WORKFLOW_USER_PROMPT_LAYER_ID",
|
||||
|
||||
@@ -67,7 +67,6 @@ class AgentBackendRunSucceededInternalEvent(AgentBackendInternalEventBase):
|
||||
type: Literal[AgentBackendInternalEventType.RUN_SUCCEEDED] = AgentBackendInternalEventType.RUN_SUCCEEDED
|
||||
output: JsonValue
|
||||
session_snapshot: CompositorSessionSnapshot
|
||||
usage: dict[str, JsonValue] | None = None
|
||||
|
||||
|
||||
class AgentBackendDeferredToolCallInternalEvent(AgentBackendInternalEventBase):
|
||||
@@ -77,7 +76,6 @@ class AgentBackendDeferredToolCallInternalEvent(AgentBackendInternalEventBase):
|
||||
deferred_tool_call: DeferredToolCallPayload
|
||||
message: str | None = None
|
||||
session_snapshot: CompositorSessionSnapshot
|
||||
usage: dict[str, JsonValue] | None = None
|
||||
|
||||
|
||||
class AgentBackendRunFailedInternalEvent(AgentBackendInternalEventBase):
|
||||
@@ -142,7 +140,6 @@ class AgentBackendRunEventAdapter:
|
||||
deferred_tool_call=event.data.deferred_tool_call,
|
||||
message=_deferred_tool_call_message(event.data.deferred_tool_call),
|
||||
session_snapshot=event.data.session_snapshot,
|
||||
usage=_agent_run_usage(event.data.usage),
|
||||
)
|
||||
]
|
||||
return [
|
||||
@@ -151,7 +148,6 @@ class AgentBackendRunEventAdapter:
|
||||
source_event_id=event.id,
|
||||
output=event.data.output,
|
||||
session_snapshot=event.data.session_snapshot,
|
||||
usage=_agent_run_usage(event.data.usage),
|
||||
)
|
||||
]
|
||||
case RunFailedEvent():
|
||||
@@ -188,13 +184,3 @@ def _deferred_tool_call_message(payload: DeferredToolCallPayload) -> str:
|
||||
return title
|
||||
|
||||
return f"Agent backend requested external input via deferred tool '{payload.tool_name}'."
|
||||
|
||||
|
||||
def _agent_run_usage(usage: object | None) -> dict[str, JsonValue] | None:
|
||||
"""Return JSON-safe usage metadata from optional Agent backend usage."""
|
||||
if usage is None:
|
||||
return None
|
||||
dumped = _EVENT_DATA_ADAPTER.dump_python(usage, mode="json")
|
||||
if not isinstance(dumped, dict):
|
||||
return None
|
||||
return cast(dict[str, JsonValue], dumped)
|
||||
|
||||
@@ -19,9 +19,6 @@ from agenton.compositor.schemas import LayerSessionSnapshot
|
||||
from agenton.layers import ExitIntent
|
||||
from agenton_collections.layers.plain import PLAIN_PROMPT_LAYER_TYPE_ID, PromptLayerConfig
|
||||
from agenton_collections.layers.pydantic_ai import PYDANTIC_AI_HISTORY_LAYER_TYPE_ID
|
||||
from dify_agent.layers.ask_human import DIFY_ASK_HUMAN_LAYER_TYPE_ID, DifyAskHumanLayerConfig
|
||||
from dify_agent.layers.config import DIFY_CONFIG_LAYER_TYPE_ID, DifyConfigLayerConfig
|
||||
from dify_agent.layers.dify_core_tools import DIFY_CORE_TOOLS_LAYER_TYPE_ID, DifyCoreToolsLayerConfig
|
||||
from dify_agent.layers.dify_plugin import (
|
||||
DIFY_PLUGIN_LLM_LAYER_TYPE_ID,
|
||||
DIFY_PLUGIN_TOOLS_LAYER_TYPE_ID,
|
||||
@@ -34,7 +31,6 @@ from dify_agent.layers.execution_context import (
|
||||
DIFY_EXECUTION_CONTEXT_LAYER_TYPE_ID,
|
||||
DifyExecutionContextLayerConfig,
|
||||
)
|
||||
from dify_agent.layers.knowledge import DIFY_KNOWLEDGE_BASE_LAYER_TYPE_ID, DifyKnowledgeBaseLayerConfig
|
||||
from dify_agent.layers.output import DIFY_OUTPUT_LAYER_TYPE_ID, DifyOutputLayerConfig
|
||||
from dify_agent.layers.shell import DIFY_SHELL_LAYER_TYPE_ID, DifyShellLayerConfig
|
||||
from dify_agent.protocol import (
|
||||
@@ -42,7 +38,6 @@ from dify_agent.protocol import (
|
||||
DIFY_AGENT_MODEL_LAYER_ID,
|
||||
DIFY_AGENT_OUTPUT_LAYER_ID,
|
||||
CreateRunRequest,
|
||||
DeferredToolResultsPayload,
|
||||
LayerExitSignals,
|
||||
RunComposition,
|
||||
RunLayerSpec,
|
||||
@@ -56,12 +51,8 @@ WORKFLOW_NODE_JOB_PROMPT_LAYER_ID = "workflow_node_job_prompt"
|
||||
WORKFLOW_USER_PROMPT_LAYER_ID = "workflow_user_prompt"
|
||||
AGENT_APP_USER_PROMPT_LAYER_ID = "agent_app_user_prompt"
|
||||
DIFY_EXECUTION_CONTEXT_LAYER_ID = "execution_context"
|
||||
DIFY_CONFIG_LAYER_ID = "config"
|
||||
DIFY_DRIVE_LAYER_ID = "drive"
|
||||
DIFY_PLUGIN_TOOLS_LAYER_ID = "tools"
|
||||
DIFY_CORE_TOOLS_LAYER_ID = "core_tools"
|
||||
DIFY_KNOWLEDGE_BASE_LAYER_ID = "knowledge"
|
||||
DIFY_ASK_HUMAN_LAYER_ID = "ask_human"
|
||||
DIFY_SHELL_LAYER_ID = "shell"
|
||||
|
||||
|
||||
@@ -82,28 +73,6 @@ def _filter_snapshot_to_specs(
|
||||
return CompositorSessionSnapshot(schema_version=snapshot.schema_version, layers=filtered_layers)
|
||||
|
||||
|
||||
def _shell_layer_deps() -> dict[str, str]:
|
||||
return {"execution_context": DIFY_EXECUTION_CONTEXT_LAYER_ID}
|
||||
|
||||
|
||||
def _drive_layer_deps() -> dict[str, str]:
|
||||
return {"shell": DIFY_SHELL_LAYER_ID}
|
||||
|
||||
|
||||
def _config_layer_deps() -> dict[str, str]:
|
||||
return {"shell": DIFY_SHELL_LAYER_ID}
|
||||
|
||||
|
||||
def _shell_config_with_drive_ref(
|
||||
shell_config: DifyShellLayerConfig | None,
|
||||
drive_config: DifyDriveLayerConfig | None,
|
||||
) -> DifyShellLayerConfig:
|
||||
config = shell_config or DifyShellLayerConfig()
|
||||
if drive_config is None:
|
||||
return config
|
||||
return config.model_copy(update={"agent_stub_drive_ref": drive_config.drive_ref})
|
||||
|
||||
|
||||
class AgentBackendModelConfig(BaseModel):
|
||||
"""API-side model/plugin selection before it is converted to Dify Agent layers."""
|
||||
|
||||
@@ -167,24 +136,14 @@ class AgentBackendWorkflowNodeRunInput(BaseModel):
|
||||
idempotency_key: str | None = None
|
||||
output: AgentBackendOutputConfig | None = None
|
||||
tools: DifyPluginToolsLayerConfig | None = None
|
||||
core_tools: DifyCoreToolsLayerConfig | None = None
|
||||
knowledge: DifyKnowledgeBaseLayerConfig | None = None
|
||||
config_layer_config: DifyConfigLayerConfig | None = None
|
||||
# Drive Skills & Files declaration (dify.drive) — an index the agent pulls
|
||||
# through the back proxy, never inline content.
|
||||
# through the back proxy, never inline content; see AGENT_DRIVE_MANIFEST_ENABLED.
|
||||
drive_config: DifyDriveLayerConfig | None = None
|
||||
# Human-in-the-loop ask_human deferred tool (dify.ask_human). Present only when
|
||||
# the Agent Soul configures human involvement; a deferred call ends the run and
|
||||
# the workflow pauses via the existing HITL form mechanism (ENG-635).
|
||||
ask_human_config: DifyAskHumanLayerConfig | None = None
|
||||
# Inject the sandboxed shell layer (dify.shell). Requires the agent backend
|
||||
# to be wired with a shellctl entrypoint; see configs AGENT_SHELL_ENABLED.
|
||||
include_shell: bool = False
|
||||
shell_config: DifyShellLayerConfig | None = None
|
||||
session_snapshot: CompositorSessionSnapshot | None = None
|
||||
# Human tool results fed back into a continuation run after a HITL submission
|
||||
# (ENG-638). Keyed by the original deferred tool_call_id.
|
||||
deferred_tool_results: DeferredToolResultsPayload | None = None
|
||||
include_history: bool = True
|
||||
suspend_on_exit: bool = True
|
||||
metadata: dict[str, JsonValue] = Field(default_factory=dict)
|
||||
@@ -216,23 +175,14 @@ class AgentBackendAgentAppRunInput(BaseModel):
|
||||
idempotency_key: str | None = None
|
||||
output: AgentBackendOutputConfig | None = None
|
||||
tools: DifyPluginToolsLayerConfig | None = None
|
||||
core_tools: DifyCoreToolsLayerConfig | None = None
|
||||
knowledge: DifyKnowledgeBaseLayerConfig | None = None
|
||||
config_layer_config: DifyConfigLayerConfig | None = None
|
||||
# Drive Skills & Files declaration (dify.drive) — an index the agent pulls
|
||||
# through the back proxy, never inline content.
|
||||
# through the back proxy, never inline content; see AGENT_DRIVE_MANIFEST_ENABLED.
|
||||
drive_config: DifyDriveLayerConfig | None = None
|
||||
# Human-in-the-loop ask_human deferred tool (dify.ask_human). Present only when
|
||||
# the Agent Soul configures human involvement (ENG-635).
|
||||
ask_human_config: DifyAskHumanLayerConfig | None = None
|
||||
# Inject the sandboxed shell layer (dify.shell). Requires the agent backend
|
||||
# to be wired with a shellctl entrypoint; see configs AGENT_SHELL_ENABLED.
|
||||
include_shell: bool = False
|
||||
shell_config: DifyShellLayerConfig | None = None
|
||||
session_snapshot: CompositorSessionSnapshot | None = None
|
||||
# Human tool results fed back into a continuation run after a HITL submission
|
||||
# (ENG-638). Keyed by the original deferred tool_call_id.
|
||||
deferred_tool_results: DeferredToolResultsPayload | None = None
|
||||
include_history: bool = True
|
||||
suspend_on_exit: bool = True
|
||||
metadata: dict[str, JsonValue] = Field(default_factory=dict)
|
||||
@@ -254,11 +204,9 @@ class AgentBackendRunRequestBuilder:
|
||||
"""Build an Agent App conversation-turn run request.
|
||||
|
||||
Layer graph: optional Agent Soul system prompt → user prompt →
|
||||
execution context → optional shell / config / drive / history
|
||||
(multi-turn) → LLM → optional plugin-direct tools / core-routed tools /
|
||||
knowledge search / ask_human / structured output. Mirrors the
|
||||
workflow-node layer ordering minus the workflow-job / previous-node
|
||||
prompt.
|
||||
execution context → optional history (multi-turn) → LLM → optional
|
||||
plugin tools → optional structured output. Mirrors the workflow-node
|
||||
layer ordering minus the workflow-job / previous-node prompt.
|
||||
"""
|
||||
layers: list[RunLayerSpec] = []
|
||||
if run_input.agent_soul_prompt:
|
||||
@@ -288,42 +236,13 @@ class AgentBackendRunRequestBuilder:
|
||||
]
|
||||
)
|
||||
|
||||
include_shell = (
|
||||
run_input.include_shell or run_input.config_layer_config is not None or run_input.drive_config is not None
|
||||
)
|
||||
if include_shell:
|
||||
# Sandboxed bash workspace (dify.shell). It enters before config/drive
|
||||
# so eager pulls materialize content in the same filesystem used by
|
||||
# model commands.
|
||||
layers.append(
|
||||
RunLayerSpec(
|
||||
name=DIFY_SHELL_LAYER_ID,
|
||||
type=DIFY_SHELL_LAYER_TYPE_ID,
|
||||
deps=_shell_layer_deps(),
|
||||
metadata=run_input.metadata,
|
||||
config=_shell_config_with_drive_ref(run_input.shell_config, run_input.drive_config),
|
||||
)
|
||||
)
|
||||
|
||||
if run_input.config_layer_config is not None:
|
||||
layers.append(
|
||||
RunLayerSpec(
|
||||
name=DIFY_CONFIG_LAYER_ID,
|
||||
type=DIFY_CONFIG_LAYER_TYPE_ID,
|
||||
deps=_config_layer_deps(),
|
||||
metadata=run_input.metadata,
|
||||
config=run_input.config_layer_config,
|
||||
)
|
||||
)
|
||||
|
||||
if run_input.drive_config is not None:
|
||||
# Drive Skills & Files declaration (dify.drive): the catalog plus
|
||||
# prompt-mentioned entries eagerly pulled through the shell layer.
|
||||
# Drive Skills & Files declaration (dify.drive): a config-only index;
|
||||
# the agent pulls listed entries through the back proxy by drive_ref.
|
||||
layers.append(
|
||||
RunLayerSpec(
|
||||
name=DIFY_DRIVE_LAYER_ID,
|
||||
type=DIFY_DRIVE_LAYER_TYPE_ID,
|
||||
deps=_drive_layer_deps(),
|
||||
metadata=run_input.metadata,
|
||||
config=run_input.drive_config,
|
||||
)
|
||||
@@ -355,51 +274,27 @@ class AgentBackendRunRequestBuilder:
|
||||
)
|
||||
|
||||
if run_input.tools is not None and run_input.tools.tools:
|
||||
plugin_tool_deps = {"execution_context": DIFY_EXECUTION_CONTEXT_LAYER_ID}
|
||||
if include_shell:
|
||||
plugin_tool_deps["shell"] = DIFY_SHELL_LAYER_ID
|
||||
layers.append(
|
||||
RunLayerSpec(
|
||||
name=DIFY_PLUGIN_TOOLS_LAYER_ID,
|
||||
type=DIFY_PLUGIN_TOOLS_LAYER_TYPE_ID,
|
||||
deps=plugin_tool_deps,
|
||||
deps={"execution_context": DIFY_EXECUTION_CONTEXT_LAYER_ID},
|
||||
metadata=run_input.metadata,
|
||||
config=run_input.tools,
|
||||
)
|
||||
)
|
||||
|
||||
if run_input.core_tools is not None and run_input.core_tools.tools:
|
||||
if run_input.include_shell:
|
||||
# Sandboxed bash workspace (dify.shell). Depends on execution_context so
|
||||
# the agent server can mint per-command Agent Stub env (back proxy);
|
||||
# shellctl connection itself is server-injected.
|
||||
layers.append(
|
||||
RunLayerSpec(
|
||||
name=DIFY_CORE_TOOLS_LAYER_ID,
|
||||
type=DIFY_CORE_TOOLS_LAYER_TYPE_ID,
|
||||
name=DIFY_SHELL_LAYER_ID,
|
||||
type=DIFY_SHELL_LAYER_TYPE_ID,
|
||||
deps={"execution_context": DIFY_EXECUTION_CONTEXT_LAYER_ID},
|
||||
metadata=run_input.metadata,
|
||||
config=run_input.core_tools,
|
||||
)
|
||||
)
|
||||
|
||||
if run_input.knowledge is not None and run_input.knowledge.sets:
|
||||
layers.append(
|
||||
RunLayerSpec(
|
||||
name=DIFY_KNOWLEDGE_BASE_LAYER_ID,
|
||||
type=DIFY_KNOWLEDGE_BASE_LAYER_TYPE_ID,
|
||||
deps={"execution_context": DIFY_EXECUTION_CONTEXT_LAYER_ID},
|
||||
metadata=run_input.metadata,
|
||||
config=run_input.knowledge,
|
||||
)
|
||||
)
|
||||
|
||||
if run_input.ask_human_config is not None:
|
||||
# Human-in-the-loop ask_human deferred tool (dify.ask_human). A call ends
|
||||
# the run with a deferred_tool_call; the caller pauses (workflow HITL) and
|
||||
# later resumes with deferred_tool_results. Needs the history layer above.
|
||||
layers.append(
|
||||
RunLayerSpec(
|
||||
name=DIFY_ASK_HUMAN_LAYER_ID,
|
||||
type=DIFY_ASK_HUMAN_LAYER_TYPE_ID,
|
||||
metadata=run_input.metadata,
|
||||
config=run_input.ask_human_config,
|
||||
config=run_input.shell_config or DifyShellLayerConfig(),
|
||||
)
|
||||
)
|
||||
|
||||
@@ -423,7 +318,6 @@ class AgentBackendRunRequestBuilder:
|
||||
idempotency_key=run_input.idempotency_key,
|
||||
metadata=run_input.metadata,
|
||||
session_snapshot=run_input.session_snapshot,
|
||||
deferred_tool_results=run_input.deferred_tool_results,
|
||||
on_exit=LayerExitSignals(
|
||||
default=ExitIntent.SUSPEND if run_input.suspend_on_exit else ExitIntent.DELETE,
|
||||
),
|
||||
@@ -444,8 +338,7 @@ class AgentBackendRunRequestBuilder:
|
||||
non-plugin layer graph that produced the snapshot. Plugin layers
|
||||
(``dify.plugin.llm``, ``dify.plugin.tools``) are excluded from both the
|
||||
composition and the snapshot before submission because their configs
|
||||
may carry credentials or runtime-only declarations that are not
|
||||
persisted between runs.
|
||||
require credentials that are not persisted between runs.
|
||||
"""
|
||||
if not runtime_layer_specs:
|
||||
raise ValueError(
|
||||
@@ -475,13 +368,7 @@ class AgentBackendRunRequestBuilder:
|
||||
)
|
||||
|
||||
def build_for_workflow_node(self, run_input: AgentBackendWorkflowNodeRunInput) -> CreateRunRequest:
|
||||
"""Build a workflow Agent Node run request without defining another wire schema.
|
||||
|
||||
Layer graph mirrors the workflow surface: prompts → execution context →
|
||||
optional shell / config / drive / history → LLM → optional
|
||||
plugin-direct tools / core-routed tools / knowledge search /
|
||||
ask_human / structured output.
|
||||
"""
|
||||
"""Build a workflow Agent Node run request without defining another wire schema."""
|
||||
layers: list[RunLayerSpec] = []
|
||||
if run_input.agent_soul_prompt:
|
||||
layers.append(
|
||||
@@ -499,7 +386,7 @@ class AgentBackendRunRequestBuilder:
|
||||
name=WORKFLOW_NODE_JOB_PROMPT_LAYER_ID,
|
||||
type=PLAIN_PROMPT_LAYER_TYPE_ID,
|
||||
metadata={**run_input.metadata, "origin": "workflow_node_job"},
|
||||
config=PromptLayerConfig(user=run_input.workflow_node_job_prompt),
|
||||
config=PromptLayerConfig(prefix=run_input.workflow_node_job_prompt),
|
||||
),
|
||||
RunLayerSpec(
|
||||
name=WORKFLOW_USER_PROMPT_LAYER_ID,
|
||||
@@ -516,42 +403,13 @@ class AgentBackendRunRequestBuilder:
|
||||
]
|
||||
)
|
||||
|
||||
include_shell = (
|
||||
run_input.include_shell or run_input.config_layer_config is not None or run_input.drive_config is not None
|
||||
)
|
||||
if include_shell:
|
||||
# Sandboxed bash workspace (dify.shell). It enters before drive so
|
||||
# drive can materialize mentioned targets with `dify-agent drive pull`
|
||||
# in the same shell-visible filesystem used by model commands.
|
||||
layers.append(
|
||||
RunLayerSpec(
|
||||
name=DIFY_SHELL_LAYER_ID,
|
||||
type=DIFY_SHELL_LAYER_TYPE_ID,
|
||||
deps=_shell_layer_deps(),
|
||||
metadata=run_input.metadata,
|
||||
config=_shell_config_with_drive_ref(run_input.shell_config, run_input.drive_config),
|
||||
)
|
||||
)
|
||||
|
||||
if run_input.config_layer_config is not None:
|
||||
layers.append(
|
||||
RunLayerSpec(
|
||||
name=DIFY_CONFIG_LAYER_ID,
|
||||
type=DIFY_CONFIG_LAYER_TYPE_ID,
|
||||
deps=_config_layer_deps(),
|
||||
metadata=run_input.metadata,
|
||||
config=run_input.config_layer_config,
|
||||
)
|
||||
)
|
||||
|
||||
if run_input.drive_config is not None:
|
||||
# Drive Skills & Files declaration (dify.drive): the catalog plus
|
||||
# prompt-mentioned entries eagerly pulled through the shell layer.
|
||||
# Drive Skills & Files declaration (dify.drive): a config-only index;
|
||||
# the agent pulls listed entries through the back proxy by drive_ref.
|
||||
layers.append(
|
||||
RunLayerSpec(
|
||||
name=DIFY_DRIVE_LAYER_ID,
|
||||
type=DIFY_DRIVE_LAYER_TYPE_ID,
|
||||
deps=_drive_layer_deps(),
|
||||
metadata=run_input.metadata,
|
||||
config=run_input.drive_config,
|
||||
)
|
||||
@@ -585,51 +443,27 @@ class AgentBackendRunRequestBuilder:
|
||||
)
|
||||
|
||||
if run_input.tools is not None and run_input.tools.tools:
|
||||
plugin_tool_deps = {"execution_context": DIFY_EXECUTION_CONTEXT_LAYER_ID}
|
||||
if include_shell:
|
||||
plugin_tool_deps["shell"] = DIFY_SHELL_LAYER_ID
|
||||
layers.append(
|
||||
RunLayerSpec(
|
||||
name=DIFY_PLUGIN_TOOLS_LAYER_ID,
|
||||
type=DIFY_PLUGIN_TOOLS_LAYER_TYPE_ID,
|
||||
deps=plugin_tool_deps,
|
||||
deps={"execution_context": DIFY_EXECUTION_CONTEXT_LAYER_ID},
|
||||
metadata=run_input.metadata,
|
||||
config=run_input.tools,
|
||||
)
|
||||
)
|
||||
|
||||
if run_input.core_tools is not None and run_input.core_tools.tools:
|
||||
if run_input.include_shell:
|
||||
# Sandboxed bash workspace (dify.shell). Depends on execution_context so
|
||||
# the agent server can mint per-command Agent Stub env (back proxy);
|
||||
# shellctl connection itself is server-injected.
|
||||
layers.append(
|
||||
RunLayerSpec(
|
||||
name=DIFY_CORE_TOOLS_LAYER_ID,
|
||||
type=DIFY_CORE_TOOLS_LAYER_TYPE_ID,
|
||||
name=DIFY_SHELL_LAYER_ID,
|
||||
type=DIFY_SHELL_LAYER_TYPE_ID,
|
||||
deps={"execution_context": DIFY_EXECUTION_CONTEXT_LAYER_ID},
|
||||
metadata=run_input.metadata,
|
||||
config=run_input.core_tools,
|
||||
)
|
||||
)
|
||||
|
||||
if run_input.knowledge is not None and run_input.knowledge.sets:
|
||||
layers.append(
|
||||
RunLayerSpec(
|
||||
name=DIFY_KNOWLEDGE_BASE_LAYER_ID,
|
||||
type=DIFY_KNOWLEDGE_BASE_LAYER_TYPE_ID,
|
||||
deps={"execution_context": DIFY_EXECUTION_CONTEXT_LAYER_ID},
|
||||
metadata=run_input.metadata,
|
||||
config=run_input.knowledge,
|
||||
)
|
||||
)
|
||||
|
||||
if run_input.ask_human_config is not None:
|
||||
# Human-in-the-loop ask_human deferred tool (dify.ask_human). A call ends
|
||||
# the run with a deferred_tool_call; the caller pauses (workflow HITL) and
|
||||
# later resumes with deferred_tool_results. Needs the history layer above.
|
||||
layers.append(
|
||||
RunLayerSpec(
|
||||
name=DIFY_ASK_HUMAN_LAYER_ID,
|
||||
type=DIFY_ASK_HUMAN_LAYER_TYPE_ID,
|
||||
metadata=run_input.metadata,
|
||||
config=run_input.ask_human_config,
|
||||
config=run_input.shell_config or DifyShellLayerConfig(),
|
||||
)
|
||||
)
|
||||
|
||||
@@ -653,7 +487,6 @@ class AgentBackendRunRequestBuilder:
|
||||
idempotency_key=run_input.idempotency_key,
|
||||
metadata=run_input.metadata,
|
||||
session_snapshot=run_input.session_snapshot,
|
||||
deferred_tool_results=run_input.deferred_tool_results,
|
||||
on_exit=LayerExitSignals(
|
||||
default=ExitIntent.SUSPEND if run_input.suspend_on_exit else ExitIntent.DELETE,
|
||||
),
|
||||
|
||||
@@ -11,7 +11,6 @@ from .data_migration import (
|
||||
migration_data_wizard,
|
||||
)
|
||||
from .plugin import (
|
||||
backfill_plugin_auto_upgrade,
|
||||
extract_plugins,
|
||||
extract_unique_plugins,
|
||||
install_plugins,
|
||||
@@ -22,10 +21,8 @@ from .plugin import (
|
||||
setup_system_trigger_oauth_client,
|
||||
transform_datasource_credentials,
|
||||
)
|
||||
from .rbac import migrate_dataset_permissions_to_rbac, migrate_member_roles_to_rbac
|
||||
from .retention import (
|
||||
archive_workflow_runs,
|
||||
archive_workflow_runs_plan,
|
||||
clean_expired_messages,
|
||||
clean_workflow_runs,
|
||||
cleanup_orphaned_draft_variables,
|
||||
@@ -52,8 +49,6 @@ from .vector import (
|
||||
__all__ = [
|
||||
"add_qdrant_index",
|
||||
"archive_workflow_runs",
|
||||
"archive_workflow_runs_plan",
|
||||
"backfill_plugin_auto_upgrade",
|
||||
"clean_expired_messages",
|
||||
"clean_workflow_runs",
|
||||
"cleanup_orphaned_draft_variables",
|
||||
@@ -76,9 +71,7 @@ __all__ = [
|
||||
"legacy_model_types",
|
||||
"migrate_annotation_vector_database",
|
||||
"migrate_data_for_plugin",
|
||||
"migrate_dataset_permissions_to_rbac",
|
||||
"migrate_knowledge_vector_database",
|
||||
"migrate_member_roles_to_rbac",
|
||||
"migrate_oss",
|
||||
"migration_data_wizard",
|
||||
"old_metadata_migration",
|
||||
|
||||
@@ -25,7 +25,7 @@ def reset_password(email, new_password, password_confirm):
|
||||
return
|
||||
normalized_email = email.strip().lower()
|
||||
|
||||
account = AccountService.get_account_by_email_with_case_fallback(db.session, email.strip())
|
||||
account = AccountService.get_account_by_email_with_case_fallback(email.strip())
|
||||
|
||||
if not account:
|
||||
click.echo(click.style(f"Account not found for email: {email}", fg="red"))
|
||||
@@ -67,7 +67,7 @@ def reset_email(email, new_email, email_confirm):
|
||||
return
|
||||
normalized_new_email = new_email.strip().lower()
|
||||
|
||||
account = AccountService.get_account_by_email_with_case_fallback(db.session, email.strip())
|
||||
account = AccountService.get_account_by_email_with_case_fallback(email.strip())
|
||||
|
||||
if not account:
|
||||
click.echo(click.style(f"Account not found for email: {email}", fg="red"))
|
||||
@@ -133,9 +133,8 @@ def create_tenant(email: str, language: str | None = None, name: str | None = No
|
||||
password=new_password,
|
||||
language=language,
|
||||
create_workspace_required=False,
|
||||
session=db.session,
|
||||
)
|
||||
TenantService.create_owner_tenant_if_not_exist(account, name, session=db.session)
|
||||
TenantService.create_owner_tenant_if_not_exist(account, name)
|
||||
|
||||
click.echo(
|
||||
click.style(
|
||||
|
||||
@@ -7,7 +7,6 @@ from typing import cast
|
||||
|
||||
import click
|
||||
|
||||
from commands.rbac import migrate_dataset_permissions_to_rbac
|
||||
from extensions.ext_database import db
|
||||
from graphon.model_runtime.entities.model_entities import ModelType
|
||||
from services.legacy_model_type_migration import (
|
||||
@@ -178,4 +177,3 @@ def legacy_model_types(
|
||||
|
||||
|
||||
data_migrate.add_command(legacy_model_types)
|
||||
data_migrate.add_command(migrate_dataset_permissions_to_rbac)
|
||||
|
||||
@@ -10,7 +10,6 @@ import click
|
||||
import sqlalchemy as sa
|
||||
import yaml
|
||||
|
||||
from core.db.session_factory import session_factory
|
||||
from extensions.ext_database import db
|
||||
from models import Tenant
|
||||
from models.model import App
|
||||
@@ -107,8 +106,7 @@ def export_migration_data(input_file: str | None, output_file: str | None, overw
|
||||
assert output_file is not None
|
||||
raw_config = _load_json_object(input_file, "Export config")
|
||||
selection = ExportConfigParser().parse(raw_config)
|
||||
with session_factory.create_session() as session:
|
||||
result = MigrationExportService().export(session, selection)
|
||||
result = MigrationExportService().export(selection)
|
||||
MigrationPackageService().save_package(result.package, output_file, overwrite=overwrite)
|
||||
click.echo(click.style(f"Output written to {output_file}", fg="green"))
|
||||
_render_report(result.report_items, context=_with_output_path(result.report_context, output_file))
|
||||
@@ -155,21 +153,19 @@ def import_migration_data(
|
||||
_require_options(("--input", input_file))
|
||||
assert input_file is not None
|
||||
package = MigrationPackageService().load_package(input_file)
|
||||
with session_factory.create_session() as session:
|
||||
result = MigrationImportService().import_package(
|
||||
session,
|
||||
ImportRequest(
|
||||
package=package,
|
||||
cli_target_tenant=target_tenant,
|
||||
operator_email=operator_email,
|
||||
options_override=_build_options_override(
|
||||
package.metadata.import_options,
|
||||
id_strategy=id_strategy,
|
||||
conflict_strategy=conflict_strategy,
|
||||
create_app_api_token_on_import=create_app_api_token_on_import,
|
||||
),
|
||||
result = MigrationImportService().import_package(
|
||||
ImportRequest(
|
||||
package=package,
|
||||
cli_target_tenant=target_tenant,
|
||||
operator_email=operator_email,
|
||||
options_override=_build_options_override(
|
||||
package.metadata.import_options,
|
||||
id_strategy=id_strategy,
|
||||
conflict_strategy=conflict_strategy,
|
||||
create_app_api_token_on_import=create_app_api_token_on_import,
|
||||
),
|
||||
)
|
||||
)
|
||||
_render_report(result.report_items, context=result.report_context)
|
||||
except MigrationDataError as exc:
|
||||
raise click.ClickException(str(exc)) from exc
|
||||
@@ -252,8 +248,7 @@ def migration_data_wizard() -> None:
|
||||
conflict_strategy=conflict_strategy,
|
||||
output_file=output_file,
|
||||
)
|
||||
with session_factory.create_session() as session:
|
||||
result = MigrationExportService().export(session, selection)
|
||||
result = MigrationExportService().export(selection)
|
||||
MigrationPackageService().save_package(result.package, output_file, overwrite=overwrite)
|
||||
click.echo(click.style(f"Output written to {output_file}", fg="green"))
|
||||
_print_wizard_step("Report")
|
||||
|
||||
+1
-108
@@ -1,11 +1,10 @@
|
||||
import json
|
||||
import logging
|
||||
import time
|
||||
from typing import Any, cast
|
||||
|
||||
import click
|
||||
from pydantic import TypeAdapter
|
||||
from sqlalchemy import delete, func, select
|
||||
from sqlalchemy import delete, select
|
||||
from sqlalchemy.engine import CursorResult
|
||||
|
||||
from configs import dify_config
|
||||
@@ -16,13 +15,11 @@ from core.plugin.plugin_service import PluginService
|
||||
from core.tools.utils.system_encryption import encrypt_system_params
|
||||
from extensions.ext_database import db
|
||||
from models import Tenant
|
||||
from models.account import TenantPluginAutoUpgradeStrategy
|
||||
from models.oauth import DatasourceOauthParamConfig, DatasourceProvider
|
||||
from models.provider_ids import DatasourceProviderID, ToolProviderID
|
||||
from models.source import DataSourceApiKeyAuthBinding, DataSourceOauthBinding
|
||||
from models.tools import ToolOAuthSystemClient
|
||||
from services.plugin.data_migration import PluginDataMigration
|
||||
from services.plugin.plugin_auto_upgrade_service import PluginAutoUpgradeService
|
||||
from services.plugin.plugin_migration import PluginMigration
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
@@ -405,110 +402,6 @@ def migrate_data_for_plugin():
|
||||
click.echo(click.style("Migrate data for plugin completed.", fg="green"))
|
||||
|
||||
|
||||
def _candidate_auto_upgrade_strategy_tenant_ids_stmt(limit: int | None = None):
|
||||
category_count = len(TenantPluginAutoUpgradeStrategy.PluginCategory)
|
||||
stmt = (
|
||||
select(TenantPluginAutoUpgradeStrategy.tenant_id)
|
||||
.group_by(TenantPluginAutoUpgradeStrategy.tenant_id)
|
||||
.having(func.count(func.distinct(TenantPluginAutoUpgradeStrategy.category)) < category_count)
|
||||
.order_by(TenantPluginAutoUpgradeStrategy.tenant_id)
|
||||
)
|
||||
|
||||
if limit is not None:
|
||||
stmt = stmt.limit(limit)
|
||||
|
||||
return stmt
|
||||
|
||||
|
||||
def _count_auto_upgrade_strategy_tenant_ids(limit: int | None) -> int:
|
||||
candidate_stmt = _candidate_auto_upgrade_strategy_tenant_ids_stmt(limit).subquery()
|
||||
return db.session.scalar(select(func.count()).select_from(candidate_stmt)) or 0
|
||||
|
||||
|
||||
def _iter_auto_upgrade_strategy_tenant_ids(limit: int | None):
|
||||
stmt = _candidate_auto_upgrade_strategy_tenant_ids_stmt(limit).execution_options(yield_per=1000)
|
||||
yield from db.session.scalars(stmt)
|
||||
|
||||
|
||||
@click.command(
|
||||
"backfill-plugin-auto-upgrade",
|
||||
help="Backfill category-scoped plugin auto-upgrade strategies and normalize plugin lists.",
|
||||
)
|
||||
@click.option("--tenant-id", multiple=True, help="Tenant ID to backfill. Can be passed multiple times.")
|
||||
@click.option("--limit", type=int, default=None, help="Maximum number of candidate tenants to process.")
|
||||
@click.option("--batch-size", type=int, default=500, show_default=True, help="Progress reporting batch size.")
|
||||
@click.option("--dry-run", is_flag=True, help="Only print candidate tenant count.")
|
||||
def backfill_plugin_auto_upgrade(
|
||||
tenant_id: tuple[str, ...],
|
||||
limit: int | None,
|
||||
batch_size: int,
|
||||
dry_run: bool,
|
||||
):
|
||||
"""
|
||||
Backfill historical auto-upgrade strategies after the category column exists.
|
||||
|
||||
Missing category rows are created from the tenant's tool/default row. Pure default
|
||||
strategies become latest for model plugins and fix-only for all other categories.
|
||||
Tenants with include/exclude plugin IDs are split
|
||||
by installed plugin category using plugin daemon metadata.
|
||||
"""
|
||||
start_at = time.perf_counter()
|
||||
candidate_count = len(tenant_id) if tenant_id else _count_auto_upgrade_strategy_tenant_ids(limit)
|
||||
click.echo(click.style(f"Found {candidate_count} candidate tenants.", fg="yellow"))
|
||||
|
||||
if dry_run:
|
||||
elapsed = time.perf_counter() - start_at
|
||||
click.echo(click.style(f"Dry run completed. elapsed={elapsed:.2f}s", fg="green"))
|
||||
return
|
||||
|
||||
tenant_ids = list(tenant_id) if tenant_id else _iter_auto_upgrade_strategy_tenant_ids(limit)
|
||||
|
||||
backfilled_count = 0
|
||||
created_count = 0
|
||||
normalized_count = 0
|
||||
skipped_count = 0
|
||||
failed_count = 0
|
||||
for index, current_tenant_id in enumerate(tenant_ids, start=1):
|
||||
try:
|
||||
result = PluginAutoUpgradeService.backfill_strategy_categories(
|
||||
current_tenant_id,
|
||||
)
|
||||
except Exception as e:
|
||||
failed_count += 1
|
||||
click.echo(click.style(f"Failed tenant {current_tenant_id}: {str(e)}", fg="red"))
|
||||
continue
|
||||
|
||||
if result.created_count > 0:
|
||||
backfilled_count += 1
|
||||
created_count += result.created_count
|
||||
elif not result.normalized:
|
||||
skipped_count += 1
|
||||
if result.normalized:
|
||||
normalized_count += 1
|
||||
|
||||
if batch_size > 0 and index % batch_size == 0:
|
||||
click.echo(
|
||||
click.style(
|
||||
f"Processed {index}/{candidate_count} tenants. "
|
||||
f"backfilled={backfilled_count}, created_rows={created_count}, "
|
||||
f"normalized={normalized_count}, skipped={skipped_count}, failed={failed_count}, "
|
||||
f"elapsed={time.perf_counter() - start_at:.2f}s",
|
||||
fg="yellow",
|
||||
)
|
||||
)
|
||||
|
||||
elapsed = time.perf_counter() - start_at
|
||||
click.echo(
|
||||
click.style(
|
||||
f"Backfill plugin auto-upgrade strategy categories completed. "
|
||||
f"backfilled={backfilled_count}, created_rows={created_count}, "
|
||||
f"normalized={normalized_count}, skipped={skipped_count}, failed={failed_count}, "
|
||||
f"elapsed={elapsed:.2f}s",
|
||||
fg="green",
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
@click.command("extract-plugins", help="Extract plugins.")
|
||||
@click.option("--output_file", prompt=True, help="The file to store the extracted plugins.", default="plugins.jsonl")
|
||||
@click.option("--workers", prompt=True, help="The number of workers to extract plugins.", default=10)
|
||||
|
||||
@@ -1,484 +0,0 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
from collections.abc import Iterator
|
||||
from concurrent.futures import ThreadPoolExecutor, as_completed
|
||||
|
||||
import click
|
||||
from sqlalchemy import select
|
||||
|
||||
from configs import dify_config
|
||||
from core.db.session_factory import session_factory
|
||||
from core.rbac import RBACResourceWhitelistScope
|
||||
from models import Dataset, DatasetPermission, DatasetPermissionEnum, TenantAccountJoin, TenantAccountRole
|
||||
from services.enterprise.rbac_service import ListOption, RBACService, ReplaceMemberBindings, ReplaceUserAccessPolicies
|
||||
|
||||
_RBAC_DEFAULT_ACCESS_POLICY_ID = "default"
|
||||
|
||||
_LEGACY_ROLE_TO_BUILTIN_TAG = {
|
||||
TenantAccountRole.OWNER.value: "owner",
|
||||
TenantAccountRole.ADMIN.value: "admin",
|
||||
TenantAccountRole.EDITOR.value: "editor",
|
||||
TenantAccountRole.NORMAL.value: "normal",
|
||||
TenantAccountRole.DATASET_OPERATOR.value: "dataset_operator",
|
||||
}
|
||||
|
||||
|
||||
def _resolve_builtin_role_ids(tenant_id: str, operator_account_id: str) -> dict[str, str]:
|
||||
"""Resolve every legacy workspace role to the current tenant's builtin RBAC role id.
|
||||
|
||||
The migration replays the old `TenantAccountJoin.role` values onto the
|
||||
RBAC member-role binding API. Builtin RBAC roles are tenant-scoped and
|
||||
identified by runtime ids, so the command must look them up per tenant.
|
||||
"""
|
||||
roles = RBACService.Roles.list(
|
||||
tenant_id=tenant_id,
|
||||
account_id=operator_account_id,
|
||||
options=ListOption(page_number=1, results_per_page=100),
|
||||
).data
|
||||
role_id_by_tag = {
|
||||
role.role_tag: role.id
|
||||
for role in roles
|
||||
if role.is_builtin and role.category == "global_system_default" and role.role_tag
|
||||
}
|
||||
resolved: dict[str, str] = {}
|
||||
for legacy_role, expected_builtin_tag in _LEGACY_ROLE_TO_BUILTIN_TAG.items():
|
||||
role_id = role_id_by_tag.get(expected_builtin_tag)
|
||||
if expected_builtin_tag == "dataset_operator" and not dify_config.DATASET_OPERATOR_ENABLED:
|
||||
continue
|
||||
if not role_id:
|
||||
raise ValueError(f"Builtin RBAC role not found for tenant={tenant_id}, legacy_role={legacy_role}")
|
||||
resolved[legacy_role] = role_id
|
||||
return resolved
|
||||
|
||||
|
||||
def _resolve_builtin_role_id(tenant_id: str, operator_account_id: str, legacy_role: str) -> str:
|
||||
"""Resolve a legacy workspace role to the current tenant's builtin RBAC role id.
|
||||
|
||||
The migration replays the old `TenantAccountJoin.role` values onto the
|
||||
RBAC member-role binding API. Builtin RBAC roles are tenant-scoped and
|
||||
identified by runtime ids, so the command must look them up per tenant.
|
||||
"""
|
||||
if legacy_role not in _LEGACY_ROLE_TO_BUILTIN_TAG:
|
||||
raise ValueError(f"Unsupported legacy workspace role: {legacy_role}")
|
||||
|
||||
return _resolve_builtin_role_ids(tenant_id, operator_account_id)[legacy_role]
|
||||
|
||||
|
||||
def _iter_tenant_member_batches(
|
||||
tenant_id: str | None,
|
||||
*,
|
||||
db_batch_size: int,
|
||||
api_batch_size: int,
|
||||
) -> Iterator[tuple[str, str, list[tuple[str, str]]]]:
|
||||
"""Yield legacy member roles in tenant-scoped API-sized batches.
|
||||
|
||||
Rows are projected to primitive values and streamed from the database, so
|
||||
the command never materializes every TenantAccountJoin ORM object. The
|
||||
iterator only keeps one tenant's API-sized batches in memory while it
|
||||
finds that tenant's owner account.
|
||||
"""
|
||||
with session_factory.create_session() as session:
|
||||
stmt = (
|
||||
select(TenantAccountJoin.tenant_id, TenantAccountJoin.account_id, TenantAccountJoin.role)
|
||||
.order_by(TenantAccountJoin.tenant_id.asc(), TenantAccountJoin.id.asc())
|
||||
.execution_options(yield_per=db_batch_size)
|
||||
)
|
||||
if tenant_id:
|
||||
stmt = stmt.where(TenantAccountJoin.tenant_id == tenant_id)
|
||||
|
||||
current_tenant_id: str | None = None
|
||||
owner_account_id: str | None = None
|
||||
batches: list[list[tuple[str, str]]] = []
|
||||
batch: list[tuple[str, str]] = []
|
||||
|
||||
def flush_current_tenant() -> Iterator[tuple[str, str, list[tuple[str, str]]]]:
|
||||
if current_tenant_id is None:
|
||||
return
|
||||
if batch:
|
||||
batches.append(batch.copy())
|
||||
if not owner_account_id:
|
||||
raise ValueError(f"Workspace owner not found for tenant={current_tenant_id}")
|
||||
for item in batches:
|
||||
yield current_tenant_id, owner_account_id, item
|
||||
|
||||
for row in session.execute(stmt):
|
||||
workspace_id = str(row.tenant_id)
|
||||
if current_tenant_id is not None and workspace_id != current_tenant_id:
|
||||
yield from flush_current_tenant()
|
||||
owner_account_id = None
|
||||
batches = []
|
||||
batch = []
|
||||
current_tenant_id = workspace_id
|
||||
account_id = str(row.account_id)
|
||||
role = str(row.role)
|
||||
if role == TenantAccountRole.OWNER.value:
|
||||
owner_account_id = account_id
|
||||
batch.append((account_id, role))
|
||||
if len(batch) >= api_batch_size:
|
||||
batches.append(batch)
|
||||
batch = []
|
||||
|
||||
yield from flush_current_tenant()
|
||||
|
||||
|
||||
def _member_already_has_role(current_roles_by_account_id: dict[str, set[str]], account_id: str, role_id: str) -> bool:
|
||||
return current_roles_by_account_id.get(account_id) == {role_id}
|
||||
|
||||
|
||||
def _replace_member_role(
|
||||
tenant_id: str,
|
||||
operator_account_id: str,
|
||||
member_account_id: str,
|
||||
role_id: str,
|
||||
) -> str:
|
||||
RBACService.MemberRoles.replace(
|
||||
tenant_id=tenant_id,
|
||||
account_id=operator_account_id,
|
||||
member_account_id=member_account_id,
|
||||
role_ids=[role_id],
|
||||
)
|
||||
return member_account_id
|
||||
|
||||
|
||||
@click.command(
|
||||
"rbac-migrate-member-roles", help="Migrate legacy workspace member roles into RBAC member-role bindings."
|
||||
)
|
||||
@click.option("--tenant-id", help="Only migrate a single workspace.")
|
||||
@click.option("--dry-run", is_flag=True, default=False, help="Preview the migration without writing RBAC bindings.")
|
||||
@click.option("--db-batch-size", default=5000, show_default=True, help="Rows fetched per database batch.")
|
||||
@click.option("--api-batch-size", default=200, show_default=True, help="Members checked per RBAC batch_get call.")
|
||||
@click.option("--workers", default=1, show_default=True, help="Concurrent member role replace calls per tenant batch.")
|
||||
def migrate_member_roles_to_rbac(
|
||||
tenant_id: str | None,
|
||||
dry_run: bool,
|
||||
db_batch_size: int,
|
||||
api_batch_size: int,
|
||||
workers: int,
|
||||
) -> None:
|
||||
"""Backfill RBAC member-role bindings from legacy `TenantAccountJoin.role` data.
|
||||
|
||||
This is an offline migration command for workspaces that already have
|
||||
members in the legacy role model but need matching records in the RBAC
|
||||
member-role binding store.
|
||||
"""
|
||||
click.echo(click.style("Starting RBAC member-role migration.", fg="green"))
|
||||
if workers < 1:
|
||||
raise click.BadParameter("workers must be >= 1", param_hint="--workers")
|
||||
|
||||
tenant_count = 0
|
||||
scanned_count = 0
|
||||
skipped_count = 0
|
||||
migrated_count = 0
|
||||
current_tenant_id: str | None = None
|
||||
role_ids_by_legacy_role: dict[str, str] = {}
|
||||
|
||||
for workspace_id, owner_account_id, batch in _iter_tenant_member_batches(
|
||||
tenant_id,
|
||||
db_batch_size=db_batch_size,
|
||||
api_batch_size=api_batch_size,
|
||||
):
|
||||
scanned_count += len(batch)
|
||||
if workspace_id != current_tenant_id:
|
||||
tenant_count += 1
|
||||
current_tenant_id = workspace_id
|
||||
role_ids_by_legacy_role = _resolve_builtin_role_ids(workspace_id, owner_account_id)
|
||||
click.echo(f"tenant={workspace_id}")
|
||||
|
||||
current_roles_by_account_id: dict[str, set[str]] = {}
|
||||
if not dry_run:
|
||||
current_roles = RBACService.MemberRoles.batch_get(
|
||||
tenant_id=workspace_id,
|
||||
account_id=owner_account_id,
|
||||
member_account_ids=[account_id for account_id, _ in batch],
|
||||
)
|
||||
current_roles_by_account_id = {
|
||||
item.account_id: {str(role.id) for role in item.roles} for item in current_roles
|
||||
}
|
||||
|
||||
replace_jobs: list[tuple[str, str]] = []
|
||||
for member_account_id, legacy_role in batch:
|
||||
resolved_role_id = role_ids_by_legacy_role.get(legacy_role)
|
||||
if not resolved_role_id:
|
||||
raise ValueError(f"Unsupported legacy workspace role: {legacy_role}")
|
||||
|
||||
if dry_run:
|
||||
click.echo(
|
||||
f"tenant={workspace_id} member={member_account_id} "
|
||||
f"legacy_role={legacy_role} -> rbac_role_id={resolved_role_id}"
|
||||
)
|
||||
continue
|
||||
|
||||
if _member_already_has_role(current_roles_by_account_id, member_account_id, resolved_role_id):
|
||||
skipped_count += 1
|
||||
continue
|
||||
|
||||
replace_jobs.append((member_account_id, resolved_role_id))
|
||||
|
||||
if replace_jobs:
|
||||
if workers == 1:
|
||||
for member_account_id, resolved_role_id in replace_jobs:
|
||||
_replace_member_role(workspace_id, owner_account_id, member_account_id, resolved_role_id)
|
||||
migrated_count += 1
|
||||
else:
|
||||
with ThreadPoolExecutor(max_workers=workers) as executor:
|
||||
futures = [
|
||||
executor.submit(
|
||||
_replace_member_role,
|
||||
workspace_id,
|
||||
owner_account_id,
|
||||
member_account_id,
|
||||
resolved_role_id,
|
||||
)
|
||||
for member_account_id, resolved_role_id in replace_jobs
|
||||
]
|
||||
for future in as_completed(futures):
|
||||
future.result()
|
||||
migrated_count += 1
|
||||
|
||||
if scanned_count % 10000 == 0:
|
||||
click.echo(
|
||||
f"progress scanned={scanned_count} migrated={migrated_count} skipped={skipped_count}",
|
||||
err=True,
|
||||
)
|
||||
|
||||
if scanned_count == 0:
|
||||
click.echo(click.style("No workspace members found for migration.", fg="yellow"))
|
||||
return
|
||||
|
||||
if dry_run:
|
||||
click.echo(
|
||||
click.style(
|
||||
f"Dry run completed. Scanned {scanned_count} members across {tenant_count} tenants. "
|
||||
"No RBAC bindings were written.",
|
||||
fg="yellow",
|
||||
)
|
||||
)
|
||||
else:
|
||||
click.echo(
|
||||
click.style(
|
||||
f"RBAC member-role migration completed. Scanned {scanned_count} members across {tenant_count} tenants, "
|
||||
f"migrated {migrated_count}, skipped {skipped_count} already up-to-date.",
|
||||
fg="green",
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
def _dataset_permission_enum(permission: DatasetPermissionEnum | str | None) -> DatasetPermissionEnum:
|
||||
if permission is None:
|
||||
return DatasetPermissionEnum.ONLY_ME
|
||||
try:
|
||||
return DatasetPermissionEnum(permission)
|
||||
except ValueError as exc:
|
||||
raise ValueError(f"Unsupported legacy dataset permission: {permission}") from exc
|
||||
|
||||
|
||||
def _rbac_dataset_scope_for_legacy_permission(permission: DatasetPermissionEnum) -> RBACResourceWhitelistScope:
|
||||
if permission is DatasetPermissionEnum.ALL_TEAM:
|
||||
return RBACResourceWhitelistScope.ALL
|
||||
if permission in {DatasetPermissionEnum.ONLY_ME, DatasetPermissionEnum.PARTIAL_TEAM}:
|
||||
return RBACResourceWhitelistScope.SPECIFIC
|
||||
raise ValueError(f"Unsupported legacy dataset permission: {permission}")
|
||||
|
||||
|
||||
def _emit_dataset_permission_migration_event(payload: dict[str, object]) -> None:
|
||||
click.echo(json.dumps(payload, sort_keys=True))
|
||||
|
||||
|
||||
@click.command(
|
||||
"rbac-migrate-dataset-permissions",
|
||||
help=(
|
||||
"Migrate legacy dataset permission scopes and partial members into RBAC dataset access bindings. "
|
||||
"Side effect: replacing each dataset whitelist clears existing per-user policy bindings; "
|
||||
"the command then recreates legacy partial-member default bindings."
|
||||
),
|
||||
)
|
||||
@click.option("--tenant-id", help="Only migrate datasets in a single workspace.")
|
||||
@click.option("--dataset-id", help="Only migrate a single dataset.")
|
||||
@click.option("--batch-size", default=500, show_default=True, type=click.IntRange(min=1))
|
||||
@click.option(
|
||||
"--dry-run/--apply",
|
||||
default=True,
|
||||
show_default=True,
|
||||
help="Preview the migration without writing RBAC bindings. Use --apply to write changes.",
|
||||
)
|
||||
def migrate_dataset_permissions_to_rbac(
|
||||
tenant_id: str | None,
|
||||
dataset_id: str | None,
|
||||
batch_size: int,
|
||||
dry_run: bool,
|
||||
) -> None:
|
||||
"""Backfill RBAC dataset access config from legacy `Dataset.permission`.
|
||||
|
||||
Legacy mapping:
|
||||
- all_team_members -> RBAC dataset whitelist scope "all"
|
||||
- partial_members -> RBAC dataset whitelist scope "specific" plus each partial member gets the
|
||||
virtual default policy
|
||||
- only_me -> RBAC dataset whitelist scope "specific" with no member policy bindings
|
||||
|
||||
The command replaces each dataset's RBAC whitelist scope first. RBAC clears
|
||||
existing per-user policy bindings during that replace, then this command
|
||||
recreates the legacy partial-member default bindings. Re-running it is
|
||||
therefore idempotent for a dataset's current legacy configuration.
|
||||
"""
|
||||
click.echo(click.style("Starting RBAC dataset permission migration.", fg="green"))
|
||||
|
||||
scanned_count = 0
|
||||
scope_migrated_count = 0
|
||||
user_policy_migrated_count = 0
|
||||
partial_dataset_count = 0
|
||||
|
||||
last_dataset_id: str | None = None
|
||||
while True:
|
||||
with session_factory.create_session() as session:
|
||||
stmt = (
|
||||
select(Dataset.id, Dataset.tenant_id, Dataset.permission, Dataset.created_by)
|
||||
.order_by(Dataset.id.asc())
|
||||
.limit(batch_size)
|
||||
)
|
||||
if tenant_id:
|
||||
stmt = stmt.where(Dataset.tenant_id == tenant_id)
|
||||
if dataset_id:
|
||||
stmt = stmt.where(Dataset.id == dataset_id)
|
||||
if last_dataset_id:
|
||||
stmt = stmt.where(Dataset.id > last_dataset_id)
|
||||
|
||||
dataset_rows = list(session.execute(stmt).all())
|
||||
if not dataset_rows:
|
||||
break
|
||||
|
||||
dataset_ids = [str(row.id) for row in dataset_rows]
|
||||
partial_members_by_dataset_id: dict[str, list[str]] = {item: [] for item in dataset_ids}
|
||||
permission_rows = session.execute(
|
||||
select(DatasetPermission.dataset_id, DatasetPermission.account_id).where(
|
||||
DatasetPermission.dataset_id.in_(dataset_ids)
|
||||
)
|
||||
).all()
|
||||
for row in permission_rows:
|
||||
partial_members_by_dataset_id[str(row.dataset_id)].append(str(row.account_id))
|
||||
|
||||
for dataset in dataset_rows:
|
||||
workspace_id = str(dataset.tenant_id)
|
||||
current_dataset_id = str(dataset.id)
|
||||
operator_account_id = str(dataset.created_by)
|
||||
permission_value = _dataset_permission_enum(dataset.permission)
|
||||
scope = _rbac_dataset_scope_for_legacy_permission(permission_value)
|
||||
partial_member_ids = sorted(set(partial_members_by_dataset_id[current_dataset_id]))
|
||||
should_bind_partial_members = permission_value is DatasetPermissionEnum.PARTIAL_TEAM
|
||||
|
||||
click.echo(
|
||||
f"tenant={workspace_id} dataset={current_dataset_id} "
|
||||
f"operator={operator_account_id} "
|
||||
f"legacy_permission={permission_value} -> rbac_scope={scope} "
|
||||
f"partial_members={len(partial_member_ids) if should_bind_partial_members else 0}"
|
||||
)
|
||||
|
||||
scanned_count += 1
|
||||
replace_whitelist_payload = ReplaceMemberBindings(scope=scope)
|
||||
if dry_run:
|
||||
_emit_dataset_permission_migration_event(
|
||||
{
|
||||
"event": "dataset_permission_migration_proposed_change",
|
||||
"action": "replace_whitelist",
|
||||
"dry_run": True,
|
||||
"tenant_id": workspace_id,
|
||||
"dataset_id": current_dataset_id,
|
||||
"operator_account_id": operator_account_id,
|
||||
"before": {
|
||||
"legacy_dataset_permission": permission_value.value,
|
||||
"legacy_partial_member_ids": partial_member_ids if should_bind_partial_members else [],
|
||||
},
|
||||
"after": {
|
||||
"rbac_whitelist_scope": scope.value,
|
||||
},
|
||||
"call": {
|
||||
"method": "RBACService.DatasetAccess.replace_whitelist",
|
||||
"kwargs": {
|
||||
"tenant_id": workspace_id,
|
||||
"account_id": operator_account_id,
|
||||
"dataset_id": current_dataset_id,
|
||||
"payload": replace_whitelist_payload.model_dump(mode="json"),
|
||||
},
|
||||
},
|
||||
}
|
||||
)
|
||||
if not dry_run:
|
||||
RBACService.DatasetAccess.replace_whitelist(
|
||||
tenant_id=workspace_id,
|
||||
account_id=operator_account_id,
|
||||
dataset_id=current_dataset_id,
|
||||
payload=replace_whitelist_payload,
|
||||
)
|
||||
scope_migrated_count += 1
|
||||
|
||||
if should_bind_partial_members:
|
||||
partial_dataset_count += 1
|
||||
for member_account_id in partial_member_ids:
|
||||
replace_user_access_policies_payload = ReplaceUserAccessPolicies(
|
||||
access_policy_ids=[_RBAC_DEFAULT_ACCESS_POLICY_ID],
|
||||
)
|
||||
if dry_run:
|
||||
_emit_dataset_permission_migration_event(
|
||||
{
|
||||
"event": "dataset_permission_migration_proposed_change",
|
||||
"action": "replace_user_access_policies",
|
||||
"dry_run": True,
|
||||
"tenant_id": workspace_id,
|
||||
"dataset_id": current_dataset_id,
|
||||
"operator_account_id": operator_account_id,
|
||||
"target_account_id": member_account_id,
|
||||
"before": {
|
||||
"legacy_dataset_permission": permission_value.value,
|
||||
"legacy_partial_member_id": member_account_id,
|
||||
},
|
||||
"after": {
|
||||
"rbac_user_access_policy_ids": [_RBAC_DEFAULT_ACCESS_POLICY_ID],
|
||||
},
|
||||
"call": {
|
||||
"method": "RBACService.DatasetAccess.replace_user_access_policies",
|
||||
"kwargs": {
|
||||
"tenant_id": workspace_id,
|
||||
"account_id": operator_account_id,
|
||||
"dataset_id": current_dataset_id,
|
||||
"target_account_id": member_account_id,
|
||||
"payload": replace_user_access_policies_payload.model_dump(mode="json"),
|
||||
},
|
||||
},
|
||||
}
|
||||
)
|
||||
continue
|
||||
RBACService.DatasetAccess.replace_user_access_policies(
|
||||
tenant_id=workspace_id,
|
||||
account_id=operator_account_id,
|
||||
dataset_id=current_dataset_id,
|
||||
target_account_id=member_account_id,
|
||||
payload=replace_user_access_policies_payload,
|
||||
)
|
||||
user_policy_migrated_count += 1
|
||||
|
||||
last_dataset_id = dataset_ids[-1]
|
||||
|
||||
if dataset_id:
|
||||
break
|
||||
|
||||
if scanned_count == 0:
|
||||
click.echo(click.style("No datasets found for migration.", fg="yellow"))
|
||||
return
|
||||
|
||||
if dry_run:
|
||||
click.echo(
|
||||
click.style(
|
||||
f"Dry run completed. Scanned {scanned_count} datasets; "
|
||||
f"{partial_dataset_count} partial-member datasets would be migrated.",
|
||||
fg="yellow",
|
||||
)
|
||||
)
|
||||
else:
|
||||
click.echo(
|
||||
click.style(
|
||||
"RBAC dataset permission migration completed. "
|
||||
f"Scanned {scanned_count} datasets, migrated {scope_migrated_count} scopes, "
|
||||
f"wrote {user_policy_migrated_count} user default-policy bindings.",
|
||||
fg="green",
|
||||
)
|
||||
)
|
||||
+95
-571
@@ -12,171 +12,10 @@ from services.clear_free_plan_tenant_expired_logs import ClearFreePlanTenantExpi
|
||||
from services.retention.conversation.messages_clean_policy import create_message_clean_policy
|
||||
from services.retention.conversation.messages_clean_service import MessagesCleanService
|
||||
from services.retention.workflow_run.clear_free_plan_expired_workflow_run_logs import WorkflowRunCleanup
|
||||
from services.retention.workflow_run.tenant_prefix import tenant_prefix_condition
|
||||
from tasks.remove_app_and_related_data_task import delete_draft_variables_batch
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
_HEX_PREFIXES = tuple("0123456789abcdef")
|
||||
|
||||
|
||||
class WorkflowRunArchivePlanRow(TypedDict):
|
||||
tenant_prefix: str
|
||||
total_tenants: int
|
||||
workflow_runs: int
|
||||
workflow_node_executions: int
|
||||
paid_tenants: int
|
||||
unpaid_tenants: int
|
||||
|
||||
|
||||
class WorkflowRunArchiveTenantPlan(TypedDict):
|
||||
archive_tenant_ids: list[str] | None
|
||||
paid_tenant_ids: list[str]
|
||||
unpaid_tenant_ids: list[str]
|
||||
|
||||
|
||||
def _normalize_utc_datetime(value: datetime.datetime) -> datetime.datetime:
|
||||
if value.tzinfo is None:
|
||||
return value.replace(tzinfo=datetime.UTC)
|
||||
return value.astimezone(datetime.UTC)
|
||||
|
||||
|
||||
def _parse_tenant_prefixes(prefixes: str | None) -> list[str]:
|
||||
if not prefixes:
|
||||
return []
|
||||
|
||||
parsed = []
|
||||
for raw_prefix in prefixes.split(","):
|
||||
prefix = raw_prefix.strip().lower()
|
||||
if not prefix:
|
||||
continue
|
||||
if len(prefix) != 1 or prefix not in _HEX_PREFIXES:
|
||||
raise click.UsageError("--tenant-prefixes must be a comma-separated list of hex digits, e.g. 0,1,a,f.")
|
||||
parsed.append(prefix)
|
||||
return sorted(set(parsed))
|
||||
|
||||
|
||||
def _get_archive_candidate_tenant_ids_by_prefix(
|
||||
prefix: str,
|
||||
*,
|
||||
start_from: datetime.datetime | None,
|
||||
end_before: datetime.datetime,
|
||||
) -> list[str]:
|
||||
from graphon.enums import WorkflowExecutionStatus
|
||||
from models.workflow import WorkflowRun
|
||||
from services.retention.workflow_run.archive_paid_plan_workflow_run import WorkflowRunArchiver
|
||||
|
||||
conditions = [
|
||||
WorkflowRun.created_at < end_before,
|
||||
WorkflowRun.status.in_(WorkflowExecutionStatus.ended_values()),
|
||||
WorkflowRun.type.in_(WorkflowRunArchiver.ARCHIVED_TYPE),
|
||||
tenant_prefix_condition(WorkflowRun.tenant_id, prefix),
|
||||
]
|
||||
if start_from is not None:
|
||||
conditions.append(WorkflowRun.created_at >= start_from)
|
||||
|
||||
tenant_ids = db.session.scalars(
|
||||
sa.select(WorkflowRun.tenant_id).where(*conditions).distinct().order_by(WorkflowRun.tenant_id)
|
||||
).all()
|
||||
return list(tenant_ids)
|
||||
|
||||
|
||||
def _filter_paid_workflow_archive_tenant_ids(tenant_ids: list[str]) -> tuple[list[str], list[str]]:
|
||||
from configs import dify_config
|
||||
from enums.cloud_plan import CloudPlan
|
||||
from services.billing_service import BillingService
|
||||
|
||||
tenant_ids = sorted(set(tenant_ids))
|
||||
if not tenant_ids:
|
||||
return [], []
|
||||
if not dify_config.BILLING_ENABLED:
|
||||
return tenant_ids, []
|
||||
|
||||
plans = BillingService.get_plan_bulk_with_cache(tenant_ids)
|
||||
paid_tenant_ids = [
|
||||
tenant_id
|
||||
for tenant_id in tenant_ids
|
||||
if plans.get(tenant_id) and plans[tenant_id].get("plan") in (CloudPlan.PROFESSIONAL, CloudPlan.TEAM)
|
||||
]
|
||||
unpaid_tenant_ids = sorted(set(tenant_ids) - set(paid_tenant_ids))
|
||||
return paid_tenant_ids, unpaid_tenant_ids
|
||||
|
||||
|
||||
def _resolve_archive_tenant_ids_from_plan(
|
||||
*,
|
||||
tenant_ids: str | None,
|
||||
tenant_prefixes: list[str],
|
||||
start_from: datetime.datetime | None,
|
||||
end_before: datetime.datetime,
|
||||
) -> WorkflowRunArchiveTenantPlan:
|
||||
"""
|
||||
Resolve the archive tenant scope once before scanning workflow_runs.
|
||||
|
||||
Prefix rollout should use the tenant list collected by the same planning path, then archive by
|
||||
tenant_id IN (...). Scanning workflow_runs with a tenant prefix range in every archive run is too expensive on
|
||||
the large production table this command is meant to shrink.
|
||||
"""
|
||||
if tenant_ids:
|
||||
requested_tenant_ids = [tid.strip() for tid in tenant_ids.split(",") if tid.strip()]
|
||||
elif tenant_prefixes:
|
||||
requested_tenant_ids = []
|
||||
for prefix in tenant_prefixes:
|
||||
requested_tenant_ids.extend(
|
||||
_get_archive_candidate_tenant_ids_by_prefix(
|
||||
prefix,
|
||||
start_from=start_from,
|
||||
end_before=end_before,
|
||||
)
|
||||
)
|
||||
else:
|
||||
return WorkflowRunArchiveTenantPlan(
|
||||
archive_tenant_ids=None,
|
||||
paid_tenant_ids=[],
|
||||
unpaid_tenant_ids=[],
|
||||
)
|
||||
|
||||
paid_tenant_ids, unpaid_tenant_ids = _filter_paid_workflow_archive_tenant_ids(requested_tenant_ids)
|
||||
return WorkflowRunArchiveTenantPlan(
|
||||
archive_tenant_ids=paid_tenant_ids,
|
||||
paid_tenant_ids=paid_tenant_ids,
|
||||
unpaid_tenant_ids=unpaid_tenant_ids,
|
||||
)
|
||||
|
||||
|
||||
def _resolve_archive_time_range(
|
||||
*,
|
||||
before_days: int,
|
||||
from_days_ago: int | None,
|
||||
to_days_ago: int | None,
|
||||
start_from: datetime.datetime | None,
|
||||
end_before: datetime.datetime | None,
|
||||
) -> tuple[int, datetime.datetime | None, datetime.datetime | None]:
|
||||
if (start_from is None) ^ (end_before is None):
|
||||
raise click.UsageError("--start-from and --end-before must be provided together.")
|
||||
|
||||
if (from_days_ago is None) ^ (to_days_ago is None):
|
||||
raise click.UsageError("--from-days-ago and --to-days-ago must be provided together.")
|
||||
|
||||
if from_days_ago is not None and to_days_ago is not None:
|
||||
if start_from or end_before:
|
||||
raise click.UsageError("Choose either day offsets or explicit dates, not both.")
|
||||
if from_days_ago <= to_days_ago:
|
||||
raise click.UsageError("--from-days-ago must be greater than --to-days-ago.")
|
||||
now = datetime.datetime.now(datetime.UTC)
|
||||
start_from = now - datetime.timedelta(days=from_days_ago)
|
||||
end_before = now - datetime.timedelta(days=to_days_ago)
|
||||
before_days = 0
|
||||
|
||||
if start_from is not None:
|
||||
start_from = _normalize_utc_datetime(start_from)
|
||||
if end_before is not None:
|
||||
end_before = _normalize_utc_datetime(end_before)
|
||||
|
||||
if start_from and end_before and start_from >= end_before:
|
||||
raise click.UsageError("--start-from must be earlier than --end-before.")
|
||||
|
||||
return before_days, start_from, end_before
|
||||
|
||||
|
||||
@click.command("clear-free-plan-tenant-expired-logs", help="Clear free plan tenant expired logs.")
|
||||
@click.option("--days", prompt=True, help="The days to clear free plan tenant expired logs.", default=30)
|
||||
@@ -300,150 +139,11 @@ def clean_workflow_runs(
|
||||
)
|
||||
|
||||
|
||||
@click.command(
|
||||
"archive-workflow-runs-plan",
|
||||
help="Plan workflow run archive rollout by tenant ID first hex digit.",
|
||||
)
|
||||
@click.option("--before-days", default=90, show_default=True, help="Plan runs older than N days.")
|
||||
@click.option(
|
||||
"--from-days-ago",
|
||||
default=None,
|
||||
type=click.IntRange(min=0),
|
||||
help="Lower bound in days ago (older). Must be paired with --to-days-ago.",
|
||||
)
|
||||
@click.option(
|
||||
"--to-days-ago",
|
||||
default=None,
|
||||
type=click.IntRange(min=0),
|
||||
help="Upper bound in days ago (newer). Must be paired with --from-days-ago.",
|
||||
)
|
||||
@click.option(
|
||||
"--start-from",
|
||||
type=click.DateTime(formats=["%Y-%m-%d", "%Y-%m-%dT%H:%M:%S"]),
|
||||
default=None,
|
||||
help="Plan runs created at or after this timestamp (UTC if no timezone).",
|
||||
)
|
||||
@click.option(
|
||||
"--end-before",
|
||||
type=click.DateTime(formats=["%Y-%m-%d", "%Y-%m-%dT%H:%M:%S"]),
|
||||
default=None,
|
||||
help="Plan runs created before this timestamp (UTC if no timezone).",
|
||||
)
|
||||
@click.option(
|
||||
"--include-archived",
|
||||
is_flag=True,
|
||||
help="Compatibility no-op for V2 bundle archive; plan counts source rows in the requested window.",
|
||||
)
|
||||
def archive_workflow_runs_plan(
|
||||
before_days: int,
|
||||
from_days_ago: int | None,
|
||||
to_days_ago: int | None,
|
||||
start_from: datetime.datetime | None,
|
||||
end_before: datetime.datetime | None,
|
||||
include_archived: bool,
|
||||
):
|
||||
"""
|
||||
Print the 16 tenant-prefix rollout rows used to choose archive execution order.
|
||||
|
||||
Counts use the same workflow run eligibility as archive-workflow-runs: ended runs,
|
||||
supported workflow types, and the requested created_at window. V2 bundle archive
|
||||
does not maintain per-run archive logs, so this plan reports source-table volume.
|
||||
"""
|
||||
from graphon.enums import WorkflowExecutionStatus
|
||||
from models.workflow import WorkflowNodeExecutionModel, WorkflowRun
|
||||
from services.retention.workflow_run.archive_paid_plan_workflow_run import WorkflowRunArchiver
|
||||
|
||||
before_days, start_from, end_before = _resolve_archive_time_range(
|
||||
before_days=before_days,
|
||||
from_days_ago=from_days_ago,
|
||||
to_days_ago=to_days_ago,
|
||||
start_from=start_from,
|
||||
end_before=end_before,
|
||||
)
|
||||
plan_end_before = end_before or datetime.datetime.now(datetime.UTC) - datetime.timedelta(days=before_days)
|
||||
if include_archived:
|
||||
click.echo(click.style("--include-archived is a no-op for V2 bundle archive plans.", fg="yellow"))
|
||||
|
||||
rows: list[WorkflowRunArchivePlanRow] = []
|
||||
for prefix in _HEX_PREFIXES:
|
||||
tenant_ids = _get_archive_candidate_tenant_ids_by_prefix(
|
||||
prefix,
|
||||
start_from=start_from,
|
||||
end_before=plan_end_before,
|
||||
)
|
||||
total_tenants = len(tenant_ids)
|
||||
paid_tenant_ids, unpaid_tenant_ids = _filter_paid_workflow_archive_tenant_ids(tenant_ids)
|
||||
|
||||
run_conditions = [
|
||||
WorkflowRun.created_at < plan_end_before,
|
||||
WorkflowRun.status.in_(WorkflowExecutionStatus.ended_values()),
|
||||
WorkflowRun.type.in_(WorkflowRunArchiver.ARCHIVED_TYPE),
|
||||
tenant_prefix_condition(WorkflowRun.tenant_id, prefix),
|
||||
]
|
||||
if start_from is not None:
|
||||
run_conditions.append(WorkflowRun.created_at >= start_from)
|
||||
workflow_runs = (
|
||||
db.session.scalar(sa.select(sa.func.count()).select_from(WorkflowRun).where(*run_conditions)) or 0
|
||||
)
|
||||
candidate_runs = sa.select(WorkflowRun.id).where(*run_conditions).subquery()
|
||||
workflow_node_executions = (
|
||||
db.session.scalar(
|
||||
sa.select(sa.func.count())
|
||||
.select_from(WorkflowNodeExecutionModel)
|
||||
.join(candidate_runs, WorkflowNodeExecutionModel.workflow_run_id == candidate_runs.c.id)
|
||||
)
|
||||
or 0
|
||||
)
|
||||
|
||||
rows.append(
|
||||
WorkflowRunArchivePlanRow(
|
||||
tenant_prefix=prefix,
|
||||
total_tenants=total_tenants,
|
||||
workflow_runs=workflow_runs,
|
||||
workflow_node_executions=workflow_node_executions,
|
||||
paid_tenants=len(paid_tenant_ids),
|
||||
unpaid_tenants=len(unpaid_tenant_ids),
|
||||
)
|
||||
)
|
||||
|
||||
click.echo(
|
||||
click.style(
|
||||
f"Workflow archive plan for runs before {plan_end_before.isoformat()}"
|
||||
f"{f' and at/after {start_from.isoformat()}' if start_from else ''}.",
|
||||
fg="white",
|
||||
)
|
||||
)
|
||||
click.echo(
|
||||
click.style(
|
||||
"fixed_archive_window="
|
||||
f"{start_from.isoformat() if start_from else 'unbounded'},{plan_end_before.isoformat()}",
|
||||
fg="white",
|
||||
)
|
||||
)
|
||||
click.echo("tenant_prefix,total_tenants,workflow_runs,workflow_node_executions,paid_tenants,unpaid_tenants")
|
||||
for row in rows:
|
||||
click.echo(
|
||||
f"{row['tenant_prefix']},{row['total_tenants']},{row['workflow_runs']},"
|
||||
f"{row['workflow_node_executions']},{row['paid_tenants']},{row['unpaid_tenants']}"
|
||||
)
|
||||
|
||||
ordered_rows = sorted(
|
||||
rows,
|
||||
key=lambda row: (row["workflow_runs"] + row["workflow_node_executions"], row["tenant_prefix"]),
|
||||
)
|
||||
click.echo("suggested_execution_order=" + ",".join(row["tenant_prefix"] for row in ordered_rows))
|
||||
|
||||
|
||||
@click.command(
|
||||
"archive-workflow-runs",
|
||||
help="Archive workflow runs for paid plan tenants to S3-compatible storage.",
|
||||
)
|
||||
@click.option("--tenant-ids", default=None, help="Optional comma-separated tenant IDs for grayscale rollout.")
|
||||
@click.option(
|
||||
"--tenant-prefixes",
|
||||
default=None,
|
||||
help="Optional comma-separated tenant ID first hex digits for rollout waves, e.g. 0,1,a,f.",
|
||||
)
|
||||
@click.option("--before-days", default=90, show_default=True, help="Archive runs older than N days.")
|
||||
@click.option(
|
||||
"--from-days-ago",
|
||||
@@ -469,36 +169,13 @@ def archive_workflow_runs_plan(
|
||||
default=None,
|
||||
help="Archive runs created before this timestamp (UTC if no timezone).",
|
||||
)
|
||||
@click.option("--batch-size", default=10000, show_default=True, help="Maximum workflow runs per archive bundle.")
|
||||
@click.option(
|
||||
"--workers",
|
||||
default=1,
|
||||
show_default=True,
|
||||
type=int,
|
||||
help="Reserved; bundle archive currently runs serially.",
|
||||
)
|
||||
@click.option(
|
||||
"--run-shard-index",
|
||||
default=None,
|
||||
type=click.IntRange(min=0),
|
||||
help="Zero-based workflow run shard index for parallel cron jobs. Must be paired with --run-shard-total.",
|
||||
)
|
||||
@click.option(
|
||||
"--run-shard-total",
|
||||
default=None,
|
||||
type=click.IntRange(min=1, max=16),
|
||||
help="Total workflow run shard count for parallel cron jobs. Must be paired with --run-shard-index.",
|
||||
)
|
||||
@click.option("--batch-size", default=100, show_default=True, help="Batch size for processing.")
|
||||
@click.option("--workers", default=1, show_default=True, type=int, help="Concurrent workflow runs to archive.")
|
||||
@click.option("--limit", default=None, type=int, help="Maximum number of runs to archive.")
|
||||
@click.option("--dry-run", is_flag=True, help="Preview without archiving.")
|
||||
@click.option(
|
||||
"--delete-after-archive",
|
||||
is_flag=True,
|
||||
help="Not supported by bundle archive; use a separate bundle delete workflow after validation.",
|
||||
)
|
||||
@click.option("--delete-after-archive", is_flag=True, help="Delete runs and related data after archiving.")
|
||||
def archive_workflow_runs(
|
||||
tenant_ids: str | None,
|
||||
tenant_prefixes: str | None,
|
||||
before_days: int,
|
||||
from_days_ago: int | None,
|
||||
to_days_ago: int | None,
|
||||
@@ -506,8 +183,6 @@ def archive_workflow_runs(
|
||||
end_before: datetime.datetime | None,
|
||||
batch_size: int,
|
||||
workers: int,
|
||||
run_shard_index: int | None,
|
||||
run_shard_total: int | None,
|
||||
limit: int | None,
|
||||
dry_run: bool,
|
||||
delete_after_archive: bool,
|
||||
@@ -515,19 +190,14 @@ def archive_workflow_runs(
|
||||
"""
|
||||
Archive workflow runs for paid plan tenants older than the specified days.
|
||||
|
||||
This command writes V2 tenant/month/shard archive bundles. Each bundle contains Parquet snapshots from:
|
||||
- workflow_runs
|
||||
- workflow_app_logs
|
||||
This command archives the following tables to storage:
|
||||
- workflow_node_executions
|
||||
- workflow_node_execution_offload
|
||||
- workflow_pauses
|
||||
- workflow_pause_reasons
|
||||
- workflow_trigger_logs
|
||||
|
||||
Source database rows are always preserved by archive. Deletion must be handled by
|
||||
a separate bundle-level delete workflow after manifest, checksum, row-count, and
|
||||
restore-sampling validation. In --dry-run mode, no storage or database writes
|
||||
happen; the command estimates per-table Parquet bytes and object size instead.
|
||||
The workflow_runs and workflow_app_logs tables are preserved for UI listing.
|
||||
"""
|
||||
from services.retention.workflow_run.archive_paid_plan_workflow_run import WorkflowRunArchiver
|
||||
|
||||
@@ -539,67 +209,32 @@ def archive_workflow_runs(
|
||||
)
|
||||
)
|
||||
|
||||
uses_relative_window = start_from is None and end_before is None
|
||||
try:
|
||||
before_days, start_from, end_before = _resolve_archive_time_range(
|
||||
before_days=before_days,
|
||||
from_days_ago=from_days_ago,
|
||||
to_days_ago=to_days_ago,
|
||||
start_from=start_from,
|
||||
end_before=end_before,
|
||||
)
|
||||
parsed_tenant_prefixes = _parse_tenant_prefixes(tenant_prefixes)
|
||||
except click.UsageError as e:
|
||||
click.echo(click.style(e.message, fg="red"))
|
||||
if (start_from is None) ^ (end_before is None):
|
||||
click.echo(click.style("start-from and end-before must be provided together.", fg="red"))
|
||||
return
|
||||
|
||||
if (from_days_ago is None) ^ (to_days_ago is None):
|
||||
click.echo(click.style("from-days-ago and to-days-ago must be provided together.", fg="red"))
|
||||
return
|
||||
|
||||
if from_days_ago is not None and to_days_ago is not None:
|
||||
if start_from or end_before:
|
||||
click.echo(click.style("Choose either day offsets or explicit dates, not both.", fg="red"))
|
||||
return
|
||||
if from_days_ago <= to_days_ago:
|
||||
click.echo(click.style("from-days-ago must be greater than to-days-ago.", fg="red"))
|
||||
return
|
||||
now = datetime.datetime.now()
|
||||
start_from = now - datetime.timedelta(days=from_days_ago)
|
||||
end_before = now - datetime.timedelta(days=to_days_ago)
|
||||
before_days = 0
|
||||
|
||||
if start_from and end_before and start_from >= end_before:
|
||||
click.echo(click.style("start-from must be earlier than end-before.", fg="red"))
|
||||
return
|
||||
plan_end_before = end_before or datetime.datetime.now(datetime.UTC) - datetime.timedelta(days=before_days)
|
||||
if workers < 1:
|
||||
click.echo(click.style("workers must be at least 1.", fg="red"))
|
||||
return
|
||||
if (run_shard_index is None) ^ (run_shard_total is None):
|
||||
click.echo(click.style("run-shard-index and run-shard-total must be provided together.", fg="red"))
|
||||
return
|
||||
if run_shard_index is not None and run_shard_total is not None and run_shard_index >= run_shard_total:
|
||||
click.echo(click.style("run-shard-index must be less than run-shard-total.", fg="red"))
|
||||
return
|
||||
if delete_after_archive:
|
||||
click.echo(click.style("delete-after-archive is not supported by bundle archive.", fg="red"))
|
||||
return
|
||||
if uses_relative_window:
|
||||
click.echo(
|
||||
click.style(
|
||||
"Relative archive windows are evaluated at command start. For multi-day prefix/shard rollout, "
|
||||
"reuse absolute --start-from/--end-before values from archive-workflow-runs-plan.",
|
||||
fg="yellow",
|
||||
)
|
||||
)
|
||||
|
||||
try:
|
||||
tenant_plan = _resolve_archive_tenant_ids_from_plan(
|
||||
tenant_ids=tenant_ids,
|
||||
tenant_prefixes=parsed_tenant_prefixes,
|
||||
start_from=start_from,
|
||||
end_before=plan_end_before,
|
||||
)
|
||||
except Exception:
|
||||
logger.exception("Failed to resolve workflow archive tenant plan")
|
||||
click.echo(click.style("Failed to resolve workflow archive tenant plan.", fg="red"))
|
||||
return
|
||||
|
||||
planned_tenant_ids = tenant_plan["archive_tenant_ids"]
|
||||
planned_paid_tenant_ids = tenant_plan["paid_tenant_ids"] if planned_tenant_ids is not None else None
|
||||
paid_tenants = len(tenant_plan["paid_tenant_ids"])
|
||||
unpaid_tenants = len(tenant_plan["unpaid_tenant_ids"])
|
||||
if planned_tenant_ids is not None:
|
||||
click.echo(
|
||||
click.style(
|
||||
f"Resolved archive tenant plan: paid_tenants={paid_tenants}, unpaid_tenants={unpaid_tenants}.",
|
||||
fg="white",
|
||||
)
|
||||
)
|
||||
if not planned_tenant_ids:
|
||||
click.echo(click.style("No paid tenants matched the archive plan; nothing to archive.", fg="yellow"))
|
||||
return
|
||||
|
||||
archiver = WorkflowRunArchiver(
|
||||
days=before_days,
|
||||
@@ -607,11 +242,7 @@ def archive_workflow_runs(
|
||||
start_from=start_from,
|
||||
end_before=end_before,
|
||||
workers=workers,
|
||||
tenant_ids=planned_tenant_ids,
|
||||
tenant_prefixes=parsed_tenant_prefixes,
|
||||
paid_tenant_ids=planned_paid_tenant_ids,
|
||||
run_shard_index=run_shard_index,
|
||||
run_shard_total=run_shard_total,
|
||||
tenant_ids=[tid.strip() for tid in tenant_ids.split(",")] if tenant_ids else None,
|
||||
limit=limit,
|
||||
dry_run=dry_run,
|
||||
delete_after_archive=delete_after_archive,
|
||||
@@ -621,9 +252,7 @@ def archive_workflow_runs(
|
||||
click.style(
|
||||
f"Summary: processed={summary.total_runs_processed}, archived={summary.runs_archived}, "
|
||||
f"skipped={summary.runs_skipped}, failed={summary.runs_failed}, "
|
||||
f"bundles_archived={summary.bundles_archived}, bundles_skipped={summary.bundles_skipped}, "
|
||||
f"bundles_failed={summary.bundles_failed}, "
|
||||
f"object_size_bytes={summary.total_object_size_bytes}, time={summary.total_elapsed_time:.2f}s",
|
||||
f"time={summary.total_elapsed_time:.2f}s",
|
||||
fg="cyan",
|
||||
)
|
||||
)
|
||||
@@ -639,52 +268,6 @@ def archive_workflow_runs(
|
||||
)
|
||||
|
||||
|
||||
def _echo_bundle_archive_operation_summary(summary) -> None:
|
||||
status = "completed successfully" if summary.bundles_failed == 0 else "completed with failures"
|
||||
fg = "green" if summary.bundles_failed == 0 else "red"
|
||||
click.echo(
|
||||
click.style(
|
||||
f"{summary.operation} {status}. "
|
||||
f"bundles_success={summary.bundles_succeeded} bundles_failed={summary.bundles_failed} "
|
||||
f"runs={summary.runs_processed} rows={summary.rows_processed} "
|
||||
f"archive_bytes={summary.archive_bytes} duration={summary.elapsed_time:.2f}s "
|
||||
f"validation_time={summary.validation_time:.2f}s "
|
||||
f"runs_per_second={summary.runs_per_second:.2f} rows_per_second={summary.rows_per_second:.2f} "
|
||||
f"bytes_per_second={summary.bytes_per_second:.2f}",
|
||||
fg=fg,
|
||||
)
|
||||
)
|
||||
click.echo(click.style("table,row_count", fg="white"))
|
||||
for table_name in [
|
||||
"workflow_runs",
|
||||
"workflow_app_logs",
|
||||
"workflow_node_executions",
|
||||
"workflow_node_execution_offload",
|
||||
"workflow_pauses",
|
||||
"workflow_pause_reasons",
|
||||
"workflow_trigger_logs",
|
||||
]:
|
||||
click.echo(f"{table_name},{summary.table_counts.get(table_name, 0)}")
|
||||
for result in summary.results:
|
||||
if result.success:
|
||||
click.echo(
|
||||
click.style(
|
||||
f" bundle={result.bundle_id} tenant={result.tenant_id} runs={result.run_count} "
|
||||
f"rows={result.row_count} archive_bytes={result.archive_bytes} "
|
||||
f"time={result.elapsed_time:.2f}s validation={result.validation_time:.2f}s",
|
||||
fg="white",
|
||||
)
|
||||
)
|
||||
else:
|
||||
click.echo(
|
||||
click.style(
|
||||
f" failed bundle={result.bundle_id} tenant={result.tenant_id} "
|
||||
f"object_prefix={result.object_prefix} error={result.error}",
|
||||
fg="red",
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
@click.command(
|
||||
"restore-workflow-runs",
|
||||
help="Restore archived workflow runs from S3-compatible storage.",
|
||||
@@ -707,8 +290,8 @@ def _echo_bundle_archive_operation_summary(summary) -> None:
|
||||
default=None,
|
||||
help="Optional upper bound (exclusive) for created_at; must be paired with --start-from.",
|
||||
)
|
||||
@click.option("--workers", default=1, show_default=True, type=int, help="V1 --run-id compatibility only.")
|
||||
@click.option("--limit", type=int, default=100, show_default=True, help="Maximum number of V2 bundles to restore.")
|
||||
@click.option("--workers", default=1, show_default=True, type=int, help="Concurrent workflow runs to restore.")
|
||||
@click.option("--limit", type=int, default=100, show_default=True, help="Maximum number of runs to restore.")
|
||||
@click.option("--dry-run", is_flag=True, help="Preview without restoring.")
|
||||
def restore_workflow_runs(
|
||||
tenant_ids: str | None,
|
||||
@@ -720,18 +303,15 @@ def restore_workflow_runs(
|
||||
dry_run: bool,
|
||||
):
|
||||
"""
|
||||
Restore archived workflow runs from storage to the database.
|
||||
Restore an archived workflow run from storage to the database.
|
||||
|
||||
Batch restore uses V2 bundle metadata and validates archive objects before writing source rows. This restores:
|
||||
- workflow_runs
|
||||
- workflow_app_logs
|
||||
This restores the following tables:
|
||||
- workflow_node_executions
|
||||
- workflow_node_execution_offload
|
||||
- workflow_pauses
|
||||
- workflow_pause_reasons
|
||||
- workflow_trigger_logs
|
||||
"""
|
||||
from services.retention.workflow_run.bundle_archive_maintenance import WorkflowRunBundleArchiveMaintenance
|
||||
from services.retention.workflow_run.restore_archived_workflow_run import WorkflowRunRestore
|
||||
|
||||
parsed_tenant_ids = None
|
||||
@@ -755,46 +335,39 @@ def restore_workflow_runs(
|
||||
)
|
||||
)
|
||||
|
||||
restorer = WorkflowRunRestore(dry_run=dry_run, workers=workers)
|
||||
if run_id:
|
||||
restorer = WorkflowRunRestore(dry_run=dry_run, workers=workers)
|
||||
results = [restorer.restore_by_run_id(run_id)]
|
||||
end_time = datetime.datetime.now(datetime.UTC)
|
||||
elapsed = end_time - start_time
|
||||
else:
|
||||
assert start_from is not None
|
||||
assert end_before is not None
|
||||
results = restorer.restore_batch(
|
||||
parsed_tenant_ids,
|
||||
start_date=start_from,
|
||||
end_date=end_before,
|
||||
limit=limit,
|
||||
)
|
||||
|
||||
successes = sum(1 for result in results if result.success)
|
||||
failures = len(results) - successes
|
||||
end_time = datetime.datetime.now(datetime.UTC)
|
||||
elapsed = end_time - start_time
|
||||
|
||||
if failures == 0:
|
||||
click.echo(
|
||||
click.style(
|
||||
f"Restore completed successfully. success={successes} duration={elapsed}",
|
||||
fg="green",
|
||||
)
|
||||
successes = sum(1 for result in results if result.success)
|
||||
failures = len(results) - successes
|
||||
|
||||
if failures == 0:
|
||||
click.echo(
|
||||
click.style(
|
||||
f"Restore completed successfully. success={successes} duration={elapsed}",
|
||||
fg="green",
|
||||
)
|
||||
else:
|
||||
click.echo(
|
||||
click.style(
|
||||
f"Restore completed with failures. success={successes} failed={failures} duration={elapsed}",
|
||||
fg="red",
|
||||
)
|
||||
)
|
||||
else:
|
||||
click.echo(
|
||||
click.style(
|
||||
f"Restore completed with failures. success={successes} failed={failures} duration={elapsed}",
|
||||
fg="red",
|
||||
)
|
||||
return
|
||||
|
||||
if workers != 1:
|
||||
click.echo(
|
||||
click.style("--workers is ignored for V2 bundle restore; bundles are processed serially.", fg="yellow")
|
||||
)
|
||||
assert start_from is not None
|
||||
assert end_before is not None
|
||||
bundle_restorer = WorkflowRunBundleArchiveMaintenance(dry_run=dry_run, strict_content_validation=True)
|
||||
summary = bundle_restorer.restore_batch(
|
||||
tenant_ids=parsed_tenant_ids,
|
||||
start_date=start_from,
|
||||
end_date=end_before,
|
||||
limit=limit,
|
||||
)
|
||||
_echo_bundle_archive_operation_summary(summary)
|
||||
return
|
||||
|
||||
|
||||
@click.command(
|
||||
@@ -819,20 +392,8 @@ def restore_workflow_runs(
|
||||
default=None,
|
||||
help="Optional upper bound (exclusive) for created_at; must be paired with --start-from.",
|
||||
)
|
||||
@click.option("--limit", type=int, default=100, show_default=True, help="Maximum number of V2 bundles to delete.")
|
||||
@click.option("--limit", type=int, default=100, show_default=True, help="Maximum number of runs to delete.")
|
||||
@click.option("--dry-run", is_flag=True, help="Preview without deleting.")
|
||||
@click.option(
|
||||
"--skip-bad-archives",
|
||||
is_flag=True,
|
||||
help="Continue batch deletion when one archive object fails validation.",
|
||||
)
|
||||
@click.option(
|
||||
"--restore-sample-interval",
|
||||
type=int,
|
||||
default=0,
|
||||
show_default=True,
|
||||
help="Run restore dry-run after every N successful deletes; 0 disables restore sampling.",
|
||||
)
|
||||
def delete_archived_workflow_runs(
|
||||
tenant_ids: str | None,
|
||||
run_id: str | None,
|
||||
@@ -840,16 +401,10 @@ def delete_archived_workflow_runs(
|
||||
end_before: datetime.datetime | None,
|
||||
limit: int,
|
||||
dry_run: bool,
|
||||
skip_bad_archives: bool,
|
||||
restore_sample_interval: int,
|
||||
):
|
||||
"""
|
||||
Delete archived workflow runs from the database.
|
||||
|
||||
Batch delete uses V2 bundle metadata and validates object existence, manifest schema, object size, checksum, row
|
||||
counts, and source/archive content checksums before deleting source rows. `--run-id` keeps the V1 per-run path.
|
||||
"""
|
||||
from services.retention.workflow_run.bundle_archive_maintenance import WorkflowRunBundleArchiveMaintenance
|
||||
from services.retention.workflow_run.delete_archived_workflow_run import ArchivedWorkflowRunDeletion
|
||||
|
||||
parsed_tenant_ids = None
|
||||
@@ -862,8 +417,6 @@ def delete_archived_workflow_runs(
|
||||
raise click.UsageError("--start-from and --end-before must be provided together.")
|
||||
if run_id is None and (start_from is None or end_before is None):
|
||||
raise click.UsageError("--start-from and --end-before are required for batch delete.")
|
||||
if restore_sample_interval < 0:
|
||||
raise click.BadParameter("restore-sample-interval must be >= 0")
|
||||
|
||||
start_time = datetime.datetime.now(datetime.UTC)
|
||||
target_desc = f"workflow run {run_id}" if run_id else "workflow runs"
|
||||
@@ -874,85 +427,56 @@ def delete_archived_workflow_runs(
|
||||
)
|
||||
)
|
||||
|
||||
deleter = ArchivedWorkflowRunDeletion(dry_run=dry_run)
|
||||
if run_id:
|
||||
deleter = ArchivedWorkflowRunDeletion(
|
||||
dry_run=dry_run,
|
||||
skip_bad_archives=skip_bad_archives,
|
||||
restore_sample_interval=restore_sample_interval,
|
||||
)
|
||||
results = [deleter.delete_by_run_id(run_id)]
|
||||
for result in results:
|
||||
if result.success:
|
||||
click.echo(
|
||||
click.style(
|
||||
f"{'[DRY RUN] Would delete' if dry_run else 'Deleted'} "
|
||||
f"workflow run {result.run_id} (tenant={result.tenant_id}, "
|
||||
f"archive_key={result.archive_key}, counts={result.validated_counts})",
|
||||
fg="green",
|
||||
)
|
||||
)
|
||||
if result.restore_sampled:
|
||||
sample_status = "passed" if result.restore_sample_success else "failed"
|
||||
click.echo(
|
||||
click.style(
|
||||
f" restore dry-run sample {sample_status} for workflow run {result.run_id}",
|
||||
fg="green" if result.restore_sample_success else "red",
|
||||
)
|
||||
)
|
||||
else:
|
||||
click.echo(
|
||||
click.style(
|
||||
f"Failed to delete workflow run {result.run_id}: {result.error}",
|
||||
fg="red",
|
||||
)
|
||||
)
|
||||
click.echo(
|
||||
click.style(
|
||||
" runbook: pause this delete window, verify archive storage object and manifest/checksum, "
|
||||
"retry the same run after fixing storage or DB drift, or rerun with --skip-bad-archives "
|
||||
"to quarantine this run and continue the batch.",
|
||||
fg="yellow",
|
||||
)
|
||||
)
|
||||
else:
|
||||
assert start_from is not None
|
||||
assert end_before is not None
|
||||
results = deleter.delete_batch(
|
||||
parsed_tenant_ids,
|
||||
start_date=start_from,
|
||||
end_date=end_before,
|
||||
limit=limit,
|
||||
)
|
||||
|
||||
end_time = datetime.datetime.now(datetime.UTC)
|
||||
elapsed = end_time - start_time
|
||||
|
||||
successes = sum(1 for result in results if result.success)
|
||||
failures = len(results) - successes
|
||||
|
||||
if failures == 0:
|
||||
for result in results:
|
||||
if result.success:
|
||||
click.echo(
|
||||
click.style(
|
||||
f"Delete completed successfully. success={successes} duration={elapsed}",
|
||||
f"{'[DRY RUN] Would delete' if dry_run else 'Deleted'} "
|
||||
f"workflow run {result.run_id} (tenant={result.tenant_id})",
|
||||
fg="green",
|
||||
)
|
||||
)
|
||||
else:
|
||||
click.echo(
|
||||
click.style(
|
||||
f"Delete completed with failures. success={successes} failed={failures} duration={elapsed}",
|
||||
f"Failed to delete workflow run {result.run_id}: {result.error}",
|
||||
fg="red",
|
||||
)
|
||||
)
|
||||
return
|
||||
|
||||
if restore_sample_interval:
|
||||
click.echo(click.style("--restore-sample-interval is ignored for V2 bundle delete.", fg="yellow"))
|
||||
assert start_from is not None
|
||||
assert end_before is not None
|
||||
bundle_deleter = WorkflowRunBundleArchiveMaintenance(
|
||||
dry_run=dry_run,
|
||||
strict_content_validation=True,
|
||||
stop_on_error=not skip_bad_archives,
|
||||
)
|
||||
summary = bundle_deleter.delete_batch(
|
||||
tenant_ids=parsed_tenant_ids,
|
||||
start_date=start_from,
|
||||
end_date=end_before,
|
||||
limit=limit,
|
||||
)
|
||||
_echo_bundle_archive_operation_summary(summary)
|
||||
end_time = datetime.datetime.now(datetime.UTC)
|
||||
elapsed = end_time - start_time
|
||||
|
||||
successes = sum(1 for result in results if result.success)
|
||||
failures = len(results) - successes
|
||||
|
||||
if failures == 0:
|
||||
click.echo(
|
||||
click.style(
|
||||
f"Delete completed successfully. success={successes} duration={elapsed}",
|
||||
fg="green",
|
||||
)
|
||||
)
|
||||
else:
|
||||
click.echo(
|
||||
click.style(
|
||||
f"Delete completed with failures. success={successes} failed={failures} duration={elapsed}",
|
||||
fg="red",
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
def _find_orphaned_draft_variables(batch_size: int = 1000) -> list[str]:
|
||||
|
||||
@@ -13,7 +13,6 @@ from core.rag.index_processor.constant.built_in_field import BuiltInField
|
||||
from core.rag.index_processor.constant.index_type import IndexStructureType, IndexTechniqueType
|
||||
from core.rag.models.document import ChildDocument, Document
|
||||
from extensions.ext_database import db
|
||||
from libs.pagination import paginate_query
|
||||
from models.dataset import Dataset, DatasetCollectionBinding, DatasetMetadata, DatasetMetadataBinding, DocumentSegment
|
||||
from models.dataset import Document as DatasetDocument
|
||||
from models.enums import DatasetMetadataType, IndexingStatus, SegmentStatus
|
||||
@@ -184,7 +183,7 @@ def migrate_knowledge_vector_database():
|
||||
.order_by(Dataset.created_at.desc())
|
||||
)
|
||||
|
||||
datasets = paginate_query(stmt, page=page, per_page=50, max_per_page=50)
|
||||
datasets = db.paginate(select=stmt, page=page, per_page=50, max_per_page=50, error_out=False)
|
||||
if not datasets.items:
|
||||
break
|
||||
except SQLAlchemyError:
|
||||
@@ -410,7 +409,7 @@ def old_metadata_migration():
|
||||
.where(DatasetDocument.doc_metadata.is_not(None))
|
||||
.order_by(DatasetDocument.created_at.desc())
|
||||
)
|
||||
documents = paginate_query(stmt, page=page, per_page=50, max_per_page=50)
|
||||
documents = db.paginate(select=stmt, page=page, per_page=50, max_per_page=50, error_out=False)
|
||||
except SQLAlchemyError:
|
||||
raise
|
||||
if not documents:
|
||||
|
||||
@@ -16,7 +16,7 @@ class EnterpriseFeatureConfig(BaseSettings):
|
||||
|
||||
CAN_REPLACE_LOGO: bool = Field(
|
||||
description="Allow customization of the enterprise logo.",
|
||||
default=False,
|
||||
default=True,
|
||||
)
|
||||
|
||||
ENTERPRISE_REQUEST_TIMEOUT: int = Field(
|
||||
@@ -29,17 +29,6 @@ class EnterpriseFeatureConfig(BaseSettings):
|
||||
"This helps gain runtime performance by trading off consistency.",
|
||||
)
|
||||
|
||||
RBAC_ENABLED: bool = Field(
|
||||
description="Enable enterprise RBAC APIs. When disabled, compatibility responses fall back to legacy roles.",
|
||||
default=False,
|
||||
)
|
||||
|
||||
ENTERPRISE_RBAC_REQUEST_TIMEOUT: int = Field(
|
||||
ge=1,
|
||||
description="Maximum timeout in seconds for inner RBAC requests.",
|
||||
default=30,
|
||||
)
|
||||
|
||||
|
||||
class EnterpriseTelemetryConfig(BaseSettings):
|
||||
"""
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
from pydantic import Field, NonNegativeFloat
|
||||
from pydantic import Field
|
||||
from pydantic_settings import BaseSettings
|
||||
|
||||
|
||||
@@ -32,10 +32,12 @@ class AgentBackendConfig(BaseSettings):
|
||||
default=False,
|
||||
)
|
||||
|
||||
AGENT_APP_TEXT_DELTA_DEBOUNCE_SECONDS: NonNegativeFloat = Field(
|
||||
AGENT_DRIVE_MANIFEST_ENABLED: bool = Field(
|
||||
description=(
|
||||
"Buffer Agent App assistant text deltas for up to this many seconds before "
|
||||
"publishing SSE chunks. Set to 0 to publish each delta immediately."
|
||||
"Inject the dify.drive layer (Skills & Files drive manifest declaration) "
|
||||
"into Agent runs. The declaration is an index only — the agent backend "
|
||||
"pulls the actual SKILL.md / files through the back proxy. Keep it off "
|
||||
"until the agent backend registers the dify.drive layer type."
|
||||
),
|
||||
default=0.5,
|
||||
default=False,
|
||||
)
|
||||
|
||||
@@ -1073,12 +1073,6 @@ class MailConfig(BaseSettings):
|
||||
default=None,
|
||||
)
|
||||
|
||||
|
||||
class HomepageConfig(BaseSettings):
|
||||
"""
|
||||
Configuration for homepage feature toggles exposed through system features.
|
||||
"""
|
||||
|
||||
ENABLE_TRIAL_APP: bool = Field(
|
||||
description="Enable trial app",
|
||||
default=False,
|
||||
@@ -1089,11 +1083,6 @@ class HomepageConfig(BaseSettings):
|
||||
default=False,
|
||||
)
|
||||
|
||||
ENABLE_LEARN_APP: bool = Field(
|
||||
description="Enable Learn App",
|
||||
default=True,
|
||||
)
|
||||
|
||||
|
||||
class RagEtlConfig(BaseSettings):
|
||||
"""
|
||||
@@ -1500,7 +1489,6 @@ class FeatureConfig(
|
||||
EndpointConfig,
|
||||
FileAccessConfig,
|
||||
FileUploadConfig,
|
||||
HomepageConfig,
|
||||
HttpConfig,
|
||||
InnerAPIConfig,
|
||||
IndexingConfig,
|
||||
|
||||
@@ -2,6 +2,7 @@ from typing import Literal, Protocol, cast
|
||||
from urllib.parse import quote_plus, urlunparse
|
||||
|
||||
from pydantic import AliasChoices, Field
|
||||
from pydantic.types import NonNegativeInt
|
||||
from pydantic_settings import BaseSettings
|
||||
|
||||
|
||||
@@ -70,6 +71,24 @@ class RedisPubSubConfig(BaseSettings):
|
||||
default=600,
|
||||
)
|
||||
|
||||
PUBSUB_LISTENER_JOIN_TIMEOUT_MS: NonNegativeInt = Field(
|
||||
validation_alias=AliasChoices("EVENT_BUS_LISTENER_JOIN_TIMEOUT_MS", "PUBSUB_LISTENER_JOIN_TIMEOUT_MS"),
|
||||
description=(
|
||||
"Maximum time (milliseconds) that ``Subscription.close()`` waits for its listener thread to "
|
||||
"finish before returning. Bounds the tail latency between a terminal event being delivered to "
|
||||
"an SSE client and the response stream actually closing.\n\n"
|
||||
"The listener thread blocks on a polling read (XREAD BLOCK for streams, get_message timeout "
|
||||
"for pubsub/sharded) with a fixed 1s window, so close() naturally has to wait up to ~1s for "
|
||||
"the thread to notice the subscription was closed. Setting this lower (e.g. 100) lets close() "
|
||||
"return promptly while the daemon listener thread cleans itself up on the next poll "
|
||||
"boundary - safe because the listener holds no critical state and exits within one poll "
|
||||
"window. Setting it higher (e.g. 5000) gives the listener more grace before close() gives up "
|
||||
"and logs a warning. Default 2000ms preserves the pre-change behaviour.\n\n"
|
||||
"Also accepts ENV: EVENT_BUS_LISTENER_JOIN_TIMEOUT_MS."
|
||||
),
|
||||
default=2000,
|
||||
)
|
||||
|
||||
def _build_default_pubsub_url(self) -> str:
|
||||
defaults = _redis_defaults(self)
|
||||
if not defaults.REDIS_HOST or not defaults.REDIS_PORT:
|
||||
|
||||
@@ -20,12 +20,6 @@ class NacosHttpClient:
|
||||
self.token: str | None = None
|
||||
self.token_ttl = 18000
|
||||
self.token_expire_time: float = 0
|
||||
# Bounded timeouts so a slow or unresponsive Nacos server cannot hang the API
|
||||
# service indefinitely during startup or token refresh.
|
||||
self.timeout = httpx.Timeout(
|
||||
float(os.getenv("DIFY_ENV_NACOS_REQUEST_TIMEOUT", "10.0")),
|
||||
connect=float(os.getenv("DIFY_ENV_NACOS_CONNECT_TIMEOUT", "3.0")),
|
||||
)
|
||||
|
||||
def http_request(
|
||||
self, url: str, method: str = "GET", headers: dict[str, str] | None = None, params: dict[str, str] | None = None
|
||||
@@ -34,17 +28,12 @@ class NacosHttpClient:
|
||||
headers = {}
|
||||
if params is None:
|
||||
params = {}
|
||||
full_url = "http://" + self.server + url
|
||||
try:
|
||||
self._inject_auth_info(headers, params)
|
||||
response = httpx.request(method, url=full_url, headers=headers, params=params, timeout=self.timeout)
|
||||
response = httpx.request(method, url="http://" + self.server + url, headers=headers, params=params)
|
||||
response.raise_for_status()
|
||||
return response.text
|
||||
except httpx.TimeoutException as e:
|
||||
logger.warning("Request to Nacos timed out (url=%s, timeout=%s): %s", full_url, self.timeout, e)
|
||||
return f"Request to Nacos timed out: {e}"
|
||||
except httpx.RequestError as e:
|
||||
logger.warning("Request to Nacos failed (url=%s): %s", full_url, e)
|
||||
return f"Request to Nacos failed: {e}"
|
||||
|
||||
def _inject_auth_info(self, headers: dict[str, str], params: dict[str, str], module: str = "config") -> None:
|
||||
@@ -89,16 +78,13 @@ class NacosHttpClient:
|
||||
params = {"username": self.username, "password": self.password}
|
||||
url = "http://" + self.server + "/nacos/v1/auth/login"
|
||||
try:
|
||||
resp = httpx.request("POST", url, headers=None, params=params, timeout=self.timeout)
|
||||
resp = httpx.request("POST", url, headers=None, params=params)
|
||||
resp.raise_for_status()
|
||||
response_data = resp.json()
|
||||
self.token = response_data.get("accessToken")
|
||||
self.token_ttl = response_data.get("tokenTtl", 18000)
|
||||
self.token_expire_time = current_time + self.token_ttl - 10
|
||||
return self.token
|
||||
except httpx.TimeoutException:
|
||||
logger.exception("[get-access-token] request to Nacos timed out (url=%s, timeout=%s)", url, self.timeout)
|
||||
raise
|
||||
except Exception:
|
||||
logger.exception("[get-access-token] exception occur")
|
||||
raise
|
||||
|
||||
File diff suppressed because one or more lines are too long
@@ -1,107 +0,0 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from collections.abc import Sequence
|
||||
from dataclasses import dataclass
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
from services.enterprise import rbac_service as enterprise_rbac_service
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from services.app_service import AppListBaseParams
|
||||
from services.enterprise.rbac_service import MyPermissionsResponse
|
||||
|
||||
# Permission keys (dot-notation, from MyPermissionsResponse) that grant
|
||||
# list/preview access to an app. Keep this the single source of truth for both
|
||||
# the console and OpenAPI app-list endpoints.
|
||||
APP_LIST_PERMISSION_KEYS: frozenset[str] = frozenset({"app.preview", "app.acl.preview", "app.full_access"})
|
||||
|
||||
# Workspace permission key that lets a caller see apps they maintain even when
|
||||
# those apps are not in their preview whitelist.
|
||||
_MANAGE_OWN_APPS_PERMISSION_KEY = "app.create_and_management"
|
||||
|
||||
|
||||
def has_app_list_permission(permission_keys: Sequence[str]) -> bool:
|
||||
"""Return True if any of ``permission_keys`` grants app list/preview access."""
|
||||
return any(permission_key in APP_LIST_PERMISSION_KEYS for permission_key in permission_keys)
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class AppAccessFilter:
|
||||
"""Resolved RBAC visibility for app list/read endpoints.
|
||||
|
||||
``accessible_app_ids`` of ``None`` means the caller can see every app in the
|
||||
workspace (unrestricted). Otherwise it is the exact set of app ids the
|
||||
caller may preview; combined with ``can_manage_own_apps`` it also covers
|
||||
apps the caller maintains.
|
||||
"""
|
||||
|
||||
accessible_app_ids: set[str] | None
|
||||
can_manage_own_apps: bool
|
||||
|
||||
@classmethod
|
||||
def unrestricted(cls) -> AppAccessFilter:
|
||||
"""Filter that imposes no restriction (RBAC disabled / not applicable)."""
|
||||
return cls(accessible_app_ids=None, can_manage_own_apps=False)
|
||||
|
||||
def is_app_accessible(self, app_id: str, maintainer: str | None, account_id: str) -> bool:
|
||||
"""Whether a single app is visible to the caller under this filter.
|
||||
|
||||
Mirrors the service-layer query gate: an app is visible when the filter
|
||||
is unrestricted, the app id is whitelisted, or the caller maintains it
|
||||
and holds ``app.create_and_management``.
|
||||
"""
|
||||
if self.accessible_app_ids is None:
|
||||
return True
|
||||
if app_id in self.accessible_app_ids:
|
||||
return True
|
||||
return self.can_manage_own_apps and maintainer is not None and maintainer == account_id
|
||||
|
||||
def apply_to_params(self, params: AppListBaseParams) -> None:
|
||||
if self.accessible_app_ids is None:
|
||||
return
|
||||
params.accessible_app_ids = sorted(self.accessible_app_ids)
|
||||
params.include_own_apps = self.can_manage_own_apps
|
||||
|
||||
|
||||
def resolve_app_access_filter(
|
||||
tenant_id: str,
|
||||
account_id: str,
|
||||
*,
|
||||
permissions: MyPermissionsResponse | None = None,
|
||||
) -> AppAccessFilter:
|
||||
"""Compute the RBAC app-access filter for ``account_id`` in ``tenant_id``.
|
||||
|
||||
Pass ``permissions`` when the caller has already fetched the snapshot (the
|
||||
console controller reuses it for per-app permission keys) to avoid a second
|
||||
inner-API round trip; otherwise it is fetched here.
|
||||
"""
|
||||
if permissions is None:
|
||||
permissions = enterprise_rbac_service.RBACService.MyPermissions.get(tenant_id, account_id)
|
||||
whitelist_scope = enterprise_rbac_service.RBACService.AppAccess.whitelist_resources(tenant_id, account_id)
|
||||
|
||||
can_manage_own_apps = _MANAGE_OWN_APPS_PERMISSION_KEY in permissions.workspace.permission_keys
|
||||
has_default_preview = has_app_list_permission(permissions.app.default_permission_keys) or has_app_list_permission(
|
||||
permissions.workspace.permission_keys
|
||||
)
|
||||
|
||||
permission_app_ids: set[str] | None = None
|
||||
if not has_default_preview:
|
||||
# Collect apps the caller can preview via per-app permission overrides.
|
||||
permission_app_ids = {
|
||||
override.resource_id
|
||||
for override in permissions.app.overrides
|
||||
if has_app_list_permission(override.permission_keys)
|
||||
}
|
||||
|
||||
accessible_app_ids: set[str] | None
|
||||
if getattr(whitelist_scope, "unrestricted", False):
|
||||
accessible_app_ids = permission_app_ids
|
||||
else:
|
||||
accessible_app_ids = set(whitelist_scope.resource_ids)
|
||||
if permission_app_ids is not None:
|
||||
accessible_app_ids |= permission_app_ids
|
||||
elif has_default_preview:
|
||||
# Default preview overrides the whitelist restriction.
|
||||
accessible_app_ids = None
|
||||
|
||||
return AppAccessFilter(accessible_app_ids=accessible_app_ids, can_manage_own_apps=can_manage_own_apps)
|
||||
@@ -1,8 +1,7 @@
|
||||
from copy import deepcopy
|
||||
from typing import Annotated, Any, Literal, override
|
||||
from typing import Any, Literal
|
||||
from uuid import UUID
|
||||
|
||||
from pydantic import BaseModel, Field, GetJsonSchemaHandler, WithJsonSchema, model_validator
|
||||
from pydantic import BaseModel, Field, model_validator
|
||||
|
||||
from libs.helper import UUIDStrOrEmpty
|
||||
|
||||
@@ -10,53 +9,8 @@ from libs.helper import UUIDStrOrEmpty
|
||||
|
||||
|
||||
class ConversationRenamePayload(BaseModel):
|
||||
name: str | None = Field(
|
||||
default=None,
|
||||
description="Conversation name. Required when `auto_generate` is `false`.",
|
||||
)
|
||||
auto_generate: bool = Field(
|
||||
default=False,
|
||||
description="Automatically generate the conversation name. When `true`, the `name` field is ignored.",
|
||||
)
|
||||
|
||||
@classmethod
|
||||
@override
|
||||
def __get_pydantic_json_schema__(cls, core_schema: Any, handler: GetJsonSchemaHandler) -> dict[str, Any]:
|
||||
schema = handler.resolve_ref_schema(handler(core_schema))
|
||||
properties = schema.get("properties")
|
||||
if not isinstance(properties, dict):
|
||||
return schema
|
||||
|
||||
auto_generate_schema = deepcopy(properties.get("auto_generate", {"type": "boolean"}))
|
||||
name_schema = deepcopy(properties.get("name", {"type": "string"}))
|
||||
non_blank_name_schema: dict[str, Any] = {"pattern": r".*\S.*", "type": "string"}
|
||||
if isinstance(name_schema, dict) and isinstance(name_schema.get("title"), str):
|
||||
non_blank_name_schema["title"] = name_schema["title"]
|
||||
|
||||
auto_generate_true_schema = {**auto_generate_schema, "enum": [True]}
|
||||
auto_generate_true_schema.pop("default", None)
|
||||
|
||||
return {
|
||||
**schema,
|
||||
"anyOf": [
|
||||
{
|
||||
"properties": {
|
||||
"auto_generate": auto_generate_true_schema,
|
||||
"name": name_schema,
|
||||
},
|
||||
"required": ["auto_generate"],
|
||||
"type": "object",
|
||||
},
|
||||
{
|
||||
"properties": {
|
||||
"auto_generate": {**auto_generate_schema, "enum": [False]},
|
||||
"name": non_blank_name_schema,
|
||||
},
|
||||
"required": ["name"],
|
||||
"type": "object",
|
||||
},
|
||||
],
|
||||
}
|
||||
name: str | None = None
|
||||
auto_generate: bool = False
|
||||
|
||||
@model_validator(mode="after")
|
||||
def validate_name_requirement(self):
|
||||
@@ -70,28 +24,14 @@ class ConversationRenamePayload(BaseModel):
|
||||
|
||||
|
||||
class MessageListQuery(BaseModel):
|
||||
conversation_id: UUIDStrOrEmpty = Field(description="Conversation ID.")
|
||||
first_id: UUIDStrOrEmpty | None = Field(
|
||||
default=None,
|
||||
description=(
|
||||
"The ID of the first chat record on the current page. Omit this value to fetch the latest messages; "
|
||||
"for subsequent pages, use the first message ID from the current list to fetch older messages."
|
||||
),
|
||||
)
|
||||
limit: int = Field(
|
||||
default=20,
|
||||
ge=1,
|
||||
le=100,
|
||||
description="Number of chat history messages to return per request.",
|
||||
)
|
||||
conversation_id: UUIDStrOrEmpty = Field(description="Conversation UUID")
|
||||
first_id: UUIDStrOrEmpty | None = Field(default=None, description="First message ID for pagination")
|
||||
limit: int = Field(default=20, ge=1, le=100, description="Number of messages to return (1-100)")
|
||||
|
||||
|
||||
class MessageFeedbackPayload(BaseModel):
|
||||
rating: Literal["like", "dislike"] | None = Field(
|
||||
default=None,
|
||||
description="Feedback rating. Set to `null` to revoke previously submitted feedback.",
|
||||
)
|
||||
content: str | None = Field(default=None, description="Optional text feedback providing additional detail.")
|
||||
rating: Literal["like", "dislike"] | None = None
|
||||
content: str | None = None
|
||||
|
||||
|
||||
# --- Saved message schemas ---
|
||||
@@ -108,39 +48,6 @@ class SavedMessageCreatePayload(BaseModel):
|
||||
|
||||
# --- Workflow schemas ---
|
||||
|
||||
WORKFLOW_INPUT_FILE_ITEM_SCHEMA: dict[str, object] = {
|
||||
"type": "object",
|
||||
"required": ["type", "transfer_method"],
|
||||
"properties": {
|
||||
"type": {
|
||||
"description": "File type.",
|
||||
"enum": ["document", "image", "audio", "video", "custom"],
|
||||
"type": "string",
|
||||
},
|
||||
"transfer_method": {
|
||||
"description": "Transfer method: `remote_url` for file URL, `local_file` for uploaded file.",
|
||||
"enum": ["remote_url", "local_file"],
|
||||
"type": "string",
|
||||
},
|
||||
"url": {
|
||||
"description": "File URL when `transfer_method` is `remote_url`.",
|
||||
"format": "url",
|
||||
"type": "string",
|
||||
},
|
||||
"upload_file_id": {
|
||||
"description": (
|
||||
"Uploaded file ID obtained from the [Upload File](/api-reference/files/upload-file) API when "
|
||||
"`transfer_method` is `local_file`."
|
||||
),
|
||||
"type": "string",
|
||||
},
|
||||
},
|
||||
}
|
||||
WORKFLOW_INPUT_FILE_LIST_SCHEMA: dict[str, object] = {
|
||||
"anyOf": [{"items": WORKFLOW_INPUT_FILE_ITEM_SCHEMA, "type": "array"}, {"type": "null"}]
|
||||
}
|
||||
WorkflowInputFileList = Annotated[list[dict[str, Any]] | None, WithJsonSchema(WORKFLOW_INPUT_FILE_LIST_SCHEMA)]
|
||||
|
||||
|
||||
class DefaultBlockConfigQuery(BaseModel):
|
||||
q: str | None = None
|
||||
@@ -154,22 +61,8 @@ class WorkflowListQuery(BaseModel):
|
||||
|
||||
|
||||
class WorkflowRunPayload(BaseModel):
|
||||
inputs: dict[str, Any] = Field(
|
||||
description=(
|
||||
"Key-value pairs for workflow input variables. Values for file-type variables should be arrays of "
|
||||
"file objects with `type`, `transfer_method`, and either `url` or `upload_file_id`. Refer to the "
|
||||
"`user_input_form` field in the [Get App Parameters](/api-reference/applications/get-app-parameters) "
|
||||
"response to discover the variable names and types expected by your app."
|
||||
)
|
||||
)
|
||||
files: WorkflowInputFileList = Field(
|
||||
default=None,
|
||||
description=(
|
||||
"File list for workflow system file inputs. Available when file upload is enabled for the workflow. "
|
||||
"To attach a local file, first upload it via [Upload File](/api-reference/files/upload-file) and use "
|
||||
"the returned `id` as `upload_file_id` with `transfer_method: local_file`."
|
||||
),
|
||||
)
|
||||
inputs: dict[str, Any]
|
||||
files: list[dict[str, Any]] | None = Field(default=None)
|
||||
|
||||
|
||||
class WorkflowUpdatePayload(BaseModel):
|
||||
@@ -184,49 +77,28 @@ DOCUMENT_BATCH_DOWNLOAD_ZIP_MAX_DOCS = 100
|
||||
|
||||
|
||||
class ChildChunkCreatePayload(BaseModel):
|
||||
content: str = Field(description="Child chunk text content.")
|
||||
content: str
|
||||
|
||||
|
||||
class ChildChunkUpdatePayload(BaseModel):
|
||||
content: str = Field(description="Child chunk text content.")
|
||||
content: str
|
||||
|
||||
|
||||
class DocumentBatchDownloadZipPayload(BaseModel):
|
||||
"""Request payload for bulk downloading documents as a zip archive."""
|
||||
|
||||
document_ids: list[UUID] = Field(
|
||||
...,
|
||||
min_length=1,
|
||||
max_length=DOCUMENT_BATCH_DOWNLOAD_ZIP_MAX_DOCS,
|
||||
description="List of document IDs to include in the ZIP download.",
|
||||
)
|
||||
document_ids: list[UUID] = Field(..., min_length=1, max_length=DOCUMENT_BATCH_DOWNLOAD_ZIP_MAX_DOCS)
|
||||
|
||||
|
||||
class MetadataUpdatePayload(BaseModel):
|
||||
name: str = Field(description="New metadata field name.")
|
||||
name: str
|
||||
|
||||
|
||||
# --- Audio schemas ---
|
||||
|
||||
|
||||
UUIDString = Annotated[str, WithJsonSchema({"format": "uuid", "type": "string"})]
|
||||
|
||||
|
||||
class TextToAudioPayload(BaseModel):
|
||||
message_id: UUIDString | None = Field(
|
||||
default=None,
|
||||
description="Message ID. Takes priority over `text` when both are provided.",
|
||||
)
|
||||
voice: str | None = Field(
|
||||
default=None,
|
||||
description=(
|
||||
"Voice to use for text-to-speech. Available voices depend on the TTS provider configured for this app. "
|
||||
"Omit to use the app's configured voice when available; that value is exposed by "
|
||||
"[Get App Parameters](/api-reference/applications/get-app-parameters) as `text_to_speech.voice`."
|
||||
),
|
||||
)
|
||||
text: str | None = Field(default=None, description="Speech content to convert.")
|
||||
streaming: bool | None = Field(
|
||||
default=None,
|
||||
description="Reserved for compatibility; TTS response streaming is determined by the provider output.",
|
||||
)
|
||||
message_id: str | None = Field(default=None, description="Message ID")
|
||||
voice: str | None = Field(default=None, description="Voice to use for TTS")
|
||||
text: str | None = Field(default=None, description="Text to convert to audio")
|
||||
streaming: bool | None = Field(default=None, description="Enable streaming response")
|
||||
|
||||
@@ -153,7 +153,6 @@ class ApiBaseUrlResponse(ResponseModel):
|
||||
|
||||
class NewAppResponse(ResponseModel):
|
||||
new_app_id: str
|
||||
permission_keys: list[str] = Field(default_factory=list)
|
||||
|
||||
|
||||
class Parameters(BaseModel):
|
||||
@@ -183,7 +182,6 @@ class Site(BaseModel):
|
||||
description: str | None = None
|
||||
copyright: str | None = None
|
||||
privacy_policy: str | None = None
|
||||
input_placeholder: str | None = None
|
||||
custom_disclaimer: str | None = None
|
||||
default_language: str
|
||||
show_workflow_steps: bool
|
||||
|
||||
@@ -35,23 +35,17 @@ class HumanInputFormSubmitPayload(BaseModel):
|
||||
),
|
||||
examples=[HUMAN_INPUT_FORM_INPUT_EXAMPLE],
|
||||
)
|
||||
action: str = Field(
|
||||
description=(
|
||||
"ID of the action button the recipient selected. Must match one of the `id` values from the form's "
|
||||
"`user_actions` list."
|
||||
)
|
||||
)
|
||||
action: str
|
||||
|
||||
|
||||
def stringify_form_default_values(values: dict[str, object]) -> dict[str, str]:
|
||||
"""Serialize default values into strings expected by human-input form clients."""
|
||||
result: dict[str, str] = {}
|
||||
for key, value in values.items():
|
||||
match value:
|
||||
case None:
|
||||
result[key] = ""
|
||||
case dict() | list():
|
||||
result[key] = json.dumps(value, ensure_ascii=False)
|
||||
case _:
|
||||
result[key] = str(value)
|
||||
if value is None:
|
||||
result[key] = ""
|
||||
elif isinstance(value, (dict, list)):
|
||||
result[key] = json.dumps(value, ensure_ascii=False)
|
||||
else:
|
||||
result[key] = str(value)
|
||||
return result
|
||||
|
||||
@@ -1,57 +0,0 @@
|
||||
"""Controller session decorators.
|
||||
|
||||
`with_session` is an HTTP controller helper: it opens one SQLAlchemy session
|
||||
for a Resource handler and injects it as the first argument after `self`.
|
||||
Handlers use a transaction by default so migrated write paths keep
|
||||
commit/rollback handling; pure read handlers may opt out with `write=False`.
|
||||
"""
|
||||
|
||||
from collections.abc import Callable
|
||||
from functools import wraps
|
||||
from typing import Concatenate, overload
|
||||
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from core.db.session_factory import session_factory
|
||||
|
||||
|
||||
@overload
|
||||
def with_session[T, **P, R](
|
||||
view: Callable[Concatenate[T, Session, P], R],
|
||||
*,
|
||||
write: bool = True,
|
||||
) -> Callable[Concatenate[T, P], R]: ...
|
||||
|
||||
|
||||
@overload
|
||||
def with_session[T, **P, R](
|
||||
view: None = None,
|
||||
*,
|
||||
write: bool = True,
|
||||
) -> Callable[[Callable[Concatenate[T, Session, P], R]], Callable[Concatenate[T, P], R]]: ...
|
||||
|
||||
|
||||
def with_session[T, **P, R](
|
||||
view: Callable[Concatenate[T, Session, P], R] | None = None,
|
||||
*,
|
||||
write: bool = True,
|
||||
) -> (
|
||||
Callable[Concatenate[T, P], R] | Callable[[Callable[Concatenate[T, Session, P], R]], Callable[Concatenate[T, P], R]]
|
||||
):
|
||||
"""Inject a request-scoped session, using a transaction only for write handlers."""
|
||||
|
||||
def decorator(view: Callable[Concatenate[T, Session, P], R]) -> Callable[Concatenate[T, P], R]:
|
||||
@wraps(view)
|
||||
def wrapper(self: T, *args: P.args, **kwargs: P.kwargs) -> R:
|
||||
if write:
|
||||
with session_factory.get_session_maker().begin() as session:
|
||||
return view(self, session, *args, **kwargs)
|
||||
|
||||
with session_factory.create_session() as session:
|
||||
return view(self, session, *args, **kwargs)
|
||||
|
||||
return wrapper
|
||||
|
||||
if view is None:
|
||||
return decorator
|
||||
return decorator(view)
|
||||
@@ -1,68 +1,9 @@
|
||||
from collections.abc import Callable
|
||||
from functools import wraps
|
||||
|
||||
from sqlalchemy import select
|
||||
from werkzeug.exceptions import Forbidden, NotFound
|
||||
|
||||
from configs import dify_config
|
||||
from core.rbac import RBACPermission, RBACResourceScope
|
||||
from extensions.ext_database import db
|
||||
from libs.login import current_account_with_tenant
|
||||
from models.dataset import Dataset
|
||||
from models.model import App
|
||||
from services.enterprise.rbac_service import RBACService
|
||||
|
||||
__all__ = ["RBACPermission", "RBACResourceScope", "enforce_rbac_access", "rbac_permission_required"]
|
||||
|
||||
|
||||
def enforce_rbac_access(
|
||||
*,
|
||||
tenant_id: str,
|
||||
account_id: str,
|
||||
resource_type: RBACResourceScope,
|
||||
scene: RBACPermission,
|
||||
resource_required: bool = True,
|
||||
path_args: dict[str, object] | None = None,
|
||||
) -> None:
|
||||
"""Enforce enterprise RBAC for an explicit account/tenant pair.
|
||||
|
||||
This is the flask-login-independent core of the RBAC gate so it can run
|
||||
inside request-handling layers that resolve the caller themselves (e.g. the
|
||||
openapi auth pipeline, which has the account on ``AuthData`` before
|
||||
flask-login is mounted).
|
||||
|
||||
No-op when ``RBAC_ENABLED`` is ``False``. For resource-scoped checks the
|
||||
resource ID is taken from ``path_args`` merged with ``request.view_args``;
|
||||
resource ownership short-circuits the check. Raises ``Forbidden`` when
|
||||
access is denied. For workspace-level checks pass ``resource_required=False``
|
||||
so the RBAC request omits ``resource_id``.
|
||||
|
||||
Args:
|
||||
tenant_id: The tenant the access is evaluated against.
|
||||
account_id: The account requesting access.
|
||||
resource_type: The :class:`RBACResourceScope` member (app/dataset/workspace).
|
||||
scene: The :class:`RBACPermission` permission point, e.g. ``RBACPermission.APP_DELETE``.
|
||||
resource_required: Whether a concrete resource ID is required.
|
||||
path_args: Extra path arguments to merge with ``request.view_args``.
|
||||
"""
|
||||
if not dify_config.RBAC_ENABLED:
|
||||
return
|
||||
|
||||
check_resource_type = None if resource_type == RBACResourceScope.WORKSPACE else resource_type
|
||||
resource_id = None
|
||||
if resource_required and check_resource_type:
|
||||
resource_id = _extract_resource_id(resource_type, path_args)
|
||||
if _is_resource_owned_by_current_user(tenant_id, account_id, resource_type, resource_id):
|
||||
return
|
||||
allowed = RBACService.CheckAccess.check(
|
||||
tenant_id,
|
||||
account_id,
|
||||
scene=scene,
|
||||
resource_type=check_resource_type,
|
||||
resource_id=resource_id,
|
||||
)
|
||||
if not allowed:
|
||||
raise Forbidden()
|
||||
__all__ = ["RBACPermission", "RBACResourceScope", "rbac_permission_required"]
|
||||
|
||||
|
||||
def rbac_permission_required[**P, R](
|
||||
@@ -71,96 +12,19 @@ def rbac_permission_required[**P, R](
|
||||
*,
|
||||
resource_required: bool = True,
|
||||
) -> Callable[[Callable[P, R]], Callable[P, R]]:
|
||||
"""Check enterprise RBAC permissions for the current flask-login user.
|
||||
|
||||
When ``RBAC_ENABLED`` is ``False`` the decorator is a no-op and the
|
||||
request passes through unchanged. When enabled it resolves the current
|
||||
account/tenant and delegates to :func:`enforce_rbac_access`, raising
|
||||
``Forbidden`` if access is denied.
|
||||
"""Check enterprise RBAC permissions for the current user.
|
||||
|
||||
Args:
|
||||
resource_type: The :class:`RBACResourceScope` member (app/dataset/workspace).
|
||||
scene: The :class:`RBACPermission` permission point, e.g. ``RBACPermission.APP_DELETE``.
|
||||
scene: The :class:`RBACPermission` permission point.
|
||||
resource_required: Whether a concrete resource ID is required.
|
||||
"""
|
||||
|
||||
def decorator(view: Callable[P, R]) -> Callable[P, R]:
|
||||
@wraps(view)
|
||||
def decorated(*args: P.args, **kwargs: P.kwargs) -> R:
|
||||
if not dify_config.RBAC_ENABLED:
|
||||
return view(*args, **kwargs)
|
||||
|
||||
current_user, current_tenant_id = current_account_with_tenant()
|
||||
enforce_rbac_access(
|
||||
tenant_id=current_tenant_id,
|
||||
account_id=current_user.id,
|
||||
resource_type=resource_type,
|
||||
scene=scene,
|
||||
resource_required=resource_required,
|
||||
path_args=kwargs,
|
||||
)
|
||||
return view(*args, **kwargs)
|
||||
|
||||
return decorated
|
||||
|
||||
return decorator
|
||||
|
||||
|
||||
def _is_resource_owned_by_current_user(
|
||||
tenant_id: str, account_id: str, resource_type: RBACResourceScope, resource_id: str
|
||||
) -> bool:
|
||||
if resource_type == RBACResourceScope.APP:
|
||||
maintainer = db.session.scalar(
|
||||
select(App.maintainer).where(
|
||||
App.id == resource_id,
|
||||
App.tenant_id == tenant_id,
|
||||
App.status == "normal",
|
||||
)
|
||||
)
|
||||
return maintainer == account_id
|
||||
|
||||
if resource_type == RBACResourceScope.DATASET:
|
||||
maintainer = db.session.scalar(
|
||||
select(Dataset.maintainer).where(
|
||||
Dataset.id == resource_id,
|
||||
Dataset.tenant_id == tenant_id,
|
||||
)
|
||||
)
|
||||
return maintainer == account_id
|
||||
|
||||
return False
|
||||
|
||||
|
||||
def _extract_resource_id(resource_type: RBACResourceScope, path_args: dict[str, object] | None = None) -> str:
|
||||
"""Extract the resource ID from matched path arguments.
|
||||
|
||||
Some legacy route classes use neutral names such as ``resource_id`` for
|
||||
app/dataset resources, and Agent App routes use ``agent_id`` as the app id.
|
||||
Dataset endpoints behind a rag-pipeline route contain ``pipeline_id``
|
||||
instead of ``dataset_id``. In that case we look up the associated
|
||||
``Dataset`` row via ``Dataset.pipeline_id``.
|
||||
"""
|
||||
from flask import request
|
||||
|
||||
view_args = request.view_args or {}
|
||||
matched_args = {**view_args, **(path_args or {})}
|
||||
|
||||
if resource_type == RBACResourceScope.APP:
|
||||
app_id = matched_args.get("app_id") or matched_args.get("agent_id") or matched_args.get("resource_id")
|
||||
if not app_id:
|
||||
raise ValueError("Missing app_id in request path")
|
||||
return str(app_id)
|
||||
|
||||
if resource_type == RBACResourceScope.DATASET:
|
||||
dataset_id = matched_args.get("dataset_id") or matched_args.get("resource_id")
|
||||
if dataset_id:
|
||||
return str(dataset_id)
|
||||
|
||||
pipeline_id = matched_args.get("pipeline_id")
|
||||
if pipeline_id:
|
||||
dataset = db.session.scalar(select(Dataset).where(Dataset.pipeline_id == str(pipeline_id)))
|
||||
if not dataset:
|
||||
raise NotFound("Dataset not found for pipeline")
|
||||
return str(dataset.id)
|
||||
raise ValueError("Missing dataset_id or pipeline_id in request path")
|
||||
raise ValueError(f"Unknown resource_type: {resource_type}")
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user