Compare commits

..
Author SHA1 Message Date
Stephen Zhou c2fc4f4822 Use node 22 2026-03-03 13:31:35 +08:00
Stephen Zhou 67e0eeefd2 Add permission 2026-03-02 21:36:34 +08:00
Stephen Zhou cf58035fa2 Add heapsnapshot-signal 2026-03-02 19:57:16 +08:00
Stephen Zhou 097f10d920 Merge branch 'hotfix/1.13.0-fix.2' into deploy/memory-usage 2026-03-02 19:33:07 +08:00
wangxiaoleiQuantumGhostautofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
c34d05141e fix: fix chat assistant response mode blocking is not work (#32394)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-03-02 18:34:17 +08:00
Stephen Zhou 302701b303 chore: remove serwist 2026-03-02 17:19:15 +08:00
Stephen ZhouandStephen Zhou 8080159eaf fix: remove REDIRECT_URL_KEY from url (#32770) 2026-03-02 16:06:33 +08:00
2663 changed files with 59145 additions and 180414 deletions
-168
View File
@@ -1,168 +0,0 @@
---
name: backend-code-review
description: Review backend code for quality, security, maintainability, and best practices based on established checklist rules. Use when the user requests a review, analysis, or improvement of backend files (e.g., `.py`) under the `api/` directory. Do NOT use for frontend files (e.g., `.tsx`, `.ts`, `.js`). Supports pending-change review, code snippets review, and file-focused review.
---
# Backend Code Review
## When to use this skill
Use this skill whenever the user asks to **review, analyze, or improve** backend code (e.g., `.py`) under the `api/` directory. Supports the following review modes:
- **Pending-change review**: when the user asks to review current changes (inspect staged/working-tree files slated for commit to get the changes).
- **Code snippets review**: when the user pastes code snippets (e.g., a function/class/module excerpt) into the chat and asks for a review.
- **File-focused review**: when the user points to specific files and asks for a review of those files (one file or a small, explicit set of files, e.g., `api/...`, `api/app.py`).
Do NOT use this skill when:
- The request is about frontend code or UI (e.g., `.tsx`, `.ts`, `.js`, `web/`).
- The user is not asking for a review/analysis/improvement of backend code.
- The scope is not under `api/` (unless the user explicitly asks to review backend-related changes outside `api/`).
## How to use this skill
Follow these steps when using this skill:
1. **Identify the review mode** (pending-change vs snippet vs file-focused) based on the users input. Keep the scope tight: review only what the user provided or explicitly referenced.
2. Follow the rules defined in **Checklist** to perform the review. If no Checklist rule matches, apply **General Review Rules** as a fallback to perform the best-effort review.
3. Compose the final output strictly follow the **Required Output Format**.
Notes when using this skill:
- Always include actionable fixes or suggestions (including possible code snippets).
- Use best-effort `File:Line` references when a file path and line numbers are available; otherwise, use the most specific identifier you can.
## Checklist
- db schema design: if the review scope includes code/files under `api/models/` or `api/migrations/`, follow [references/db-schema-rule.md](references/db-schema-rule.md) to perform the review
- architecture: if the review scope involves controller/service/core-domain/libs/model layering, dependency direction, or moving responsibilities across modules, follow [references/architecture-rule.md](references/architecture-rule.md) to perform the review
- repositories abstraction: if the review scope contains table/model operations (e.g., `select(...)`, `session.execute(...)`, joins, CRUD) and is not under `api/repositories`, `api/core/repositories`, or `api/extensions/*/repositories/`, follow [references/repositories-rule.md](references/repositories-rule.md) to perform the review
- sqlalchemy patterns: if the review scope involves SQLAlchemy session/query usage, db transaction/crud usage, or raw SQL usage, follow [references/sqlalchemy-rule.md](references/sqlalchemy-rule.md) to perform the review
## General Review Rules
### 1. Security Review
Check for:
- SQL injection vulnerabilities
- Server-Side Request Forgery (SSRF)
- Command injection
- Insecure deserialization
- Hardcoded secrets/credentials
- Improper authentication/authorization
- Insecure direct object references
### 2. Performance Review
Check for:
- N+1 queries
- Missing database indexes
- Memory leaks
- Blocking operations in async code
- Missing caching opportunities
### 3. Code Quality Review
Check for:
- Code forward compatibility
- Code duplication (DRY violations)
- Functions doing too much (SRP violations)
- Deep nesting / complex conditionals
- Magic numbers/strings
- Poor naming
- Missing error handling
- Incomplete type coverage
### 4. Testing Review
Check for:
- Missing test coverage for new code
- Tests that don't test behavior
- Flaky test patterns
- Missing edge cases
## Required Output Format
When this skill invoked, the response must exactly follow one of the two templates:
### Template A (any findings)
```markdown
# Code Review Summary
Found <X> critical issues need to be fixed:
## 🔴 Critical (Must Fix)
### 1. <brief description of the issue>
FilePath: <path> line <line>
<relevant code snippet or pointer>
#### Explanation
<detailed explanation and references of the issue>
#### Suggested Fix
1. <brief description of suggested fix>
2. <code example> (optional, omit if not applicable)
---
... (repeat for each critical issue) ...
Found <Y> suggestions for improvement:
## 🟡 Suggestions (Should Consider)
### 1. <brief description of the suggestion>
FilePath: <path> line <line>
<relevant code snippet or pointer>
#### Explanation
<detailed explanation and references of the suggestion>
#### Suggested Fix
1. <brief description of suggested fix>
2. <code example> (optional, omit if not applicable)
---
... (repeat for each suggestion) ...
Found <Z> optional nits:
## 🟢 Nits (Optional)
### 1. <brief description of the nit>
FilePath: <path> line <line>
<relevant code snippet or pointer>
#### Explanation
<explanation and references of the optional nit>
#### Suggested Fix
- <minor suggestions>
---
... (repeat for each nits) ...
## ✅ What's Good
- <Positive feedback on good patterns>
```
- If there are no critical issues or suggestions or option nits or good points, just omit that section.
- If the issue number is more than 10, summarize as "Found 10+ critical issues/suggestions/optional nits" and only output the first 10 items.
- Don't compress the blank lines between sections; keep them as-is for readability.
- If there is any issue requires code changes, append a brief follow-up question to ask whether the user wants to apply the fix(es) after the structured output. For example: "Would you like me to use the Suggested fix(es) to address these issues?"
### Template B (no issues)
```markdown
## Code Review Summary
✅ No issues found.
```
@@ -1,91 +0,0 @@
# Rule Catalog — Architecture
## Scope
- Covers: controller/service/core-domain/libs/model layering, dependency direction, responsibility placement, observability-friendly flow.
## Rules
### Keep business logic out of controllers
- Category: maintainability
- Severity: critical
- Description: Controllers should parse input, call services, and return serialized responses. Business decisions inside controllers make behavior hard to reuse and test.
- Suggested fix: Move domain/business logic into the service or core/domain layer. Keep controller handlers thin and orchestration-focused.
- Example:
- Bad:
```python
@bp.post("/apps/<app_id>/publish")
def publish_app(app_id: str):
payload = request.get_json() or {}
if payload.get("force") and current_user.role != "admin":
raise ValueError("only admin can force publish")
app = App.query.get(app_id)
app.status = "published"
db.session.commit()
return {"result": "ok"}
```
- Good:
```python
@bp.post("/apps/<app_id>/publish")
def publish_app(app_id: str):
payload = PublishRequest.model_validate(request.get_json() or {})
app_service.publish_app(app_id=app_id, force=payload.force, actor_id=current_user.id)
return {"result": "ok"}
```
### Preserve layer dependency direction
- Category: best practices
- Severity: critical
- Description: Controllers may depend on services, and services may depend on core/domain abstractions. Reversing this direction (for example, core importing controller/web modules) creates cycles and leaks transport concerns into domain code.
- Suggested fix: Extract shared contracts into core/domain or service-level modules and make upper layers depend on lower, not the reverse.
- Example:
- Bad:
```python
# core/policy/publish_policy.py
from controllers.console.app import request_context
def can_publish() -> bool:
return request_context.current_user.is_admin
```
- Good:
```python
# core/policy/publish_policy.py
def can_publish(role: str) -> bool:
return role == "admin"
# service layer adapts web/user context to domain input
allowed = can_publish(role=current_user.role)
```
### Keep libs business-agnostic
- Category: maintainability
- Severity: critical
- Description: Modules under `api/libs/` should remain reusable, business-agnostic building blocks. They must not encode product/domain-specific rules, workflow orchestration, or business decisions.
- Suggested fix:
- If business logic appears in `api/libs/`, extract it into the appropriate `services/` or `core/` module and keep `libs` focused on generic, cross-cutting helpers.
- Keep `libs` dependencies clean: avoid importing service/controller/domain-specific modules into `api/libs/`.
- Example:
- Bad:
```python
# api/libs/conversation_filter.py
from services.conversation_service import ConversationService
def should_archive_conversation(conversation, tenant_id: str) -> bool:
# Domain policy and service dependency are leaking into libs.
service = ConversationService()
if service.has_paid_plan(tenant_id):
return conversation.idle_days > 90
return conversation.idle_days > 30
```
- Good:
```python
# api/libs/datetime_utils.py (business-agnostic helper)
def older_than_days(idle_days: int, threshold_days: int) -> bool:
return idle_days > threshold_days
# services/conversation_service.py (business logic stays in service/core)
from libs.datetime_utils import older_than_days
def should_archive_conversation(conversation, tenant_id: str) -> bool:
threshold_days = 90 if has_paid_plan(tenant_id) else 30
return older_than_days(conversation.idle_days, threshold_days)
```
@@ -1,157 +0,0 @@
# Rule Catalog — DB Schema Design
## Scope
- Covers: model/base inheritance, schema boundaries in model properties, tenant-aware schema design, index redundancy checks, dialect portability in models, and cross-database compatibility in migrations.
- Does NOT cover: session lifecycle, transaction boundaries, and query execution patterns (handled by `sqlalchemy-rule.md`).
## Rules
### Do not query other tables inside `@property`
- Category: [maintainability, performance]
- Severity: critical
- Description: A model `@property` must not open sessions or query other tables. This hides dependencies across models, tightly couples schema objects to data access, and can cause N+1 query explosions when iterating collections.
- Suggested fix:
- Keep model properties pure and local to already-loaded fields.
- Move cross-table data fetching to service/repository methods.
- For list/batch reads, fetch required related data explicitly (join/preload/bulk query) before rendering derived values.
- Example:
- Bad:
```python
class Conversation(TypeBase):
__tablename__ = "conversations"
@property
def app_name(self) -> str:
with Session(db.engine, expire_on_commit=False) as session:
app = session.execute(select(App).where(App.id == self.app_id)).scalar_one()
return app.name
```
- Good:
```python
class Conversation(TypeBase):
__tablename__ = "conversations"
@property
def display_title(self) -> str:
return self.name or "Untitled"
# Service/repository layer performs explicit batch fetch for related App rows.
```
### Prefer including `tenant_id` in model definitions
- Category: maintainability
- Severity: suggestion
- Description: In multi-tenant domains, include `tenant_id` in schema definitions whenever the entity belongs to tenant-owned data. This improves data isolation safety and keeps future partitioning/sharding strategies practical as data volume grows.
- Suggested fix:
- Add a `tenant_id` column and ensure related unique/index constraints include tenant dimension when applicable.
- Propagate `tenant_id` through service/repository contracts to keep access paths tenant-aware.
- Exception: if a table is explicitly designed as non-tenant-scoped global metadata, document that design decision clearly.
- Example:
- Bad:
```python
from sqlalchemy.orm import Mapped
class Dataset(TypeBase):
__tablename__ = "datasets"
id: Mapped[str] = mapped_column(StringUUID, primary_key=True)
name: Mapped[str] = mapped_column(sa.String(255), nullable=False)
```
- Good:
```python
from sqlalchemy.orm import Mapped
class Dataset(TypeBase):
__tablename__ = "datasets"
id: Mapped[str] = mapped_column(StringUUID, primary_key=True)
tenant_id: Mapped[str] = mapped_column(StringUUID, nullable=False, index=True)
name: Mapped[str] = mapped_column(sa.String(255), nullable=False)
```
### Detect and avoid duplicate/redundant indexes
- Category: performance
- Severity: suggestion
- Description: Review index definitions for leftmost-prefix redundancy. For example, index `(a, b, c)` can safely cover most lookups for `(a, b)`. Keeping both may increase write overhead and can mislead the optimizer into suboptimal execution plans.
- Suggested fix:
- Before adding an index, compare against existing composite indexes by leftmost-prefix rules.
- Drop or avoid creating redundant prefixes unless there is a proven query-pattern need.
- Apply the same review standard in both model `__table_args__` and migration index DDL.
- Example:
- Bad:
```python
__table_args__ = (
sa.Index("idx_msg_tenant_app", "tenant_id", "app_id"),
sa.Index("idx_msg_tenant_app_created", "tenant_id", "app_id", "created_at"),
)
```
- Good:
```python
__table_args__ = (
# Keep the wider index unless profiling proves a dedicated short index is needed.
sa.Index("idx_msg_tenant_app_created", "tenant_id", "app_id", "created_at"),
)
```
### Avoid PostgreSQL-only dialect usage in models; wrap in `models.types`
- Category: maintainability
- Severity: critical
- Description: Model/schema definitions should avoid PostgreSQL-only constructs directly in business models. When database-specific behavior is required, encapsulate it in `api/models/types.py` using both PostgreSQL and MySQL dialect implementations, then consume that abstraction from model code.
- Suggested fix:
- Do not directly place dialect-only types/operators in model columns when a portable wrapper can be used.
- Add or extend wrappers in `models.types` (for example, `AdjustedJSON`, `LongText`, `BinaryData`) to normalize behavior across PostgreSQL and MySQL.
- Example:
- Bad:
```python
from sqlalchemy.dialects.postgresql import JSONB
from sqlalchemy.orm import Mapped
class ToolConfig(TypeBase):
__tablename__ = "tool_configs"
config: Mapped[dict] = mapped_column(JSONB, nullable=False)
```
- Good:
```python
from sqlalchemy.orm import Mapped
from models.types import AdjustedJSON
class ToolConfig(TypeBase):
__tablename__ = "tool_configs"
config: Mapped[dict] = mapped_column(AdjustedJSON(), nullable=False)
```
### Guard migration incompatibilities with dialect checks and shared types
- Category: maintainability
- Severity: critical
- Description: Migration scripts under `api/migrations/versions/` must account for PostgreSQL/MySQL incompatibilities explicitly. For dialect-sensitive DDL or defaults, branch on the active dialect (for example, `conn.dialect.name == "postgresql"`), and prefer reusable compatibility abstractions from `models.types` where applicable.
- Suggested fix:
- In migration upgrades/downgrades, bind connection and branch by dialect for incompatible SQL fragments.
- Reuse `models.types` wrappers in column definitions when that keeps behavior aligned with runtime models.
- Avoid one-dialect-only migration logic unless there is a documented, deliberate compatibility exception.
- Example:
- Bad:
```python
with op.batch_alter_table("dataset_keyword_tables") as batch_op:
batch_op.add_column(
sa.Column(
"data_source_type",
sa.String(255),
server_default=sa.text("'database'::character varying"),
nullable=False,
)
)
```
- Good:
```python
def _is_pg(conn) -> bool:
return conn.dialect.name == "postgresql"
conn = op.get_bind()
default_expr = sa.text("'database'::character varying") if _is_pg(conn) else sa.text("'database'")
with op.batch_alter_table("dataset_keyword_tables") as batch_op:
batch_op.add_column(
sa.Column("data_source_type", sa.String(255), server_default=default_expr, nullable=False)
)
```
@@ -1,61 +0,0 @@
# Rule Catalog - Repositories Abstraction
## Scope
- Covers: when to reuse existing repository abstractions, when to introduce new repositories, and how to preserve dependency direction between service/core and infrastructure implementations.
- Does NOT cover: SQLAlchemy session lifecycle and query-shape specifics (handled by `sqlalchemy-rule.md`), and table schema/migration design (handled by `db-schema-rule.md`).
## Rules
### Introduce repositories abstraction
- Category: maintainability
- Severity: suggestion
- Description: If a table/model already has a repository abstraction, all reads/writes/queries for that table should use the existing repository. If no repository exists, introduce one only when complexity justifies it, such as large/high-volume tables, repeated complex query logic, or likely storage-strategy variation.
- Suggested fix:
- First check `api/repositories`, `api/core/repositories`, and `api/extensions/*/repositories/` to verify whether the table/model already has a repository abstraction. If it exists, route all operations through it and add missing repository methods instead of bypassing it with ad-hoc SQLAlchemy access.
- If no repository exists, add one only when complexity warrants it (for example, repeated complex queries, large data domains, or multiple storage strategies), while preserving dependency direction (service/core depends on abstraction; infra provides implementation).
- Example:
- Bad:
```python
# Existing repository is ignored and service uses ad-hoc table queries.
class AppService:
def archive_app(self, app_id: str, tenant_id: str) -> None:
app = self.session.execute(
select(App).where(App.id == app_id, App.tenant_id == tenant_id)
).scalar_one()
app.archived = True
self.session.commit()
```
- Good:
```python
# Case A: Existing repository must be reused for all table operations.
class AppService:
def archive_app(self, app_id: str, tenant_id: str) -> None:
app = self.app_repo.get_by_id(app_id=app_id, tenant_id=tenant_id)
app.archived = True
self.app_repo.save(app)
# If the query is missing, extend the existing abstraction.
active_apps = self.app_repo.list_active_for_tenant(tenant_id=tenant_id)
```
- Bad:
```python
# No repository exists, but large-domain query logic is scattered in service code.
class ConversationService:
def list_recent_for_app(self, app_id: str, tenant_id: str, limit: int) -> list[Conversation]:
...
# many filters/joins/pagination variants duplicated across services
```
- Good:
```python
# Case B: Introduce repository for large/complex domains or storage variation.
class ConversationRepository(Protocol):
def list_recent_for_app(self, app_id: str, tenant_id: str, limit: int) -> list[Conversation]: ...
class SqlAlchemyConversationRepository:
def list_recent_for_app(self, app_id: str, tenant_id: str, limit: int) -> list[Conversation]:
...
class ConversationService:
def __init__(self, conversation_repo: ConversationRepository):
self.conversation_repo = conversation_repo
```
@@ -1,139 +0,0 @@
# Rule Catalog — SQLAlchemy Patterns
## Scope
- Covers: SQLAlchemy session and transaction lifecycle, query construction, tenant scoping, raw SQL boundaries, and write-path concurrency safeguards.
- Does NOT cover: table/model schema and migration design details (handled by `db-schema-rule.md`).
## Rules
### Use Session context manager with explicit transaction control behavior
- Category: best practices
- Severity: critical
- Description: Session and transaction lifecycle must be explicit and bounded on write paths. Missing commits can silently drop intended updates, while ad-hoc or long-lived transactions increase contention, lock duration, and deadlock risk.
- Suggested fix:
- Use **explicit `session.commit()`** after completing a related write unit.
- Or use **`session.begin()` context manager** for automatic commit/rollback on a scoped block.
- Keep transaction windows short: avoid network I/O, heavy computation, or unrelated work inside the transaction.
- Example:
- Bad:
```python
# Missing commit: write may never be persisted.
with Session(db.engine, expire_on_commit=False) as session:
run = session.get(WorkflowRun, run_id)
run.status = "cancelled"
# Long transaction: external I/O inside a DB transaction.
with Session(db.engine, expire_on_commit=False) as session, session.begin():
run = session.get(WorkflowRun, run_id)
run.status = "cancelled"
call_external_api()
```
- Good:
```python
# Option 1: explicit commit.
with Session(db.engine, expire_on_commit=False) as session:
run = session.get(WorkflowRun, run_id)
run.status = "cancelled"
session.commit()
# Option 2: scoped transaction with automatic commit/rollback.
with Session(db.engine, expire_on_commit=False) as session, session.begin():
run = session.get(WorkflowRun, run_id)
run.status = "cancelled"
# Keep non-DB work outside transaction scope.
call_external_api()
```
### Enforce tenant_id scoping on shared-resource queries
- Category: security
- Severity: critical
- Description: Reads and writes against shared tables must be scoped by `tenant_id` to prevent cross-tenant data leakage or corruption.
- Suggested fix: Add `tenant_id` predicate to all tenant-owned entity queries and propagate tenant context through service/repository interfaces.
- Example:
- Bad:
```python
stmt = select(Workflow).where(Workflow.id == workflow_id)
workflow = session.execute(stmt).scalar_one_or_none()
```
- Good:
```python
stmt = select(Workflow).where(
Workflow.id == workflow_id,
Workflow.tenant_id == tenant_id,
)
workflow = session.execute(stmt).scalar_one_or_none()
```
### Prefer SQLAlchemy expressions over raw SQL by default
- Category: maintainability
- Severity: suggestion
- Description: Raw SQL should be exceptional. ORM/Core expressions are easier to evolve, safer to compose, and more consistent with the codebase.
- Suggested fix: Rewrite straightforward raw SQL into SQLAlchemy `select/update/delete` expressions; keep raw SQL only when required by clear technical constraints.
- Example:
- Bad:
```python
row = session.execute(
text("SELECT * FROM workflows WHERE id = :id AND tenant_id = :tenant_id"),
{"id": workflow_id, "tenant_id": tenant_id},
).first()
```
- Good:
```python
stmt = select(Workflow).where(
Workflow.id == workflow_id,
Workflow.tenant_id == tenant_id,
)
row = session.execute(stmt).scalar_one_or_none()
```
### Protect write paths with concurrency safeguards
- Category: quality
- Severity: critical
- Description: Multi-writer paths without explicit concurrency control can silently overwrite data. Choose the safeguard based on contention level, lock scope, and throughput cost instead of defaulting to one strategy.
- Suggested fix:
- **Optimistic locking**: Use when contention is usually low and retries are acceptable. Add a version (or updated_at) guard in `WHERE` and treat `rowcount == 0` as a conflict.
- **Redis distributed lock**: Use when the critical section spans multiple steps/processes (or includes non-DB side effects) and you need cross-worker mutual exclusion.
- **SELECT ... FOR UPDATE**: Use when contention is high on the same rows and strict in-transaction serialization is required. Keep transactions short to reduce lock wait/deadlock risk.
- In all cases, scope by `tenant_id` and verify affected row counts for conditional writes.
- Example:
- Bad:
```python
# No tenant scope, no conflict detection, and no lock on a contested write path.
session.execute(update(WorkflowRun).where(WorkflowRun.id == run_id).values(status="cancelled"))
session.commit() # silently overwrites concurrent updates
```
- Good:
```python
# 1) Optimistic lock (low contention, retry on conflict)
result = session.execute(
update(WorkflowRun)
.where(
WorkflowRun.id == run_id,
WorkflowRun.tenant_id == tenant_id,
WorkflowRun.version == expected_version,
)
.values(status="cancelled", version=WorkflowRun.version + 1)
)
if result.rowcount == 0:
raise WorkflowStateConflictError("stale version, retry")
# 2) Redis distributed lock (cross-worker critical section)
lock_name = f"workflow_run_lock:{tenant_id}:{run_id}"
with redis_client.lock(lock_name, timeout=20):
session.execute(
update(WorkflowRun)
.where(WorkflowRun.id == run_id, WorkflowRun.tenant_id == tenant_id)
.values(status="cancelled")
)
session.commit()
# 3) Pessimistic lock with SELECT ... FOR UPDATE (high contention)
run = session.execute(
select(WorkflowRun)
.where(WorkflowRun.id == run_id, WorkflowRun.tenant_id == tenant_id)
.with_for_update()
).scalar_one()
run.status = "cancelled"
session.commit()
```
-1
View File
@@ -1 +0,0 @@
../../.agents/skills/backend-code-review
+1 -1
View File
@@ -36,7 +36,7 @@
/api/core/workflow/graph/ @laipz8200 @QuantumGhost /api/core/workflow/graph/ @laipz8200 @QuantumGhost
/api/core/workflow/graph_events/ @laipz8200 @QuantumGhost /api/core/workflow/graph_events/ @laipz8200 @QuantumGhost
/api/core/workflow/node_events/ @laipz8200 @QuantumGhost /api/core/workflow/node_events/ @laipz8200 @QuantumGhost
/api/dify_graph/model_runtime/ @laipz8200 @QuantumGhost /api/core/model_runtime/ @laipz8200 @QuantumGhost
# Backend - Workflow - Nodes (Agent, Iteration, Loop, LLM) # Backend - Workflow - Nodes (Agent, Iteration, Loop, LLM)
/api/core/workflow/nodes/agent/ @Nov1c444 /api/core/workflow/nodes/agent/ @Nov1c444
+5 -18
View File
@@ -1,25 +1,12 @@
version: 2 version: 2
multi-ecosystem-groups:
python:
schedule:
interval: "weekly" # or whatever schedule you want
updates: updates:
- package-ecosystem: "pip"
directory: "/api"
open-pull-requests-limit: 2
patterns: ["*"]
schedule:
interval: "weekly"
- package-ecosystem: "uv"
directory: "/api"
open-pull-requests-limit: 2
patterns: ["*"]
schedule:
interval: "weekly"
- package-ecosystem: "npm" - package-ecosystem: "npm"
directory: "/web" directory: "/web"
schedule: schedule:
interval: "weekly" interval: "weekly"
open-pull-requests-limit: 2 open-pull-requests-limit: 2
- package-ecosystem: "uv"
directory: "/api"
schedule:
interval: "weekly"
open-pull-requests-limit: 2
@@ -1,88 +0,0 @@
name: Comment with Pyrefly Diff
on:
workflow_run:
workflows:
- Pyrefly Diff Check
types:
- completed
permissions: {}
jobs:
comment:
name: Comment PR with pyrefly diff
runs-on: ubuntu-latest
permissions:
actions: read
contents: read
issues: write
pull-requests: write
if: ${{ github.event.workflow_run.conclusion == 'success' && github.event.workflow_run.pull_requests[0].head.repo.full_name != github.repository }}
steps:
- name: Download pyrefly diff artifact
uses: actions/github-script@v8
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
script: |
const fs = require('fs');
const artifacts = await github.rest.actions.listWorkflowRunArtifacts({
owner: context.repo.owner,
repo: context.repo.repo,
run_id: ${{ github.event.workflow_run.id }},
});
const match = artifacts.data.artifacts.find((artifact) =>
artifact.name === 'pyrefly_diff'
);
if (!match) {
throw new Error('pyrefly_diff artifact not found');
}
const download = await github.rest.actions.downloadArtifact({
owner: context.repo.owner,
repo: context.repo.repo,
artifact_id: match.id,
archive_format: 'zip',
});
fs.writeFileSync('pyrefly_diff.zip', Buffer.from(download.data));
- name: Unzip artifact
run: unzip -o pyrefly_diff.zip
- name: Post comment
uses: actions/github-script@v8
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
script: |
const fs = require('fs');
let diff = fs.readFileSync('pyrefly_diff.txt', { encoding: 'utf8' });
let prNumber = null;
try {
prNumber = parseInt(fs.readFileSync('pr_number.txt', { encoding: 'utf8' }), 10);
} catch (err) {
// Fallback to workflow_run payload if artifact is missing or incomplete.
const prs = context.payload.workflow_run.pull_requests || [];
if (prs.length > 0 && prs[0].number) {
prNumber = prs[0].number;
}
}
if (!prNumber) {
throw new Error('PR number not found in artifact or workflow_run payload');
}
const MAX_CHARS = 65000;
if (diff.length > MAX_CHARS) {
diff = diff.slice(0, MAX_CHARS);
diff = diff.slice(0, diff.lastIndexOf('\\n'));
diff += '\\n\\n... (truncated) ...';
}
const body = diff.trim()
? '### Pyrefly Diff\n<details>\n<summary>base → PR</summary>\n\n```diff\n' + diff + '\n```\n</details>'
: '### Pyrefly Diff\nNo changes detected.';
await github.rest.issues.createComment({
issue_number: prNumber,
owner: context.repo.owner,
repo: context.repo.repo,
body,
});
-100
View File
@@ -1,100 +0,0 @@
name: Pyrefly Diff Check
on:
pull_request:
paths:
- 'api/**/*.py'
permissions:
contents: read
jobs:
pyrefly-diff:
runs-on: ubuntu-latest
permissions:
contents: read
issues: write
pull-requests: write
steps:
- name: Checkout PR branch
uses: actions/checkout@v6
with:
fetch-depth: 0
- name: Setup Python & UV
uses: astral-sh/setup-uv@v5
with:
enable-cache: true
- name: Install dependencies
run: uv sync --project api --dev
- name: Prepare diagnostics extractor
run: |
git show ${{ github.event.pull_request.head.sha }}:api/libs/pyrefly_diagnostics.py > /tmp/pyrefly_diagnostics.py
- name: Run pyrefly on PR branch
run: |
uv run --directory api --dev pyrefly check 2>&1 \
| uv run --directory api python /tmp/pyrefly_diagnostics.py > /tmp/pyrefly_pr.txt || true
- name: Checkout base branch
run: git checkout ${{ github.base_ref }}
- name: Run pyrefly on base branch
run: |
uv run --directory api --dev pyrefly check 2>&1 \
| uv run --directory api python /tmp/pyrefly_diagnostics.py > /tmp/pyrefly_base.txt || true
- name: Compute diff
run: |
diff -u /tmp/pyrefly_base.txt /tmp/pyrefly_pr.txt > pyrefly_diff.txt || true
- name: Save PR number
run: |
echo ${{ github.event.pull_request.number }} > pr_number.txt
- name: Upload pyrefly diff
uses: actions/upload-artifact@v4
with:
name: pyrefly_diff
path: |
pyrefly_diff.txt
pr_number.txt
- name: Comment PR with pyrefly diff
if: ${{ github.event.pull_request.head.repo.full_name == github.repository }}
uses: actions/github-script@v8
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
script: |
const fs = require('fs');
let diff = fs.readFileSync('pyrefly_diff.txt', { encoding: 'utf8' });
const prNumber = context.payload.pull_request.number;
const MAX_CHARS = 65000;
if (diff.length > MAX_CHARS) {
diff = diff.slice(0, MAX_CHARS);
diff = diff.slice(0, diff.lastIndexOf('\n'));
diff += '\n\n... (truncated) ...';
}
const body = diff.trim()
? [
'### Pyrefly Diff',
'<details>',
'<summary>base → PR</summary>',
'',
'```diff',
diff,
'```',
'</details>',
].join('\n')
: '### Pyrefly Diff\nNo changes detected.';
await github.rest.issues.createComment({
issue_number: prNumber,
owner: context.repo.owner,
repo: context.repo.repo,
body,
});
+2 -61
View File
@@ -3,22 +3,14 @@ name: Web Tests
on: on:
workflow_call: workflow_call:
permissions:
contents: read
concurrency: concurrency:
group: web-tests-${{ github.head_ref || github.run_id }} group: web-tests-${{ github.head_ref || github.run_id }}
cancel-in-progress: true cancel-in-progress: true
jobs: jobs:
test: test:
name: Web Tests (${{ matrix.shardIndex }}/${{ matrix.shardTotal }}) name: Web Tests
runs-on: ubuntu-latest runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
shardIndex: [1, 2, 3, 4]
shardTotal: [4]
defaults: defaults:
run: run:
shell: bash shell: bash
@@ -47,58 +39,7 @@ jobs:
run: pnpm install --frozen-lockfile run: pnpm install --frozen-lockfile
- name: Run tests - name: Run tests
run: pnpm vitest run --reporter=blob --shard=${{ matrix.shardIndex }}/${{ matrix.shardTotal }} --coverage run: pnpm test:ci
- name: Upload blob report
if: ${{ !cancelled() }}
uses: actions/upload-artifact@v6
with:
name: blob-report-${{ matrix.shardIndex }}
path: web/.vitest-reports/*
include-hidden-files: true
retention-days: 1
merge-reports:
name: Merge Test Reports
if: ${{ !cancelled() }}
needs: [test]
runs-on: ubuntu-latest
defaults:
run:
shell: bash
working-directory: ./web
steps:
- name: Checkout code
uses: actions/checkout@v6
with:
persist-credentials: false
- name: Install pnpm
uses: pnpm/action-setup@v4
with:
package_json_file: web/package.json
run_install: false
- name: Setup Node.js
uses: actions/setup-node@v6
with:
node-version: 24
cache: pnpm
cache-dependency-path: ./web/pnpm-lock.yaml
- name: Install dependencies
run: pnpm install --frozen-lockfile
- name: Download blob reports
uses: actions/download-artifact@v6
with:
path: web/.vitest-reports
pattern: blob-report-*
merge-multiple: true
- name: Merge reports
run: pnpm vitest --merge-reports --coverage --silent=passed-only
- name: Coverage Summary - name: Coverage Summary
if: always() if: always()
+3 -2
View File
@@ -68,9 +68,10 @@ lint:
@echo "✅ Linting complete" @echo "✅ Linting complete"
type-check: type-check:
@echo "📝 Running type checks (basedpyright + mypy)..." @echo "📝 Running type checks (basedpyright + mypy + ty)..."
@./dev/basedpyright-check $(PATH_TO_CHECK) @./dev/basedpyright-check $(PATH_TO_CHECK)
@uv --directory api run mypy --exclude-gitignore --exclude 'tests/' --exclude 'migrations/' --check-untyped-defs --disable-error-code=import-untyped . @uv --directory api run mypy --exclude-gitignore --exclude 'tests/' --exclude 'migrations/' --check-untyped-defs --disable-error-code=import-untyped .
@cd api && uv run ty check
@echo "✅ Type checks complete" @echo "✅ Type checks complete"
test: test:
@@ -131,7 +132,7 @@ help:
@echo " make format - Format code with ruff" @echo " make format - Format code with ruff"
@echo " make check - Check code with ruff" @echo " make check - Check code with ruff"
@echo " make lint - Format, fix, and lint code (ruff, imports, dotenv)" @echo " make lint - Format, fix, and lint code (ruff, imports, dotenv)"
@echo " make type-check - Run type checks (basedpyright, mypy)" @echo " make type-check - Run type checks (basedpyright, mypy, ty)"
@echo " make test - Run backend unit tests (or TARGET_TESTS=./api/tests/<target_tests>)" @echo " make test - Run backend unit tests (or TARGET_TESTS=./api/tests/<target_tests>)"
@echo "" @echo ""
@echo "Docker Build Targets:" @echo "Docker Build Targets:"
+4
View File
@@ -1,5 +1,9 @@
![cover-v5-optimized](./images/GitHub_README_if.png) ![cover-v5-optimized](./images/GitHub_README_if.png)
<p align="center">
📌 <a href="https://dify.ai/blog/introducing-dify-workflow-file-upload-a-demo-on-ai-podcast">Introducing Dify Workflow File Upload: Recreate Google NotebookLM Podcast</a>
</p>
<p align="center"> <p align="center">
<a href="https://cloud.dify.ai">Dify Cloud</a> · <a href="https://cloud.dify.ai">Dify Cloud</a> ·
<a href="https://docs.dify.ai/getting-started/install-self-hosted">Self-hosting</a> · <a href="https://docs.dify.ai/getting-started/install-self-hosted">Self-hosting</a> ·
-2
View File
@@ -553,8 +553,6 @@ WORKFLOW_LOG_CLEANUP_ENABLED=false
WORKFLOW_LOG_RETENTION_DAYS=30 WORKFLOW_LOG_RETENTION_DAYS=30
# Batch size for workflow log cleanup operations (default: 100) # Batch size for workflow log cleanup operations (default: 100)
WORKFLOW_LOG_CLEANUP_BATCH_SIZE=100 WORKFLOW_LOG_CLEANUP_BATCH_SIZE=100
# Comma-separated list of workflow IDs to clean logs for
WORKFLOW_LOG_CLEANUP_SPECIFIC_WORKFLOW_IDS=
# App configuration # App configuration
APP_MAX_EXECUTION_TIME=1200 APP_MAX_EXECUTION_TIME=1200
+283 -104
View File
@@ -1,7 +1,6 @@
[importlinter] [importlinter]
root_packages = root_packages =
core core
dify_graph
configs configs
controllers controllers
extensions extensions
@@ -22,50 +21,51 @@ layers =
runtime runtime
entities entities
containers = containers =
dify_graph core.workflow
ignore_imports = ignore_imports =
dify_graph.nodes.base.node -> dify_graph.graph_events core.workflow.nodes.base.node -> core.workflow.graph_events
dify_graph.nodes.iteration.iteration_node -> dify_graph.graph_events core.workflow.nodes.iteration.iteration_node -> core.workflow.graph_events
dify_graph.nodes.loop.loop_node -> dify_graph.graph_events core.workflow.nodes.loop.loop_node -> core.workflow.graph_events
dify_graph.nodes.iteration.iteration_node -> core.workflow.node_factory core.workflow.nodes.iteration.iteration_node -> core.app.workflow.node_factory
dify_graph.nodes.loop.loop_node -> core.workflow.node_factory core.workflow.nodes.loop.loop_node -> core.app.workflow.node_factory
dify_graph.nodes.iteration.iteration_node -> core.app.workflow.layers.llm_quota
dify_graph.nodes.loop.loop_node -> core.app.workflow.layers.llm_quota
dify_graph.nodes.iteration.iteration_node -> dify_graph.graph_engine core.workflow.nodes.iteration.iteration_node -> core.workflow.graph_engine
dify_graph.nodes.iteration.iteration_node -> dify_graph.graph core.workflow.nodes.iteration.iteration_node -> core.workflow.graph
dify_graph.nodes.iteration.iteration_node -> dify_graph.graph_engine.command_channels core.workflow.nodes.iteration.iteration_node -> core.workflow.graph_engine.command_channels
dify_graph.nodes.loop.loop_node -> dify_graph.graph_engine core.workflow.nodes.loop.loop_node -> core.workflow.graph_engine
dify_graph.nodes.loop.loop_node -> dify_graph.graph core.workflow.nodes.loop.loop_node -> core.workflow.graph
dify_graph.nodes.loop.loop_node -> dify_graph.graph_engine.command_channels core.workflow.nodes.loop.loop_node -> core.workflow.graph_engine.command_channels
# TODO(QuantumGhost): fix the import violation later # TODO(QuantumGhost): fix the import violation later
dify_graph.entities.pause_reason -> dify_graph.nodes.human_input.entities core.workflow.entities.pause_reason -> core.workflow.nodes.human_input.entities
[importlinter:contract:workflow-infrastructure-dependencies] [importlinter:contract:workflow-infrastructure-dependencies]
name = Workflow Infrastructure Dependencies name = Workflow Infrastructure Dependencies
type = forbidden type = forbidden
source_modules = source_modules =
dify_graph core.workflow
forbidden_modules = forbidden_modules =
extensions.ext_database extensions.ext_database
extensions.ext_redis extensions.ext_redis
allow_indirect_imports = True allow_indirect_imports = True
ignore_imports = ignore_imports =
dify_graph.nodes.agent.agent_node -> extensions.ext_database core.workflow.nodes.agent.agent_node -> extensions.ext_database
dify_graph.nodes.llm.file_saver -> extensions.ext_database core.workflow.nodes.datasource.datasource_node -> extensions.ext_database
dify_graph.nodes.llm.node -> extensions.ext_database core.workflow.nodes.knowledge_index.knowledge_index_node -> extensions.ext_database
dify_graph.nodes.tool.tool_node -> extensions.ext_database core.workflow.nodes.llm.file_saver -> extensions.ext_database
dify_graph.model_runtime.model_providers.__base.ai_model -> extensions.ext_redis core.workflow.nodes.llm.llm_utils -> extensions.ext_database
dify_graph.model_runtime.model_providers.model_provider_factory -> extensions.ext_redis core.workflow.nodes.llm.node -> extensions.ext_database
core.workflow.nodes.tool.tool_node -> extensions.ext_database
core.workflow.graph_engine.command_channels.redis_channel -> extensions.ext_redis
core.workflow.graph_engine.manager -> extensions.ext_redis
# TODO(QuantumGhost): use DI to avoid depending on global DB. # TODO(QuantumGhost): use DI to avoid depending on global DB.
dify_graph.nodes.human_input.human_input_node -> extensions.ext_database core.workflow.nodes.human_input.human_input_node -> extensions.ext_database
[importlinter:contract:workflow-external-imports] [importlinter:contract:workflow-external-imports]
name = Workflow External Imports name = Workflow External Imports
type = forbidden type = forbidden
source_modules = source_modules =
dify_graph core.workflow
forbidden_modules = forbidden_modules =
configs configs
controllers controllers
@@ -91,6 +91,7 @@ forbidden_modules =
core.logging core.logging
core.mcp core.mcp
core.memory core.memory
core.model_manager
core.moderation core.moderation
core.ops core.ops
core.plugin core.plugin
@@ -103,70 +104,248 @@ forbidden_modules =
core.trigger core.trigger
core.variables core.variables
ignore_imports = ignore_imports =
dify_graph.nodes.loop.loop_node -> core.workflow.node_factory core.workflow.nodes.loop.loop_node -> core.app.workflow.node_factory
dify_graph.nodes.agent.agent_node -> core.model_manager core.workflow.graph_engine.command_channels.redis_channel -> extensions.ext_redis
dify_graph.nodes.agent.agent_node -> core.provider_manager core.workflow.workflow_entry -> core.app.workflow.layers.observability
dify_graph.nodes.agent.agent_node -> core.tools.tool_manager core.workflow.nodes.agent.agent_node -> core.model_manager
dify_graph.nodes.document_extractor.node -> core.helper.ssrf_proxy core.workflow.nodes.agent.agent_node -> core.provider_manager
dify_graph.nodes.iteration.iteration_node -> core.workflow.node_factory core.workflow.nodes.agent.agent_node -> core.tools.tool_manager
dify_graph.nodes.iteration.iteration_node -> core.app.workflow.layers.llm_quota core.workflow.nodes.code.code_node -> core.helper.code_executor.code_executor
dify_graph.nodes.llm.llm_utils -> core.model_manager core.workflow.nodes.datasource.datasource_node -> models.model
dify_graph.nodes.llm.protocols -> core.model_manager core.workflow.nodes.datasource.datasource_node -> models.tools
dify_graph.nodes.llm.llm_utils -> dify_graph.model_runtime.model_providers.__base.large_language_model core.workflow.nodes.datasource.datasource_node -> services.datasource_provider_service
dify_graph.nodes.llm.node -> core.tools.signature core.workflow.nodes.document_extractor.node -> configs
dify_graph.nodes.tool.tool_node -> core.callback_handler.workflow_tool_callback_handler core.workflow.nodes.document_extractor.node -> core.file.file_manager
dify_graph.nodes.tool.tool_node -> core.tools.tool_engine core.workflow.nodes.document_extractor.node -> core.helper.ssrf_proxy
dify_graph.nodes.tool.tool_node -> core.tools.tool_manager core.workflow.nodes.http_request.entities -> configs
dify_graph.nodes.agent.agent_node -> core.agent.entities core.workflow.nodes.http_request.executor -> configs
dify_graph.nodes.agent.agent_node -> core.agent.plugin_entities core.workflow.nodes.http_request.executor -> core.file.file_manager
dify_graph.nodes.knowledge_retrieval.knowledge_retrieval_node -> core.app.app_config.entities core.workflow.nodes.http_request.node -> configs
dify_graph.nodes.parameter_extractor.parameter_extractor_node -> core.prompt.advanced_prompt_transform core.workflow.nodes.http_request.node -> core.tools.tool_file_manager
dify_graph.nodes.parameter_extractor.parameter_extractor_node -> core.prompt.simple_prompt_transform core.workflow.nodes.iteration.iteration_node -> core.app.workflow.node_factory
dify_graph.nodes.parameter_extractor.parameter_extractor_node -> dify_graph.model_runtime.model_providers.__base.large_language_model core.workflow.nodes.knowledge_index.knowledge_index_node -> core.rag.index_processor.index_processor_factory
dify_graph.nodes.question_classifier.question_classifier_node -> core.prompt.simple_prompt_transform core.workflow.nodes.llm.llm_utils -> configs
dify_graph.nodes.parameter_extractor.parameter_extractor_node -> core.model_manager core.workflow.nodes.llm.llm_utils -> core.app.entities.app_invoke_entities
dify_graph.nodes.question_classifier.question_classifier_node -> core.model_manager core.workflow.nodes.llm.llm_utils -> core.file.models
dify_graph.nodes.tool.tool_node -> core.tools.utils.message_transformer core.workflow.nodes.llm.llm_utils -> core.model_manager
dify_graph.nodes.tool.tool_node -> models core.workflow.nodes.llm.llm_utils -> core.model_runtime.model_providers.__base.large_language_model
dify_graph.nodes.agent.agent_node -> models.model core.workflow.nodes.llm.llm_utils -> models.model
dify_graph.nodes.llm.file_saver -> core.helper.ssrf_proxy core.workflow.nodes.llm.llm_utils -> models.provider
dify_graph.nodes.llm.node -> core.helper.code_executor core.workflow.nodes.llm.llm_utils -> services.credit_pool_service
dify_graph.nodes.llm.node -> core.llm_generator.output_parser.errors core.workflow.nodes.llm.node -> core.tools.signature
dify_graph.nodes.llm.node -> core.llm_generator.output_parser.structured_output core.workflow.nodes.tool.tool_node -> core.callback_handler.workflow_tool_callback_handler
dify_graph.nodes.llm.node -> core.model_manager core.workflow.nodes.tool.tool_node -> core.tools.tool_engine
dify_graph.nodes.agent.entities -> core.prompt.entities.advanced_prompt_entities core.workflow.nodes.tool.tool_node -> core.tools.tool_manager
dify_graph.nodes.llm.entities -> core.prompt.entities.advanced_prompt_entities core.workflow.workflow_entry -> configs
dify_graph.nodes.llm.node -> core.prompt.entities.advanced_prompt_entities core.workflow.workflow_entry -> models.workflow
dify_graph.nodes.llm.node -> core.prompt.utils.prompt_message_util core.workflow.nodes.agent.agent_node -> core.agent.entities
dify_graph.nodes.parameter_extractor.entities -> core.prompt.entities.advanced_prompt_entities core.workflow.nodes.agent.agent_node -> core.agent.plugin_entities
dify_graph.nodes.parameter_extractor.parameter_extractor_node -> core.prompt.entities.advanced_prompt_entities core.workflow.nodes.base.node -> core.app.entities.app_invoke_entities
dify_graph.nodes.parameter_extractor.parameter_extractor_node -> core.prompt.utils.prompt_message_util core.workflow.nodes.human_input.human_input_node -> core.app.entities.app_invoke_entities
dify_graph.nodes.question_classifier.entities -> core.prompt.entities.advanced_prompt_entities core.workflow.nodes.knowledge_index.knowledge_index_node -> core.app.entities.app_invoke_entities
dify_graph.nodes.question_classifier.question_classifier_node -> core.prompt.utils.prompt_message_util core.workflow.nodes.knowledge_retrieval.knowledge_retrieval_node -> core.app.app_config.entities
dify_graph.nodes.knowledge_index.entities -> core.rag.retrieval.retrieval_methods core.workflow.nodes.llm.node -> core.app.entities.app_invoke_entities
dify_graph.nodes.llm.node -> models.dataset core.workflow.nodes.parameter_extractor.parameter_extractor_node -> core.app.entities.app_invoke_entities
dify_graph.nodes.agent.agent_node -> core.tools.utils.message_transformer core.workflow.nodes.parameter_extractor.parameter_extractor_node -> core.prompt.advanced_prompt_transform
dify_graph.nodes.llm.file_saver -> core.tools.signature core.workflow.nodes.parameter_extractor.parameter_extractor_node -> core.prompt.simple_prompt_transform
dify_graph.nodes.llm.file_saver -> core.tools.tool_file_manager core.workflow.nodes.parameter_extractor.parameter_extractor_node -> core.model_runtime.model_providers.__base.large_language_model
dify_graph.nodes.tool.tool_node -> core.tools.errors core.workflow.nodes.question_classifier.question_classifier_node -> core.app.entities.app_invoke_entities
dify_graph.nodes.agent.agent_node -> extensions.ext_database core.workflow.nodes.question_classifier.question_classifier_node -> core.prompt.advanced_prompt_transform
dify_graph.nodes.llm.file_saver -> extensions.ext_database core.workflow.nodes.question_classifier.question_classifier_node -> core.prompt.simple_prompt_transform
dify_graph.nodes.llm.node -> extensions.ext_database core.workflow.nodes.start.entities -> core.app.app_config.entities
dify_graph.nodes.tool.tool_node -> extensions.ext_database core.workflow.nodes.start.start_node -> core.app.app_config.entities
dify_graph.nodes.human_input.human_input_node -> extensions.ext_database core.workflow.workflow_entry -> core.app.apps.exc
dify_graph.nodes.human_input.human_input_node -> core.repositories.human_input_repository core.workflow.workflow_entry -> core.app.entities.app_invoke_entities
dify_graph.nodes.agent.agent_node -> models core.workflow.workflow_entry -> core.app.workflow.node_factory
dify_graph.nodes.loop.loop_node -> core.app.workflow.layers.llm_quota core.workflow.nodes.datasource.datasource_node -> core.datasource.datasource_manager
dify_graph.nodes.llm.node -> models.model core.workflow.nodes.datasource.datasource_node -> core.datasource.utils.message_transformer
dify_graph.nodes.agent.agent_node -> services core.workflow.nodes.llm.llm_utils -> core.entities.provider_entities
dify_graph.nodes.tool.tool_node -> services core.workflow.nodes.parameter_extractor.parameter_extractor_node -> core.model_manager
dify_graph.model_runtime.model_providers.__base.ai_model -> configs core.workflow.nodes.question_classifier.question_classifier_node -> core.model_manager
dify_graph.model_runtime.model_providers.__base.ai_model -> extensions.ext_redis core.workflow.node_events.node -> core.file
dify_graph.model_runtime.model_providers.__base.large_language_model -> configs core.workflow.nodes.agent.agent_node -> core.file
dify_graph.model_runtime.model_providers.__base.text_embedding_model -> core.entities.embedding_type core.workflow.nodes.datasource.datasource_node -> core.file
dify_graph.model_runtime.model_providers.model_provider_factory -> configs core.workflow.nodes.datasource.datasource_node -> core.file.enums
dify_graph.model_runtime.model_providers.model_provider_factory -> extensions.ext_redis core.workflow.nodes.document_extractor.node -> core.file
dify_graph.model_runtime.model_providers.model_provider_factory -> models.provider_ids core.workflow.nodes.http_request.executor -> core.file.enums
core.workflow.nodes.http_request.node -> core.file
core.workflow.nodes.http_request.node -> core.file.file_manager
core.workflow.nodes.knowledge_retrieval.knowledge_retrieval_node -> core.file.models
core.workflow.nodes.list_operator.node -> core.file
core.workflow.nodes.llm.file_saver -> core.file
core.workflow.nodes.llm.llm_utils -> core.variables.segments
core.workflow.nodes.llm.node -> core.file
core.workflow.nodes.llm.node -> core.file.file_manager
core.workflow.nodes.llm.node -> core.file.models
core.workflow.nodes.loop.entities -> core.variables.types
core.workflow.nodes.parameter_extractor.parameter_extractor_node -> core.file
core.workflow.nodes.protocols -> core.file
core.workflow.nodes.question_classifier.question_classifier_node -> core.file.models
core.workflow.nodes.tool.tool_node -> core.file
core.workflow.nodes.tool.tool_node -> core.tools.utils.message_transformer
core.workflow.nodes.tool.tool_node -> models
core.workflow.nodes.trigger_webhook.node -> core.file
core.workflow.runtime.variable_pool -> core.file
core.workflow.runtime.variable_pool -> core.file.file_manager
core.workflow.system_variable -> core.file.models
core.workflow.utils.condition.processor -> core.file
core.workflow.utils.condition.processor -> core.file.file_manager
core.workflow.workflow_entry -> core.file.models
core.workflow.workflow_type_encoder -> core.file.models
core.workflow.nodes.agent.agent_node -> models.model
core.workflow.nodes.code.code_node -> core.helper.code_executor.code_node_provider
core.workflow.nodes.code.code_node -> core.helper.code_executor.javascript.javascript_code_provider
core.workflow.nodes.code.code_node -> core.helper.code_executor.python3.python3_code_provider
core.workflow.nodes.code.entities -> core.helper.code_executor.code_executor
core.workflow.nodes.datasource.datasource_node -> core.variables.variables
core.workflow.nodes.http_request.executor -> core.helper.ssrf_proxy
core.workflow.nodes.http_request.node -> core.helper.ssrf_proxy
core.workflow.nodes.llm.file_saver -> core.helper.ssrf_proxy
core.workflow.nodes.llm.node -> core.helper.code_executor
core.workflow.nodes.template_transform.template_renderer -> core.helper.code_executor.code_executor
core.workflow.nodes.llm.node -> core.llm_generator.output_parser.errors
core.workflow.nodes.llm.node -> core.llm_generator.output_parser.structured_output
core.workflow.nodes.llm.node -> core.model_manager
core.workflow.nodes.agent.entities -> core.prompt.entities.advanced_prompt_entities
core.workflow.nodes.llm.entities -> core.prompt.entities.advanced_prompt_entities
core.workflow.nodes.llm.llm_utils -> core.prompt.entities.advanced_prompt_entities
core.workflow.nodes.llm.node -> core.prompt.entities.advanced_prompt_entities
core.workflow.nodes.llm.node -> core.prompt.utils.prompt_message_util
core.workflow.nodes.parameter_extractor.entities -> core.prompt.entities.advanced_prompt_entities
core.workflow.nodes.parameter_extractor.parameter_extractor_node -> core.prompt.entities.advanced_prompt_entities
core.workflow.nodes.parameter_extractor.parameter_extractor_node -> core.prompt.utils.prompt_message_util
core.workflow.nodes.question_classifier.entities -> core.prompt.entities.advanced_prompt_entities
core.workflow.nodes.question_classifier.question_classifier_node -> core.prompt.utils.prompt_message_util
core.workflow.nodes.knowledge_index.entities -> core.rag.retrieval.retrieval_methods
core.workflow.nodes.knowledge_index.knowledge_index_node -> core.rag.retrieval.retrieval_methods
core.workflow.nodes.knowledge_index.knowledge_index_node -> models.dataset
core.workflow.nodes.knowledge_index.knowledge_index_node -> services.summary_index_service
core.workflow.nodes.knowledge_index.knowledge_index_node -> tasks.generate_summary_index_task
core.workflow.nodes.knowledge_index.knowledge_index_node -> core.rag.index_processor.processor.paragraph_index_processor
core.workflow.nodes.llm.node -> models.dataset
core.workflow.nodes.agent.agent_node -> core.tools.utils.message_transformer
core.workflow.nodes.llm.file_saver -> core.tools.signature
core.workflow.nodes.llm.file_saver -> core.tools.tool_file_manager
core.workflow.nodes.tool.tool_node -> core.tools.errors
core.workflow.conversation_variable_updater -> core.variables
core.workflow.graph_engine.entities.commands -> core.variables.variables
core.workflow.nodes.agent.agent_node -> core.variables.segments
core.workflow.nodes.answer.answer_node -> core.variables
core.workflow.nodes.code.code_node -> core.variables.segments
core.workflow.nodes.code.code_node -> core.variables.types
core.workflow.nodes.code.entities -> core.variables.types
core.workflow.nodes.datasource.datasource_node -> core.variables.segments
core.workflow.nodes.document_extractor.node -> core.variables
core.workflow.nodes.document_extractor.node -> core.variables.segments
core.workflow.nodes.http_request.executor -> core.variables.segments
core.workflow.nodes.http_request.node -> core.variables.segments
core.workflow.nodes.human_input.entities -> core.variables.consts
core.workflow.nodes.iteration.iteration_node -> core.variables
core.workflow.nodes.iteration.iteration_node -> core.variables.segments
core.workflow.nodes.iteration.iteration_node -> core.variables.variables
core.workflow.nodes.knowledge_retrieval.knowledge_retrieval_node -> core.variables
core.workflow.nodes.knowledge_retrieval.knowledge_retrieval_node -> core.variables.segments
core.workflow.nodes.list_operator.node -> core.variables
core.workflow.nodes.list_operator.node -> core.variables.segments
core.workflow.nodes.llm.node -> core.variables
core.workflow.nodes.loop.loop_node -> core.variables
core.workflow.nodes.parameter_extractor.entities -> core.variables.types
core.workflow.nodes.parameter_extractor.exc -> core.variables.types
core.workflow.nodes.parameter_extractor.parameter_extractor_node -> core.variables.types
core.workflow.nodes.tool.tool_node -> core.variables.segments
core.workflow.nodes.tool.tool_node -> core.variables.variables
core.workflow.nodes.trigger_webhook.node -> core.variables.types
core.workflow.nodes.trigger_webhook.node -> core.variables.variables
core.workflow.nodes.variable_aggregator.entities -> core.variables.types
core.workflow.nodes.variable_aggregator.variable_aggregator_node -> core.variables.segments
core.workflow.nodes.variable_assigner.common.helpers -> core.variables
core.workflow.nodes.variable_assigner.common.helpers -> core.variables.consts
core.workflow.nodes.variable_assigner.common.helpers -> core.variables.types
core.workflow.nodes.variable_assigner.v1.node -> core.variables
core.workflow.nodes.variable_assigner.v2.helpers -> core.variables
core.workflow.nodes.variable_assigner.v2.node -> core.variables
core.workflow.nodes.variable_assigner.v2.node -> core.variables.consts
core.workflow.runtime.graph_runtime_state_protocol -> core.variables.segments
core.workflow.runtime.read_only_wrappers -> core.variables.segments
core.workflow.runtime.variable_pool -> core.variables
core.workflow.runtime.variable_pool -> core.variables.consts
core.workflow.runtime.variable_pool -> core.variables.segments
core.workflow.runtime.variable_pool -> core.variables.variables
core.workflow.utils.condition.processor -> core.variables
core.workflow.utils.condition.processor -> core.variables.segments
core.workflow.variable_loader -> core.variables
core.workflow.variable_loader -> core.variables.consts
core.workflow.workflow_type_encoder -> core.variables
core.workflow.graph_engine.manager -> extensions.ext_redis
core.workflow.nodes.agent.agent_node -> extensions.ext_database
core.workflow.nodes.datasource.datasource_node -> extensions.ext_database
core.workflow.nodes.knowledge_index.knowledge_index_node -> extensions.ext_database
core.workflow.nodes.llm.file_saver -> extensions.ext_database
core.workflow.nodes.llm.llm_utils -> extensions.ext_database
core.workflow.nodes.llm.node -> extensions.ext_database
core.workflow.nodes.tool.tool_node -> extensions.ext_database
core.workflow.nodes.human_input.human_input_node -> extensions.ext_database
core.workflow.nodes.human_input.human_input_node -> core.repositories.human_input_repository
core.workflow.workflow_entry -> extensions.otel.runtime
core.workflow.nodes.agent.agent_node -> models
core.workflow.nodes.base.node -> models.enums
core.workflow.nodes.llm.llm_utils -> models.provider_ids
core.workflow.nodes.llm.node -> models.model
core.workflow.workflow_entry -> models.enums
core.workflow.nodes.agent.agent_node -> services
core.workflow.nodes.tool.tool_node -> services
[importlinter:contract:model-runtime-no-internal-imports]
name = Model Runtime Internal Imports
type = forbidden
source_modules =
core.model_runtime
forbidden_modules =
configs
controllers
extensions
models
services
tasks
core.agent
core.app
core.base
core.callback_handler
core.datasource
core.db
core.entities
core.errors
core.extension
core.external_data_tool
core.file
core.helper
core.hosting_configuration
core.indexing_runner
core.llm_generator
core.logging
core.mcp
core.memory
core.model_manager
core.moderation
core.ops
core.plugin
core.prompt
core.provider_manager
core.rag
core.repositories
core.schemas
core.tools
core.trigger
core.variables
core.workflow
ignore_imports =
core.model_runtime.model_providers.__base.ai_model -> configs
core.model_runtime.model_providers.__base.ai_model -> extensions.ext_redis
core.model_runtime.model_providers.__base.large_language_model -> configs
core.model_runtime.model_providers.__base.text_embedding_model -> core.entities.embedding_type
core.model_runtime.model_providers.model_provider_factory -> configs
core.model_runtime.model_providers.model_provider_factory -> extensions.ext_redis
core.model_runtime.model_providers.model_provider_factory -> models.provider_ids
[importlinter:contract:rsc] [importlinter:contract:rsc]
name = RSC name = RSC
@@ -175,7 +354,7 @@ layers =
graph_engine graph_engine
response_coordinator response_coordinator
containers = containers =
dify_graph.graph_engine core.workflow.graph_engine
[importlinter:contract:worker] [importlinter:contract:worker]
name = Worker name = Worker
@@ -184,7 +363,7 @@ layers =
graph_engine graph_engine
worker worker
containers = containers =
dify_graph.graph_engine core.workflow.graph_engine
[importlinter:contract:graph-engine-architecture] [importlinter:contract:graph-engine-architecture]
name = Graph Engine Architecture name = Graph Engine Architecture
@@ -200,28 +379,28 @@ layers =
worker_management worker_management
domain domain
containers = containers =
dify_graph.graph_engine core.workflow.graph_engine
[importlinter:contract:domain-isolation] [importlinter:contract:domain-isolation]
name = Domain Model Isolation name = Domain Model Isolation
type = forbidden type = forbidden
source_modules = source_modules =
dify_graph.graph_engine.domain core.workflow.graph_engine.domain
forbidden_modules = forbidden_modules =
dify_graph.graph_engine.worker_management core.workflow.graph_engine.worker_management
dify_graph.graph_engine.command_channels core.workflow.graph_engine.command_channels
dify_graph.graph_engine.layers core.workflow.graph_engine.layers
dify_graph.graph_engine.protocols core.workflow.graph_engine.protocols
[importlinter:contract:worker-management] [importlinter:contract:worker-management]
name = Worker Management name = Worker Management
type = forbidden type = forbidden
source_modules = source_modules =
dify_graph.graph_engine.worker_management core.workflow.graph_engine.worker_management
forbidden_modules = forbidden_modules =
dify_graph.graph_engine.orchestration core.workflow.graph_engine.orchestration
dify_graph.graph_engine.command_processing core.workflow.graph_engine.command_processing
dify_graph.graph_engine.event_management core.workflow.graph_engine.event_management
[importlinter:contract:graph-traversal-components] [importlinter:contract:graph-traversal-components]
@@ -231,11 +410,11 @@ layers =
edge_processor edge_processor
skip_propagator skip_propagator
containers = containers =
dify_graph.graph_engine.graph_traversal core.workflow.graph_engine.graph_traversal
[importlinter:contract:command-channels] [importlinter:contract:command-channels]
name = Command Channels Independence name = Command Channels Independence
type = independence type = independence
modules = modules =
dify_graph.graph_engine.command_channels.in_memory_channel core.workflow.graph_engine.command_channels.in_memory_channel
dify_graph.graph_engine.command_channels.redis_channel core.workflow.graph_engine.command_channels.redis_channel
+1 -1
View File
@@ -100,7 +100,7 @@ ignore = [
"configs/*" = [ "configs/*" = [
"N802", # invalid-function-name "N802", # invalid-function-name
] ]
"dify_graph/model_runtime/callbacks/base_callback.py" = ["T201"] "core/model_runtime/callbacks/base_callback.py" = ["T201"]
"core/workflow/callbacks/workflow_logging_callback.py" = ["T201"] "core/workflow/callbacks/workflow_logging_callback.py" = ["T201"]
"libs/gmpy2_pkcs10aep_cipher.py" = [ "libs/gmpy2_pkcs10aep_cipher.py" = [
"N803", # invalid-argument-name "N803", # invalid-argument-name
+81 -1
View File
@@ -42,7 +42,7 @@ The scripts resolve paths relative to their location, so you can run them from a
1. Set up your application by visiting `http://localhost:3000`. 1. Set up your application by visiting `http://localhost:3000`.
1. Start the worker service (async and scheduler tasks, runs from `api`). 1. Optional: start the worker service (async tasks, runs from `api`).
```bash ```bash
./dev/start-worker ./dev/start-worker
@@ -54,6 +54,86 @@ The scripts resolve paths relative to their location, so you can run them from a
./dev/start-beat ./dev/start-beat
``` ```
### Manual commands
<details>
<summary>Show manual setup and run steps</summary>
These commands assume you start from the repository root.
1. Start the docker-compose stack.
The backend requires middleware, including PostgreSQL, Redis, and Weaviate, which can be started together using `docker-compose`.
```bash
cp docker/middleware.env.example docker/middleware.env
# Use mysql or another vector database profile if you are not using postgres/weaviate.
docker compose -f docker/docker-compose.middleware.yaml --profile postgresql --profile weaviate -p dify up -d
```
1. Copy env files.
```bash
cp api/.env.example api/.env
cp web/.env.example web/.env.local
```
1. Install UV if needed.
```bash
pip install uv
# Or on macOS
brew install uv
```
1. Install API dependencies.
```bash
cd api
uv sync --group dev
```
1. Install web dependencies.
```bash
cd web
pnpm install
cd ..
```
1. Start backend (runs migrations first, in a new terminal).
```bash
cd api
uv run flask db upgrade
uv run flask run --host 0.0.0.0 --port=5001 --debug
```
1. Start Dify [web](../web) service (in a new terminal).
```bash
cd web
pnpm dev:inspect
```
1. Set up your application by visiting `http://localhost:3000`.
1. Optional: start the worker service (async tasks, in a new terminal).
```bash
cd api
uv run celery -A app.celery worker -P threads -c 2 --loglevel INFO -Q api_token,dataset,priority_dataset,priority_pipeline,pipeline,mail,ops_trace,app_deletion,plugin,workflow_storage,conversation,workflow,schedule_poller,schedule_executor,triggered_workflow_dispatcher,trigger_refresh_executor,retention
```
1. Optional: start Celery Beat (scheduled tasks, in a new terminal).
```bash
cd api
uv run celery -A app.celery beat
```
</details>
### Environment notes ### Environment notes
> [!IMPORTANT] > [!IMPORTANT]
+2 -14
View File
@@ -30,7 +30,6 @@ from extensions.ext_redis import redis_client
from extensions.ext_storage import storage from extensions.ext_storage import storage
from extensions.storage.opendal_storage import OpenDALStorage from extensions.storage.opendal_storage import OpenDALStorage
from extensions.storage.storage_type import StorageType from extensions.storage.storage_type import StorageType
from libs.db_migration_lock import DbMigrationAutoRenewLock
from libs.helper import email as email_validate from libs.helper import email as email_validate
from libs.password import hash_password, password_pattern, valid_password from libs.password import hash_password, password_pattern, valid_password
from libs.rsa import generate_key_pair from libs.rsa import generate_key_pair
@@ -55,8 +54,6 @@ from tasks.remove_app_and_related_data_task import delete_draft_variables_batch
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
DB_UPGRADE_LOCK_TTL_SECONDS = 60
@click.command("reset-password", help="Reset the account password.") @click.command("reset-password", help="Reset the account password.")
@click.option("--email", prompt=True, help="Account email to reset password for") @click.option("--email", prompt=True, help="Account email to reset password for")
@@ -730,15 +727,8 @@ def create_tenant(email: str, language: str | None = None, name: str | None = No
@click.command("upgrade-db", help="Upgrade the database") @click.command("upgrade-db", help="Upgrade the database")
def upgrade_db(): def upgrade_db():
click.echo("Preparing database migration...") click.echo("Preparing database migration...")
lock = DbMigrationAutoRenewLock( lock = redis_client.lock(name="db_upgrade_lock", timeout=60)
redis_client=redis_client,
name="db_upgrade_lock",
ttl_seconds=DB_UPGRADE_LOCK_TTL_SECONDS,
logger=logger,
log_context="db_migration",
)
if lock.acquire(blocking=False): if lock.acquire(blocking=False):
migration_succeeded = False
try: try:
click.echo(click.style("Starting database migration.", fg="green")) click.echo(click.style("Starting database migration.", fg="green"))
@@ -747,7 +737,6 @@ def upgrade_db():
flask_migrate.upgrade() flask_migrate.upgrade()
migration_succeeded = True
click.echo(click.style("Database migration successful!", fg="green")) click.echo(click.style("Database migration successful!", fg="green"))
except Exception as e: except Exception as e:
@@ -755,8 +744,7 @@ def upgrade_db():
click.echo(click.style(f"Database migration failed: {e}", fg="red")) click.echo(click.style(f"Database migration failed: {e}", fg="red"))
raise SystemExit(1) raise SystemExit(1)
finally: finally:
status = "successful" if migration_succeeded else "failed" lock.release()
lock.release_safely(status=status)
else: else:
click.echo("Database migration skipped") click.echo("Database migration skipped")
-8
View File
@@ -265,11 +265,6 @@ class PluginConfig(BaseSettings):
default=60 * 60, default=60 * 60,
) )
PLUGIN_MAX_FILE_SIZE: PositiveInt = Field(
description="Maximum allowed size (bytes) for plugin-generated files",
default=50 * 1024 * 1024,
)
class MarketplaceConfig(BaseSettings): class MarketplaceConfig(BaseSettings):
""" """
@@ -1319,9 +1314,6 @@ class WorkflowLogConfig(BaseSettings):
WORKFLOW_LOG_CLEANUP_BATCH_SIZE: int = Field( WORKFLOW_LOG_CLEANUP_BATCH_SIZE: int = Field(
default=100, description="Batch size for workflow run log cleanup operations" default=100, description="Batch size for workflow run log cleanup operations"
) )
WORKFLOW_LOG_CLEANUP_SPECIFIC_WORKFLOW_IDS: str = Field(
default="", description="Comma-separated list of workflow IDs to clean logs for"
)
class SwaggerUIConfig(BaseSettings): class SwaggerUIConfig(BaseSettings):
@@ -1,5 +1,3 @@
from typing import Literal
from pydantic import Field, PositiveInt from pydantic import Field, PositiveInt
from pydantic_settings import BaseSettings from pydantic_settings import BaseSettings
@@ -51,43 +49,3 @@ class OceanBaseVectorConfig(BaseSettings):
), ),
default="ik", default="ik",
) )
OCEANBASE_VECTOR_BATCH_SIZE: PositiveInt = Field(
description="Number of documents to insert per batch",
default=100,
)
OCEANBASE_VECTOR_METRIC_TYPE: Literal["l2", "cosine", "inner_product"] = Field(
description="Distance metric type for vector index: l2, cosine, or inner_product",
default="l2",
)
OCEANBASE_HNSW_M: PositiveInt = Field(
description="HNSW M parameter (max number of connections per node)",
default=16,
)
OCEANBASE_HNSW_EF_CONSTRUCTION: PositiveInt = Field(
description="HNSW efConstruction parameter (index build-time search width)",
default=256,
)
OCEANBASE_HNSW_EF_SEARCH: int = Field(
description="HNSW efSearch parameter (query-time search width, -1 uses server default)",
default=-1,
)
OCEANBASE_VECTOR_POOL_SIZE: PositiveInt = Field(
description="SQLAlchemy connection pool size",
default=5,
)
OCEANBASE_VECTOR_MAX_OVERFLOW: int = Field(
description="SQLAlchemy connection pool max overflow connections",
default=10,
)
OCEANBASE_HNSW_REFRESH_THRESHOLD: int = Field(
description="Minimum number of inserted documents to trigger an automatic HNSW index refresh (0 to disable)",
default=1000,
)
File diff suppressed because one or more lines are too long
+1 -1
View File
@@ -12,7 +12,7 @@ or any other web framework.
import contextvars import contextvars
from collections.abc import Callable from collections.abc import Callable
from dify_graph.context.execution_context import ( from core.workflow.context.execution_context import (
ExecutionContext, ExecutionContext,
IExecutionContext, IExecutionContext,
NullAppContext, NullAppContext,
+2 -2
View File
@@ -10,8 +10,8 @@ from typing import Any, final
from flask import Flask, current_app, g from flask import Flask, current_app, g
from dify_graph.context import register_context_capturer from core.workflow.context import register_context_capturer
from dify_graph.context.execution_context import ( from core.workflow.context.execution_context import (
AppContext, AppContext,
IExecutionContext, IExecutionContext,
) )
+1 -1
View File
@@ -4,7 +4,7 @@ from typing import Any, TypeAlias
from pydantic import BaseModel, ConfigDict, computed_field from pydantic import BaseModel, ConfigDict, computed_field
from dify_graph.file import helpers as file_helpers from core.file import helpers as file_helpers
from models.model import IconType from models.model import IconType
JSONValue: TypeAlias = str | int | float | bool | None | dict[str, Any] | list[Any] JSONValue: TypeAlias = str | int | float | bool | None | dict[str, Any] | list[Any]
+2 -15
View File
@@ -23,10 +23,10 @@ from controllers.console.wraps import (
is_admin_or_owner_required, is_admin_or_owner_required,
setup_required, setup_required,
) )
from core.file import helpers as file_helpers
from core.ops.ops_trace_manager import OpsTraceManager from core.ops.ops_trace_manager import OpsTraceManager
from core.rag.retrieval.retrieval_methods import RetrievalMethod from core.rag.retrieval.retrieval_methods import RetrievalMethod
from dify_graph.enums import NodeType, WorkflowExecutionStatus from core.workflow.enums import NodeType, WorkflowExecutionStatus
from dify_graph.file import helpers as file_helpers
from extensions.ext_database import db from extensions.ext_database import db
from libs.login import current_account_with_tenant, login_required from libs.login import current_account_with_tenant, login_required
from models import App, DatasetPermissionEnum, Workflow from models import App, DatasetPermissionEnum, Workflow
@@ -660,19 +660,6 @@ class AppCopyApi(Resource):
) )
session.commit() session.commit()
# Inherit web app permission from original app
if result.app_id and FeatureService.get_system_features().webapp_auth.enabled:
try:
# Get the original app's access mode
original_settings = EnterpriseService.WebAppAuth.get_app_access_mode_by_id(app_model.id)
access_mode = original_settings.access_mode
except Exception:
# If original app has no settings (old app), default to public to match fallback behavior
access_mode = "public"
# Apply the same access mode to the copied app
EnterpriseService.WebAppAuth.update_app_access_mode(result.app_id, access_mode)
stmt = select(App).where(App.id == result.app_id) stmt = select(App).where(App.id == result.app_id)
app = session.scalar(stmt) app = session.scalar(stmt)
+1 -1
View File
@@ -22,7 +22,7 @@ from controllers.console.app.error import (
from controllers.console.app.wraps import get_app_model from controllers.console.app.wraps import get_app_model
from controllers.console.wraps import account_initialization_required, setup_required from controllers.console.wraps import account_initialization_required, setup_required
from core.errors.error import ModelCurrentlyNotSupportError, ProviderTokenNotInitError, QuotaExceededError from core.errors.error import ModelCurrentlyNotSupportError, ProviderTokenNotInitError, QuotaExceededError
from dify_graph.model_runtime.errors.invoke import InvokeError from core.model_runtime.errors.invoke import InvokeError
from libs.login import login_required from libs.login import login_required
from models import App, AppMode from models import App, AppMode
from services.audio_service import AudioService from services.audio_service import AudioService
+1 -1
View File
@@ -26,7 +26,7 @@ from core.errors.error import (
QuotaExceededError, QuotaExceededError,
) )
from core.helper.trace_id_helper import get_external_trace_id from core.helper.trace_id_helper import get_external_trace_id
from dify_graph.model_runtime.errors.invoke import InvokeError from core.model_runtime.errors.invoke import InvokeError
from libs import helper from libs import helper
from libs.helper import uuid_value from libs.helper import uuid_value
from libs.login import current_user, login_required from libs.login import current_user, login_required
+1 -1
View File
@@ -18,7 +18,7 @@ from core.helper.code_executor.javascript.javascript_code_provider import Javasc
from core.helper.code_executor.python3.python3_code_provider import Python3CodeProvider from core.helper.code_executor.python3.python3_code_provider import Python3CodeProvider
from core.llm_generator.entities import RuleCodeGeneratePayload, RuleGeneratePayload, RuleStructuredOutputPayload from core.llm_generator.entities import RuleCodeGeneratePayload, RuleGeneratePayload, RuleStructuredOutputPayload
from core.llm_generator.llm_generator import LLMGenerator from core.llm_generator.llm_generator import LLMGenerator
from dify_graph.model_runtime.errors.invoke import InvokeError from core.model_runtime.errors.invoke import InvokeError
from extensions.ext_database import db from extensions.ext_database import db
from libs.login import current_account_with_tenant, login_required from libs.login import current_account_with_tenant, login_required
from models import App from models import App
+1 -1
View File
@@ -24,7 +24,7 @@ from controllers.console.wraps import (
) )
from core.app.entities.app_invoke_entities import InvokeFrom from core.app.entities.app_invoke_entities import InvokeFrom
from core.errors.error import ModelCurrentlyNotSupportError, ProviderTokenNotInitError, QuotaExceededError from core.errors.error import ModelCurrentlyNotSupportError, ProviderTokenNotInitError, QuotaExceededError
from dify_graph.model_runtime.errors.invoke import InvokeError from core.model_runtime.errors.invoke import InvokeError
from extensions.ext_database import db from extensions.ext_database import db
from fields.raws import FilesContainedField from fields.raws import FilesContainedField
from libs.helper import TimestampField, uuid_value from libs.helper import TimestampField, uuid_value
+5 -6
View File
@@ -20,7 +20,9 @@ from core.app.app_config.features.file_upload.manager import FileUploadConfigMan
from core.app.apps.base_app_queue_manager import AppQueueManager from core.app.apps.base_app_queue_manager import AppQueueManager
from core.app.apps.workflow.app_generator import SKIP_PREPARE_USER_INPUTS_KEY from core.app.apps.workflow.app_generator import SKIP_PREPARE_USER_INPUTS_KEY
from core.app.entities.app_invoke_entities import InvokeFrom from core.app.entities.app_invoke_entities import InvokeFrom
from core.file.models import File
from core.helper.trace_id_helper import get_external_trace_id from core.helper.trace_id_helper import get_external_trace_id
from core.model_runtime.utils.encoders import jsonable_encoder
from core.plugin.impl.exc import PluginInvokeError from core.plugin.impl.exc import PluginInvokeError
from core.trigger.debug.event_selectors import ( from core.trigger.debug.event_selectors import (
TriggerDebugEvent, TriggerDebugEvent,
@@ -28,12 +30,9 @@ from core.trigger.debug.event_selectors import (
create_event_poller, create_event_poller,
select_trigger_debug_events, select_trigger_debug_events,
) )
from dify_graph.enums import NodeType from core.workflow.enums import NodeType
from dify_graph.file.models import File from core.workflow.graph_engine.manager import GraphEngineManager
from dify_graph.graph_engine.manager import GraphEngineManager
from dify_graph.model_runtime.utils.encoders import jsonable_encoder
from extensions.ext_database import db from extensions.ext_database import db
from extensions.ext_redis import redis_client
from factories import file_factory, variable_factory from factories import file_factory, variable_factory
from fields.member_fields import simple_account_fields from fields.member_fields import simple_account_fields
from fields.workflow_fields import workflow_fields, workflow_pagination_fields from fields.workflow_fields import workflow_fields, workflow_pagination_fields
@@ -741,7 +740,7 @@ class WorkflowTaskStopApi(Resource):
AppQueueManager.set_stop_flag_no_user_check(task_id) AppQueueManager.set_stop_flag_no_user_check(task_id)
# New graph engine command channel mechanism # New graph engine command channel mechanism
GraphEngineManager(redis_client).send_stop_command(task_id) GraphEngineManager.send_stop_command(task_id)
return {"result": "success"} return {"result": "success"}
@@ -9,7 +9,7 @@ from sqlalchemy.orm import Session
from controllers.console import console_ns from controllers.console import console_ns
from controllers.console.app.wraps import get_app_model from controllers.console.app.wraps import get_app_model
from controllers.console.wraps import account_initialization_required, setup_required from controllers.console.wraps import account_initialization_required, setup_required
from dify_graph.enums import WorkflowExecutionStatus from core.workflow.enums import WorkflowExecutionStatus
from extensions.ext_database import db from extensions.ext_database import db
from fields.workflow_app_log_fields import ( from fields.workflow_app_log_fields import (
build_workflow_app_log_pagination_model, build_workflow_app_log_pagination_model,
@@ -15,11 +15,11 @@ from controllers.console.app.error import (
from controllers.console.app.wraps import get_app_model from controllers.console.app.wraps import get_app_model
from controllers.console.wraps import account_initialization_required, edit_permission_required, setup_required from controllers.console.wraps import account_initialization_required, edit_permission_required, setup_required
from controllers.web.error import InvalidArgumentError, NotFoundError from controllers.web.error import InvalidArgumentError, NotFoundError
from dify_graph.constants import CONVERSATION_VARIABLE_NODE_ID, SYSTEM_VARIABLE_NODE_ID from core.file import helpers as file_helpers
from dify_graph.file import helpers as file_helpers from core.variables.segment_group import SegmentGroup
from dify_graph.variables.segment_group import SegmentGroup from core.variables.segments import ArrayFileSegment, FileSegment, Segment
from dify_graph.variables.segments import ArrayFileSegment, FileSegment, Segment from core.variables.types import SegmentType
from dify_graph.variables.types import SegmentType from core.workflow.constants import CONVERSATION_VARIABLE_NODE_ID, SYSTEM_VARIABLE_NODE_ID
from extensions.ext_database import db from extensions.ext_database import db
from factories.file_factory import build_from_mapping, build_from_mappings from factories.file_factory import build_from_mapping, build_from_mappings
from factories.variable_factory import build_segment_with_type from factories.variable_factory import build_segment_with_type
@@ -112,11 +112,11 @@ _WORKFLOW_DRAFT_VARIABLE_WITHOUT_VALUE_FIELDS = {
"is_truncated": fields.Boolean(attribute=lambda model: model.file_id is not None), "is_truncated": fields.Boolean(attribute=lambda model: model.file_id is not None),
} }
_WORKFLOW_DRAFT_VARIABLE_FIELDS = { _WORKFLOW_DRAFT_VARIABLE_FIELDS = dict(
**_WORKFLOW_DRAFT_VARIABLE_WITHOUT_VALUE_FIELDS, _WORKFLOW_DRAFT_VARIABLE_WITHOUT_VALUE_FIELDS,
"value": fields.Raw(attribute=_serialize_var_value), value=fields.Raw(attribute=_serialize_var_value),
"full_content": fields.Raw(attribute=_serialize_full_content), full_content=fields.Raw(attribute=_serialize_full_content),
} )
_WORKFLOW_DRAFT_ENV_VARIABLE_FIELDS = { _WORKFLOW_DRAFT_ENV_VARIABLE_FIELDS = {
"id": fields.String, "id": fields.String,
+2 -2
View File
@@ -12,8 +12,8 @@ from controllers.console import console_ns
from controllers.console.app.wraps import get_app_model from controllers.console.app.wraps import get_app_model
from controllers.console.wraps import account_initialization_required, setup_required from controllers.console.wraps import account_initialization_required, setup_required
from controllers.web.error import NotFoundError from controllers.web.error import NotFoundError
from dify_graph.entities.pause_reason import HumanInputRequired from core.workflow.entities.pause_reason import HumanInputRequired
from dify_graph.enums import WorkflowExecutionStatus from core.workflow.enums import WorkflowExecutionStatus
from extensions.ext_database import db from extensions.ext_database import db
from fields.end_user_fields import simple_end_user_fields from fields.end_user_fields import simple_end_user_fields
from fields.member_fields import simple_account_fields from fields.member_fields import simple_account_fields
+1 -1
View File
@@ -8,7 +8,7 @@ from pydantic import BaseModel
from werkzeug.exceptions import BadRequest, NotFound from werkzeug.exceptions import BadRequest, NotFound
from controllers.console.wraps import account_initialization_required, setup_required from controllers.console.wraps import account_initialization_required, setup_required
from dify_graph.model_runtime.utils.encoders import jsonable_encoder from core.model_runtime.utils.encoders import jsonable_encoder
from libs.login import current_account_with_tenant, login_required from libs.login import current_account_with_tenant, login_required
from models import Account from models import Account
from models.model import OAuthProviderApp from models.model import OAuthProviderApp
+4 -31
View File
@@ -25,12 +25,12 @@ from controllers.console.wraps import (
) )
from core.errors.error import LLMBadRequestError, ProviderTokenNotInitError from core.errors.error import LLMBadRequestError, ProviderTokenNotInitError
from core.indexing_runner import IndexingRunner from core.indexing_runner import IndexingRunner
from core.model_runtime.entities.model_entities import ModelType
from core.provider_manager import ProviderManager from core.provider_manager import ProviderManager
from core.rag.datasource.vdb.vector_type import VectorType from core.rag.datasource.vdb.vector_type import VectorType
from core.rag.extractor.entity.datasource_type import DatasourceType from core.rag.extractor.entity.datasource_type import DatasourceType
from core.rag.extractor.entity.extract_setting import ExtractSetting, NotionInfo, WebsiteInfo from core.rag.extractor.entity.extract_setting import ExtractSetting, NotionInfo, WebsiteInfo
from core.rag.retrieval.retrieval_methods import RetrievalMethod from core.rag.retrieval.retrieval_methods import RetrievalMethod
from dify_graph.model_runtime.entities.model_entities import ModelType
from extensions.ext_database import db from extensions.ext_database import db
from fields.app_fields import app_detail_kernel_fields, related_app_list from fields.app_fields import app_detail_kernel_fields, related_app_list
from fields.dataset_fields import ( from fields.dataset_fields import (
@@ -53,7 +53,7 @@ from fields.dataset_fields import (
from fields.document_fields import document_status_fields from fields.document_fields import document_status_fields
from libs.login import current_account_with_tenant, login_required from libs.login import current_account_with_tenant, login_required
from models import ApiToken, Dataset, Document, DocumentSegment, UploadFile from models import ApiToken, Dataset, Document, DocumentSegment, UploadFile
from models.dataset import DatasetPermission, DatasetPermissionEnum from models.dataset import DatasetPermissionEnum
from models.provider_ids import ModelProviderID from models.provider_ids import ModelProviderID
from services.api_token_service import ApiTokenCache from services.api_token_service import ApiTokenCache
from services.dataset_service import DatasetPermissionService, DatasetService, DocumentService from services.dataset_service import DatasetPermissionService, DatasetService, DocumentService
@@ -119,14 +119,6 @@ def _validate_indexing_technique(value: str | None) -> str | None:
return value return value
def _validate_doc_form(value: str | None) -> str | None:
if value is None:
return value
if value not in Dataset.DOC_FORM_LIST:
raise ValueError("Invalid doc_form.")
return value
class DatasetCreatePayload(BaseModel): class DatasetCreatePayload(BaseModel):
name: str = Field(..., min_length=1, max_length=40) name: str = Field(..., min_length=1, max_length=40)
description: str = Field("", max_length=400) description: str = Field("", max_length=400)
@@ -187,14 +179,6 @@ class IndexingEstimatePayload(BaseModel):
raise ValueError("indexing_technique is required.") raise ValueError("indexing_technique is required.")
return result return result
@field_validator("doc_form")
@classmethod
def validate_doc_form(cls, value: str) -> str:
result = _validate_doc_form(value)
if result is None:
return "text_model"
return result
class ConsoleDatasetListQuery(BaseModel): class ConsoleDatasetListQuery(BaseModel):
page: int = Field(default=1, description="Page number") page: int = Field(default=1, description="Page number")
@@ -339,18 +323,6 @@ class DatasetListApi(Resource):
model_names.append(f"{embedding_model.model}:{embedding_model.provider.provider}") model_names.append(f"{embedding_model.model}:{embedding_model.provider.provider}")
data = cast(list[dict[str, Any]], marshal(datasets, dataset_detail_fields)) data = cast(list[dict[str, Any]], marshal(datasets, dataset_detail_fields))
dataset_ids = [item["id"] for item in data if item.get("permission") == "partial_members"]
partial_members_map: dict[str, list[str]] = {}
if dataset_ids:
permissions = db.session.execute(
select(DatasetPermission.dataset_id, DatasetPermission.account_id).where(
DatasetPermission.dataset_id.in_(dataset_ids)
)
).all()
for dataset_id, account_id in permissions:
partial_members_map.setdefault(dataset_id, []).append(account_id)
for item in data: for item in data:
# convert embedding_model_provider to plugin standard format # convert embedding_model_provider to plugin standard format
if item["indexing_technique"] == "high_quality" and item["embedding_model_provider"]: if item["indexing_technique"] == "high_quality" and item["embedding_model_provider"]:
@@ -364,7 +336,8 @@ class DatasetListApi(Resource):
item["embedding_available"] = True item["embedding_available"] = True
if item.get("permission") == "partial_members": if item.get("permission") == "partial_members":
item.update({"partial_member_list": partial_members_map.get(item["id"], [])}) part_users_list = DatasetPermissionService.get_dataset_partial_member_list(item["id"])
item.update({"partial_member_list": part_users_list})
else: else:
item.update({"partial_member_list": []}) item.update({"partial_member_list": []})
@@ -24,11 +24,11 @@ from core.errors.error import (
) )
from core.indexing_runner import IndexingRunner from core.indexing_runner import IndexingRunner
from core.model_manager import ModelManager from core.model_manager import ModelManager
from core.model_runtime.entities.model_entities import ModelType
from core.model_runtime.errors.invoke import InvokeAuthorizationError
from core.plugin.impl.exc import PluginDaemonClientSideError from core.plugin.impl.exc import PluginDaemonClientSideError
from core.rag.extractor.entity.datasource_type import DatasourceType from core.rag.extractor.entity.datasource_type import DatasourceType
from core.rag.extractor.entity.extract_setting import ExtractSetting, NotionInfo, WebsiteInfo from core.rag.extractor.entity.extract_setting import ExtractSetting, NotionInfo, WebsiteInfo
from dify_graph.model_runtime.entities.model_entities import ModelType
from dify_graph.model_runtime.errors.invoke import InvokeAuthorizationError
from extensions.ext_database import db from extensions.ext_database import db
from fields.dataset_fields import dataset_fields from fields.dataset_fields import dataset_fields
from fields.document_fields import ( from fields.document_fields import (
@@ -26,7 +26,7 @@ from controllers.console.wraps import (
) )
from core.errors.error import LLMBadRequestError, ProviderTokenNotInitError from core.errors.error import LLMBadRequestError, ProviderTokenNotInitError
from core.model_manager import ModelManager from core.model_manager import ModelManager
from dify_graph.model_runtime.entities.model_entities import ModelType from core.model_runtime.entities.model_entities import ModelType
from extensions.ext_database import db from extensions.ext_database import db
from extensions.ext_redis import redis_client from extensions.ext_redis import redis_client
from fields.segment_fields import child_chunk_fields, segment_fields from fields.segment_fields import child_chunk_fields, segment_fields
@@ -19,7 +19,7 @@ from core.errors.error import (
ProviderTokenNotInitError, ProviderTokenNotInitError,
QuotaExceededError, QuotaExceededError,
) )
from dify_graph.model_runtime.errors.invoke import InvokeError from core.model_runtime.errors.invoke import InvokeError
from fields.hit_testing_fields import hit_testing_record_fields from fields.hit_testing_fields import hit_testing_record_fields
from libs.login import current_user from libs.login import current_user
from models.account import Account from models.account import Account
@@ -9,9 +9,9 @@ from configs import dify_config
from controllers.common.schema import register_schema_models from controllers.common.schema import register_schema_models
from controllers.console import console_ns from controllers.console import console_ns
from controllers.console.wraps import account_initialization_required, edit_permission_required, setup_required from controllers.console.wraps import account_initialization_required, edit_permission_required, setup_required
from core.model_runtime.errors.validate import CredentialsValidateFailedError
from core.model_runtime.utils.encoders import jsonable_encoder
from core.plugin.impl.oauth import OAuthHandler from core.plugin.impl.oauth import OAuthHandler
from dify_graph.model_runtime.errors.validate import CredentialsValidateFailedError
from dify_graph.model_runtime.utils.encoders import jsonable_encoder
from libs.login import current_account_with_tenant, login_required from libs.login import current_account_with_tenant, login_required
from models.provider_ids import DatasourceProviderID from models.provider_ids import DatasourceProviderID
from services.datasource_provider_service import DatasourceProviderService from services.datasource_provider_service import DatasourceProviderService
@@ -21,8 +21,8 @@ from controllers.console.app.workflow_draft_variable import (
from controllers.console.datasets.wraps import get_rag_pipeline from controllers.console.datasets.wraps import get_rag_pipeline
from controllers.console.wraps import account_initialization_required, setup_required from controllers.console.wraps import account_initialization_required, setup_required
from controllers.web.error import InvalidArgumentError, NotFoundError from controllers.web.error import InvalidArgumentError, NotFoundError
from dify_graph.constants import CONVERSATION_VARIABLE_NODE_ID, SYSTEM_VARIABLE_NODE_ID from core.variables.types import SegmentType
from dify_graph.variables.types import SegmentType from core.workflow.constants import CONVERSATION_VARIABLE_NODE_ID, SYSTEM_VARIABLE_NODE_ID
from extensions.ext_database import db from extensions.ext_database import db
from factories.file_factory import build_from_mapping, build_from_mappings from factories.file_factory import build_from_mapping, build_from_mappings
from factories.variable_factory import build_segment_with_type from factories.variable_factory import build_segment_with_type
@@ -33,7 +33,7 @@ from controllers.web.error import InvokeRateLimitError as InvokeRateLimitHttpErr
from core.app.apps.base_app_queue_manager import AppQueueManager from core.app.apps.base_app_queue_manager import AppQueueManager
from core.app.apps.pipeline.pipeline_generator import PipelineGenerator from core.app.apps.pipeline.pipeline_generator import PipelineGenerator
from core.app.entities.app_invoke_entities import InvokeFrom from core.app.entities.app_invoke_entities import InvokeFrom
from dify_graph.model_runtime.utils.encoders import jsonable_encoder from core.model_runtime.utils.encoders import jsonable_encoder
from extensions.ext_database import db from extensions.ext_database import db
from factories import variable_factory from factories import variable_factory
from libs import helper from libs import helper
+1 -1
View File
@@ -19,7 +19,7 @@ from controllers.console.app.error import (
) )
from controllers.console.explore.wraps import InstalledAppResource from controllers.console.explore.wraps import InstalledAppResource
from core.errors.error import ModelCurrentlyNotSupportError, ProviderTokenNotInitError, QuotaExceededError from core.errors.error import ModelCurrentlyNotSupportError, ProviderTokenNotInitError, QuotaExceededError
from dify_graph.model_runtime.errors.invoke import InvokeError from core.model_runtime.errors.invoke import InvokeError
from services.audio_service import AudioService from services.audio_service import AudioService
from services.errors.audio import ( from services.errors.audio import (
AudioTooLargeServiceError, AudioTooLargeServiceError,
@@ -24,7 +24,7 @@ from core.errors.error import (
ProviderTokenNotInitError, ProviderTokenNotInitError,
QuotaExceededError, QuotaExceededError,
) )
from dify_graph.model_runtime.errors.invoke import InvokeError from core.model_runtime.errors.invoke import InvokeError
from extensions.ext_database import db from extensions.ext_database import db
from libs import helper from libs import helper
from libs.datetime_utils import naive_utc_now from libs.datetime_utils import naive_utc_now
+1 -1
View File
@@ -21,7 +21,7 @@ from controllers.console.explore.error import (
from controllers.console.explore.wraps import InstalledAppResource from controllers.console.explore.wraps import InstalledAppResource
from core.app.entities.app_invoke_entities import InvokeFrom from core.app.entities.app_invoke_entities import InvokeFrom
from core.errors.error import ModelCurrentlyNotSupportError, ProviderTokenNotInitError, QuotaExceededError from core.errors.error import ModelCurrentlyNotSupportError, ProviderTokenNotInitError, QuotaExceededError
from dify_graph.model_runtime.errors.invoke import InvokeError from core.model_runtime.errors.invoke import InvokeError
from fields.conversation_fields import ResultResponse from fields.conversation_fields import ResultResponse
from fields.message_fields import MessageInfiniteScrollPagination, MessageListItem, SuggestedQuestionsResponse from fields.message_fields import MessageInfiniteScrollPagination, MessageListItem, SuggestedQuestionsResponse
from libs import helper from libs import helper
+21 -26
View File
@@ -10,7 +10,7 @@ import services
from controllers.common.fields import Parameters as ParametersResponse from controllers.common.fields import Parameters as ParametersResponse
from controllers.common.fields import Site as SiteResponse from controllers.common.fields import Site as SiteResponse
from controllers.common.schema import get_or_create_model from controllers.common.schema import get_or_create_model
from controllers.console import console_ns from controllers.console import api, console_ns
from controllers.console.app.error import ( from controllers.console.app.error import (
AppUnavailableError, AppUnavailableError,
AudioTooLargeError, AudioTooLargeError,
@@ -41,10 +41,9 @@ from core.errors.error import (
ProviderTokenNotInitError, ProviderTokenNotInitError,
QuotaExceededError, QuotaExceededError,
) )
from dify_graph.graph_engine.manager import GraphEngineManager from core.model_runtime.errors.invoke import InvokeError
from dify_graph.model_runtime.errors.invoke import InvokeError from core.workflow.graph_engine.manager import GraphEngineManager
from extensions.ext_database import db from extensions.ext_database import db
from extensions.ext_redis import redis_client
from fields.app_fields import ( from fields.app_fields import (
app_detail_fields_with_site, app_detail_fields_with_site,
deleted_tool_fields, deleted_tool_fields,
@@ -226,7 +225,7 @@ class TrialAppWorkflowTaskStopApi(TrialAppResource):
AppQueueManager.set_stop_flag_no_user_check(task_id) AppQueueManager.set_stop_flag_no_user_check(task_id)
# New graph engine command channel mechanism # New graph engine command channel mechanism
GraphEngineManager(redis_client).send_stop_command(task_id) GraphEngineManager.send_stop_command(task_id)
return {"result": "success"} return {"result": "success"}
@@ -470,7 +469,7 @@ class TrialSitApi(Resource):
"""Resource for trial app sites.""" """Resource for trial app sites."""
@trial_feature_enable @trial_feature_enable
@get_app_model_with_trial(None) @get_app_model_with_trial
def get(self, app_model): def get(self, app_model):
"""Retrieve app site info. """Retrieve app site info.
@@ -492,7 +491,7 @@ class TrialAppParameterApi(Resource):
"""Resource for app variables.""" """Resource for app variables."""
@trial_feature_enable @trial_feature_enable
@get_app_model_with_trial(None) @get_app_model_with_trial
def get(self, app_model): def get(self, app_model):
"""Retrieve app parameters.""" """Retrieve app parameters."""
@@ -521,7 +520,7 @@ class TrialAppParameterApi(Resource):
class AppApi(Resource): class AppApi(Resource):
@trial_feature_enable @trial_feature_enable
@get_app_model_with_trial(None) @get_app_model_with_trial
@marshal_with(app_detail_with_site_model) @marshal_with(app_detail_with_site_model)
def get(self, app_model): def get(self, app_model):
"""Get app detail""" """Get app detail"""
@@ -534,7 +533,7 @@ class AppApi(Resource):
class AppWorkflowApi(Resource): class AppWorkflowApi(Resource):
@trial_feature_enable @trial_feature_enable
@get_app_model_with_trial(None) @get_app_model_with_trial
@marshal_with(workflow_model) @marshal_with(workflow_model)
def get(self, app_model): def get(self, app_model):
"""Get workflow detail""" """Get workflow detail"""
@@ -553,7 +552,7 @@ class AppWorkflowApi(Resource):
class DatasetListApi(Resource): class DatasetListApi(Resource):
@trial_feature_enable @trial_feature_enable
@get_app_model_with_trial(None) @get_app_model_with_trial
def get(self, app_model): def get(self, app_model):
page = request.args.get("page", default=1, type=int) page = request.args.get("page", default=1, type=int)
limit = request.args.get("limit", default=20, type=int) limit = request.args.get("limit", default=20, type=int)
@@ -571,31 +570,27 @@ class DatasetListApi(Resource):
return response return response
console_ns.add_resource(TrialChatApi, "/trial-apps/<uuid:app_id>/chat-messages", endpoint="trial_app_chat_completion") api.add_resource(TrialChatApi, "/trial-apps/<uuid:app_id>/chat-messages", endpoint="trial_app_chat_completion")
console_ns.add_resource( api.add_resource(
TrialMessageSuggestedQuestionApi, TrialMessageSuggestedQuestionApi,
"/trial-apps/<uuid:app_id>/messages/<uuid:message_id>/suggested-questions", "/trial-apps/<uuid:app_id>/messages/<uuid:message_id>/suggested-questions",
endpoint="trial_app_suggested_question", endpoint="trial_app_suggested_question",
) )
console_ns.add_resource(TrialChatAudioApi, "/trial-apps/<uuid:app_id>/audio-to-text", endpoint="trial_app_audio") api.add_resource(TrialChatAudioApi, "/trial-apps/<uuid:app_id>/audio-to-text", endpoint="trial_app_audio")
console_ns.add_resource(TrialChatTextApi, "/trial-apps/<uuid:app_id>/text-to-audio", endpoint="trial_app_text") api.add_resource(TrialChatTextApi, "/trial-apps/<uuid:app_id>/text-to-audio", endpoint="trial_app_text")
console_ns.add_resource( api.add_resource(TrialCompletionApi, "/trial-apps/<uuid:app_id>/completion-messages", endpoint="trial_app_completion")
TrialCompletionApi, "/trial-apps/<uuid:app_id>/completion-messages", endpoint="trial_app_completion"
)
console_ns.add_resource(TrialSitApi, "/trial-apps/<uuid:app_id>/site") api.add_resource(TrialSitApi, "/trial-apps/<uuid:app_id>/site")
console_ns.add_resource(TrialAppParameterApi, "/trial-apps/<uuid:app_id>/parameters", endpoint="trial_app_parameters") api.add_resource(TrialAppParameterApi, "/trial-apps/<uuid:app_id>/parameters", endpoint="trial_app_parameters")
console_ns.add_resource(AppApi, "/trial-apps/<uuid:app_id>", endpoint="trial_app") api.add_resource(AppApi, "/trial-apps/<uuid:app_id>", endpoint="trial_app")
console_ns.add_resource( api.add_resource(TrialAppWorkflowRunApi, "/trial-apps/<uuid:app_id>/workflows/run", endpoint="trial_app_workflow_run")
TrialAppWorkflowRunApi, "/trial-apps/<uuid:app_id>/workflows/run", endpoint="trial_app_workflow_run" api.add_resource(TrialAppWorkflowTaskStopApi, "/trial-apps/<uuid:app_id>/workflows/tasks/<string:task_id>/stop")
)
console_ns.add_resource(TrialAppWorkflowTaskStopApi, "/trial-apps/<uuid:app_id>/workflows/tasks/<string:task_id>/stop")
console_ns.add_resource(AppWorkflowApi, "/trial-apps/<uuid:app_id>/workflows", endpoint="trial_app_workflow") api.add_resource(AppWorkflowApi, "/trial-apps/<uuid:app_id>/workflows", endpoint="trial_app_workflow")
console_ns.add_resource(DatasetListApi, "/trial-apps/<uuid:app_id>/datasets", endpoint="trial_app_datasets") api.add_resource(DatasetListApi, "/trial-apps/<uuid:app_id>/datasets", endpoint="trial_app_datasets")
+3 -4
View File
@@ -21,9 +21,8 @@ from core.errors.error import (
ProviderTokenNotInitError, ProviderTokenNotInitError,
QuotaExceededError, QuotaExceededError,
) )
from dify_graph.graph_engine.manager import GraphEngineManager from core.model_runtime.errors.invoke import InvokeError
from dify_graph.model_runtime.errors.invoke import InvokeError from core.workflow.graph_engine.manager import GraphEngineManager
from extensions.ext_redis import redis_client
from libs import helper from libs import helper
from libs.login import current_account_with_tenant from libs.login import current_account_with_tenant
from models.model import AppMode, InstalledApp from models.model import AppMode, InstalledApp
@@ -101,6 +100,6 @@ class InstalledAppWorkflowTaskStopApi(InstalledAppResource):
AppQueueManager.set_stop_flag_no_user_check(task_id) AppQueueManager.set_stop_flag_no_user_check(task_id)
# New graph engine command channel mechanism # New graph engine command channel mechanism
GraphEngineManager(redis_client).send_stop_command(task_id) GraphEngineManager.send_stop_command(task_id)
return {"result": "success"} return {"result": "success"}
+4 -4
View File
@@ -105,9 +105,9 @@ def trial_app_required(view: Callable[Concatenate[App, P], R] | None = None):
return decorator return decorator
def trial_feature_enable(view: Callable[P, R]): def trial_feature_enable(view: Callable[..., R]) -> Callable[..., R]:
@wraps(view) @wraps(view)
def decorated(*args: P.args, **kwargs: P.kwargs): def decorated(*args, **kwargs):
features = FeatureService.get_system_features() features = FeatureService.get_system_features()
if not features.enable_trial_app: if not features.enable_trial_app:
abort(403, "Trial app feature is not enabled.") abort(403, "Trial app feature is not enabled.")
@@ -116,9 +116,9 @@ def trial_feature_enable(view: Callable[P, R]):
return decorated return decorated
def explore_banner_enabled(view: Callable[P, R]): def explore_banner_enabled(view: Callable[..., R]) -> Callable[..., R]:
@wraps(view) @wraps(view)
def decorated(*args: P.args, **kwargs: P.kwargs): def decorated(*args, **kwargs):
features = FeatureService.get_system_features() features = FeatureService.get_system_features()
if not features.enable_explore_banner: if not features.enable_explore_banner:
abort(403, "Explore banner feature is not enabled.") abort(403, "Explore banner feature is not enabled.")
+1 -1
View File
@@ -12,8 +12,8 @@ from controllers.common.errors import (
UnsupportedFileTypeError, UnsupportedFileTypeError,
) )
from controllers.console import console_ns from controllers.console import console_ns
from core.file import helpers as file_helpers
from core.helper import ssrf_proxy from core.helper import ssrf_proxy
from dify_graph.file import helpers as file_helpers
from extensions.ext_database import db from extensions.ext_database import db
from fields.file_fields import FileWithSignedUrl, RemoteFileInfo from fields.file_fields import FileWithSignedUrl, RemoteFileInfo
from libs.login import current_account_with_tenant, login_required from libs.login import current_account_with_tenant, login_required
@@ -2,7 +2,7 @@ from flask_restx import Resource, fields
from controllers.console import console_ns from controllers.console import console_ns
from controllers.console.wraps import account_initialization_required, setup_required from controllers.console.wraps import account_initialization_required, setup_required
from dify_graph.model_runtime.utils.encoders import jsonable_encoder from core.model_runtime.utils.encoders import jsonable_encoder
from libs.login import current_account_with_tenant, login_required from libs.login import current_account_with_tenant, login_required
from services.agent_service import AgentService from services.agent_service import AgentService
@@ -7,8 +7,8 @@ from pydantic import BaseModel, Field
from controllers.common.schema import register_schema_models from controllers.common.schema import register_schema_models
from controllers.console import console_ns from controllers.console import console_ns
from controllers.console.wraps import account_initialization_required, is_admin_or_owner_required, setup_required from controllers.console.wraps import account_initialization_required, is_admin_or_owner_required, setup_required
from core.model_runtime.utils.encoders import jsonable_encoder
from core.plugin.impl.exc import PluginPermissionDeniedError from core.plugin.impl.exc import PluginPermissionDeniedError
from dify_graph.model_runtime.utils.encoders import jsonable_encoder
from libs.login import current_account_with_tenant, login_required from libs.login import current_account_with_tenant, login_required
from services.plugin.endpoint_service import EndpointService from services.plugin.endpoint_service import EndpointService
@@ -5,8 +5,8 @@ from werkzeug.exceptions import Forbidden
from controllers.common.schema import register_schema_models from controllers.common.schema import register_schema_models
from controllers.console import console_ns from controllers.console import console_ns
from controllers.console.wraps import account_initialization_required, setup_required from controllers.console.wraps import account_initialization_required, setup_required
from dify_graph.model_runtime.entities.model_entities import ModelType from core.model_runtime.entities.model_entities import ModelType
from dify_graph.model_runtime.errors.validate import CredentialsValidateFailedError from core.model_runtime.errors.validate import CredentialsValidateFailedError
from libs.login import current_account_with_tenant, login_required from libs.login import current_account_with_tenant, login_required
from models import TenantAccountRole from models import TenantAccountRole
from services.model_load_balancing_service import ModelLoadBalancingService from services.model_load_balancing_service import ModelLoadBalancingService
@@ -7,9 +7,9 @@ from pydantic import BaseModel, Field, field_validator
from controllers.console import console_ns from controllers.console import console_ns
from controllers.console.wraps import account_initialization_required, is_admin_or_owner_required, setup_required from controllers.console.wraps import account_initialization_required, is_admin_or_owner_required, setup_required
from dify_graph.model_runtime.entities.model_entities import ModelType from core.model_runtime.entities.model_entities import ModelType
from dify_graph.model_runtime.errors.validate import CredentialsValidateFailedError from core.model_runtime.errors.validate import CredentialsValidateFailedError
from dify_graph.model_runtime.utils.encoders import jsonable_encoder from core.model_runtime.utils.encoders import jsonable_encoder
from libs.helper import uuid_value from libs.helper import uuid_value
from libs.login import current_account_with_tenant, login_required from libs.login import current_account_with_tenant, login_required
from services.billing_service import BillingService from services.billing_service import BillingService
+3 -3
View File
@@ -8,9 +8,9 @@ from pydantic import BaseModel, Field, field_validator
from controllers.common.schema import register_enum_models, register_schema_models from controllers.common.schema import register_enum_models, register_schema_models
from controllers.console import console_ns from controllers.console import console_ns
from controllers.console.wraps import account_initialization_required, is_admin_or_owner_required, setup_required from controllers.console.wraps import account_initialization_required, is_admin_or_owner_required, setup_required
from dify_graph.model_runtime.entities.model_entities import ModelType from core.model_runtime.entities.model_entities import ModelType
from dify_graph.model_runtime.errors.validate import CredentialsValidateFailedError from core.model_runtime.errors.validate import CredentialsValidateFailedError
from dify_graph.model_runtime.utils.encoders import jsonable_encoder from core.model_runtime.utils.encoders import jsonable_encoder
from libs.helper import uuid_value from libs.helper import uuid_value
from libs.login import current_account_with_tenant, login_required from libs.login import current_account_with_tenant, login_required
from services.model_load_balancing_service import ModelLoadBalancingService from services.model_load_balancing_service import ModelLoadBalancingService
+1 -1
View File
@@ -12,8 +12,8 @@ from controllers.common.schema import register_enum_models, register_schema_mode
from controllers.console import console_ns from controllers.console import console_ns
from controllers.console.workspace import plugin_permission_required from controllers.console.workspace import plugin_permission_required
from controllers.console.wraps import account_initialization_required, is_admin_or_owner_required, setup_required from controllers.console.wraps import account_initialization_required, is_admin_or_owner_required, setup_required
from core.model_runtime.utils.encoders import jsonable_encoder
from core.plugin.impl.exc import PluginDaemonClientSideError from core.plugin.impl.exc import PluginDaemonClientSideError
from dify_graph.model_runtime.utils.encoders import jsonable_encoder
from libs.login import current_account_with_tenant, login_required from libs.login import current_account_with_tenant, login_required
from models.account import TenantPluginAutoUpgradeStrategy, TenantPluginPermission from models.account import TenantPluginAutoUpgradeStrategy, TenantPluginPermission
from services.plugin.plugin_auto_upgrade_service import PluginAutoUpgradeService from services.plugin.plugin_auto_upgrade_service import PluginAutoUpgradeService
@@ -23,10 +23,10 @@ from core.entities.mcp_provider import MCPAuthentication, MCPConfiguration
from core.mcp.auth.auth_flow import auth, handle_callback from core.mcp.auth.auth_flow import auth, handle_callback
from core.mcp.error import MCPAuthError, MCPError, MCPRefreshTokenError from core.mcp.error import MCPAuthError, MCPError, MCPRefreshTokenError
from core.mcp.mcp_client import MCPClient from core.mcp.mcp_client import MCPClient
from core.model_runtime.utils.encoders import jsonable_encoder
from core.plugin.entities.plugin_daemon import CredentialType from core.plugin.entities.plugin_daemon import CredentialType
from core.plugin.impl.oauth import OAuthHandler from core.plugin.impl.oauth import OAuthHandler
from core.tools.entities.tool_entities import ApiProviderSchemaType, WorkflowToolParameterConfiguration from core.tools.entities.tool_entities import ApiProviderSchemaType, WorkflowToolParameterConfiguration
from dify_graph.model_runtime.utils.encoders import jsonable_encoder
from extensions.ext_database import db from extensions.ext_database import db
from libs.helper import alphanumeric, uuid_value from libs.helper import alphanumeric, uuid_value
from libs.login import current_account_with_tenant, login_required from libs.login import current_account_with_tenant, login_required
@@ -10,11 +10,11 @@ from werkzeug.exceptions import BadRequest, Forbidden
from configs import dify_config from configs import dify_config
from controllers.common.schema import register_schema_models from controllers.common.schema import register_schema_models
from controllers.web.error import NotFoundError from controllers.web.error import NotFoundError
from core.model_runtime.utils.encoders import jsonable_encoder
from core.plugin.entities.plugin_daemon import CredentialType from core.plugin.entities.plugin_daemon import CredentialType
from core.plugin.impl.oauth import OAuthHandler from core.plugin.impl.oauth import OAuthHandler
from core.trigger.entities.entities import SubscriptionBuilderUpdater from core.trigger.entities.entities import SubscriptionBuilderUpdater
from core.trigger.trigger_manager import TriggerManager from core.trigger.trigger_manager import TriggerManager
from dify_graph.model_runtime.utils.encoders import jsonable_encoder
from extensions.ext_database import db from extensions.ext_database import db
from libs.login import current_user, login_required from libs.login import current_user, login_required
from models.account import Account from models.account import Account
+4 -4
View File
@@ -36,9 +36,9 @@ ERROR_MSG_INVALID_ENCRYPTED_DATA = "Invalid encrypted data"
ERROR_MSG_INVALID_ENCRYPTED_CODE = "Invalid encrypted code" ERROR_MSG_INVALID_ENCRYPTED_CODE = "Invalid encrypted code"
def account_initialization_required(view: Callable[P, R]) -> Callable[P, R]: def account_initialization_required(view: Callable[P, R]):
@wraps(view) @wraps(view)
def decorated(*args: P.args, **kwargs: P.kwargs) -> R: def decorated(*args: P.args, **kwargs: P.kwargs):
# check account initialization # check account initialization
current_user, _ = current_account_with_tenant() current_user, _ = current_account_with_tenant()
if current_user.status == AccountStatus.UNINITIALIZED: if current_user.status == AccountStatus.UNINITIALIZED:
@@ -214,9 +214,9 @@ def cloud_utm_record(view: Callable[P, R]):
return decorated return decorated
def setup_required(view: Callable[P, R]) -> Callable[P, R]: def setup_required(view: Callable[P, R]):
@wraps(view) @wraps(view)
def decorated(*args: P.args, **kwargs: P.kwargs) -> R: def decorated(*args: P.args, **kwargs: P.kwargs):
# check setup # check setup
if ( if (
dify_config.EDITION == "SELF_HOSTED" dify_config.EDITION == "SELF_HOSTED"
+1 -1
View File
@@ -137,7 +137,7 @@ class FilePreviewApi(Resource):
if args.as_attachment: if args.as_attachment:
encoded_filename = quote(upload_file.name) encoded_filename = quote(upload_file.name)
response.headers["Content-Disposition"] = f"attachment; filename*=UTF-8''{encoded_filename}" response.headers["Content-Disposition"] = f"attachment; filename*=UTF-8''{encoded_filename}"
response.headers["Content-Type"] = "application/octet-stream" response.headers["Content-Type"] = "application/octet-stream"
enforce_download_for_html( enforce_download_for_html(
response, response,
-4
View File
@@ -64,10 +64,6 @@ class ToolFileApi(Resource):
if not stream or not tool_file: if not stream or not tool_file:
raise NotFound("file is not found") raise NotFound("file is not found")
except NotFound:
raise
except Exception: except Exception:
raise UnsupportedFileTypeError() raise UnsupportedFileTypeError()
+1 -1
View File
@@ -7,8 +7,8 @@ from pydantic import BaseModel, Field
from werkzeug.exceptions import Forbidden from werkzeug.exceptions import Forbidden
import services import services
from core.file.helpers import verify_plugin_file_signature
from core.tools.tool_file_manager import ToolFileManager from core.tools.tool_file_manager import ToolFileManager
from dify_graph.file.helpers import verify_plugin_file_signature
from fields.file_fields import FileResponse from fields.file_fields import FileResponse
from ..common.errors import ( from ..common.errors import (
+2 -2
View File
@@ -4,6 +4,8 @@ from controllers.console.wraps import setup_required
from controllers.inner_api import inner_api_ns from controllers.inner_api import inner_api_ns
from controllers.inner_api.plugin.wraps import get_user_tenant, plugin_data from controllers.inner_api.plugin.wraps import get_user_tenant, plugin_data
from controllers.inner_api.wraps import plugin_inner_api_only from controllers.inner_api.wraps import plugin_inner_api_only
from core.file.helpers import get_signed_file_url_for_plugin
from core.model_runtime.utils.encoders import jsonable_encoder
from core.plugin.backwards_invocation.app import PluginAppBackwardsInvocation from core.plugin.backwards_invocation.app import PluginAppBackwardsInvocation
from core.plugin.backwards_invocation.base import BaseBackwardsInvocationResponse from core.plugin.backwards_invocation.base import BaseBackwardsInvocationResponse
from core.plugin.backwards_invocation.encrypt import PluginEncrypter from core.plugin.backwards_invocation.encrypt import PluginEncrypter
@@ -28,8 +30,6 @@ from core.plugin.entities.request import (
RequestRequestUploadFile, RequestRequestUploadFile,
) )
from core.tools.entities.tool_entities import ToolProviderType from core.tools.entities.tool_entities import ToolProviderType
from dify_graph.file.helpers import get_signed_file_url_for_plugin
from dify_graph.model_runtime.utils.encoders import jsonable_encoder
from libs.helper import length_prefixed_response from libs.helper import length_prefixed_response
from models import Account, Tenant from models import Account, Tenant
from models.model import EndUser from models.model import EndUser
+1 -1
View File
@@ -8,9 +8,9 @@ from sqlalchemy.orm import Session
from controllers.common.schema import register_schema_model from controllers.common.schema import register_schema_model
from controllers.console.app.mcp_server import AppMCPServerStatus from controllers.console.app.mcp_server import AppMCPServerStatus
from controllers.mcp import mcp_ns from controllers.mcp import mcp_ns
from core.app.app_config.entities import VariableEntity
from core.mcp import types as mcp_types from core.mcp import types as mcp_types
from core.mcp.server.streamable_http import handle_mcp_request from core.mcp.server.streamable_http import handle_mcp_request
from dify_graph.variables.input_entities import VariableEntity
from extensions.ext_database import db from extensions.ext_database import db
from libs import helper from libs import helper
from models.model import App, AppMCPServer, AppMode, EndUser from models.model import App, AppMCPServer, AppMode, EndUser
+1 -1
View File
@@ -21,7 +21,7 @@ from controllers.service_api.app.error import (
) )
from controllers.service_api.wraps import FetchUserArg, WhereisUserArg, validate_app_token from controllers.service_api.wraps import FetchUserArg, WhereisUserArg, validate_app_token
from core.errors.error import ModelCurrentlyNotSupportError, ProviderTokenNotInitError, QuotaExceededError from core.errors.error import ModelCurrentlyNotSupportError, ProviderTokenNotInitError, QuotaExceededError
from dify_graph.model_runtime.errors.invoke import InvokeError from core.model_runtime.errors.invoke import InvokeError
from models.model import App, EndUser from models.model import App, EndUser
from services.audio_service import AudioService from services.audio_service import AudioService
from services.errors.audio import ( from services.errors.audio import (
@@ -28,7 +28,7 @@ from core.errors.error import (
QuotaExceededError, QuotaExceededError,
) )
from core.helper.trace_id_helper import get_external_trace_id from core.helper.trace_id_helper import get_external_trace_id
from dify_graph.model_runtime.errors.invoke import InvokeError from core.model_runtime.errors.invoke import InvokeError
from libs import helper from libs import helper
from libs.helper import UUIDStrOrEmpty from libs.helper import UUIDStrOrEmpty
from models.model import App, AppMode, EndUser from models.model import App, AppMode, EndUser
+4 -5
View File
@@ -27,11 +27,10 @@ from core.errors.error import (
QuotaExceededError, QuotaExceededError,
) )
from core.helper.trace_id_helper import get_external_trace_id from core.helper.trace_id_helper import get_external_trace_id
from dify_graph.enums import WorkflowExecutionStatus from core.model_runtime.errors.invoke import InvokeError
from dify_graph.graph_engine.manager import GraphEngineManager from core.workflow.enums import WorkflowExecutionStatus
from dify_graph.model_runtime.errors.invoke import InvokeError from core.workflow.graph_engine.manager import GraphEngineManager
from extensions.ext_database import db from extensions.ext_database import db
from extensions.ext_redis import redis_client
from fields.workflow_app_log_fields import build_workflow_app_log_pagination_model from fields.workflow_app_log_fields import build_workflow_app_log_pagination_model
from libs import helper from libs import helper
from libs.helper import OptionalTimestampField, TimestampField from libs.helper import OptionalTimestampField, TimestampField
@@ -281,7 +280,7 @@ class WorkflowTaskStopApi(Resource):
AppQueueManager.set_stop_flag_no_user_check(task_id) AppQueueManager.set_stop_flag_no_user_check(task_id)
# New graph engine command channel mechanism # New graph engine command channel mechanism
GraphEngineManager(redis_client).send_stop_command(task_id) GraphEngineManager.send_stop_command(task_id)
return {"result": "success"} return {"result": "success"}
@@ -14,8 +14,8 @@ from controllers.service_api.wraps import (
DatasetApiResource, DatasetApiResource,
cloud_edition_billing_rate_limit_check, cloud_edition_billing_rate_limit_check,
) )
from core.model_runtime.entities.model_entities import ModelType
from core.provider_manager import ProviderManager from core.provider_manager import ProviderManager
from dify_graph.model_runtime.entities.model_entities import ModelType
from fields.dataset_fields import dataset_detail_fields from fields.dataset_fields import dataset_detail_fields
from fields.tag_fields import DataSetTag from fields.tag_fields import DataSetTag
from libs.login import current_user from libs.login import current_user
@@ -4,7 +4,7 @@ from uuid import UUID
from flask import request from flask import request
from flask_restx import marshal from flask_restx import marshal
from pydantic import BaseModel, Field, field_validator, model_validator from pydantic import BaseModel, Field, model_validator
from sqlalchemy import desc, select from sqlalchemy import desc, select
from werkzeug.exceptions import Forbidden, NotFound from werkzeug.exceptions import Forbidden, NotFound
@@ -60,13 +60,6 @@ class DocumentTextCreatePayload(BaseModel):
embedding_model: str | None = None embedding_model: str | None = None
embedding_model_provider: str | None = None embedding_model_provider: str | None = None
@field_validator("doc_form")
@classmethod
def validate_doc_form(cls, value: str) -> str:
if value not in Dataset.DOC_FORM_LIST:
raise ValueError("Invalid doc_form.")
return value
DEFAULT_REF_TEMPLATE_SWAGGER_2_0 = "#/definitions/{model}" DEFAULT_REF_TEMPLATE_SWAGGER_2_0 = "#/definitions/{model}"
@@ -79,13 +72,6 @@ class DocumentTextUpdate(BaseModel):
doc_language: str = "English" doc_language: str = "English"
retrieval_model: RetrievalModel | None = None retrieval_model: RetrievalModel | None = None
@field_validator("doc_form")
@classmethod
def validate_doc_form(cls, value: str) -> str:
if value not in Dataset.DOC_FORM_LIST:
raise ValueError("Invalid doc_form.")
return value
@model_validator(mode="after") @model_validator(mode="after")
def check_text_and_name(self) -> Self: def check_text_and_name(self) -> Self:
if self.text is not None and self.name is None: if self.text is not None and self.name is None:
@@ -3,8 +3,7 @@ from typing import Any
from flask import request from flask import request
from pydantic import BaseModel from pydantic import BaseModel
from sqlalchemy import select from werkzeug.exceptions import Forbidden
from werkzeug.exceptions import Forbidden, NotFound
import services import services
from controllers.common.errors import FilenameNotExistsError, NoFileUploadedError, TooManyFilesError from controllers.common.errors import FilenameNotExistsError, NoFileUploadedError, TooManyFilesError
@@ -18,7 +17,7 @@ from core.app.entities.app_invoke_entities import InvokeFrom
from libs import helper from libs import helper
from libs.login import current_user from libs.login import current_user
from models import Account from models import Account
from models.dataset import Dataset, Pipeline from models.dataset import Pipeline
from models.engine import db from models.engine import db
from services.errors.file import FileTooLargeError, UnsupportedFileTypeError from services.errors.file import FileTooLargeError, UnsupportedFileTypeError
from services.file_service import FileService from services.file_service import FileService
@@ -66,12 +65,6 @@ class DatasourcePluginsApi(DatasetApiResource):
) )
def get(self, tenant_id: str, dataset_id: str): def get(self, tenant_id: str, dataset_id: str):
"""Resource for getting datasource plugins.""" """Resource for getting datasource plugins."""
# Verify dataset ownership
stmt = select(Dataset).where(Dataset.tenant_id == tenant_id, Dataset.id == dataset_id)
dataset = db.session.scalar(stmt)
if not dataset:
raise NotFound("Dataset not found.")
# Get query parameter to determine published or draft # Get query parameter to determine published or draft
is_published: bool = request.args.get("is_published", default=True, type=bool) is_published: bool = request.args.get("is_published", default=True, type=bool)
@@ -111,12 +104,6 @@ class DatasourceNodeRunApi(DatasetApiResource):
@service_api_ns.expect(service_api_ns.models[DatasourceNodeRunPayload.__name__]) @service_api_ns.expect(service_api_ns.models[DatasourceNodeRunPayload.__name__])
def post(self, tenant_id: str, dataset_id: str, node_id: str): def post(self, tenant_id: str, dataset_id: str, node_id: str):
"""Resource for getting datasource plugins.""" """Resource for getting datasource plugins."""
# Verify dataset ownership
stmt = select(Dataset).where(Dataset.tenant_id == tenant_id, Dataset.id == dataset_id)
dataset = db.session.scalar(stmt)
if not dataset:
raise NotFound("Dataset not found.")
payload = DatasourceNodeRunPayload.model_validate(service_api_ns.payload or {}) payload = DatasourceNodeRunPayload.model_validate(service_api_ns.payload or {})
assert isinstance(current_user, Account) assert isinstance(current_user, Account)
rag_pipeline_service: RagPipelineService = RagPipelineService() rag_pipeline_service: RagPipelineService = RagPipelineService()
@@ -174,12 +161,6 @@ class PipelineRunApi(DatasetApiResource):
@service_api_ns.expect(service_api_ns.models[PipelineRunApiEntity.__name__]) @service_api_ns.expect(service_api_ns.models[PipelineRunApiEntity.__name__])
def post(self, tenant_id: str, dataset_id: str): def post(self, tenant_id: str, dataset_id: str):
"""Resource for running a rag pipeline.""" """Resource for running a rag pipeline."""
# Verify dataset ownership
stmt = select(Dataset).where(Dataset.tenant_id == tenant_id, Dataset.id == dataset_id)
dataset = db.session.scalar(stmt)
if not dataset:
raise NotFound("Dataset not found.")
payload = PipelineRunApiEntity.model_validate(service_api_ns.payload or {}) payload = PipelineRunApiEntity.model_validate(service_api_ns.payload or {})
if not isinstance(current_user, Account): if not isinstance(current_user, Account):
@@ -17,7 +17,7 @@ from controllers.service_api.wraps import (
) )
from core.errors.error import LLMBadRequestError, ProviderTokenNotInitError from core.errors.error import LLMBadRequestError, ProviderTokenNotInitError
from core.model_manager import ModelManager from core.model_manager import ModelManager
from dify_graph.model_runtime.entities.model_entities import ModelType from core.model_runtime.entities.model_entities import ModelType
from extensions.ext_database import db from extensions.ext_database import db
from fields.segment_fields import child_chunk_fields, segment_fields from fields.segment_fields import child_chunk_fields, segment_fields
from libs.login import current_account_with_tenant from libs.login import current_account_with_tenant
@@ -3,7 +3,7 @@ from flask_restx import Resource
from controllers.service_api import service_api_ns from controllers.service_api import service_api_ns
from controllers.service_api.wraps import validate_dataset_token from controllers.service_api.wraps import validate_dataset_token
from dify_graph.model_runtime.utils.encoders import jsonable_encoder from core.model_runtime.utils.encoders import jsonable_encoder
from services.model_provider_service import ModelProviderService from services.model_provider_service import ModelProviderService
+1 -1
View File
@@ -20,7 +20,7 @@ from controllers.web.error import (
) )
from controllers.web.wraps import WebApiResource from controllers.web.wraps import WebApiResource
from core.errors.error import ModelCurrentlyNotSupportError, ProviderTokenNotInitError, QuotaExceededError from core.errors.error import ModelCurrentlyNotSupportError, ProviderTokenNotInitError, QuotaExceededError
from dify_graph.model_runtime.errors.invoke import InvokeError from core.model_runtime.errors.invoke import InvokeError
from libs.helper import uuid_value from libs.helper import uuid_value
from models.model import App from models.model import App
from services.audio_service import AudioService from services.audio_service import AudioService
+1 -1
View File
@@ -25,7 +25,7 @@ from core.errors.error import (
ProviderTokenNotInitError, ProviderTokenNotInitError,
QuotaExceededError, QuotaExceededError,
) )
from dify_graph.model_runtime.errors.invoke import InvokeError from core.model_runtime.errors.invoke import InvokeError
from libs import helper from libs import helper
from libs.helper import uuid_value from libs.helper import uuid_value
from models.model import AppMode from models.model import AppMode
+1 -1
View File
@@ -20,7 +20,7 @@ from controllers.web.error import (
from controllers.web.wraps import WebApiResource from controllers.web.wraps import WebApiResource
from core.app.entities.app_invoke_entities import InvokeFrom from core.app.entities.app_invoke_entities import InvokeFrom
from core.errors.error import ModelCurrentlyNotSupportError, ProviderTokenNotInitError, QuotaExceededError from core.errors.error import ModelCurrentlyNotSupportError, ProviderTokenNotInitError, QuotaExceededError
from dify_graph.model_runtime.errors.invoke import InvokeError from core.model_runtime.errors.invoke import InvokeError
from fields.conversation_fields import ResultResponse from fields.conversation_fields import ResultResponse
from fields.message_fields import SuggestedQuestionsResponse, WebMessageInfiniteScrollPagination, WebMessageListItem from fields.message_fields import SuggestedQuestionsResponse, WebMessageInfiniteScrollPagination, WebMessageListItem
from libs import helper from libs import helper
+1 -1
View File
@@ -10,8 +10,8 @@ from controllers.common.errors import (
RemoteFileUploadError, RemoteFileUploadError,
UnsupportedFileTypeError, UnsupportedFileTypeError,
) )
from core.file import helpers as file_helpers
from core.helper import ssrf_proxy from core.helper import ssrf_proxy
from dify_graph.file import helpers as file_helpers
from extensions.ext_database import db from extensions.ext_database import db
from fields.file_fields import FileWithSignedUrl, RemoteFileInfo from fields.file_fields import FileWithSignedUrl, RemoteFileInfo
from services.file_service import FileService from services.file_service import FileService
+3 -4
View File
@@ -22,9 +22,8 @@ from core.errors.error import (
ProviderTokenNotInitError, ProviderTokenNotInitError,
QuotaExceededError, QuotaExceededError,
) )
from dify_graph.graph_engine.manager import GraphEngineManager from core.model_runtime.errors.invoke import InvokeError
from dify_graph.model_runtime.errors.invoke import InvokeError from core.workflow.graph_engine.manager import GraphEngineManager
from extensions.ext_redis import redis_client
from libs import helper from libs import helper
from models.model import App, AppMode, EndUser from models.model import App, AppMode, EndUser
from services.app_generate_service import AppGenerateService from services.app_generate_service import AppGenerateService
@@ -122,6 +121,6 @@ class WorkflowTaskStopApi(WebApiResource):
AppQueueManager.set_stop_flag_no_user_check(task_id) AppQueueManager.set_stop_flag_no_user_check(task_id)
# New graph engine command channel mechanism # New graph engine command channel mechanism
GraphEngineManager(redis_client).send_stop_command(task_id) GraphEngineManager.send_stop_command(task_id)
return {"result": "success"} return {"result": "success"}
+13 -13
View File
@@ -17,17 +17,10 @@ from core.app.entities.app_invoke_entities import (
) )
from core.callback_handler.agent_tool_callback_handler import DifyAgentCallbackHandler from core.callback_handler.agent_tool_callback_handler import DifyAgentCallbackHandler
from core.callback_handler.index_tool_callback_handler import DatasetIndexToolCallbackHandler from core.callback_handler.index_tool_callback_handler import DatasetIndexToolCallbackHandler
from core.file import file_manager
from core.memory.token_buffer_memory import TokenBufferMemory from core.memory.token_buffer_memory import TokenBufferMemory
from core.model_manager import ModelInstance from core.model_manager import ModelInstance
from core.prompt.utils.extract_thread_messages import extract_thread_messages from core.model_runtime.entities import (
from core.tools.__base.tool import Tool
from core.tools.entities.tool_entities import (
ToolParameter,
)
from core.tools.tool_manager import ToolManager
from core.tools.utils.dataset_retriever_tool import DatasetRetrieverTool
from dify_graph.file import file_manager
from dify_graph.model_runtime.entities import (
AssistantPromptMessage, AssistantPromptMessage,
LLMUsage, LLMUsage,
PromptMessage, PromptMessage,
@@ -37,9 +30,16 @@ from dify_graph.model_runtime.entities import (
ToolPromptMessage, ToolPromptMessage,
UserPromptMessage, UserPromptMessage,
) )
from dify_graph.model_runtime.entities.message_entities import ImagePromptMessageContent, PromptMessageContentUnionTypes from core.model_runtime.entities.message_entities import ImagePromptMessageContent, PromptMessageContentUnionTypes
from dify_graph.model_runtime.entities.model_entities import ModelFeature from core.model_runtime.entities.model_entities import ModelFeature
from dify_graph.model_runtime.model_providers.__base.large_language_model import LargeLanguageModel from core.model_runtime.model_providers.__base.large_language_model import LargeLanguageModel
from core.prompt.utils.extract_thread_messages import extract_thread_messages
from core.tools.__base.tool import Tool
from core.tools.entities.tool_entities import (
ToolParameter,
)
from core.tools.tool_manager import ToolManager
from core.tools.utils.dataset_retriever_tool import DatasetRetrieverTool
from extensions.ext_database import db from extensions.ext_database import db
from factories import file_factory from factories import file_factory
from models.enums import CreatorUserRole from models.enums import CreatorUserRole
@@ -112,7 +112,7 @@ class BaseAgentRunner(AppRunner):
# check if model supports stream tool call # check if model supports stream tool call
llm_model = cast(LargeLanguageModel, model_instance.model_type_instance) llm_model = cast(LargeLanguageModel, model_instance.model_type_instance)
model_schema = llm_model.get_model_schema(model_instance.model_name, model_instance.credentials) model_schema = llm_model.get_model_schema(model_instance.model, model_instance.credentials)
features = model_schema.features if model_schema and model_schema.features else [] features = model_schema.features if model_schema and model_schema.features else []
self.stream_tool_call = ModelFeature.STREAM_TOOL_CALL in features self.stream_tool_call = ModelFeature.STREAM_TOOL_CALL in features
self.files = application_generate_entity.files if ModelFeature.VISION in features else [] self.files = application_generate_entity.files if ModelFeature.VISION in features else []
+10 -10
View File
@@ -9,20 +9,20 @@ from core.agent.entities import AgentScratchpadUnit
from core.agent.output_parser.cot_output_parser import CotAgentOutputParser from core.agent.output_parser.cot_output_parser import CotAgentOutputParser
from core.app.apps.base_app_queue_manager import PublishFrom from core.app.apps.base_app_queue_manager import PublishFrom
from core.app.entities.queue_entities import QueueAgentThoughtEvent, QueueMessageEndEvent, QueueMessageFileEvent from core.app.entities.queue_entities import QueueAgentThoughtEvent, QueueMessageEndEvent, QueueMessageFileEvent
from core.ops.ops_trace_manager import TraceQueueManager from core.model_runtime.entities.llm_entities import LLMResult, LLMResultChunk, LLMResultChunkDelta, LLMUsage
from core.prompt.agent_history_prompt_transform import AgentHistoryPromptTransform from core.model_runtime.entities.message_entities import (
from core.tools.__base.tool import Tool
from core.tools.entities.tool_entities import ToolInvokeMeta
from core.tools.tool_engine import ToolEngine
from dify_graph.model_runtime.entities.llm_entities import LLMResult, LLMResultChunk, LLMResultChunkDelta, LLMUsage
from dify_graph.model_runtime.entities.message_entities import (
AssistantPromptMessage, AssistantPromptMessage,
PromptMessage, PromptMessage,
PromptMessageTool, PromptMessageTool,
ToolPromptMessage, ToolPromptMessage,
UserPromptMessage, UserPromptMessage,
) )
from dify_graph.nodes.agent.exc import AgentMaxIterationError from core.ops.ops_trace_manager import TraceQueueManager
from core.prompt.agent_history_prompt_transform import AgentHistoryPromptTransform
from core.tools.__base.tool import Tool
from core.tools.entities.tool_entities import ToolInvokeMeta
from core.tools.tool_engine import ToolEngine
from core.workflow.nodes.agent.exc import AgentMaxIterationError
from models.model import Message from models.model import Message
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
@@ -245,7 +245,7 @@ class CotAgentRunner(BaseAgentRunner, ABC):
iteration_step += 1 iteration_step += 1
yield LLMResultChunk( yield LLMResultChunk(
model=model_instance.model_name, model=model_instance.model,
prompt_messages=prompt_messages, prompt_messages=prompt_messages,
delta=LLMResultChunkDelta( delta=LLMResultChunkDelta(
index=0, message=AssistantPromptMessage(content=final_answer), usage=llm_usage["usage"] index=0, message=AssistantPromptMessage(content=final_answer), usage=llm_usage["usage"]
@@ -268,7 +268,7 @@ class CotAgentRunner(BaseAgentRunner, ABC):
self.queue_manager.publish( self.queue_manager.publish(
QueueMessageEndEvent( QueueMessageEndEvent(
llm_result=LLMResult( llm_result=LLMResult(
model=model_instance.model_name, model=model_instance.model,
prompt_messages=prompt_messages, prompt_messages=prompt_messages,
message=AssistantPromptMessage(content=final_answer), message=AssistantPromptMessage(content=final_answer),
usage=llm_usage["usage"] or LLMUsage.empty_usage(), usage=llm_usage["usage"] or LLMUsage.empty_usage(),
+4 -4
View File
@@ -1,16 +1,16 @@
import json import json
from core.agent.cot_agent_runner import CotAgentRunner from core.agent.cot_agent_runner import CotAgentRunner
from dify_graph.file import file_manager from core.file import file_manager
from dify_graph.model_runtime.entities import ( from core.model_runtime.entities import (
AssistantPromptMessage, AssistantPromptMessage,
PromptMessage, PromptMessage,
SystemPromptMessage, SystemPromptMessage,
TextPromptMessageContent, TextPromptMessageContent,
UserPromptMessage, UserPromptMessage,
) )
from dify_graph.model_runtime.entities.message_entities import ImagePromptMessageContent, PromptMessageContentUnionTypes from core.model_runtime.entities.message_entities import ImagePromptMessageContent, PromptMessageContentUnionTypes
from dify_graph.model_runtime.utils.encoders import jsonable_encoder from core.model_runtime.utils.encoders import jsonable_encoder
class CotChatAgentRunner(CotAgentRunner): class CotChatAgentRunner(CotAgentRunner):
@@ -1,13 +1,13 @@
import json import json
from core.agent.cot_agent_runner import CotAgentRunner from core.agent.cot_agent_runner import CotAgentRunner
from dify_graph.model_runtime.entities.message_entities import ( from core.model_runtime.entities.message_entities import (
AssistantPromptMessage, AssistantPromptMessage,
PromptMessage, PromptMessage,
TextPromptMessageContent, TextPromptMessageContent,
UserPromptMessage, UserPromptMessage,
) )
from dify_graph.model_runtime.utils.encoders import jsonable_encoder from core.model_runtime.utils.encoders import jsonable_encoder
class CotCompletionAgentRunner(CotAgentRunner): class CotCompletionAgentRunner(CotAgentRunner):
+9 -9
View File
@@ -7,11 +7,8 @@ from typing import Any, Union
from core.agent.base_agent_runner import BaseAgentRunner from core.agent.base_agent_runner import BaseAgentRunner
from core.app.apps.base_app_queue_manager import PublishFrom from core.app.apps.base_app_queue_manager import PublishFrom
from core.app.entities.queue_entities import QueueAgentThoughtEvent, QueueMessageEndEvent, QueueMessageFileEvent from core.app.entities.queue_entities import QueueAgentThoughtEvent, QueueMessageEndEvent, QueueMessageFileEvent
from core.prompt.agent_history_prompt_transform import AgentHistoryPromptTransform from core.file import file_manager
from core.tools.entities.tool_entities import ToolInvokeMeta from core.model_runtime.entities import (
from core.tools.tool_engine import ToolEngine
from dify_graph.file import file_manager
from dify_graph.model_runtime.entities import (
AssistantPromptMessage, AssistantPromptMessage,
LLMResult, LLMResult,
LLMResultChunk, LLMResultChunk,
@@ -24,8 +21,11 @@ from dify_graph.model_runtime.entities import (
ToolPromptMessage, ToolPromptMessage,
UserPromptMessage, UserPromptMessage,
) )
from dify_graph.model_runtime.entities.message_entities import ImagePromptMessageContent, PromptMessageContentUnionTypes from core.model_runtime.entities.message_entities import ImagePromptMessageContent, PromptMessageContentUnionTypes
from dify_graph.nodes.agent.exc import AgentMaxIterationError from core.prompt.agent_history_prompt_transform import AgentHistoryPromptTransform
from core.tools.entities.tool_entities import ToolInvokeMeta
from core.tools.tool_engine import ToolEngine
from core.workflow.nodes.agent.exc import AgentMaxIterationError
from models.model import Message from models.model import Message
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
@@ -178,7 +178,7 @@ class FunctionCallAgentRunner(BaseAgentRunner):
) )
yield LLMResultChunk( yield LLMResultChunk(
model=model_instance.model_name, model=model_instance.model,
prompt_messages=result.prompt_messages, prompt_messages=result.prompt_messages,
system_fingerprint=result.system_fingerprint, system_fingerprint=result.system_fingerprint,
delta=LLMResultChunkDelta( delta=LLMResultChunkDelta(
@@ -308,7 +308,7 @@ class FunctionCallAgentRunner(BaseAgentRunner):
self.queue_manager.publish( self.queue_manager.publish(
QueueMessageEndEvent( QueueMessageEndEvent(
llm_result=LLMResult( llm_result=LLMResult(
model=model_instance.model_name, model=model_instance.model,
prompt_messages=prompt_messages, prompt_messages=prompt_messages,
message=AssistantPromptMessage(content=final_answer), message=AssistantPromptMessage(content=final_answer),
usage=llm_usage["usage"] or LLMUsage.empty_usage(), usage=llm_usage["usage"] or LLMUsage.empty_usage(),
@@ -4,7 +4,7 @@ from collections.abc import Generator
from typing import Union from typing import Union
from core.agent.entities import AgentScratchpadUnit from core.agent.entities import AgentScratchpadUnit
from dify_graph.model_runtime.entities.llm_entities import LLMResultChunk from core.model_runtime.entities.llm_entities import LLMResultChunk
class CotAgentOutputParser: class CotAgentOutputParser:
@@ -4,10 +4,10 @@ from core.app.app_config.entities import EasyUIBasedAppConfig
from core.app.entities.app_invoke_entities import ModelConfigWithCredentialsEntity from core.app.entities.app_invoke_entities import ModelConfigWithCredentialsEntity
from core.entities.model_entities import ModelStatus from core.entities.model_entities import ModelStatus
from core.errors.error import ModelCurrentlyNotSupportError, ProviderTokenNotInitError, QuotaExceededError from core.errors.error import ModelCurrentlyNotSupportError, ProviderTokenNotInitError, QuotaExceededError
from core.model_runtime.entities.llm_entities import LLMMode
from core.model_runtime.entities.model_entities import ModelPropertyKey, ModelType
from core.model_runtime.model_providers.__base.large_language_model import LargeLanguageModel
from core.provider_manager import ProviderManager from core.provider_manager import ProviderManager
from dify_graph.model_runtime.entities.llm_entities import LLMMode
from dify_graph.model_runtime.entities.model_entities import ModelPropertyKey, ModelType
from dify_graph.model_runtime.model_providers.__base.large_language_model import LargeLanguageModel
class ModelConfigConverter: class ModelConfigConverter:
@@ -2,9 +2,9 @@ from collections.abc import Mapping
from typing import Any from typing import Any
from core.app.app_config.entities import ModelConfigEntity from core.app.app_config.entities import ModelConfigEntity
from core.model_runtime.entities.model_entities import ModelPropertyKey, ModelType
from core.model_runtime.model_providers.model_provider_factory import ModelProviderFactory
from core.provider_manager import ProviderManager from core.provider_manager import ProviderManager
from dify_graph.model_runtime.entities.model_entities import ModelPropertyKey, ModelType
from dify_graph.model_runtime.model_providers.model_provider_factory import ModelProviderFactory
from models.provider_ids import ModelProviderID from models.provider_ids import ModelProviderID
@@ -4,8 +4,8 @@ from core.app.app_config.entities import (
AdvancedCompletionPromptTemplateEntity, AdvancedCompletionPromptTemplateEntity,
PromptTemplateEntity, PromptTemplateEntity,
) )
from core.model_runtime.entities.message_entities import PromptMessageRole
from core.prompt.simple_prompt_transform import ModelMode from core.prompt.simple_prompt_transform import ModelMode
from dify_graph.model_runtime.entities.message_entities import PromptMessageRole
from models.model import AppMode from models.model import AppMode
@@ -1,8 +1,7 @@
import re import re
from core.app.app_config.entities import ExternalDataVariableEntity from core.app.app_config.entities import ExternalDataVariableEntity, VariableEntity, VariableEntityType
from core.external_data_tool.factory import ExternalDataToolFactory from core.external_data_tool.factory import ExternalDataToolFactory
from dify_graph.variables.input_entities import VariableEntity, VariableEntityType
_ALLOWED_VARIABLE_ENTITY_TYPE = frozenset( _ALLOWED_VARIABLE_ENTITY_TYPE = frozenset(
[ [
+61 -7
View File
@@ -2,12 +2,12 @@ from collections.abc import Sequence
from enum import StrEnum, auto from enum import StrEnum, auto
from typing import Any, Literal from typing import Any, Literal
from pydantic import BaseModel, Field from jsonschema import Draft7Validator, SchemaError
from pydantic import BaseModel, Field, field_validator
from dify_graph.file import FileUploadConfig from core.file import FileTransferMethod, FileType, FileUploadConfig
from dify_graph.model_runtime.entities.llm_entities import LLMMode from core.model_runtime.entities.llm_entities import LLMMode
from dify_graph.model_runtime.entities.message_entities import PromptMessageRole from core.model_runtime.entities.message_entities import PromptMessageRole
from dify_graph.variables.input_entities import VariableEntity as WorkflowVariableEntity
from models.model import AppMode from models.model import AppMode
@@ -90,7 +90,61 @@ class PromptTemplateEntity(BaseModel):
advanced_completion_prompt_template: AdvancedCompletionPromptTemplateEntity | None = None advanced_completion_prompt_template: AdvancedCompletionPromptTemplateEntity | None = None
class RagPipelineVariableEntity(WorkflowVariableEntity): class VariableEntityType(StrEnum):
TEXT_INPUT = "text-input"
SELECT = "select"
PARAGRAPH = "paragraph"
NUMBER = "number"
EXTERNAL_DATA_TOOL = "external_data_tool"
FILE = "file"
FILE_LIST = "file-list"
CHECKBOX = "checkbox"
JSON_OBJECT = "json_object"
class VariableEntity(BaseModel):
"""
Variable Entity.
"""
# `variable` records the name of the variable in user inputs.
variable: str
label: str
description: str = ""
type: VariableEntityType
required: bool = False
hide: bool = False
default: Any = None
max_length: int | None = None
options: Sequence[str] = Field(default_factory=list)
allowed_file_types: Sequence[FileType] | None = Field(default_factory=list)
allowed_file_extensions: Sequence[str] | None = Field(default_factory=list)
allowed_file_upload_methods: Sequence[FileTransferMethod] | None = Field(default_factory=list)
json_schema: dict | None = Field(default=None)
@field_validator("description", mode="before")
@classmethod
def convert_none_description(cls, v: Any) -> str:
return v or ""
@field_validator("options", mode="before")
@classmethod
def convert_none_options(cls, v: Any) -> Sequence[str]:
return v or []
@field_validator("json_schema")
@classmethod
def validate_json_schema(cls, schema: dict | None) -> dict | None:
if schema is None:
return None
try:
Draft7Validator.check_schema(schema)
except SchemaError as e:
raise ValueError(f"Invalid JSON schema: {e.message}")
return schema
class RagPipelineVariableEntity(VariableEntity):
""" """
Rag Pipeline Variable Entity. Rag Pipeline Variable Entity.
""" """
@@ -260,7 +314,7 @@ class AppConfig(BaseModel):
app_id: str app_id: str
app_mode: AppMode app_mode: AppMode
additional_features: AppAdditionalFeatures | None = None additional_features: AppAdditionalFeatures | None = None
variables: list[WorkflowVariableEntity] = [] variables: list[VariableEntity] = []
sensitive_word_avoidance: SensitiveWordAvoidanceEntity | None = None sensitive_word_avoidance: SensitiveWordAvoidanceEntity | None = None
@@ -2,7 +2,7 @@ from collections.abc import Mapping
from typing import Any from typing import Any
from constants import DEFAULT_FILE_NUMBER_LIMITS from constants import DEFAULT_FILE_NUMBER_LIMITS
from dify_graph.file import FileUploadConfig from core.file import FileUploadConfig
class FileUploadConfigManager: class FileUploadConfigManager:
@@ -1,7 +1,6 @@
import re import re
from core.app.app_config.entities import RagPipelineVariableEntity from core.app.app_config.entities import RagPipelineVariableEntity, VariableEntity
from dify_graph.variables.input_entities import VariableEntity
from models.workflow import Workflow from models.workflow import Workflow
@@ -31,18 +31,18 @@ from core.app.entities.app_invoke_entities import AdvancedChatAppGenerateEntity,
from core.app.entities.task_entities import ChatbotAppBlockingResponse, ChatbotAppStreamResponse from core.app.entities.task_entities import ChatbotAppBlockingResponse, ChatbotAppStreamResponse
from core.app.layers.pause_state_persist_layer import PauseStateLayerConfig, PauseStatePersistenceLayer from core.app.layers.pause_state_persist_layer import PauseStateLayerConfig, PauseStatePersistenceLayer
from core.helper.trace_id_helper import extract_external_trace_id_from_args from core.helper.trace_id_helper import extract_external_trace_id_from_args
from core.model_runtime.errors.invoke import InvokeAuthorizationError
from core.ops.ops_trace_manager import TraceQueueManager from core.ops.ops_trace_manager import TraceQueueManager
from core.prompt.utils.get_thread_messages_length import get_thread_messages_length from core.prompt.utils.get_thread_messages_length import get_thread_messages_length
from core.repositories import DifyCoreRepositoryFactory from core.repositories import DifyCoreRepositoryFactory
from dify_graph.graph_engine.layers.base import GraphEngineLayer from core.workflow.graph_engine.layers.base import GraphEngineLayer
from dify_graph.model_runtime.errors.invoke import InvokeAuthorizationError from core.workflow.repositories.draft_variable_repository import (
from dify_graph.repositories.draft_variable_repository import (
DraftVariableSaverFactory, DraftVariableSaverFactory,
) )
from dify_graph.repositories.workflow_execution_repository import WorkflowExecutionRepository from core.workflow.repositories.workflow_execution_repository import WorkflowExecutionRepository
from dify_graph.repositories.workflow_node_execution_repository import WorkflowNodeExecutionRepository from core.workflow.repositories.workflow_node_execution_repository import WorkflowNodeExecutionRepository
from dify_graph.runtime import GraphRuntimeState from core.workflow.runtime import GraphRuntimeState
from dify_graph.variable_loader import DUMMY_VARIABLE_LOADER, VariableLoader from core.workflow.variable_loader import DUMMY_VARIABLE_LOADER, VariableLoader
from extensions.ext_database import db from extensions.ext_database import db
from factories import file_factory from factories import file_factory
from libs.flask_utils import preserve_flask_contexts from libs.flask_utils import preserve_flask_contexts
@@ -25,16 +25,16 @@ from core.app.workflow.layers.persistence import PersistenceWorkflowInfo, Workfl
from core.db.session_factory import session_factory from core.db.session_factory import session_factory
from core.moderation.base import ModerationError from core.moderation.base import ModerationError
from core.moderation.input_moderation import InputModeration from core.moderation.input_moderation import InputModeration
from core.variables.variables import Variable
from core.workflow.enums import WorkflowType
from core.workflow.graph_engine.command_channels.redis_channel import RedisChannel
from core.workflow.graph_engine.layers.base import GraphEngineLayer
from core.workflow.repositories.workflow_execution_repository import WorkflowExecutionRepository
from core.workflow.repositories.workflow_node_execution_repository import WorkflowNodeExecutionRepository
from core.workflow.runtime import GraphRuntimeState, VariablePool
from core.workflow.system_variable import SystemVariable
from core.workflow.variable_loader import VariableLoader
from core.workflow.workflow_entry import WorkflowEntry from core.workflow.workflow_entry import WorkflowEntry
from dify_graph.enums import WorkflowType
from dify_graph.graph_engine.command_channels.redis_channel import RedisChannel
from dify_graph.graph_engine.layers.base import GraphEngineLayer
from dify_graph.repositories.workflow_execution_repository import WorkflowExecutionRepository
from dify_graph.repositories.workflow_node_execution_repository import WorkflowNodeExecutionRepository
from dify_graph.runtime import GraphRuntimeState, VariablePool
from dify_graph.system_variable import SystemVariable
from dify_graph.variable_loader import VariableLoader
from dify_graph.variables.variables import Variable
from extensions.ext_database import db from extensions.ext_database import db
from extensions.ext_redis import redis_client from extensions.ext_redis import redis_client
from extensions.otel import WorkflowAppRunnerHandler, trace_span from extensions.otel import WorkflowAppRunnerHandler, trace_span
@@ -63,16 +63,16 @@ from core.app.entities.task_entities import (
from core.app.task_pipeline.based_generate_task_pipeline import BasedGenerateTaskPipeline from core.app.task_pipeline.based_generate_task_pipeline import BasedGenerateTaskPipeline
from core.app.task_pipeline.message_cycle_manager import MessageCycleManager from core.app.task_pipeline.message_cycle_manager import MessageCycleManager
from core.base.tts import AppGeneratorTTSPublisher, AudioTrunk from core.base.tts import AppGeneratorTTSPublisher, AudioTrunk
from core.model_runtime.entities.llm_entities import LLMUsage
from core.model_runtime.utils.encoders import jsonable_encoder
from core.ops.ops_trace_manager import TraceQueueManager from core.ops.ops_trace_manager import TraceQueueManager
from core.repositories.human_input_repository import HumanInputFormRepositoryImpl from core.repositories.human_input_repository import HumanInputFormRepositoryImpl
from dify_graph.entities.pause_reason import HumanInputRequired from core.workflow.entities.pause_reason import HumanInputRequired
from dify_graph.enums import WorkflowExecutionStatus from core.workflow.enums import WorkflowExecutionStatus
from dify_graph.model_runtime.entities.llm_entities import LLMUsage from core.workflow.nodes import NodeType
from dify_graph.model_runtime.utils.encoders import jsonable_encoder from core.workflow.repositories.draft_variable_repository import DraftVariableSaverFactory
from dify_graph.nodes import NodeType from core.workflow.runtime import GraphRuntimeState
from dify_graph.repositories.draft_variable_repository import DraftVariableSaverFactory from core.workflow.system_variable import SystemVariable
from dify_graph.runtime import GraphRuntimeState
from dify_graph.system_variable import SystemVariable
from extensions.ext_database import db from extensions.ext_database import db
from libs.datetime_utils import naive_utc_now from libs.datetime_utils import naive_utc_now
from models import Account, Conversation, EndUser, Message, MessageFile from models import Account, Conversation, EndUser, Message, MessageFile
@@ -669,14 +669,16 @@ class AdvancedChatAppGenerateTaskPipeline(GraphRuntimeStateSupport):
) -> Generator[StreamResponse, None, None]: ) -> Generator[StreamResponse, None, None]:
"""Handle retriever resources events.""" """Handle retriever resources events."""
self._message_cycle_manager.handle_retriever_resources(event) self._message_cycle_manager.handle_retriever_resources(event)
yield from () return
yield # Make this a generator
def _handle_annotation_reply_event( def _handle_annotation_reply_event(
self, event: QueueAnnotationReplyEvent, **kwargs self, event: QueueAnnotationReplyEvent, **kwargs
) -> Generator[StreamResponse, None, None]: ) -> Generator[StreamResponse, None, None]:
"""Handle annotation reply events.""" """Handle annotation reply events."""
self._message_cycle_manager.handle_annotation_reply(event) self._message_cycle_manager.handle_annotation_reply(event)
yield from () return
yield # Make this a generator
def _handle_message_replace_event( def _handle_message_replace_event(
self, event: QueueMessageReplaceEvent, **kwargs self, event: QueueMessageReplaceEvent, **kwargs
@@ -20,8 +20,8 @@ from core.app.apps.exc import GenerateTaskStoppedError
from core.app.apps.message_based_app_generator import MessageBasedAppGenerator from core.app.apps.message_based_app_generator import MessageBasedAppGenerator
from core.app.apps.message_based_app_queue_manager import MessageBasedAppQueueManager from core.app.apps.message_based_app_queue_manager import MessageBasedAppQueueManager
from core.app.entities.app_invoke_entities import AgentChatAppGenerateEntity, InvokeFrom from core.app.entities.app_invoke_entities import AgentChatAppGenerateEntity, InvokeFrom
from core.model_runtime.errors.invoke import InvokeAuthorizationError
from core.ops.ops_trace_manager import TraceQueueManager from core.ops.ops_trace_manager import TraceQueueManager
from dify_graph.model_runtime.errors.invoke import InvokeAuthorizationError
from extensions.ext_database import db from extensions.ext_database import db
from factories import file_factory from factories import file_factory
from libs.flask_utils import preserve_flask_contexts from libs.flask_utils import preserve_flask_contexts
+4 -4
View File
@@ -14,10 +14,10 @@ from core.app.entities.app_invoke_entities import AgentChatAppGenerateEntity
from core.app.entities.queue_entities import QueueAnnotationReplyEvent from core.app.entities.queue_entities import QueueAnnotationReplyEvent
from core.memory.token_buffer_memory import TokenBufferMemory from core.memory.token_buffer_memory import TokenBufferMemory
from core.model_manager import ModelInstance from core.model_manager import ModelInstance
from core.model_runtime.entities.llm_entities import LLMMode
from core.model_runtime.entities.model_entities import ModelFeature, ModelPropertyKey
from core.model_runtime.model_providers.__base.large_language_model import LargeLanguageModel
from core.moderation.base import ModerationError from core.moderation.base import ModerationError
from dify_graph.model_runtime.entities.llm_entities import LLMMode
from dify_graph.model_runtime.entities.model_entities import ModelFeature, ModelPropertyKey
from dify_graph.model_runtime.model_providers.__base.large_language_model import LargeLanguageModel
from extensions.ext_database import db from extensions.ext_database import db
from models.model import App, Conversation, Message from models.model import App, Conversation, Message
@@ -178,7 +178,7 @@ class AgentChatAppRunner(AppRunner):
# change function call strategy based on LLM model # change function call strategy based on LLM model
llm_model = cast(LargeLanguageModel, model_instance.model_type_instance) llm_model = cast(LargeLanguageModel, model_instance.model_type_instance)
model_schema = llm_model.get_model_schema(model_instance.model_name, model_instance.credentials) model_schema = llm_model.get_model_schema(model_instance.model, model_instance.credentials)
if not model_schema: if not model_schema:
raise ValueError("Model schema not found") raise ValueError("Model schema not found")
@@ -6,7 +6,7 @@ from typing import Any, Union
from core.app.entities.app_invoke_entities import InvokeFrom from core.app.entities.app_invoke_entities import InvokeFrom
from core.app.entities.task_entities import AppBlockingResponse, AppStreamResponse from core.app.entities.task_entities import AppBlockingResponse, AppStreamResponse
from core.errors.error import ModelCurrentlyNotSupportError, ProviderTokenNotInitError, QuotaExceededError from core.errors.error import ModelCurrentlyNotSupportError, ProviderTokenNotInitError, QuotaExceededError
from dify_graph.model_runtime.errors.invoke import InvokeError from core.model_runtime.errors.invoke import InvokeError
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
+5 -5
View File
@@ -3,22 +3,22 @@ from typing import TYPE_CHECKING, Any, Union, final
from sqlalchemy.orm import Session from sqlalchemy.orm import Session
from core.app.app_config.entities import VariableEntityType
from core.app.entities.app_invoke_entities import InvokeFrom from core.app.entities.app_invoke_entities import InvokeFrom
from dify_graph.enums import NodeType from core.file import File, FileUploadConfig
from dify_graph.file import File, FileUploadConfig from core.workflow.enums import NodeType
from dify_graph.repositories.draft_variable_repository import ( from core.workflow.repositories.draft_variable_repository import (
DraftVariableSaver, DraftVariableSaver,
DraftVariableSaverFactory, DraftVariableSaverFactory,
NoopDraftVariableSaver, NoopDraftVariableSaver,
) )
from dify_graph.variables.input_entities import VariableEntityType
from factories import file_factory from factories import file_factory
from libs.orjson import orjson_dumps from libs.orjson import orjson_dumps
from models import Account, EndUser from models import Account, EndUser
from services.workflow_draft_variable_service import DraftVariableSaver as DraftVariableSaverImpl from services.workflow_draft_variable_service import DraftVariableSaver as DraftVariableSaverImpl
if TYPE_CHECKING: if TYPE_CHECKING:
from dify_graph.variables.input_entities import VariableEntity from core.app.app_config.entities import VariableEntity
class BaseAppGenerator: class BaseAppGenerator:
+5 -5
View File
@@ -2,7 +2,7 @@ import logging
import queue import queue
import threading import threading
import time import time
from abc import ABC, abstractmethod from abc import abstractmethod
from enum import IntEnum, auto from enum import IntEnum, auto
from typing import Any from typing import Any
@@ -20,7 +20,7 @@ from core.app.entities.queue_entities import (
QueueStopEvent, QueueStopEvent,
WorkflowQueueMessage, WorkflowQueueMessage,
) )
from dify_graph.runtime import GraphRuntimeState from core.workflow.runtime import GraphRuntimeState
from extensions.ext_redis import redis_client from extensions.ext_redis import redis_client
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
@@ -31,7 +31,7 @@ class PublishFrom(IntEnum):
TASK_PIPELINE = auto() TASK_PIPELINE = auto()
class AppQueueManager(ABC): class AppQueueManager:
def __init__(self, task_id: str, user_id: str, invoke_from: InvokeFrom): def __init__(self, task_id: str, user_id: str, invoke_from: InvokeFrom):
if not user_id: if not user_id:
raise ValueError("user is required") raise ValueError("user is required")
@@ -122,7 +122,7 @@ class AppQueueManager(ABC):
"""Attach the live graph runtime state reference for downstream consumers.""" """Attach the live graph runtime state reference for downstream consumers."""
self._graph_runtime_state = graph_runtime_state self._graph_runtime_state = graph_runtime_state
def publish(self, event: AppQueueEvent, pub_from: PublishFrom) -> None: def publish(self, event: AppQueueEvent, pub_from: PublishFrom):
""" """
Publish event to queue Publish event to queue
:param event: :param event:
@@ -133,7 +133,7 @@ class AppQueueManager(ABC):
self._publish(event, pub_from) self._publish(event, pub_from)
@abstractmethod @abstractmethod
def _publish(self, event: AppQueueEvent, pub_from: PublishFrom) -> None: def _publish(self, event: AppQueueEvent, pub_from: PublishFrom):
""" """
Publish event to queue Publish event to queue
:param event: :param event:
+11 -11
View File
@@ -22,29 +22,29 @@ from core.app.entities.queue_entities import (
from core.app.features.annotation_reply.annotation_reply import AnnotationReplyFeature from core.app.features.annotation_reply.annotation_reply import AnnotationReplyFeature
from core.app.features.hosting_moderation.hosting_moderation import HostingModerationFeature from core.app.features.hosting_moderation.hosting_moderation import HostingModerationFeature
from core.external_data_tool.external_data_fetch import ExternalDataFetch from core.external_data_tool.external_data_fetch import ExternalDataFetch
from core.file.enums import FileTransferMethod, FileType
from core.memory.token_buffer_memory import TokenBufferMemory from core.memory.token_buffer_memory import TokenBufferMemory
from core.model_manager import ModelInstance from core.model_manager import ModelInstance
from core.moderation.input_moderation import InputModeration from core.model_runtime.entities.llm_entities import LLMResult, LLMResultChunk, LLMResultChunkDelta, LLMUsage
from core.prompt.advanced_prompt_transform import AdvancedPromptTransform from core.model_runtime.entities.message_entities import (
from core.prompt.entities.advanced_prompt_entities import ChatModelMessage, CompletionModelPromptTemplate, MemoryConfig
from core.prompt.simple_prompt_transform import ModelMode, SimplePromptTransform
from core.tools.tool_file_manager import ToolFileManager
from dify_graph.file.enums import FileTransferMethod, FileType
from dify_graph.model_runtime.entities.llm_entities import LLMResult, LLMResultChunk, LLMResultChunkDelta, LLMUsage
from dify_graph.model_runtime.entities.message_entities import (
AssistantPromptMessage, AssistantPromptMessage,
ImagePromptMessageContent, ImagePromptMessageContent,
PromptMessage, PromptMessage,
TextPromptMessageContent, TextPromptMessageContent,
) )
from dify_graph.model_runtime.entities.model_entities import ModelPropertyKey from core.model_runtime.entities.model_entities import ModelPropertyKey
from dify_graph.model_runtime.errors.invoke import InvokeBadRequestError from core.model_runtime.errors.invoke import InvokeBadRequestError
from core.moderation.input_moderation import InputModeration
from core.prompt.advanced_prompt_transform import AdvancedPromptTransform
from core.prompt.entities.advanced_prompt_entities import ChatModelMessage, CompletionModelPromptTemplate, MemoryConfig
from core.prompt.simple_prompt_transform import ModelMode, SimplePromptTransform
from core.tools.tool_file_manager import ToolFileManager
from extensions.ext_database import db from extensions.ext_database import db
from models.enums import CreatorUserRole from models.enums import CreatorUserRole
from models.model import App, AppMode, Message, MessageAnnotation, MessageFile from models.model import App, AppMode, Message, MessageAnnotation, MessageFile
if TYPE_CHECKING: if TYPE_CHECKING:
from dify_graph.file.models import File from core.file.models import File
_logger = logging.getLogger(__name__) _logger = logging.getLogger(__name__)
+1 -1
View File
@@ -19,8 +19,8 @@ from core.app.apps.exc import GenerateTaskStoppedError
from core.app.apps.message_based_app_generator import MessageBasedAppGenerator from core.app.apps.message_based_app_generator import MessageBasedAppGenerator
from core.app.apps.message_based_app_queue_manager import MessageBasedAppQueueManager from core.app.apps.message_based_app_queue_manager import MessageBasedAppQueueManager
from core.app.entities.app_invoke_entities import ChatAppGenerateEntity, InvokeFrom from core.app.entities.app_invoke_entities import ChatAppGenerateEntity, InvokeFrom
from core.model_runtime.errors.invoke import InvokeAuthorizationError
from core.ops.ops_trace_manager import TraceQueueManager from core.ops.ops_trace_manager import TraceQueueManager
from dify_graph.model_runtime.errors.invoke import InvokeAuthorizationError
from extensions.ext_database import db from extensions.ext_database import db
from factories import file_factory from factories import file_factory
from models import Account from models import Account
+2 -2
View File
@@ -11,12 +11,12 @@ from core.app.entities.app_invoke_entities import (
) )
from core.app.entities.queue_entities import QueueAnnotationReplyEvent from core.app.entities.queue_entities import QueueAnnotationReplyEvent
from core.callback_handler.index_tool_callback_handler import DatasetIndexToolCallbackHandler from core.callback_handler.index_tool_callback_handler import DatasetIndexToolCallbackHandler
from core.file import File
from core.memory.token_buffer_memory import TokenBufferMemory from core.memory.token_buffer_memory import TokenBufferMemory
from core.model_manager import ModelInstance from core.model_manager import ModelInstance
from core.model_runtime.entities.message_entities import ImagePromptMessageContent
from core.moderation.base import ModerationError from core.moderation.base import ModerationError
from core.rag.retrieval.dataset_retrieval import DatasetRetrieval from core.rag.retrieval.dataset_retrieval import DatasetRetrieval
from dify_graph.file import File
from dify_graph.model_runtime.entities.message_entities import ImagePromptMessageContent
from extensions.ext_database import db from extensions.ext_database import db
from models.model import App, Conversation, Message from models.model import App, Conversation, Message
@@ -4,7 +4,7 @@ from __future__ import annotations
from typing import TYPE_CHECKING from typing import TYPE_CHECKING
from dify_graph.runtime import GraphRuntimeState from core.workflow.runtime import GraphRuntimeState
if TYPE_CHECKING: if TYPE_CHECKING:
from core.app.task_pipeline.based_generate_task_pipeline import BasedGenerateTaskPipeline from core.app.task_pipeline.based_generate_task_pipeline import BasedGenerateTaskPipeline

Some files were not shown because too many files have changed in this diff Show More