Compare commits
7
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
c2fc4f4822 | ||
|
|
67e0eeefd2 | ||
|
|
cf58035fa2 | ||
|
|
097f10d920 | ||
|
|
c34d05141e | ||
|
|
302701b303 | ||
|
|
8080159eaf |
@@ -1,168 +0,0 @@
|
|||||||
---
|
|
||||||
name: backend-code-review
|
|
||||||
description: Review backend code for quality, security, maintainability, and best practices based on established checklist rules. Use when the user requests a review, analysis, or improvement of backend files (e.g., `.py`) under the `api/` directory. Do NOT use for frontend files (e.g., `.tsx`, `.ts`, `.js`). Supports pending-change review, code snippets review, and file-focused review.
|
|
||||||
---
|
|
||||||
|
|
||||||
# Backend Code Review
|
|
||||||
|
|
||||||
## When to use this skill
|
|
||||||
|
|
||||||
Use this skill whenever the user asks to **review, analyze, or improve** backend code (e.g., `.py`) under the `api/` directory. Supports the following review modes:
|
|
||||||
|
|
||||||
- **Pending-change review**: when the user asks to review current changes (inspect staged/working-tree files slated for commit to get the changes).
|
|
||||||
- **Code snippets review**: when the user pastes code snippets (e.g., a function/class/module excerpt) into the chat and asks for a review.
|
|
||||||
- **File-focused review**: when the user points to specific files and asks for a review of those files (one file or a small, explicit set of files, e.g., `api/...`, `api/app.py`).
|
|
||||||
|
|
||||||
Do NOT use this skill when:
|
|
||||||
|
|
||||||
- The request is about frontend code or UI (e.g., `.tsx`, `.ts`, `.js`, `web/`).
|
|
||||||
- The user is not asking for a review/analysis/improvement of backend code.
|
|
||||||
- The scope is not under `api/` (unless the user explicitly asks to review backend-related changes outside `api/`).
|
|
||||||
|
|
||||||
## How to use this skill
|
|
||||||
|
|
||||||
Follow these steps when using this skill:
|
|
||||||
|
|
||||||
1. **Identify the review mode** (pending-change vs snippet vs file-focused) based on the user’s input. Keep the scope tight: review only what the user provided or explicitly referenced.
|
|
||||||
2. Follow the rules defined in **Checklist** to perform the review. If no Checklist rule matches, apply **General Review Rules** as a fallback to perform the best-effort review.
|
|
||||||
3. Compose the final output strictly follow the **Required Output Format**.
|
|
||||||
|
|
||||||
Notes when using this skill:
|
|
||||||
- Always include actionable fixes or suggestions (including possible code snippets).
|
|
||||||
- Use best-effort `File:Line` references when a file path and line numbers are available; otherwise, use the most specific identifier you can.
|
|
||||||
|
|
||||||
## Checklist
|
|
||||||
|
|
||||||
- db schema design: if the review scope includes code/files under `api/models/` or `api/migrations/`, follow [references/db-schema-rule.md](references/db-schema-rule.md) to perform the review
|
|
||||||
- architecture: if the review scope involves controller/service/core-domain/libs/model layering, dependency direction, or moving responsibilities across modules, follow [references/architecture-rule.md](references/architecture-rule.md) to perform the review
|
|
||||||
- repositories abstraction: if the review scope contains table/model operations (e.g., `select(...)`, `session.execute(...)`, joins, CRUD) and is not under `api/repositories`, `api/core/repositories`, or `api/extensions/*/repositories/`, follow [references/repositories-rule.md](references/repositories-rule.md) to perform the review
|
|
||||||
- sqlalchemy patterns: if the review scope involves SQLAlchemy session/query usage, db transaction/crud usage, or raw SQL usage, follow [references/sqlalchemy-rule.md](references/sqlalchemy-rule.md) to perform the review
|
|
||||||
|
|
||||||
## General Review Rules
|
|
||||||
|
|
||||||
### 1. Security Review
|
|
||||||
|
|
||||||
Check for:
|
|
||||||
- SQL injection vulnerabilities
|
|
||||||
- Server-Side Request Forgery (SSRF)
|
|
||||||
- Command injection
|
|
||||||
- Insecure deserialization
|
|
||||||
- Hardcoded secrets/credentials
|
|
||||||
- Improper authentication/authorization
|
|
||||||
- Insecure direct object references
|
|
||||||
|
|
||||||
### 2. Performance Review
|
|
||||||
|
|
||||||
Check for:
|
|
||||||
- N+1 queries
|
|
||||||
- Missing database indexes
|
|
||||||
- Memory leaks
|
|
||||||
- Blocking operations in async code
|
|
||||||
- Missing caching opportunities
|
|
||||||
|
|
||||||
### 3. Code Quality Review
|
|
||||||
|
|
||||||
Check for:
|
|
||||||
- Code forward compatibility
|
|
||||||
- Code duplication (DRY violations)
|
|
||||||
- Functions doing too much (SRP violations)
|
|
||||||
- Deep nesting / complex conditionals
|
|
||||||
- Magic numbers/strings
|
|
||||||
- Poor naming
|
|
||||||
- Missing error handling
|
|
||||||
- Incomplete type coverage
|
|
||||||
|
|
||||||
### 4. Testing Review
|
|
||||||
|
|
||||||
Check for:
|
|
||||||
- Missing test coverage for new code
|
|
||||||
- Tests that don't test behavior
|
|
||||||
- Flaky test patterns
|
|
||||||
- Missing edge cases
|
|
||||||
|
|
||||||
## Required Output Format
|
|
||||||
|
|
||||||
When this skill invoked, the response must exactly follow one of the two templates:
|
|
||||||
|
|
||||||
### Template A (any findings)
|
|
||||||
|
|
||||||
```markdown
|
|
||||||
# Code Review Summary
|
|
||||||
|
|
||||||
Found <X> critical issues need to be fixed:
|
|
||||||
|
|
||||||
## 🔴 Critical (Must Fix)
|
|
||||||
|
|
||||||
### 1. <brief description of the issue>
|
|
||||||
|
|
||||||
FilePath: <path> line <line>
|
|
||||||
<relevant code snippet or pointer>
|
|
||||||
|
|
||||||
#### Explanation
|
|
||||||
|
|
||||||
<detailed explanation and references of the issue>
|
|
||||||
|
|
||||||
#### Suggested Fix
|
|
||||||
|
|
||||||
1. <brief description of suggested fix>
|
|
||||||
2. <code example> (optional, omit if not applicable)
|
|
||||||
|
|
||||||
---
|
|
||||||
... (repeat for each critical issue) ...
|
|
||||||
|
|
||||||
Found <Y> suggestions for improvement:
|
|
||||||
|
|
||||||
## 🟡 Suggestions (Should Consider)
|
|
||||||
|
|
||||||
### 1. <brief description of the suggestion>
|
|
||||||
|
|
||||||
FilePath: <path> line <line>
|
|
||||||
<relevant code snippet or pointer>
|
|
||||||
|
|
||||||
#### Explanation
|
|
||||||
|
|
||||||
<detailed explanation and references of the suggestion>
|
|
||||||
|
|
||||||
#### Suggested Fix
|
|
||||||
|
|
||||||
1. <brief description of suggested fix>
|
|
||||||
2. <code example> (optional, omit if not applicable)
|
|
||||||
|
|
||||||
---
|
|
||||||
... (repeat for each suggestion) ...
|
|
||||||
|
|
||||||
Found <Z> optional nits:
|
|
||||||
|
|
||||||
## 🟢 Nits (Optional)
|
|
||||||
### 1. <brief description of the nit>
|
|
||||||
|
|
||||||
FilePath: <path> line <line>
|
|
||||||
<relevant code snippet or pointer>
|
|
||||||
|
|
||||||
#### Explanation
|
|
||||||
|
|
||||||
<explanation and references of the optional nit>
|
|
||||||
|
|
||||||
#### Suggested Fix
|
|
||||||
|
|
||||||
- <minor suggestions>
|
|
||||||
|
|
||||||
---
|
|
||||||
... (repeat for each nits) ...
|
|
||||||
|
|
||||||
## ✅ What's Good
|
|
||||||
|
|
||||||
- <Positive feedback on good patterns>
|
|
||||||
```
|
|
||||||
|
|
||||||
- If there are no critical issues or suggestions or option nits or good points, just omit that section.
|
|
||||||
- If the issue number is more than 10, summarize as "Found 10+ critical issues/suggestions/optional nits" and only output the first 10 items.
|
|
||||||
- Don't compress the blank lines between sections; keep them as-is for readability.
|
|
||||||
- If there is any issue requires code changes, append a brief follow-up question to ask whether the user wants to apply the fix(es) after the structured output. For example: "Would you like me to use the Suggested fix(es) to address these issues?"
|
|
||||||
|
|
||||||
### Template B (no issues)
|
|
||||||
|
|
||||||
```markdown
|
|
||||||
## Code Review Summary
|
|
||||||
✅ No issues found.
|
|
||||||
```
|
|
||||||
@@ -1,91 +0,0 @@
|
|||||||
# Rule Catalog — Architecture
|
|
||||||
|
|
||||||
## Scope
|
|
||||||
- Covers: controller/service/core-domain/libs/model layering, dependency direction, responsibility placement, observability-friendly flow.
|
|
||||||
|
|
||||||
## Rules
|
|
||||||
|
|
||||||
### Keep business logic out of controllers
|
|
||||||
- Category: maintainability
|
|
||||||
- Severity: critical
|
|
||||||
- Description: Controllers should parse input, call services, and return serialized responses. Business decisions inside controllers make behavior hard to reuse and test.
|
|
||||||
- Suggested fix: Move domain/business logic into the service or core/domain layer. Keep controller handlers thin and orchestration-focused.
|
|
||||||
- Example:
|
|
||||||
- Bad:
|
|
||||||
```python
|
|
||||||
@bp.post("/apps/<app_id>/publish")
|
|
||||||
def publish_app(app_id: str):
|
|
||||||
payload = request.get_json() or {}
|
|
||||||
if payload.get("force") and current_user.role != "admin":
|
|
||||||
raise ValueError("only admin can force publish")
|
|
||||||
app = App.query.get(app_id)
|
|
||||||
app.status = "published"
|
|
||||||
db.session.commit()
|
|
||||||
return {"result": "ok"}
|
|
||||||
```
|
|
||||||
- Good:
|
|
||||||
```python
|
|
||||||
@bp.post("/apps/<app_id>/publish")
|
|
||||||
def publish_app(app_id: str):
|
|
||||||
payload = PublishRequest.model_validate(request.get_json() or {})
|
|
||||||
app_service.publish_app(app_id=app_id, force=payload.force, actor_id=current_user.id)
|
|
||||||
return {"result": "ok"}
|
|
||||||
```
|
|
||||||
|
|
||||||
### Preserve layer dependency direction
|
|
||||||
- Category: best practices
|
|
||||||
- Severity: critical
|
|
||||||
- Description: Controllers may depend on services, and services may depend on core/domain abstractions. Reversing this direction (for example, core importing controller/web modules) creates cycles and leaks transport concerns into domain code.
|
|
||||||
- Suggested fix: Extract shared contracts into core/domain or service-level modules and make upper layers depend on lower, not the reverse.
|
|
||||||
- Example:
|
|
||||||
- Bad:
|
|
||||||
```python
|
|
||||||
# core/policy/publish_policy.py
|
|
||||||
from controllers.console.app import request_context
|
|
||||||
|
|
||||||
def can_publish() -> bool:
|
|
||||||
return request_context.current_user.is_admin
|
|
||||||
```
|
|
||||||
- Good:
|
|
||||||
```python
|
|
||||||
# core/policy/publish_policy.py
|
|
||||||
def can_publish(role: str) -> bool:
|
|
||||||
return role == "admin"
|
|
||||||
|
|
||||||
# service layer adapts web/user context to domain input
|
|
||||||
allowed = can_publish(role=current_user.role)
|
|
||||||
```
|
|
||||||
|
|
||||||
### Keep libs business-agnostic
|
|
||||||
- Category: maintainability
|
|
||||||
- Severity: critical
|
|
||||||
- Description: Modules under `api/libs/` should remain reusable, business-agnostic building blocks. They must not encode product/domain-specific rules, workflow orchestration, or business decisions.
|
|
||||||
- Suggested fix:
|
|
||||||
- If business logic appears in `api/libs/`, extract it into the appropriate `services/` or `core/` module and keep `libs` focused on generic, cross-cutting helpers.
|
|
||||||
- Keep `libs` dependencies clean: avoid importing service/controller/domain-specific modules into `api/libs/`.
|
|
||||||
- Example:
|
|
||||||
- Bad:
|
|
||||||
```python
|
|
||||||
# api/libs/conversation_filter.py
|
|
||||||
from services.conversation_service import ConversationService
|
|
||||||
|
|
||||||
def should_archive_conversation(conversation, tenant_id: str) -> bool:
|
|
||||||
# Domain policy and service dependency are leaking into libs.
|
|
||||||
service = ConversationService()
|
|
||||||
if service.has_paid_plan(tenant_id):
|
|
||||||
return conversation.idle_days > 90
|
|
||||||
return conversation.idle_days > 30
|
|
||||||
```
|
|
||||||
- Good:
|
|
||||||
```python
|
|
||||||
# api/libs/datetime_utils.py (business-agnostic helper)
|
|
||||||
def older_than_days(idle_days: int, threshold_days: int) -> bool:
|
|
||||||
return idle_days > threshold_days
|
|
||||||
|
|
||||||
# services/conversation_service.py (business logic stays in service/core)
|
|
||||||
from libs.datetime_utils import older_than_days
|
|
||||||
|
|
||||||
def should_archive_conversation(conversation, tenant_id: str) -> bool:
|
|
||||||
threshold_days = 90 if has_paid_plan(tenant_id) else 30
|
|
||||||
return older_than_days(conversation.idle_days, threshold_days)
|
|
||||||
```
|
|
||||||
@@ -1,157 +0,0 @@
|
|||||||
# Rule Catalog — DB Schema Design
|
|
||||||
|
|
||||||
## Scope
|
|
||||||
- Covers: model/base inheritance, schema boundaries in model properties, tenant-aware schema design, index redundancy checks, dialect portability in models, and cross-database compatibility in migrations.
|
|
||||||
- Does NOT cover: session lifecycle, transaction boundaries, and query execution patterns (handled by `sqlalchemy-rule.md`).
|
|
||||||
|
|
||||||
## Rules
|
|
||||||
|
|
||||||
### Do not query other tables inside `@property`
|
|
||||||
- Category: [maintainability, performance]
|
|
||||||
- Severity: critical
|
|
||||||
- Description: A model `@property` must not open sessions or query other tables. This hides dependencies across models, tightly couples schema objects to data access, and can cause N+1 query explosions when iterating collections.
|
|
||||||
- Suggested fix:
|
|
||||||
- Keep model properties pure and local to already-loaded fields.
|
|
||||||
- Move cross-table data fetching to service/repository methods.
|
|
||||||
- For list/batch reads, fetch required related data explicitly (join/preload/bulk query) before rendering derived values.
|
|
||||||
- Example:
|
|
||||||
- Bad:
|
|
||||||
```python
|
|
||||||
class Conversation(TypeBase):
|
|
||||||
__tablename__ = "conversations"
|
|
||||||
|
|
||||||
@property
|
|
||||||
def app_name(self) -> str:
|
|
||||||
with Session(db.engine, expire_on_commit=False) as session:
|
|
||||||
app = session.execute(select(App).where(App.id == self.app_id)).scalar_one()
|
|
||||||
return app.name
|
|
||||||
```
|
|
||||||
- Good:
|
|
||||||
```python
|
|
||||||
class Conversation(TypeBase):
|
|
||||||
__tablename__ = "conversations"
|
|
||||||
|
|
||||||
@property
|
|
||||||
def display_title(self) -> str:
|
|
||||||
return self.name or "Untitled"
|
|
||||||
|
|
||||||
|
|
||||||
# Service/repository layer performs explicit batch fetch for related App rows.
|
|
||||||
```
|
|
||||||
|
|
||||||
### Prefer including `tenant_id` in model definitions
|
|
||||||
- Category: maintainability
|
|
||||||
- Severity: suggestion
|
|
||||||
- Description: In multi-tenant domains, include `tenant_id` in schema definitions whenever the entity belongs to tenant-owned data. This improves data isolation safety and keeps future partitioning/sharding strategies practical as data volume grows.
|
|
||||||
- Suggested fix:
|
|
||||||
- Add a `tenant_id` column and ensure related unique/index constraints include tenant dimension when applicable.
|
|
||||||
- Propagate `tenant_id` through service/repository contracts to keep access paths tenant-aware.
|
|
||||||
- Exception: if a table is explicitly designed as non-tenant-scoped global metadata, document that design decision clearly.
|
|
||||||
- Example:
|
|
||||||
- Bad:
|
|
||||||
```python
|
|
||||||
from sqlalchemy.orm import Mapped
|
|
||||||
|
|
||||||
class Dataset(TypeBase):
|
|
||||||
__tablename__ = "datasets"
|
|
||||||
id: Mapped[str] = mapped_column(StringUUID, primary_key=True)
|
|
||||||
name: Mapped[str] = mapped_column(sa.String(255), nullable=False)
|
|
||||||
```
|
|
||||||
- Good:
|
|
||||||
```python
|
|
||||||
from sqlalchemy.orm import Mapped
|
|
||||||
|
|
||||||
class Dataset(TypeBase):
|
|
||||||
__tablename__ = "datasets"
|
|
||||||
id: Mapped[str] = mapped_column(StringUUID, primary_key=True)
|
|
||||||
tenant_id: Mapped[str] = mapped_column(StringUUID, nullable=False, index=True)
|
|
||||||
name: Mapped[str] = mapped_column(sa.String(255), nullable=False)
|
|
||||||
```
|
|
||||||
|
|
||||||
### Detect and avoid duplicate/redundant indexes
|
|
||||||
- Category: performance
|
|
||||||
- Severity: suggestion
|
|
||||||
- Description: Review index definitions for leftmost-prefix redundancy. For example, index `(a, b, c)` can safely cover most lookups for `(a, b)`. Keeping both may increase write overhead and can mislead the optimizer into suboptimal execution plans.
|
|
||||||
- Suggested fix:
|
|
||||||
- Before adding an index, compare against existing composite indexes by leftmost-prefix rules.
|
|
||||||
- Drop or avoid creating redundant prefixes unless there is a proven query-pattern need.
|
|
||||||
- Apply the same review standard in both model `__table_args__` and migration index DDL.
|
|
||||||
- Example:
|
|
||||||
- Bad:
|
|
||||||
```python
|
|
||||||
__table_args__ = (
|
|
||||||
sa.Index("idx_msg_tenant_app", "tenant_id", "app_id"),
|
|
||||||
sa.Index("idx_msg_tenant_app_created", "tenant_id", "app_id", "created_at"),
|
|
||||||
)
|
|
||||||
```
|
|
||||||
- Good:
|
|
||||||
```python
|
|
||||||
__table_args__ = (
|
|
||||||
# Keep the wider index unless profiling proves a dedicated short index is needed.
|
|
||||||
sa.Index("idx_msg_tenant_app_created", "tenant_id", "app_id", "created_at"),
|
|
||||||
)
|
|
||||||
```
|
|
||||||
|
|
||||||
### Avoid PostgreSQL-only dialect usage in models; wrap in `models.types`
|
|
||||||
- Category: maintainability
|
|
||||||
- Severity: critical
|
|
||||||
- Description: Model/schema definitions should avoid PostgreSQL-only constructs directly in business models. When database-specific behavior is required, encapsulate it in `api/models/types.py` using both PostgreSQL and MySQL dialect implementations, then consume that abstraction from model code.
|
|
||||||
- Suggested fix:
|
|
||||||
- Do not directly place dialect-only types/operators in model columns when a portable wrapper can be used.
|
|
||||||
- Add or extend wrappers in `models.types` (for example, `AdjustedJSON`, `LongText`, `BinaryData`) to normalize behavior across PostgreSQL and MySQL.
|
|
||||||
- Example:
|
|
||||||
- Bad:
|
|
||||||
```python
|
|
||||||
from sqlalchemy.dialects.postgresql import JSONB
|
|
||||||
from sqlalchemy.orm import Mapped
|
|
||||||
|
|
||||||
class ToolConfig(TypeBase):
|
|
||||||
__tablename__ = "tool_configs"
|
|
||||||
config: Mapped[dict] = mapped_column(JSONB, nullable=False)
|
|
||||||
```
|
|
||||||
- Good:
|
|
||||||
```python
|
|
||||||
from sqlalchemy.orm import Mapped
|
|
||||||
|
|
||||||
from models.types import AdjustedJSON
|
|
||||||
|
|
||||||
class ToolConfig(TypeBase):
|
|
||||||
__tablename__ = "tool_configs"
|
|
||||||
config: Mapped[dict] = mapped_column(AdjustedJSON(), nullable=False)
|
|
||||||
```
|
|
||||||
|
|
||||||
### Guard migration incompatibilities with dialect checks and shared types
|
|
||||||
- Category: maintainability
|
|
||||||
- Severity: critical
|
|
||||||
- Description: Migration scripts under `api/migrations/versions/` must account for PostgreSQL/MySQL incompatibilities explicitly. For dialect-sensitive DDL or defaults, branch on the active dialect (for example, `conn.dialect.name == "postgresql"`), and prefer reusable compatibility abstractions from `models.types` where applicable.
|
|
||||||
- Suggested fix:
|
|
||||||
- In migration upgrades/downgrades, bind connection and branch by dialect for incompatible SQL fragments.
|
|
||||||
- Reuse `models.types` wrappers in column definitions when that keeps behavior aligned with runtime models.
|
|
||||||
- Avoid one-dialect-only migration logic unless there is a documented, deliberate compatibility exception.
|
|
||||||
- Example:
|
|
||||||
- Bad:
|
|
||||||
```python
|
|
||||||
with op.batch_alter_table("dataset_keyword_tables") as batch_op:
|
|
||||||
batch_op.add_column(
|
|
||||||
sa.Column(
|
|
||||||
"data_source_type",
|
|
||||||
sa.String(255),
|
|
||||||
server_default=sa.text("'database'::character varying"),
|
|
||||||
nullable=False,
|
|
||||||
)
|
|
||||||
)
|
|
||||||
```
|
|
||||||
- Good:
|
|
||||||
```python
|
|
||||||
def _is_pg(conn) -> bool:
|
|
||||||
return conn.dialect.name == "postgresql"
|
|
||||||
|
|
||||||
|
|
||||||
conn = op.get_bind()
|
|
||||||
default_expr = sa.text("'database'::character varying") if _is_pg(conn) else sa.text("'database'")
|
|
||||||
|
|
||||||
with op.batch_alter_table("dataset_keyword_tables") as batch_op:
|
|
||||||
batch_op.add_column(
|
|
||||||
sa.Column("data_source_type", sa.String(255), server_default=default_expr, nullable=False)
|
|
||||||
)
|
|
||||||
```
|
|
||||||
@@ -1,61 +0,0 @@
|
|||||||
# Rule Catalog - Repositories Abstraction
|
|
||||||
|
|
||||||
## Scope
|
|
||||||
- Covers: when to reuse existing repository abstractions, when to introduce new repositories, and how to preserve dependency direction between service/core and infrastructure implementations.
|
|
||||||
- Does NOT cover: SQLAlchemy session lifecycle and query-shape specifics (handled by `sqlalchemy-rule.md`), and table schema/migration design (handled by `db-schema-rule.md`).
|
|
||||||
|
|
||||||
## Rules
|
|
||||||
|
|
||||||
### Introduce repositories abstraction
|
|
||||||
- Category: maintainability
|
|
||||||
- Severity: suggestion
|
|
||||||
- Description: If a table/model already has a repository abstraction, all reads/writes/queries for that table should use the existing repository. If no repository exists, introduce one only when complexity justifies it, such as large/high-volume tables, repeated complex query logic, or likely storage-strategy variation.
|
|
||||||
- Suggested fix:
|
|
||||||
- First check `api/repositories`, `api/core/repositories`, and `api/extensions/*/repositories/` to verify whether the table/model already has a repository abstraction. If it exists, route all operations through it and add missing repository methods instead of bypassing it with ad-hoc SQLAlchemy access.
|
|
||||||
- If no repository exists, add one only when complexity warrants it (for example, repeated complex queries, large data domains, or multiple storage strategies), while preserving dependency direction (service/core depends on abstraction; infra provides implementation).
|
|
||||||
- Example:
|
|
||||||
- Bad:
|
|
||||||
```python
|
|
||||||
# Existing repository is ignored and service uses ad-hoc table queries.
|
|
||||||
class AppService:
|
|
||||||
def archive_app(self, app_id: str, tenant_id: str) -> None:
|
|
||||||
app = self.session.execute(
|
|
||||||
select(App).where(App.id == app_id, App.tenant_id == tenant_id)
|
|
||||||
).scalar_one()
|
|
||||||
app.archived = True
|
|
||||||
self.session.commit()
|
|
||||||
```
|
|
||||||
- Good:
|
|
||||||
```python
|
|
||||||
# Case A: Existing repository must be reused for all table operations.
|
|
||||||
class AppService:
|
|
||||||
def archive_app(self, app_id: str, tenant_id: str) -> None:
|
|
||||||
app = self.app_repo.get_by_id(app_id=app_id, tenant_id=tenant_id)
|
|
||||||
app.archived = True
|
|
||||||
self.app_repo.save(app)
|
|
||||||
|
|
||||||
# If the query is missing, extend the existing abstraction.
|
|
||||||
active_apps = self.app_repo.list_active_for_tenant(tenant_id=tenant_id)
|
|
||||||
```
|
|
||||||
- Bad:
|
|
||||||
```python
|
|
||||||
# No repository exists, but large-domain query logic is scattered in service code.
|
|
||||||
class ConversationService:
|
|
||||||
def list_recent_for_app(self, app_id: str, tenant_id: str, limit: int) -> list[Conversation]:
|
|
||||||
...
|
|
||||||
# many filters/joins/pagination variants duplicated across services
|
|
||||||
```
|
|
||||||
- Good:
|
|
||||||
```python
|
|
||||||
# Case B: Introduce repository for large/complex domains or storage variation.
|
|
||||||
class ConversationRepository(Protocol):
|
|
||||||
def list_recent_for_app(self, app_id: str, tenant_id: str, limit: int) -> list[Conversation]: ...
|
|
||||||
|
|
||||||
class SqlAlchemyConversationRepository:
|
|
||||||
def list_recent_for_app(self, app_id: str, tenant_id: str, limit: int) -> list[Conversation]:
|
|
||||||
...
|
|
||||||
|
|
||||||
class ConversationService:
|
|
||||||
def __init__(self, conversation_repo: ConversationRepository):
|
|
||||||
self.conversation_repo = conversation_repo
|
|
||||||
```
|
|
||||||
@@ -1,139 +0,0 @@
|
|||||||
# Rule Catalog — SQLAlchemy Patterns
|
|
||||||
|
|
||||||
## Scope
|
|
||||||
- Covers: SQLAlchemy session and transaction lifecycle, query construction, tenant scoping, raw SQL boundaries, and write-path concurrency safeguards.
|
|
||||||
- Does NOT cover: table/model schema and migration design details (handled by `db-schema-rule.md`).
|
|
||||||
|
|
||||||
## Rules
|
|
||||||
|
|
||||||
### Use Session context manager with explicit transaction control behavior
|
|
||||||
- Category: best practices
|
|
||||||
- Severity: critical
|
|
||||||
- Description: Session and transaction lifecycle must be explicit and bounded on write paths. Missing commits can silently drop intended updates, while ad-hoc or long-lived transactions increase contention, lock duration, and deadlock risk.
|
|
||||||
- Suggested fix:
|
|
||||||
- Use **explicit `session.commit()`** after completing a related write unit.
|
|
||||||
- Or use **`session.begin()` context manager** for automatic commit/rollback on a scoped block.
|
|
||||||
- Keep transaction windows short: avoid network I/O, heavy computation, or unrelated work inside the transaction.
|
|
||||||
- Example:
|
|
||||||
- Bad:
|
|
||||||
```python
|
|
||||||
# Missing commit: write may never be persisted.
|
|
||||||
with Session(db.engine, expire_on_commit=False) as session:
|
|
||||||
run = session.get(WorkflowRun, run_id)
|
|
||||||
run.status = "cancelled"
|
|
||||||
|
|
||||||
# Long transaction: external I/O inside a DB transaction.
|
|
||||||
with Session(db.engine, expire_on_commit=False) as session, session.begin():
|
|
||||||
run = session.get(WorkflowRun, run_id)
|
|
||||||
run.status = "cancelled"
|
|
||||||
call_external_api()
|
|
||||||
```
|
|
||||||
- Good:
|
|
||||||
```python
|
|
||||||
# Option 1: explicit commit.
|
|
||||||
with Session(db.engine, expire_on_commit=False) as session:
|
|
||||||
run = session.get(WorkflowRun, run_id)
|
|
||||||
run.status = "cancelled"
|
|
||||||
session.commit()
|
|
||||||
|
|
||||||
# Option 2: scoped transaction with automatic commit/rollback.
|
|
||||||
with Session(db.engine, expire_on_commit=False) as session, session.begin():
|
|
||||||
run = session.get(WorkflowRun, run_id)
|
|
||||||
run.status = "cancelled"
|
|
||||||
|
|
||||||
# Keep non-DB work outside transaction scope.
|
|
||||||
call_external_api()
|
|
||||||
```
|
|
||||||
|
|
||||||
### Enforce tenant_id scoping on shared-resource queries
|
|
||||||
- Category: security
|
|
||||||
- Severity: critical
|
|
||||||
- Description: Reads and writes against shared tables must be scoped by `tenant_id` to prevent cross-tenant data leakage or corruption.
|
|
||||||
- Suggested fix: Add `tenant_id` predicate to all tenant-owned entity queries and propagate tenant context through service/repository interfaces.
|
|
||||||
- Example:
|
|
||||||
- Bad:
|
|
||||||
```python
|
|
||||||
stmt = select(Workflow).where(Workflow.id == workflow_id)
|
|
||||||
workflow = session.execute(stmt).scalar_one_or_none()
|
|
||||||
```
|
|
||||||
- Good:
|
|
||||||
```python
|
|
||||||
stmt = select(Workflow).where(
|
|
||||||
Workflow.id == workflow_id,
|
|
||||||
Workflow.tenant_id == tenant_id,
|
|
||||||
)
|
|
||||||
workflow = session.execute(stmt).scalar_one_or_none()
|
|
||||||
```
|
|
||||||
|
|
||||||
### Prefer SQLAlchemy expressions over raw SQL by default
|
|
||||||
- Category: maintainability
|
|
||||||
- Severity: suggestion
|
|
||||||
- Description: Raw SQL should be exceptional. ORM/Core expressions are easier to evolve, safer to compose, and more consistent with the codebase.
|
|
||||||
- Suggested fix: Rewrite straightforward raw SQL into SQLAlchemy `select/update/delete` expressions; keep raw SQL only when required by clear technical constraints.
|
|
||||||
- Example:
|
|
||||||
- Bad:
|
|
||||||
```python
|
|
||||||
row = session.execute(
|
|
||||||
text("SELECT * FROM workflows WHERE id = :id AND tenant_id = :tenant_id"),
|
|
||||||
{"id": workflow_id, "tenant_id": tenant_id},
|
|
||||||
).first()
|
|
||||||
```
|
|
||||||
- Good:
|
|
||||||
```python
|
|
||||||
stmt = select(Workflow).where(
|
|
||||||
Workflow.id == workflow_id,
|
|
||||||
Workflow.tenant_id == tenant_id,
|
|
||||||
)
|
|
||||||
row = session.execute(stmt).scalar_one_or_none()
|
|
||||||
```
|
|
||||||
|
|
||||||
### Protect write paths with concurrency safeguards
|
|
||||||
- Category: quality
|
|
||||||
- Severity: critical
|
|
||||||
- Description: Multi-writer paths without explicit concurrency control can silently overwrite data. Choose the safeguard based on contention level, lock scope, and throughput cost instead of defaulting to one strategy.
|
|
||||||
- Suggested fix:
|
|
||||||
- **Optimistic locking**: Use when contention is usually low and retries are acceptable. Add a version (or updated_at) guard in `WHERE` and treat `rowcount == 0` as a conflict.
|
|
||||||
- **Redis distributed lock**: Use when the critical section spans multiple steps/processes (or includes non-DB side effects) and you need cross-worker mutual exclusion.
|
|
||||||
- **SELECT ... FOR UPDATE**: Use when contention is high on the same rows and strict in-transaction serialization is required. Keep transactions short to reduce lock wait/deadlock risk.
|
|
||||||
- In all cases, scope by `tenant_id` and verify affected row counts for conditional writes.
|
|
||||||
- Example:
|
|
||||||
- Bad:
|
|
||||||
```python
|
|
||||||
# No tenant scope, no conflict detection, and no lock on a contested write path.
|
|
||||||
session.execute(update(WorkflowRun).where(WorkflowRun.id == run_id).values(status="cancelled"))
|
|
||||||
session.commit() # silently overwrites concurrent updates
|
|
||||||
```
|
|
||||||
- Good:
|
|
||||||
```python
|
|
||||||
# 1) Optimistic lock (low contention, retry on conflict)
|
|
||||||
result = session.execute(
|
|
||||||
update(WorkflowRun)
|
|
||||||
.where(
|
|
||||||
WorkflowRun.id == run_id,
|
|
||||||
WorkflowRun.tenant_id == tenant_id,
|
|
||||||
WorkflowRun.version == expected_version,
|
|
||||||
)
|
|
||||||
.values(status="cancelled", version=WorkflowRun.version + 1)
|
|
||||||
)
|
|
||||||
if result.rowcount == 0:
|
|
||||||
raise WorkflowStateConflictError("stale version, retry")
|
|
||||||
|
|
||||||
# 2) Redis distributed lock (cross-worker critical section)
|
|
||||||
lock_name = f"workflow_run_lock:{tenant_id}:{run_id}"
|
|
||||||
with redis_client.lock(lock_name, timeout=20):
|
|
||||||
session.execute(
|
|
||||||
update(WorkflowRun)
|
|
||||||
.where(WorkflowRun.id == run_id, WorkflowRun.tenant_id == tenant_id)
|
|
||||||
.values(status="cancelled")
|
|
||||||
)
|
|
||||||
session.commit()
|
|
||||||
|
|
||||||
# 3) Pessimistic lock with SELECT ... FOR UPDATE (high contention)
|
|
||||||
run = session.execute(
|
|
||||||
select(WorkflowRun)
|
|
||||||
.where(WorkflowRun.id == run_id, WorkflowRun.tenant_id == tenant_id)
|
|
||||||
.with_for_update()
|
|
||||||
).scalar_one()
|
|
||||||
run.status = "cancelled"
|
|
||||||
session.commit()
|
|
||||||
```
|
|
||||||
@@ -1 +0,0 @@
|
|||||||
../../.agents/skills/backend-code-review
|
|
||||||
+1
-1
@@ -36,7 +36,7 @@
|
|||||||
/api/core/workflow/graph/ @laipz8200 @QuantumGhost
|
/api/core/workflow/graph/ @laipz8200 @QuantumGhost
|
||||||
/api/core/workflow/graph_events/ @laipz8200 @QuantumGhost
|
/api/core/workflow/graph_events/ @laipz8200 @QuantumGhost
|
||||||
/api/core/workflow/node_events/ @laipz8200 @QuantumGhost
|
/api/core/workflow/node_events/ @laipz8200 @QuantumGhost
|
||||||
/api/dify_graph/model_runtime/ @laipz8200 @QuantumGhost
|
/api/core/model_runtime/ @laipz8200 @QuantumGhost
|
||||||
|
|
||||||
# Backend - Workflow - Nodes (Agent, Iteration, Loop, LLM)
|
# Backend - Workflow - Nodes (Agent, Iteration, Loop, LLM)
|
||||||
/api/core/workflow/nodes/agent/ @Nov1c444
|
/api/core/workflow/nodes/agent/ @Nov1c444
|
||||||
|
|||||||
+5
-18
@@ -1,25 +1,12 @@
|
|||||||
version: 2
|
version: 2
|
||||||
|
|
||||||
multi-ecosystem-groups:
|
|
||||||
python:
|
|
||||||
schedule:
|
|
||||||
interval: "weekly" # or whatever schedule you want
|
|
||||||
|
|
||||||
updates:
|
updates:
|
||||||
- package-ecosystem: "pip"
|
|
||||||
directory: "/api"
|
|
||||||
open-pull-requests-limit: 2
|
|
||||||
patterns: ["*"]
|
|
||||||
schedule:
|
|
||||||
interval: "weekly"
|
|
||||||
- package-ecosystem: "uv"
|
|
||||||
directory: "/api"
|
|
||||||
open-pull-requests-limit: 2
|
|
||||||
patterns: ["*"]
|
|
||||||
schedule:
|
|
||||||
interval: "weekly"
|
|
||||||
- package-ecosystem: "npm"
|
- package-ecosystem: "npm"
|
||||||
directory: "/web"
|
directory: "/web"
|
||||||
schedule:
|
schedule:
|
||||||
interval: "weekly"
|
interval: "weekly"
|
||||||
open-pull-requests-limit: 2
|
open-pull-requests-limit: 2
|
||||||
|
- package-ecosystem: "uv"
|
||||||
|
directory: "/api"
|
||||||
|
schedule:
|
||||||
|
interval: "weekly"
|
||||||
|
open-pull-requests-limit: 2
|
||||||
|
|||||||
@@ -1,88 +0,0 @@
|
|||||||
name: Comment with Pyrefly Diff
|
|
||||||
|
|
||||||
on:
|
|
||||||
workflow_run:
|
|
||||||
workflows:
|
|
||||||
- Pyrefly Diff Check
|
|
||||||
types:
|
|
||||||
- completed
|
|
||||||
|
|
||||||
permissions: {}
|
|
||||||
|
|
||||||
jobs:
|
|
||||||
comment:
|
|
||||||
name: Comment PR with pyrefly diff
|
|
||||||
runs-on: ubuntu-latest
|
|
||||||
permissions:
|
|
||||||
actions: read
|
|
||||||
contents: read
|
|
||||||
issues: write
|
|
||||||
pull-requests: write
|
|
||||||
if: ${{ github.event.workflow_run.conclusion == 'success' && github.event.workflow_run.pull_requests[0].head.repo.full_name != github.repository }}
|
|
||||||
steps:
|
|
||||||
- name: Download pyrefly diff artifact
|
|
||||||
uses: actions/github-script@v8
|
|
||||||
with:
|
|
||||||
github-token: ${{ secrets.GITHUB_TOKEN }}
|
|
||||||
script: |
|
|
||||||
const fs = require('fs');
|
|
||||||
const artifacts = await github.rest.actions.listWorkflowRunArtifacts({
|
|
||||||
owner: context.repo.owner,
|
|
||||||
repo: context.repo.repo,
|
|
||||||
run_id: ${{ github.event.workflow_run.id }},
|
|
||||||
});
|
|
||||||
const match = artifacts.data.artifacts.find((artifact) =>
|
|
||||||
artifact.name === 'pyrefly_diff'
|
|
||||||
);
|
|
||||||
if (!match) {
|
|
||||||
throw new Error('pyrefly_diff artifact not found');
|
|
||||||
}
|
|
||||||
const download = await github.rest.actions.downloadArtifact({
|
|
||||||
owner: context.repo.owner,
|
|
||||||
repo: context.repo.repo,
|
|
||||||
artifact_id: match.id,
|
|
||||||
archive_format: 'zip',
|
|
||||||
});
|
|
||||||
fs.writeFileSync('pyrefly_diff.zip', Buffer.from(download.data));
|
|
||||||
|
|
||||||
- name: Unzip artifact
|
|
||||||
run: unzip -o pyrefly_diff.zip
|
|
||||||
|
|
||||||
- name: Post comment
|
|
||||||
uses: actions/github-script@v8
|
|
||||||
with:
|
|
||||||
github-token: ${{ secrets.GITHUB_TOKEN }}
|
|
||||||
script: |
|
|
||||||
const fs = require('fs');
|
|
||||||
let diff = fs.readFileSync('pyrefly_diff.txt', { encoding: 'utf8' });
|
|
||||||
let prNumber = null;
|
|
||||||
try {
|
|
||||||
prNumber = parseInt(fs.readFileSync('pr_number.txt', { encoding: 'utf8' }), 10);
|
|
||||||
} catch (err) {
|
|
||||||
// Fallback to workflow_run payload if artifact is missing or incomplete.
|
|
||||||
const prs = context.payload.workflow_run.pull_requests || [];
|
|
||||||
if (prs.length > 0 && prs[0].number) {
|
|
||||||
prNumber = prs[0].number;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
if (!prNumber) {
|
|
||||||
throw new Error('PR number not found in artifact or workflow_run payload');
|
|
||||||
}
|
|
||||||
|
|
||||||
const MAX_CHARS = 65000;
|
|
||||||
if (diff.length > MAX_CHARS) {
|
|
||||||
diff = diff.slice(0, MAX_CHARS);
|
|
||||||
diff = diff.slice(0, diff.lastIndexOf('\\n'));
|
|
||||||
diff += '\\n\\n... (truncated) ...';
|
|
||||||
}
|
|
||||||
|
|
||||||
const body = diff.trim()
|
|
||||||
? '### Pyrefly Diff\n<details>\n<summary>base → PR</summary>\n\n```diff\n' + diff + '\n```\n</details>'
|
|
||||||
: '### Pyrefly Diff\nNo changes detected.';
|
|
||||||
|
|
||||||
await github.rest.issues.createComment({
|
|
||||||
issue_number: prNumber,
|
|
||||||
owner: context.repo.owner,
|
|
||||||
repo: context.repo.repo,
|
|
||||||
body,
|
|
||||||
});
|
|
||||||
@@ -1,100 +0,0 @@
|
|||||||
name: Pyrefly Diff Check
|
|
||||||
|
|
||||||
on:
|
|
||||||
pull_request:
|
|
||||||
paths:
|
|
||||||
- 'api/**/*.py'
|
|
||||||
|
|
||||||
permissions:
|
|
||||||
contents: read
|
|
||||||
|
|
||||||
jobs:
|
|
||||||
pyrefly-diff:
|
|
||||||
runs-on: ubuntu-latest
|
|
||||||
permissions:
|
|
||||||
contents: read
|
|
||||||
issues: write
|
|
||||||
pull-requests: write
|
|
||||||
steps:
|
|
||||||
- name: Checkout PR branch
|
|
||||||
uses: actions/checkout@v6
|
|
||||||
with:
|
|
||||||
fetch-depth: 0
|
|
||||||
|
|
||||||
- name: Setup Python & UV
|
|
||||||
uses: astral-sh/setup-uv@v5
|
|
||||||
with:
|
|
||||||
enable-cache: true
|
|
||||||
|
|
||||||
- name: Install dependencies
|
|
||||||
run: uv sync --project api --dev
|
|
||||||
|
|
||||||
- name: Prepare diagnostics extractor
|
|
||||||
run: |
|
|
||||||
git show ${{ github.event.pull_request.head.sha }}:api/libs/pyrefly_diagnostics.py > /tmp/pyrefly_diagnostics.py
|
|
||||||
|
|
||||||
- name: Run pyrefly on PR branch
|
|
||||||
run: |
|
|
||||||
uv run --directory api --dev pyrefly check 2>&1 \
|
|
||||||
| uv run --directory api python /tmp/pyrefly_diagnostics.py > /tmp/pyrefly_pr.txt || true
|
|
||||||
|
|
||||||
- name: Checkout base branch
|
|
||||||
run: git checkout ${{ github.base_ref }}
|
|
||||||
|
|
||||||
- name: Run pyrefly on base branch
|
|
||||||
run: |
|
|
||||||
uv run --directory api --dev pyrefly check 2>&1 \
|
|
||||||
| uv run --directory api python /tmp/pyrefly_diagnostics.py > /tmp/pyrefly_base.txt || true
|
|
||||||
|
|
||||||
- name: Compute diff
|
|
||||||
run: |
|
|
||||||
diff -u /tmp/pyrefly_base.txt /tmp/pyrefly_pr.txt > pyrefly_diff.txt || true
|
|
||||||
|
|
||||||
- name: Save PR number
|
|
||||||
run: |
|
|
||||||
echo ${{ github.event.pull_request.number }} > pr_number.txt
|
|
||||||
|
|
||||||
- name: Upload pyrefly diff
|
|
||||||
uses: actions/upload-artifact@v4
|
|
||||||
with:
|
|
||||||
name: pyrefly_diff
|
|
||||||
path: |
|
|
||||||
pyrefly_diff.txt
|
|
||||||
pr_number.txt
|
|
||||||
|
|
||||||
- name: Comment PR with pyrefly diff
|
|
||||||
if: ${{ github.event.pull_request.head.repo.full_name == github.repository }}
|
|
||||||
uses: actions/github-script@v8
|
|
||||||
with:
|
|
||||||
github-token: ${{ secrets.GITHUB_TOKEN }}
|
|
||||||
script: |
|
|
||||||
const fs = require('fs');
|
|
||||||
let diff = fs.readFileSync('pyrefly_diff.txt', { encoding: 'utf8' });
|
|
||||||
const prNumber = context.payload.pull_request.number;
|
|
||||||
|
|
||||||
const MAX_CHARS = 65000;
|
|
||||||
if (diff.length > MAX_CHARS) {
|
|
||||||
diff = diff.slice(0, MAX_CHARS);
|
|
||||||
diff = diff.slice(0, diff.lastIndexOf('\n'));
|
|
||||||
diff += '\n\n... (truncated) ...';
|
|
||||||
}
|
|
||||||
|
|
||||||
const body = diff.trim()
|
|
||||||
? [
|
|
||||||
'### Pyrefly Diff',
|
|
||||||
'<details>',
|
|
||||||
'<summary>base → PR</summary>',
|
|
||||||
'',
|
|
||||||
'```diff',
|
|
||||||
diff,
|
|
||||||
'```',
|
|
||||||
'</details>',
|
|
||||||
].join('\n')
|
|
||||||
: '### Pyrefly Diff\nNo changes detected.';
|
|
||||||
|
|
||||||
await github.rest.issues.createComment({
|
|
||||||
issue_number: prNumber,
|
|
||||||
owner: context.repo.owner,
|
|
||||||
repo: context.repo.repo,
|
|
||||||
body,
|
|
||||||
});
|
|
||||||
@@ -3,22 +3,14 @@ name: Web Tests
|
|||||||
on:
|
on:
|
||||||
workflow_call:
|
workflow_call:
|
||||||
|
|
||||||
permissions:
|
|
||||||
contents: read
|
|
||||||
|
|
||||||
concurrency:
|
concurrency:
|
||||||
group: web-tests-${{ github.head_ref || github.run_id }}
|
group: web-tests-${{ github.head_ref || github.run_id }}
|
||||||
cancel-in-progress: true
|
cancel-in-progress: true
|
||||||
|
|
||||||
jobs:
|
jobs:
|
||||||
test:
|
test:
|
||||||
name: Web Tests (${{ matrix.shardIndex }}/${{ matrix.shardTotal }})
|
name: Web Tests
|
||||||
runs-on: ubuntu-latest
|
runs-on: ubuntu-latest
|
||||||
strategy:
|
|
||||||
fail-fast: false
|
|
||||||
matrix:
|
|
||||||
shardIndex: [1, 2, 3, 4]
|
|
||||||
shardTotal: [4]
|
|
||||||
defaults:
|
defaults:
|
||||||
run:
|
run:
|
||||||
shell: bash
|
shell: bash
|
||||||
@@ -47,58 +39,7 @@ jobs:
|
|||||||
run: pnpm install --frozen-lockfile
|
run: pnpm install --frozen-lockfile
|
||||||
|
|
||||||
- name: Run tests
|
- name: Run tests
|
||||||
run: pnpm vitest run --reporter=blob --shard=${{ matrix.shardIndex }}/${{ matrix.shardTotal }} --coverage
|
run: pnpm test:ci
|
||||||
|
|
||||||
- name: Upload blob report
|
|
||||||
if: ${{ !cancelled() }}
|
|
||||||
uses: actions/upload-artifact@v6
|
|
||||||
with:
|
|
||||||
name: blob-report-${{ matrix.shardIndex }}
|
|
||||||
path: web/.vitest-reports/*
|
|
||||||
include-hidden-files: true
|
|
||||||
retention-days: 1
|
|
||||||
|
|
||||||
merge-reports:
|
|
||||||
name: Merge Test Reports
|
|
||||||
if: ${{ !cancelled() }}
|
|
||||||
needs: [test]
|
|
||||||
runs-on: ubuntu-latest
|
|
||||||
defaults:
|
|
||||||
run:
|
|
||||||
shell: bash
|
|
||||||
working-directory: ./web
|
|
||||||
|
|
||||||
steps:
|
|
||||||
- name: Checkout code
|
|
||||||
uses: actions/checkout@v6
|
|
||||||
with:
|
|
||||||
persist-credentials: false
|
|
||||||
|
|
||||||
- name: Install pnpm
|
|
||||||
uses: pnpm/action-setup@v4
|
|
||||||
with:
|
|
||||||
package_json_file: web/package.json
|
|
||||||
run_install: false
|
|
||||||
|
|
||||||
- name: Setup Node.js
|
|
||||||
uses: actions/setup-node@v6
|
|
||||||
with:
|
|
||||||
node-version: 24
|
|
||||||
cache: pnpm
|
|
||||||
cache-dependency-path: ./web/pnpm-lock.yaml
|
|
||||||
|
|
||||||
- name: Install dependencies
|
|
||||||
run: pnpm install --frozen-lockfile
|
|
||||||
|
|
||||||
- name: Download blob reports
|
|
||||||
uses: actions/download-artifact@v6
|
|
||||||
with:
|
|
||||||
path: web/.vitest-reports
|
|
||||||
pattern: blob-report-*
|
|
||||||
merge-multiple: true
|
|
||||||
|
|
||||||
- name: Merge reports
|
|
||||||
run: pnpm vitest --merge-reports --coverage --silent=passed-only
|
|
||||||
|
|
||||||
- name: Coverage Summary
|
- name: Coverage Summary
|
||||||
if: always()
|
if: always()
|
||||||
|
|||||||
@@ -68,9 +68,10 @@ lint:
|
|||||||
@echo "✅ Linting complete"
|
@echo "✅ Linting complete"
|
||||||
|
|
||||||
type-check:
|
type-check:
|
||||||
@echo "📝 Running type checks (basedpyright + mypy)..."
|
@echo "📝 Running type checks (basedpyright + mypy + ty)..."
|
||||||
@./dev/basedpyright-check $(PATH_TO_CHECK)
|
@./dev/basedpyright-check $(PATH_TO_CHECK)
|
||||||
@uv --directory api run mypy --exclude-gitignore --exclude 'tests/' --exclude 'migrations/' --check-untyped-defs --disable-error-code=import-untyped .
|
@uv --directory api run mypy --exclude-gitignore --exclude 'tests/' --exclude 'migrations/' --check-untyped-defs --disable-error-code=import-untyped .
|
||||||
|
@cd api && uv run ty check
|
||||||
@echo "✅ Type checks complete"
|
@echo "✅ Type checks complete"
|
||||||
|
|
||||||
test:
|
test:
|
||||||
@@ -131,7 +132,7 @@ help:
|
|||||||
@echo " make format - Format code with ruff"
|
@echo " make format - Format code with ruff"
|
||||||
@echo " make check - Check code with ruff"
|
@echo " make check - Check code with ruff"
|
||||||
@echo " make lint - Format, fix, and lint code (ruff, imports, dotenv)"
|
@echo " make lint - Format, fix, and lint code (ruff, imports, dotenv)"
|
||||||
@echo " make type-check - Run type checks (basedpyright, mypy)"
|
@echo " make type-check - Run type checks (basedpyright, mypy, ty)"
|
||||||
@echo " make test - Run backend unit tests (or TARGET_TESTS=./api/tests/<target_tests>)"
|
@echo " make test - Run backend unit tests (or TARGET_TESTS=./api/tests/<target_tests>)"
|
||||||
@echo ""
|
@echo ""
|
||||||
@echo "Docker Build Targets:"
|
@echo "Docker Build Targets:"
|
||||||
|
|||||||
@@ -1,5 +1,9 @@
|
|||||||

|

|
||||||
|
|
||||||
|
<p align="center">
|
||||||
|
📌 <a href="https://dify.ai/blog/introducing-dify-workflow-file-upload-a-demo-on-ai-podcast">Introducing Dify Workflow File Upload: Recreate Google NotebookLM Podcast</a>
|
||||||
|
</p>
|
||||||
|
|
||||||
<p align="center">
|
<p align="center">
|
||||||
<a href="https://cloud.dify.ai">Dify Cloud</a> ·
|
<a href="https://cloud.dify.ai">Dify Cloud</a> ·
|
||||||
<a href="https://docs.dify.ai/getting-started/install-self-hosted">Self-hosting</a> ·
|
<a href="https://docs.dify.ai/getting-started/install-self-hosted">Self-hosting</a> ·
|
||||||
|
|||||||
@@ -553,8 +553,6 @@ WORKFLOW_LOG_CLEANUP_ENABLED=false
|
|||||||
WORKFLOW_LOG_RETENTION_DAYS=30
|
WORKFLOW_LOG_RETENTION_DAYS=30
|
||||||
# Batch size for workflow log cleanup operations (default: 100)
|
# Batch size for workflow log cleanup operations (default: 100)
|
||||||
WORKFLOW_LOG_CLEANUP_BATCH_SIZE=100
|
WORKFLOW_LOG_CLEANUP_BATCH_SIZE=100
|
||||||
# Comma-separated list of workflow IDs to clean logs for
|
|
||||||
WORKFLOW_LOG_CLEANUP_SPECIFIC_WORKFLOW_IDS=
|
|
||||||
|
|
||||||
# App configuration
|
# App configuration
|
||||||
APP_MAX_EXECUTION_TIME=1200
|
APP_MAX_EXECUTION_TIME=1200
|
||||||
|
|||||||
+283
-104
@@ -1,7 +1,6 @@
|
|||||||
[importlinter]
|
[importlinter]
|
||||||
root_packages =
|
root_packages =
|
||||||
core
|
core
|
||||||
dify_graph
|
|
||||||
configs
|
configs
|
||||||
controllers
|
controllers
|
||||||
extensions
|
extensions
|
||||||
@@ -22,50 +21,51 @@ layers =
|
|||||||
runtime
|
runtime
|
||||||
entities
|
entities
|
||||||
containers =
|
containers =
|
||||||
dify_graph
|
core.workflow
|
||||||
ignore_imports =
|
ignore_imports =
|
||||||
dify_graph.nodes.base.node -> dify_graph.graph_events
|
core.workflow.nodes.base.node -> core.workflow.graph_events
|
||||||
dify_graph.nodes.iteration.iteration_node -> dify_graph.graph_events
|
core.workflow.nodes.iteration.iteration_node -> core.workflow.graph_events
|
||||||
dify_graph.nodes.loop.loop_node -> dify_graph.graph_events
|
core.workflow.nodes.loop.loop_node -> core.workflow.graph_events
|
||||||
|
|
||||||
dify_graph.nodes.iteration.iteration_node -> core.workflow.node_factory
|
core.workflow.nodes.iteration.iteration_node -> core.app.workflow.node_factory
|
||||||
dify_graph.nodes.loop.loop_node -> core.workflow.node_factory
|
core.workflow.nodes.loop.loop_node -> core.app.workflow.node_factory
|
||||||
dify_graph.nodes.iteration.iteration_node -> core.app.workflow.layers.llm_quota
|
|
||||||
dify_graph.nodes.loop.loop_node -> core.app.workflow.layers.llm_quota
|
|
||||||
|
|
||||||
dify_graph.nodes.iteration.iteration_node -> dify_graph.graph_engine
|
core.workflow.nodes.iteration.iteration_node -> core.workflow.graph_engine
|
||||||
dify_graph.nodes.iteration.iteration_node -> dify_graph.graph
|
core.workflow.nodes.iteration.iteration_node -> core.workflow.graph
|
||||||
dify_graph.nodes.iteration.iteration_node -> dify_graph.graph_engine.command_channels
|
core.workflow.nodes.iteration.iteration_node -> core.workflow.graph_engine.command_channels
|
||||||
dify_graph.nodes.loop.loop_node -> dify_graph.graph_engine
|
core.workflow.nodes.loop.loop_node -> core.workflow.graph_engine
|
||||||
dify_graph.nodes.loop.loop_node -> dify_graph.graph
|
core.workflow.nodes.loop.loop_node -> core.workflow.graph
|
||||||
dify_graph.nodes.loop.loop_node -> dify_graph.graph_engine.command_channels
|
core.workflow.nodes.loop.loop_node -> core.workflow.graph_engine.command_channels
|
||||||
# TODO(QuantumGhost): fix the import violation later
|
# TODO(QuantumGhost): fix the import violation later
|
||||||
dify_graph.entities.pause_reason -> dify_graph.nodes.human_input.entities
|
core.workflow.entities.pause_reason -> core.workflow.nodes.human_input.entities
|
||||||
|
|
||||||
[importlinter:contract:workflow-infrastructure-dependencies]
|
[importlinter:contract:workflow-infrastructure-dependencies]
|
||||||
name = Workflow Infrastructure Dependencies
|
name = Workflow Infrastructure Dependencies
|
||||||
type = forbidden
|
type = forbidden
|
||||||
source_modules =
|
source_modules =
|
||||||
dify_graph
|
core.workflow
|
||||||
forbidden_modules =
|
forbidden_modules =
|
||||||
extensions.ext_database
|
extensions.ext_database
|
||||||
extensions.ext_redis
|
extensions.ext_redis
|
||||||
allow_indirect_imports = True
|
allow_indirect_imports = True
|
||||||
ignore_imports =
|
ignore_imports =
|
||||||
dify_graph.nodes.agent.agent_node -> extensions.ext_database
|
core.workflow.nodes.agent.agent_node -> extensions.ext_database
|
||||||
dify_graph.nodes.llm.file_saver -> extensions.ext_database
|
core.workflow.nodes.datasource.datasource_node -> extensions.ext_database
|
||||||
dify_graph.nodes.llm.node -> extensions.ext_database
|
core.workflow.nodes.knowledge_index.knowledge_index_node -> extensions.ext_database
|
||||||
dify_graph.nodes.tool.tool_node -> extensions.ext_database
|
core.workflow.nodes.llm.file_saver -> extensions.ext_database
|
||||||
dify_graph.model_runtime.model_providers.__base.ai_model -> extensions.ext_redis
|
core.workflow.nodes.llm.llm_utils -> extensions.ext_database
|
||||||
dify_graph.model_runtime.model_providers.model_provider_factory -> extensions.ext_redis
|
core.workflow.nodes.llm.node -> extensions.ext_database
|
||||||
|
core.workflow.nodes.tool.tool_node -> extensions.ext_database
|
||||||
|
core.workflow.graph_engine.command_channels.redis_channel -> extensions.ext_redis
|
||||||
|
core.workflow.graph_engine.manager -> extensions.ext_redis
|
||||||
# TODO(QuantumGhost): use DI to avoid depending on global DB.
|
# TODO(QuantumGhost): use DI to avoid depending on global DB.
|
||||||
dify_graph.nodes.human_input.human_input_node -> extensions.ext_database
|
core.workflow.nodes.human_input.human_input_node -> extensions.ext_database
|
||||||
|
|
||||||
[importlinter:contract:workflow-external-imports]
|
[importlinter:contract:workflow-external-imports]
|
||||||
name = Workflow External Imports
|
name = Workflow External Imports
|
||||||
type = forbidden
|
type = forbidden
|
||||||
source_modules =
|
source_modules =
|
||||||
dify_graph
|
core.workflow
|
||||||
forbidden_modules =
|
forbidden_modules =
|
||||||
configs
|
configs
|
||||||
controllers
|
controllers
|
||||||
@@ -91,6 +91,7 @@ forbidden_modules =
|
|||||||
core.logging
|
core.logging
|
||||||
core.mcp
|
core.mcp
|
||||||
core.memory
|
core.memory
|
||||||
|
core.model_manager
|
||||||
core.moderation
|
core.moderation
|
||||||
core.ops
|
core.ops
|
||||||
core.plugin
|
core.plugin
|
||||||
@@ -103,70 +104,248 @@ forbidden_modules =
|
|||||||
core.trigger
|
core.trigger
|
||||||
core.variables
|
core.variables
|
||||||
ignore_imports =
|
ignore_imports =
|
||||||
dify_graph.nodes.loop.loop_node -> core.workflow.node_factory
|
core.workflow.nodes.loop.loop_node -> core.app.workflow.node_factory
|
||||||
dify_graph.nodes.agent.agent_node -> core.model_manager
|
core.workflow.graph_engine.command_channels.redis_channel -> extensions.ext_redis
|
||||||
dify_graph.nodes.agent.agent_node -> core.provider_manager
|
core.workflow.workflow_entry -> core.app.workflow.layers.observability
|
||||||
dify_graph.nodes.agent.agent_node -> core.tools.tool_manager
|
core.workflow.nodes.agent.agent_node -> core.model_manager
|
||||||
dify_graph.nodes.document_extractor.node -> core.helper.ssrf_proxy
|
core.workflow.nodes.agent.agent_node -> core.provider_manager
|
||||||
dify_graph.nodes.iteration.iteration_node -> core.workflow.node_factory
|
core.workflow.nodes.agent.agent_node -> core.tools.tool_manager
|
||||||
dify_graph.nodes.iteration.iteration_node -> core.app.workflow.layers.llm_quota
|
core.workflow.nodes.code.code_node -> core.helper.code_executor.code_executor
|
||||||
dify_graph.nodes.llm.llm_utils -> core.model_manager
|
core.workflow.nodes.datasource.datasource_node -> models.model
|
||||||
dify_graph.nodes.llm.protocols -> core.model_manager
|
core.workflow.nodes.datasource.datasource_node -> models.tools
|
||||||
dify_graph.nodes.llm.llm_utils -> dify_graph.model_runtime.model_providers.__base.large_language_model
|
core.workflow.nodes.datasource.datasource_node -> services.datasource_provider_service
|
||||||
dify_graph.nodes.llm.node -> core.tools.signature
|
core.workflow.nodes.document_extractor.node -> configs
|
||||||
dify_graph.nodes.tool.tool_node -> core.callback_handler.workflow_tool_callback_handler
|
core.workflow.nodes.document_extractor.node -> core.file.file_manager
|
||||||
dify_graph.nodes.tool.tool_node -> core.tools.tool_engine
|
core.workflow.nodes.document_extractor.node -> core.helper.ssrf_proxy
|
||||||
dify_graph.nodes.tool.tool_node -> core.tools.tool_manager
|
core.workflow.nodes.http_request.entities -> configs
|
||||||
dify_graph.nodes.agent.agent_node -> core.agent.entities
|
core.workflow.nodes.http_request.executor -> configs
|
||||||
dify_graph.nodes.agent.agent_node -> core.agent.plugin_entities
|
core.workflow.nodes.http_request.executor -> core.file.file_manager
|
||||||
dify_graph.nodes.knowledge_retrieval.knowledge_retrieval_node -> core.app.app_config.entities
|
core.workflow.nodes.http_request.node -> configs
|
||||||
dify_graph.nodes.parameter_extractor.parameter_extractor_node -> core.prompt.advanced_prompt_transform
|
core.workflow.nodes.http_request.node -> core.tools.tool_file_manager
|
||||||
dify_graph.nodes.parameter_extractor.parameter_extractor_node -> core.prompt.simple_prompt_transform
|
core.workflow.nodes.iteration.iteration_node -> core.app.workflow.node_factory
|
||||||
dify_graph.nodes.parameter_extractor.parameter_extractor_node -> dify_graph.model_runtime.model_providers.__base.large_language_model
|
core.workflow.nodes.knowledge_index.knowledge_index_node -> core.rag.index_processor.index_processor_factory
|
||||||
dify_graph.nodes.question_classifier.question_classifier_node -> core.prompt.simple_prompt_transform
|
core.workflow.nodes.llm.llm_utils -> configs
|
||||||
dify_graph.nodes.parameter_extractor.parameter_extractor_node -> core.model_manager
|
core.workflow.nodes.llm.llm_utils -> core.app.entities.app_invoke_entities
|
||||||
dify_graph.nodes.question_classifier.question_classifier_node -> core.model_manager
|
core.workflow.nodes.llm.llm_utils -> core.file.models
|
||||||
dify_graph.nodes.tool.tool_node -> core.tools.utils.message_transformer
|
core.workflow.nodes.llm.llm_utils -> core.model_manager
|
||||||
dify_graph.nodes.tool.tool_node -> models
|
core.workflow.nodes.llm.llm_utils -> core.model_runtime.model_providers.__base.large_language_model
|
||||||
dify_graph.nodes.agent.agent_node -> models.model
|
core.workflow.nodes.llm.llm_utils -> models.model
|
||||||
dify_graph.nodes.llm.file_saver -> core.helper.ssrf_proxy
|
core.workflow.nodes.llm.llm_utils -> models.provider
|
||||||
dify_graph.nodes.llm.node -> core.helper.code_executor
|
core.workflow.nodes.llm.llm_utils -> services.credit_pool_service
|
||||||
dify_graph.nodes.llm.node -> core.llm_generator.output_parser.errors
|
core.workflow.nodes.llm.node -> core.tools.signature
|
||||||
dify_graph.nodes.llm.node -> core.llm_generator.output_parser.structured_output
|
core.workflow.nodes.tool.tool_node -> core.callback_handler.workflow_tool_callback_handler
|
||||||
dify_graph.nodes.llm.node -> core.model_manager
|
core.workflow.nodes.tool.tool_node -> core.tools.tool_engine
|
||||||
dify_graph.nodes.agent.entities -> core.prompt.entities.advanced_prompt_entities
|
core.workflow.nodes.tool.tool_node -> core.tools.tool_manager
|
||||||
dify_graph.nodes.llm.entities -> core.prompt.entities.advanced_prompt_entities
|
core.workflow.workflow_entry -> configs
|
||||||
dify_graph.nodes.llm.node -> core.prompt.entities.advanced_prompt_entities
|
core.workflow.workflow_entry -> models.workflow
|
||||||
dify_graph.nodes.llm.node -> core.prompt.utils.prompt_message_util
|
core.workflow.nodes.agent.agent_node -> core.agent.entities
|
||||||
dify_graph.nodes.parameter_extractor.entities -> core.prompt.entities.advanced_prompt_entities
|
core.workflow.nodes.agent.agent_node -> core.agent.plugin_entities
|
||||||
dify_graph.nodes.parameter_extractor.parameter_extractor_node -> core.prompt.entities.advanced_prompt_entities
|
core.workflow.nodes.base.node -> core.app.entities.app_invoke_entities
|
||||||
dify_graph.nodes.parameter_extractor.parameter_extractor_node -> core.prompt.utils.prompt_message_util
|
core.workflow.nodes.human_input.human_input_node -> core.app.entities.app_invoke_entities
|
||||||
dify_graph.nodes.question_classifier.entities -> core.prompt.entities.advanced_prompt_entities
|
core.workflow.nodes.knowledge_index.knowledge_index_node -> core.app.entities.app_invoke_entities
|
||||||
dify_graph.nodes.question_classifier.question_classifier_node -> core.prompt.utils.prompt_message_util
|
core.workflow.nodes.knowledge_retrieval.knowledge_retrieval_node -> core.app.app_config.entities
|
||||||
dify_graph.nodes.knowledge_index.entities -> core.rag.retrieval.retrieval_methods
|
core.workflow.nodes.llm.node -> core.app.entities.app_invoke_entities
|
||||||
dify_graph.nodes.llm.node -> models.dataset
|
core.workflow.nodes.parameter_extractor.parameter_extractor_node -> core.app.entities.app_invoke_entities
|
||||||
dify_graph.nodes.agent.agent_node -> core.tools.utils.message_transformer
|
core.workflow.nodes.parameter_extractor.parameter_extractor_node -> core.prompt.advanced_prompt_transform
|
||||||
dify_graph.nodes.llm.file_saver -> core.tools.signature
|
core.workflow.nodes.parameter_extractor.parameter_extractor_node -> core.prompt.simple_prompt_transform
|
||||||
dify_graph.nodes.llm.file_saver -> core.tools.tool_file_manager
|
core.workflow.nodes.parameter_extractor.parameter_extractor_node -> core.model_runtime.model_providers.__base.large_language_model
|
||||||
dify_graph.nodes.tool.tool_node -> core.tools.errors
|
core.workflow.nodes.question_classifier.question_classifier_node -> core.app.entities.app_invoke_entities
|
||||||
dify_graph.nodes.agent.agent_node -> extensions.ext_database
|
core.workflow.nodes.question_classifier.question_classifier_node -> core.prompt.advanced_prompt_transform
|
||||||
dify_graph.nodes.llm.file_saver -> extensions.ext_database
|
core.workflow.nodes.question_classifier.question_classifier_node -> core.prompt.simple_prompt_transform
|
||||||
dify_graph.nodes.llm.node -> extensions.ext_database
|
core.workflow.nodes.start.entities -> core.app.app_config.entities
|
||||||
dify_graph.nodes.tool.tool_node -> extensions.ext_database
|
core.workflow.nodes.start.start_node -> core.app.app_config.entities
|
||||||
dify_graph.nodes.human_input.human_input_node -> extensions.ext_database
|
core.workflow.workflow_entry -> core.app.apps.exc
|
||||||
dify_graph.nodes.human_input.human_input_node -> core.repositories.human_input_repository
|
core.workflow.workflow_entry -> core.app.entities.app_invoke_entities
|
||||||
dify_graph.nodes.agent.agent_node -> models
|
core.workflow.workflow_entry -> core.app.workflow.node_factory
|
||||||
dify_graph.nodes.loop.loop_node -> core.app.workflow.layers.llm_quota
|
core.workflow.nodes.datasource.datasource_node -> core.datasource.datasource_manager
|
||||||
dify_graph.nodes.llm.node -> models.model
|
core.workflow.nodes.datasource.datasource_node -> core.datasource.utils.message_transformer
|
||||||
dify_graph.nodes.agent.agent_node -> services
|
core.workflow.nodes.llm.llm_utils -> core.entities.provider_entities
|
||||||
dify_graph.nodes.tool.tool_node -> services
|
core.workflow.nodes.parameter_extractor.parameter_extractor_node -> core.model_manager
|
||||||
dify_graph.model_runtime.model_providers.__base.ai_model -> configs
|
core.workflow.nodes.question_classifier.question_classifier_node -> core.model_manager
|
||||||
dify_graph.model_runtime.model_providers.__base.ai_model -> extensions.ext_redis
|
core.workflow.node_events.node -> core.file
|
||||||
dify_graph.model_runtime.model_providers.__base.large_language_model -> configs
|
core.workflow.nodes.agent.agent_node -> core.file
|
||||||
dify_graph.model_runtime.model_providers.__base.text_embedding_model -> core.entities.embedding_type
|
core.workflow.nodes.datasource.datasource_node -> core.file
|
||||||
dify_graph.model_runtime.model_providers.model_provider_factory -> configs
|
core.workflow.nodes.datasource.datasource_node -> core.file.enums
|
||||||
dify_graph.model_runtime.model_providers.model_provider_factory -> extensions.ext_redis
|
core.workflow.nodes.document_extractor.node -> core.file
|
||||||
dify_graph.model_runtime.model_providers.model_provider_factory -> models.provider_ids
|
core.workflow.nodes.http_request.executor -> core.file.enums
|
||||||
|
core.workflow.nodes.http_request.node -> core.file
|
||||||
|
core.workflow.nodes.http_request.node -> core.file.file_manager
|
||||||
|
core.workflow.nodes.knowledge_retrieval.knowledge_retrieval_node -> core.file.models
|
||||||
|
core.workflow.nodes.list_operator.node -> core.file
|
||||||
|
core.workflow.nodes.llm.file_saver -> core.file
|
||||||
|
core.workflow.nodes.llm.llm_utils -> core.variables.segments
|
||||||
|
core.workflow.nodes.llm.node -> core.file
|
||||||
|
core.workflow.nodes.llm.node -> core.file.file_manager
|
||||||
|
core.workflow.nodes.llm.node -> core.file.models
|
||||||
|
core.workflow.nodes.loop.entities -> core.variables.types
|
||||||
|
core.workflow.nodes.parameter_extractor.parameter_extractor_node -> core.file
|
||||||
|
core.workflow.nodes.protocols -> core.file
|
||||||
|
core.workflow.nodes.question_classifier.question_classifier_node -> core.file.models
|
||||||
|
core.workflow.nodes.tool.tool_node -> core.file
|
||||||
|
core.workflow.nodes.tool.tool_node -> core.tools.utils.message_transformer
|
||||||
|
core.workflow.nodes.tool.tool_node -> models
|
||||||
|
core.workflow.nodes.trigger_webhook.node -> core.file
|
||||||
|
core.workflow.runtime.variable_pool -> core.file
|
||||||
|
core.workflow.runtime.variable_pool -> core.file.file_manager
|
||||||
|
core.workflow.system_variable -> core.file.models
|
||||||
|
core.workflow.utils.condition.processor -> core.file
|
||||||
|
core.workflow.utils.condition.processor -> core.file.file_manager
|
||||||
|
core.workflow.workflow_entry -> core.file.models
|
||||||
|
core.workflow.workflow_type_encoder -> core.file.models
|
||||||
|
core.workflow.nodes.agent.agent_node -> models.model
|
||||||
|
core.workflow.nodes.code.code_node -> core.helper.code_executor.code_node_provider
|
||||||
|
core.workflow.nodes.code.code_node -> core.helper.code_executor.javascript.javascript_code_provider
|
||||||
|
core.workflow.nodes.code.code_node -> core.helper.code_executor.python3.python3_code_provider
|
||||||
|
core.workflow.nodes.code.entities -> core.helper.code_executor.code_executor
|
||||||
|
core.workflow.nodes.datasource.datasource_node -> core.variables.variables
|
||||||
|
core.workflow.nodes.http_request.executor -> core.helper.ssrf_proxy
|
||||||
|
core.workflow.nodes.http_request.node -> core.helper.ssrf_proxy
|
||||||
|
core.workflow.nodes.llm.file_saver -> core.helper.ssrf_proxy
|
||||||
|
core.workflow.nodes.llm.node -> core.helper.code_executor
|
||||||
|
core.workflow.nodes.template_transform.template_renderer -> core.helper.code_executor.code_executor
|
||||||
|
core.workflow.nodes.llm.node -> core.llm_generator.output_parser.errors
|
||||||
|
core.workflow.nodes.llm.node -> core.llm_generator.output_parser.structured_output
|
||||||
|
core.workflow.nodes.llm.node -> core.model_manager
|
||||||
|
core.workflow.nodes.agent.entities -> core.prompt.entities.advanced_prompt_entities
|
||||||
|
core.workflow.nodes.llm.entities -> core.prompt.entities.advanced_prompt_entities
|
||||||
|
core.workflow.nodes.llm.llm_utils -> core.prompt.entities.advanced_prompt_entities
|
||||||
|
core.workflow.nodes.llm.node -> core.prompt.entities.advanced_prompt_entities
|
||||||
|
core.workflow.nodes.llm.node -> core.prompt.utils.prompt_message_util
|
||||||
|
core.workflow.nodes.parameter_extractor.entities -> core.prompt.entities.advanced_prompt_entities
|
||||||
|
core.workflow.nodes.parameter_extractor.parameter_extractor_node -> core.prompt.entities.advanced_prompt_entities
|
||||||
|
core.workflow.nodes.parameter_extractor.parameter_extractor_node -> core.prompt.utils.prompt_message_util
|
||||||
|
core.workflow.nodes.question_classifier.entities -> core.prompt.entities.advanced_prompt_entities
|
||||||
|
core.workflow.nodes.question_classifier.question_classifier_node -> core.prompt.utils.prompt_message_util
|
||||||
|
core.workflow.nodes.knowledge_index.entities -> core.rag.retrieval.retrieval_methods
|
||||||
|
core.workflow.nodes.knowledge_index.knowledge_index_node -> core.rag.retrieval.retrieval_methods
|
||||||
|
core.workflow.nodes.knowledge_index.knowledge_index_node -> models.dataset
|
||||||
|
core.workflow.nodes.knowledge_index.knowledge_index_node -> services.summary_index_service
|
||||||
|
core.workflow.nodes.knowledge_index.knowledge_index_node -> tasks.generate_summary_index_task
|
||||||
|
core.workflow.nodes.knowledge_index.knowledge_index_node -> core.rag.index_processor.processor.paragraph_index_processor
|
||||||
|
core.workflow.nodes.llm.node -> models.dataset
|
||||||
|
core.workflow.nodes.agent.agent_node -> core.tools.utils.message_transformer
|
||||||
|
core.workflow.nodes.llm.file_saver -> core.tools.signature
|
||||||
|
core.workflow.nodes.llm.file_saver -> core.tools.tool_file_manager
|
||||||
|
core.workflow.nodes.tool.tool_node -> core.tools.errors
|
||||||
|
core.workflow.conversation_variable_updater -> core.variables
|
||||||
|
core.workflow.graph_engine.entities.commands -> core.variables.variables
|
||||||
|
core.workflow.nodes.agent.agent_node -> core.variables.segments
|
||||||
|
core.workflow.nodes.answer.answer_node -> core.variables
|
||||||
|
core.workflow.nodes.code.code_node -> core.variables.segments
|
||||||
|
core.workflow.nodes.code.code_node -> core.variables.types
|
||||||
|
core.workflow.nodes.code.entities -> core.variables.types
|
||||||
|
core.workflow.nodes.datasource.datasource_node -> core.variables.segments
|
||||||
|
core.workflow.nodes.document_extractor.node -> core.variables
|
||||||
|
core.workflow.nodes.document_extractor.node -> core.variables.segments
|
||||||
|
core.workflow.nodes.http_request.executor -> core.variables.segments
|
||||||
|
core.workflow.nodes.http_request.node -> core.variables.segments
|
||||||
|
core.workflow.nodes.human_input.entities -> core.variables.consts
|
||||||
|
core.workflow.nodes.iteration.iteration_node -> core.variables
|
||||||
|
core.workflow.nodes.iteration.iteration_node -> core.variables.segments
|
||||||
|
core.workflow.nodes.iteration.iteration_node -> core.variables.variables
|
||||||
|
core.workflow.nodes.knowledge_retrieval.knowledge_retrieval_node -> core.variables
|
||||||
|
core.workflow.nodes.knowledge_retrieval.knowledge_retrieval_node -> core.variables.segments
|
||||||
|
core.workflow.nodes.list_operator.node -> core.variables
|
||||||
|
core.workflow.nodes.list_operator.node -> core.variables.segments
|
||||||
|
core.workflow.nodes.llm.node -> core.variables
|
||||||
|
core.workflow.nodes.loop.loop_node -> core.variables
|
||||||
|
core.workflow.nodes.parameter_extractor.entities -> core.variables.types
|
||||||
|
core.workflow.nodes.parameter_extractor.exc -> core.variables.types
|
||||||
|
core.workflow.nodes.parameter_extractor.parameter_extractor_node -> core.variables.types
|
||||||
|
core.workflow.nodes.tool.tool_node -> core.variables.segments
|
||||||
|
core.workflow.nodes.tool.tool_node -> core.variables.variables
|
||||||
|
core.workflow.nodes.trigger_webhook.node -> core.variables.types
|
||||||
|
core.workflow.nodes.trigger_webhook.node -> core.variables.variables
|
||||||
|
core.workflow.nodes.variable_aggregator.entities -> core.variables.types
|
||||||
|
core.workflow.nodes.variable_aggregator.variable_aggregator_node -> core.variables.segments
|
||||||
|
core.workflow.nodes.variable_assigner.common.helpers -> core.variables
|
||||||
|
core.workflow.nodes.variable_assigner.common.helpers -> core.variables.consts
|
||||||
|
core.workflow.nodes.variable_assigner.common.helpers -> core.variables.types
|
||||||
|
core.workflow.nodes.variable_assigner.v1.node -> core.variables
|
||||||
|
core.workflow.nodes.variable_assigner.v2.helpers -> core.variables
|
||||||
|
core.workflow.nodes.variable_assigner.v2.node -> core.variables
|
||||||
|
core.workflow.nodes.variable_assigner.v2.node -> core.variables.consts
|
||||||
|
core.workflow.runtime.graph_runtime_state_protocol -> core.variables.segments
|
||||||
|
core.workflow.runtime.read_only_wrappers -> core.variables.segments
|
||||||
|
core.workflow.runtime.variable_pool -> core.variables
|
||||||
|
core.workflow.runtime.variable_pool -> core.variables.consts
|
||||||
|
core.workflow.runtime.variable_pool -> core.variables.segments
|
||||||
|
core.workflow.runtime.variable_pool -> core.variables.variables
|
||||||
|
core.workflow.utils.condition.processor -> core.variables
|
||||||
|
core.workflow.utils.condition.processor -> core.variables.segments
|
||||||
|
core.workflow.variable_loader -> core.variables
|
||||||
|
core.workflow.variable_loader -> core.variables.consts
|
||||||
|
core.workflow.workflow_type_encoder -> core.variables
|
||||||
|
core.workflow.graph_engine.manager -> extensions.ext_redis
|
||||||
|
core.workflow.nodes.agent.agent_node -> extensions.ext_database
|
||||||
|
core.workflow.nodes.datasource.datasource_node -> extensions.ext_database
|
||||||
|
core.workflow.nodes.knowledge_index.knowledge_index_node -> extensions.ext_database
|
||||||
|
core.workflow.nodes.llm.file_saver -> extensions.ext_database
|
||||||
|
core.workflow.nodes.llm.llm_utils -> extensions.ext_database
|
||||||
|
core.workflow.nodes.llm.node -> extensions.ext_database
|
||||||
|
core.workflow.nodes.tool.tool_node -> extensions.ext_database
|
||||||
|
core.workflow.nodes.human_input.human_input_node -> extensions.ext_database
|
||||||
|
core.workflow.nodes.human_input.human_input_node -> core.repositories.human_input_repository
|
||||||
|
core.workflow.workflow_entry -> extensions.otel.runtime
|
||||||
|
core.workflow.nodes.agent.agent_node -> models
|
||||||
|
core.workflow.nodes.base.node -> models.enums
|
||||||
|
core.workflow.nodes.llm.llm_utils -> models.provider_ids
|
||||||
|
core.workflow.nodes.llm.node -> models.model
|
||||||
|
core.workflow.workflow_entry -> models.enums
|
||||||
|
core.workflow.nodes.agent.agent_node -> services
|
||||||
|
core.workflow.nodes.tool.tool_node -> services
|
||||||
|
|
||||||
|
[importlinter:contract:model-runtime-no-internal-imports]
|
||||||
|
name = Model Runtime Internal Imports
|
||||||
|
type = forbidden
|
||||||
|
source_modules =
|
||||||
|
core.model_runtime
|
||||||
|
forbidden_modules =
|
||||||
|
configs
|
||||||
|
controllers
|
||||||
|
extensions
|
||||||
|
models
|
||||||
|
services
|
||||||
|
tasks
|
||||||
|
core.agent
|
||||||
|
core.app
|
||||||
|
core.base
|
||||||
|
core.callback_handler
|
||||||
|
core.datasource
|
||||||
|
core.db
|
||||||
|
core.entities
|
||||||
|
core.errors
|
||||||
|
core.extension
|
||||||
|
core.external_data_tool
|
||||||
|
core.file
|
||||||
|
core.helper
|
||||||
|
core.hosting_configuration
|
||||||
|
core.indexing_runner
|
||||||
|
core.llm_generator
|
||||||
|
core.logging
|
||||||
|
core.mcp
|
||||||
|
core.memory
|
||||||
|
core.model_manager
|
||||||
|
core.moderation
|
||||||
|
core.ops
|
||||||
|
core.plugin
|
||||||
|
core.prompt
|
||||||
|
core.provider_manager
|
||||||
|
core.rag
|
||||||
|
core.repositories
|
||||||
|
core.schemas
|
||||||
|
core.tools
|
||||||
|
core.trigger
|
||||||
|
core.variables
|
||||||
|
core.workflow
|
||||||
|
ignore_imports =
|
||||||
|
core.model_runtime.model_providers.__base.ai_model -> configs
|
||||||
|
core.model_runtime.model_providers.__base.ai_model -> extensions.ext_redis
|
||||||
|
core.model_runtime.model_providers.__base.large_language_model -> configs
|
||||||
|
core.model_runtime.model_providers.__base.text_embedding_model -> core.entities.embedding_type
|
||||||
|
core.model_runtime.model_providers.model_provider_factory -> configs
|
||||||
|
core.model_runtime.model_providers.model_provider_factory -> extensions.ext_redis
|
||||||
|
core.model_runtime.model_providers.model_provider_factory -> models.provider_ids
|
||||||
|
|
||||||
[importlinter:contract:rsc]
|
[importlinter:contract:rsc]
|
||||||
name = RSC
|
name = RSC
|
||||||
@@ -175,7 +354,7 @@ layers =
|
|||||||
graph_engine
|
graph_engine
|
||||||
response_coordinator
|
response_coordinator
|
||||||
containers =
|
containers =
|
||||||
dify_graph.graph_engine
|
core.workflow.graph_engine
|
||||||
|
|
||||||
[importlinter:contract:worker]
|
[importlinter:contract:worker]
|
||||||
name = Worker
|
name = Worker
|
||||||
@@ -184,7 +363,7 @@ layers =
|
|||||||
graph_engine
|
graph_engine
|
||||||
worker
|
worker
|
||||||
containers =
|
containers =
|
||||||
dify_graph.graph_engine
|
core.workflow.graph_engine
|
||||||
|
|
||||||
[importlinter:contract:graph-engine-architecture]
|
[importlinter:contract:graph-engine-architecture]
|
||||||
name = Graph Engine Architecture
|
name = Graph Engine Architecture
|
||||||
@@ -200,28 +379,28 @@ layers =
|
|||||||
worker_management
|
worker_management
|
||||||
domain
|
domain
|
||||||
containers =
|
containers =
|
||||||
dify_graph.graph_engine
|
core.workflow.graph_engine
|
||||||
|
|
||||||
[importlinter:contract:domain-isolation]
|
[importlinter:contract:domain-isolation]
|
||||||
name = Domain Model Isolation
|
name = Domain Model Isolation
|
||||||
type = forbidden
|
type = forbidden
|
||||||
source_modules =
|
source_modules =
|
||||||
dify_graph.graph_engine.domain
|
core.workflow.graph_engine.domain
|
||||||
forbidden_modules =
|
forbidden_modules =
|
||||||
dify_graph.graph_engine.worker_management
|
core.workflow.graph_engine.worker_management
|
||||||
dify_graph.graph_engine.command_channels
|
core.workflow.graph_engine.command_channels
|
||||||
dify_graph.graph_engine.layers
|
core.workflow.graph_engine.layers
|
||||||
dify_graph.graph_engine.protocols
|
core.workflow.graph_engine.protocols
|
||||||
|
|
||||||
[importlinter:contract:worker-management]
|
[importlinter:contract:worker-management]
|
||||||
name = Worker Management
|
name = Worker Management
|
||||||
type = forbidden
|
type = forbidden
|
||||||
source_modules =
|
source_modules =
|
||||||
dify_graph.graph_engine.worker_management
|
core.workflow.graph_engine.worker_management
|
||||||
forbidden_modules =
|
forbidden_modules =
|
||||||
dify_graph.graph_engine.orchestration
|
core.workflow.graph_engine.orchestration
|
||||||
dify_graph.graph_engine.command_processing
|
core.workflow.graph_engine.command_processing
|
||||||
dify_graph.graph_engine.event_management
|
core.workflow.graph_engine.event_management
|
||||||
|
|
||||||
|
|
||||||
[importlinter:contract:graph-traversal-components]
|
[importlinter:contract:graph-traversal-components]
|
||||||
@@ -231,11 +410,11 @@ layers =
|
|||||||
edge_processor
|
edge_processor
|
||||||
skip_propagator
|
skip_propagator
|
||||||
containers =
|
containers =
|
||||||
dify_graph.graph_engine.graph_traversal
|
core.workflow.graph_engine.graph_traversal
|
||||||
|
|
||||||
[importlinter:contract:command-channels]
|
[importlinter:contract:command-channels]
|
||||||
name = Command Channels Independence
|
name = Command Channels Independence
|
||||||
type = independence
|
type = independence
|
||||||
modules =
|
modules =
|
||||||
dify_graph.graph_engine.command_channels.in_memory_channel
|
core.workflow.graph_engine.command_channels.in_memory_channel
|
||||||
dify_graph.graph_engine.command_channels.redis_channel
|
core.workflow.graph_engine.command_channels.redis_channel
|
||||||
|
|||||||
+1
-1
@@ -100,7 +100,7 @@ ignore = [
|
|||||||
"configs/*" = [
|
"configs/*" = [
|
||||||
"N802", # invalid-function-name
|
"N802", # invalid-function-name
|
||||||
]
|
]
|
||||||
"dify_graph/model_runtime/callbacks/base_callback.py" = ["T201"]
|
"core/model_runtime/callbacks/base_callback.py" = ["T201"]
|
||||||
"core/workflow/callbacks/workflow_logging_callback.py" = ["T201"]
|
"core/workflow/callbacks/workflow_logging_callback.py" = ["T201"]
|
||||||
"libs/gmpy2_pkcs10aep_cipher.py" = [
|
"libs/gmpy2_pkcs10aep_cipher.py" = [
|
||||||
"N803", # invalid-argument-name
|
"N803", # invalid-argument-name
|
||||||
|
|||||||
+81
-1
@@ -42,7 +42,7 @@ The scripts resolve paths relative to their location, so you can run them from a
|
|||||||
|
|
||||||
1. Set up your application by visiting `http://localhost:3000`.
|
1. Set up your application by visiting `http://localhost:3000`.
|
||||||
|
|
||||||
1. Start the worker service (async and scheduler tasks, runs from `api`).
|
1. Optional: start the worker service (async tasks, runs from `api`).
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
./dev/start-worker
|
./dev/start-worker
|
||||||
@@ -54,6 +54,86 @@ The scripts resolve paths relative to their location, so you can run them from a
|
|||||||
./dev/start-beat
|
./dev/start-beat
|
||||||
```
|
```
|
||||||
|
|
||||||
|
### Manual commands
|
||||||
|
|
||||||
|
<details>
|
||||||
|
<summary>Show manual setup and run steps</summary>
|
||||||
|
|
||||||
|
These commands assume you start from the repository root.
|
||||||
|
|
||||||
|
1. Start the docker-compose stack.
|
||||||
|
|
||||||
|
The backend requires middleware, including PostgreSQL, Redis, and Weaviate, which can be started together using `docker-compose`.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cp docker/middleware.env.example docker/middleware.env
|
||||||
|
# Use mysql or another vector database profile if you are not using postgres/weaviate.
|
||||||
|
docker compose -f docker/docker-compose.middleware.yaml --profile postgresql --profile weaviate -p dify up -d
|
||||||
|
```
|
||||||
|
|
||||||
|
1. Copy env files.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cp api/.env.example api/.env
|
||||||
|
cp web/.env.example web/.env.local
|
||||||
|
```
|
||||||
|
|
||||||
|
1. Install UV if needed.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
pip install uv
|
||||||
|
# Or on macOS
|
||||||
|
brew install uv
|
||||||
|
```
|
||||||
|
|
||||||
|
1. Install API dependencies.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd api
|
||||||
|
uv sync --group dev
|
||||||
|
```
|
||||||
|
|
||||||
|
1. Install web dependencies.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd web
|
||||||
|
pnpm install
|
||||||
|
cd ..
|
||||||
|
```
|
||||||
|
|
||||||
|
1. Start backend (runs migrations first, in a new terminal).
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd api
|
||||||
|
uv run flask db upgrade
|
||||||
|
uv run flask run --host 0.0.0.0 --port=5001 --debug
|
||||||
|
```
|
||||||
|
|
||||||
|
1. Start Dify [web](../web) service (in a new terminal).
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd web
|
||||||
|
pnpm dev:inspect
|
||||||
|
```
|
||||||
|
|
||||||
|
1. Set up your application by visiting `http://localhost:3000`.
|
||||||
|
|
||||||
|
1. Optional: start the worker service (async tasks, in a new terminal).
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd api
|
||||||
|
uv run celery -A app.celery worker -P threads -c 2 --loglevel INFO -Q api_token,dataset,priority_dataset,priority_pipeline,pipeline,mail,ops_trace,app_deletion,plugin,workflow_storage,conversation,workflow,schedule_poller,schedule_executor,triggered_workflow_dispatcher,trigger_refresh_executor,retention
|
||||||
|
```
|
||||||
|
|
||||||
|
1. Optional: start Celery Beat (scheduled tasks, in a new terminal).
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd api
|
||||||
|
uv run celery -A app.celery beat
|
||||||
|
```
|
||||||
|
|
||||||
|
</details>
|
||||||
|
|
||||||
### Environment notes
|
### Environment notes
|
||||||
|
|
||||||
> [!IMPORTANT]
|
> [!IMPORTANT]
|
||||||
|
|||||||
+2
-14
@@ -30,7 +30,6 @@ from extensions.ext_redis import redis_client
|
|||||||
from extensions.ext_storage import storage
|
from extensions.ext_storage import storage
|
||||||
from extensions.storage.opendal_storage import OpenDALStorage
|
from extensions.storage.opendal_storage import OpenDALStorage
|
||||||
from extensions.storage.storage_type import StorageType
|
from extensions.storage.storage_type import StorageType
|
||||||
from libs.db_migration_lock import DbMigrationAutoRenewLock
|
|
||||||
from libs.helper import email as email_validate
|
from libs.helper import email as email_validate
|
||||||
from libs.password import hash_password, password_pattern, valid_password
|
from libs.password import hash_password, password_pattern, valid_password
|
||||||
from libs.rsa import generate_key_pair
|
from libs.rsa import generate_key_pair
|
||||||
@@ -55,8 +54,6 @@ from tasks.remove_app_and_related_data_task import delete_draft_variables_batch
|
|||||||
|
|
||||||
logger = logging.getLogger(__name__)
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
DB_UPGRADE_LOCK_TTL_SECONDS = 60
|
|
||||||
|
|
||||||
|
|
||||||
@click.command("reset-password", help="Reset the account password.")
|
@click.command("reset-password", help="Reset the account password.")
|
||||||
@click.option("--email", prompt=True, help="Account email to reset password for")
|
@click.option("--email", prompt=True, help="Account email to reset password for")
|
||||||
@@ -730,15 +727,8 @@ def create_tenant(email: str, language: str | None = None, name: str | None = No
|
|||||||
@click.command("upgrade-db", help="Upgrade the database")
|
@click.command("upgrade-db", help="Upgrade the database")
|
||||||
def upgrade_db():
|
def upgrade_db():
|
||||||
click.echo("Preparing database migration...")
|
click.echo("Preparing database migration...")
|
||||||
lock = DbMigrationAutoRenewLock(
|
lock = redis_client.lock(name="db_upgrade_lock", timeout=60)
|
||||||
redis_client=redis_client,
|
|
||||||
name="db_upgrade_lock",
|
|
||||||
ttl_seconds=DB_UPGRADE_LOCK_TTL_SECONDS,
|
|
||||||
logger=logger,
|
|
||||||
log_context="db_migration",
|
|
||||||
)
|
|
||||||
if lock.acquire(blocking=False):
|
if lock.acquire(blocking=False):
|
||||||
migration_succeeded = False
|
|
||||||
try:
|
try:
|
||||||
click.echo(click.style("Starting database migration.", fg="green"))
|
click.echo(click.style("Starting database migration.", fg="green"))
|
||||||
|
|
||||||
@@ -747,7 +737,6 @@ def upgrade_db():
|
|||||||
|
|
||||||
flask_migrate.upgrade()
|
flask_migrate.upgrade()
|
||||||
|
|
||||||
migration_succeeded = True
|
|
||||||
click.echo(click.style("Database migration successful!", fg="green"))
|
click.echo(click.style("Database migration successful!", fg="green"))
|
||||||
|
|
||||||
except Exception as e:
|
except Exception as e:
|
||||||
@@ -755,8 +744,7 @@ def upgrade_db():
|
|||||||
click.echo(click.style(f"Database migration failed: {e}", fg="red"))
|
click.echo(click.style(f"Database migration failed: {e}", fg="red"))
|
||||||
raise SystemExit(1)
|
raise SystemExit(1)
|
||||||
finally:
|
finally:
|
||||||
status = "successful" if migration_succeeded else "failed"
|
lock.release()
|
||||||
lock.release_safely(status=status)
|
|
||||||
else:
|
else:
|
||||||
click.echo("Database migration skipped")
|
click.echo("Database migration skipped")
|
||||||
|
|
||||||
|
|||||||
@@ -265,11 +265,6 @@ class PluginConfig(BaseSettings):
|
|||||||
default=60 * 60,
|
default=60 * 60,
|
||||||
)
|
)
|
||||||
|
|
||||||
PLUGIN_MAX_FILE_SIZE: PositiveInt = Field(
|
|
||||||
description="Maximum allowed size (bytes) for plugin-generated files",
|
|
||||||
default=50 * 1024 * 1024,
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
class MarketplaceConfig(BaseSettings):
|
class MarketplaceConfig(BaseSettings):
|
||||||
"""
|
"""
|
||||||
@@ -1319,9 +1314,6 @@ class WorkflowLogConfig(BaseSettings):
|
|||||||
WORKFLOW_LOG_CLEANUP_BATCH_SIZE: int = Field(
|
WORKFLOW_LOG_CLEANUP_BATCH_SIZE: int = Field(
|
||||||
default=100, description="Batch size for workflow run log cleanup operations"
|
default=100, description="Batch size for workflow run log cleanup operations"
|
||||||
)
|
)
|
||||||
WORKFLOW_LOG_CLEANUP_SPECIFIC_WORKFLOW_IDS: str = Field(
|
|
||||||
default="", description="Comma-separated list of workflow IDs to clean logs for"
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
class SwaggerUIConfig(BaseSettings):
|
class SwaggerUIConfig(BaseSettings):
|
||||||
|
|||||||
@@ -1,5 +1,3 @@
|
|||||||
from typing import Literal
|
|
||||||
|
|
||||||
from pydantic import Field, PositiveInt
|
from pydantic import Field, PositiveInt
|
||||||
from pydantic_settings import BaseSettings
|
from pydantic_settings import BaseSettings
|
||||||
|
|
||||||
@@ -51,43 +49,3 @@ class OceanBaseVectorConfig(BaseSettings):
|
|||||||
),
|
),
|
||||||
default="ik",
|
default="ik",
|
||||||
)
|
)
|
||||||
|
|
||||||
OCEANBASE_VECTOR_BATCH_SIZE: PositiveInt = Field(
|
|
||||||
description="Number of documents to insert per batch",
|
|
||||||
default=100,
|
|
||||||
)
|
|
||||||
|
|
||||||
OCEANBASE_VECTOR_METRIC_TYPE: Literal["l2", "cosine", "inner_product"] = Field(
|
|
||||||
description="Distance metric type for vector index: l2, cosine, or inner_product",
|
|
||||||
default="l2",
|
|
||||||
)
|
|
||||||
|
|
||||||
OCEANBASE_HNSW_M: PositiveInt = Field(
|
|
||||||
description="HNSW M parameter (max number of connections per node)",
|
|
||||||
default=16,
|
|
||||||
)
|
|
||||||
|
|
||||||
OCEANBASE_HNSW_EF_CONSTRUCTION: PositiveInt = Field(
|
|
||||||
description="HNSW efConstruction parameter (index build-time search width)",
|
|
||||||
default=256,
|
|
||||||
)
|
|
||||||
|
|
||||||
OCEANBASE_HNSW_EF_SEARCH: int = Field(
|
|
||||||
description="HNSW efSearch parameter (query-time search width, -1 uses server default)",
|
|
||||||
default=-1,
|
|
||||||
)
|
|
||||||
|
|
||||||
OCEANBASE_VECTOR_POOL_SIZE: PositiveInt = Field(
|
|
||||||
description="SQLAlchemy connection pool size",
|
|
||||||
default=5,
|
|
||||||
)
|
|
||||||
|
|
||||||
OCEANBASE_VECTOR_MAX_OVERFLOW: int = Field(
|
|
||||||
description="SQLAlchemy connection pool max overflow connections",
|
|
||||||
default=10,
|
|
||||||
)
|
|
||||||
|
|
||||||
OCEANBASE_HNSW_REFRESH_THRESHOLD: int = Field(
|
|
||||||
description="Minimum number of inserted documents to trigger an automatic HNSW index refresh (0 to disable)",
|
|
||||||
default=1000,
|
|
||||||
)
|
|
||||||
|
|||||||
File diff suppressed because one or more lines are too long
@@ -12,7 +12,7 @@ or any other web framework.
|
|||||||
import contextvars
|
import contextvars
|
||||||
from collections.abc import Callable
|
from collections.abc import Callable
|
||||||
|
|
||||||
from dify_graph.context.execution_context import (
|
from core.workflow.context.execution_context import (
|
||||||
ExecutionContext,
|
ExecutionContext,
|
||||||
IExecutionContext,
|
IExecutionContext,
|
||||||
NullAppContext,
|
NullAppContext,
|
||||||
|
|||||||
@@ -10,8 +10,8 @@ from typing import Any, final
|
|||||||
|
|
||||||
from flask import Flask, current_app, g
|
from flask import Flask, current_app, g
|
||||||
|
|
||||||
from dify_graph.context import register_context_capturer
|
from core.workflow.context import register_context_capturer
|
||||||
from dify_graph.context.execution_context import (
|
from core.workflow.context.execution_context import (
|
||||||
AppContext,
|
AppContext,
|
||||||
IExecutionContext,
|
IExecutionContext,
|
||||||
)
|
)
|
||||||
|
|||||||
@@ -4,7 +4,7 @@ from typing import Any, TypeAlias
|
|||||||
|
|
||||||
from pydantic import BaseModel, ConfigDict, computed_field
|
from pydantic import BaseModel, ConfigDict, computed_field
|
||||||
|
|
||||||
from dify_graph.file import helpers as file_helpers
|
from core.file import helpers as file_helpers
|
||||||
from models.model import IconType
|
from models.model import IconType
|
||||||
|
|
||||||
JSONValue: TypeAlias = str | int | float | bool | None | dict[str, Any] | list[Any]
|
JSONValue: TypeAlias = str | int | float | bool | None | dict[str, Any] | list[Any]
|
||||||
|
|||||||
@@ -23,10 +23,10 @@ from controllers.console.wraps import (
|
|||||||
is_admin_or_owner_required,
|
is_admin_or_owner_required,
|
||||||
setup_required,
|
setup_required,
|
||||||
)
|
)
|
||||||
|
from core.file import helpers as file_helpers
|
||||||
from core.ops.ops_trace_manager import OpsTraceManager
|
from core.ops.ops_trace_manager import OpsTraceManager
|
||||||
from core.rag.retrieval.retrieval_methods import RetrievalMethod
|
from core.rag.retrieval.retrieval_methods import RetrievalMethod
|
||||||
from dify_graph.enums import NodeType, WorkflowExecutionStatus
|
from core.workflow.enums import NodeType, WorkflowExecutionStatus
|
||||||
from dify_graph.file import helpers as file_helpers
|
|
||||||
from extensions.ext_database import db
|
from extensions.ext_database import db
|
||||||
from libs.login import current_account_with_tenant, login_required
|
from libs.login import current_account_with_tenant, login_required
|
||||||
from models import App, DatasetPermissionEnum, Workflow
|
from models import App, DatasetPermissionEnum, Workflow
|
||||||
@@ -660,19 +660,6 @@ class AppCopyApi(Resource):
|
|||||||
)
|
)
|
||||||
session.commit()
|
session.commit()
|
||||||
|
|
||||||
# Inherit web app permission from original app
|
|
||||||
if result.app_id and FeatureService.get_system_features().webapp_auth.enabled:
|
|
||||||
try:
|
|
||||||
# Get the original app's access mode
|
|
||||||
original_settings = EnterpriseService.WebAppAuth.get_app_access_mode_by_id(app_model.id)
|
|
||||||
access_mode = original_settings.access_mode
|
|
||||||
except Exception:
|
|
||||||
# If original app has no settings (old app), default to public to match fallback behavior
|
|
||||||
access_mode = "public"
|
|
||||||
|
|
||||||
# Apply the same access mode to the copied app
|
|
||||||
EnterpriseService.WebAppAuth.update_app_access_mode(result.app_id, access_mode)
|
|
||||||
|
|
||||||
stmt = select(App).where(App.id == result.app_id)
|
stmt = select(App).where(App.id == result.app_id)
|
||||||
app = session.scalar(stmt)
|
app = session.scalar(stmt)
|
||||||
|
|
||||||
|
|||||||
@@ -22,7 +22,7 @@ from controllers.console.app.error import (
|
|||||||
from controllers.console.app.wraps import get_app_model
|
from controllers.console.app.wraps import get_app_model
|
||||||
from controllers.console.wraps import account_initialization_required, setup_required
|
from controllers.console.wraps import account_initialization_required, setup_required
|
||||||
from core.errors.error import ModelCurrentlyNotSupportError, ProviderTokenNotInitError, QuotaExceededError
|
from core.errors.error import ModelCurrentlyNotSupportError, ProviderTokenNotInitError, QuotaExceededError
|
||||||
from dify_graph.model_runtime.errors.invoke import InvokeError
|
from core.model_runtime.errors.invoke import InvokeError
|
||||||
from libs.login import login_required
|
from libs.login import login_required
|
||||||
from models import App, AppMode
|
from models import App, AppMode
|
||||||
from services.audio_service import AudioService
|
from services.audio_service import AudioService
|
||||||
|
|||||||
@@ -26,7 +26,7 @@ from core.errors.error import (
|
|||||||
QuotaExceededError,
|
QuotaExceededError,
|
||||||
)
|
)
|
||||||
from core.helper.trace_id_helper import get_external_trace_id
|
from core.helper.trace_id_helper import get_external_trace_id
|
||||||
from dify_graph.model_runtime.errors.invoke import InvokeError
|
from core.model_runtime.errors.invoke import InvokeError
|
||||||
from libs import helper
|
from libs import helper
|
||||||
from libs.helper import uuid_value
|
from libs.helper import uuid_value
|
||||||
from libs.login import current_user, login_required
|
from libs.login import current_user, login_required
|
||||||
|
|||||||
@@ -18,7 +18,7 @@ from core.helper.code_executor.javascript.javascript_code_provider import Javasc
|
|||||||
from core.helper.code_executor.python3.python3_code_provider import Python3CodeProvider
|
from core.helper.code_executor.python3.python3_code_provider import Python3CodeProvider
|
||||||
from core.llm_generator.entities import RuleCodeGeneratePayload, RuleGeneratePayload, RuleStructuredOutputPayload
|
from core.llm_generator.entities import RuleCodeGeneratePayload, RuleGeneratePayload, RuleStructuredOutputPayload
|
||||||
from core.llm_generator.llm_generator import LLMGenerator
|
from core.llm_generator.llm_generator import LLMGenerator
|
||||||
from dify_graph.model_runtime.errors.invoke import InvokeError
|
from core.model_runtime.errors.invoke import InvokeError
|
||||||
from extensions.ext_database import db
|
from extensions.ext_database import db
|
||||||
from libs.login import current_account_with_tenant, login_required
|
from libs.login import current_account_with_tenant, login_required
|
||||||
from models import App
|
from models import App
|
||||||
|
|||||||
@@ -24,7 +24,7 @@ from controllers.console.wraps import (
|
|||||||
)
|
)
|
||||||
from core.app.entities.app_invoke_entities import InvokeFrom
|
from core.app.entities.app_invoke_entities import InvokeFrom
|
||||||
from core.errors.error import ModelCurrentlyNotSupportError, ProviderTokenNotInitError, QuotaExceededError
|
from core.errors.error import ModelCurrentlyNotSupportError, ProviderTokenNotInitError, QuotaExceededError
|
||||||
from dify_graph.model_runtime.errors.invoke import InvokeError
|
from core.model_runtime.errors.invoke import InvokeError
|
||||||
from extensions.ext_database import db
|
from extensions.ext_database import db
|
||||||
from fields.raws import FilesContainedField
|
from fields.raws import FilesContainedField
|
||||||
from libs.helper import TimestampField, uuid_value
|
from libs.helper import TimestampField, uuid_value
|
||||||
|
|||||||
@@ -20,7 +20,9 @@ from core.app.app_config.features.file_upload.manager import FileUploadConfigMan
|
|||||||
from core.app.apps.base_app_queue_manager import AppQueueManager
|
from core.app.apps.base_app_queue_manager import AppQueueManager
|
||||||
from core.app.apps.workflow.app_generator import SKIP_PREPARE_USER_INPUTS_KEY
|
from core.app.apps.workflow.app_generator import SKIP_PREPARE_USER_INPUTS_KEY
|
||||||
from core.app.entities.app_invoke_entities import InvokeFrom
|
from core.app.entities.app_invoke_entities import InvokeFrom
|
||||||
|
from core.file.models import File
|
||||||
from core.helper.trace_id_helper import get_external_trace_id
|
from core.helper.trace_id_helper import get_external_trace_id
|
||||||
|
from core.model_runtime.utils.encoders import jsonable_encoder
|
||||||
from core.plugin.impl.exc import PluginInvokeError
|
from core.plugin.impl.exc import PluginInvokeError
|
||||||
from core.trigger.debug.event_selectors import (
|
from core.trigger.debug.event_selectors import (
|
||||||
TriggerDebugEvent,
|
TriggerDebugEvent,
|
||||||
@@ -28,12 +30,9 @@ from core.trigger.debug.event_selectors import (
|
|||||||
create_event_poller,
|
create_event_poller,
|
||||||
select_trigger_debug_events,
|
select_trigger_debug_events,
|
||||||
)
|
)
|
||||||
from dify_graph.enums import NodeType
|
from core.workflow.enums import NodeType
|
||||||
from dify_graph.file.models import File
|
from core.workflow.graph_engine.manager import GraphEngineManager
|
||||||
from dify_graph.graph_engine.manager import GraphEngineManager
|
|
||||||
from dify_graph.model_runtime.utils.encoders import jsonable_encoder
|
|
||||||
from extensions.ext_database import db
|
from extensions.ext_database import db
|
||||||
from extensions.ext_redis import redis_client
|
|
||||||
from factories import file_factory, variable_factory
|
from factories import file_factory, variable_factory
|
||||||
from fields.member_fields import simple_account_fields
|
from fields.member_fields import simple_account_fields
|
||||||
from fields.workflow_fields import workflow_fields, workflow_pagination_fields
|
from fields.workflow_fields import workflow_fields, workflow_pagination_fields
|
||||||
@@ -741,7 +740,7 @@ class WorkflowTaskStopApi(Resource):
|
|||||||
AppQueueManager.set_stop_flag_no_user_check(task_id)
|
AppQueueManager.set_stop_flag_no_user_check(task_id)
|
||||||
|
|
||||||
# New graph engine command channel mechanism
|
# New graph engine command channel mechanism
|
||||||
GraphEngineManager(redis_client).send_stop_command(task_id)
|
GraphEngineManager.send_stop_command(task_id)
|
||||||
|
|
||||||
return {"result": "success"}
|
return {"result": "success"}
|
||||||
|
|
||||||
|
|||||||
@@ -9,7 +9,7 @@ from sqlalchemy.orm import Session
|
|||||||
from controllers.console import console_ns
|
from controllers.console import console_ns
|
||||||
from controllers.console.app.wraps import get_app_model
|
from controllers.console.app.wraps import get_app_model
|
||||||
from controllers.console.wraps import account_initialization_required, setup_required
|
from controllers.console.wraps import account_initialization_required, setup_required
|
||||||
from dify_graph.enums import WorkflowExecutionStatus
|
from core.workflow.enums import WorkflowExecutionStatus
|
||||||
from extensions.ext_database import db
|
from extensions.ext_database import db
|
||||||
from fields.workflow_app_log_fields import (
|
from fields.workflow_app_log_fields import (
|
||||||
build_workflow_app_log_pagination_model,
|
build_workflow_app_log_pagination_model,
|
||||||
|
|||||||
@@ -15,11 +15,11 @@ from controllers.console.app.error import (
|
|||||||
from controllers.console.app.wraps import get_app_model
|
from controllers.console.app.wraps import get_app_model
|
||||||
from controllers.console.wraps import account_initialization_required, edit_permission_required, setup_required
|
from controllers.console.wraps import account_initialization_required, edit_permission_required, setup_required
|
||||||
from controllers.web.error import InvalidArgumentError, NotFoundError
|
from controllers.web.error import InvalidArgumentError, NotFoundError
|
||||||
from dify_graph.constants import CONVERSATION_VARIABLE_NODE_ID, SYSTEM_VARIABLE_NODE_ID
|
from core.file import helpers as file_helpers
|
||||||
from dify_graph.file import helpers as file_helpers
|
from core.variables.segment_group import SegmentGroup
|
||||||
from dify_graph.variables.segment_group import SegmentGroup
|
from core.variables.segments import ArrayFileSegment, FileSegment, Segment
|
||||||
from dify_graph.variables.segments import ArrayFileSegment, FileSegment, Segment
|
from core.variables.types import SegmentType
|
||||||
from dify_graph.variables.types import SegmentType
|
from core.workflow.constants import CONVERSATION_VARIABLE_NODE_ID, SYSTEM_VARIABLE_NODE_ID
|
||||||
from extensions.ext_database import db
|
from extensions.ext_database import db
|
||||||
from factories.file_factory import build_from_mapping, build_from_mappings
|
from factories.file_factory import build_from_mapping, build_from_mappings
|
||||||
from factories.variable_factory import build_segment_with_type
|
from factories.variable_factory import build_segment_with_type
|
||||||
@@ -112,11 +112,11 @@ _WORKFLOW_DRAFT_VARIABLE_WITHOUT_VALUE_FIELDS = {
|
|||||||
"is_truncated": fields.Boolean(attribute=lambda model: model.file_id is not None),
|
"is_truncated": fields.Boolean(attribute=lambda model: model.file_id is not None),
|
||||||
}
|
}
|
||||||
|
|
||||||
_WORKFLOW_DRAFT_VARIABLE_FIELDS = {
|
_WORKFLOW_DRAFT_VARIABLE_FIELDS = dict(
|
||||||
**_WORKFLOW_DRAFT_VARIABLE_WITHOUT_VALUE_FIELDS,
|
_WORKFLOW_DRAFT_VARIABLE_WITHOUT_VALUE_FIELDS,
|
||||||
"value": fields.Raw(attribute=_serialize_var_value),
|
value=fields.Raw(attribute=_serialize_var_value),
|
||||||
"full_content": fields.Raw(attribute=_serialize_full_content),
|
full_content=fields.Raw(attribute=_serialize_full_content),
|
||||||
}
|
)
|
||||||
|
|
||||||
_WORKFLOW_DRAFT_ENV_VARIABLE_FIELDS = {
|
_WORKFLOW_DRAFT_ENV_VARIABLE_FIELDS = {
|
||||||
"id": fields.String,
|
"id": fields.String,
|
||||||
|
|||||||
@@ -12,8 +12,8 @@ from controllers.console import console_ns
|
|||||||
from controllers.console.app.wraps import get_app_model
|
from controllers.console.app.wraps import get_app_model
|
||||||
from controllers.console.wraps import account_initialization_required, setup_required
|
from controllers.console.wraps import account_initialization_required, setup_required
|
||||||
from controllers.web.error import NotFoundError
|
from controllers.web.error import NotFoundError
|
||||||
from dify_graph.entities.pause_reason import HumanInputRequired
|
from core.workflow.entities.pause_reason import HumanInputRequired
|
||||||
from dify_graph.enums import WorkflowExecutionStatus
|
from core.workflow.enums import WorkflowExecutionStatus
|
||||||
from extensions.ext_database import db
|
from extensions.ext_database import db
|
||||||
from fields.end_user_fields import simple_end_user_fields
|
from fields.end_user_fields import simple_end_user_fields
|
||||||
from fields.member_fields import simple_account_fields
|
from fields.member_fields import simple_account_fields
|
||||||
|
|||||||
@@ -8,7 +8,7 @@ from pydantic import BaseModel
|
|||||||
from werkzeug.exceptions import BadRequest, NotFound
|
from werkzeug.exceptions import BadRequest, NotFound
|
||||||
|
|
||||||
from controllers.console.wraps import account_initialization_required, setup_required
|
from controllers.console.wraps import account_initialization_required, setup_required
|
||||||
from dify_graph.model_runtime.utils.encoders import jsonable_encoder
|
from core.model_runtime.utils.encoders import jsonable_encoder
|
||||||
from libs.login import current_account_with_tenant, login_required
|
from libs.login import current_account_with_tenant, login_required
|
||||||
from models import Account
|
from models import Account
|
||||||
from models.model import OAuthProviderApp
|
from models.model import OAuthProviderApp
|
||||||
|
|||||||
@@ -25,12 +25,12 @@ from controllers.console.wraps import (
|
|||||||
)
|
)
|
||||||
from core.errors.error import LLMBadRequestError, ProviderTokenNotInitError
|
from core.errors.error import LLMBadRequestError, ProviderTokenNotInitError
|
||||||
from core.indexing_runner import IndexingRunner
|
from core.indexing_runner import IndexingRunner
|
||||||
|
from core.model_runtime.entities.model_entities import ModelType
|
||||||
from core.provider_manager import ProviderManager
|
from core.provider_manager import ProviderManager
|
||||||
from core.rag.datasource.vdb.vector_type import VectorType
|
from core.rag.datasource.vdb.vector_type import VectorType
|
||||||
from core.rag.extractor.entity.datasource_type import DatasourceType
|
from core.rag.extractor.entity.datasource_type import DatasourceType
|
||||||
from core.rag.extractor.entity.extract_setting import ExtractSetting, NotionInfo, WebsiteInfo
|
from core.rag.extractor.entity.extract_setting import ExtractSetting, NotionInfo, WebsiteInfo
|
||||||
from core.rag.retrieval.retrieval_methods import RetrievalMethod
|
from core.rag.retrieval.retrieval_methods import RetrievalMethod
|
||||||
from dify_graph.model_runtime.entities.model_entities import ModelType
|
|
||||||
from extensions.ext_database import db
|
from extensions.ext_database import db
|
||||||
from fields.app_fields import app_detail_kernel_fields, related_app_list
|
from fields.app_fields import app_detail_kernel_fields, related_app_list
|
||||||
from fields.dataset_fields import (
|
from fields.dataset_fields import (
|
||||||
@@ -53,7 +53,7 @@ from fields.dataset_fields import (
|
|||||||
from fields.document_fields import document_status_fields
|
from fields.document_fields import document_status_fields
|
||||||
from libs.login import current_account_with_tenant, login_required
|
from libs.login import current_account_with_tenant, login_required
|
||||||
from models import ApiToken, Dataset, Document, DocumentSegment, UploadFile
|
from models import ApiToken, Dataset, Document, DocumentSegment, UploadFile
|
||||||
from models.dataset import DatasetPermission, DatasetPermissionEnum
|
from models.dataset import DatasetPermissionEnum
|
||||||
from models.provider_ids import ModelProviderID
|
from models.provider_ids import ModelProviderID
|
||||||
from services.api_token_service import ApiTokenCache
|
from services.api_token_service import ApiTokenCache
|
||||||
from services.dataset_service import DatasetPermissionService, DatasetService, DocumentService
|
from services.dataset_service import DatasetPermissionService, DatasetService, DocumentService
|
||||||
@@ -119,14 +119,6 @@ def _validate_indexing_technique(value: str | None) -> str | None:
|
|||||||
return value
|
return value
|
||||||
|
|
||||||
|
|
||||||
def _validate_doc_form(value: str | None) -> str | None:
|
|
||||||
if value is None:
|
|
||||||
return value
|
|
||||||
if value not in Dataset.DOC_FORM_LIST:
|
|
||||||
raise ValueError("Invalid doc_form.")
|
|
||||||
return value
|
|
||||||
|
|
||||||
|
|
||||||
class DatasetCreatePayload(BaseModel):
|
class DatasetCreatePayload(BaseModel):
|
||||||
name: str = Field(..., min_length=1, max_length=40)
|
name: str = Field(..., min_length=1, max_length=40)
|
||||||
description: str = Field("", max_length=400)
|
description: str = Field("", max_length=400)
|
||||||
@@ -187,14 +179,6 @@ class IndexingEstimatePayload(BaseModel):
|
|||||||
raise ValueError("indexing_technique is required.")
|
raise ValueError("indexing_technique is required.")
|
||||||
return result
|
return result
|
||||||
|
|
||||||
@field_validator("doc_form")
|
|
||||||
@classmethod
|
|
||||||
def validate_doc_form(cls, value: str) -> str:
|
|
||||||
result = _validate_doc_form(value)
|
|
||||||
if result is None:
|
|
||||||
return "text_model"
|
|
||||||
return result
|
|
||||||
|
|
||||||
|
|
||||||
class ConsoleDatasetListQuery(BaseModel):
|
class ConsoleDatasetListQuery(BaseModel):
|
||||||
page: int = Field(default=1, description="Page number")
|
page: int = Field(default=1, description="Page number")
|
||||||
@@ -339,18 +323,6 @@ class DatasetListApi(Resource):
|
|||||||
model_names.append(f"{embedding_model.model}:{embedding_model.provider.provider}")
|
model_names.append(f"{embedding_model.model}:{embedding_model.provider.provider}")
|
||||||
|
|
||||||
data = cast(list[dict[str, Any]], marshal(datasets, dataset_detail_fields))
|
data = cast(list[dict[str, Any]], marshal(datasets, dataset_detail_fields))
|
||||||
dataset_ids = [item["id"] for item in data if item.get("permission") == "partial_members"]
|
|
||||||
partial_members_map: dict[str, list[str]] = {}
|
|
||||||
if dataset_ids:
|
|
||||||
permissions = db.session.execute(
|
|
||||||
select(DatasetPermission.dataset_id, DatasetPermission.account_id).where(
|
|
||||||
DatasetPermission.dataset_id.in_(dataset_ids)
|
|
||||||
)
|
|
||||||
).all()
|
|
||||||
|
|
||||||
for dataset_id, account_id in permissions:
|
|
||||||
partial_members_map.setdefault(dataset_id, []).append(account_id)
|
|
||||||
|
|
||||||
for item in data:
|
for item in data:
|
||||||
# convert embedding_model_provider to plugin standard format
|
# convert embedding_model_provider to plugin standard format
|
||||||
if item["indexing_technique"] == "high_quality" and item["embedding_model_provider"]:
|
if item["indexing_technique"] == "high_quality" and item["embedding_model_provider"]:
|
||||||
@@ -364,7 +336,8 @@ class DatasetListApi(Resource):
|
|||||||
item["embedding_available"] = True
|
item["embedding_available"] = True
|
||||||
|
|
||||||
if item.get("permission") == "partial_members":
|
if item.get("permission") == "partial_members":
|
||||||
item.update({"partial_member_list": partial_members_map.get(item["id"], [])})
|
part_users_list = DatasetPermissionService.get_dataset_partial_member_list(item["id"])
|
||||||
|
item.update({"partial_member_list": part_users_list})
|
||||||
else:
|
else:
|
||||||
item.update({"partial_member_list": []})
|
item.update({"partial_member_list": []})
|
||||||
|
|
||||||
|
|||||||
@@ -24,11 +24,11 @@ from core.errors.error import (
|
|||||||
)
|
)
|
||||||
from core.indexing_runner import IndexingRunner
|
from core.indexing_runner import IndexingRunner
|
||||||
from core.model_manager import ModelManager
|
from core.model_manager import ModelManager
|
||||||
|
from core.model_runtime.entities.model_entities import ModelType
|
||||||
|
from core.model_runtime.errors.invoke import InvokeAuthorizationError
|
||||||
from core.plugin.impl.exc import PluginDaemonClientSideError
|
from core.plugin.impl.exc import PluginDaemonClientSideError
|
||||||
from core.rag.extractor.entity.datasource_type import DatasourceType
|
from core.rag.extractor.entity.datasource_type import DatasourceType
|
||||||
from core.rag.extractor.entity.extract_setting import ExtractSetting, NotionInfo, WebsiteInfo
|
from core.rag.extractor.entity.extract_setting import ExtractSetting, NotionInfo, WebsiteInfo
|
||||||
from dify_graph.model_runtime.entities.model_entities import ModelType
|
|
||||||
from dify_graph.model_runtime.errors.invoke import InvokeAuthorizationError
|
|
||||||
from extensions.ext_database import db
|
from extensions.ext_database import db
|
||||||
from fields.dataset_fields import dataset_fields
|
from fields.dataset_fields import dataset_fields
|
||||||
from fields.document_fields import (
|
from fields.document_fields import (
|
||||||
|
|||||||
@@ -26,7 +26,7 @@ from controllers.console.wraps import (
|
|||||||
)
|
)
|
||||||
from core.errors.error import LLMBadRequestError, ProviderTokenNotInitError
|
from core.errors.error import LLMBadRequestError, ProviderTokenNotInitError
|
||||||
from core.model_manager import ModelManager
|
from core.model_manager import ModelManager
|
||||||
from dify_graph.model_runtime.entities.model_entities import ModelType
|
from core.model_runtime.entities.model_entities import ModelType
|
||||||
from extensions.ext_database import db
|
from extensions.ext_database import db
|
||||||
from extensions.ext_redis import redis_client
|
from extensions.ext_redis import redis_client
|
||||||
from fields.segment_fields import child_chunk_fields, segment_fields
|
from fields.segment_fields import child_chunk_fields, segment_fields
|
||||||
|
|||||||
@@ -19,7 +19,7 @@ from core.errors.error import (
|
|||||||
ProviderTokenNotInitError,
|
ProviderTokenNotInitError,
|
||||||
QuotaExceededError,
|
QuotaExceededError,
|
||||||
)
|
)
|
||||||
from dify_graph.model_runtime.errors.invoke import InvokeError
|
from core.model_runtime.errors.invoke import InvokeError
|
||||||
from fields.hit_testing_fields import hit_testing_record_fields
|
from fields.hit_testing_fields import hit_testing_record_fields
|
||||||
from libs.login import current_user
|
from libs.login import current_user
|
||||||
from models.account import Account
|
from models.account import Account
|
||||||
|
|||||||
@@ -9,9 +9,9 @@ from configs import dify_config
|
|||||||
from controllers.common.schema import register_schema_models
|
from controllers.common.schema import register_schema_models
|
||||||
from controllers.console import console_ns
|
from controllers.console import console_ns
|
||||||
from controllers.console.wraps import account_initialization_required, edit_permission_required, setup_required
|
from controllers.console.wraps import account_initialization_required, edit_permission_required, setup_required
|
||||||
|
from core.model_runtime.errors.validate import CredentialsValidateFailedError
|
||||||
|
from core.model_runtime.utils.encoders import jsonable_encoder
|
||||||
from core.plugin.impl.oauth import OAuthHandler
|
from core.plugin.impl.oauth import OAuthHandler
|
||||||
from dify_graph.model_runtime.errors.validate import CredentialsValidateFailedError
|
|
||||||
from dify_graph.model_runtime.utils.encoders import jsonable_encoder
|
|
||||||
from libs.login import current_account_with_tenant, login_required
|
from libs.login import current_account_with_tenant, login_required
|
||||||
from models.provider_ids import DatasourceProviderID
|
from models.provider_ids import DatasourceProviderID
|
||||||
from services.datasource_provider_service import DatasourceProviderService
|
from services.datasource_provider_service import DatasourceProviderService
|
||||||
|
|||||||
@@ -21,8 +21,8 @@ from controllers.console.app.workflow_draft_variable import (
|
|||||||
from controllers.console.datasets.wraps import get_rag_pipeline
|
from controllers.console.datasets.wraps import get_rag_pipeline
|
||||||
from controllers.console.wraps import account_initialization_required, setup_required
|
from controllers.console.wraps import account_initialization_required, setup_required
|
||||||
from controllers.web.error import InvalidArgumentError, NotFoundError
|
from controllers.web.error import InvalidArgumentError, NotFoundError
|
||||||
from dify_graph.constants import CONVERSATION_VARIABLE_NODE_ID, SYSTEM_VARIABLE_NODE_ID
|
from core.variables.types import SegmentType
|
||||||
from dify_graph.variables.types import SegmentType
|
from core.workflow.constants import CONVERSATION_VARIABLE_NODE_ID, SYSTEM_VARIABLE_NODE_ID
|
||||||
from extensions.ext_database import db
|
from extensions.ext_database import db
|
||||||
from factories.file_factory import build_from_mapping, build_from_mappings
|
from factories.file_factory import build_from_mapping, build_from_mappings
|
||||||
from factories.variable_factory import build_segment_with_type
|
from factories.variable_factory import build_segment_with_type
|
||||||
|
|||||||
@@ -33,7 +33,7 @@ from controllers.web.error import InvokeRateLimitError as InvokeRateLimitHttpErr
|
|||||||
from core.app.apps.base_app_queue_manager import AppQueueManager
|
from core.app.apps.base_app_queue_manager import AppQueueManager
|
||||||
from core.app.apps.pipeline.pipeline_generator import PipelineGenerator
|
from core.app.apps.pipeline.pipeline_generator import PipelineGenerator
|
||||||
from core.app.entities.app_invoke_entities import InvokeFrom
|
from core.app.entities.app_invoke_entities import InvokeFrom
|
||||||
from dify_graph.model_runtime.utils.encoders import jsonable_encoder
|
from core.model_runtime.utils.encoders import jsonable_encoder
|
||||||
from extensions.ext_database import db
|
from extensions.ext_database import db
|
||||||
from factories import variable_factory
|
from factories import variable_factory
|
||||||
from libs import helper
|
from libs import helper
|
||||||
|
|||||||
@@ -19,7 +19,7 @@ from controllers.console.app.error import (
|
|||||||
)
|
)
|
||||||
from controllers.console.explore.wraps import InstalledAppResource
|
from controllers.console.explore.wraps import InstalledAppResource
|
||||||
from core.errors.error import ModelCurrentlyNotSupportError, ProviderTokenNotInitError, QuotaExceededError
|
from core.errors.error import ModelCurrentlyNotSupportError, ProviderTokenNotInitError, QuotaExceededError
|
||||||
from dify_graph.model_runtime.errors.invoke import InvokeError
|
from core.model_runtime.errors.invoke import InvokeError
|
||||||
from services.audio_service import AudioService
|
from services.audio_service import AudioService
|
||||||
from services.errors.audio import (
|
from services.errors.audio import (
|
||||||
AudioTooLargeServiceError,
|
AudioTooLargeServiceError,
|
||||||
|
|||||||
@@ -24,7 +24,7 @@ from core.errors.error import (
|
|||||||
ProviderTokenNotInitError,
|
ProviderTokenNotInitError,
|
||||||
QuotaExceededError,
|
QuotaExceededError,
|
||||||
)
|
)
|
||||||
from dify_graph.model_runtime.errors.invoke import InvokeError
|
from core.model_runtime.errors.invoke import InvokeError
|
||||||
from extensions.ext_database import db
|
from extensions.ext_database import db
|
||||||
from libs import helper
|
from libs import helper
|
||||||
from libs.datetime_utils import naive_utc_now
|
from libs.datetime_utils import naive_utc_now
|
||||||
|
|||||||
@@ -21,7 +21,7 @@ from controllers.console.explore.error import (
|
|||||||
from controllers.console.explore.wraps import InstalledAppResource
|
from controllers.console.explore.wraps import InstalledAppResource
|
||||||
from core.app.entities.app_invoke_entities import InvokeFrom
|
from core.app.entities.app_invoke_entities import InvokeFrom
|
||||||
from core.errors.error import ModelCurrentlyNotSupportError, ProviderTokenNotInitError, QuotaExceededError
|
from core.errors.error import ModelCurrentlyNotSupportError, ProviderTokenNotInitError, QuotaExceededError
|
||||||
from dify_graph.model_runtime.errors.invoke import InvokeError
|
from core.model_runtime.errors.invoke import InvokeError
|
||||||
from fields.conversation_fields import ResultResponse
|
from fields.conversation_fields import ResultResponse
|
||||||
from fields.message_fields import MessageInfiniteScrollPagination, MessageListItem, SuggestedQuestionsResponse
|
from fields.message_fields import MessageInfiniteScrollPagination, MessageListItem, SuggestedQuestionsResponse
|
||||||
from libs import helper
|
from libs import helper
|
||||||
|
|||||||
@@ -10,7 +10,7 @@ import services
|
|||||||
from controllers.common.fields import Parameters as ParametersResponse
|
from controllers.common.fields import Parameters as ParametersResponse
|
||||||
from controllers.common.fields import Site as SiteResponse
|
from controllers.common.fields import Site as SiteResponse
|
||||||
from controllers.common.schema import get_or_create_model
|
from controllers.common.schema import get_or_create_model
|
||||||
from controllers.console import console_ns
|
from controllers.console import api, console_ns
|
||||||
from controllers.console.app.error import (
|
from controllers.console.app.error import (
|
||||||
AppUnavailableError,
|
AppUnavailableError,
|
||||||
AudioTooLargeError,
|
AudioTooLargeError,
|
||||||
@@ -41,10 +41,9 @@ from core.errors.error import (
|
|||||||
ProviderTokenNotInitError,
|
ProviderTokenNotInitError,
|
||||||
QuotaExceededError,
|
QuotaExceededError,
|
||||||
)
|
)
|
||||||
from dify_graph.graph_engine.manager import GraphEngineManager
|
from core.model_runtime.errors.invoke import InvokeError
|
||||||
from dify_graph.model_runtime.errors.invoke import InvokeError
|
from core.workflow.graph_engine.manager import GraphEngineManager
|
||||||
from extensions.ext_database import db
|
from extensions.ext_database import db
|
||||||
from extensions.ext_redis import redis_client
|
|
||||||
from fields.app_fields import (
|
from fields.app_fields import (
|
||||||
app_detail_fields_with_site,
|
app_detail_fields_with_site,
|
||||||
deleted_tool_fields,
|
deleted_tool_fields,
|
||||||
@@ -226,7 +225,7 @@ class TrialAppWorkflowTaskStopApi(TrialAppResource):
|
|||||||
AppQueueManager.set_stop_flag_no_user_check(task_id)
|
AppQueueManager.set_stop_flag_no_user_check(task_id)
|
||||||
|
|
||||||
# New graph engine command channel mechanism
|
# New graph engine command channel mechanism
|
||||||
GraphEngineManager(redis_client).send_stop_command(task_id)
|
GraphEngineManager.send_stop_command(task_id)
|
||||||
|
|
||||||
return {"result": "success"}
|
return {"result": "success"}
|
||||||
|
|
||||||
@@ -470,7 +469,7 @@ class TrialSitApi(Resource):
|
|||||||
"""Resource for trial app sites."""
|
"""Resource for trial app sites."""
|
||||||
|
|
||||||
@trial_feature_enable
|
@trial_feature_enable
|
||||||
@get_app_model_with_trial(None)
|
@get_app_model_with_trial
|
||||||
def get(self, app_model):
|
def get(self, app_model):
|
||||||
"""Retrieve app site info.
|
"""Retrieve app site info.
|
||||||
|
|
||||||
@@ -492,7 +491,7 @@ class TrialAppParameterApi(Resource):
|
|||||||
"""Resource for app variables."""
|
"""Resource for app variables."""
|
||||||
|
|
||||||
@trial_feature_enable
|
@trial_feature_enable
|
||||||
@get_app_model_with_trial(None)
|
@get_app_model_with_trial
|
||||||
def get(self, app_model):
|
def get(self, app_model):
|
||||||
"""Retrieve app parameters."""
|
"""Retrieve app parameters."""
|
||||||
|
|
||||||
@@ -521,7 +520,7 @@ class TrialAppParameterApi(Resource):
|
|||||||
|
|
||||||
class AppApi(Resource):
|
class AppApi(Resource):
|
||||||
@trial_feature_enable
|
@trial_feature_enable
|
||||||
@get_app_model_with_trial(None)
|
@get_app_model_with_trial
|
||||||
@marshal_with(app_detail_with_site_model)
|
@marshal_with(app_detail_with_site_model)
|
||||||
def get(self, app_model):
|
def get(self, app_model):
|
||||||
"""Get app detail"""
|
"""Get app detail"""
|
||||||
@@ -534,7 +533,7 @@ class AppApi(Resource):
|
|||||||
|
|
||||||
class AppWorkflowApi(Resource):
|
class AppWorkflowApi(Resource):
|
||||||
@trial_feature_enable
|
@trial_feature_enable
|
||||||
@get_app_model_with_trial(None)
|
@get_app_model_with_trial
|
||||||
@marshal_with(workflow_model)
|
@marshal_with(workflow_model)
|
||||||
def get(self, app_model):
|
def get(self, app_model):
|
||||||
"""Get workflow detail"""
|
"""Get workflow detail"""
|
||||||
@@ -553,7 +552,7 @@ class AppWorkflowApi(Resource):
|
|||||||
|
|
||||||
class DatasetListApi(Resource):
|
class DatasetListApi(Resource):
|
||||||
@trial_feature_enable
|
@trial_feature_enable
|
||||||
@get_app_model_with_trial(None)
|
@get_app_model_with_trial
|
||||||
def get(self, app_model):
|
def get(self, app_model):
|
||||||
page = request.args.get("page", default=1, type=int)
|
page = request.args.get("page", default=1, type=int)
|
||||||
limit = request.args.get("limit", default=20, type=int)
|
limit = request.args.get("limit", default=20, type=int)
|
||||||
@@ -571,31 +570,27 @@ class DatasetListApi(Resource):
|
|||||||
return response
|
return response
|
||||||
|
|
||||||
|
|
||||||
console_ns.add_resource(TrialChatApi, "/trial-apps/<uuid:app_id>/chat-messages", endpoint="trial_app_chat_completion")
|
api.add_resource(TrialChatApi, "/trial-apps/<uuid:app_id>/chat-messages", endpoint="trial_app_chat_completion")
|
||||||
|
|
||||||
console_ns.add_resource(
|
api.add_resource(
|
||||||
TrialMessageSuggestedQuestionApi,
|
TrialMessageSuggestedQuestionApi,
|
||||||
"/trial-apps/<uuid:app_id>/messages/<uuid:message_id>/suggested-questions",
|
"/trial-apps/<uuid:app_id>/messages/<uuid:message_id>/suggested-questions",
|
||||||
endpoint="trial_app_suggested_question",
|
endpoint="trial_app_suggested_question",
|
||||||
)
|
)
|
||||||
|
|
||||||
console_ns.add_resource(TrialChatAudioApi, "/trial-apps/<uuid:app_id>/audio-to-text", endpoint="trial_app_audio")
|
api.add_resource(TrialChatAudioApi, "/trial-apps/<uuid:app_id>/audio-to-text", endpoint="trial_app_audio")
|
||||||
console_ns.add_resource(TrialChatTextApi, "/trial-apps/<uuid:app_id>/text-to-audio", endpoint="trial_app_text")
|
api.add_resource(TrialChatTextApi, "/trial-apps/<uuid:app_id>/text-to-audio", endpoint="trial_app_text")
|
||||||
|
|
||||||
console_ns.add_resource(
|
api.add_resource(TrialCompletionApi, "/trial-apps/<uuid:app_id>/completion-messages", endpoint="trial_app_completion")
|
||||||
TrialCompletionApi, "/trial-apps/<uuid:app_id>/completion-messages", endpoint="trial_app_completion"
|
|
||||||
)
|
|
||||||
|
|
||||||
console_ns.add_resource(TrialSitApi, "/trial-apps/<uuid:app_id>/site")
|
api.add_resource(TrialSitApi, "/trial-apps/<uuid:app_id>/site")
|
||||||
|
|
||||||
console_ns.add_resource(TrialAppParameterApi, "/trial-apps/<uuid:app_id>/parameters", endpoint="trial_app_parameters")
|
api.add_resource(TrialAppParameterApi, "/trial-apps/<uuid:app_id>/parameters", endpoint="trial_app_parameters")
|
||||||
|
|
||||||
console_ns.add_resource(AppApi, "/trial-apps/<uuid:app_id>", endpoint="trial_app")
|
api.add_resource(AppApi, "/trial-apps/<uuid:app_id>", endpoint="trial_app")
|
||||||
|
|
||||||
console_ns.add_resource(
|
api.add_resource(TrialAppWorkflowRunApi, "/trial-apps/<uuid:app_id>/workflows/run", endpoint="trial_app_workflow_run")
|
||||||
TrialAppWorkflowRunApi, "/trial-apps/<uuid:app_id>/workflows/run", endpoint="trial_app_workflow_run"
|
api.add_resource(TrialAppWorkflowTaskStopApi, "/trial-apps/<uuid:app_id>/workflows/tasks/<string:task_id>/stop")
|
||||||
)
|
|
||||||
console_ns.add_resource(TrialAppWorkflowTaskStopApi, "/trial-apps/<uuid:app_id>/workflows/tasks/<string:task_id>/stop")
|
|
||||||
|
|
||||||
console_ns.add_resource(AppWorkflowApi, "/trial-apps/<uuid:app_id>/workflows", endpoint="trial_app_workflow")
|
api.add_resource(AppWorkflowApi, "/trial-apps/<uuid:app_id>/workflows", endpoint="trial_app_workflow")
|
||||||
console_ns.add_resource(DatasetListApi, "/trial-apps/<uuid:app_id>/datasets", endpoint="trial_app_datasets")
|
api.add_resource(DatasetListApi, "/trial-apps/<uuid:app_id>/datasets", endpoint="trial_app_datasets")
|
||||||
|
|||||||
@@ -21,9 +21,8 @@ from core.errors.error import (
|
|||||||
ProviderTokenNotInitError,
|
ProviderTokenNotInitError,
|
||||||
QuotaExceededError,
|
QuotaExceededError,
|
||||||
)
|
)
|
||||||
from dify_graph.graph_engine.manager import GraphEngineManager
|
from core.model_runtime.errors.invoke import InvokeError
|
||||||
from dify_graph.model_runtime.errors.invoke import InvokeError
|
from core.workflow.graph_engine.manager import GraphEngineManager
|
||||||
from extensions.ext_redis import redis_client
|
|
||||||
from libs import helper
|
from libs import helper
|
||||||
from libs.login import current_account_with_tenant
|
from libs.login import current_account_with_tenant
|
||||||
from models.model import AppMode, InstalledApp
|
from models.model import AppMode, InstalledApp
|
||||||
@@ -101,6 +100,6 @@ class InstalledAppWorkflowTaskStopApi(InstalledAppResource):
|
|||||||
AppQueueManager.set_stop_flag_no_user_check(task_id)
|
AppQueueManager.set_stop_flag_no_user_check(task_id)
|
||||||
|
|
||||||
# New graph engine command channel mechanism
|
# New graph engine command channel mechanism
|
||||||
GraphEngineManager(redis_client).send_stop_command(task_id)
|
GraphEngineManager.send_stop_command(task_id)
|
||||||
|
|
||||||
return {"result": "success"}
|
return {"result": "success"}
|
||||||
|
|||||||
@@ -105,9 +105,9 @@ def trial_app_required(view: Callable[Concatenate[App, P], R] | None = None):
|
|||||||
return decorator
|
return decorator
|
||||||
|
|
||||||
|
|
||||||
def trial_feature_enable(view: Callable[P, R]):
|
def trial_feature_enable(view: Callable[..., R]) -> Callable[..., R]:
|
||||||
@wraps(view)
|
@wraps(view)
|
||||||
def decorated(*args: P.args, **kwargs: P.kwargs):
|
def decorated(*args, **kwargs):
|
||||||
features = FeatureService.get_system_features()
|
features = FeatureService.get_system_features()
|
||||||
if not features.enable_trial_app:
|
if not features.enable_trial_app:
|
||||||
abort(403, "Trial app feature is not enabled.")
|
abort(403, "Trial app feature is not enabled.")
|
||||||
@@ -116,9 +116,9 @@ def trial_feature_enable(view: Callable[P, R]):
|
|||||||
return decorated
|
return decorated
|
||||||
|
|
||||||
|
|
||||||
def explore_banner_enabled(view: Callable[P, R]):
|
def explore_banner_enabled(view: Callable[..., R]) -> Callable[..., R]:
|
||||||
@wraps(view)
|
@wraps(view)
|
||||||
def decorated(*args: P.args, **kwargs: P.kwargs):
|
def decorated(*args, **kwargs):
|
||||||
features = FeatureService.get_system_features()
|
features = FeatureService.get_system_features()
|
||||||
if not features.enable_explore_banner:
|
if not features.enable_explore_banner:
|
||||||
abort(403, "Explore banner feature is not enabled.")
|
abort(403, "Explore banner feature is not enabled.")
|
||||||
|
|||||||
@@ -12,8 +12,8 @@ from controllers.common.errors import (
|
|||||||
UnsupportedFileTypeError,
|
UnsupportedFileTypeError,
|
||||||
)
|
)
|
||||||
from controllers.console import console_ns
|
from controllers.console import console_ns
|
||||||
|
from core.file import helpers as file_helpers
|
||||||
from core.helper import ssrf_proxy
|
from core.helper import ssrf_proxy
|
||||||
from dify_graph.file import helpers as file_helpers
|
|
||||||
from extensions.ext_database import db
|
from extensions.ext_database import db
|
||||||
from fields.file_fields import FileWithSignedUrl, RemoteFileInfo
|
from fields.file_fields import FileWithSignedUrl, RemoteFileInfo
|
||||||
from libs.login import current_account_with_tenant, login_required
|
from libs.login import current_account_with_tenant, login_required
|
||||||
|
|||||||
@@ -2,7 +2,7 @@ from flask_restx import Resource, fields
|
|||||||
|
|
||||||
from controllers.console import console_ns
|
from controllers.console import console_ns
|
||||||
from controllers.console.wraps import account_initialization_required, setup_required
|
from controllers.console.wraps import account_initialization_required, setup_required
|
||||||
from dify_graph.model_runtime.utils.encoders import jsonable_encoder
|
from core.model_runtime.utils.encoders import jsonable_encoder
|
||||||
from libs.login import current_account_with_tenant, login_required
|
from libs.login import current_account_with_tenant, login_required
|
||||||
from services.agent_service import AgentService
|
from services.agent_service import AgentService
|
||||||
|
|
||||||
|
|||||||
@@ -7,8 +7,8 @@ from pydantic import BaseModel, Field
|
|||||||
from controllers.common.schema import register_schema_models
|
from controllers.common.schema import register_schema_models
|
||||||
from controllers.console import console_ns
|
from controllers.console import console_ns
|
||||||
from controllers.console.wraps import account_initialization_required, is_admin_or_owner_required, setup_required
|
from controllers.console.wraps import account_initialization_required, is_admin_or_owner_required, setup_required
|
||||||
|
from core.model_runtime.utils.encoders import jsonable_encoder
|
||||||
from core.plugin.impl.exc import PluginPermissionDeniedError
|
from core.plugin.impl.exc import PluginPermissionDeniedError
|
||||||
from dify_graph.model_runtime.utils.encoders import jsonable_encoder
|
|
||||||
from libs.login import current_account_with_tenant, login_required
|
from libs.login import current_account_with_tenant, login_required
|
||||||
from services.plugin.endpoint_service import EndpointService
|
from services.plugin.endpoint_service import EndpointService
|
||||||
|
|
||||||
|
|||||||
@@ -5,8 +5,8 @@ from werkzeug.exceptions import Forbidden
|
|||||||
from controllers.common.schema import register_schema_models
|
from controllers.common.schema import register_schema_models
|
||||||
from controllers.console import console_ns
|
from controllers.console import console_ns
|
||||||
from controllers.console.wraps import account_initialization_required, setup_required
|
from controllers.console.wraps import account_initialization_required, setup_required
|
||||||
from dify_graph.model_runtime.entities.model_entities import ModelType
|
from core.model_runtime.entities.model_entities import ModelType
|
||||||
from dify_graph.model_runtime.errors.validate import CredentialsValidateFailedError
|
from core.model_runtime.errors.validate import CredentialsValidateFailedError
|
||||||
from libs.login import current_account_with_tenant, login_required
|
from libs.login import current_account_with_tenant, login_required
|
||||||
from models import TenantAccountRole
|
from models import TenantAccountRole
|
||||||
from services.model_load_balancing_service import ModelLoadBalancingService
|
from services.model_load_balancing_service import ModelLoadBalancingService
|
||||||
|
|||||||
@@ -7,9 +7,9 @@ from pydantic import BaseModel, Field, field_validator
|
|||||||
|
|
||||||
from controllers.console import console_ns
|
from controllers.console import console_ns
|
||||||
from controllers.console.wraps import account_initialization_required, is_admin_or_owner_required, setup_required
|
from controllers.console.wraps import account_initialization_required, is_admin_or_owner_required, setup_required
|
||||||
from dify_graph.model_runtime.entities.model_entities import ModelType
|
from core.model_runtime.entities.model_entities import ModelType
|
||||||
from dify_graph.model_runtime.errors.validate import CredentialsValidateFailedError
|
from core.model_runtime.errors.validate import CredentialsValidateFailedError
|
||||||
from dify_graph.model_runtime.utils.encoders import jsonable_encoder
|
from core.model_runtime.utils.encoders import jsonable_encoder
|
||||||
from libs.helper import uuid_value
|
from libs.helper import uuid_value
|
||||||
from libs.login import current_account_with_tenant, login_required
|
from libs.login import current_account_with_tenant, login_required
|
||||||
from services.billing_service import BillingService
|
from services.billing_service import BillingService
|
||||||
|
|||||||
@@ -8,9 +8,9 @@ from pydantic import BaseModel, Field, field_validator
|
|||||||
from controllers.common.schema import register_enum_models, register_schema_models
|
from controllers.common.schema import register_enum_models, register_schema_models
|
||||||
from controllers.console import console_ns
|
from controllers.console import console_ns
|
||||||
from controllers.console.wraps import account_initialization_required, is_admin_or_owner_required, setup_required
|
from controllers.console.wraps import account_initialization_required, is_admin_or_owner_required, setup_required
|
||||||
from dify_graph.model_runtime.entities.model_entities import ModelType
|
from core.model_runtime.entities.model_entities import ModelType
|
||||||
from dify_graph.model_runtime.errors.validate import CredentialsValidateFailedError
|
from core.model_runtime.errors.validate import CredentialsValidateFailedError
|
||||||
from dify_graph.model_runtime.utils.encoders import jsonable_encoder
|
from core.model_runtime.utils.encoders import jsonable_encoder
|
||||||
from libs.helper import uuid_value
|
from libs.helper import uuid_value
|
||||||
from libs.login import current_account_with_tenant, login_required
|
from libs.login import current_account_with_tenant, login_required
|
||||||
from services.model_load_balancing_service import ModelLoadBalancingService
|
from services.model_load_balancing_service import ModelLoadBalancingService
|
||||||
|
|||||||
@@ -12,8 +12,8 @@ from controllers.common.schema import register_enum_models, register_schema_mode
|
|||||||
from controllers.console import console_ns
|
from controllers.console import console_ns
|
||||||
from controllers.console.workspace import plugin_permission_required
|
from controllers.console.workspace import plugin_permission_required
|
||||||
from controllers.console.wraps import account_initialization_required, is_admin_or_owner_required, setup_required
|
from controllers.console.wraps import account_initialization_required, is_admin_or_owner_required, setup_required
|
||||||
|
from core.model_runtime.utils.encoders import jsonable_encoder
|
||||||
from core.plugin.impl.exc import PluginDaemonClientSideError
|
from core.plugin.impl.exc import PluginDaemonClientSideError
|
||||||
from dify_graph.model_runtime.utils.encoders import jsonable_encoder
|
|
||||||
from libs.login import current_account_with_tenant, login_required
|
from libs.login import current_account_with_tenant, login_required
|
||||||
from models.account import TenantPluginAutoUpgradeStrategy, TenantPluginPermission
|
from models.account import TenantPluginAutoUpgradeStrategy, TenantPluginPermission
|
||||||
from services.plugin.plugin_auto_upgrade_service import PluginAutoUpgradeService
|
from services.plugin.plugin_auto_upgrade_service import PluginAutoUpgradeService
|
||||||
|
|||||||
@@ -23,10 +23,10 @@ from core.entities.mcp_provider import MCPAuthentication, MCPConfiguration
|
|||||||
from core.mcp.auth.auth_flow import auth, handle_callback
|
from core.mcp.auth.auth_flow import auth, handle_callback
|
||||||
from core.mcp.error import MCPAuthError, MCPError, MCPRefreshTokenError
|
from core.mcp.error import MCPAuthError, MCPError, MCPRefreshTokenError
|
||||||
from core.mcp.mcp_client import MCPClient
|
from core.mcp.mcp_client import MCPClient
|
||||||
|
from core.model_runtime.utils.encoders import jsonable_encoder
|
||||||
from core.plugin.entities.plugin_daemon import CredentialType
|
from core.plugin.entities.plugin_daemon import CredentialType
|
||||||
from core.plugin.impl.oauth import OAuthHandler
|
from core.plugin.impl.oauth import OAuthHandler
|
||||||
from core.tools.entities.tool_entities import ApiProviderSchemaType, WorkflowToolParameterConfiguration
|
from core.tools.entities.tool_entities import ApiProviderSchemaType, WorkflowToolParameterConfiguration
|
||||||
from dify_graph.model_runtime.utils.encoders import jsonable_encoder
|
|
||||||
from extensions.ext_database import db
|
from extensions.ext_database import db
|
||||||
from libs.helper import alphanumeric, uuid_value
|
from libs.helper import alphanumeric, uuid_value
|
||||||
from libs.login import current_account_with_tenant, login_required
|
from libs.login import current_account_with_tenant, login_required
|
||||||
|
|||||||
@@ -10,11 +10,11 @@ from werkzeug.exceptions import BadRequest, Forbidden
|
|||||||
from configs import dify_config
|
from configs import dify_config
|
||||||
from controllers.common.schema import register_schema_models
|
from controllers.common.schema import register_schema_models
|
||||||
from controllers.web.error import NotFoundError
|
from controllers.web.error import NotFoundError
|
||||||
|
from core.model_runtime.utils.encoders import jsonable_encoder
|
||||||
from core.plugin.entities.plugin_daemon import CredentialType
|
from core.plugin.entities.plugin_daemon import CredentialType
|
||||||
from core.plugin.impl.oauth import OAuthHandler
|
from core.plugin.impl.oauth import OAuthHandler
|
||||||
from core.trigger.entities.entities import SubscriptionBuilderUpdater
|
from core.trigger.entities.entities import SubscriptionBuilderUpdater
|
||||||
from core.trigger.trigger_manager import TriggerManager
|
from core.trigger.trigger_manager import TriggerManager
|
||||||
from dify_graph.model_runtime.utils.encoders import jsonable_encoder
|
|
||||||
from extensions.ext_database import db
|
from extensions.ext_database import db
|
||||||
from libs.login import current_user, login_required
|
from libs.login import current_user, login_required
|
||||||
from models.account import Account
|
from models.account import Account
|
||||||
|
|||||||
@@ -36,9 +36,9 @@ ERROR_MSG_INVALID_ENCRYPTED_DATA = "Invalid encrypted data"
|
|||||||
ERROR_MSG_INVALID_ENCRYPTED_CODE = "Invalid encrypted code"
|
ERROR_MSG_INVALID_ENCRYPTED_CODE = "Invalid encrypted code"
|
||||||
|
|
||||||
|
|
||||||
def account_initialization_required(view: Callable[P, R]) -> Callable[P, R]:
|
def account_initialization_required(view: Callable[P, R]):
|
||||||
@wraps(view)
|
@wraps(view)
|
||||||
def decorated(*args: P.args, **kwargs: P.kwargs) -> R:
|
def decorated(*args: P.args, **kwargs: P.kwargs):
|
||||||
# check account initialization
|
# check account initialization
|
||||||
current_user, _ = current_account_with_tenant()
|
current_user, _ = current_account_with_tenant()
|
||||||
if current_user.status == AccountStatus.UNINITIALIZED:
|
if current_user.status == AccountStatus.UNINITIALIZED:
|
||||||
@@ -214,9 +214,9 @@ def cloud_utm_record(view: Callable[P, R]):
|
|||||||
return decorated
|
return decorated
|
||||||
|
|
||||||
|
|
||||||
def setup_required(view: Callable[P, R]) -> Callable[P, R]:
|
def setup_required(view: Callable[P, R]):
|
||||||
@wraps(view)
|
@wraps(view)
|
||||||
def decorated(*args: P.args, **kwargs: P.kwargs) -> R:
|
def decorated(*args: P.args, **kwargs: P.kwargs):
|
||||||
# check setup
|
# check setup
|
||||||
if (
|
if (
|
||||||
dify_config.EDITION == "SELF_HOSTED"
|
dify_config.EDITION == "SELF_HOSTED"
|
||||||
|
|||||||
@@ -137,7 +137,7 @@ class FilePreviewApi(Resource):
|
|||||||
if args.as_attachment:
|
if args.as_attachment:
|
||||||
encoded_filename = quote(upload_file.name)
|
encoded_filename = quote(upload_file.name)
|
||||||
response.headers["Content-Disposition"] = f"attachment; filename*=UTF-8''{encoded_filename}"
|
response.headers["Content-Disposition"] = f"attachment; filename*=UTF-8''{encoded_filename}"
|
||||||
response.headers["Content-Type"] = "application/octet-stream"
|
response.headers["Content-Type"] = "application/octet-stream"
|
||||||
|
|
||||||
enforce_download_for_html(
|
enforce_download_for_html(
|
||||||
response,
|
response,
|
||||||
|
|||||||
@@ -64,10 +64,6 @@ class ToolFileApi(Resource):
|
|||||||
|
|
||||||
if not stream or not tool_file:
|
if not stream or not tool_file:
|
||||||
raise NotFound("file is not found")
|
raise NotFound("file is not found")
|
||||||
|
|
||||||
except NotFound:
|
|
||||||
raise
|
|
||||||
|
|
||||||
except Exception:
|
except Exception:
|
||||||
raise UnsupportedFileTypeError()
|
raise UnsupportedFileTypeError()
|
||||||
|
|
||||||
|
|||||||
@@ -7,8 +7,8 @@ from pydantic import BaseModel, Field
|
|||||||
from werkzeug.exceptions import Forbidden
|
from werkzeug.exceptions import Forbidden
|
||||||
|
|
||||||
import services
|
import services
|
||||||
|
from core.file.helpers import verify_plugin_file_signature
|
||||||
from core.tools.tool_file_manager import ToolFileManager
|
from core.tools.tool_file_manager import ToolFileManager
|
||||||
from dify_graph.file.helpers import verify_plugin_file_signature
|
|
||||||
from fields.file_fields import FileResponse
|
from fields.file_fields import FileResponse
|
||||||
|
|
||||||
from ..common.errors import (
|
from ..common.errors import (
|
||||||
|
|||||||
@@ -4,6 +4,8 @@ from controllers.console.wraps import setup_required
|
|||||||
from controllers.inner_api import inner_api_ns
|
from controllers.inner_api import inner_api_ns
|
||||||
from controllers.inner_api.plugin.wraps import get_user_tenant, plugin_data
|
from controllers.inner_api.plugin.wraps import get_user_tenant, plugin_data
|
||||||
from controllers.inner_api.wraps import plugin_inner_api_only
|
from controllers.inner_api.wraps import plugin_inner_api_only
|
||||||
|
from core.file.helpers import get_signed_file_url_for_plugin
|
||||||
|
from core.model_runtime.utils.encoders import jsonable_encoder
|
||||||
from core.plugin.backwards_invocation.app import PluginAppBackwardsInvocation
|
from core.plugin.backwards_invocation.app import PluginAppBackwardsInvocation
|
||||||
from core.plugin.backwards_invocation.base import BaseBackwardsInvocationResponse
|
from core.plugin.backwards_invocation.base import BaseBackwardsInvocationResponse
|
||||||
from core.plugin.backwards_invocation.encrypt import PluginEncrypter
|
from core.plugin.backwards_invocation.encrypt import PluginEncrypter
|
||||||
@@ -28,8 +30,6 @@ from core.plugin.entities.request import (
|
|||||||
RequestRequestUploadFile,
|
RequestRequestUploadFile,
|
||||||
)
|
)
|
||||||
from core.tools.entities.tool_entities import ToolProviderType
|
from core.tools.entities.tool_entities import ToolProviderType
|
||||||
from dify_graph.file.helpers import get_signed_file_url_for_plugin
|
|
||||||
from dify_graph.model_runtime.utils.encoders import jsonable_encoder
|
|
||||||
from libs.helper import length_prefixed_response
|
from libs.helper import length_prefixed_response
|
||||||
from models import Account, Tenant
|
from models import Account, Tenant
|
||||||
from models.model import EndUser
|
from models.model import EndUser
|
||||||
|
|||||||
@@ -8,9 +8,9 @@ from sqlalchemy.orm import Session
|
|||||||
from controllers.common.schema import register_schema_model
|
from controllers.common.schema import register_schema_model
|
||||||
from controllers.console.app.mcp_server import AppMCPServerStatus
|
from controllers.console.app.mcp_server import AppMCPServerStatus
|
||||||
from controllers.mcp import mcp_ns
|
from controllers.mcp import mcp_ns
|
||||||
|
from core.app.app_config.entities import VariableEntity
|
||||||
from core.mcp import types as mcp_types
|
from core.mcp import types as mcp_types
|
||||||
from core.mcp.server.streamable_http import handle_mcp_request
|
from core.mcp.server.streamable_http import handle_mcp_request
|
||||||
from dify_graph.variables.input_entities import VariableEntity
|
|
||||||
from extensions.ext_database import db
|
from extensions.ext_database import db
|
||||||
from libs import helper
|
from libs import helper
|
||||||
from models.model import App, AppMCPServer, AppMode, EndUser
|
from models.model import App, AppMCPServer, AppMode, EndUser
|
||||||
|
|||||||
@@ -21,7 +21,7 @@ from controllers.service_api.app.error import (
|
|||||||
)
|
)
|
||||||
from controllers.service_api.wraps import FetchUserArg, WhereisUserArg, validate_app_token
|
from controllers.service_api.wraps import FetchUserArg, WhereisUserArg, validate_app_token
|
||||||
from core.errors.error import ModelCurrentlyNotSupportError, ProviderTokenNotInitError, QuotaExceededError
|
from core.errors.error import ModelCurrentlyNotSupportError, ProviderTokenNotInitError, QuotaExceededError
|
||||||
from dify_graph.model_runtime.errors.invoke import InvokeError
|
from core.model_runtime.errors.invoke import InvokeError
|
||||||
from models.model import App, EndUser
|
from models.model import App, EndUser
|
||||||
from services.audio_service import AudioService
|
from services.audio_service import AudioService
|
||||||
from services.errors.audio import (
|
from services.errors.audio import (
|
||||||
|
|||||||
@@ -28,7 +28,7 @@ from core.errors.error import (
|
|||||||
QuotaExceededError,
|
QuotaExceededError,
|
||||||
)
|
)
|
||||||
from core.helper.trace_id_helper import get_external_trace_id
|
from core.helper.trace_id_helper import get_external_trace_id
|
||||||
from dify_graph.model_runtime.errors.invoke import InvokeError
|
from core.model_runtime.errors.invoke import InvokeError
|
||||||
from libs import helper
|
from libs import helper
|
||||||
from libs.helper import UUIDStrOrEmpty
|
from libs.helper import UUIDStrOrEmpty
|
||||||
from models.model import App, AppMode, EndUser
|
from models.model import App, AppMode, EndUser
|
||||||
|
|||||||
@@ -27,11 +27,10 @@ from core.errors.error import (
|
|||||||
QuotaExceededError,
|
QuotaExceededError,
|
||||||
)
|
)
|
||||||
from core.helper.trace_id_helper import get_external_trace_id
|
from core.helper.trace_id_helper import get_external_trace_id
|
||||||
from dify_graph.enums import WorkflowExecutionStatus
|
from core.model_runtime.errors.invoke import InvokeError
|
||||||
from dify_graph.graph_engine.manager import GraphEngineManager
|
from core.workflow.enums import WorkflowExecutionStatus
|
||||||
from dify_graph.model_runtime.errors.invoke import InvokeError
|
from core.workflow.graph_engine.manager import GraphEngineManager
|
||||||
from extensions.ext_database import db
|
from extensions.ext_database import db
|
||||||
from extensions.ext_redis import redis_client
|
|
||||||
from fields.workflow_app_log_fields import build_workflow_app_log_pagination_model
|
from fields.workflow_app_log_fields import build_workflow_app_log_pagination_model
|
||||||
from libs import helper
|
from libs import helper
|
||||||
from libs.helper import OptionalTimestampField, TimestampField
|
from libs.helper import OptionalTimestampField, TimestampField
|
||||||
@@ -281,7 +280,7 @@ class WorkflowTaskStopApi(Resource):
|
|||||||
AppQueueManager.set_stop_flag_no_user_check(task_id)
|
AppQueueManager.set_stop_flag_no_user_check(task_id)
|
||||||
|
|
||||||
# New graph engine command channel mechanism
|
# New graph engine command channel mechanism
|
||||||
GraphEngineManager(redis_client).send_stop_command(task_id)
|
GraphEngineManager.send_stop_command(task_id)
|
||||||
|
|
||||||
return {"result": "success"}
|
return {"result": "success"}
|
||||||
|
|
||||||
|
|||||||
@@ -14,8 +14,8 @@ from controllers.service_api.wraps import (
|
|||||||
DatasetApiResource,
|
DatasetApiResource,
|
||||||
cloud_edition_billing_rate_limit_check,
|
cloud_edition_billing_rate_limit_check,
|
||||||
)
|
)
|
||||||
|
from core.model_runtime.entities.model_entities import ModelType
|
||||||
from core.provider_manager import ProviderManager
|
from core.provider_manager import ProviderManager
|
||||||
from dify_graph.model_runtime.entities.model_entities import ModelType
|
|
||||||
from fields.dataset_fields import dataset_detail_fields
|
from fields.dataset_fields import dataset_detail_fields
|
||||||
from fields.tag_fields import DataSetTag
|
from fields.tag_fields import DataSetTag
|
||||||
from libs.login import current_user
|
from libs.login import current_user
|
||||||
|
|||||||
@@ -4,7 +4,7 @@ from uuid import UUID
|
|||||||
|
|
||||||
from flask import request
|
from flask import request
|
||||||
from flask_restx import marshal
|
from flask_restx import marshal
|
||||||
from pydantic import BaseModel, Field, field_validator, model_validator
|
from pydantic import BaseModel, Field, model_validator
|
||||||
from sqlalchemy import desc, select
|
from sqlalchemy import desc, select
|
||||||
from werkzeug.exceptions import Forbidden, NotFound
|
from werkzeug.exceptions import Forbidden, NotFound
|
||||||
|
|
||||||
@@ -60,13 +60,6 @@ class DocumentTextCreatePayload(BaseModel):
|
|||||||
embedding_model: str | None = None
|
embedding_model: str | None = None
|
||||||
embedding_model_provider: str | None = None
|
embedding_model_provider: str | None = None
|
||||||
|
|
||||||
@field_validator("doc_form")
|
|
||||||
@classmethod
|
|
||||||
def validate_doc_form(cls, value: str) -> str:
|
|
||||||
if value not in Dataset.DOC_FORM_LIST:
|
|
||||||
raise ValueError("Invalid doc_form.")
|
|
||||||
return value
|
|
||||||
|
|
||||||
|
|
||||||
DEFAULT_REF_TEMPLATE_SWAGGER_2_0 = "#/definitions/{model}"
|
DEFAULT_REF_TEMPLATE_SWAGGER_2_0 = "#/definitions/{model}"
|
||||||
|
|
||||||
@@ -79,13 +72,6 @@ class DocumentTextUpdate(BaseModel):
|
|||||||
doc_language: str = "English"
|
doc_language: str = "English"
|
||||||
retrieval_model: RetrievalModel | None = None
|
retrieval_model: RetrievalModel | None = None
|
||||||
|
|
||||||
@field_validator("doc_form")
|
|
||||||
@classmethod
|
|
||||||
def validate_doc_form(cls, value: str) -> str:
|
|
||||||
if value not in Dataset.DOC_FORM_LIST:
|
|
||||||
raise ValueError("Invalid doc_form.")
|
|
||||||
return value
|
|
||||||
|
|
||||||
@model_validator(mode="after")
|
@model_validator(mode="after")
|
||||||
def check_text_and_name(self) -> Self:
|
def check_text_and_name(self) -> Self:
|
||||||
if self.text is not None and self.name is None:
|
if self.text is not None and self.name is None:
|
||||||
|
|||||||
@@ -3,8 +3,7 @@ from typing import Any
|
|||||||
|
|
||||||
from flask import request
|
from flask import request
|
||||||
from pydantic import BaseModel
|
from pydantic import BaseModel
|
||||||
from sqlalchemy import select
|
from werkzeug.exceptions import Forbidden
|
||||||
from werkzeug.exceptions import Forbidden, NotFound
|
|
||||||
|
|
||||||
import services
|
import services
|
||||||
from controllers.common.errors import FilenameNotExistsError, NoFileUploadedError, TooManyFilesError
|
from controllers.common.errors import FilenameNotExistsError, NoFileUploadedError, TooManyFilesError
|
||||||
@@ -18,7 +17,7 @@ from core.app.entities.app_invoke_entities import InvokeFrom
|
|||||||
from libs import helper
|
from libs import helper
|
||||||
from libs.login import current_user
|
from libs.login import current_user
|
||||||
from models import Account
|
from models import Account
|
||||||
from models.dataset import Dataset, Pipeline
|
from models.dataset import Pipeline
|
||||||
from models.engine import db
|
from models.engine import db
|
||||||
from services.errors.file import FileTooLargeError, UnsupportedFileTypeError
|
from services.errors.file import FileTooLargeError, UnsupportedFileTypeError
|
||||||
from services.file_service import FileService
|
from services.file_service import FileService
|
||||||
@@ -66,12 +65,6 @@ class DatasourcePluginsApi(DatasetApiResource):
|
|||||||
)
|
)
|
||||||
def get(self, tenant_id: str, dataset_id: str):
|
def get(self, tenant_id: str, dataset_id: str):
|
||||||
"""Resource for getting datasource plugins."""
|
"""Resource for getting datasource plugins."""
|
||||||
# Verify dataset ownership
|
|
||||||
stmt = select(Dataset).where(Dataset.tenant_id == tenant_id, Dataset.id == dataset_id)
|
|
||||||
dataset = db.session.scalar(stmt)
|
|
||||||
if not dataset:
|
|
||||||
raise NotFound("Dataset not found.")
|
|
||||||
|
|
||||||
# Get query parameter to determine published or draft
|
# Get query parameter to determine published or draft
|
||||||
is_published: bool = request.args.get("is_published", default=True, type=bool)
|
is_published: bool = request.args.get("is_published", default=True, type=bool)
|
||||||
|
|
||||||
@@ -111,12 +104,6 @@ class DatasourceNodeRunApi(DatasetApiResource):
|
|||||||
@service_api_ns.expect(service_api_ns.models[DatasourceNodeRunPayload.__name__])
|
@service_api_ns.expect(service_api_ns.models[DatasourceNodeRunPayload.__name__])
|
||||||
def post(self, tenant_id: str, dataset_id: str, node_id: str):
|
def post(self, tenant_id: str, dataset_id: str, node_id: str):
|
||||||
"""Resource for getting datasource plugins."""
|
"""Resource for getting datasource plugins."""
|
||||||
# Verify dataset ownership
|
|
||||||
stmt = select(Dataset).where(Dataset.tenant_id == tenant_id, Dataset.id == dataset_id)
|
|
||||||
dataset = db.session.scalar(stmt)
|
|
||||||
if not dataset:
|
|
||||||
raise NotFound("Dataset not found.")
|
|
||||||
|
|
||||||
payload = DatasourceNodeRunPayload.model_validate(service_api_ns.payload or {})
|
payload = DatasourceNodeRunPayload.model_validate(service_api_ns.payload or {})
|
||||||
assert isinstance(current_user, Account)
|
assert isinstance(current_user, Account)
|
||||||
rag_pipeline_service: RagPipelineService = RagPipelineService()
|
rag_pipeline_service: RagPipelineService = RagPipelineService()
|
||||||
@@ -174,12 +161,6 @@ class PipelineRunApi(DatasetApiResource):
|
|||||||
@service_api_ns.expect(service_api_ns.models[PipelineRunApiEntity.__name__])
|
@service_api_ns.expect(service_api_ns.models[PipelineRunApiEntity.__name__])
|
||||||
def post(self, tenant_id: str, dataset_id: str):
|
def post(self, tenant_id: str, dataset_id: str):
|
||||||
"""Resource for running a rag pipeline."""
|
"""Resource for running a rag pipeline."""
|
||||||
# Verify dataset ownership
|
|
||||||
stmt = select(Dataset).where(Dataset.tenant_id == tenant_id, Dataset.id == dataset_id)
|
|
||||||
dataset = db.session.scalar(stmt)
|
|
||||||
if not dataset:
|
|
||||||
raise NotFound("Dataset not found.")
|
|
||||||
|
|
||||||
payload = PipelineRunApiEntity.model_validate(service_api_ns.payload or {})
|
payload = PipelineRunApiEntity.model_validate(service_api_ns.payload or {})
|
||||||
|
|
||||||
if not isinstance(current_user, Account):
|
if not isinstance(current_user, Account):
|
||||||
|
|||||||
@@ -17,7 +17,7 @@ from controllers.service_api.wraps import (
|
|||||||
)
|
)
|
||||||
from core.errors.error import LLMBadRequestError, ProviderTokenNotInitError
|
from core.errors.error import LLMBadRequestError, ProviderTokenNotInitError
|
||||||
from core.model_manager import ModelManager
|
from core.model_manager import ModelManager
|
||||||
from dify_graph.model_runtime.entities.model_entities import ModelType
|
from core.model_runtime.entities.model_entities import ModelType
|
||||||
from extensions.ext_database import db
|
from extensions.ext_database import db
|
||||||
from fields.segment_fields import child_chunk_fields, segment_fields
|
from fields.segment_fields import child_chunk_fields, segment_fields
|
||||||
from libs.login import current_account_with_tenant
|
from libs.login import current_account_with_tenant
|
||||||
|
|||||||
@@ -3,7 +3,7 @@ from flask_restx import Resource
|
|||||||
|
|
||||||
from controllers.service_api import service_api_ns
|
from controllers.service_api import service_api_ns
|
||||||
from controllers.service_api.wraps import validate_dataset_token
|
from controllers.service_api.wraps import validate_dataset_token
|
||||||
from dify_graph.model_runtime.utils.encoders import jsonable_encoder
|
from core.model_runtime.utils.encoders import jsonable_encoder
|
||||||
from services.model_provider_service import ModelProviderService
|
from services.model_provider_service import ModelProviderService
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
@@ -20,7 +20,7 @@ from controllers.web.error import (
|
|||||||
)
|
)
|
||||||
from controllers.web.wraps import WebApiResource
|
from controllers.web.wraps import WebApiResource
|
||||||
from core.errors.error import ModelCurrentlyNotSupportError, ProviderTokenNotInitError, QuotaExceededError
|
from core.errors.error import ModelCurrentlyNotSupportError, ProviderTokenNotInitError, QuotaExceededError
|
||||||
from dify_graph.model_runtime.errors.invoke import InvokeError
|
from core.model_runtime.errors.invoke import InvokeError
|
||||||
from libs.helper import uuid_value
|
from libs.helper import uuid_value
|
||||||
from models.model import App
|
from models.model import App
|
||||||
from services.audio_service import AudioService
|
from services.audio_service import AudioService
|
||||||
|
|||||||
@@ -25,7 +25,7 @@ from core.errors.error import (
|
|||||||
ProviderTokenNotInitError,
|
ProviderTokenNotInitError,
|
||||||
QuotaExceededError,
|
QuotaExceededError,
|
||||||
)
|
)
|
||||||
from dify_graph.model_runtime.errors.invoke import InvokeError
|
from core.model_runtime.errors.invoke import InvokeError
|
||||||
from libs import helper
|
from libs import helper
|
||||||
from libs.helper import uuid_value
|
from libs.helper import uuid_value
|
||||||
from models.model import AppMode
|
from models.model import AppMode
|
||||||
|
|||||||
@@ -20,7 +20,7 @@ from controllers.web.error import (
|
|||||||
from controllers.web.wraps import WebApiResource
|
from controllers.web.wraps import WebApiResource
|
||||||
from core.app.entities.app_invoke_entities import InvokeFrom
|
from core.app.entities.app_invoke_entities import InvokeFrom
|
||||||
from core.errors.error import ModelCurrentlyNotSupportError, ProviderTokenNotInitError, QuotaExceededError
|
from core.errors.error import ModelCurrentlyNotSupportError, ProviderTokenNotInitError, QuotaExceededError
|
||||||
from dify_graph.model_runtime.errors.invoke import InvokeError
|
from core.model_runtime.errors.invoke import InvokeError
|
||||||
from fields.conversation_fields import ResultResponse
|
from fields.conversation_fields import ResultResponse
|
||||||
from fields.message_fields import SuggestedQuestionsResponse, WebMessageInfiniteScrollPagination, WebMessageListItem
|
from fields.message_fields import SuggestedQuestionsResponse, WebMessageInfiniteScrollPagination, WebMessageListItem
|
||||||
from libs import helper
|
from libs import helper
|
||||||
|
|||||||
@@ -10,8 +10,8 @@ from controllers.common.errors import (
|
|||||||
RemoteFileUploadError,
|
RemoteFileUploadError,
|
||||||
UnsupportedFileTypeError,
|
UnsupportedFileTypeError,
|
||||||
)
|
)
|
||||||
|
from core.file import helpers as file_helpers
|
||||||
from core.helper import ssrf_proxy
|
from core.helper import ssrf_proxy
|
||||||
from dify_graph.file import helpers as file_helpers
|
|
||||||
from extensions.ext_database import db
|
from extensions.ext_database import db
|
||||||
from fields.file_fields import FileWithSignedUrl, RemoteFileInfo
|
from fields.file_fields import FileWithSignedUrl, RemoteFileInfo
|
||||||
from services.file_service import FileService
|
from services.file_service import FileService
|
||||||
|
|||||||
@@ -22,9 +22,8 @@ from core.errors.error import (
|
|||||||
ProviderTokenNotInitError,
|
ProviderTokenNotInitError,
|
||||||
QuotaExceededError,
|
QuotaExceededError,
|
||||||
)
|
)
|
||||||
from dify_graph.graph_engine.manager import GraphEngineManager
|
from core.model_runtime.errors.invoke import InvokeError
|
||||||
from dify_graph.model_runtime.errors.invoke import InvokeError
|
from core.workflow.graph_engine.manager import GraphEngineManager
|
||||||
from extensions.ext_redis import redis_client
|
|
||||||
from libs import helper
|
from libs import helper
|
||||||
from models.model import App, AppMode, EndUser
|
from models.model import App, AppMode, EndUser
|
||||||
from services.app_generate_service import AppGenerateService
|
from services.app_generate_service import AppGenerateService
|
||||||
@@ -122,6 +121,6 @@ class WorkflowTaskStopApi(WebApiResource):
|
|||||||
AppQueueManager.set_stop_flag_no_user_check(task_id)
|
AppQueueManager.set_stop_flag_no_user_check(task_id)
|
||||||
|
|
||||||
# New graph engine command channel mechanism
|
# New graph engine command channel mechanism
|
||||||
GraphEngineManager(redis_client).send_stop_command(task_id)
|
GraphEngineManager.send_stop_command(task_id)
|
||||||
|
|
||||||
return {"result": "success"}
|
return {"result": "success"}
|
||||||
|
|||||||
@@ -17,17 +17,10 @@ from core.app.entities.app_invoke_entities import (
|
|||||||
)
|
)
|
||||||
from core.callback_handler.agent_tool_callback_handler import DifyAgentCallbackHandler
|
from core.callback_handler.agent_tool_callback_handler import DifyAgentCallbackHandler
|
||||||
from core.callback_handler.index_tool_callback_handler import DatasetIndexToolCallbackHandler
|
from core.callback_handler.index_tool_callback_handler import DatasetIndexToolCallbackHandler
|
||||||
|
from core.file import file_manager
|
||||||
from core.memory.token_buffer_memory import TokenBufferMemory
|
from core.memory.token_buffer_memory import TokenBufferMemory
|
||||||
from core.model_manager import ModelInstance
|
from core.model_manager import ModelInstance
|
||||||
from core.prompt.utils.extract_thread_messages import extract_thread_messages
|
from core.model_runtime.entities import (
|
||||||
from core.tools.__base.tool import Tool
|
|
||||||
from core.tools.entities.tool_entities import (
|
|
||||||
ToolParameter,
|
|
||||||
)
|
|
||||||
from core.tools.tool_manager import ToolManager
|
|
||||||
from core.tools.utils.dataset_retriever_tool import DatasetRetrieverTool
|
|
||||||
from dify_graph.file import file_manager
|
|
||||||
from dify_graph.model_runtime.entities import (
|
|
||||||
AssistantPromptMessage,
|
AssistantPromptMessage,
|
||||||
LLMUsage,
|
LLMUsage,
|
||||||
PromptMessage,
|
PromptMessage,
|
||||||
@@ -37,9 +30,16 @@ from dify_graph.model_runtime.entities import (
|
|||||||
ToolPromptMessage,
|
ToolPromptMessage,
|
||||||
UserPromptMessage,
|
UserPromptMessage,
|
||||||
)
|
)
|
||||||
from dify_graph.model_runtime.entities.message_entities import ImagePromptMessageContent, PromptMessageContentUnionTypes
|
from core.model_runtime.entities.message_entities import ImagePromptMessageContent, PromptMessageContentUnionTypes
|
||||||
from dify_graph.model_runtime.entities.model_entities import ModelFeature
|
from core.model_runtime.entities.model_entities import ModelFeature
|
||||||
from dify_graph.model_runtime.model_providers.__base.large_language_model import LargeLanguageModel
|
from core.model_runtime.model_providers.__base.large_language_model import LargeLanguageModel
|
||||||
|
from core.prompt.utils.extract_thread_messages import extract_thread_messages
|
||||||
|
from core.tools.__base.tool import Tool
|
||||||
|
from core.tools.entities.tool_entities import (
|
||||||
|
ToolParameter,
|
||||||
|
)
|
||||||
|
from core.tools.tool_manager import ToolManager
|
||||||
|
from core.tools.utils.dataset_retriever_tool import DatasetRetrieverTool
|
||||||
from extensions.ext_database import db
|
from extensions.ext_database import db
|
||||||
from factories import file_factory
|
from factories import file_factory
|
||||||
from models.enums import CreatorUserRole
|
from models.enums import CreatorUserRole
|
||||||
@@ -112,7 +112,7 @@ class BaseAgentRunner(AppRunner):
|
|||||||
|
|
||||||
# check if model supports stream tool call
|
# check if model supports stream tool call
|
||||||
llm_model = cast(LargeLanguageModel, model_instance.model_type_instance)
|
llm_model = cast(LargeLanguageModel, model_instance.model_type_instance)
|
||||||
model_schema = llm_model.get_model_schema(model_instance.model_name, model_instance.credentials)
|
model_schema = llm_model.get_model_schema(model_instance.model, model_instance.credentials)
|
||||||
features = model_schema.features if model_schema and model_schema.features else []
|
features = model_schema.features if model_schema and model_schema.features else []
|
||||||
self.stream_tool_call = ModelFeature.STREAM_TOOL_CALL in features
|
self.stream_tool_call = ModelFeature.STREAM_TOOL_CALL in features
|
||||||
self.files = application_generate_entity.files if ModelFeature.VISION in features else []
|
self.files = application_generate_entity.files if ModelFeature.VISION in features else []
|
||||||
|
|||||||
@@ -9,20 +9,20 @@ from core.agent.entities import AgentScratchpadUnit
|
|||||||
from core.agent.output_parser.cot_output_parser import CotAgentOutputParser
|
from core.agent.output_parser.cot_output_parser import CotAgentOutputParser
|
||||||
from core.app.apps.base_app_queue_manager import PublishFrom
|
from core.app.apps.base_app_queue_manager import PublishFrom
|
||||||
from core.app.entities.queue_entities import QueueAgentThoughtEvent, QueueMessageEndEvent, QueueMessageFileEvent
|
from core.app.entities.queue_entities import QueueAgentThoughtEvent, QueueMessageEndEvent, QueueMessageFileEvent
|
||||||
from core.ops.ops_trace_manager import TraceQueueManager
|
from core.model_runtime.entities.llm_entities import LLMResult, LLMResultChunk, LLMResultChunkDelta, LLMUsage
|
||||||
from core.prompt.agent_history_prompt_transform import AgentHistoryPromptTransform
|
from core.model_runtime.entities.message_entities import (
|
||||||
from core.tools.__base.tool import Tool
|
|
||||||
from core.tools.entities.tool_entities import ToolInvokeMeta
|
|
||||||
from core.tools.tool_engine import ToolEngine
|
|
||||||
from dify_graph.model_runtime.entities.llm_entities import LLMResult, LLMResultChunk, LLMResultChunkDelta, LLMUsage
|
|
||||||
from dify_graph.model_runtime.entities.message_entities import (
|
|
||||||
AssistantPromptMessage,
|
AssistantPromptMessage,
|
||||||
PromptMessage,
|
PromptMessage,
|
||||||
PromptMessageTool,
|
PromptMessageTool,
|
||||||
ToolPromptMessage,
|
ToolPromptMessage,
|
||||||
UserPromptMessage,
|
UserPromptMessage,
|
||||||
)
|
)
|
||||||
from dify_graph.nodes.agent.exc import AgentMaxIterationError
|
from core.ops.ops_trace_manager import TraceQueueManager
|
||||||
|
from core.prompt.agent_history_prompt_transform import AgentHistoryPromptTransform
|
||||||
|
from core.tools.__base.tool import Tool
|
||||||
|
from core.tools.entities.tool_entities import ToolInvokeMeta
|
||||||
|
from core.tools.tool_engine import ToolEngine
|
||||||
|
from core.workflow.nodes.agent.exc import AgentMaxIterationError
|
||||||
from models.model import Message
|
from models.model import Message
|
||||||
|
|
||||||
logger = logging.getLogger(__name__)
|
logger = logging.getLogger(__name__)
|
||||||
@@ -245,7 +245,7 @@ class CotAgentRunner(BaseAgentRunner, ABC):
|
|||||||
iteration_step += 1
|
iteration_step += 1
|
||||||
|
|
||||||
yield LLMResultChunk(
|
yield LLMResultChunk(
|
||||||
model=model_instance.model_name,
|
model=model_instance.model,
|
||||||
prompt_messages=prompt_messages,
|
prompt_messages=prompt_messages,
|
||||||
delta=LLMResultChunkDelta(
|
delta=LLMResultChunkDelta(
|
||||||
index=0, message=AssistantPromptMessage(content=final_answer), usage=llm_usage["usage"]
|
index=0, message=AssistantPromptMessage(content=final_answer), usage=llm_usage["usage"]
|
||||||
@@ -268,7 +268,7 @@ class CotAgentRunner(BaseAgentRunner, ABC):
|
|||||||
self.queue_manager.publish(
|
self.queue_manager.publish(
|
||||||
QueueMessageEndEvent(
|
QueueMessageEndEvent(
|
||||||
llm_result=LLMResult(
|
llm_result=LLMResult(
|
||||||
model=model_instance.model_name,
|
model=model_instance.model,
|
||||||
prompt_messages=prompt_messages,
|
prompt_messages=prompt_messages,
|
||||||
message=AssistantPromptMessage(content=final_answer),
|
message=AssistantPromptMessage(content=final_answer),
|
||||||
usage=llm_usage["usage"] or LLMUsage.empty_usage(),
|
usage=llm_usage["usage"] or LLMUsage.empty_usage(),
|
||||||
|
|||||||
@@ -1,16 +1,16 @@
|
|||||||
import json
|
import json
|
||||||
|
|
||||||
from core.agent.cot_agent_runner import CotAgentRunner
|
from core.agent.cot_agent_runner import CotAgentRunner
|
||||||
from dify_graph.file import file_manager
|
from core.file import file_manager
|
||||||
from dify_graph.model_runtime.entities import (
|
from core.model_runtime.entities import (
|
||||||
AssistantPromptMessage,
|
AssistantPromptMessage,
|
||||||
PromptMessage,
|
PromptMessage,
|
||||||
SystemPromptMessage,
|
SystemPromptMessage,
|
||||||
TextPromptMessageContent,
|
TextPromptMessageContent,
|
||||||
UserPromptMessage,
|
UserPromptMessage,
|
||||||
)
|
)
|
||||||
from dify_graph.model_runtime.entities.message_entities import ImagePromptMessageContent, PromptMessageContentUnionTypes
|
from core.model_runtime.entities.message_entities import ImagePromptMessageContent, PromptMessageContentUnionTypes
|
||||||
from dify_graph.model_runtime.utils.encoders import jsonable_encoder
|
from core.model_runtime.utils.encoders import jsonable_encoder
|
||||||
|
|
||||||
|
|
||||||
class CotChatAgentRunner(CotAgentRunner):
|
class CotChatAgentRunner(CotAgentRunner):
|
||||||
|
|||||||
@@ -1,13 +1,13 @@
|
|||||||
import json
|
import json
|
||||||
|
|
||||||
from core.agent.cot_agent_runner import CotAgentRunner
|
from core.agent.cot_agent_runner import CotAgentRunner
|
||||||
from dify_graph.model_runtime.entities.message_entities import (
|
from core.model_runtime.entities.message_entities import (
|
||||||
AssistantPromptMessage,
|
AssistantPromptMessage,
|
||||||
PromptMessage,
|
PromptMessage,
|
||||||
TextPromptMessageContent,
|
TextPromptMessageContent,
|
||||||
UserPromptMessage,
|
UserPromptMessage,
|
||||||
)
|
)
|
||||||
from dify_graph.model_runtime.utils.encoders import jsonable_encoder
|
from core.model_runtime.utils.encoders import jsonable_encoder
|
||||||
|
|
||||||
|
|
||||||
class CotCompletionAgentRunner(CotAgentRunner):
|
class CotCompletionAgentRunner(CotAgentRunner):
|
||||||
|
|||||||
@@ -7,11 +7,8 @@ from typing import Any, Union
|
|||||||
from core.agent.base_agent_runner import BaseAgentRunner
|
from core.agent.base_agent_runner import BaseAgentRunner
|
||||||
from core.app.apps.base_app_queue_manager import PublishFrom
|
from core.app.apps.base_app_queue_manager import PublishFrom
|
||||||
from core.app.entities.queue_entities import QueueAgentThoughtEvent, QueueMessageEndEvent, QueueMessageFileEvent
|
from core.app.entities.queue_entities import QueueAgentThoughtEvent, QueueMessageEndEvent, QueueMessageFileEvent
|
||||||
from core.prompt.agent_history_prompt_transform import AgentHistoryPromptTransform
|
from core.file import file_manager
|
||||||
from core.tools.entities.tool_entities import ToolInvokeMeta
|
from core.model_runtime.entities import (
|
||||||
from core.tools.tool_engine import ToolEngine
|
|
||||||
from dify_graph.file import file_manager
|
|
||||||
from dify_graph.model_runtime.entities import (
|
|
||||||
AssistantPromptMessage,
|
AssistantPromptMessage,
|
||||||
LLMResult,
|
LLMResult,
|
||||||
LLMResultChunk,
|
LLMResultChunk,
|
||||||
@@ -24,8 +21,11 @@ from dify_graph.model_runtime.entities import (
|
|||||||
ToolPromptMessage,
|
ToolPromptMessage,
|
||||||
UserPromptMessage,
|
UserPromptMessage,
|
||||||
)
|
)
|
||||||
from dify_graph.model_runtime.entities.message_entities import ImagePromptMessageContent, PromptMessageContentUnionTypes
|
from core.model_runtime.entities.message_entities import ImagePromptMessageContent, PromptMessageContentUnionTypes
|
||||||
from dify_graph.nodes.agent.exc import AgentMaxIterationError
|
from core.prompt.agent_history_prompt_transform import AgentHistoryPromptTransform
|
||||||
|
from core.tools.entities.tool_entities import ToolInvokeMeta
|
||||||
|
from core.tools.tool_engine import ToolEngine
|
||||||
|
from core.workflow.nodes.agent.exc import AgentMaxIterationError
|
||||||
from models.model import Message
|
from models.model import Message
|
||||||
|
|
||||||
logger = logging.getLogger(__name__)
|
logger = logging.getLogger(__name__)
|
||||||
@@ -178,7 +178,7 @@ class FunctionCallAgentRunner(BaseAgentRunner):
|
|||||||
)
|
)
|
||||||
|
|
||||||
yield LLMResultChunk(
|
yield LLMResultChunk(
|
||||||
model=model_instance.model_name,
|
model=model_instance.model,
|
||||||
prompt_messages=result.prompt_messages,
|
prompt_messages=result.prompt_messages,
|
||||||
system_fingerprint=result.system_fingerprint,
|
system_fingerprint=result.system_fingerprint,
|
||||||
delta=LLMResultChunkDelta(
|
delta=LLMResultChunkDelta(
|
||||||
@@ -308,7 +308,7 @@ class FunctionCallAgentRunner(BaseAgentRunner):
|
|||||||
self.queue_manager.publish(
|
self.queue_manager.publish(
|
||||||
QueueMessageEndEvent(
|
QueueMessageEndEvent(
|
||||||
llm_result=LLMResult(
|
llm_result=LLMResult(
|
||||||
model=model_instance.model_name,
|
model=model_instance.model,
|
||||||
prompt_messages=prompt_messages,
|
prompt_messages=prompt_messages,
|
||||||
message=AssistantPromptMessage(content=final_answer),
|
message=AssistantPromptMessage(content=final_answer),
|
||||||
usage=llm_usage["usage"] or LLMUsage.empty_usage(),
|
usage=llm_usage["usage"] or LLMUsage.empty_usage(),
|
||||||
|
|||||||
@@ -4,7 +4,7 @@ from collections.abc import Generator
|
|||||||
from typing import Union
|
from typing import Union
|
||||||
|
|
||||||
from core.agent.entities import AgentScratchpadUnit
|
from core.agent.entities import AgentScratchpadUnit
|
||||||
from dify_graph.model_runtime.entities.llm_entities import LLMResultChunk
|
from core.model_runtime.entities.llm_entities import LLMResultChunk
|
||||||
|
|
||||||
|
|
||||||
class CotAgentOutputParser:
|
class CotAgentOutputParser:
|
||||||
|
|||||||
@@ -4,10 +4,10 @@ from core.app.app_config.entities import EasyUIBasedAppConfig
|
|||||||
from core.app.entities.app_invoke_entities import ModelConfigWithCredentialsEntity
|
from core.app.entities.app_invoke_entities import ModelConfigWithCredentialsEntity
|
||||||
from core.entities.model_entities import ModelStatus
|
from core.entities.model_entities import ModelStatus
|
||||||
from core.errors.error import ModelCurrentlyNotSupportError, ProviderTokenNotInitError, QuotaExceededError
|
from core.errors.error import ModelCurrentlyNotSupportError, ProviderTokenNotInitError, QuotaExceededError
|
||||||
|
from core.model_runtime.entities.llm_entities import LLMMode
|
||||||
|
from core.model_runtime.entities.model_entities import ModelPropertyKey, ModelType
|
||||||
|
from core.model_runtime.model_providers.__base.large_language_model import LargeLanguageModel
|
||||||
from core.provider_manager import ProviderManager
|
from core.provider_manager import ProviderManager
|
||||||
from dify_graph.model_runtime.entities.llm_entities import LLMMode
|
|
||||||
from dify_graph.model_runtime.entities.model_entities import ModelPropertyKey, ModelType
|
|
||||||
from dify_graph.model_runtime.model_providers.__base.large_language_model import LargeLanguageModel
|
|
||||||
|
|
||||||
|
|
||||||
class ModelConfigConverter:
|
class ModelConfigConverter:
|
||||||
|
|||||||
@@ -2,9 +2,9 @@ from collections.abc import Mapping
|
|||||||
from typing import Any
|
from typing import Any
|
||||||
|
|
||||||
from core.app.app_config.entities import ModelConfigEntity
|
from core.app.app_config.entities import ModelConfigEntity
|
||||||
|
from core.model_runtime.entities.model_entities import ModelPropertyKey, ModelType
|
||||||
|
from core.model_runtime.model_providers.model_provider_factory import ModelProviderFactory
|
||||||
from core.provider_manager import ProviderManager
|
from core.provider_manager import ProviderManager
|
||||||
from dify_graph.model_runtime.entities.model_entities import ModelPropertyKey, ModelType
|
|
||||||
from dify_graph.model_runtime.model_providers.model_provider_factory import ModelProviderFactory
|
|
||||||
from models.provider_ids import ModelProviderID
|
from models.provider_ids import ModelProviderID
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
@@ -4,8 +4,8 @@ from core.app.app_config.entities import (
|
|||||||
AdvancedCompletionPromptTemplateEntity,
|
AdvancedCompletionPromptTemplateEntity,
|
||||||
PromptTemplateEntity,
|
PromptTemplateEntity,
|
||||||
)
|
)
|
||||||
|
from core.model_runtime.entities.message_entities import PromptMessageRole
|
||||||
from core.prompt.simple_prompt_transform import ModelMode
|
from core.prompt.simple_prompt_transform import ModelMode
|
||||||
from dify_graph.model_runtime.entities.message_entities import PromptMessageRole
|
|
||||||
from models.model import AppMode
|
from models.model import AppMode
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
@@ -1,8 +1,7 @@
|
|||||||
import re
|
import re
|
||||||
|
|
||||||
from core.app.app_config.entities import ExternalDataVariableEntity
|
from core.app.app_config.entities import ExternalDataVariableEntity, VariableEntity, VariableEntityType
|
||||||
from core.external_data_tool.factory import ExternalDataToolFactory
|
from core.external_data_tool.factory import ExternalDataToolFactory
|
||||||
from dify_graph.variables.input_entities import VariableEntity, VariableEntityType
|
|
||||||
|
|
||||||
_ALLOWED_VARIABLE_ENTITY_TYPE = frozenset(
|
_ALLOWED_VARIABLE_ENTITY_TYPE = frozenset(
|
||||||
[
|
[
|
||||||
|
|||||||
@@ -2,12 +2,12 @@ from collections.abc import Sequence
|
|||||||
from enum import StrEnum, auto
|
from enum import StrEnum, auto
|
||||||
from typing import Any, Literal
|
from typing import Any, Literal
|
||||||
|
|
||||||
from pydantic import BaseModel, Field
|
from jsonschema import Draft7Validator, SchemaError
|
||||||
|
from pydantic import BaseModel, Field, field_validator
|
||||||
|
|
||||||
from dify_graph.file import FileUploadConfig
|
from core.file import FileTransferMethod, FileType, FileUploadConfig
|
||||||
from dify_graph.model_runtime.entities.llm_entities import LLMMode
|
from core.model_runtime.entities.llm_entities import LLMMode
|
||||||
from dify_graph.model_runtime.entities.message_entities import PromptMessageRole
|
from core.model_runtime.entities.message_entities import PromptMessageRole
|
||||||
from dify_graph.variables.input_entities import VariableEntity as WorkflowVariableEntity
|
|
||||||
from models.model import AppMode
|
from models.model import AppMode
|
||||||
|
|
||||||
|
|
||||||
@@ -90,7 +90,61 @@ class PromptTemplateEntity(BaseModel):
|
|||||||
advanced_completion_prompt_template: AdvancedCompletionPromptTemplateEntity | None = None
|
advanced_completion_prompt_template: AdvancedCompletionPromptTemplateEntity | None = None
|
||||||
|
|
||||||
|
|
||||||
class RagPipelineVariableEntity(WorkflowVariableEntity):
|
class VariableEntityType(StrEnum):
|
||||||
|
TEXT_INPUT = "text-input"
|
||||||
|
SELECT = "select"
|
||||||
|
PARAGRAPH = "paragraph"
|
||||||
|
NUMBER = "number"
|
||||||
|
EXTERNAL_DATA_TOOL = "external_data_tool"
|
||||||
|
FILE = "file"
|
||||||
|
FILE_LIST = "file-list"
|
||||||
|
CHECKBOX = "checkbox"
|
||||||
|
JSON_OBJECT = "json_object"
|
||||||
|
|
||||||
|
|
||||||
|
class VariableEntity(BaseModel):
|
||||||
|
"""
|
||||||
|
Variable Entity.
|
||||||
|
"""
|
||||||
|
|
||||||
|
# `variable` records the name of the variable in user inputs.
|
||||||
|
variable: str
|
||||||
|
label: str
|
||||||
|
description: str = ""
|
||||||
|
type: VariableEntityType
|
||||||
|
required: bool = False
|
||||||
|
hide: bool = False
|
||||||
|
default: Any = None
|
||||||
|
max_length: int | None = None
|
||||||
|
options: Sequence[str] = Field(default_factory=list)
|
||||||
|
allowed_file_types: Sequence[FileType] | None = Field(default_factory=list)
|
||||||
|
allowed_file_extensions: Sequence[str] | None = Field(default_factory=list)
|
||||||
|
allowed_file_upload_methods: Sequence[FileTransferMethod] | None = Field(default_factory=list)
|
||||||
|
json_schema: dict | None = Field(default=None)
|
||||||
|
|
||||||
|
@field_validator("description", mode="before")
|
||||||
|
@classmethod
|
||||||
|
def convert_none_description(cls, v: Any) -> str:
|
||||||
|
return v or ""
|
||||||
|
|
||||||
|
@field_validator("options", mode="before")
|
||||||
|
@classmethod
|
||||||
|
def convert_none_options(cls, v: Any) -> Sequence[str]:
|
||||||
|
return v or []
|
||||||
|
|
||||||
|
@field_validator("json_schema")
|
||||||
|
@classmethod
|
||||||
|
def validate_json_schema(cls, schema: dict | None) -> dict | None:
|
||||||
|
if schema is None:
|
||||||
|
return None
|
||||||
|
try:
|
||||||
|
Draft7Validator.check_schema(schema)
|
||||||
|
except SchemaError as e:
|
||||||
|
raise ValueError(f"Invalid JSON schema: {e.message}")
|
||||||
|
return schema
|
||||||
|
|
||||||
|
|
||||||
|
class RagPipelineVariableEntity(VariableEntity):
|
||||||
"""
|
"""
|
||||||
Rag Pipeline Variable Entity.
|
Rag Pipeline Variable Entity.
|
||||||
"""
|
"""
|
||||||
@@ -260,7 +314,7 @@ class AppConfig(BaseModel):
|
|||||||
app_id: str
|
app_id: str
|
||||||
app_mode: AppMode
|
app_mode: AppMode
|
||||||
additional_features: AppAdditionalFeatures | None = None
|
additional_features: AppAdditionalFeatures | None = None
|
||||||
variables: list[WorkflowVariableEntity] = []
|
variables: list[VariableEntity] = []
|
||||||
sensitive_word_avoidance: SensitiveWordAvoidanceEntity | None = None
|
sensitive_word_avoidance: SensitiveWordAvoidanceEntity | None = None
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
@@ -2,7 +2,7 @@ from collections.abc import Mapping
|
|||||||
from typing import Any
|
from typing import Any
|
||||||
|
|
||||||
from constants import DEFAULT_FILE_NUMBER_LIMITS
|
from constants import DEFAULT_FILE_NUMBER_LIMITS
|
||||||
from dify_graph.file import FileUploadConfig
|
from core.file import FileUploadConfig
|
||||||
|
|
||||||
|
|
||||||
class FileUploadConfigManager:
|
class FileUploadConfigManager:
|
||||||
|
|||||||
@@ -1,7 +1,6 @@
|
|||||||
import re
|
import re
|
||||||
|
|
||||||
from core.app.app_config.entities import RagPipelineVariableEntity
|
from core.app.app_config.entities import RagPipelineVariableEntity, VariableEntity
|
||||||
from dify_graph.variables.input_entities import VariableEntity
|
|
||||||
from models.workflow import Workflow
|
from models.workflow import Workflow
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
@@ -31,18 +31,18 @@ from core.app.entities.app_invoke_entities import AdvancedChatAppGenerateEntity,
|
|||||||
from core.app.entities.task_entities import ChatbotAppBlockingResponse, ChatbotAppStreamResponse
|
from core.app.entities.task_entities import ChatbotAppBlockingResponse, ChatbotAppStreamResponse
|
||||||
from core.app.layers.pause_state_persist_layer import PauseStateLayerConfig, PauseStatePersistenceLayer
|
from core.app.layers.pause_state_persist_layer import PauseStateLayerConfig, PauseStatePersistenceLayer
|
||||||
from core.helper.trace_id_helper import extract_external_trace_id_from_args
|
from core.helper.trace_id_helper import extract_external_trace_id_from_args
|
||||||
|
from core.model_runtime.errors.invoke import InvokeAuthorizationError
|
||||||
from core.ops.ops_trace_manager import TraceQueueManager
|
from core.ops.ops_trace_manager import TraceQueueManager
|
||||||
from core.prompt.utils.get_thread_messages_length import get_thread_messages_length
|
from core.prompt.utils.get_thread_messages_length import get_thread_messages_length
|
||||||
from core.repositories import DifyCoreRepositoryFactory
|
from core.repositories import DifyCoreRepositoryFactory
|
||||||
from dify_graph.graph_engine.layers.base import GraphEngineLayer
|
from core.workflow.graph_engine.layers.base import GraphEngineLayer
|
||||||
from dify_graph.model_runtime.errors.invoke import InvokeAuthorizationError
|
from core.workflow.repositories.draft_variable_repository import (
|
||||||
from dify_graph.repositories.draft_variable_repository import (
|
|
||||||
DraftVariableSaverFactory,
|
DraftVariableSaverFactory,
|
||||||
)
|
)
|
||||||
from dify_graph.repositories.workflow_execution_repository import WorkflowExecutionRepository
|
from core.workflow.repositories.workflow_execution_repository import WorkflowExecutionRepository
|
||||||
from dify_graph.repositories.workflow_node_execution_repository import WorkflowNodeExecutionRepository
|
from core.workflow.repositories.workflow_node_execution_repository import WorkflowNodeExecutionRepository
|
||||||
from dify_graph.runtime import GraphRuntimeState
|
from core.workflow.runtime import GraphRuntimeState
|
||||||
from dify_graph.variable_loader import DUMMY_VARIABLE_LOADER, VariableLoader
|
from core.workflow.variable_loader import DUMMY_VARIABLE_LOADER, VariableLoader
|
||||||
from extensions.ext_database import db
|
from extensions.ext_database import db
|
||||||
from factories import file_factory
|
from factories import file_factory
|
||||||
from libs.flask_utils import preserve_flask_contexts
|
from libs.flask_utils import preserve_flask_contexts
|
||||||
|
|||||||
@@ -25,16 +25,16 @@ from core.app.workflow.layers.persistence import PersistenceWorkflowInfo, Workfl
|
|||||||
from core.db.session_factory import session_factory
|
from core.db.session_factory import session_factory
|
||||||
from core.moderation.base import ModerationError
|
from core.moderation.base import ModerationError
|
||||||
from core.moderation.input_moderation import InputModeration
|
from core.moderation.input_moderation import InputModeration
|
||||||
|
from core.variables.variables import Variable
|
||||||
|
from core.workflow.enums import WorkflowType
|
||||||
|
from core.workflow.graph_engine.command_channels.redis_channel import RedisChannel
|
||||||
|
from core.workflow.graph_engine.layers.base import GraphEngineLayer
|
||||||
|
from core.workflow.repositories.workflow_execution_repository import WorkflowExecutionRepository
|
||||||
|
from core.workflow.repositories.workflow_node_execution_repository import WorkflowNodeExecutionRepository
|
||||||
|
from core.workflow.runtime import GraphRuntimeState, VariablePool
|
||||||
|
from core.workflow.system_variable import SystemVariable
|
||||||
|
from core.workflow.variable_loader import VariableLoader
|
||||||
from core.workflow.workflow_entry import WorkflowEntry
|
from core.workflow.workflow_entry import WorkflowEntry
|
||||||
from dify_graph.enums import WorkflowType
|
|
||||||
from dify_graph.graph_engine.command_channels.redis_channel import RedisChannel
|
|
||||||
from dify_graph.graph_engine.layers.base import GraphEngineLayer
|
|
||||||
from dify_graph.repositories.workflow_execution_repository import WorkflowExecutionRepository
|
|
||||||
from dify_graph.repositories.workflow_node_execution_repository import WorkflowNodeExecutionRepository
|
|
||||||
from dify_graph.runtime import GraphRuntimeState, VariablePool
|
|
||||||
from dify_graph.system_variable import SystemVariable
|
|
||||||
from dify_graph.variable_loader import VariableLoader
|
|
||||||
from dify_graph.variables.variables import Variable
|
|
||||||
from extensions.ext_database import db
|
from extensions.ext_database import db
|
||||||
from extensions.ext_redis import redis_client
|
from extensions.ext_redis import redis_client
|
||||||
from extensions.otel import WorkflowAppRunnerHandler, trace_span
|
from extensions.otel import WorkflowAppRunnerHandler, trace_span
|
||||||
|
|||||||
@@ -63,16 +63,16 @@ from core.app.entities.task_entities import (
|
|||||||
from core.app.task_pipeline.based_generate_task_pipeline import BasedGenerateTaskPipeline
|
from core.app.task_pipeline.based_generate_task_pipeline import BasedGenerateTaskPipeline
|
||||||
from core.app.task_pipeline.message_cycle_manager import MessageCycleManager
|
from core.app.task_pipeline.message_cycle_manager import MessageCycleManager
|
||||||
from core.base.tts import AppGeneratorTTSPublisher, AudioTrunk
|
from core.base.tts import AppGeneratorTTSPublisher, AudioTrunk
|
||||||
|
from core.model_runtime.entities.llm_entities import LLMUsage
|
||||||
|
from core.model_runtime.utils.encoders import jsonable_encoder
|
||||||
from core.ops.ops_trace_manager import TraceQueueManager
|
from core.ops.ops_trace_manager import TraceQueueManager
|
||||||
from core.repositories.human_input_repository import HumanInputFormRepositoryImpl
|
from core.repositories.human_input_repository import HumanInputFormRepositoryImpl
|
||||||
from dify_graph.entities.pause_reason import HumanInputRequired
|
from core.workflow.entities.pause_reason import HumanInputRequired
|
||||||
from dify_graph.enums import WorkflowExecutionStatus
|
from core.workflow.enums import WorkflowExecutionStatus
|
||||||
from dify_graph.model_runtime.entities.llm_entities import LLMUsage
|
from core.workflow.nodes import NodeType
|
||||||
from dify_graph.model_runtime.utils.encoders import jsonable_encoder
|
from core.workflow.repositories.draft_variable_repository import DraftVariableSaverFactory
|
||||||
from dify_graph.nodes import NodeType
|
from core.workflow.runtime import GraphRuntimeState
|
||||||
from dify_graph.repositories.draft_variable_repository import DraftVariableSaverFactory
|
from core.workflow.system_variable import SystemVariable
|
||||||
from dify_graph.runtime import GraphRuntimeState
|
|
||||||
from dify_graph.system_variable import SystemVariable
|
|
||||||
from extensions.ext_database import db
|
from extensions.ext_database import db
|
||||||
from libs.datetime_utils import naive_utc_now
|
from libs.datetime_utils import naive_utc_now
|
||||||
from models import Account, Conversation, EndUser, Message, MessageFile
|
from models import Account, Conversation, EndUser, Message, MessageFile
|
||||||
@@ -669,14 +669,16 @@ class AdvancedChatAppGenerateTaskPipeline(GraphRuntimeStateSupport):
|
|||||||
) -> Generator[StreamResponse, None, None]:
|
) -> Generator[StreamResponse, None, None]:
|
||||||
"""Handle retriever resources events."""
|
"""Handle retriever resources events."""
|
||||||
self._message_cycle_manager.handle_retriever_resources(event)
|
self._message_cycle_manager.handle_retriever_resources(event)
|
||||||
yield from ()
|
return
|
||||||
|
yield # Make this a generator
|
||||||
|
|
||||||
def _handle_annotation_reply_event(
|
def _handle_annotation_reply_event(
|
||||||
self, event: QueueAnnotationReplyEvent, **kwargs
|
self, event: QueueAnnotationReplyEvent, **kwargs
|
||||||
) -> Generator[StreamResponse, None, None]:
|
) -> Generator[StreamResponse, None, None]:
|
||||||
"""Handle annotation reply events."""
|
"""Handle annotation reply events."""
|
||||||
self._message_cycle_manager.handle_annotation_reply(event)
|
self._message_cycle_manager.handle_annotation_reply(event)
|
||||||
yield from ()
|
return
|
||||||
|
yield # Make this a generator
|
||||||
|
|
||||||
def _handle_message_replace_event(
|
def _handle_message_replace_event(
|
||||||
self, event: QueueMessageReplaceEvent, **kwargs
|
self, event: QueueMessageReplaceEvent, **kwargs
|
||||||
|
|||||||
@@ -20,8 +20,8 @@ from core.app.apps.exc import GenerateTaskStoppedError
|
|||||||
from core.app.apps.message_based_app_generator import MessageBasedAppGenerator
|
from core.app.apps.message_based_app_generator import MessageBasedAppGenerator
|
||||||
from core.app.apps.message_based_app_queue_manager import MessageBasedAppQueueManager
|
from core.app.apps.message_based_app_queue_manager import MessageBasedAppQueueManager
|
||||||
from core.app.entities.app_invoke_entities import AgentChatAppGenerateEntity, InvokeFrom
|
from core.app.entities.app_invoke_entities import AgentChatAppGenerateEntity, InvokeFrom
|
||||||
|
from core.model_runtime.errors.invoke import InvokeAuthorizationError
|
||||||
from core.ops.ops_trace_manager import TraceQueueManager
|
from core.ops.ops_trace_manager import TraceQueueManager
|
||||||
from dify_graph.model_runtime.errors.invoke import InvokeAuthorizationError
|
|
||||||
from extensions.ext_database import db
|
from extensions.ext_database import db
|
||||||
from factories import file_factory
|
from factories import file_factory
|
||||||
from libs.flask_utils import preserve_flask_contexts
|
from libs.flask_utils import preserve_flask_contexts
|
||||||
|
|||||||
@@ -14,10 +14,10 @@ from core.app.entities.app_invoke_entities import AgentChatAppGenerateEntity
|
|||||||
from core.app.entities.queue_entities import QueueAnnotationReplyEvent
|
from core.app.entities.queue_entities import QueueAnnotationReplyEvent
|
||||||
from core.memory.token_buffer_memory import TokenBufferMemory
|
from core.memory.token_buffer_memory import TokenBufferMemory
|
||||||
from core.model_manager import ModelInstance
|
from core.model_manager import ModelInstance
|
||||||
|
from core.model_runtime.entities.llm_entities import LLMMode
|
||||||
|
from core.model_runtime.entities.model_entities import ModelFeature, ModelPropertyKey
|
||||||
|
from core.model_runtime.model_providers.__base.large_language_model import LargeLanguageModel
|
||||||
from core.moderation.base import ModerationError
|
from core.moderation.base import ModerationError
|
||||||
from dify_graph.model_runtime.entities.llm_entities import LLMMode
|
|
||||||
from dify_graph.model_runtime.entities.model_entities import ModelFeature, ModelPropertyKey
|
|
||||||
from dify_graph.model_runtime.model_providers.__base.large_language_model import LargeLanguageModel
|
|
||||||
from extensions.ext_database import db
|
from extensions.ext_database import db
|
||||||
from models.model import App, Conversation, Message
|
from models.model import App, Conversation, Message
|
||||||
|
|
||||||
@@ -178,7 +178,7 @@ class AgentChatAppRunner(AppRunner):
|
|||||||
|
|
||||||
# change function call strategy based on LLM model
|
# change function call strategy based on LLM model
|
||||||
llm_model = cast(LargeLanguageModel, model_instance.model_type_instance)
|
llm_model = cast(LargeLanguageModel, model_instance.model_type_instance)
|
||||||
model_schema = llm_model.get_model_schema(model_instance.model_name, model_instance.credentials)
|
model_schema = llm_model.get_model_schema(model_instance.model, model_instance.credentials)
|
||||||
if not model_schema:
|
if not model_schema:
|
||||||
raise ValueError("Model schema not found")
|
raise ValueError("Model schema not found")
|
||||||
|
|
||||||
|
|||||||
@@ -6,7 +6,7 @@ from typing import Any, Union
|
|||||||
from core.app.entities.app_invoke_entities import InvokeFrom
|
from core.app.entities.app_invoke_entities import InvokeFrom
|
||||||
from core.app.entities.task_entities import AppBlockingResponse, AppStreamResponse
|
from core.app.entities.task_entities import AppBlockingResponse, AppStreamResponse
|
||||||
from core.errors.error import ModelCurrentlyNotSupportError, ProviderTokenNotInitError, QuotaExceededError
|
from core.errors.error import ModelCurrentlyNotSupportError, ProviderTokenNotInitError, QuotaExceededError
|
||||||
from dify_graph.model_runtime.errors.invoke import InvokeError
|
from core.model_runtime.errors.invoke import InvokeError
|
||||||
|
|
||||||
logger = logging.getLogger(__name__)
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
|||||||
@@ -3,22 +3,22 @@ from typing import TYPE_CHECKING, Any, Union, final
|
|||||||
|
|
||||||
from sqlalchemy.orm import Session
|
from sqlalchemy.orm import Session
|
||||||
|
|
||||||
|
from core.app.app_config.entities import VariableEntityType
|
||||||
from core.app.entities.app_invoke_entities import InvokeFrom
|
from core.app.entities.app_invoke_entities import InvokeFrom
|
||||||
from dify_graph.enums import NodeType
|
from core.file import File, FileUploadConfig
|
||||||
from dify_graph.file import File, FileUploadConfig
|
from core.workflow.enums import NodeType
|
||||||
from dify_graph.repositories.draft_variable_repository import (
|
from core.workflow.repositories.draft_variable_repository import (
|
||||||
DraftVariableSaver,
|
DraftVariableSaver,
|
||||||
DraftVariableSaverFactory,
|
DraftVariableSaverFactory,
|
||||||
NoopDraftVariableSaver,
|
NoopDraftVariableSaver,
|
||||||
)
|
)
|
||||||
from dify_graph.variables.input_entities import VariableEntityType
|
|
||||||
from factories import file_factory
|
from factories import file_factory
|
||||||
from libs.orjson import orjson_dumps
|
from libs.orjson import orjson_dumps
|
||||||
from models import Account, EndUser
|
from models import Account, EndUser
|
||||||
from services.workflow_draft_variable_service import DraftVariableSaver as DraftVariableSaverImpl
|
from services.workflow_draft_variable_service import DraftVariableSaver as DraftVariableSaverImpl
|
||||||
|
|
||||||
if TYPE_CHECKING:
|
if TYPE_CHECKING:
|
||||||
from dify_graph.variables.input_entities import VariableEntity
|
from core.app.app_config.entities import VariableEntity
|
||||||
|
|
||||||
|
|
||||||
class BaseAppGenerator:
|
class BaseAppGenerator:
|
||||||
|
|||||||
@@ -2,7 +2,7 @@ import logging
|
|||||||
import queue
|
import queue
|
||||||
import threading
|
import threading
|
||||||
import time
|
import time
|
||||||
from abc import ABC, abstractmethod
|
from abc import abstractmethod
|
||||||
from enum import IntEnum, auto
|
from enum import IntEnum, auto
|
||||||
from typing import Any
|
from typing import Any
|
||||||
|
|
||||||
@@ -20,7 +20,7 @@ from core.app.entities.queue_entities import (
|
|||||||
QueueStopEvent,
|
QueueStopEvent,
|
||||||
WorkflowQueueMessage,
|
WorkflowQueueMessage,
|
||||||
)
|
)
|
||||||
from dify_graph.runtime import GraphRuntimeState
|
from core.workflow.runtime import GraphRuntimeState
|
||||||
from extensions.ext_redis import redis_client
|
from extensions.ext_redis import redis_client
|
||||||
|
|
||||||
logger = logging.getLogger(__name__)
|
logger = logging.getLogger(__name__)
|
||||||
@@ -31,7 +31,7 @@ class PublishFrom(IntEnum):
|
|||||||
TASK_PIPELINE = auto()
|
TASK_PIPELINE = auto()
|
||||||
|
|
||||||
|
|
||||||
class AppQueueManager(ABC):
|
class AppQueueManager:
|
||||||
def __init__(self, task_id: str, user_id: str, invoke_from: InvokeFrom):
|
def __init__(self, task_id: str, user_id: str, invoke_from: InvokeFrom):
|
||||||
if not user_id:
|
if not user_id:
|
||||||
raise ValueError("user is required")
|
raise ValueError("user is required")
|
||||||
@@ -122,7 +122,7 @@ class AppQueueManager(ABC):
|
|||||||
"""Attach the live graph runtime state reference for downstream consumers."""
|
"""Attach the live graph runtime state reference for downstream consumers."""
|
||||||
self._graph_runtime_state = graph_runtime_state
|
self._graph_runtime_state = graph_runtime_state
|
||||||
|
|
||||||
def publish(self, event: AppQueueEvent, pub_from: PublishFrom) -> None:
|
def publish(self, event: AppQueueEvent, pub_from: PublishFrom):
|
||||||
"""
|
"""
|
||||||
Publish event to queue
|
Publish event to queue
|
||||||
:param event:
|
:param event:
|
||||||
@@ -133,7 +133,7 @@ class AppQueueManager(ABC):
|
|||||||
self._publish(event, pub_from)
|
self._publish(event, pub_from)
|
||||||
|
|
||||||
@abstractmethod
|
@abstractmethod
|
||||||
def _publish(self, event: AppQueueEvent, pub_from: PublishFrom) -> None:
|
def _publish(self, event: AppQueueEvent, pub_from: PublishFrom):
|
||||||
"""
|
"""
|
||||||
Publish event to queue
|
Publish event to queue
|
||||||
:param event:
|
:param event:
|
||||||
|
|||||||
@@ -22,29 +22,29 @@ from core.app.entities.queue_entities import (
|
|||||||
from core.app.features.annotation_reply.annotation_reply import AnnotationReplyFeature
|
from core.app.features.annotation_reply.annotation_reply import AnnotationReplyFeature
|
||||||
from core.app.features.hosting_moderation.hosting_moderation import HostingModerationFeature
|
from core.app.features.hosting_moderation.hosting_moderation import HostingModerationFeature
|
||||||
from core.external_data_tool.external_data_fetch import ExternalDataFetch
|
from core.external_data_tool.external_data_fetch import ExternalDataFetch
|
||||||
|
from core.file.enums import FileTransferMethod, FileType
|
||||||
from core.memory.token_buffer_memory import TokenBufferMemory
|
from core.memory.token_buffer_memory import TokenBufferMemory
|
||||||
from core.model_manager import ModelInstance
|
from core.model_manager import ModelInstance
|
||||||
from core.moderation.input_moderation import InputModeration
|
from core.model_runtime.entities.llm_entities import LLMResult, LLMResultChunk, LLMResultChunkDelta, LLMUsage
|
||||||
from core.prompt.advanced_prompt_transform import AdvancedPromptTransform
|
from core.model_runtime.entities.message_entities import (
|
||||||
from core.prompt.entities.advanced_prompt_entities import ChatModelMessage, CompletionModelPromptTemplate, MemoryConfig
|
|
||||||
from core.prompt.simple_prompt_transform import ModelMode, SimplePromptTransform
|
|
||||||
from core.tools.tool_file_manager import ToolFileManager
|
|
||||||
from dify_graph.file.enums import FileTransferMethod, FileType
|
|
||||||
from dify_graph.model_runtime.entities.llm_entities import LLMResult, LLMResultChunk, LLMResultChunkDelta, LLMUsage
|
|
||||||
from dify_graph.model_runtime.entities.message_entities import (
|
|
||||||
AssistantPromptMessage,
|
AssistantPromptMessage,
|
||||||
ImagePromptMessageContent,
|
ImagePromptMessageContent,
|
||||||
PromptMessage,
|
PromptMessage,
|
||||||
TextPromptMessageContent,
|
TextPromptMessageContent,
|
||||||
)
|
)
|
||||||
from dify_graph.model_runtime.entities.model_entities import ModelPropertyKey
|
from core.model_runtime.entities.model_entities import ModelPropertyKey
|
||||||
from dify_graph.model_runtime.errors.invoke import InvokeBadRequestError
|
from core.model_runtime.errors.invoke import InvokeBadRequestError
|
||||||
|
from core.moderation.input_moderation import InputModeration
|
||||||
|
from core.prompt.advanced_prompt_transform import AdvancedPromptTransform
|
||||||
|
from core.prompt.entities.advanced_prompt_entities import ChatModelMessage, CompletionModelPromptTemplate, MemoryConfig
|
||||||
|
from core.prompt.simple_prompt_transform import ModelMode, SimplePromptTransform
|
||||||
|
from core.tools.tool_file_manager import ToolFileManager
|
||||||
from extensions.ext_database import db
|
from extensions.ext_database import db
|
||||||
from models.enums import CreatorUserRole
|
from models.enums import CreatorUserRole
|
||||||
from models.model import App, AppMode, Message, MessageAnnotation, MessageFile
|
from models.model import App, AppMode, Message, MessageAnnotation, MessageFile
|
||||||
|
|
||||||
if TYPE_CHECKING:
|
if TYPE_CHECKING:
|
||||||
from dify_graph.file.models import File
|
from core.file.models import File
|
||||||
|
|
||||||
_logger = logging.getLogger(__name__)
|
_logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
|||||||
@@ -19,8 +19,8 @@ from core.app.apps.exc import GenerateTaskStoppedError
|
|||||||
from core.app.apps.message_based_app_generator import MessageBasedAppGenerator
|
from core.app.apps.message_based_app_generator import MessageBasedAppGenerator
|
||||||
from core.app.apps.message_based_app_queue_manager import MessageBasedAppQueueManager
|
from core.app.apps.message_based_app_queue_manager import MessageBasedAppQueueManager
|
||||||
from core.app.entities.app_invoke_entities import ChatAppGenerateEntity, InvokeFrom
|
from core.app.entities.app_invoke_entities import ChatAppGenerateEntity, InvokeFrom
|
||||||
|
from core.model_runtime.errors.invoke import InvokeAuthorizationError
|
||||||
from core.ops.ops_trace_manager import TraceQueueManager
|
from core.ops.ops_trace_manager import TraceQueueManager
|
||||||
from dify_graph.model_runtime.errors.invoke import InvokeAuthorizationError
|
|
||||||
from extensions.ext_database import db
|
from extensions.ext_database import db
|
||||||
from factories import file_factory
|
from factories import file_factory
|
||||||
from models import Account
|
from models import Account
|
||||||
|
|||||||
@@ -11,12 +11,12 @@ from core.app.entities.app_invoke_entities import (
|
|||||||
)
|
)
|
||||||
from core.app.entities.queue_entities import QueueAnnotationReplyEvent
|
from core.app.entities.queue_entities import QueueAnnotationReplyEvent
|
||||||
from core.callback_handler.index_tool_callback_handler import DatasetIndexToolCallbackHandler
|
from core.callback_handler.index_tool_callback_handler import DatasetIndexToolCallbackHandler
|
||||||
|
from core.file import File
|
||||||
from core.memory.token_buffer_memory import TokenBufferMemory
|
from core.memory.token_buffer_memory import TokenBufferMemory
|
||||||
from core.model_manager import ModelInstance
|
from core.model_manager import ModelInstance
|
||||||
|
from core.model_runtime.entities.message_entities import ImagePromptMessageContent
|
||||||
from core.moderation.base import ModerationError
|
from core.moderation.base import ModerationError
|
||||||
from core.rag.retrieval.dataset_retrieval import DatasetRetrieval
|
from core.rag.retrieval.dataset_retrieval import DatasetRetrieval
|
||||||
from dify_graph.file import File
|
|
||||||
from dify_graph.model_runtime.entities.message_entities import ImagePromptMessageContent
|
|
||||||
from extensions.ext_database import db
|
from extensions.ext_database import db
|
||||||
from models.model import App, Conversation, Message
|
from models.model import App, Conversation, Message
|
||||||
|
|
||||||
|
|||||||
@@ -4,7 +4,7 @@ from __future__ import annotations
|
|||||||
|
|
||||||
from typing import TYPE_CHECKING
|
from typing import TYPE_CHECKING
|
||||||
|
|
||||||
from dify_graph.runtime import GraphRuntimeState
|
from core.workflow.runtime import GraphRuntimeState
|
||||||
|
|
||||||
if TYPE_CHECKING:
|
if TYPE_CHECKING:
|
||||||
from core.app.task_pipeline.based_generate_task_pipeline import BasedGenerateTaskPipeline
|
from core.app.task_pipeline.based_generate_task_pipeline import BasedGenerateTaskPipeline
|
||||||
|
|||||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user