Compare commits

..
33 Commits
Author SHA1 Message Date
QuantumGhost f943f1e39b chore: bump version to 1.13.1 (#33505) 2026-03-17 18:31:12 +08:00
-LAN-andGitHub 239e09473e fix(web): preserve public workflow SSE reconnect after pause (#33552) 2026-03-17 16:41:08 +08:00
非法操作GitHubgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>hj24
18ff5d9288 feat: add metrics to clean message and workflow-run task (#33143)
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: hj24 <mambahj24@gmail.com>
2026-03-17 13:55:28 +08:00
Stephen ZhouandGitHub 18af5fc8c7 chore: revert #33540 #32966 (#33566) 2026-03-17 12:55:58 +08:00
9e2123c655 test(web): ignore default-branch merge commits in diff coverage (#33564)
Co-authored-by: CodingOnStar <hanxujiang@dify.com>
2026-03-17 12:33:52 +08:00
statxcandGitHub 7e34faaf51 refactor(api): replace dict/Mapping with TypedDict in trigger.py and workflow.py (#33562) 2026-03-17 13:14:42 +09:00
Stephen ZhouandGitHub 569748189e refactor: use hoisted modern monaco (#33540) 2026-03-17 10:56:46 +08:00
yyhandGitHub f198f5b0ab fix: raise block selector overlay z-index (#33557) 2026-03-17 10:39:48 +08:00
KVOJJJinGitHubautofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
49e0e1b939 fix(web): page crash in knowledge retrieval node caused by dataset selection and score threshold (#33553)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-03-17 10:35:07 +08:00
statxcandGitHub f886f11094 refactor(api): replace dict/Mapping with TypedDict in dataset models (#33550) 2026-03-17 10:33:29 +09:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
fa82a0f708 chore(deps): bump authlib from 1.6.7 to 1.6.9 in /api (#33544)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-17 05:06:07 +09:00
0a3275fbe8 chore: update coverage summary check in web tests workflow (#33533)
Co-authored-by: CodingOnStar <hanxujiang@dify.com>
Co-authored-by: yyh <92089059+lyzno1@users.noreply.github.com>
2026-03-16 23:09:33 +08:00
-LAN-andGitHub e445f69604 refactor(api): simplify response session eligibility (#33538) 2026-03-16 21:22:37 +08:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
c7f86dba09 chore(deps-dev): bump the dev group across 1 directory with 19 updates (#33525)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-03-16 20:31:58 +09:00
Coding On StarGitHubCodingOnStarautofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
6da802eb2a refactor(custom): reorganize web app brand module and raise coverage threshold (#33531)
Co-authored-by: CodingOnStar <hanxujiang@dify.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-03-16 18:17:21 +08:00
yyhandGitHub c3ee83645f fix(web): migrate InputNumber to Base UI NumberField (#33520) 2026-03-16 17:59:30 +08:00
QuantumGhostGitHubautofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
4a090876f1 chore(api): rename configuration EVENT_BUS_REDIS_CLUSTERS (#33528)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-03-16 17:50:15 +08:00
Stephen ZhouandGitHub 598189d307 chore: fix dep alerts (#33527) 2026-03-16 17:08:36 +08:00
QuantumGhostandGitHub 1f3fa95e2c chore: update plugin daemon version to 0.5.4-local in Docker compose files (#33526) 2026-03-16 17:06:32 +08:00
0d72d99263 test: limit web diff coverage to current push range (#33523)
Co-authored-by: CodingOnStar <hanxujiang@dify.com>
2026-03-16 16:59:25 +08:00
BlackouttaGitHubgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>QuantumGhostautofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
57d476d4e2 feat: apply markdown rendering to HITL email, sanitize email subject and body (#32305)
This PR:

1. Fixes the bug that email body of `HumanInput` node are sent as-is, without markdown rendering or sanitization
2. Applies HTML sanitization to email subject and body
3. Removes `\r` and `\n` from email subject to prevent SMTP header injection

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: QuantumGhost <obelisk.reg+git@gmail.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-03-16 16:52:46 +08:00
Stephen ZhouandGitHub 4822d550b6 chore: remove next img (#33517) 2026-03-16 16:48:22 +08:00
wangxiaoleiGitHubautofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
041d7ffe3d chore: compatiable resource_metadata return without scheme (#33506)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-03-16 15:44:48 +08:00
7daec9717d feat(diff-coverage): enhance coverage reporting for multi-line statements and branches (#33516)
Co-authored-by: CodingOnStar <hanxujiang@dify.com>
2026-03-16 15:09:46 +08:00
Stephen ZhouandGitHub a01c384f5b chore: remove next font (#33512) 2026-03-16 14:57:25 +08:00
Coding On StarGitHubCodingOnStarautofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
378577767b refactor(web): split text-generation result flow and raise coverage (#33499)
Co-authored-by: CodingOnStar <hanxujiang@dify.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-03-16 14:42:32 +08:00
59327e4f10 feat(diff-coverage): implement coverage analysis for changed components (#33514)
Co-authored-by: CodingOnStar <hanxujiang@dify.com>
2026-03-16 14:39:57 +08:00
wangxiaoleiandGitHub 915ee385db fix: fix weaviate_vector test failed (#33511) 2026-03-16 14:32:09 +08:00
非法操作GitHubautofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>QuantumGhost
98e72521f4 chore: change draft var to user scoped (#33066)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: QuantumGhost <obelisk.reg+git@gmail.com>
2026-03-16 14:04:41 +08:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
df570df238 chore(deps-dev): bump the vdb group across 1 directory with 15 updates (#33502)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-16 14:42:27 +09:00
yyhandGitHub 7ac482d776 refactor(web): consolidate query/mutation guidance and deprecate use-base wrappers (#33456)
Signed-off-by: yyh <yuanyouhuilyz@gmail.com>
2026-03-16 13:38:29 +08:00
SageGitHublijiezhaoautofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
3920d67b8e feat(api): Making WeaviateClient a singleton
Co-authored-by: lijiezhao <lijiezhao@perfect99.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-03-16 13:38:28 +08:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>Asuka Minatoautofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
29b724e23d chore(deps): bump google-auth from 2.49.0 to 2.49.1 in /api in the google group (#33483)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Asuka Minato <i@asukaminato.eu.org>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-03-16 13:55:34 +09:00
738 changed files with 65453 additions and 5463 deletions
+6 -47
View File
@@ -187,53 +187,12 @@ const Template = useMemo(() => {
**When**: Component directly handles API calls, data transformation, or complex async operations.
**Dify Convention**: Use `@tanstack/react-query` hooks from `web/service/use-*.ts` or create custom data hooks.
```typescript
// ❌ Before: API logic in component
const MCPServiceCard = () => {
const [basicAppConfig, setBasicAppConfig] = useState({})
useEffect(() => {
if (isBasicApp && appId) {
(async () => {
const res = await fetchAppDetail({ url: '/apps', id: appId })
setBasicAppConfig(res?.model_config || {})
})()
}
}, [appId, isBasicApp])
// More API-related logic...
}
// ✅ After: Extract to data hook using React Query
// use-app-config.ts
import { useQuery } from '@tanstack/react-query'
import { get } from '@/service/base'
const NAME_SPACE = 'appConfig'
export const useAppConfig = (appId: string, isBasicApp: boolean) => {
return useQuery({
enabled: isBasicApp && !!appId,
queryKey: [NAME_SPACE, 'detail', appId],
queryFn: () => get<AppDetailResponse>(`/apps/${appId}`),
select: data => data?.model_config || {},
})
}
// Component becomes cleaner
const MCPServiceCard = () => {
const { data: config, isLoading } = useAppConfig(appId, isBasicApp)
// UI only
}
```
**React Query Best Practices in Dify**:
- Define `NAME_SPACE` for query key organization
- Use `enabled` option for conditional fetching
- Use `select` for data transformation
- Export invalidation hooks: `useInvalidXxx`
**Dify Convention**:
- This skill is for component decomposition, not query/mutation design.
- When refactoring data fetching, follow `web/AGENTS.md`.
- Use `frontend-query-mutation` for contracts, query shape, data-fetching wrappers, query/mutation call-site patterns, conditional queries, invalidation, and mutation error handling.
- Do not introduce deprecated `useInvalid` / `useReset`.
- Do not add thin passthrough `useQuery` wrappers during refactoring; only extract a custom hook when it truly orchestrates multiple queries/mutations or shared derived state.
**Dify Examples**:
- `web/service/use-workflow.ts`
@@ -155,48 +155,14 @@ const Configuration: FC = () => {
## Common Hook Patterns in Dify
### 1. Data Fetching Hook (React Query)
### 1. Data Fetching / Mutation Hooks
```typescript
// Pattern: Use @tanstack/react-query for data fetching
import { useQuery, useQueryClient } from '@tanstack/react-query'
import { get } from '@/service/base'
import { useInvalid } from '@/service/use-base'
When hook extraction touches query or mutation code, do not use this reference as the source of truth for data-layer patterns.
const NAME_SPACE = 'appConfig'
// Query keys for cache management
export const appConfigQueryKeys = {
detail: (appId: string) => [NAME_SPACE, 'detail', appId] as const,
}
// Main data hook
export const useAppConfig = (appId: string) => {
return useQuery({
enabled: !!appId,
queryKey: appConfigQueryKeys.detail(appId),
queryFn: () => get<AppDetailResponse>(`/apps/${appId}`),
select: data => data?.model_config || null,
})
}
// Invalidation hook for refreshing data
export const useInvalidAppConfig = () => {
return useInvalid([NAME_SPACE])
}
// Usage in component
const Component = () => {
const { data: config, isLoading, error, refetch } = useAppConfig(appId)
const invalidAppConfig = useInvalidAppConfig()
const handleRefresh = () => {
invalidAppConfig() // Invalidates cache and triggers refetch
}
return <div>...</div>
}
```
- Follow `web/AGENTS.md` first.
- Use `frontend-query-mutation` for contracts, query shape, data-fetching wrappers, query/mutation call-site patterns, conditional queries, invalidation, and mutation error handling.
- Do not introduce deprecated `useInvalid` / `useReset`.
- Do not extract thin passthrough `useQuery` hooks; only extract orchestration hooks.
### 2. Form State Hook
@@ -0,0 +1,44 @@
---
name: frontend-query-mutation
description: Guide for implementing Dify frontend query and mutation patterns with TanStack Query and oRPC. Trigger when creating or updating contracts in web/contract, wiring router composition, consuming consoleQuery or marketplaceQuery in components or services, deciding whether to call queryOptions() directly or extract a helper or use-* hook, handling conditional queries, cache invalidation, mutation error handling, or migrating legacy service calls to contract-first query and mutation helpers.
---
# Frontend Query & Mutation
## Intent
- Keep contract as the single source of truth in `web/contract/*`.
- Prefer contract-shaped `queryOptions()` and `mutationOptions()`.
- Keep invalidation and mutation flow knowledge in the service layer.
- Keep abstractions minimal to preserve TypeScript inference.
## Workflow
1. Identify the change surface.
- Read `references/contract-patterns.md` for contract files, router composition, client helpers, and query or mutation call-site shape.
- Read `references/runtime-rules.md` for conditional queries, invalidation, error handling, and legacy migrations.
- Read both references when a task spans contract shape and runtime behavior.
2. Implement the smallest abstraction that fits the task.
- Default to direct `useQuery(...)` or `useMutation(...)` calls with oRPC helpers at the call site.
- Extract a small shared query helper only when multiple call sites share the same extra options.
- Create `web/service/use-{domain}.ts` only for orchestration or shared domain behavior.
3. Preserve Dify conventions.
- Keep contract inputs in `{ params, query?, body? }` shape.
- Bind invalidation in the service-layer mutation definition.
- Prefer `mutate(...)`; use `mutateAsync(...)` only when Promise semantics are required.
## Files Commonly Touched
- `web/contract/console/*.ts`
- `web/contract/marketplace.ts`
- `web/contract/router.ts`
- `web/service/client.ts`
- `web/service/use-*.ts`
- component and hook call sites using `consoleQuery` or `marketplaceQuery`
## References
- Use `references/contract-patterns.md` for contract shape, router registration, query and mutation helpers, and anti-patterns that degrade inference.
- Use `references/runtime-rules.md` for conditional queries, invalidation, `mutate` versus `mutateAsync`, and legacy migration rules.
Treat this skill as the single query and mutation entry point for Dify frontend work. Keep detailed rules in the reference files instead of duplicating them in project docs.
@@ -0,0 +1,4 @@
interface:
display_name: "Frontend Query & Mutation"
short_description: "Dify TanStack Query and oRPC patterns"
default_prompt: "Use this skill when implementing or reviewing Dify frontend contracts, query and mutation call sites, conditional queries, invalidation, or legacy query/mutation migrations."
@@ -0,0 +1,98 @@
# Contract Patterns
## Table of Contents
- Intent
- Minimal structure
- Core workflow
- Query usage decision rule
- Mutation usage decision rule
- Anti-patterns
- Contract rules
- Type export
## Intent
- Keep contract as the single source of truth in `web/contract/*`.
- Default query usage to call-site `useQuery(consoleQuery|marketplaceQuery.xxx.queryOptions(...))` when endpoint behavior maps 1:1 to the contract.
- Keep abstractions minimal and preserve TypeScript inference.
## Minimal Structure
```text
web/contract/
├── base.ts
├── router.ts
├── marketplace.ts
└── console/
├── billing.ts
└── ...other domains
web/service/client.ts
```
## Core Workflow
1. Define contract in `web/contract/console/{domain}.ts` or `web/contract/marketplace.ts`.
- Use `base.route({...}).output(type<...>())` as the baseline.
- Add `.input(type<...>())` only when the request has `params`, `query`, or `body`.
- For `GET` without input, omit `.input(...)`; do not use `.input(type<unknown>())`.
2. Register contract in `web/contract/router.ts`.
- Import directly from domain files and nest by API prefix.
3. Consume from UI call sites via oRPC query utilities.
```typescript
import { useQuery } from '@tanstack/react-query'
import { consoleQuery } from '@/service/client'
const invoiceQuery = useQuery(consoleQuery.billing.invoices.queryOptions({
staleTime: 5 * 60 * 1000,
throwOnError: true,
select: invoice => invoice.url,
}))
```
## Query Usage Decision Rule
1. Default to direct `*.queryOptions(...)` usage at the call site.
2. If 3 or more call sites share the same extra options, extract a small query helper, not a `use-*` passthrough hook.
3. Create `web/service/use-{domain}.ts` only for orchestration.
- Combine multiple queries or mutations.
- Share domain-level derived state or invalidation helpers.
```typescript
const invoicesBaseQueryOptions = () =>
consoleQuery.billing.invoices.queryOptions({ retry: false })
const invoiceQuery = useQuery({
...invoicesBaseQueryOptions(),
throwOnError: true,
})
```
## Mutation Usage Decision Rule
1. Default to mutation helpers from `consoleQuery` or `marketplaceQuery`, for example `useMutation(consoleQuery.billing.bindPartnerStack.mutationOptions(...))`.
2. If the mutation flow is heavily custom, use oRPC clients as `mutationFn`, for example `consoleClient.xxx` or `marketplaceClient.xxx`, instead of handwritten non-oRPC mutation logic.
## Anti-Patterns
- Do not wrap `useQuery` with `options?: Partial<UseQueryOptions>`.
- Do not split local `queryKey` and `queryFn` when oRPC `queryOptions` already exists and fits the use case.
- Do not create thin `use-*` passthrough hooks for a single endpoint.
- These patterns can degrade inference, especially around `throwOnError` and `select`, and add unnecessary indirection.
## Contract Rules
- Input structure: always use `{ params, query?, body? }`.
- No-input `GET`: omit `.input(...)`; do not use `.input(type<unknown>())`.
- Path params: use `{paramName}` in the path and match it in the `params` object.
- Router nesting: group by API prefix, for example `/billing/*` becomes `billing: {}`.
- No barrel files: import directly from specific files.
- Types: import from `@/types/` and use the `type<T>()` helper.
- Mutations: prefer `mutationOptions`; use explicit `mutationKey` mainly for defaults, filtering, and devtools.
## Type Export
```typescript
export type ConsoleInputs = InferContractRouterInputs<typeof consoleRouterContract>
```
@@ -0,0 +1,133 @@
# Runtime Rules
## Table of Contents
- Conditional queries
- Cache invalidation
- Key API guide
- `mutate` vs `mutateAsync`
- Legacy migration
## Conditional Queries
Prefer contract-shaped `queryOptions(...)`.
When required input is missing, prefer `input: skipToken` instead of placeholder params or non-null assertions.
Use `enabled` only for extra business gating after the input itself is already valid.
```typescript
import { skipToken, useQuery } from '@tanstack/react-query'
// Disable the query by skipping input construction.
function useAccessMode(appId: string | undefined) {
return useQuery(consoleQuery.accessControl.appAccessMode.queryOptions({
input: appId
? { params: { appId } }
: skipToken,
}))
}
// Avoid runtime-only guards that bypass type checking.
function useBadAccessMode(appId: string | undefined) {
return useQuery(consoleQuery.accessControl.appAccessMode.queryOptions({
input: { params: { appId: appId! } },
enabled: !!appId,
}))
}
```
## Cache Invalidation
Bind invalidation in the service-layer mutation definition.
Components may add UI feedback in call-site callbacks, but they should not decide which queries to invalidate.
Use:
- `.key()` for namespace or prefix invalidation
- `.queryKey(...)` only for exact cache reads or writes such as `getQueryData` and `setQueryData`
- `queryClient.invalidateQueries(...)` in mutation `onSuccess`
Do not use deprecated `useInvalid` from `use-base.ts`.
```typescript
// Service layer owns cache invalidation.
export const useUpdateAccessMode = () => {
const queryClient = useQueryClient()
return useMutation(consoleQuery.accessControl.updateAccessMode.mutationOptions({
onSuccess: () => {
queryClient.invalidateQueries({
queryKey: consoleQuery.accessControl.appWhitelistSubjects.key(),
})
},
}))
}
// Component only adds UI behavior.
updateAccessMode({ appId, mode }, {
onSuccess: () => Toast.notify({ type: 'success', message: '...' }),
})
// Avoid putting invalidation knowledge in the component.
mutate({ appId, mode }, {
onSuccess: () => {
queryClient.invalidateQueries({
queryKey: consoleQuery.accessControl.appWhitelistSubjects.key(),
})
},
})
```
## Key API Guide
- `.key(...)`
- Use for partial matching operations.
- Prefer it for invalidation, refetch, and cancel patterns.
- Example: `queryClient.invalidateQueries({ queryKey: consoleQuery.billing.key() })`
- `.queryKey(...)`
- Use for a specific query's full key.
- Prefer it for exact cache addressing and direct reads or writes.
- `.mutationKey(...)`
- Use for a specific mutation's full key.
- Prefer it for mutation defaults registration, mutation-status filtering, and devtools grouping.
## `mutate` vs `mutateAsync`
Prefer `mutate` by default.
Use `mutateAsync` only when Promise semantics are truly required, such as parallel mutations or sequential steps with result dependencies.
Rules:
- Event handlers should usually call `mutate(...)` with `onSuccess` or `onError`.
- Every `await mutateAsync(...)` must be wrapped in `try/catch`.
- Do not use `mutateAsync` when callbacks already express the flow clearly.
```typescript
// Default case.
mutation.mutate(data, {
onSuccess: result => router.push(result.url),
})
// Promise semantics are required.
try {
const order = await createOrder.mutateAsync(orderData)
await confirmPayment.mutateAsync({ orderId: order.id, token })
router.push(`/orders/${order.id}`)
}
catch (error) {
Toast.notify({
type: 'error',
message: error instanceof Error ? error.message : 'Unknown error',
})
}
```
## Legacy Migration
When touching old code, migrate it toward these rules:
| Old pattern | New pattern |
|---|---|
| `useInvalid(key)` in service layer | `queryClient.invalidateQueries(...)` inside mutation `onSuccess` |
| component-triggered invalidation after mutation | move invalidation into the service-layer mutation definition |
| imperative fetch plus manual invalidation | wrap it in `useMutation(...mutationOptions(...))` |
| `await mutateAsync()` without `try/catch` | switch to `mutate(...)` or add `try/catch` |
-103
View File
@@ -1,103 +0,0 @@
---
name: orpc-contract-first
description: Guide for implementing oRPC contract-first API patterns in Dify frontend. Trigger when creating or updating contracts in web/contract, wiring router composition, integrating TanStack Query with typed contracts, migrating legacy service calls to oRPC, or deciding whether to call queryOptions directly vs extracting a helper or use-* hook in web/service.
---
# oRPC Contract-First Development
## Intent
- Keep contract as single source of truth in `web/contract/*`.
- Default query usage: call-site `useQuery(consoleQuery|marketplaceQuery.xxx.queryOptions(...))` when endpoint behavior maps 1:1 to the contract.
- Keep abstractions minimal and preserve TypeScript inference.
## Minimal Structure
```text
web/contract/
├── base.ts
├── router.ts
├── marketplace.ts
└── console/
├── billing.ts
└── ...other domains
web/service/client.ts
```
## Core Workflow
1. Define contract in `web/contract/console/{domain}.ts` or `web/contract/marketplace.ts`
- Use `base.route({...}).output(type<...>())` as baseline.
- Add `.input(type<...>())` only when request has `params/query/body`.
- For `GET` without input, omit `.input(...)` (do not use `.input(type<unknown>())`).
2. Register contract in `web/contract/router.ts`
- Import directly from domain files and nest by API prefix.
3. Consume from UI call sites via oRPC query utils.
```typescript
import { useQuery } from '@tanstack/react-query'
import { consoleQuery } from '@/service/client'
const invoiceQuery = useQuery(consoleQuery.billing.invoices.queryOptions({
staleTime: 5 * 60 * 1000,
throwOnError: true,
select: invoice => invoice.url,
}))
```
## Query Usage Decision Rule
1. Default: call site directly uses `*.queryOptions(...)`.
2. If 3+ call sites share the same extra options (for example `retry: false`), extract a small queryOptions helper, not a `use-*` passthrough hook.
3. Create `web/service/use-{domain}.ts` only for orchestration:
- Combine multiple queries/mutations.
- Share domain-level derived state or invalidation helpers.
```typescript
const invoicesBaseQueryOptions = () =>
consoleQuery.billing.invoices.queryOptions({ retry: false })
const invoiceQuery = useQuery({
...invoicesBaseQueryOptions(),
throwOnError: true,
})
```
## Mutation Usage Decision Rule
1. Default: call mutation helpers from `consoleQuery` / `marketplaceQuery`, for example `useMutation(consoleQuery.billing.bindPartnerStack.mutationOptions(...))`.
2. If mutation flow is heavily custom, use oRPC clients as `mutationFn` (for example `consoleClient.xxx` / `marketplaceClient.xxx`), instead of generic handwritten non-oRPC mutation logic.
## Key API Guide (`.key` vs `.queryKey` vs `.mutationKey`)
- `.key(...)`:
- Use for partial matching operations (recommended for invalidation/refetch/cancel patterns).
- Example: `queryClient.invalidateQueries({ queryKey: consoleQuery.billing.key() })`
- `.queryKey(...)`:
- Use for a specific query's full key (exact query identity / direct cache addressing).
- `.mutationKey(...)`:
- Use for a specific mutation's full key.
- Typical use cases: mutation defaults registration, mutation-status filtering (`useIsMutating`, `queryClient.isMutating`), or explicit devtools grouping.
## Anti-Patterns
- Do not wrap `useQuery` with `options?: Partial<UseQueryOptions>`.
- Do not split local `queryKey/queryFn` when oRPC `queryOptions` already exists and fits the use case.
- Do not create thin `use-*` passthrough hooks for a single endpoint.
- Reason: these patterns can degrade inference (`data` may become `unknown`, especially around `throwOnError`/`select`) and add unnecessary indirection.
## Contract Rules
- **Input structure**: Always use `{ params, query?, body? }` format
- **No-input GET**: Omit `.input(...)`; do not use `.input(type<unknown>())`
- **Path params**: Use `{paramName}` in path, match in `params` object
- **Router nesting**: Group by API prefix (e.g., `/billing/*` -> `billing: {}`)
- **No barrel files**: Import directly from specific files
- **Types**: Import from `@/types/`, use `type<T>()` helper
- **Mutations**: Prefer `mutationOptions`; use explicit `mutationKey` mainly for defaults/filtering/devtools
## Type Export
```typescript
export type ConsoleInputs = InferContractRouterInputs<typeof consoleRouterContract>
```
+1
View File
@@ -0,0 +1 @@
../../.agents/skills/frontend-query-mutation
-1
View File
@@ -1 +0,0 @@
../../.agents/skills/orpc-contract-first
+3 -2
View File
@@ -63,8 +63,9 @@ jobs:
if: needs.check-changes.outputs.web-changed == 'true'
uses: ./.github/workflows/web-tests.yml
with:
base_sha: ${{ github.event_name == 'pull_request' && github.event.pull_request.base.sha || github.event.before }}
head_sha: ${{ github.event_name == 'pull_request' && github.event.pull_request.head.sha || github.sha }}
base_sha: ${{ github.event.before || github.event.pull_request.base.sha }}
diff_range_mode: ${{ github.event.before && 'exact' || 'merge-base' }}
head_sha: ${{ github.event.after || github.event.pull_request.head.sha || github.sha }}
style-check:
name: Style Check
+22 -306
View File
@@ -6,6 +6,9 @@ on:
base_sha:
required: false
type: string
diff_range_mode:
required: false
type: string
head_sha:
required: false
type: string
@@ -86,13 +89,24 @@ jobs:
- name: Merge reports
run: vp test --merge-reports --reporter=json --reporter=agent --coverage
- name: Check app/components diff coverage
- name: Report app/components baseline coverage
run: node ./scripts/report-components-coverage-baseline.mjs
- name: Report app/components test touch
env:
BASE_SHA: ${{ inputs.base_sha }}
DIFF_RANGE_MODE: ${{ inputs.diff_range_mode }}
HEAD_SHA: ${{ inputs.head_sha }}
run: node ./scripts/report-components-test-touch.mjs
- name: Check app/components pure diff coverage
env:
BASE_SHA: ${{ inputs.base_sha }}
DIFF_RANGE_MODE: ${{ inputs.diff_range_mode }}
HEAD_SHA: ${{ inputs.head_sha }}
run: node ./scripts/check-components-diff-coverage.mjs
- name: Coverage Summary
- name: Check Coverage Summary
if: always()
id: coverage-summary
run: |
@@ -101,313 +115,15 @@ jobs:
COVERAGE_FILE="coverage/coverage-final.json"
COVERAGE_SUMMARY_FILE="coverage/coverage-summary.json"
if [ ! -f "$COVERAGE_FILE" ] && [ ! -f "$COVERAGE_SUMMARY_FILE" ]; then
echo "has_coverage=false" >> "$GITHUB_OUTPUT"
echo "### 🚨 Test Coverage Report :test_tube:" >> "$GITHUB_STEP_SUMMARY"
echo "Coverage data not found. Ensure Vitest runs with coverage enabled." >> "$GITHUB_STEP_SUMMARY"
if [ -f "$COVERAGE_FILE" ] || [ -f "$COVERAGE_SUMMARY_FILE" ]; then
echo "has_coverage=true" >> "$GITHUB_OUTPUT"
exit 0
fi
echo "has_coverage=true" >> "$GITHUB_OUTPUT"
node <<'NODE' >> "$GITHUB_STEP_SUMMARY"
const fs = require('fs');
const path = require('path');
let libCoverage = null;
try {
libCoverage = require('istanbul-lib-coverage');
} catch (error) {
libCoverage = null;
}
const summaryPath = path.join('coverage', 'coverage-summary.json');
const finalPath = path.join('coverage', 'coverage-final.json');
const hasSummary = fs.existsSync(summaryPath);
const hasFinal = fs.existsSync(finalPath);
if (!hasSummary && !hasFinal) {
console.log('### Test Coverage Summary :test_tube:');
console.log('');
console.log('No coverage data found.');
process.exit(0);
}
const summary = hasSummary
? JSON.parse(fs.readFileSync(summaryPath, 'utf8'))
: null;
const coverage = hasFinal
? JSON.parse(fs.readFileSync(finalPath, 'utf8'))
: null;
const getLineCoverageFromStatements = (statementMap, statementHits) => {
const lineHits = {};
if (!statementMap || !statementHits) {
return lineHits;
}
Object.entries(statementMap).forEach(([key, statement]) => {
const line = statement?.start?.line;
if (!line) {
return;
}
const hits = statementHits[key] ?? 0;
const previous = lineHits[line];
lineHits[line] = previous === undefined ? hits : Math.max(previous, hits);
});
return lineHits;
};
const getFileCoverage = (entry) => (
libCoverage ? libCoverage.createFileCoverage(entry) : null
);
const getLineHits = (entry, fileCoverage) => {
const lineHits = entry.l ?? {};
if (Object.keys(lineHits).length > 0) {
return lineHits;
}
if (fileCoverage) {
return fileCoverage.getLineCoverage();
}
return getLineCoverageFromStatements(entry.statementMap ?? {}, entry.s ?? {});
};
const getUncoveredLines = (entry, fileCoverage, lineHits) => {
if (lineHits && Object.keys(lineHits).length > 0) {
return Object.entries(lineHits)
.filter(([, count]) => count === 0)
.map(([line]) => Number(line))
.sort((a, b) => a - b);
}
if (fileCoverage) {
return fileCoverage.getUncoveredLines();
}
return [];
};
const totals = {
lines: { covered: 0, total: 0 },
statements: { covered: 0, total: 0 },
branches: { covered: 0, total: 0 },
functions: { covered: 0, total: 0 },
};
const fileSummaries = [];
if (summary) {
const totalEntry = summary.total ?? {};
['lines', 'statements', 'branches', 'functions'].forEach((key) => {
if (totalEntry[key]) {
totals[key].covered = totalEntry[key].covered ?? 0;
totals[key].total = totalEntry[key].total ?? 0;
}
});
Object.entries(summary)
.filter(([file]) => file !== 'total')
.forEach(([file, data]) => {
fileSummaries.push({
file,
pct: data.lines?.pct ?? data.statements?.pct ?? 0,
lines: {
covered: data.lines?.covered ?? 0,
total: data.lines?.total ?? 0,
},
});
});
} else if (coverage) {
Object.entries(coverage).forEach(([file, entry]) => {
const fileCoverage = getFileCoverage(entry);
const lineHits = getLineHits(entry, fileCoverage);
const statementHits = entry.s ?? {};
const branchHits = entry.b ?? {};
const functionHits = entry.f ?? {};
const lineTotal = Object.keys(lineHits).length;
const lineCovered = Object.values(lineHits).filter((n) => n > 0).length;
const statementTotal = Object.keys(statementHits).length;
const statementCovered = Object.values(statementHits).filter((n) => n > 0).length;
const branchTotal = Object.values(branchHits).reduce((acc, branches) => acc + branches.length, 0);
const branchCovered = Object.values(branchHits).reduce(
(acc, branches) => acc + branches.filter((n) => n > 0).length,
0,
);
const functionTotal = Object.keys(functionHits).length;
const functionCovered = Object.values(functionHits).filter((n) => n > 0).length;
totals.lines.total += lineTotal;
totals.lines.covered += lineCovered;
totals.statements.total += statementTotal;
totals.statements.covered += statementCovered;
totals.branches.total += branchTotal;
totals.branches.covered += branchCovered;
totals.functions.total += functionTotal;
totals.functions.covered += functionCovered;
const pct = (covered, tot) => (tot > 0 ? (covered / tot) * 100 : 0);
fileSummaries.push({
file,
pct: pct(lineCovered || statementCovered, lineTotal || statementTotal),
lines: {
covered: lineCovered || statementCovered,
total: lineTotal || statementTotal,
},
});
});
}
const pct = (covered, tot) => (tot > 0 ? ((covered / tot) * 100).toFixed(2) : '0.00');
console.log('### Test Coverage Summary :test_tube:');
console.log('');
console.log('| Metric | Coverage | Covered / Total |');
console.log('|--------|----------|-----------------|');
console.log(`| Lines | ${pct(totals.lines.covered, totals.lines.total)}% | ${totals.lines.covered} / ${totals.lines.total} |`);
console.log(`| Statements | ${pct(totals.statements.covered, totals.statements.total)}% | ${totals.statements.covered} / ${totals.statements.total} |`);
console.log(`| Branches | ${pct(totals.branches.covered, totals.branches.total)}% | ${totals.branches.covered} / ${totals.branches.total} |`);
console.log(`| Functions | ${pct(totals.functions.covered, totals.functions.total)}% | ${totals.functions.covered} / ${totals.functions.total} |`);
console.log('');
console.log('<details><summary>File coverage (lowest lines first)</summary>');
console.log('');
console.log('```');
fileSummaries
.sort((a, b) => (a.pct - b.pct) || (b.lines.total - a.lines.total))
.slice(0, 25)
.forEach(({ file, pct, lines }) => {
console.log(`${pct.toFixed(2)}%\t${lines.covered}/${lines.total}\t${file}`);
});
console.log('```');
console.log('</details>');
if (coverage) {
const pctValue = (covered, tot) => {
if (tot === 0) {
return '0';
}
return ((covered / tot) * 100)
.toFixed(2)
.replace(/\.?0+$/, '');
};
const formatLineRanges = (lines) => {
if (lines.length === 0) {
return '';
}
const ranges = [];
let start = lines[0];
let end = lines[0];
for (let i = 1; i < lines.length; i += 1) {
const current = lines[i];
if (current === end + 1) {
end = current;
continue;
}
ranges.push(start === end ? `${start}` : `${start}-${end}`);
start = current;
end = current;
}
ranges.push(start === end ? `${start}` : `${start}-${end}`);
return ranges.join(',');
};
const tableTotals = {
statements: { covered: 0, total: 0 },
branches: { covered: 0, total: 0 },
functions: { covered: 0, total: 0 },
lines: { covered: 0, total: 0 },
};
const tableRows = Object.entries(coverage)
.map(([file, entry]) => {
const fileCoverage = getFileCoverage(entry);
const lineHits = getLineHits(entry, fileCoverage);
const statementHits = entry.s ?? {};
const branchHits = entry.b ?? {};
const functionHits = entry.f ?? {};
const lineTotal = Object.keys(lineHits).length;
const lineCovered = Object.values(lineHits).filter((n) => n > 0).length;
const statementTotal = Object.keys(statementHits).length;
const statementCovered = Object.values(statementHits).filter((n) => n > 0).length;
const branchTotal = Object.values(branchHits).reduce((acc, branches) => acc + branches.length, 0);
const branchCovered = Object.values(branchHits).reduce(
(acc, branches) => acc + branches.filter((n) => n > 0).length,
0,
);
const functionTotal = Object.keys(functionHits).length;
const functionCovered = Object.values(functionHits).filter((n) => n > 0).length;
tableTotals.lines.total += lineTotal;
tableTotals.lines.covered += lineCovered;
tableTotals.statements.total += statementTotal;
tableTotals.statements.covered += statementCovered;
tableTotals.branches.total += branchTotal;
tableTotals.branches.covered += branchCovered;
tableTotals.functions.total += functionTotal;
tableTotals.functions.covered += functionCovered;
const uncoveredLines = getUncoveredLines(entry, fileCoverage, lineHits);
const filePath = entry.path ?? file;
const relativePath = path.isAbsolute(filePath)
? path.relative(process.cwd(), filePath)
: filePath;
return {
file: relativePath || file,
statements: pctValue(statementCovered, statementTotal),
branches: pctValue(branchCovered, branchTotal),
functions: pctValue(functionCovered, functionTotal),
lines: pctValue(lineCovered, lineTotal),
uncovered: formatLineRanges(uncoveredLines),
};
})
.sort((a, b) => a.file.localeCompare(b.file));
const columns = [
{ key: 'file', header: 'File', align: 'left' },
{ key: 'statements', header: '% Stmts', align: 'right' },
{ key: 'branches', header: '% Branch', align: 'right' },
{ key: 'functions', header: '% Funcs', align: 'right' },
{ key: 'lines', header: '% Lines', align: 'right' },
{ key: 'uncovered', header: 'Uncovered Line #s', align: 'left' },
];
const allFilesRow = {
file: 'All files',
statements: pctValue(tableTotals.statements.covered, tableTotals.statements.total),
branches: pctValue(tableTotals.branches.covered, tableTotals.branches.total),
functions: pctValue(tableTotals.functions.covered, tableTotals.functions.total),
lines: pctValue(tableTotals.lines.covered, tableTotals.lines.total),
uncovered: '',
};
const rowsForOutput = [allFilesRow, ...tableRows];
const formatRow = (row) => `| ${columns
.map(({ key }) => String(row[key] ?? ''))
.join(' | ')} |`;
const headerRow = `| ${columns.map(({ header }) => header).join(' | ')} |`;
const dividerRow = `| ${columns
.map(({ align }) => (align === 'right' ? '---:' : ':---'))
.join(' | ')} |`;
console.log('');
console.log('<details><summary>Vitest coverage table</summary>');
console.log('');
console.log(headerRow);
console.log(dividerRow);
rowsForOutput.forEach((row) => console.log(formatRow(row)));
console.log('</details>');
}
NODE
echo "has_coverage=false" >> "$GITHUB_OUTPUT"
echo "### 🚨 app/components Diff Coverage" >> "$GITHUB_STEP_SUMMARY"
echo "" >> "$GITHUB_STEP_SUMMARY"
echo "Coverage artifacts not found. Ensure Vitest merge reports ran with coverage enabled." >> "$GITHUB_STEP_SUMMARY"
- name: Upload Coverage Artifact
if: steps.coverage-summary.outputs.has_coverage == 'true'
+13 -12
View File
@@ -737,24 +737,25 @@ SANDBOX_EXPIRED_RECORDS_RETENTION_DAYS=30
SANDBOX_EXPIRED_RECORDS_CLEAN_TASK_LOCK_TTL=90000
# Redis URL used for PubSub between API and
# Redis URL used for event bus between API and
# celery worker
# defaults to url constructed from `REDIS_*`
# configurations
PUBSUB_REDIS_URL=
# Pub/sub channel type for streaming events.
# valid options are:
EVENT_BUS_REDIS_URL=
# Event transport type. Options are:
#
# - pubsub: for normal Pub/Sub
# - sharded: for sharded Pub/Sub
# - pubsub: normal Pub/Sub (at-most-once)
# - sharded: sharded Pub/Sub (at-most-once)
# - streams: Redis Streams (at-least-once, recommended to avoid subscriber races)
#
# It's highly recommended to use sharded Pub/Sub AND redis cluster
# for large deployments.
PUBSUB_REDIS_CHANNEL_TYPE=pubsub
# Whether to use Redis cluster mode while running
# PubSub.
# Note: Before enabling 'streams' in production, estimate your expected event volume and retention needs.
# Configure Redis memory limits and stream trimming appropriately (e.g., MAXLEN and key expiry) to reduce
# the risk of data loss from Redis auto-eviction under memory pressure.
# Also accepts ENV: EVENT_BUS_REDIS_CHANNEL_TYPE.
EVENT_BUS_REDIS_CHANNEL_TYPE=pubsub
# Whether to use Redis cluster mode while use redis as event bus.
# It's highly recommended to enable this for large deployments.
PUBSUB_REDIS_USE_CLUSTERS=false
EVENT_BUS_REDIS_USE_CLUSTERS=false
# Whether to Enable human input timeout check task
ENABLE_HUMAN_INPUT_TIMEOUT_TASK=true
+34 -7
View File
@@ -88,6 +88,8 @@ def clean_workflow_runs(
"""
Clean workflow runs and related workflow data for free tenants.
"""
from extensions.otel.runtime import flush_telemetry
if (start_from is None) ^ (end_before is None):
raise click.UsageError("--start-from and --end-before must be provided together.")
@@ -104,16 +106,27 @@ def clean_workflow_runs(
end_before = now - datetime.timedelta(days=to_days_ago)
before_days = 0
if from_days_ago is not None and to_days_ago is not None:
task_label = f"{from_days_ago}to{to_days_ago}"
elif start_from is None:
task_label = f"before-{before_days}"
else:
task_label = "custom"
start_time = datetime.datetime.now(datetime.UTC)
click.echo(click.style(f"Starting workflow run cleanup at {start_time.isoformat()}.", fg="white"))
WorkflowRunCleanup(
days=before_days,
batch_size=batch_size,
start_from=start_from,
end_before=end_before,
dry_run=dry_run,
).run()
try:
WorkflowRunCleanup(
days=before_days,
batch_size=batch_size,
start_from=start_from,
end_before=end_before,
dry_run=dry_run,
task_label=task_label,
).run()
finally:
flush_telemetry()
end_time = datetime.datetime.now(datetime.UTC)
elapsed = end_time - start_time
@@ -659,6 +672,8 @@ def clean_expired_messages(
"""
Clean expired messages and related data for tenants based on clean policy.
"""
from extensions.otel.runtime import flush_telemetry
click.echo(click.style("clean_messages: start clean messages.", fg="green"))
start_at = time.perf_counter()
@@ -698,6 +713,13 @@ def clean_expired_messages(
# NOTE: graceful_period will be ignored when billing is disabled.
policy = create_message_clean_policy(graceful_period_days=graceful_period)
if from_days_ago is not None and before_days is not None:
task_label = f"{from_days_ago}to{before_days}"
elif start_from is None and before_days is not None:
task_label = f"before-{before_days}"
else:
task_label = "custom"
# Create and run the cleanup service
if abs_mode:
assert start_from is not None
@@ -708,6 +730,7 @@ def clean_expired_messages(
end_before=end_before,
batch_size=batch_size,
dry_run=dry_run,
task_label=task_label,
)
elif from_days_ago is None:
assert before_days is not None
@@ -716,6 +739,7 @@ def clean_expired_messages(
days=before_days,
batch_size=batch_size,
dry_run=dry_run,
task_label=task_label,
)
else:
assert before_days is not None
@@ -727,6 +751,7 @@ def clean_expired_messages(
end_before=now - datetime.timedelta(days=before_days),
batch_size=batch_size,
dry_run=dry_run,
task_label=task_label,
)
stats = service.run()
@@ -752,6 +777,8 @@ def clean_expired_messages(
)
)
raise
finally:
flush_telemetry()
click.echo(click.style("messages cleanup completed.", fg="green"))
+2 -2
View File
@@ -41,10 +41,10 @@ class RedisPubSubConfig(BaseSettings, RedisConfigDefaultsMixin):
)
PUBSUB_REDIS_USE_CLUSTERS: bool = Field(
validation_alias=AliasChoices("EVENT_BUS_REDIS_CLUSTERS", "PUBSUB_REDIS_USE_CLUSTERS"),
validation_alias=AliasChoices("EVENT_BUS_REDIS_USE_CLUSTERS", "PUBSUB_REDIS_USE_CLUSTERS"),
description=(
"Enable Redis Cluster mode for pub/sub or streams transport. Recommended for large deployments. "
"Also accepts ENV: EVENT_BUS_REDIS_CLUSTERS."
"Also accepts ENV: EVENT_BUS_REDIS_USE_CLUSTERS."
),
default=False,
)
@@ -23,7 +23,7 @@ from dify_graph.variables.types import SegmentType
from extensions.ext_database import db
from factories.file_factory import build_from_mapping, build_from_mappings
from factories.variable_factory import build_segment_with_type
from libs.login import login_required
from libs.login import current_user, login_required
from models import App, AppMode
from models.workflow import WorkflowDraftVariable
from services.workflow_draft_variable_service import WorkflowDraftVariableList, WorkflowDraftVariableService
@@ -100,6 +100,18 @@ def _serialize_full_content(variable: WorkflowDraftVariable) -> dict | None:
}
def _ensure_variable_access(
variable: WorkflowDraftVariable | None,
app_id: str,
variable_id: str,
) -> WorkflowDraftVariable:
if variable is None:
raise NotFoundError(description=f"variable not found, id={variable_id}")
if variable.app_id != app_id or variable.user_id != current_user.id:
raise NotFoundError(description=f"variable not found, id={variable_id}")
return variable
_WORKFLOW_DRAFT_VARIABLE_WITHOUT_VALUE_FIELDS = {
"id": fields.String,
"type": fields.String(attribute=lambda model: model.get_variable_type()),
@@ -238,6 +250,7 @@ class WorkflowVariableCollectionApi(Resource):
app_id=app_model.id,
page=args.page,
limit=args.limit,
user_id=current_user.id,
)
return workflow_vars
@@ -250,7 +263,7 @@ class WorkflowVariableCollectionApi(Resource):
draft_var_srv = WorkflowDraftVariableService(
session=db.session(),
)
draft_var_srv.delete_workflow_variables(app_model.id)
draft_var_srv.delete_user_workflow_variables(app_model.id, user_id=current_user.id)
db.session.commit()
return Response("", 204)
@@ -287,7 +300,7 @@ class NodeVariableCollectionApi(Resource):
draft_var_srv = WorkflowDraftVariableService(
session=session,
)
node_vars = draft_var_srv.list_node_variables(app_model.id, node_id)
node_vars = draft_var_srv.list_node_variables(app_model.id, node_id, user_id=current_user.id)
return node_vars
@@ -298,7 +311,7 @@ class NodeVariableCollectionApi(Resource):
def delete(self, app_model: App, node_id: str):
validate_node_id(node_id)
srv = WorkflowDraftVariableService(db.session())
srv.delete_node_variables(app_model.id, node_id)
srv.delete_node_variables(app_model.id, node_id, user_id=current_user.id)
db.session.commit()
return Response("", 204)
@@ -319,11 +332,11 @@ class VariableApi(Resource):
draft_var_srv = WorkflowDraftVariableService(
session=db.session(),
)
variable = draft_var_srv.get_variable(variable_id=variable_id)
if variable is None:
raise NotFoundError(description=f"variable not found, id={variable_id}")
if variable.app_id != app_model.id:
raise NotFoundError(description=f"variable not found, id={variable_id}")
variable = _ensure_variable_access(
variable=draft_var_srv.get_variable(variable_id=variable_id),
app_id=app_model.id,
variable_id=variable_id,
)
return variable
@console_ns.doc("update_variable")
@@ -360,11 +373,11 @@ class VariableApi(Resource):
)
args_model = WorkflowDraftVariableUpdatePayload.model_validate(console_ns.payload or {})
variable = draft_var_srv.get_variable(variable_id=variable_id)
if variable is None:
raise NotFoundError(description=f"variable not found, id={variable_id}")
if variable.app_id != app_model.id:
raise NotFoundError(description=f"variable not found, id={variable_id}")
variable = _ensure_variable_access(
variable=draft_var_srv.get_variable(variable_id=variable_id),
app_id=app_model.id,
variable_id=variable_id,
)
new_name = args_model.name
raw_value = args_model.value
@@ -397,11 +410,11 @@ class VariableApi(Resource):
draft_var_srv = WorkflowDraftVariableService(
session=db.session(),
)
variable = draft_var_srv.get_variable(variable_id=variable_id)
if variable is None:
raise NotFoundError(description=f"variable not found, id={variable_id}")
if variable.app_id != app_model.id:
raise NotFoundError(description=f"variable not found, id={variable_id}")
variable = _ensure_variable_access(
variable=draft_var_srv.get_variable(variable_id=variable_id),
app_id=app_model.id,
variable_id=variable_id,
)
draft_var_srv.delete_variable(variable)
db.session.commit()
return Response("", 204)
@@ -427,11 +440,11 @@ class VariableResetApi(Resource):
raise NotFoundError(
f"Draft workflow not found, app_id={app_model.id}",
)
variable = draft_var_srv.get_variable(variable_id=variable_id)
if variable is None:
raise NotFoundError(description=f"variable not found, id={variable_id}")
if variable.app_id != app_model.id:
raise NotFoundError(description=f"variable not found, id={variable_id}")
variable = _ensure_variable_access(
variable=draft_var_srv.get_variable(variable_id=variable_id),
app_id=app_model.id,
variable_id=variable_id,
)
resetted = draft_var_srv.reset_variable(draft_workflow, variable)
db.session.commit()
@@ -447,11 +460,15 @@ def _get_variable_list(app_model: App, node_id) -> WorkflowDraftVariableList:
session=session,
)
if node_id == CONVERSATION_VARIABLE_NODE_ID:
draft_vars = draft_var_srv.list_conversation_variables(app_model.id)
draft_vars = draft_var_srv.list_conversation_variables(app_model.id, user_id=current_user.id)
elif node_id == SYSTEM_VARIABLE_NODE_ID:
draft_vars = draft_var_srv.list_system_variables(app_model.id)
draft_vars = draft_var_srv.list_system_variables(app_model.id, user_id=current_user.id)
else:
draft_vars = draft_var_srv.list_node_variables(app_id=app_model.id, node_id=node_id)
draft_vars = draft_var_srv.list_node_variables(
app_id=app_model.id,
node_id=node_id,
user_id=current_user.id,
)
return draft_vars
@@ -472,7 +489,7 @@ class ConversationVariableCollectionApi(Resource):
if draft_workflow is None:
raise NotFoundError(description=f"draft workflow not found, id={app_model.id}")
draft_var_srv = WorkflowDraftVariableService(db.session())
draft_var_srv.prefill_conversation_variable_default_values(draft_workflow)
draft_var_srv.prefill_conversation_variable_default_values(draft_workflow, user_id=current_user.id)
db.session.commit()
return _get_variable_list(app_model, CONVERSATION_VARIABLE_NODE_ID)
@@ -102,6 +102,7 @@ class RagPipelineVariableCollectionApi(Resource):
app_id=pipeline.id,
page=query.page,
limit=query.limit,
user_id=current_user.id,
)
return workflow_vars
@@ -111,7 +112,7 @@ class RagPipelineVariableCollectionApi(Resource):
draft_var_srv = WorkflowDraftVariableService(
session=db.session(),
)
draft_var_srv.delete_workflow_variables(pipeline.id)
draft_var_srv.delete_user_workflow_variables(pipeline.id, user_id=current_user.id)
db.session.commit()
return Response("", 204)
@@ -144,7 +145,7 @@ class RagPipelineNodeVariableCollectionApi(Resource):
draft_var_srv = WorkflowDraftVariableService(
session=session,
)
node_vars = draft_var_srv.list_node_variables(pipeline.id, node_id)
node_vars = draft_var_srv.list_node_variables(pipeline.id, node_id, user_id=current_user.id)
return node_vars
@@ -152,7 +153,7 @@ class RagPipelineNodeVariableCollectionApi(Resource):
def delete(self, pipeline: Pipeline, node_id: str):
validate_node_id(node_id)
srv = WorkflowDraftVariableService(db.session())
srv.delete_node_variables(pipeline.id, node_id)
srv.delete_node_variables(pipeline.id, node_id, user_id=current_user.id)
db.session.commit()
return Response("", 204)
@@ -283,11 +284,11 @@ def _get_variable_list(pipeline: Pipeline, node_id) -> WorkflowDraftVariableList
session=session,
)
if node_id == CONVERSATION_VARIABLE_NODE_ID:
draft_vars = draft_var_srv.list_conversation_variables(pipeline.id)
draft_vars = draft_var_srv.list_conversation_variables(pipeline.id, user_id=current_user.id)
elif node_id == SYSTEM_VARIABLE_NODE_ID:
draft_vars = draft_var_srv.list_system_variables(pipeline.id)
draft_vars = draft_var_srv.list_system_variables(pipeline.id, user_id=current_user.id)
else:
draft_vars = draft_var_srv.list_node_variables(app_id=pipeline.id, node_id=node_id)
draft_vars = draft_var_srv.list_node_variables(app_id=pipeline.id, node_id=node_id, user_id=current_user.id)
return draft_vars
@@ -330,9 +330,10 @@ class AdvancedChatAppGenerator(MessageBasedAppGenerator):
engine=db.engine,
app_id=application_generate_entity.app_config.app_id,
tenant_id=application_generate_entity.app_config.tenant_id,
user_id=user.id,
)
draft_var_srv = WorkflowDraftVariableService(db.session())
draft_var_srv.prefill_conversation_variable_default_values(workflow)
draft_var_srv.prefill_conversation_variable_default_values(workflow, user_id=user.id)
return self._generate(
workflow=workflow,
@@ -413,9 +414,10 @@ class AdvancedChatAppGenerator(MessageBasedAppGenerator):
engine=db.engine,
app_id=application_generate_entity.app_config.app_id,
tenant_id=application_generate_entity.app_config.tenant_id,
user_id=user.id,
)
draft_var_srv = WorkflowDraftVariableService(db.session())
draft_var_srv.prefill_conversation_variable_default_values(workflow)
draft_var_srv.prefill_conversation_variable_default_values(workflow, user_id=user.id)
return self._generate(
workflow=workflow,
@@ -419,11 +419,12 @@ class PipelineGenerator(BaseAppGenerator):
triggered_from=WorkflowNodeExecutionTriggeredFrom.SINGLE_STEP,
)
draft_var_srv = WorkflowDraftVariableService(db.session())
draft_var_srv.prefill_conversation_variable_default_values(workflow)
draft_var_srv.prefill_conversation_variable_default_values(workflow, user_id=user.id)
var_loader = DraftVarLoader(
engine=db.engine,
app_id=application_generate_entity.app_config.app_id,
tenant_id=application_generate_entity.app_config.tenant_id,
user_id=user.id,
)
return self._generate(
@@ -514,11 +515,12 @@ class PipelineGenerator(BaseAppGenerator):
triggered_from=WorkflowNodeExecutionTriggeredFrom.SINGLE_STEP,
)
draft_var_srv = WorkflowDraftVariableService(db.session())
draft_var_srv.prefill_conversation_variable_default_values(workflow)
draft_var_srv.prefill_conversation_variable_default_values(workflow, user_id=user.id)
var_loader = DraftVarLoader(
engine=db.engine,
app_id=application_generate_entity.app_config.app_id,
tenant_id=application_generate_entity.app_config.tenant_id,
user_id=user.id,
)
return self._generate(
+4 -2
View File
@@ -414,11 +414,12 @@ class WorkflowAppGenerator(BaseAppGenerator):
triggered_from=WorkflowNodeExecutionTriggeredFrom.SINGLE_STEP,
)
draft_var_srv = WorkflowDraftVariableService(db.session())
draft_var_srv.prefill_conversation_variable_default_values(workflow)
draft_var_srv.prefill_conversation_variable_default_values(workflow, user_id=user.id)
var_loader = DraftVarLoader(
engine=db.engine,
app_id=application_generate_entity.app_config.app_id,
tenant_id=application_generate_entity.app_config.tenant_id,
user_id=user.id,
)
return self._generate(
@@ -497,11 +498,12 @@ class WorkflowAppGenerator(BaseAppGenerator):
triggered_from=WorkflowNodeExecutionTriggeredFrom.SINGLE_STEP,
)
draft_var_srv = WorkflowDraftVariableService(db.session())
draft_var_srv.prefill_conversation_variable_default_values(workflow)
draft_var_srv.prefill_conversation_variable_default_values(workflow, user_id=user.id)
var_loader = DraftVarLoader(
engine=db.engine,
app_id=application_generate_entity.app_config.app_id,
tenant_id=application_generate_entity.app_config.tenant_id,
user_id=user.id,
)
return self._generate(
app_model=app_model,
+6 -4
View File
@@ -5,6 +5,7 @@ import re
import threading
import time
import uuid
from collections.abc import Mapping
from typing import Any
from flask import Flask, current_app
@@ -37,7 +38,7 @@ from extensions.ext_storage import storage
from libs import helper
from libs.datetime_utils import naive_utc_now
from models import Account
from models.dataset import ChildChunk, Dataset, DatasetProcessRule, DocumentSegment
from models.dataset import AutomaticRulesConfig, ChildChunk, Dataset, DatasetProcessRule, DocumentSegment
from models.dataset import Document as DatasetDocument
from models.model import UploadFile
from services.feature_service import FeatureService
@@ -265,7 +266,7 @@ class IndexingRunner:
self,
tenant_id: str,
extract_settings: list[ExtractSetting],
tmp_processing_rule: dict,
tmp_processing_rule: Mapping[str, Any],
doc_form: str | None = None,
doc_language: str = "English",
dataset_id: str | None = None,
@@ -376,7 +377,7 @@ class IndexingRunner:
return IndexingEstimate(total_segments=total_segments, preview=preview_texts)
def _extract(
self, index_processor: BaseIndexProcessor, dataset_document: DatasetDocument, process_rule: dict
self, index_processor: BaseIndexProcessor, dataset_document: DatasetDocument, process_rule: Mapping[str, Any]
) -> list[Document]:
data_source_info = dataset_document.data_source_info_dict
text_docs = []
@@ -543,6 +544,7 @@ class IndexingRunner:
"""
Clean the document text according to the processing rules.
"""
rules: AutomaticRulesConfig | dict[str, Any]
if processing_rule.mode == "automatic":
rules = DatasetProcessRule.AUTOMATIC_RULES
else:
@@ -756,7 +758,7 @@ class IndexingRunner:
dataset: Dataset,
text_docs: list[Document],
doc_language: str,
process_rule: dict,
process_rule: Mapping[str, Any],
current_user: Account | None = None,
) -> list[Document]:
# get embedding model instance
+21 -5
View File
@@ -55,15 +55,31 @@ def build_protected_resource_metadata_discovery_urls(
"""
urls = []
parsed_server_url = urlparse(server_url)
base_url = f"{parsed_server_url.scheme}://{parsed_server_url.netloc}"
path = parsed_server_url.path.rstrip("/")
# First priority: URL from WWW-Authenticate header
if www_auth_resource_metadata_url:
urls.append(www_auth_resource_metadata_url)
parsed_metadata_url = urlparse(www_auth_resource_metadata_url)
normalized_metadata_url = None
if parsed_metadata_url.scheme and parsed_metadata_url.netloc:
normalized_metadata_url = www_auth_resource_metadata_url
elif not parsed_metadata_url.scheme and parsed_metadata_url.netloc:
normalized_metadata_url = f"{parsed_server_url.scheme}:{www_auth_resource_metadata_url}"
elif (
not parsed_metadata_url.scheme
and not parsed_metadata_url.netloc
and parsed_metadata_url.path.startswith("/")
):
first_segment = parsed_metadata_url.path.lstrip("/").split("/", 1)[0]
if first_segment == ".well-known" or "." not in first_segment:
normalized_metadata_url = urljoin(base_url, parsed_metadata_url.path)
if normalized_metadata_url:
urls.append(normalized_metadata_url)
# Fallback: construct from server URL
parsed = urlparse(server_url)
base_url = f"{parsed.scheme}://{parsed.netloc}"
path = parsed.path.rstrip("/")
# Priority 2: With path insertion (e.g., /.well-known/oauth-protected-resource/public/mcp)
if path:
path_url = f"{base_url}/.well-known/oauth-protected-resource{path}"
@@ -8,6 +8,7 @@ document embeddings used in retrieval-augmented generation workflows.
import datetime
import json
import logging
import threading
import uuid as _uuid
from typing import Any
from urllib.parse import urlparse
@@ -32,6 +33,9 @@ from models.dataset import Dataset
logger = logging.getLogger(__name__)
_weaviate_client: weaviate.WeaviateClient | None = None
_weaviate_client_lock = threading.Lock()
class WeaviateConfig(BaseModel):
"""
@@ -99,43 +103,52 @@ class WeaviateVector(BaseVector):
Configures both HTTP and gRPC connections with proper authentication.
"""
p = urlparse(config.endpoint)
host = p.hostname or config.endpoint.replace("https://", "").replace("http://", "")
http_secure = p.scheme == "https"
http_port = p.port or (443 if http_secure else 80)
global _weaviate_client
if _weaviate_client and _weaviate_client.is_ready():
return _weaviate_client
# Parse gRPC configuration
if config.grpc_endpoint:
# Urls without scheme won't be parsed correctly in some python versions,
# see https://bugs.python.org/issue27657
grpc_endpoint_with_scheme = (
config.grpc_endpoint if "://" in config.grpc_endpoint else f"grpc://{config.grpc_endpoint}"
with _weaviate_client_lock:
if _weaviate_client and _weaviate_client.is_ready():
return _weaviate_client
p = urlparse(config.endpoint)
host = p.hostname or config.endpoint.replace("https://", "").replace("http://", "")
http_secure = p.scheme == "https"
http_port = p.port or (443 if http_secure else 80)
# Parse gRPC configuration
if config.grpc_endpoint:
# Urls without scheme won't be parsed correctly in some python versions,
# see https://bugs.python.org/issue27657
grpc_endpoint_with_scheme = (
config.grpc_endpoint if "://" in config.grpc_endpoint else f"grpc://{config.grpc_endpoint}"
)
grpc_p = urlparse(grpc_endpoint_with_scheme)
grpc_host = grpc_p.hostname or "localhost"
grpc_port = grpc_p.port or (443 if grpc_p.scheme == "grpcs" else 50051)
grpc_secure = grpc_p.scheme == "grpcs"
else:
# Infer from HTTP endpoint as fallback
grpc_host = host
grpc_secure = http_secure
grpc_port = 443 if grpc_secure else 50051
client = weaviate.connect_to_custom(
http_host=host,
http_port=http_port,
http_secure=http_secure,
grpc_host=grpc_host,
grpc_port=grpc_port,
grpc_secure=grpc_secure,
auth_credentials=Auth.api_key(config.api_key) if config.api_key else None,
skip_init_checks=True, # Skip PyPI version check to avoid unnecessary HTTP requests
)
grpc_p = urlparse(grpc_endpoint_with_scheme)
grpc_host = grpc_p.hostname or "localhost"
grpc_port = grpc_p.port or (443 if grpc_p.scheme == "grpcs" else 50051)
grpc_secure = grpc_p.scheme == "grpcs"
else:
# Infer from HTTP endpoint as fallback
grpc_host = host
grpc_secure = http_secure
grpc_port = 443 if grpc_secure else 50051
client = weaviate.connect_to_custom(
http_host=host,
http_port=http_port,
http_secure=http_secure,
grpc_host=grpc_host,
grpc_port=grpc_port,
grpc_secure=grpc_secure,
auth_credentials=Auth.api_key(config.api_key) if config.api_key else None,
skip_init_checks=True, # Skip PyPI version check to avoid unnecessary HTTP requests
)
if not client.is_ready():
raise ConnectionError("Vector database is not ready")
if not client.is_ready():
raise ConnectionError("Vector database is not ready")
return client
_weaviate_client = client
return client
def get_type(self) -> str:
"""Returns the vector database type identifier."""
@@ -6,6 +6,5 @@ of responses based on upstream node outputs and constants.
"""
from .coordinator import ResponseStreamCoordinator
from .session import RESPONSE_SESSION_NODE_TYPES
__all__ = ["RESPONSE_SESSION_NODE_TYPES", "ResponseStreamCoordinator"]
__all__ = ["ResponseStreamCoordinator"]
@@ -3,10 +3,6 @@ Internal response session management for response coordinator.
This module contains the private ResponseSession class used internally
by ResponseStreamCoordinator to manage streaming sessions.
`RESPONSE_SESSION_NODE_TYPES` is intentionally mutable so downstream applications
can opt additional response-capable node types into session creation without
patching the coordinator.
"""
from __future__ import annotations
@@ -14,7 +10,6 @@ from __future__ import annotations
from dataclasses import dataclass
from typing import Protocol, cast
from dify_graph.enums import BuiltinNodeTypes, NodeType
from dify_graph.nodes.base.template import Template
from dify_graph.runtime.graph_runtime_state import NodeProtocol
@@ -25,12 +20,6 @@ class _ResponseSessionNodeProtocol(NodeProtocol, Protocol):
def get_streaming_template(self) -> Template: ...
RESPONSE_SESSION_NODE_TYPES: list[NodeType] = [
BuiltinNodeTypes.ANSWER,
BuiltinNodeTypes.END,
]
@dataclass
class ResponseSession:
"""
@@ -49,8 +38,8 @@ class ResponseSession:
Create a ResponseSession from a response-capable node.
The parameter is typed as `NodeProtocol` because the graph is exposed behind a protocol at the runtime layer.
At runtime this must be a node whose `node_type` is listed in `RESPONSE_SESSION_NODE_TYPES`
and which implements `get_streaming_template()`.
At runtime this must be a node that implements `get_streaming_template()`. The coordinator decides which
graph nodes should be treated as response-capable before they reach this factory.
Args:
node: Node from the materialized workflow graph.
@@ -59,15 +48,8 @@ class ResponseSession:
ResponseSession configured with the node's streaming template
Raises:
TypeError: If node is not a supported response node type.
TypeError: If node does not implement the response-session streaming contract.
"""
if node.node_type not in RESPONSE_SESSION_NODE_TYPES:
supported_node_types = ", ".join(RESPONSE_SESSION_NODE_TYPES)
raise TypeError(
"ResponseSession.from_node only supports node types in "
f"RESPONSE_SESSION_NODE_TYPES: {supported_node_types}"
)
response_node = cast(_ResponseSessionNodeProtocol, node)
try:
template = response_node.get_streaming_template()
@@ -8,6 +8,8 @@ from collections.abc import Mapping, Sequence
from datetime import datetime, timedelta
from typing import Annotated, Any, ClassVar, Literal, Self
import bleach
import markdown
from pydantic import BaseModel, Field, field_validator, model_validator
from dify_graph.entities.base_node_data import BaseNodeData
@@ -58,6 +60,39 @@ class EmailDeliveryConfig(BaseModel):
"""Configuration for email delivery method."""
URL_PLACEHOLDER: ClassVar[str] = "{{#url#}}"
_SUBJECT_NEWLINE_PATTERN: ClassVar[re.Pattern[str]] = re.compile(r"[\r\n]+")
_ALLOWED_HTML_TAGS: ClassVar[list[str]] = [
"a",
"blockquote",
"br",
"code",
"em",
"h1",
"h2",
"h3",
"h4",
"h5",
"h6",
"hr",
"li",
"ol",
"p",
"pre",
"strong",
"table",
"tbody",
"td",
"th",
"thead",
"tr",
"ul",
]
_ALLOWED_HTML_ATTRIBUTES: ClassVar[dict[str, list[str]]] = {
"a": ["href", "title"],
"td": ["align"],
"th": ["align"],
}
_ALLOWED_PROTOCOLS: ClassVar[list[str]] = ["http", "https", "mailto"]
recipients: EmailRecipients
@@ -98,6 +133,43 @@ class EmailDeliveryConfig(BaseModel):
return templated_body
return variable_pool.convert_template(templated_body).text
@classmethod
def render_markdown_body(cls, body: str) -> str:
"""Render markdown to safe HTML for email delivery."""
sanitized_markdown = bleach.clean(
body,
tags=[],
attributes={},
strip=True,
strip_comments=True,
)
rendered_html = markdown.markdown(
sanitized_markdown,
extensions=["nl2br", "tables"],
extension_configs={"tables": {"use_align_attribute": True}},
)
return bleach.clean(
rendered_html,
tags=cls._ALLOWED_HTML_TAGS,
attributes=cls._ALLOWED_HTML_ATTRIBUTES,
protocols=cls._ALLOWED_PROTOCOLS,
strip=True,
strip_comments=True,
)
@classmethod
def sanitize_subject(cls, subject: str) -> str:
"""Sanitize email subject to plain text and prevent CRLF injection."""
sanitized_subject = bleach.clean(
subject,
tags=[],
attributes={},
strip=True,
strip_comments=True,
)
sanitized_subject = cls._SUBJECT_NEWLINE_PATTERN.sub(" ", sanitized_subject)
return " ".join(sanitized_subject.split())
class _DeliveryMethodBase(BaseModel):
"""Base delivery method configuration."""
+22 -2
View File
@@ -5,7 +5,7 @@ from typing import Union
from celery.signals import worker_init
from flask_login import user_loaded_from_request, user_logged_in
from opentelemetry import trace
from opentelemetry import metrics, trace
from opentelemetry.propagate import set_global_textmap
from opentelemetry.propagators.b3 import B3MultiFormat
from opentelemetry.propagators.composite import CompositePropagator
@@ -31,9 +31,29 @@ def setup_context_propagation() -> None:
def shutdown_tracer() -> None:
flush_telemetry()
def flush_telemetry() -> None:
"""
Best-effort flush for telemetry providers.
This is mainly used by short-lived command processes (e.g. Kubernetes CronJob)
so counters/histograms are exported before the process exits.
"""
provider = trace.get_tracer_provider()
if hasattr(provider, "force_flush"):
provider.force_flush()
try:
provider.force_flush()
except Exception:
logger.exception("otel: failed to flush trace provider")
metric_provider = metrics.get_meter_provider()
if hasattr(metric_provider, "force_flush"):
try:
metric_provider.force_flush()
except Exception:
logger.exception("otel: failed to flush metric provider")
def is_celery_worker():
@@ -0,0 +1,69 @@
"""add user_id and switch workflow_draft_variables unique key to user scope
Revision ID: 6b5f9f8b1a2c
Revises: 0ec65df55790
Create Date: 2026-03-04 16:00:00.000000
"""
import sqlalchemy as sa
from alembic import op
import models as models
# revision identifiers, used by Alembic.
revision = "6b5f9f8b1a2c"
down_revision = "0ec65df55790"
branch_labels = None
depends_on = None
def _is_pg(conn) -> bool:
return conn.dialect.name == "postgresql"
def upgrade():
conn = op.get_bind()
table_name = "workflow_draft_variables"
with op.batch_alter_table(table_name, schema=None) as batch_op:
batch_op.add_column(sa.Column("user_id", models.types.StringUUID(), nullable=True))
if _is_pg(conn):
with op.get_context().autocommit_block():
op.create_index(
"workflow_draft_variables_app_id_user_id_key",
"workflow_draft_variables",
["app_id", "user_id", "node_id", "name"],
unique=True,
postgresql_concurrently=True,
)
else:
op.create_index(
"workflow_draft_variables_app_id_user_id_key",
"workflow_draft_variables",
["app_id", "user_id", "node_id", "name"],
unique=True,
)
with op.batch_alter_table(table_name, schema=None) as batch_op:
batch_op.drop_constraint(op.f("workflow_draft_variables_app_id_key"), type_="unique")
def downgrade():
conn = op.get_bind()
with op.batch_alter_table("workflow_draft_variables", schema=None) as batch_op:
batch_op.create_unique_constraint(
op.f("workflow_draft_variables_app_id_key"),
["app_id", "node_id", "name"],
)
if _is_pg(conn):
with op.get_context().autocommit_block():
op.drop_index("workflow_draft_variables_app_id_user_id_key", postgresql_concurrently=True)
else:
op.drop_index("workflow_draft_variables_app_id_user_id_key", table_name="workflow_draft_variables")
with op.batch_alter_table("workflow_draft_variables", schema=None) as batch_op:
batch_op.drop_column("user_id")
+69 -14
View File
@@ -10,7 +10,7 @@ import re
import time
from datetime import datetime
from json import JSONDecodeError
from typing import Any, cast
from typing import Any, TypedDict, cast
from uuid import uuid4
import sqlalchemy as sa
@@ -37,6 +37,61 @@ from .types import AdjustedJSON, BinaryData, EnumText, LongText, StringUUID, adj
logger = logging.getLogger(__name__)
class PreProcessingRuleItem(TypedDict):
id: str
enabled: bool
class SegmentationConfig(TypedDict):
delimiter: str
max_tokens: int
chunk_overlap: int
class AutomaticRulesConfig(TypedDict):
pre_processing_rules: list[PreProcessingRuleItem]
segmentation: SegmentationConfig
class ProcessRuleDict(TypedDict):
id: str
dataset_id: str
mode: str
rules: dict[str, Any] | None
class DocMetadataDetailItem(TypedDict):
id: str
name: str
type: str
value: Any
class AttachmentItem(TypedDict):
id: str
name: str
size: int
extension: str
mime_type: str
source_url: str
class DatasetBindingItem(TypedDict):
id: str
name: str
class ExternalKnowledgeApiDict(TypedDict):
id: str
tenant_id: str
name: str
description: str
settings: dict[str, Any] | None
dataset_bindings: list[DatasetBindingItem]
created_by: str
created_at: str
class DatasetPermissionEnum(enum.StrEnum):
ONLY_ME = "only_me"
ALL_TEAM = "all_team_members"
@@ -334,7 +389,7 @@ class DatasetProcessRule(Base): # bug
MODES = ["automatic", "custom", "hierarchical"]
PRE_PROCESSING_RULES = ["remove_stopwords", "remove_extra_spaces", "remove_urls_emails"]
AUTOMATIC_RULES: dict[str, Any] = {
AUTOMATIC_RULES: AutomaticRulesConfig = {
"pre_processing_rules": [
{"id": "remove_extra_spaces", "enabled": True},
{"id": "remove_urls_emails", "enabled": False},
@@ -342,7 +397,7 @@ class DatasetProcessRule(Base): # bug
"segmentation": {"delimiter": "\n", "max_tokens": 500, "chunk_overlap": 50},
}
def to_dict(self) -> dict[str, Any]:
def to_dict(self) -> ProcessRuleDict:
return {
"id": self.id,
"dataset_id": self.dataset_id,
@@ -531,7 +586,7 @@ class Document(Base):
return self.updated_at
@property
def doc_metadata_details(self) -> list[dict[str, Any]] | None:
def doc_metadata_details(self) -> list[DocMetadataDetailItem] | None:
if self.doc_metadata:
document_metadatas = (
db.session.query(DatasetMetadata)
@@ -541,9 +596,9 @@ class Document(Base):
)
.all()
)
metadata_list: list[dict[str, Any]] = []
metadata_list: list[DocMetadataDetailItem] = []
for metadata in document_metadatas:
metadata_dict: dict[str, Any] = {
metadata_dict: DocMetadataDetailItem = {
"id": metadata.id,
"name": metadata.name,
"type": metadata.type,
@@ -557,13 +612,13 @@ class Document(Base):
return None
@property
def process_rule_dict(self) -> dict[str, Any] | None:
def process_rule_dict(self) -> ProcessRuleDict | None:
if self.dataset_process_rule_id and self.dataset_process_rule:
return self.dataset_process_rule.to_dict()
return None
def get_built_in_fields(self) -> list[dict[str, Any]]:
built_in_fields: list[dict[str, Any]] = []
def get_built_in_fields(self) -> list[DocMetadataDetailItem]:
built_in_fields: list[DocMetadataDetailItem] = []
built_in_fields.append(
{
"id": "built-in",
@@ -877,7 +932,7 @@ class DocumentSegment(Base):
return text
@property
def attachments(self) -> list[dict[str, Any]]:
def attachments(self) -> list[AttachmentItem]:
# Use JOIN to fetch attachments in a single query instead of two separate queries
attachments_with_bindings = db.session.execute(
select(SegmentAttachmentBinding, UploadFile)
@@ -891,7 +946,7 @@ class DocumentSegment(Base):
).all()
if not attachments_with_bindings:
return []
attachment_list = []
attachment_list: list[AttachmentItem] = []
for _, attachment in attachments_with_bindings:
upload_file_id = attachment.id
nonce = os.urandom(16).hex()
@@ -1261,7 +1316,7 @@ class ExternalKnowledgeApis(TypeBase):
DateTime, nullable=False, server_default=func.current_timestamp(), onupdate=func.current_timestamp(), init=False
)
def to_dict(self) -> dict[str, Any]:
def to_dict(self) -> ExternalKnowledgeApiDict:
return {
"id": self.id,
"tenant_id": self.tenant_id,
@@ -1281,13 +1336,13 @@ class ExternalKnowledgeApis(TypeBase):
return None
@property
def dataset_bindings(self) -> list[dict[str, Any]]:
def dataset_bindings(self) -> list[DatasetBindingItem]:
external_knowledge_bindings = db.session.scalars(
select(ExternalKnowledgeBindings).where(ExternalKnowledgeBindings.external_knowledge_api_id == self.id)
).all()
dataset_ids = [binding.dataset_id for binding in external_knowledge_bindings]
datasets = db.session.scalars(select(Dataset).where(Dataset.id.in_(dataset_ids))).all()
dataset_bindings: list[dict[str, Any]] = []
dataset_bindings: list[DatasetBindingItem] = []
for dataset in datasets:
dataset_bindings.append({"id": dataset.id, "name": dataset.name})
+41 -3
View File
@@ -3,7 +3,7 @@ import time
from collections.abc import Mapping
from datetime import datetime
from functools import cached_property
from typing import Any, cast
from typing import Any, TypedDict, cast
from uuid import uuid4
import sqlalchemy as sa
@@ -24,6 +24,44 @@ from .model import Account
from .types import EnumText, LongText, StringUUID
class WorkflowTriggerLogDict(TypedDict):
id: str
tenant_id: str
app_id: str
workflow_id: str
workflow_run_id: str | None
root_node_id: str | None
trigger_metadata: Any
trigger_type: str
trigger_data: Any
inputs: Any
outputs: Any
status: str
error: str | None
queue_name: str
celery_task_id: str | None
retry_count: int
elapsed_time: float | None
total_tokens: int | None
created_by_role: str
created_by: str
created_at: str | None
triggered_at: str | None
finished_at: str | None
class WorkflowSchedulePlanDict(TypedDict):
id: str
app_id: str
node_id: str
tenant_id: str
cron_expression: str
timezone: str
next_run_at: str | None
created_at: str
updated_at: str
class TriggerSubscription(TypeBase):
"""
Trigger provider model for managing credentials
@@ -250,7 +288,7 @@ class WorkflowTriggerLog(TypeBase):
created_by_role = CreatorUserRole(self.created_by_role)
return db.session.get(EndUser, self.created_by) if created_by_role == CreatorUserRole.END_USER else None
def to_dict(self) -> dict[str, Any]:
def to_dict(self) -> WorkflowTriggerLogDict:
"""Convert to dictionary for API responses"""
return {
"id": self.id,
@@ -481,7 +519,7 @@ class WorkflowSchedulePlan(TypeBase):
DateTime, nullable=False, server_default=func.current_timestamp(), onupdate=func.current_timestamp(), init=False
)
def to_dict(self) -> dict[str, Any]:
def to_dict(self) -> WorkflowSchedulePlanDict:
"""Convert to dictionary representation"""
return {
"id": self.id,
+36 -6
View File
@@ -3,7 +3,7 @@ import logging
from collections.abc import Generator, Mapping, Sequence
from datetime import datetime
from enum import StrEnum
from typing import TYPE_CHECKING, Any, Optional, Union, cast
from typing import TYPE_CHECKING, Any, Optional, TypedDict, Union, cast
from uuid import uuid4
import sqlalchemy as sa
@@ -60,6 +60,22 @@ from .types import EnumText, LongText, StringUUID
logger = logging.getLogger(__name__)
class WorkflowContentDict(TypedDict):
graph: Mapping[str, Any]
features: dict[str, Any]
environment_variables: list[dict[str, Any]]
conversation_variables: list[dict[str, Any]]
rag_pipeline_variables: list[dict[str, Any]]
class WorkflowRunSummaryDict(TypedDict):
id: str
status: str
triggered_from: str
elapsed_time: float
total_tokens: int
class WorkflowType(StrEnum):
"""
Workflow Type Enum
@@ -502,14 +518,14 @@ class Workflow(Base): # bug
)
self._environment_variables = environment_variables_json
def to_dict(self, *, include_secret: bool = False) -> Mapping[str, Any]:
def to_dict(self, *, include_secret: bool = False) -> WorkflowContentDict:
environment_variables = list(self.environment_variables)
environment_variables = [
v if not isinstance(v, SecretVariable) or include_secret else v.model_copy(update={"value": ""})
for v in environment_variables
]
result = {
result: WorkflowContentDict = {
"graph": self.graph_dict,
"features": self.features_dict,
"environment_variables": [var.model_dump(mode="json") for var in environment_variables],
@@ -1231,7 +1247,7 @@ class WorkflowArchiveLog(TypeBase):
)
@property
def workflow_run_summary(self) -> dict[str, Any]:
def workflow_run_summary(self) -> WorkflowRunSummaryDict:
return {
"id": self.workflow_run_id,
"status": self.run_status,
@@ -1286,16 +1302,17 @@ class WorkflowDraftVariable(Base):
"""
@staticmethod
def unique_app_id_node_id_name() -> list[str]:
def unique_app_id_user_id_node_id_name() -> list[str]:
return [
"app_id",
"user_id",
"node_id",
"name",
]
__tablename__ = "workflow_draft_variables"
__table_args__ = (
UniqueConstraint(*unique_app_id_node_id_name()),
UniqueConstraint(*unique_app_id_user_id_node_id_name()),
Index("workflow_draft_variable_file_id_idx", "file_id"),
)
# Required for instance variable annotation.
@@ -1321,6 +1338,11 @@ class WorkflowDraftVariable(Base):
# "`app_id` maps to the `id` field in the `model.App` model."
app_id: Mapped[str] = mapped_column(StringUUID, nullable=False)
# Owner of this draft variable.
#
# This field is nullable during migration and will be migrated to NOT NULL
# in a follow-up release.
user_id: Mapped[str | None] = mapped_column(StringUUID, nullable=True, default=None)
# `last_edited_at` records when the value of a given draft variable
# is edited.
@@ -1573,6 +1595,7 @@ class WorkflowDraftVariable(Base):
cls,
*,
app_id: str,
user_id: str | None,
node_id: str,
name: str,
value: Segment,
@@ -1586,6 +1609,7 @@ class WorkflowDraftVariable(Base):
variable.updated_at = naive_utc_now()
variable.description = description
variable.app_id = app_id
variable.user_id = user_id
variable.node_id = node_id
variable.name = name
variable.set_value(value)
@@ -1599,12 +1623,14 @@ class WorkflowDraftVariable(Base):
cls,
*,
app_id: str,
user_id: str | None = None,
name: str,
value: Segment,
description: str = "",
) -> "WorkflowDraftVariable":
variable = cls._new(
app_id=app_id,
user_id=user_id,
node_id=CONVERSATION_VARIABLE_NODE_ID,
name=name,
value=value,
@@ -1619,6 +1645,7 @@ class WorkflowDraftVariable(Base):
cls,
*,
app_id: str,
user_id: str | None = None,
name: str,
value: Segment,
node_execution_id: str,
@@ -1626,6 +1653,7 @@ class WorkflowDraftVariable(Base):
) -> "WorkflowDraftVariable":
variable = cls._new(
app_id=app_id,
user_id=user_id,
node_id=SYSTEM_VARIABLE_NODE_ID,
name=name,
node_execution_id=node_execution_id,
@@ -1639,6 +1667,7 @@ class WorkflowDraftVariable(Base):
cls,
*,
app_id: str,
user_id: str | None = None,
node_id: str,
name: str,
value: Segment,
@@ -1649,6 +1678,7 @@ class WorkflowDraftVariable(Base):
) -> "WorkflowDraftVariable":
variable = cls._new(
app_id=app_id,
user_id=user_id,
node_id=node_id,
name=name,
node_execution_id=node_execution_id,
+19 -21
View File
@@ -1,6 +1,6 @@
[project]
name = "dify-api"
version = "1.13.0"
version = "1.13.1"
requires-python = ">=3.11,<3.13"
dependencies = [
@@ -40,7 +40,7 @@ dependencies = [
"numpy~=1.26.4",
"openpyxl~=3.1.5",
"opik~=1.10.37",
"litellm==1.82.2", # Pinned to avoid madoka dependency issue
"litellm==1.82.2", # Pinned to avoid madoka dependency issue
"opentelemetry-api==1.28.0",
"opentelemetry-distro==0.49b0",
"opentelemetry-exporter-otlp==1.28.0",
@@ -87,10 +87,11 @@ dependencies = [
"flask-restx~=1.3.2",
"packaging~=23.2",
"croniter>=6.0.0",
"weaviate-client==4.17.0",
"weaviate-client==4.20.4",
"apscheduler>=3.11.0",
"weave>=0.52.16",
"fastopenapi[flask]>=0.7.0",
"bleach~=6.2.0",
]
# Before adding new dependency, consider place it in
# alphabet order (a-z) and suitable group.
@@ -118,7 +119,7 @@ dev = [
"pytest~=9.0.2",
"pytest-benchmark~=5.2.3",
"pytest-cov~=7.0.0",
"pytest-env~=1.1.3",
"pytest-env~=1.6.0",
"pytest-mock~=3.15.1",
"testcontainers~=4.14.1",
"types-aiofiles~=25.1.0",
@@ -202,28 +203,28 @@ tools = ["cloudscraper~=1.2.71", "nltk~=3.9.1"]
############################################################
vdb = [
"alibabacloud_gpdb20160503~=3.8.0",
"alibabacloud_tea_openapi~=0.3.9",
"alibabacloud_tea_openapi~=0.4.3",
"chromadb==0.5.20",
"clickhouse-connect~=0.10.0",
"clickhouse-connect~=0.14.1",
"clickzetta-connector-python>=0.8.102",
"couchbase~=4.3.0",
"couchbase~=4.5.0",
"elasticsearch==8.14.0",
"opensearch-py==3.1.0",
"oracledb==3.3.0",
"oracledb==3.4.2",
"pgvecto-rs[sqlalchemy]~=0.2.1",
"pgvector==0.2.5",
"pymilvus~=2.5.0",
"pymochow==2.2.9",
"pgvector==0.4.2",
"pymilvus~=2.6.10",
"pymochow==2.3.6",
"pyobvector~=0.2.17",
"qdrant-client==1.9.0",
"intersystems-irispython>=5.1.0",
"tablestore==6.3.7",
"tcvectordb~=1.6.4",
"tidb-vector==0.0.9",
"upstash-vector==0.6.0",
"tablestore==6.4.1",
"tcvectordb~=2.0.0",
"tidb-vector==0.0.15",
"upstash-vector==0.8.0",
"volcengine-compat~=1.0.0",
"weaviate-client==4.17.0",
"xinference-client~=1.2.2",
"weaviate-client==4.20.4",
"xinference-client~=2.3.1",
"mo-vector~=0.1.13",
"mysql-connector-python>=9.3.0",
"holo-search-sdk>=0.4.1",
@@ -251,10 +252,7 @@ ignore_errors = true
[tool.pyrefly]
project-includes = ["."]
project-excludes = [
".venv",
"migrations/",
]
project-excludes = [".venv", "migrations/"]
python-platform = "linux"
python-version = "3.11.0"
infer-with-first-use = false
+1 -1
View File
@@ -304,7 +304,7 @@ class AppDslService:
)
draft_var_srv = WorkflowDraftVariableService(session=self._session)
draft_var_srv.delete_workflow_variables(app_id=app.id)
draft_var_srv.delete_app_workflow_variables(app_id=app.id)
return Import(
id=import_id,
status=status,
+6 -4
View File
@@ -18,7 +18,7 @@ from extensions.ext_database import db
from models.account import Account
from models.enums import CreatorUserRole, WorkflowTriggerStatus
from models.model import App, EndUser
from models.trigger import WorkflowTriggerLog
from models.trigger import WorkflowTriggerLog, WorkflowTriggerLogDict
from models.workflow import Workflow
from repositories.sqlalchemy_workflow_trigger_log_repository import SQLAlchemyWorkflowTriggerLogRepository
from services.errors.app import QuotaExceededError, WorkflowNotFoundError, WorkflowQuotaLimitError
@@ -224,7 +224,9 @@ class AsyncWorkflowService:
return cls.trigger_workflow_async(session, user, trigger_data)
@classmethod
def get_trigger_log(cls, workflow_trigger_log_id: str, tenant_id: str | None = None) -> dict[str, Any] | None:
def get_trigger_log(
cls, workflow_trigger_log_id: str, tenant_id: str | None = None
) -> WorkflowTriggerLogDict | None:
"""
Get trigger log by ID
@@ -247,7 +249,7 @@ class AsyncWorkflowService:
@classmethod
def get_recent_logs(
cls, tenant_id: str, app_id: str, hours: int = 24, limit: int = 100, offset: int = 0
) -> list[dict[str, Any]]:
) -> list[WorkflowTriggerLogDict]:
"""
Get recent trigger logs
@@ -272,7 +274,7 @@ class AsyncWorkflowService:
@classmethod
def get_failed_logs_for_retry(
cls, tenant_id: str, max_retry_count: int = 3, limit: int = 100
) -> list[dict[str, Any]]:
) -> list[WorkflowTriggerLogDict]:
"""
Get failed logs eligible for retry
@@ -155,13 +155,15 @@ class EmailDeliveryTestHandler:
context=context,
recipient_email=recipient_email,
)
subject = render_email_template(method.config.subject, substitutions)
subject_template = render_email_template(method.config.subject, substitutions)
subject = EmailDeliveryConfig.sanitize_subject(subject_template)
templated_body = EmailDeliveryConfig.render_body_template(
body=method.config.body,
url=substitutions.get("form_link"),
variable_pool=context.variable_pool,
)
body = render_email_template(templated_body, substitutions)
body = EmailDeliveryConfig.render_markdown_body(body)
mail.send(
to=recipient_email,
@@ -472,6 +472,7 @@ class RagPipelineService:
engine=db.engine,
app_id=pipeline.id,
tenant_id=pipeline.tenant_id,
user_id=account.id,
),
),
start_at=start_at,
@@ -1237,6 +1238,7 @@ class RagPipelineService:
engine=db.engine,
app_id=pipeline.id,
tenant_id=pipeline.tenant_id,
user_id=current_user.id,
),
),
start_at=start_at,
@@ -1,16 +1,16 @@
import datetime
import logging
import os
import random
import time
from collections.abc import Sequence
from typing import cast
from typing import TYPE_CHECKING, cast
import sqlalchemy as sa
from sqlalchemy import delete, select, tuple_
from sqlalchemy.engine import CursorResult
from sqlalchemy.orm import Session
from configs import dify_config
from extensions.ext_database import db
from libs.datetime_utils import naive_utc_now
from models.model import (
@@ -33,6 +33,131 @@ from services.retention.conversation.messages_clean_policy import (
logger = logging.getLogger(__name__)
if TYPE_CHECKING:
from opentelemetry.metrics import Counter, Histogram
class MessagesCleanupMetrics:
"""
Records low-cardinality OpenTelemetry metrics for expired message cleanup jobs.
We keep labels stable (dry_run/window_mode/task_label/status) so these metrics remain
dashboard-friendly for long-running CronJob executions.
"""
_job_runs_total: "Counter | None"
_batches_total: "Counter | None"
_messages_scanned_total: "Counter | None"
_messages_filtered_total: "Counter | None"
_messages_deleted_total: "Counter | None"
_job_duration_seconds: "Histogram | None"
_batch_duration_seconds: "Histogram | None"
_base_attributes: dict[str, str]
def __init__(self, *, dry_run: bool, has_window: bool, task_label: str) -> None:
self._job_runs_total = None
self._batches_total = None
self._messages_scanned_total = None
self._messages_filtered_total = None
self._messages_deleted_total = None
self._job_duration_seconds = None
self._batch_duration_seconds = None
self._base_attributes = {
"job_name": "messages_cleanup",
"dry_run": str(dry_run).lower(),
"window_mode": "between" if has_window else "before_cutoff",
"task_label": task_label,
}
self._init_instruments()
def _init_instruments(self) -> None:
if not dify_config.ENABLE_OTEL:
return
try:
from opentelemetry.metrics import get_meter
meter = get_meter("messages_cleanup", version=dify_config.project.version)
self._job_runs_total = meter.create_counter(
"messages_cleanup_jobs_total",
description="Total number of expired message cleanup jobs by status.",
unit="{job}",
)
self._batches_total = meter.create_counter(
"messages_cleanup_batches_total",
description="Total number of message cleanup batches processed.",
unit="{batch}",
)
self._messages_scanned_total = meter.create_counter(
"messages_cleanup_scanned_messages_total",
description="Total messages scanned by cleanup jobs.",
unit="{message}",
)
self._messages_filtered_total = meter.create_counter(
"messages_cleanup_filtered_messages_total",
description="Total messages selected by cleanup policy.",
unit="{message}",
)
self._messages_deleted_total = meter.create_counter(
"messages_cleanup_deleted_messages_total",
description="Total messages deleted by cleanup jobs.",
unit="{message}",
)
self._job_duration_seconds = meter.create_histogram(
"messages_cleanup_job_duration_seconds",
description="Duration of expired message cleanup jobs in seconds.",
unit="s",
)
self._batch_duration_seconds = meter.create_histogram(
"messages_cleanup_batch_duration_seconds",
description="Duration of expired message cleanup batch processing in seconds.",
unit="s",
)
except Exception:
logger.exception("messages_cleanup_metrics: failed to initialize instruments")
def _attrs(self, **extra: str) -> dict[str, str]:
return {**self._base_attributes, **extra}
@staticmethod
def _add(counter: "Counter | None", value: int, attributes: dict[str, str]) -> None:
if not counter or value <= 0:
return
try:
counter.add(value, attributes)
except Exception:
logger.exception("messages_cleanup_metrics: failed to add counter value")
@staticmethod
def _record(histogram: "Histogram | None", value: float, attributes: dict[str, str]) -> None:
if not histogram:
return
try:
histogram.record(value, attributes)
except Exception:
logger.exception("messages_cleanup_metrics: failed to record histogram value")
def record_batch(
self,
*,
scanned_messages: int,
filtered_messages: int,
deleted_messages: int,
batch_duration_seconds: float,
) -> None:
attributes = self._attrs()
self._add(self._batches_total, 1, attributes)
self._add(self._messages_scanned_total, scanned_messages, attributes)
self._add(self._messages_filtered_total, filtered_messages, attributes)
self._add(self._messages_deleted_total, deleted_messages, attributes)
self._record(self._batch_duration_seconds, batch_duration_seconds, attributes)
def record_completion(self, *, status: str, job_duration_seconds: float) -> None:
attributes = self._attrs(status=status)
self._add(self._job_runs_total, 1, attributes)
self._record(self._job_duration_seconds, job_duration_seconds, attributes)
class MessagesCleanService:
"""
Service for cleaning expired messages based on retention policies.
@@ -48,6 +173,7 @@ class MessagesCleanService:
start_from: datetime.datetime | None = None,
batch_size: int = 1000,
dry_run: bool = False,
task_label: str = "custom",
) -> None:
"""
Initialize the service with cleanup parameters.
@@ -58,12 +184,18 @@ class MessagesCleanService:
start_from: Optional start time (inclusive) of the range
batch_size: Number of messages to process per batch
dry_run: Whether to perform a dry run (no actual deletion)
task_label: Optional task label for retention metrics
"""
self._policy = policy
self._end_before = end_before
self._start_from = start_from
self._batch_size = batch_size
self._dry_run = dry_run
self._metrics = MessagesCleanupMetrics(
dry_run=dry_run,
has_window=bool(start_from),
task_label=task_label,
)
@classmethod
def from_time_range(
@@ -73,6 +205,7 @@ class MessagesCleanService:
end_before: datetime.datetime,
batch_size: int = 1000,
dry_run: bool = False,
task_label: str = "custom",
) -> "MessagesCleanService":
"""
Create a service instance for cleaning messages within a specific time range.
@@ -85,6 +218,7 @@ class MessagesCleanService:
end_before: End time (exclusive) of the range
batch_size: Number of messages to process per batch
dry_run: Whether to perform a dry run (no actual deletion)
task_label: Optional task label for retention metrics
Returns:
MessagesCleanService instance
@@ -112,6 +246,7 @@ class MessagesCleanService:
start_from=start_from,
batch_size=batch_size,
dry_run=dry_run,
task_label=task_label,
)
@classmethod
@@ -121,6 +256,7 @@ class MessagesCleanService:
days: int = 30,
batch_size: int = 1000,
dry_run: bool = False,
task_label: str = "custom",
) -> "MessagesCleanService":
"""
Create a service instance for cleaning messages older than specified days.
@@ -130,6 +266,7 @@ class MessagesCleanService:
days: Number of days to look back from now
batch_size: Number of messages to process per batch
dry_run: Whether to perform a dry run (no actual deletion)
task_label: Optional task label for retention metrics
Returns:
MessagesCleanService instance
@@ -153,7 +290,14 @@ class MessagesCleanService:
policy.__class__.__name__,
)
return cls(policy=policy, end_before=end_before, start_from=None, batch_size=batch_size, dry_run=dry_run)
return cls(
policy=policy,
end_before=end_before,
start_from=None,
batch_size=batch_size,
dry_run=dry_run,
task_label=task_label,
)
def run(self) -> dict[str, int]:
"""
@@ -162,7 +306,18 @@ class MessagesCleanService:
Returns:
Dict with statistics: batches, filtered_messages, total_deleted
"""
return self._clean_messages_by_time_range()
status = "success"
run_start = time.monotonic()
try:
return self._clean_messages_by_time_range()
except Exception:
status = "failed"
raise
finally:
self._metrics.record_completion(
status=status,
job_duration_seconds=time.monotonic() - run_start,
)
def _clean_messages_by_time_range(self) -> dict[str, int]:
"""
@@ -197,11 +352,14 @@ class MessagesCleanService:
self._end_before,
)
max_batch_interval_ms = int(os.environ.get("SANDBOX_EXPIRED_RECORDS_CLEAN_BATCH_MAX_INTERVAL", 200))
max_batch_interval_ms = dify_config.SANDBOX_EXPIRED_RECORDS_CLEAN_BATCH_MAX_INTERVAL
while True:
stats["batches"] += 1
batch_start = time.monotonic()
batch_scanned_messages = 0
batch_filtered_messages = 0
batch_deleted_messages = 0
# Step 1: Fetch a batch of messages using cursor
with Session(db.engine, expire_on_commit=False) as session:
@@ -240,9 +398,16 @@ class MessagesCleanService:
# Track total messages fetched across all batches
stats["total_messages"] += len(messages)
batch_scanned_messages = len(messages)
if not messages:
logger.info("clean_messages (batch %s): no more messages to process", stats["batches"])
self._metrics.record_batch(
scanned_messages=batch_scanned_messages,
filtered_messages=batch_filtered_messages,
deleted_messages=batch_deleted_messages,
batch_duration_seconds=time.monotonic() - batch_start,
)
break
# Update cursor to the last message's (created_at, id)
@@ -268,6 +433,12 @@ class MessagesCleanService:
if not apps:
logger.info("clean_messages (batch %s): no apps found, skip", stats["batches"])
self._metrics.record_batch(
scanned_messages=batch_scanned_messages,
filtered_messages=batch_filtered_messages,
deleted_messages=batch_deleted_messages,
batch_duration_seconds=time.monotonic() - batch_start,
)
continue
# Build app_id -> tenant_id mapping
@@ -286,9 +457,16 @@ class MessagesCleanService:
if not message_ids_to_delete:
logger.info("clean_messages (batch %s): no messages to delete, skip", stats["batches"])
self._metrics.record_batch(
scanned_messages=batch_scanned_messages,
filtered_messages=batch_filtered_messages,
deleted_messages=batch_deleted_messages,
batch_duration_seconds=time.monotonic() - batch_start,
)
continue
stats["filtered_messages"] += len(message_ids_to_delete)
batch_filtered_messages = len(message_ids_to_delete)
# Step 4: Batch delete messages and their relations
if not self._dry_run:
@@ -309,6 +487,7 @@ class MessagesCleanService:
commit_ms = int((time.monotonic() - commit_start) * 1000)
stats["total_deleted"] += messages_deleted
batch_deleted_messages = messages_deleted
logger.info(
"clean_messages (batch %s): processed %s messages, deleted %s messages",
@@ -343,6 +522,13 @@ class MessagesCleanService:
for msg_id in sampled_ids:
logger.info("clean_messages (batch %s, dry_run) sample: message_id=%s", stats["batches"], msg_id)
self._metrics.record_batch(
scanned_messages=batch_scanned_messages,
filtered_messages=batch_filtered_messages,
deleted_messages=batch_deleted_messages,
batch_duration_seconds=time.monotonic() - batch_start,
)
logger.info(
"clean_messages completed: total batches: %s, total messages: %s, filtered messages: %s, total deleted: %s",
stats["batches"],
@@ -1,9 +1,9 @@
import datetime
import logging
import os
import random
import time
from collections.abc import Iterable, Sequence
from typing import TYPE_CHECKING
import click
from sqlalchemy.orm import Session, sessionmaker
@@ -20,6 +20,159 @@ from services.billing_service import BillingService, SubscriptionPlan
logger = logging.getLogger(__name__)
if TYPE_CHECKING:
from opentelemetry.metrics import Counter, Histogram
class WorkflowRunCleanupMetrics:
"""
Records low-cardinality OpenTelemetry metrics for workflow run cleanup jobs.
Metrics are emitted with stable labels only (dry_run/window_mode/task_label/status)
to keep dashboard and alert cardinality predictable in production clusters.
"""
_job_runs_total: "Counter | None"
_batches_total: "Counter | None"
_runs_scanned_total: "Counter | None"
_runs_targeted_total: "Counter | None"
_runs_deleted_total: "Counter | None"
_runs_skipped_total: "Counter | None"
_related_records_total: "Counter | None"
_job_duration_seconds: "Histogram | None"
_batch_duration_seconds: "Histogram | None"
_base_attributes: dict[str, str]
def __init__(self, *, dry_run: bool, has_window: bool, task_label: str) -> None:
self._job_runs_total = None
self._batches_total = None
self._runs_scanned_total = None
self._runs_targeted_total = None
self._runs_deleted_total = None
self._runs_skipped_total = None
self._related_records_total = None
self._job_duration_seconds = None
self._batch_duration_seconds = None
self._base_attributes = {
"job_name": "workflow_run_cleanup",
"dry_run": str(dry_run).lower(),
"window_mode": "between" if has_window else "before_cutoff",
"task_label": task_label,
}
self._init_instruments()
def _init_instruments(self) -> None:
if not dify_config.ENABLE_OTEL:
return
try:
from opentelemetry.metrics import get_meter
meter = get_meter("workflow_run_cleanup", version=dify_config.project.version)
self._job_runs_total = meter.create_counter(
"workflow_run_cleanup_jobs_total",
description="Total number of workflow run cleanup jobs by status.",
unit="{job}",
)
self._batches_total = meter.create_counter(
"workflow_run_cleanup_batches_total",
description="Total number of processed cleanup batches.",
unit="{batch}",
)
self._runs_scanned_total = meter.create_counter(
"workflow_run_cleanup_scanned_runs_total",
description="Total workflow runs scanned by cleanup jobs.",
unit="{run}",
)
self._runs_targeted_total = meter.create_counter(
"workflow_run_cleanup_targeted_runs_total",
description="Total workflow runs targeted by cleanup policy.",
unit="{run}",
)
self._runs_deleted_total = meter.create_counter(
"workflow_run_cleanup_deleted_runs_total",
description="Total workflow runs deleted by cleanup jobs.",
unit="{run}",
)
self._runs_skipped_total = meter.create_counter(
"workflow_run_cleanup_skipped_runs_total",
description="Total workflow runs skipped because tenant is paid/unknown.",
unit="{run}",
)
self._related_records_total = meter.create_counter(
"workflow_run_cleanup_related_records_total",
description="Total related records processed by cleanup jobs.",
unit="{record}",
)
self._job_duration_seconds = meter.create_histogram(
"workflow_run_cleanup_job_duration_seconds",
description="Duration of workflow run cleanup jobs in seconds.",
unit="s",
)
self._batch_duration_seconds = meter.create_histogram(
"workflow_run_cleanup_batch_duration_seconds",
description="Duration of workflow run cleanup batch processing in seconds.",
unit="s",
)
except Exception:
logger.exception("workflow_run_cleanup_metrics: failed to initialize instruments")
def _attrs(self, **extra: str) -> dict[str, str]:
return {**self._base_attributes, **extra}
@staticmethod
def _add(counter: "Counter | None", value: int, attributes: dict[str, str]) -> None:
if not counter or value <= 0:
return
try:
counter.add(value, attributes)
except Exception:
logger.exception("workflow_run_cleanup_metrics: failed to add counter value")
@staticmethod
def _record(histogram: "Histogram | None", value: float, attributes: dict[str, str]) -> None:
if not histogram:
return
try:
histogram.record(value, attributes)
except Exception:
logger.exception("workflow_run_cleanup_metrics: failed to record histogram value")
def record_batch(
self,
*,
batch_rows: int,
targeted_runs: int,
skipped_runs: int,
deleted_runs: int,
related_counts: dict[str, int] | None,
related_action: str | None,
batch_duration_seconds: float,
) -> None:
attributes = self._attrs()
self._add(self._batches_total, 1, attributes)
self._add(self._runs_scanned_total, batch_rows, attributes)
self._add(self._runs_targeted_total, targeted_runs, attributes)
self._add(self._runs_skipped_total, skipped_runs, attributes)
self._add(self._runs_deleted_total, deleted_runs, attributes)
self._record(self._batch_duration_seconds, batch_duration_seconds, attributes)
if not related_counts or not related_action:
return
for record_type, count in related_counts.items():
self._add(
self._related_records_total,
count,
self._attrs(action=related_action, record_type=record_type),
)
def record_completion(self, *, status: str, job_duration_seconds: float) -> None:
attributes = self._attrs(status=status)
self._add(self._job_runs_total, 1, attributes)
self._record(self._job_duration_seconds, job_duration_seconds, attributes)
class WorkflowRunCleanup:
def __init__(
self,
@@ -29,6 +182,7 @@ class WorkflowRunCleanup:
end_before: datetime.datetime | None = None,
workflow_run_repo: APIWorkflowRunRepository | None = None,
dry_run: bool = False,
task_label: str = "custom",
):
if (start_from is None) ^ (end_before is None):
raise ValueError("start_from and end_before must be both set or both omitted.")
@@ -46,6 +200,11 @@ class WorkflowRunCleanup:
self.batch_size = batch_size
self._cleanup_whitelist: set[str] | None = None
self.dry_run = dry_run
self._metrics = WorkflowRunCleanupMetrics(
dry_run=dry_run,
has_window=bool(start_from),
task_label=task_label,
)
self.free_plan_grace_period_days = dify_config.SANDBOX_EXPIRED_RECORDS_CLEAN_GRACEFUL_PERIOD
self.workflow_run_repo: APIWorkflowRunRepository
if workflow_run_repo:
@@ -74,153 +233,193 @@ class WorkflowRunCleanup:
related_totals = self._empty_related_counts() if self.dry_run else None
batch_index = 0
last_seen: tuple[datetime.datetime, str] | None = None
status = "success"
run_start = time.monotonic()
max_batch_interval_ms = dify_config.SANDBOX_EXPIRED_RECORDS_CLEAN_BATCH_MAX_INTERVAL
max_batch_interval_ms = int(os.environ.get("SANDBOX_EXPIRED_RECORDS_CLEAN_BATCH_MAX_INTERVAL", 200))
try:
while True:
batch_start = time.monotonic()
while True:
batch_start = time.monotonic()
fetch_start = time.monotonic()
run_rows = self.workflow_run_repo.get_runs_batch_by_time_range(
start_from=self.window_start,
end_before=self.window_end,
last_seen=last_seen,
batch_size=self.batch_size,
)
if not run_rows:
logger.info("workflow_run_cleanup (batch #%s): no more rows to process", batch_index + 1)
break
batch_index += 1
last_seen = (run_rows[-1].created_at, run_rows[-1].id)
logger.info(
"workflow_run_cleanup (batch #%s): fetched %s rows in %sms",
batch_index,
len(run_rows),
int((time.monotonic() - fetch_start) * 1000),
)
tenant_ids = {row.tenant_id for row in run_rows}
filter_start = time.monotonic()
free_tenants = self._filter_free_tenants(tenant_ids)
logger.info(
"workflow_run_cleanup (batch #%s): filtered %s free tenants from %s tenants in %sms",
batch_index,
len(free_tenants),
len(tenant_ids),
int((time.monotonic() - filter_start) * 1000),
)
free_runs = [row for row in run_rows if row.tenant_id in free_tenants]
paid_or_skipped = len(run_rows) - len(free_runs)
if not free_runs:
skipped_message = (
f"[batch #{batch_index}] skipped (no sandbox runs in batch, {paid_or_skipped} paid/unknown)"
fetch_start = time.monotonic()
run_rows = self.workflow_run_repo.get_runs_batch_by_time_range(
start_from=self.window_start,
end_before=self.window_end,
last_seen=last_seen,
batch_size=self.batch_size,
)
click.echo(
click.style(
skipped_message,
fg="yellow",
)
)
continue
if not run_rows:
logger.info("workflow_run_cleanup (batch #%s): no more rows to process", batch_index + 1)
break
total_runs_targeted += len(free_runs)
if self.dry_run:
count_start = time.monotonic()
batch_counts = self.workflow_run_repo.count_runs_with_related(
free_runs,
count_node_executions=self._count_node_executions,
count_trigger_logs=self._count_trigger_logs,
)
batch_index += 1
last_seen = (run_rows[-1].created_at, run_rows[-1].id)
logger.info(
"workflow_run_cleanup (batch #%s, dry_run): counted related records in %sms",
"workflow_run_cleanup (batch #%s): fetched %s rows in %sms",
batch_index,
int((time.monotonic() - count_start) * 1000),
len(run_rows),
int((time.monotonic() - fetch_start) * 1000),
)
if related_totals is not None:
for key in related_totals:
related_totals[key] += batch_counts.get(key, 0)
sample_ids = ", ".join(run.id for run in free_runs[:5])
tenant_ids = {row.tenant_id for row in run_rows}
filter_start = time.monotonic()
free_tenants = self._filter_free_tenants(tenant_ids)
logger.info(
"workflow_run_cleanup (batch #%s): filtered %s free tenants from %s tenants in %sms",
batch_index,
len(free_tenants),
len(tenant_ids),
int((time.monotonic() - filter_start) * 1000),
)
free_runs = [row for row in run_rows if row.tenant_id in free_tenants]
paid_or_skipped = len(run_rows) - len(free_runs)
if not free_runs:
skipped_message = (
f"[batch #{batch_index}] skipped (no sandbox runs in batch, {paid_or_skipped} paid/unknown)"
)
click.echo(
click.style(
skipped_message,
fg="yellow",
)
)
self._metrics.record_batch(
batch_rows=len(run_rows),
targeted_runs=0,
skipped_runs=paid_or_skipped,
deleted_runs=0,
related_counts=None,
related_action=None,
batch_duration_seconds=time.monotonic() - batch_start,
)
continue
total_runs_targeted += len(free_runs)
if self.dry_run:
count_start = time.monotonic()
batch_counts = self.workflow_run_repo.count_runs_with_related(
free_runs,
count_node_executions=self._count_node_executions,
count_trigger_logs=self._count_trigger_logs,
)
logger.info(
"workflow_run_cleanup (batch #%s, dry_run): counted related records in %sms",
batch_index,
int((time.monotonic() - count_start) * 1000),
)
if related_totals is not None:
for key in related_totals:
related_totals[key] += batch_counts.get(key, 0)
sample_ids = ", ".join(run.id for run in free_runs[:5])
click.echo(
click.style(
f"[batch #{batch_index}] would delete {len(free_runs)} runs "
f"(sample ids: {sample_ids}) and skip {paid_or_skipped} paid/unknown",
fg="yellow",
)
)
logger.info(
"workflow_run_cleanup (batch #%s, dry_run): batch total %sms",
batch_index,
int((time.monotonic() - batch_start) * 1000),
)
self._metrics.record_batch(
batch_rows=len(run_rows),
targeted_runs=len(free_runs),
skipped_runs=paid_or_skipped,
deleted_runs=0,
related_counts={key: batch_counts.get(key, 0) for key in self._empty_related_counts()},
related_action="would_delete",
batch_duration_seconds=time.monotonic() - batch_start,
)
continue
try:
delete_start = time.monotonic()
counts = self.workflow_run_repo.delete_runs_with_related(
free_runs,
delete_node_executions=self._delete_node_executions,
delete_trigger_logs=self._delete_trigger_logs,
)
delete_ms = int((time.monotonic() - delete_start) * 1000)
except Exception:
logger.exception("Failed to delete workflow runs batch ending at %s", last_seen[0])
raise
total_runs_deleted += counts["runs"]
click.echo(
click.style(
f"[batch #{batch_index}] would delete {len(free_runs)} runs "
f"(sample ids: {sample_ids}) and skip {paid_or_skipped} paid/unknown",
fg="yellow",
f"[batch #{batch_index}] deleted runs: {counts['runs']} "
f"(nodes {counts['node_executions']}, offloads {counts['offloads']}, "
f"app_logs {counts['app_logs']}, trigger_logs {counts['trigger_logs']}, "
f"pauses {counts['pauses']}, pause_reasons {counts['pause_reasons']}); "
f"skipped {paid_or_skipped} paid/unknown",
fg="green",
)
)
logger.info(
"workflow_run_cleanup (batch #%s, dry_run): batch total %sms",
"workflow_run_cleanup (batch #%s): delete %sms, batch total %sms",
batch_index,
delete_ms,
int((time.monotonic() - batch_start) * 1000),
)
continue
try:
delete_start = time.monotonic()
counts = self.workflow_run_repo.delete_runs_with_related(
free_runs,
delete_node_executions=self._delete_node_executions,
delete_trigger_logs=self._delete_trigger_logs,
self._metrics.record_batch(
batch_rows=len(run_rows),
targeted_runs=len(free_runs),
skipped_runs=paid_or_skipped,
deleted_runs=counts["runs"],
related_counts={key: counts.get(key, 0) for key in self._empty_related_counts()},
related_action="deleted",
batch_duration_seconds=time.monotonic() - batch_start,
)
delete_ms = int((time.monotonic() - delete_start) * 1000)
except Exception:
logger.exception("Failed to delete workflow runs batch ending at %s", last_seen[0])
raise
total_runs_deleted += counts["runs"]
click.echo(
click.style(
f"[batch #{batch_index}] deleted runs: {counts['runs']} "
f"(nodes {counts['node_executions']}, offloads {counts['offloads']}, "
f"app_logs {counts['app_logs']}, trigger_logs {counts['trigger_logs']}, "
f"pauses {counts['pauses']}, pause_reasons {counts['pause_reasons']}); "
f"skipped {paid_or_skipped} paid/unknown",
fg="green",
)
)
logger.info(
"workflow_run_cleanup (batch #%s): delete %sms, batch total %sms",
batch_index,
delete_ms,
int((time.monotonic() - batch_start) * 1000),
)
# Random sleep between batches to avoid overwhelming the database
sleep_ms = random.uniform(0, max_batch_interval_ms) # noqa: S311
logger.info("workflow_run_cleanup (batch #%s): sleeping for %.2fms", batch_index, sleep_ms)
time.sleep(sleep_ms / 1000)
# Random sleep between batches to avoid overwhelming the database
sleep_ms = random.uniform(0, max_batch_interval_ms) # noqa: S311
logger.info("workflow_run_cleanup (batch #%s): sleeping for %.2fms", batch_index, sleep_ms)
time.sleep(sleep_ms / 1000)
if self.dry_run:
if self.window_start:
summary_message = (
f"Dry run complete. Would delete {total_runs_targeted} workflow runs "
f"between {self.window_start.isoformat()} and {self.window_end.isoformat()}"
)
if self.dry_run:
if self.window_start:
summary_message = (
f"Dry run complete. Would delete {total_runs_targeted} workflow runs "
f"between {self.window_start.isoformat()} and {self.window_end.isoformat()}"
)
else:
summary_message = (
f"Dry run complete. Would delete {total_runs_targeted} workflow runs "
f"before {self.window_end.isoformat()}"
)
if related_totals is not None:
summary_message = (
f"{summary_message}; related records: {self._format_related_counts(related_totals)}"
)
summary_color = "yellow"
else:
summary_message = (
f"Dry run complete. Would delete {total_runs_targeted} workflow runs "
f"before {self.window_end.isoformat()}"
)
if related_totals is not None:
summary_message = f"{summary_message}; related records: {self._format_related_counts(related_totals)}"
summary_color = "yellow"
else:
if self.window_start:
summary_message = (
f"Cleanup complete. Deleted {total_runs_deleted} workflow runs "
f"between {self.window_start.isoformat()} and {self.window_end.isoformat()}"
)
else:
summary_message = (
f"Cleanup complete. Deleted {total_runs_deleted} workflow runs before {self.window_end.isoformat()}"
)
summary_color = "white"
if self.window_start:
summary_message = (
f"Cleanup complete. Deleted {total_runs_deleted} workflow runs "
f"between {self.window_start.isoformat()} and {self.window_end.isoformat()}"
)
else:
summary_message = (
f"Cleanup complete. Deleted {total_runs_deleted} workflow runs "
f"before {self.window_end.isoformat()}"
)
summary_color = "white"
click.echo(click.style(summary_message, fg=summary_color))
click.echo(click.style(summary_message, fg=summary_color))
except Exception:
status = "failed"
raise
finally:
self._metrics.record_completion(
status=status,
job_duration_seconds=time.monotonic() - run_start,
)
def _filter_free_tenants(self, tenant_ids: Iterable[str]) -> set[str]:
tenant_id_list = list(tenant_ids)
+2 -1
View File
@@ -156,7 +156,8 @@ class VectorService:
)
# use full doc mode to generate segment's child chunk
processing_rule_dict = processing_rule.to_dict()
processing_rule_dict["rules"]["parent_mode"] = ParentMode.FULL_DOC
if processing_rule_dict["rules"] is not None:
processing_rule_dict["rules"]["parent_mode"] = ParentMode.FULL_DOC
documents = index_processor.transform(
[document],
embedding_model_instance=embedding_model_instance,
+80 -40
View File
@@ -77,6 +77,7 @@ class DraftVarLoader(VariableLoader):
_engine: Engine
# Application ID for which variables are being loaded.
_app_id: str
_user_id: str
_tenant_id: str
_fallback_variables: Sequence[VariableBase]
@@ -85,10 +86,12 @@ class DraftVarLoader(VariableLoader):
engine: Engine,
app_id: str,
tenant_id: str,
user_id: str,
fallback_variables: Sequence[VariableBase] | None = None,
):
self._engine = engine
self._app_id = app_id
self._user_id = user_id
self._tenant_id = tenant_id
self._fallback_variables = fallback_variables or []
@@ -104,7 +107,7 @@ class DraftVarLoader(VariableLoader):
with Session(bind=self._engine, expire_on_commit=False) as session:
srv = WorkflowDraftVariableService(session)
draft_vars = srv.get_draft_variables_by_selectors(self._app_id, selectors)
draft_vars = srv.get_draft_variables_by_selectors(self._app_id, selectors, user_id=self._user_id)
# Important:
files: list[File] = []
@@ -218,6 +221,7 @@ class WorkflowDraftVariableService:
self,
app_id: str,
selectors: Sequence[list[str]],
user_id: str,
) -> list[WorkflowDraftVariable]:
"""
Retrieve WorkflowDraftVariable instances based on app_id and selectors.
@@ -238,22 +242,30 @@ class WorkflowDraftVariableService:
# Alternatively, a `SELECT` statement could be constructed for each selector and
# combined using `UNION` to fetch all rows.
# Benchmarking indicates that both approaches yield comparable performance.
variables = (
query = (
self._session.query(WorkflowDraftVariable)
.options(
orm.selectinload(WorkflowDraftVariable.variable_file).selectinload(
WorkflowDraftVariableFile.upload_file
)
)
.where(WorkflowDraftVariable.app_id == app_id, or_(*ors))
.all()
.where(
WorkflowDraftVariable.app_id == app_id,
WorkflowDraftVariable.user_id == user_id,
or_(*ors),
)
)
return variables
return query.all()
def list_variables_without_values(self, app_id: str, page: int, limit: int) -> WorkflowDraftVariableList:
criteria = WorkflowDraftVariable.app_id == app_id
def list_variables_without_values(
self, app_id: str, page: int, limit: int, user_id: str
) -> WorkflowDraftVariableList:
criteria = [
WorkflowDraftVariable.app_id == app_id,
WorkflowDraftVariable.user_id == user_id,
]
total = None
query = self._session.query(WorkflowDraftVariable).where(criteria)
query = self._session.query(WorkflowDraftVariable).where(*criteria)
if page == 1:
total = query.count()
variables = (
@@ -269,11 +281,12 @@ class WorkflowDraftVariableService:
return WorkflowDraftVariableList(variables=variables, total=total)
def _list_node_variables(self, app_id: str, node_id: str) -> WorkflowDraftVariableList:
criteria = (
def _list_node_variables(self, app_id: str, node_id: str, user_id: str) -> WorkflowDraftVariableList:
criteria = [
WorkflowDraftVariable.app_id == app_id,
WorkflowDraftVariable.node_id == node_id,
)
WorkflowDraftVariable.user_id == user_id,
]
query = self._session.query(WorkflowDraftVariable).where(*criteria)
variables = (
query.options(orm.selectinload(WorkflowDraftVariable.variable_file))
@@ -282,36 +295,36 @@ class WorkflowDraftVariableService:
)
return WorkflowDraftVariableList(variables=variables)
def list_node_variables(self, app_id: str, node_id: str) -> WorkflowDraftVariableList:
return self._list_node_variables(app_id, node_id)
def list_node_variables(self, app_id: str, node_id: str, user_id: str) -> WorkflowDraftVariableList:
return self._list_node_variables(app_id, node_id, user_id=user_id)
def list_conversation_variables(self, app_id: str) -> WorkflowDraftVariableList:
return self._list_node_variables(app_id, CONVERSATION_VARIABLE_NODE_ID)
def list_conversation_variables(self, app_id: str, user_id: str) -> WorkflowDraftVariableList:
return self._list_node_variables(app_id, CONVERSATION_VARIABLE_NODE_ID, user_id=user_id)
def list_system_variables(self, app_id: str) -> WorkflowDraftVariableList:
return self._list_node_variables(app_id, SYSTEM_VARIABLE_NODE_ID)
def list_system_variables(self, app_id: str, user_id: str) -> WorkflowDraftVariableList:
return self._list_node_variables(app_id, SYSTEM_VARIABLE_NODE_ID, user_id=user_id)
def get_conversation_variable(self, app_id: str, name: str) -> WorkflowDraftVariable | None:
return self._get_variable(app_id=app_id, node_id=CONVERSATION_VARIABLE_NODE_ID, name=name)
def get_conversation_variable(self, app_id: str, name: str, user_id: str) -> WorkflowDraftVariable | None:
return self._get_variable(app_id=app_id, node_id=CONVERSATION_VARIABLE_NODE_ID, name=name, user_id=user_id)
def get_system_variable(self, app_id: str, name: str) -> WorkflowDraftVariable | None:
return self._get_variable(app_id=app_id, node_id=SYSTEM_VARIABLE_NODE_ID, name=name)
def get_system_variable(self, app_id: str, name: str, user_id: str) -> WorkflowDraftVariable | None:
return self._get_variable(app_id=app_id, node_id=SYSTEM_VARIABLE_NODE_ID, name=name, user_id=user_id)
def get_node_variable(self, app_id: str, node_id: str, name: str) -> WorkflowDraftVariable | None:
return self._get_variable(app_id, node_id, name)
def get_node_variable(self, app_id: str, node_id: str, name: str, user_id: str) -> WorkflowDraftVariable | None:
return self._get_variable(app_id, node_id, name, user_id=user_id)
def _get_variable(self, app_id: str, node_id: str, name: str) -> WorkflowDraftVariable | None:
variable = (
def _get_variable(self, app_id: str, node_id: str, name: str, user_id: str) -> WorkflowDraftVariable | None:
return (
self._session.query(WorkflowDraftVariable)
.options(orm.selectinload(WorkflowDraftVariable.variable_file))
.where(
WorkflowDraftVariable.app_id == app_id,
WorkflowDraftVariable.node_id == node_id,
WorkflowDraftVariable.name == name,
WorkflowDraftVariable.user_id == user_id,
)
.first()
)
return variable
def update_variable(
self,
@@ -462,7 +475,17 @@ class WorkflowDraftVariableService:
self._session.delete(upload_file)
self._session.delete(variable)
def delete_workflow_variables(self, app_id: str):
def delete_user_workflow_variables(self, app_id: str, user_id: str):
(
self._session.query(WorkflowDraftVariable)
.where(
WorkflowDraftVariable.app_id == app_id,
WorkflowDraftVariable.user_id == user_id,
)
.delete(synchronize_session=False)
)
def delete_app_workflow_variables(self, app_id: str):
(
self._session.query(WorkflowDraftVariable)
.where(WorkflowDraftVariable.app_id == app_id)
@@ -501,28 +524,35 @@ class WorkflowDraftVariableService:
self._session.delete(upload_file)
self._session.delete(variable_file)
def delete_node_variables(self, app_id: str, node_id: str):
return self._delete_node_variables(app_id, node_id)
def delete_node_variables(self, app_id: str, node_id: str, user_id: str):
return self._delete_node_variables(app_id, node_id, user_id=user_id)
def _delete_node_variables(self, app_id: str, node_id: str):
self._session.query(WorkflowDraftVariable).where(
WorkflowDraftVariable.app_id == app_id,
WorkflowDraftVariable.node_id == node_id,
).delete()
def _delete_node_variables(self, app_id: str, node_id: str, user_id: str):
(
self._session.query(WorkflowDraftVariable)
.where(
WorkflowDraftVariable.app_id == app_id,
WorkflowDraftVariable.node_id == node_id,
WorkflowDraftVariable.user_id == user_id,
)
.delete(synchronize_session=False)
)
def _get_conversation_id_from_draft_variable(self, app_id: str) -> str | None:
def _get_conversation_id_from_draft_variable(self, app_id: str, user_id: str) -> str | None:
draft_var = self._get_variable(
app_id=app_id,
node_id=SYSTEM_VARIABLE_NODE_ID,
name=str(SystemVariableKey.CONVERSATION_ID),
user_id=user_id,
)
if draft_var is None:
return None
segment = draft_var.get_value()
if not isinstance(segment, StringSegment):
logger.warning(
"sys.conversation_id variable is not a string: app_id=%s, id=%s",
"sys.conversation_id variable is not a string: app_id=%s, user_id=%s, id=%s",
app_id,
user_id,
draft_var.id,
)
return None
@@ -543,7 +573,7 @@ class WorkflowDraftVariableService:
If no such conversation exists, a new conversation is created and its ID is returned.
"""
conv_id = self._get_conversation_id_from_draft_variable(workflow.app_id)
conv_id = self._get_conversation_id_from_draft_variable(workflow.app_id, account_id)
if conv_id is not None:
conversation = (
@@ -580,12 +610,13 @@ class WorkflowDraftVariableService:
self._session.flush()
return conversation.id
def prefill_conversation_variable_default_values(self, workflow: Workflow):
def prefill_conversation_variable_default_values(self, workflow: Workflow, user_id: str):
""""""
draft_conv_vars: list[WorkflowDraftVariable] = []
for conv_var in workflow.conversation_variables:
draft_var = WorkflowDraftVariable.new_conversation_variable(
app_id=workflow.app_id,
user_id=user_id,
name=conv_var.name,
value=conv_var,
description=conv_var.description,
@@ -635,7 +666,7 @@ def _batch_upsert_draft_variable(
stmt = pg_insert(WorkflowDraftVariable).values([_model_to_insertion_dict(v) for v in draft_vars])
if policy == _UpsertPolicy.OVERWRITE:
stmt = stmt.on_conflict_do_update(
index_elements=WorkflowDraftVariable.unique_app_id_node_id_name(),
index_elements=WorkflowDraftVariable.unique_app_id_user_id_node_id_name(),
set_={
# Refresh creation timestamp to ensure updated variables
# appear first in chronologically sorted result sets.
@@ -652,7 +683,9 @@ def _batch_upsert_draft_variable(
},
)
elif policy == _UpsertPolicy.IGNORE:
stmt = stmt.on_conflict_do_nothing(index_elements=WorkflowDraftVariable.unique_app_id_node_id_name())
stmt = stmt.on_conflict_do_nothing(
index_elements=WorkflowDraftVariable.unique_app_id_user_id_node_id_name()
)
else:
stmt = mysql_insert(WorkflowDraftVariable).values([_model_to_insertion_dict(v) for v in draft_vars]) # type: ignore[assignment]
if policy == _UpsertPolicy.OVERWRITE:
@@ -682,6 +715,7 @@ def _model_to_insertion_dict(model: WorkflowDraftVariable) -> dict[str, Any]:
d: dict[str, Any] = {
"id": model.id,
"app_id": model.app_id,
"user_id": model.user_id,
"last_edited_at": None,
"node_id": model.node_id,
"name": model.name,
@@ -807,6 +841,7 @@ class DraftVariableSaver:
def _create_dummy_output_variable(self):
return WorkflowDraftVariable.new_node_variable(
app_id=self._app_id,
user_id=self._user.id,
node_id=self._node_id,
name=self._DUMMY_OUTPUT_IDENTITY,
node_execution_id=self._node_execution_id,
@@ -842,6 +877,7 @@ class DraftVariableSaver:
draft_vars.append(
WorkflowDraftVariable.new_conversation_variable(
app_id=self._app_id,
user_id=self._user.id,
name=item.name,
value=segment,
)
@@ -862,6 +898,7 @@ class DraftVariableSaver:
draft_vars.append(
WorkflowDraftVariable.new_node_variable(
app_id=self._app_id,
user_id=self._user.id,
node_id=self._node_id,
name=name,
node_execution_id=self._node_execution_id,
@@ -884,6 +921,7 @@ class DraftVariableSaver:
draft_vars.append(
WorkflowDraftVariable.new_sys_variable(
app_id=self._app_id,
user_id=self._user.id,
name=name,
node_execution_id=self._node_execution_id,
value=value_seg,
@@ -1019,6 +1057,7 @@ class DraftVariableSaver:
# Create the draft variable
draft_var = WorkflowDraftVariable.new_node_variable(
app_id=self._app_id,
user_id=self._user.id,
node_id=self._node_id,
name=name,
node_execution_id=self._node_execution_id,
@@ -1032,6 +1071,7 @@ class DraftVariableSaver:
# Create the draft variable
draft_var = WorkflowDraftVariable.new_node_variable(
app_id=self._app_id,
user_id=self._user.id,
node_id=self._node_id,
name=name,
node_execution_id=self._node_execution_id,
+8 -2
View File
@@ -697,7 +697,7 @@ class WorkflowService:
with Session(bind=db.engine, expire_on_commit=False) as session, session.begin():
draft_var_srv = WorkflowDraftVariableService(session)
draft_var_srv.prefill_conversation_variable_default_values(draft_workflow)
draft_var_srv.prefill_conversation_variable_default_values(draft_workflow, user_id=account.id)
node_config = draft_workflow.get_node_config_by_id(node_id)
node_type = Workflow.get_node_type_from_node_config(node_config)
@@ -740,6 +740,7 @@ class WorkflowService:
engine=db.engine,
app_id=app_model.id,
tenant_id=app_model.tenant_id,
user_id=account.id,
)
enclosing_node_type_and_id = draft_workflow.get_enclosing_node_type_and_id(node_config)
@@ -831,6 +832,7 @@ class WorkflowService:
workflow=draft_workflow,
node_config=node_config,
manual_inputs=inputs or {},
user_id=account.id,
)
node = self._build_human_input_node(
workflow=draft_workflow,
@@ -891,6 +893,7 @@ class WorkflowService:
workflow=draft_workflow,
node_config=node_config,
manual_inputs=inputs or {},
user_id=account.id,
)
node = self._build_human_input_node(
workflow=draft_workflow,
@@ -967,6 +970,7 @@ class WorkflowService:
workflow=draft_workflow,
node_config=node_config,
manual_inputs=inputs or {},
user_id=account.id,
)
node = self._build_human_input_node(
workflow=draft_workflow,
@@ -1102,10 +1106,11 @@ class WorkflowService:
workflow: Workflow,
node_config: NodeConfigDict,
manual_inputs: Mapping[str, Any],
user_id: str,
) -> VariablePool:
with Session(bind=db.engine, expire_on_commit=False) as session, session.begin():
draft_var_srv = WorkflowDraftVariableService(session)
draft_var_srv.prefill_conversation_variable_default_values(workflow)
draft_var_srv.prefill_conversation_variable_default_values(workflow, user_id=user_id)
variable_pool = VariablePool(
system_variables=SystemVariable.default(),
@@ -1118,6 +1123,7 @@ class WorkflowService:
engine=db.engine,
app_id=app_model.id,
tenant_id=app_model.tenant_id,
user_id=user_id,
)
variable_mapping = HumanInputNode.extract_variable_selector_to_variable_mapping(
graph_config=workflow.graph_dict,
+3 -2
View File
@@ -111,7 +111,7 @@ def _render_body(
url=form_link,
variable_pool=variable_pool,
)
return body
return EmailDeliveryConfig.render_markdown_body(body)
def _load_variable_pool(workflow_run_id: str | None) -> VariablePool | None:
@@ -173,10 +173,11 @@ def dispatch_human_input_email_task(form_id: str, node_title: str | None = None,
for recipient in job.recipients:
form_link = _build_form_link(recipient.token)
body = _render_body(job.body, form_link, variable_pool=variable_pool)
subject = EmailDeliveryConfig.sanitize_subject(job.subject)
mail.send(
to=recipient.email,
subject=job.subject,
subject=subject,
html=body,
)
@@ -30,6 +30,7 @@ from services.workflow_draft_variable_service import (
class TestWorkflowDraftVariableService(unittest.TestCase):
_test_app_id: str
_session: Session
_test_user_id: str
_node1_id = "test_node_1"
_node2_id = "test_node_2"
_node_exec_id = str(uuid.uuid4())
@@ -99,13 +100,13 @@ class TestWorkflowDraftVariableService(unittest.TestCase):
def test_list_variables(self):
srv = self._get_test_srv()
var_list = srv.list_variables_without_values(self._test_app_id, page=1, limit=2)
var_list = srv.list_variables_without_values(self._test_app_id, page=1, limit=2, user_id=self._test_user_id)
assert var_list.total == 5
assert len(var_list.variables) == 2
page1_var_ids = {v.id for v in var_list.variables}
assert page1_var_ids.issubset(self._variable_ids)
var_list_2 = srv.list_variables_without_values(self._test_app_id, page=2, limit=2)
var_list_2 = srv.list_variables_without_values(self._test_app_id, page=2, limit=2, user_id=self._test_user_id)
assert var_list_2.total is None
assert len(var_list_2.variables) == 2
page2_var_ids = {v.id for v in var_list_2.variables}
@@ -114,7 +115,7 @@ class TestWorkflowDraftVariableService(unittest.TestCase):
def test_get_node_variable(self):
srv = self._get_test_srv()
node_var = srv.get_node_variable(self._test_app_id, self._node1_id, "str_var")
node_var = srv.get_node_variable(self._test_app_id, self._node1_id, "str_var", user_id=self._test_user_id)
assert node_var is not None
assert node_var.id == self._node1_str_var_id
assert node_var.name == "str_var"
@@ -122,7 +123,7 @@ class TestWorkflowDraftVariableService(unittest.TestCase):
def test_get_system_variable(self):
srv = self._get_test_srv()
sys_var = srv.get_system_variable(self._test_app_id, "sys_var")
sys_var = srv.get_system_variable(self._test_app_id, "sys_var", user_id=self._test_user_id)
assert sys_var is not None
assert sys_var.id == self._sys_var_id
assert sys_var.name == "sys_var"
@@ -130,7 +131,7 @@ class TestWorkflowDraftVariableService(unittest.TestCase):
def test_get_conversation_variable(self):
srv = self._get_test_srv()
conv_var = srv.get_conversation_variable(self._test_app_id, "conv_var")
conv_var = srv.get_conversation_variable(self._test_app_id, "conv_var", user_id=self._test_user_id)
assert conv_var is not None
assert conv_var.id == self._conv_var_id
assert conv_var.name == "conv_var"
@@ -138,7 +139,7 @@ class TestWorkflowDraftVariableService(unittest.TestCase):
def test_delete_node_variables(self):
srv = self._get_test_srv()
srv.delete_node_variables(self._test_app_id, self._node2_id)
srv.delete_node_variables(self._test_app_id, self._node2_id, user_id=self._test_user_id)
node2_var_count = (
self._session.query(WorkflowDraftVariable)
.where(
@@ -162,7 +163,7 @@ class TestWorkflowDraftVariableService(unittest.TestCase):
def test__list_node_variables(self):
srv = self._get_test_srv()
node_vars = srv._list_node_variables(self._test_app_id, self._node2_id)
node_vars = srv._list_node_variables(self._test_app_id, self._node2_id, user_id=self._test_user_id)
assert len(node_vars.variables) == 2
assert {v.id for v in node_vars.variables} == set(self._node2_var_ids)
@@ -173,7 +174,7 @@ class TestWorkflowDraftVariableService(unittest.TestCase):
[self._node2_id, "str_var"],
[self._node2_id, "int_var"],
]
variables = srv.get_draft_variables_by_selectors(self._test_app_id, selectors)
variables = srv.get_draft_variables_by_selectors(self._test_app_id, selectors, user_id=self._test_user_id)
assert len(variables) == 3
assert {v.id for v in variables} == {self._node1_str_var_id} | set(self._node2_var_ids)
@@ -206,19 +207,23 @@ class TestDraftVariableLoader(unittest.TestCase):
def setUp(self):
self._test_app_id = str(uuid.uuid4())
self._test_tenant_id = str(uuid.uuid4())
self._test_user_id = str(uuid.uuid4())
sys_var = WorkflowDraftVariable.new_sys_variable(
app_id=self._test_app_id,
user_id=self._test_user_id,
name="sys_var",
value=build_segment("sys_value"),
node_execution_id=self._node_exec_id,
)
conv_var = WorkflowDraftVariable.new_conversation_variable(
app_id=self._test_app_id,
user_id=self._test_user_id,
name="conv_var",
value=build_segment("conv_value"),
)
node_var = WorkflowDraftVariable.new_node_variable(
app_id=self._test_app_id,
user_id=self._test_user_id,
node_id=self._node1_id,
name="str_var",
value=build_segment("str_value"),
@@ -248,12 +253,22 @@ class TestDraftVariableLoader(unittest.TestCase):
session.commit()
def test_variable_loader_with_empty_selector(self):
var_loader = DraftVarLoader(engine=db.engine, app_id=self._test_app_id, tenant_id=self._test_tenant_id)
var_loader = DraftVarLoader(
engine=db.engine,
app_id=self._test_app_id,
tenant_id=self._test_tenant_id,
user_id=self._test_user_id,
)
variables = var_loader.load_variables([])
assert len(variables) == 0
def test_variable_loader_with_non_empty_selector(self):
var_loader = DraftVarLoader(engine=db.engine, app_id=self._test_app_id, tenant_id=self._test_tenant_id)
var_loader = DraftVarLoader(
engine=db.engine,
app_id=self._test_app_id,
tenant_id=self._test_tenant_id,
user_id=self._test_user_id,
)
variables = var_loader.load_variables(
[
[SYSTEM_VARIABLE_NODE_ID, "sys_var"],
@@ -296,7 +311,12 @@ class TestDraftVariableLoader(unittest.TestCase):
session.commit()
# Now test loading using DraftVarLoader
var_loader = DraftVarLoader(engine=db.engine, app_id=self._test_app_id, tenant_id=self._test_tenant_id)
var_loader = DraftVarLoader(
engine=db.engine,
app_id=self._test_app_id,
tenant_id=self._test_tenant_id,
user_id=setup_account.id,
)
# Load the variable using the standard workflow
variables = var_loader.load_variables([["test_offload_node", "offloaded_string_var"]])
@@ -313,7 +333,7 @@ class TestDraftVariableLoader(unittest.TestCase):
# Clean up - delete all draft variables for this app
with Session(bind=db.engine) as session:
service = WorkflowDraftVariableService(session)
service.delete_workflow_variables(self._test_app_id)
service.delete_app_workflow_variables(self._test_app_id)
session.commit()
def test_load_offloaded_variable_object_type_integration(self):
@@ -364,6 +384,7 @@ class TestDraftVariableLoader(unittest.TestCase):
# Now create the offloaded draft variable with the correct file_id
offloaded_var = WorkflowDraftVariable.new_node_variable(
app_id=self._test_app_id,
user_id=self._test_user_id,
node_id="test_offload_node",
name="offloaded_object_var",
value=build_segment({"truncated": True}),
@@ -379,7 +400,9 @@ class TestDraftVariableLoader(unittest.TestCase):
# Use the service method that properly preloads relationships
service = WorkflowDraftVariableService(session)
draft_vars = service.get_draft_variables_by_selectors(
self._test_app_id, [["test_offload_node", "offloaded_object_var"]]
self._test_app_id,
[["test_offload_node", "offloaded_object_var"]],
user_id=self._test_user_id,
)
assert len(draft_vars) == 1
@@ -387,7 +410,12 @@ class TestDraftVariableLoader(unittest.TestCase):
assert loaded_var.is_truncated()
# Create DraftVarLoader and test loading
var_loader = DraftVarLoader(engine=db.engine, app_id=self._test_app_id, tenant_id=self._test_tenant_id)
var_loader = DraftVarLoader(
engine=db.engine,
app_id=self._test_app_id,
tenant_id=self._test_tenant_id,
user_id=self._test_user_id,
)
# Test the _load_offloaded_variable method
selector_tuple, variable = var_loader._load_offloaded_variable(loaded_var)
@@ -459,6 +487,7 @@ class TestDraftVariableLoader(unittest.TestCase):
# Now create the offloaded draft variable with the correct file_id
offloaded_var = WorkflowDraftVariable.new_node_variable(
app_id=self._test_app_id,
user_id=self._test_user_id,
node_id="test_integration_node",
name="offloaded_integration_var",
value=build_segment("truncated"),
@@ -473,7 +502,12 @@ class TestDraftVariableLoader(unittest.TestCase):
# Test load_variables with both regular and offloaded variables
# This method should handle the relationship preloading internally
var_loader = DraftVarLoader(engine=db.engine, app_id=self._test_app_id, tenant_id=self._test_tenant_id)
var_loader = DraftVarLoader(
engine=db.engine,
app_id=self._test_app_id,
tenant_id=self._test_tenant_id,
user_id=self._test_user_id,
)
variables = var_loader.load_variables(
[
@@ -572,6 +606,7 @@ class TestWorkflowDraftVariableServiceResetVariable(unittest.TestCase):
# Create test variables
self._node_var_with_exec = WorkflowDraftVariable.new_node_variable(
app_id=self._test_app_id,
user_id=self._test_user_id,
node_id=self._node_id,
name="test_var",
value=build_segment("old_value"),
@@ -581,6 +616,7 @@ class TestWorkflowDraftVariableServiceResetVariable(unittest.TestCase):
self._node_var_without_exec = WorkflowDraftVariable.new_node_variable(
app_id=self._test_app_id,
user_id=self._test_user_id,
node_id=self._node_id,
name="no_exec_var",
value=build_segment("some_value"),
@@ -591,6 +627,7 @@ class TestWorkflowDraftVariableServiceResetVariable(unittest.TestCase):
self._node_var_missing_exec = WorkflowDraftVariable.new_node_variable(
app_id=self._test_app_id,
user_id=self._test_user_id,
node_id=self._node_id,
name="missing_exec_var",
value=build_segment("some_value"),
@@ -599,6 +636,7 @@ class TestWorkflowDraftVariableServiceResetVariable(unittest.TestCase):
self._conv_var = WorkflowDraftVariable.new_conversation_variable(
app_id=self._test_app_id,
user_id=self._test_user_id,
name="conv_var_1",
value=build_segment("old_conv_value"),
)
@@ -764,6 +802,7 @@ class TestWorkflowDraftVariableServiceResetVariable(unittest.TestCase):
# Create a system variable
sys_var = WorkflowDraftVariable.new_sys_variable(
app_id=self._test_app_id,
user_id=self._test_user_id,
name="sys_var",
value=build_segment("sys_value"),
node_execution_id=self._node_exec_id,
@@ -186,7 +186,7 @@ class DifyTestContainers:
# Start Dify Plugin Daemon container for plugin management
# Dify Plugin Daemon provides plugin lifecycle management and execution
logger.info("Initializing Dify Plugin Daemon container...")
self.dify_plugin_daemon = DockerContainer(image="langgenius/dify-plugin-daemon:0.3.0-local").with_network(
self.dify_plugin_daemon = DockerContainer(image="langgenius/dify-plugin-daemon:0.5.4-local").with_network(
self.network
)
self.dify_plugin_daemon.with_exposed_ports(5002)
@@ -122,6 +122,7 @@ class TestWorkflowDraftVariableService:
name,
value,
variable_type: DraftVariableType = DraftVariableType.CONVERSATION,
user_id: str | None = None,
fake=None,
):
"""
@@ -144,10 +145,15 @@ class TestWorkflowDraftVariableService:
WorkflowDraftVariable: Created test variable instance with proper type configuration
"""
fake = fake or Faker()
if user_id is None:
app = db_session_with_containers.query(App).filter_by(id=app_id).first()
assert app is not None
user_id = app.created_by
if variable_type == "conversation":
# Create conversation variable using the appropriate factory method
variable = WorkflowDraftVariable.new_conversation_variable(
app_id=app_id,
user_id=user_id,
name=name,
value=value,
description=fake.text(max_nb_chars=20),
@@ -156,6 +162,7 @@ class TestWorkflowDraftVariableService:
# Create system variable with editable flag and execution context
variable = WorkflowDraftVariable.new_sys_variable(
app_id=app_id,
user_id=user_id,
name=name,
value=value,
node_execution_id=fake.uuid4(),
@@ -165,6 +172,7 @@ class TestWorkflowDraftVariableService:
# Create node variable with visibility and editability settings
variable = WorkflowDraftVariable.new_node_variable(
app_id=app_id,
user_id=user_id,
node_id=node_id,
name=name,
value=value,
@@ -189,7 +197,13 @@ class TestWorkflowDraftVariableService:
app = self._create_test_app(db_session_with_containers, mock_external_service_dependencies, fake=fake)
test_value = StringSegment(value=fake.word())
variable = self._create_test_variable(
db_session_with_containers, app.id, CONVERSATION_VARIABLE_NODE_ID, "test_var", test_value, fake=fake
db_session_with_containers,
app.id,
CONVERSATION_VARIABLE_NODE_ID,
"test_var",
test_value,
user_id=app.created_by,
fake=fake,
)
service = WorkflowDraftVariableService(db_session_with_containers)
retrieved_variable = service.get_variable(variable.id)
@@ -250,7 +264,7 @@ class TestWorkflowDraftVariableService:
["test_node_1", "var3"],
]
service = WorkflowDraftVariableService(db_session_with_containers)
retrieved_variables = service.get_draft_variables_by_selectors(app.id, selectors)
retrieved_variables = service.get_draft_variables_by_selectors(app.id, selectors, user_id=app.created_by)
assert len(retrieved_variables) == 3
var_names = [var.name for var in retrieved_variables]
assert "var1" in var_names
@@ -288,7 +302,7 @@ class TestWorkflowDraftVariableService:
fake=fake,
)
service = WorkflowDraftVariableService(db_session_with_containers)
result = service.list_variables_without_values(app.id, page=1, limit=3)
result = service.list_variables_without_values(app.id, page=1, limit=3, user_id=app.created_by)
assert result.total == 5
assert len(result.variables) == 3
assert result.variables[0].created_at >= result.variables[1].created_at
@@ -339,7 +353,7 @@ class TestWorkflowDraftVariableService:
fake=fake,
)
service = WorkflowDraftVariableService(db_session_with_containers)
result = service.list_node_variables(app.id, node_id)
result = service.list_node_variables(app.id, node_id, user_id=app.created_by)
assert len(result.variables) == 2
for var in result.variables:
assert var.node_id == node_id
@@ -381,7 +395,7 @@ class TestWorkflowDraftVariableService:
fake=fake,
)
service = WorkflowDraftVariableService(db_session_with_containers)
result = service.list_conversation_variables(app.id)
result = service.list_conversation_variables(app.id, user_id=app.created_by)
assert len(result.variables) == 2
for var in result.variables:
assert var.node_id == CONVERSATION_VARIABLE_NODE_ID
@@ -559,7 +573,7 @@ class TestWorkflowDraftVariableService:
assert len(app_variables) == 3
assert len(other_app_variables) == 1
service = WorkflowDraftVariableService(db_session_with_containers)
service.delete_workflow_variables(app.id)
service.delete_user_workflow_variables(app.id, user_id=app.created_by)
app_variables_after = db_session_with_containers.query(WorkflowDraftVariable).filter_by(app_id=app.id).all()
other_app_variables_after = (
db_session_with_containers.query(WorkflowDraftVariable).filter_by(app_id=other_app.id).all()
@@ -567,6 +581,69 @@ class TestWorkflowDraftVariableService:
assert len(app_variables_after) == 0
assert len(other_app_variables_after) == 1
def test_draft_variables_are_isolated_between_users(
self, db_session_with_containers: Session, mock_external_service_dependencies
):
"""
Test draft variable isolation for different users in the same app.
This test verifies that:
1. Query APIs return only variables owned by the target user.
2. User-scoped deletion only removes variables for that user and keeps
other users' variables in the same app untouched.
"""
fake = Faker()
app = self._create_test_app(db_session_with_containers, mock_external_service_dependencies, fake=fake)
user_a = app.created_by
user_b = fake.uuid4()
# Use identical variable names on purpose to verify uniqueness scope includes user_id.
self._create_test_variable(
db_session_with_containers,
app.id,
CONVERSATION_VARIABLE_NODE_ID,
"shared_name",
StringSegment(value="value_a"),
user_id=user_a,
fake=fake,
)
self._create_test_variable(
db_session_with_containers,
app.id,
CONVERSATION_VARIABLE_NODE_ID,
"shared_name",
StringSegment(value="value_b"),
user_id=user_b,
fake=fake,
)
self._create_test_variable(
db_session_with_containers,
app.id,
CONVERSATION_VARIABLE_NODE_ID,
"only_a",
StringSegment(value="only_a"),
user_id=user_a,
fake=fake,
)
service = WorkflowDraftVariableService(db_session_with_containers)
user_a_vars = service.list_conversation_variables(app.id, user_id=user_a)
user_b_vars = service.list_conversation_variables(app.id, user_id=user_b)
assert {v.name for v in user_a_vars.variables} == {"shared_name", "only_a"}
assert {v.name for v in user_b_vars.variables} == {"shared_name"}
service.delete_user_workflow_variables(app.id, user_id=user_a)
user_a_remaining = (
db_session_with_containers.query(WorkflowDraftVariable).filter_by(app_id=app.id, user_id=user_a).count()
)
user_b_remaining = (
db_session_with_containers.query(WorkflowDraftVariable).filter_by(app_id=app.id, user_id=user_b).count()
)
assert user_a_remaining == 0
assert user_b_remaining == 1
def test_delete_node_variables_success(
self, db_session_with_containers: Session, mock_external_service_dependencies
):
@@ -627,7 +704,7 @@ class TestWorkflowDraftVariableService:
assert len(other_node_variables) == 1
assert len(conv_variables) == 1
service = WorkflowDraftVariableService(db_session_with_containers)
service.delete_node_variables(app.id, node_id)
service.delete_node_variables(app.id, node_id, user_id=app.created_by)
target_node_variables_after = (
db_session_with_containers.query(WorkflowDraftVariable).filter_by(app_id=app.id, node_id=node_id).all()
)
@@ -675,7 +752,7 @@ class TestWorkflowDraftVariableService:
db_session_with_containers.commit()
service = WorkflowDraftVariableService(db_session_with_containers)
service.prefill_conversation_variable_default_values(workflow)
service.prefill_conversation_variable_default_values(workflow, user_id="00000000-0000-0000-0000-000000000001")
draft_variables = (
db_session_with_containers.query(WorkflowDraftVariable)
.filter_by(app_id=app.id, node_id=CONVERSATION_VARIABLE_NODE_ID)
@@ -715,7 +792,7 @@ class TestWorkflowDraftVariableService:
fake=fake,
)
service = WorkflowDraftVariableService(db_session_with_containers)
retrieved_conv_id = service._get_conversation_id_from_draft_variable(app.id)
retrieved_conv_id = service._get_conversation_id_from_draft_variable(app.id, app.created_by)
assert retrieved_conv_id == conversation_id
def test_get_conversation_id_from_draft_variable_not_found(
@@ -731,7 +808,7 @@ class TestWorkflowDraftVariableService:
fake = Faker()
app = self._create_test_app(db_session_with_containers, mock_external_service_dependencies, fake=fake)
service = WorkflowDraftVariableService(db_session_with_containers)
retrieved_conv_id = service._get_conversation_id_from_draft_variable(app.id)
retrieved_conv_id = service._get_conversation_id_from_draft_variable(app.id, app.created_by)
assert retrieved_conv_id is None
def test_list_system_variables_success(
@@ -772,7 +849,7 @@ class TestWorkflowDraftVariableService:
db_session_with_containers, app.id, CONVERSATION_VARIABLE_NODE_ID, "conv_var", conv_var_value, fake=fake
)
service = WorkflowDraftVariableService(db_session_with_containers)
result = service.list_system_variables(app.id)
result = service.list_system_variables(app.id, user_id=app.created_by)
assert len(result.variables) == 2
for var in result.variables:
assert var.node_id == SYSTEM_VARIABLE_NODE_ID
@@ -819,15 +896,15 @@ class TestWorkflowDraftVariableService:
fake=fake,
)
service = WorkflowDraftVariableService(db_session_with_containers)
retrieved_conv_var = service.get_conversation_variable(app.id, "test_conv_var")
retrieved_conv_var = service.get_conversation_variable(app.id, "test_conv_var", user_id=app.created_by)
assert retrieved_conv_var is not None
assert retrieved_conv_var.name == "test_conv_var"
assert retrieved_conv_var.node_id == CONVERSATION_VARIABLE_NODE_ID
retrieved_sys_var = service.get_system_variable(app.id, "test_sys_var")
retrieved_sys_var = service.get_system_variable(app.id, "test_sys_var", user_id=app.created_by)
assert retrieved_sys_var is not None
assert retrieved_sys_var.name == "test_sys_var"
assert retrieved_sys_var.node_id == SYSTEM_VARIABLE_NODE_ID
retrieved_node_var = service.get_node_variable(app.id, "test_node", "test_node_var")
retrieved_node_var = service.get_node_variable(app.id, "test_node", "test_node_var", user_id=app.created_by)
assert retrieved_node_var is not None
assert retrieved_node_var.name == "test_node_var"
assert retrieved_node_var.node_id == "test_node"
@@ -845,9 +922,14 @@ class TestWorkflowDraftVariableService:
fake = Faker()
app = self._create_test_app(db_session_with_containers, mock_external_service_dependencies, fake=fake)
service = WorkflowDraftVariableService(db_session_with_containers)
retrieved_conv_var = service.get_conversation_variable(app.id, "non_existent_conv_var")
retrieved_conv_var = service.get_conversation_variable(app.id, "non_existent_conv_var", user_id=app.created_by)
assert retrieved_conv_var is None
retrieved_sys_var = service.get_system_variable(app.id, "non_existent_sys_var")
retrieved_sys_var = service.get_system_variable(app.id, "non_existent_sys_var", user_id=app.created_by)
assert retrieved_sys_var is None
retrieved_node_var = service.get_node_variable(app.id, "test_node", "non_existent_node_var")
retrieved_node_var = service.get_node_variable(
app.id,
"test_node",
"non_existent_node_var",
user_id=app.created_by,
)
assert retrieved_node_var is None
@@ -46,6 +46,7 @@ def test_absolute_mode_calls_from_time_range():
end_before=end_before,
batch_size=200,
dry_run=True,
task_label="custom",
)
mock_from_days.assert_not_called()
@@ -74,6 +75,7 @@ def test_relative_mode_before_days_only_calls_from_days():
days=30,
batch_size=500,
dry_run=False,
task_label="before-30",
)
mock_from_time_range.assert_not_called()
@@ -105,6 +107,7 @@ def test_relative_mode_with_from_days_ago_calls_from_time_range():
end_before=fixed_now - datetime.timedelta(days=30),
batch_size=1000,
dry_run=False,
task_label="60to30",
)
mock_from_days.assert_not_called()
@@ -398,6 +398,7 @@ class TestWorkflowDraftVariableEndpoints:
method = _unwrap(api.get)
monkeypatch.setattr(workflow_draft_variable_module, "db", SimpleNamespace(engine=MagicMock()))
monkeypatch.setattr(workflow_draft_variable_module, "current_user", SimpleNamespace(id="user-1"))
class DummySession:
def __enter__(self):
@@ -22,7 +22,7 @@ from controllers.console.extension import (
)
if _NEEDS_METHOD_VIEW_CLEANUP:
delattr(builtins, "MethodView")
del builtins.MethodView
from models.account import AccountStatus
from models.api_based_extension import APIBasedExtension
@@ -234,6 +234,7 @@ class TestAdvancedChatAppGeneratorInternals:
captured: dict[str, object] = {}
prefill_calls: list[object] = []
var_loader = SimpleNamespace(loader="draft")
workflow = SimpleNamespace(id="workflow-id")
monkeypatch.setattr(
"core.app.apps.advanced_chat.app_generator.AdvancedChatAppConfigManager.get_app_config",
@@ -260,8 +261,8 @@ class TestAdvancedChatAppGeneratorInternals:
def __init__(self, session):
_ = session
def prefill_conversation_variable_default_values(self, workflow):
prefill_calls.append(workflow)
def prefill_conversation_variable_default_values(self, workflow, user_id):
prefill_calls.append((workflow, user_id))
monkeypatch.setattr("core.app.apps.advanced_chat.app_generator.WorkflowDraftVariableService", _DraftVarService)
@@ -273,7 +274,7 @@ class TestAdvancedChatAppGeneratorInternals:
result = generator.single_iteration_generate(
app_model=SimpleNamespace(id="app", tenant_id="tenant"),
workflow=SimpleNamespace(id="workflow-id"),
workflow=workflow,
node_id="node-1",
user=SimpleNamespace(id="user-id"),
args={"inputs": {"foo": "bar"}},
@@ -281,7 +282,7 @@ class TestAdvancedChatAppGeneratorInternals:
)
assert result == {"ok": True}
assert prefill_calls
assert prefill_calls == [(workflow, "user-id")]
assert captured["variable_loader"] is var_loader
assert captured["application_generate_entity"].single_iteration_run.node_id == "node-1"
@@ -291,6 +292,7 @@ class TestAdvancedChatAppGeneratorInternals:
captured: dict[str, object] = {}
prefill_calls: list[object] = []
var_loader = SimpleNamespace(loader="draft")
workflow = SimpleNamespace(id="workflow-id")
monkeypatch.setattr(
"core.app.apps.advanced_chat.app_generator.AdvancedChatAppConfigManager.get_app_config",
@@ -317,8 +319,8 @@ class TestAdvancedChatAppGeneratorInternals:
def __init__(self, session):
_ = session
def prefill_conversation_variable_default_values(self, workflow):
prefill_calls.append(workflow)
def prefill_conversation_variable_default_values(self, workflow, user_id):
prefill_calls.append((workflow, user_id))
monkeypatch.setattr("core.app.apps.advanced_chat.app_generator.WorkflowDraftVariableService", _DraftVarService)
@@ -330,7 +332,7 @@ class TestAdvancedChatAppGeneratorInternals:
result = generator.single_loop_generate(
app_model=SimpleNamespace(id="app", tenant_id="tenant"),
workflow=SimpleNamespace(id="workflow-id"),
workflow=workflow,
node_id="node-2",
user=SimpleNamespace(id="user-id"),
args=SimpleNamespace(inputs={"foo": "bar"}),
@@ -338,7 +340,7 @@ class TestAdvancedChatAppGeneratorInternals:
)
assert result == {"ok": True}
assert prefill_calls
assert prefill_calls == [(workflow, "user-id")]
assert captured["variable_loader"] is var_loader
assert captured["application_generate_entity"].single_loop_run.node_id == "node-2"
@@ -801,6 +801,27 @@ class TestAuthOrchestration:
urls = build_protected_resource_metadata_discovery_urls(None, "https://api.example.com")
assert urls == ["https://api.example.com/.well-known/oauth-protected-resource"]
def test_build_protected_resource_metadata_discovery_urls_with_relative_hint(self):
urls = build_protected_resource_metadata_discovery_urls(
"/.well-known/oauth-protected-resource/tenant/mcp",
"https://api.example.com/tenant/mcp",
)
assert urls == [
"https://api.example.com/.well-known/oauth-protected-resource/tenant/mcp",
"https://api.example.com/.well-known/oauth-protected-resource",
]
def test_build_protected_resource_metadata_discovery_urls_ignores_scheme_less_hint(self):
urls = build_protected_resource_metadata_discovery_urls(
"/openapi-mcp.cn-hangzhou.aliyuncs.com/.well-known/oauth-protected-resource/tenant/mcp",
"https://openapi-mcp.cn-hangzhou.aliyuncs.com/tenant/mcp",
)
assert urls == [
"https://openapi-mcp.cn-hangzhou.aliyuncs.com/.well-known/oauth-protected-resource/tenant/mcp",
"https://openapi-mcp.cn-hangzhou.aliyuncs.com/.well-known/oauth-protected-resource",
]
def test_build_oauth_authorization_server_metadata_discovery_urls(self):
# Case 1: with auth_server_url
urls = build_oauth_authorization_server_metadata_discovery_urls(
@@ -0,0 +1,33 @@
from unittest.mock import MagicMock, patch
from core.rag.datasource.vdb.weaviate.weaviate_vector import WeaviateConfig, WeaviateVector
def test_init_client_with_valid_config():
"""Test successful client initialization with valid configuration."""
config = WeaviateConfig(
endpoint="http://localhost:8080",
api_key="WVF5YThaHlkYwhGUSmCRgsX3tD5ngdN8pkih",
)
with patch("weaviate.connect_to_custom") as mock_connect:
mock_client = MagicMock()
mock_client.is_ready.return_value = True
mock_connect.return_value = mock_client
vector = WeaviateVector(
collection_name="test_collection",
config=config,
attributes=["doc_id"],
)
assert vector._client == mock_client
mock_connect.assert_called_once()
call_kwargs = mock_connect.call_args[1]
assert call_kwargs["http_host"] == "localhost"
assert call_kwargs["http_port"] == 8080
assert call_kwargs["http_secure"] is False
assert call_kwargs["grpc_host"] == "localhost"
assert call_kwargs["grpc_port"] == 50051
assert call_kwargs["grpc_secure"] is False
assert call_kwargs["auth_credentials"] is not None
@@ -11,6 +11,7 @@ import unittest
from types import SimpleNamespace
from unittest.mock import MagicMock, patch
from core.rag.datasource.vdb.weaviate import weaviate_vector as weaviate_vector_module
from core.rag.datasource.vdb.weaviate.weaviate_vector import WeaviateConfig, WeaviateVector
from core.rag.models.document import Document
@@ -19,6 +20,7 @@ class TestWeaviateVector(unittest.TestCase):
"""Tests for WeaviateVector class with focus on doc_type metadata handling."""
def setUp(self):
weaviate_vector_module._weaviate_client = None
self.config = WeaviateConfig(
endpoint="http://localhost:8080",
api_key="test-key",
@@ -27,6 +29,9 @@ class TestWeaviateVector(unittest.TestCase):
self.collection_name = "Test_Collection_Node"
self.attributes = ["doc_id", "dataset_id", "document_id", "doc_hash", "doc_type"]
def tearDown(self):
weaviate_vector_module._weaviate_client = None
@patch("core.rag.datasource.vdb.weaviate.weaviate_vector.weaviate")
def _create_weaviate_vector(self, mock_weaviate_module):
"""Helper to create a WeaviateVector instance with mocked client."""
@@ -4,9 +4,7 @@ from __future__ import annotations
import pytest
import dify_graph.graph_engine.response_coordinator.session as response_session_module
from dify_graph.enums import BuiltinNodeTypes, NodeExecutionType, NodeState, NodeType
from dify_graph.graph_engine.response_coordinator import RESPONSE_SESSION_NODE_TYPES
from dify_graph.graph_engine.response_coordinator.session import ResponseSession
from dify_graph.nodes.base.template import Template, TextSegment
@@ -35,28 +33,14 @@ class DummyNodeWithoutStreamingTemplate:
self.state = NodeState.UNKNOWN
def test_response_session_from_node_rejects_node_types_outside_allowlist() -> None:
"""Unsupported node types are rejected even if they expose a template."""
def test_response_session_from_node_accepts_nodes_outside_previous_allowlist() -> None:
"""Session creation depends on the streaming-template contract rather than node type."""
node = DummyResponseNode(
node_id="llm-node",
node_type=BuiltinNodeTypes.LLM,
template=Template(segments=[TextSegment(text="hello")]),
)
with pytest.raises(TypeError, match="RESPONSE_SESSION_NODE_TYPES"):
ResponseSession.from_node(node)
def test_response_session_from_node_supports_downstream_allowlist_extension(monkeypatch) -> None:
"""Downstream applications can extend the supported node-type list."""
node = DummyResponseNode(
node_id="llm-node",
node_type=BuiltinNodeTypes.LLM,
template=Template(segments=[TextSegment(text="hello")]),
)
extended_node_types = [*RESPONSE_SESSION_NODE_TYPES, BuiltinNodeTypes.LLM]
monkeypatch.setattr(response_session_module, "RESPONSE_SESSION_NODE_TYPES", extended_node_types)
session = ResponseSession.from_node(node)
assert session.node_id == "llm-node"
@@ -14,3 +14,64 @@ def test_render_body_template_replaces_variable_values():
result = config.render_body_template(body=config.body, url="https://example.com", variable_pool=variable_pool)
assert result == "Hello World https://example.com"
def test_render_markdown_body_renders_markdown_to_html():
rendered = EmailDeliveryConfig.render_markdown_body("**Bold** and [link](https://example.com)")
assert "<strong>Bold</strong>" in rendered
assert '<a href="https://example.com">link</a>' in rendered
def test_render_markdown_body_sanitizes_unsafe_html():
rendered = EmailDeliveryConfig.render_markdown_body(
'<script>alert("xss")</script><a href="javascript:alert(1)" onclick="alert(2)">Click</a>'
)
assert "<script" not in rendered
assert "<a" not in rendered
assert "onclick" not in rendered
assert "javascript:" not in rendered
assert "Click" in rendered
def test_render_markdown_body_sanitizes_markdown_link_with_javascript_href():
rendered = EmailDeliveryConfig.render_markdown_body("[bad](javascript:alert(1)) and [ok](https://example.com)")
assert "javascript:" not in rendered
assert "<a>bad</a>" in rendered
assert '<a href="https://example.com">ok</a>' in rendered
def test_render_markdown_body_does_not_allow_raw_html_tags():
rendered = EmailDeliveryConfig.render_markdown_body("<b>raw html</b> and **markdown**")
assert "<b>" not in rendered
assert "raw html" in rendered
assert "<strong>markdown</strong>" in rendered
def test_render_markdown_body_supports_table_syntax():
rendered = EmailDeliveryConfig.render_markdown_body("| h1 | h2 |\n| --- | ---: |\n| v1 | v2 |")
assert "<table>" in rendered
assert "<thead>" in rendered
assert "<tbody>" in rendered
assert 'align="right"' in rendered
assert "style=" not in rendered
def test_sanitize_subject_removes_crlf():
sanitized = EmailDeliveryConfig.sanitize_subject("Notice\r\nBCC:attacker@example.com")
assert "\r" not in sanitized
assert "\n" not in sanitized
assert sanitized == "Notice BCC:attacker@example.com"
def test_sanitize_subject_removes_html_tags():
sanitized = EmailDeliveryConfig.sanitize_subject("<b>Alert</b><img src=x onerror=1>")
assert "<" not in sanitized
assert ">" not in sanitized
assert sanitized == "Alert"
+1 -1
View File
@@ -140,7 +140,7 @@ class TestLoginRequired:
# Remove ensure_sync to simulate Flask 1.x
if hasattr(setup_app, "ensure_sync"):
delattr(setup_app, "ensure_sync")
del setup_app.ensure_sync
with setup_app.test_request_context():
mock_user = MockUser("test_user", is_authenticated=True)
+3 -4
View File
@@ -1,13 +1,12 @@
import rsa as pyrsa
from Crypto.PublicKey import RSA
from libs import gmpy2_pkcs10aep_cipher
def test_gmpy2_pkcs10aep_cipher():
rsa_key_pair = pyrsa.newkeys(2048)
public_key = rsa_key_pair[0].save_pkcs1()
private_key = rsa_key_pair[1].save_pkcs1()
rsa_key = RSA.generate(2048)
public_key = rsa_key.publickey().export_key(format="PEM")
private_key = rsa_key.export_key(format="PEM")
public_rsa_key = RSA.import_key(public_key)
public_cipher_rsa2 = gmpy2_pkcs10aep_cipher.new(public_rsa_key)
@@ -1,5 +1,4 @@
import datetime
import os
from unittest.mock import MagicMock, patch
import pytest
@@ -282,7 +281,6 @@ class TestMessagesCleanService:
MessagesCleanService._batch_delete_message_relations(mock_db_session, ["msg1", "msg2"])
assert mock_db_session.execute.call_count == 8 # 8 tables to clean up
@patch.dict(os.environ, {"SANDBOX_EXPIRED_RECORDS_CLEAN_BATCH_MAX_INTERVAL": "500"})
def test_clean_messages_interval_from_env(self, mock_db_session, mock_policy):
service = MessagesCleanService(
policy=mock_policy,
@@ -301,9 +299,13 @@ class TestMessagesCleanService:
mock_db_session.execute.side_effect = mock_returns
mock_policy.filter_message_ids.return_value = ["msg1"]
with patch("services.retention.conversation.messages_clean_service.time.sleep") as mock_sleep:
with patch("services.retention.conversation.messages_clean_service.random.uniform") as mock_uniform:
mock_uniform.return_value = 300.0
service.run()
mock_uniform.assert_called_with(0, 500)
mock_sleep.assert_called_with(0.3)
with patch(
"services.retention.conversation.messages_clean_service.dify_config.SANDBOX_EXPIRED_RECORDS_CLEAN_BATCH_MAX_INTERVAL",
500,
):
with patch("services.retention.conversation.messages_clean_service.time.sleep") as mock_sleep:
with patch("services.retention.conversation.messages_clean_service.random.uniform") as mock_uniform:
mock_uniform.return_value = 300.0
service.run()
mock_uniform.assert_called_with(0, 500)
mock_sleep.assert_called_with(0.3)
@@ -80,7 +80,13 @@ class TestWorkflowRunCleanupInit:
cfg.SANDBOX_EXPIRED_RECORDS_CLEAN_GRACEFUL_PERIOD = 0
cfg.BILLING_ENABLED = False
with pytest.raises(ValueError):
WorkflowRunCleanup(days=30, batch_size=10, start_from=dt, end_before=dt, workflow_run_repo=mock_repo)
WorkflowRunCleanup(
days=30,
batch_size=10,
start_from=dt,
end_before=dt,
workflow_run_repo=mock_repo,
)
def test_zero_batch_size_raises(self, mock_repo):
with patch("services.retention.workflow_run.clear_free_plan_expired_workflow_run_logs.dify_config") as cfg:
@@ -102,10 +108,24 @@ class TestWorkflowRunCleanupInit:
cfg.BILLING_ENABLED = False
start = datetime.datetime(2024, 1, 1)
end = datetime.datetime(2024, 6, 1)
c = WorkflowRunCleanup(days=30, batch_size=5, start_from=start, end_before=end, workflow_run_repo=mock_repo)
c = WorkflowRunCleanup(
days=30,
batch_size=5,
start_from=start,
end_before=end,
workflow_run_repo=mock_repo,
)
assert c.window_start == start
assert c.window_end == end
def test_default_task_label_is_custom(self, mock_repo):
with patch("services.retention.workflow_run.clear_free_plan_expired_workflow_run_logs.dify_config") as cfg:
cfg.SANDBOX_EXPIRED_RECORDS_CLEAN_GRACEFUL_PERIOD = 0
cfg.BILLING_ENABLED = False
c = WorkflowRunCleanup(days=30, batch_size=10, workflow_run_repo=mock_repo)
assert c._metrics._base_attributes["task_label"] == "custom"
# ---------------------------------------------------------------------------
# _empty_related_counts / _format_related_counts
@@ -393,7 +413,12 @@ class TestRunDryRunMode:
with patch("services.retention.workflow_run.clear_free_plan_expired_workflow_run_logs.dify_config") as cfg:
cfg.SANDBOX_EXPIRED_RECORDS_CLEAN_GRACEFUL_PERIOD = 0
cfg.BILLING_ENABLED = False
return WorkflowRunCleanup(days=30, batch_size=10, workflow_run_repo=mock_repo, dry_run=True)
return WorkflowRunCleanup(
days=30,
batch_size=10,
workflow_run_repo=mock_repo,
dry_run=True,
)
def test_dry_run_no_delete_called(self, mock_repo):
run = make_run("t1")
@@ -263,7 +263,7 @@ def test_import_app_completed_uses_declared_dependencies(monkeypatch):
assert result.status == ImportStatus.COMPLETED
assert result.app_id == "app-new"
draft_var_service.delete_workflow_variables.assert_called_once_with(app_id="app-new")
draft_var_service.delete_app_workflow_variables.assert_called_once_with(app_id="app-new")
@pytest.mark.parametrize("has_workflow", [True, False])
@@ -305,7 +305,7 @@ def test_import_app_legacy_versions_extract_dependencies(monkeypatch, has_workfl
account=_account_mock(), import_mode=ImportMode.YAML_CONTENT, yaml_content=_yaml_dump(data)
)
assert result.status == ImportStatus.COMPLETED_WITH_WARNINGS
draft_var_service.delete_workflow_variables.assert_called_once_with(app_id="app-legacy")
draft_var_service.delete_app_workflow_variables.assert_called_once_with(app_id="app-legacy")
def test_import_app_yaml_error_returns_failed(monkeypatch):
@@ -265,6 +265,61 @@ def test_run_exits_on_empty_batch(monkeypatch: pytest.MonkeyPatch) -> None:
cleanup.run()
def test_run_records_metrics_on_success(monkeypatch: pytest.MonkeyPatch) -> None:
cutoff = datetime.datetime.now()
repo = FakeRepo(
batches=[[FakeRun("run-free", "t_free", cutoff)]],
delete_result={
"runs": 0,
"node_executions": 2,
"offloads": 1,
"app_logs": 3,
"trigger_logs": 4,
"pauses": 5,
"pause_reasons": 6,
},
)
cleanup = create_cleanup(monkeypatch, repo=repo, days=30, batch_size=10)
monkeypatch.setattr(cleanup_module.dify_config, "BILLING_ENABLED", False)
batch_calls: list[dict[str, object]] = []
completion_calls: list[dict[str, object]] = []
monkeypatch.setattr(cleanup._metrics, "record_batch", lambda **kwargs: batch_calls.append(kwargs))
monkeypatch.setattr(cleanup._metrics, "record_completion", lambda **kwargs: completion_calls.append(kwargs))
cleanup.run()
assert len(batch_calls) == 1
assert batch_calls[0]["batch_rows"] == 1
assert batch_calls[0]["targeted_runs"] == 1
assert batch_calls[0]["deleted_runs"] == 1
assert batch_calls[0]["related_action"] == "deleted"
assert len(completion_calls) == 1
assert completion_calls[0]["status"] == "success"
def test_run_records_failed_metrics(monkeypatch: pytest.MonkeyPatch) -> None:
class FailingRepo(FakeRepo):
def delete_runs_with_related(
self, runs: list[FakeRun], delete_node_executions=None, delete_trigger_logs=None
) -> dict[str, int]:
raise RuntimeError("delete failed")
cutoff = datetime.datetime.now()
repo = FailingRepo(batches=[[FakeRun("run-free", "t_free", cutoff)]])
cleanup = create_cleanup(monkeypatch, repo=repo, days=30, batch_size=10)
monkeypatch.setattr(cleanup_module.dify_config, "BILLING_ENABLED", False)
completion_calls: list[dict[str, object]] = []
monkeypatch.setattr(cleanup._metrics, "record_completion", lambda **kwargs: completion_calls.append(kwargs))
with pytest.raises(RuntimeError, match="delete failed"):
cleanup.run()
assert len(completion_calls) == 1
assert completion_calls[0]["status"] == "failed"
def test_run_dry_run_skips_deletions(monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str]) -> None:
cutoff = datetime.datetime.now()
repo = FakeRepo(
@@ -207,6 +207,45 @@ class TestEmailDeliveryTestHandler:
assert kwargs["to"] == "test@example.com"
assert "RENDERED_Subj" in kwargs["subject"]
def test_send_test_sanitizes_subject(self, monkeypatch):
monkeypatch.setattr(
service_module.FeatureService,
"get_features",
lambda _id: SimpleNamespace(human_input_email_delivery_enabled=True),
)
monkeypatch.setattr(service_module.mail, "is_inited", lambda: True)
mock_mail_send = MagicMock()
monkeypatch.setattr(service_module.mail, "send", mock_mail_send)
monkeypatch.setattr(
service_module,
"render_email_template",
lambda template, substitutions: template.replace("{{ recipient_email }}", substitutions["recipient_email"]),
)
handler = EmailDeliveryTestHandler(session_factory=MagicMock())
handler._resolve_recipients = MagicMock(return_value=["test@example.com"])
context = DeliveryTestContext(
tenant_id="t1",
app_id="a1",
node_id="n1",
node_title="title",
rendered_content="content",
recipients=[DeliveryTestEmailRecipient(email="test@example.com", form_token="token123")],
)
method = EmailDeliveryMethod(
config=EmailDeliveryConfig(
recipients=EmailRecipients(whole_workspace=False, items=[]),
subject="<b>Notice</b>\r\nBCC:{{ recipient_email }}",
body="Body",
)
)
handler.send_test(context=context, method=method)
_, kwargs = mock_mail_send.call_args
assert kwargs["subject"] == "Notice BCC:test@example.com"
def test_resolve_recipients(self):
handler = EmailDeliveryTestHandler(session_factory=MagicMock())
@@ -540,6 +540,20 @@ class TestMessagesCleanServiceFromTimeRange:
assert service._batch_size == 1000 # default
assert service._dry_run is False # default
def test_explicit_task_label(self):
start_from = datetime.datetime(2024, 1, 1)
end_before = datetime.datetime(2024, 1, 2)
policy = BillingDisabledPolicy()
service = MessagesCleanService.from_time_range(
policy=policy,
start_from=start_from,
end_before=end_before,
task_label="60to30",
)
assert service._metrics._base_attributes["task_label"] == "60to30"
class TestMessagesCleanServiceFromDays:
"""Unit tests for MessagesCleanService.from_days factory method."""
@@ -619,3 +633,54 @@ class TestMessagesCleanServiceFromDays:
assert service._end_before == expected_end_before
assert service._batch_size == 1000 # default
assert service._dry_run is False # default
assert service._metrics._base_attributes["task_label"] == "custom"
class TestMessagesCleanServiceRun:
"""Unit tests for MessagesCleanService.run instrumentation behavior."""
def test_run_records_completion_metrics_on_success(self):
# Arrange
service = MessagesCleanService(
policy=BillingDisabledPolicy(),
start_from=datetime.datetime(2024, 1, 1),
end_before=datetime.datetime(2024, 1, 2),
batch_size=100,
dry_run=False,
)
expected_stats = {
"batches": 1,
"total_messages": 10,
"filtered_messages": 5,
"total_deleted": 5,
}
service._clean_messages_by_time_range = MagicMock(return_value=expected_stats) # type: ignore[method-assign]
completion_calls: list[dict[str, object]] = []
service._metrics.record_completion = lambda **kwargs: completion_calls.append(kwargs) # type: ignore[method-assign]
# Act
result = service.run()
# Assert
assert result == expected_stats
assert len(completion_calls) == 1
assert completion_calls[0]["status"] == "success"
def test_run_records_completion_metrics_on_failure(self):
# Arrange
service = MessagesCleanService(
policy=BillingDisabledPolicy(),
start_from=datetime.datetime(2024, 1, 1),
end_before=datetime.datetime(2024, 1, 2),
batch_size=100,
dry_run=False,
)
service._clean_messages_by_time_range = MagicMock(side_effect=RuntimeError("clean failed")) # type: ignore[method-assign]
completion_calls: list[dict[str, object]] = []
service._metrics.record_completion = lambda **kwargs: completion_calls.append(kwargs) # type: ignore[method-assign]
# Act & Assert
with pytest.raises(RuntimeError, match="clean failed"):
service.run()
assert len(completion_calls) == 1
assert completion_calls[0]["status"] == "failed"
@@ -24,7 +24,11 @@ class TestDraftVarLoaderSimple:
def draft_var_loader(self, mock_engine):
"""Create DraftVarLoader instance for testing."""
return DraftVarLoader(
engine=mock_engine, app_id="test-app-id", tenant_id="test-tenant-id", fallback_variables=[]
engine=mock_engine,
app_id="test-app-id",
tenant_id="test-tenant-id",
user_id="test-user-id",
fallback_variables=[],
)
def test_load_offloaded_variable_string_type_unit(self, draft_var_loader):
@@ -323,7 +327,9 @@ class TestDraftVarLoaderSimple:
# Verify service method was called
mock_service.get_draft_variables_by_selectors.assert_called_once_with(
draft_var_loader._app_id, selectors
draft_var_loader._app_id,
selectors,
user_id=draft_var_loader._user_id,
)
# Verify offloaded variable loading was called
@@ -8,7 +8,7 @@ from sqlalchemy import Engine
from sqlalchemy.orm import Session
from dify_graph.constants import SYSTEM_VARIABLE_NODE_ID
from dify_graph.enums import BuiltinNodeTypes
from dify_graph.enums import BuiltinNodeTypes, SystemVariableKey
from dify_graph.variables.segments import StringSegment
from dify_graph.variables.types import SegmentType
from libs.uuid_utils import uuidv7
@@ -182,6 +182,42 @@ class TestDraftVariableSaver:
draft_vars = mock_batch_upsert.call_args[0][1]
assert len(draft_vars) == 2
@patch("services.workflow_draft_variable_service._batch_upsert_draft_variable", autospec=True)
def test_start_node_save_persists_sys_timestamp_and_workflow_run_id(self, mock_batch_upsert):
"""Start node should persist common `sys.*` variables, not only `sys.files`."""
mock_session = MagicMock(spec=Session)
mock_user = MagicMock(spec=Account)
mock_user.id = "test-user-id"
mock_user.tenant_id = "test-tenant-id"
saver = DraftVariableSaver(
session=mock_session,
app_id="test-app-id",
node_id="start-node-id",
node_type=BuiltinNodeTypes.START,
node_execution_id="exec-id",
user=mock_user,
)
outputs = {
f"{SYSTEM_VARIABLE_NODE_ID}.{SystemVariableKey.TIMESTAMP}": 1700000000,
f"{SYSTEM_VARIABLE_NODE_ID}.{SystemVariableKey.WORKFLOW_EXECUTION_ID}": "run-id-123",
}
saver.save(outputs=outputs)
mock_batch_upsert.assert_called_once()
draft_vars = mock_batch_upsert.call_args[0][1]
# plus one dummy output because there are no non-sys Start inputs
assert len(draft_vars) == 3
sys_vars = [v for v in draft_vars if v.node_id == SYSTEM_VARIABLE_NODE_ID]
assert {v.name for v in sys_vars} == {
str(SystemVariableKey.TIMESTAMP),
str(SystemVariableKey.WORKFLOW_EXECUTION_ID),
}
class TestWorkflowDraftVariableService:
def _get_test_app_id(self):
@@ -245,6 +245,7 @@ class TestWorkflowService:
workflow=workflow,
node_config=node_config,
manual_inputs={"#node-0.result#": "LLM output"},
user_id="account-1",
)
node.render_form_content_with_outputs.assert_called_once()
@@ -120,4 +120,37 @@ def test_dispatch_human_input_email_task_replaces_body_variables(monkeypatch: py
session_factory=lambda: _DummySession(form),
)
assert mail.sent[0]["html"] == "Body OK"
assert mail.sent[0]["html"] == "<p>Body OK</p>"
@pytest.mark.parametrize("line_break", ["\r\n", "\r", "\n"])
def test_dispatch_human_input_email_task_sanitizes_subject(
monkeypatch: pytest.MonkeyPatch,
line_break: str,
):
mail = _DummyMail()
form = SimpleNamespace(id="form-1", tenant_id="tenant-1", workflow_run_id=None)
job = task_module._EmailDeliveryJob(
form_id="form-1",
subject=f"Notice{line_break}BCC:attacker@example.com <b>Alert</b>",
body="Body",
form_content="content",
recipients=[task_module._EmailRecipient(email="user@example.com", token="token-1")],
)
monkeypatch.setattr(task_module, "mail", mail)
monkeypatch.setattr(
task_module.FeatureService,
"get_features",
lambda _tenant_id: SimpleNamespace(human_input_email_delivery_enabled=True),
)
monkeypatch.setattr(task_module, "_load_email_jobs", lambda _session, _form: [job])
monkeypatch.setattr(task_module, "_load_variable_pool", lambda _workflow_run_id: None)
task_module.dispatch_human_input_email_task(
form_id="form-1",
node_title="Approve",
session_factory=lambda: _DummySession(form),
)
assert mail.sent[0]["subject"] == "Notice BCC:attacker@example.com Alert"
Generated
+273 -259
View File
File diff suppressed because it is too large Load Diff
+13 -12
View File
@@ -1546,24 +1546,25 @@ SANDBOX_EXPIRED_RECORDS_CLEAN_BATCH_MAX_INTERVAL=200
SANDBOX_EXPIRED_RECORDS_RETENTION_DAYS=30
# Redis URL used for PubSub between API and
# Redis URL used for event bus between API and
# celery worker
# defaults to url constructed from `REDIS_*`
# configurations
PUBSUB_REDIS_URL=
# Pub/sub channel type for streaming events.
# valid options are:
EVENT_BUS_REDIS_URL=
# Event transport type. Options are:
#
# - pubsub: for normal Pub/Sub
# - sharded: for sharded Pub/Sub
# - pubsub: normal Pub/Sub (at-most-once)
# - sharded: sharded Pub/Sub (at-most-once)
# - streams: Redis Streams (at-least-once, recommended to avoid subscriber races)
#
# It's highly recommended to use sharded Pub/Sub AND redis cluster
# for large deployments.
PUBSUB_REDIS_CHANNEL_TYPE=pubsub
# Whether to use Redis cluster mode while running
# PubSub.
# Note: Before enabling 'streams' in production, estimate your expected event volume and retention needs.
# Configure Redis memory limits and stream trimming appropriately (e.g., MAXLEN and key expiry) to reduce
# the risk of data loss from Redis auto-eviction under memory pressure.
# Also accepts ENV: EVENT_BUS_REDIS_CHANNEL_TYPE.
EVENT_BUS_REDIS_CHANNEL_TYPE=pubsub
# Whether to use Redis cluster mode while use redis as event bus.
# It's highly recommended to enable this for large deployments.
PUBSUB_REDIS_USE_CLUSTERS=false
EVENT_BUS_REDIS_USE_CLUSTERS=false
# Whether to Enable human input timeout check task
ENABLE_HUMAN_INPUT_TIMEOUT_TASK=true
+5 -5
View File
@@ -21,7 +21,7 @@ services:
# API service
api:
image: langgenius/dify-api:1.13.0
image: langgenius/dify-api:1.13.1
restart: always
environment:
# Use the shared environment variables.
@@ -63,7 +63,7 @@ services:
# worker service
# The Celery worker for processing all queues (dataset, workflow, mail, etc.)
worker:
image: langgenius/dify-api:1.13.0
image: langgenius/dify-api:1.13.1
restart: always
environment:
# Use the shared environment variables.
@@ -102,7 +102,7 @@ services:
# worker_beat service
# Celery beat for scheduling periodic tasks.
worker_beat:
image: langgenius/dify-api:1.13.0
image: langgenius/dify-api:1.13.1
restart: always
environment:
# Use the shared environment variables.
@@ -132,7 +132,7 @@ services:
# Frontend web application.
web:
image: langgenius/dify-web:1.13.0
image: langgenius/dify-web:1.13.1
restart: always
environment:
CONSOLE_API_URL: ${CONSOLE_API_URL:-}
@@ -269,7 +269,7 @@ services:
# plugin daemon
plugin_daemon:
image: langgenius/dify-plugin-daemon:0.5.3-local
image: langgenius/dify-plugin-daemon:0.5.4-local
restart: always
environment:
# Use the shared environment variables.
+1 -1
View File
@@ -123,7 +123,7 @@ services:
# plugin daemon
plugin_daemon:
image: langgenius/dify-plugin-daemon:0.5.3-local
image: langgenius/dify-plugin-daemon:0.5.4-local
restart: always
env_file:
- ./middleware.env
+8 -8
View File
@@ -699,9 +699,9 @@ x-shared-env: &shared-api-worker-env
SANDBOX_EXPIRED_RECORDS_CLEAN_BATCH_SIZE: ${SANDBOX_EXPIRED_RECORDS_CLEAN_BATCH_SIZE:-1000}
SANDBOX_EXPIRED_RECORDS_CLEAN_BATCH_MAX_INTERVAL: ${SANDBOX_EXPIRED_RECORDS_CLEAN_BATCH_MAX_INTERVAL:-200}
SANDBOX_EXPIRED_RECORDS_RETENTION_DAYS: ${SANDBOX_EXPIRED_RECORDS_RETENTION_DAYS:-30}
PUBSUB_REDIS_URL: ${PUBSUB_REDIS_URL:-}
PUBSUB_REDIS_CHANNEL_TYPE: ${PUBSUB_REDIS_CHANNEL_TYPE:-pubsub}
PUBSUB_REDIS_USE_CLUSTERS: ${PUBSUB_REDIS_USE_CLUSTERS:-false}
EVENT_BUS_REDIS_URL: ${EVENT_BUS_REDIS_URL:-}
EVENT_BUS_REDIS_CHANNEL_TYPE: ${EVENT_BUS_REDIS_CHANNEL_TYPE:-pubsub}
EVENT_BUS_REDIS_USE_CLUSTERS: ${EVENT_BUS_REDIS_USE_CLUSTERS:-false}
ENABLE_HUMAN_INPUT_TIMEOUT_TASK: ${ENABLE_HUMAN_INPUT_TIMEOUT_TASK:-true}
HUMAN_INPUT_TIMEOUT_TASK_INTERVAL: ${HUMAN_INPUT_TIMEOUT_TASK_INTERVAL:-1}
SANDBOX_EXPIRED_RECORDS_CLEAN_TASK_LOCK_TTL: ${SANDBOX_EXPIRED_RECORDS_CLEAN_TASK_LOCK_TTL:-90000}
@@ -728,7 +728,7 @@ services:
# API service
api:
image: langgenius/dify-api:1.13.0
image: langgenius/dify-api:1.13.1
restart: always
environment:
# Use the shared environment variables.
@@ -770,7 +770,7 @@ services:
# worker service
# The Celery worker for processing all queues (dataset, workflow, mail, etc.)
worker:
image: langgenius/dify-api:1.13.0
image: langgenius/dify-api:1.13.1
restart: always
environment:
# Use the shared environment variables.
@@ -809,7 +809,7 @@ services:
# worker_beat service
# Celery beat for scheduling periodic tasks.
worker_beat:
image: langgenius/dify-api:1.13.0
image: langgenius/dify-api:1.13.1
restart: always
environment:
# Use the shared environment variables.
@@ -839,7 +839,7 @@ services:
# Frontend web application.
web:
image: langgenius/dify-web:1.13.0
image: langgenius/dify-web:1.13.1
restart: always
environment:
CONSOLE_API_URL: ${CONSOLE_API_URL:-}
@@ -976,7 +976,7 @@ services:
# plugin daemon
plugin_daemon:
image: langgenius/dify-plugin-daemon:0.5.3-local
image: langgenius/dify-plugin-daemon:0.5.4-local
restart: always
environment:
# Use the shared environment variables.
+4
View File
@@ -8,6 +8,10 @@
- In new or modified code, use only overlay primitives from `@/app/components/base/ui/*`.
- Do not introduce deprecated overlay imports from `@/app/components/base/*`; when touching legacy callers, prefer migrating them and keep the allowlist shrinking (never expanding).
## Query & Mutation (Mandatory)
- `frontend-query-mutation` is the source of truth for Dify frontend contracts, query and mutation call-site patterns, conditional queries, invalidation, and mutation error handling.
## Automated Test Generation
- Use `./docs/test.md` as the canonical instruction set for generating frontend automated tests.
@@ -29,7 +29,7 @@ const mockOnPlanInfoChanged = vi.fn()
const mockDeleteAppMutation = vi.fn().mockResolvedValue(undefined)
let mockDeleteMutationPending = false
vi.mock('@/next/navigation', () => ({
vi.mock('next/navigation', () => ({
useRouter: () => ({
push: mockRouterPush,
}),
@@ -57,7 +57,7 @@ vi.mock('@headlessui/react', async () => {
}
})
vi.mock('@/next/dynamic', () => ({
vi.mock('next/dynamic', () => ({
default: (loader: () => Promise<{ default: React.ComponentType }>) => {
let Component: React.ComponentType<Record<string, unknown>> | null = null
loader().then((mod) => {
@@ -38,7 +38,7 @@ let mockShowTagManagementModal = false
const mockRouterPush = vi.fn()
const mockRouterReplace = vi.fn()
vi.mock('@/next/navigation', () => ({
vi.mock('next/navigation', () => ({
useRouter: () => ({
push: mockRouterPush,
replace: mockRouterReplace,
@@ -46,7 +46,7 @@ vi.mock('@/next/navigation', () => ({
useSearchParams: () => new URLSearchParams(),
}))
vi.mock('@/next/dynamic', () => ({
vi.mock('next/dynamic', () => ({
default: (_loader: () => Promise<{ default: React.ComponentType }>) => {
const LazyComponent = (props: Record<string, unknown>) => {
return <div data-testid="dynamic-component" {...props} />
+2 -2
View File
@@ -35,7 +35,7 @@ const mockRouterPush = vi.fn()
const mockRouterReplace = vi.fn()
const mockOnPlanInfoChanged = vi.fn()
vi.mock('@/next/navigation', () => ({
vi.mock('next/navigation', () => ({
useRouter: () => ({
push: mockRouterPush,
replace: mockRouterReplace,
@@ -117,7 +117,7 @@ vi.mock('ahooks', async () => {
})
// Mock dynamically loaded modals with test stubs
vi.mock('@/next/dynamic', () => ({
vi.mock('next/dynamic', () => ({
default: (loader: () => Promise<{ default: React.ComponentType }>) => {
let Component: React.ComponentType<Record<string, unknown>> | null = null
loader().then((mod) => {
@@ -64,7 +64,7 @@ vi.mock('@/service/use-education', () => ({
// ─── Navigation mocks ───────────────────────────────────────────────────────
const mockRouterPush = vi.fn()
vi.mock('@/next/navigation', () => ({
vi.mock('next/navigation', () => ({
useRouter: () => ({ push: mockRouterPush }),
usePathname: () => '/billing',
useSearchParams: () => new URLSearchParams(),
@@ -54,7 +54,7 @@ vi.mock('@/app/components/base/toast', () => ({
}))
// ─── Navigation mocks ───────────────────────────────────────────────────────
vi.mock('@/next/navigation', () => ({
vi.mock('next/navigation', () => ({
useRouter: () => ({ push: vi.fn() }),
usePathname: () => '/billing',
useSearchParams: () => new URLSearchParams(),
@@ -63,7 +63,7 @@ vi.mock('@/service/use-billing', () => ({
}))
// ─── Navigation mocks ───────────────────────────────────────────────────────
vi.mock('@/next/navigation', () => ({
vi.mock('next/navigation', () => ({
useRouter: () => ({ push: mockRouterPush }),
usePathname: () => '/billing',
useSearchParams: () => new URLSearchParams(),
@@ -18,7 +18,7 @@ let mockSearchParams = new URLSearchParams()
const mockMutateAsync = vi.fn()
// ─── Module mocks ────────────────────────────────────────────────────────────
vi.mock('@/next/navigation', () => ({
vi.mock('next/navigation', () => ({
useSearchParams: () => mockSearchParams,
useRouter: () => ({ push: vi.fn() }),
usePathname: () => '/',
@@ -51,7 +51,7 @@ vi.mock('@/hooks/use-async-window-open', () => ({
}))
// ─── Navigation mocks ───────────────────────────────────────────────────────
vi.mock('@/next/navigation', () => ({
vi.mock('next/navigation', () => ({
useRouter: () => ({ push: vi.fn() }),
usePathname: () => '/billing',
useSearchParams: () => new URLSearchParams(),
@@ -0,0 +1,221 @@
import {
buildGitDiffRevisionArgs,
getChangedBranchCoverage,
getChangedStatementCoverage,
getIgnoredChangedLinesFromSource,
normalizeToRepoRelative,
parseChangedLineMap,
} from '../scripts/check-components-diff-coverage-lib.mjs'
describe('check-components-diff-coverage helpers', () => {
it('should build exact and merge-base git diff revision args', () => {
expect(buildGitDiffRevisionArgs('base-sha', 'head-sha', 'exact')).toEqual(['base-sha', 'head-sha'])
expect(buildGitDiffRevisionArgs('base-sha', 'head-sha')).toEqual(['base-sha...head-sha'])
})
it('should parse changed line maps from unified diffs', () => {
const diff = [
'diff --git a/web/app/components/share/a.ts b/web/app/components/share/a.ts',
'+++ b/web/app/components/share/a.ts',
'@@ -10,0 +11,2 @@',
'+const a = 1',
'+const b = 2',
'diff --git a/web/app/components/base/b.ts b/web/app/components/base/b.ts',
'+++ b/web/app/components/base/b.ts',
'@@ -20 +21 @@',
'+const c = 3',
'diff --git a/web/README.md b/web/README.md',
'+++ b/web/README.md',
'@@ -1 +1 @@',
'+ignore me',
].join('\n')
const lineMap = parseChangedLineMap(diff, (filePath: string) => filePath.startsWith('web/app/components/'))
expect([...lineMap.entries()]).toEqual([
['web/app/components/share/a.ts', new Set([11, 12])],
['web/app/components/base/b.ts', new Set([21])],
])
})
it('should normalize coverage and absolute paths to repo-relative paths', () => {
const repoRoot = '/repo'
const webRoot = '/repo/web'
expect(normalizeToRepoRelative('web/app/components/share/a.ts', {
appComponentsCoveragePrefix: 'app/components/',
appComponentsPrefix: 'web/app/components/',
repoRoot,
sharedTestPrefix: 'web/__tests__/',
webRoot,
})).toBe('web/app/components/share/a.ts')
expect(normalizeToRepoRelative('app/components/share/a.ts', {
appComponentsCoveragePrefix: 'app/components/',
appComponentsPrefix: 'web/app/components/',
repoRoot,
sharedTestPrefix: 'web/__tests__/',
webRoot,
})).toBe('web/app/components/share/a.ts')
expect(normalizeToRepoRelative('/repo/web/app/components/share/a.ts', {
appComponentsCoveragePrefix: 'app/components/',
appComponentsPrefix: 'web/app/components/',
repoRoot,
sharedTestPrefix: 'web/__tests__/',
webRoot,
})).toBe('web/app/components/share/a.ts')
})
it('should calculate changed statement coverage from changed lines', () => {
const entry = {
s: { 0: 1, 1: 0 },
statementMap: {
0: { start: { line: 10 }, end: { line: 10 } },
1: { start: { line: 12 }, end: { line: 13 } },
},
}
const coverage = getChangedStatementCoverage(entry, new Set([10, 12]))
expect(coverage).toEqual({
covered: 1,
total: 2,
uncoveredLines: [12],
})
})
it('should report the first changed line inside a multi-line uncovered statement', () => {
const entry = {
s: { 0: 0 },
statementMap: {
0: { start: { line: 10 }, end: { line: 14 } },
},
}
const coverage = getChangedStatementCoverage(entry, new Set([13, 14]))
expect(coverage).toEqual({
covered: 0,
total: 1,
uncoveredLines: [13],
})
})
it('should fail changed lines when a source file has no coverage entry', () => {
const coverage = getChangedStatementCoverage(undefined, new Set([42, 43]))
expect(coverage).toEqual({
covered: 0,
total: 2,
uncoveredLines: [42, 43],
})
})
it('should calculate changed branch coverage using changed branch definitions', () => {
const entry = {
b: {
0: [1, 0],
},
branchMap: {
0: {
line: 20,
loc: { start: { line: 20 }, end: { line: 20 } },
locations: [
{ start: { line: 20 }, end: { line: 20 } },
{ start: { line: 21 }, end: { line: 21 } },
],
type: 'if',
},
},
}
const coverage = getChangedBranchCoverage(entry, new Set([20]))
expect(coverage).toEqual({
covered: 1,
total: 2,
uncoveredBranches: [
{ armIndex: 1, line: 21 },
],
})
})
it('should report the first changed line inside a multi-line uncovered branch arm', () => {
const entry = {
b: {
0: [0, 0],
},
branchMap: {
0: {
line: 30,
loc: { start: { line: 30 }, end: { line: 35 } },
locations: [
{ start: { line: 31 }, end: { line: 34 } },
{ start: { line: 35 }, end: { line: 38 } },
],
type: 'if',
},
},
}
const coverage = getChangedBranchCoverage(entry, new Set([33]))
expect(coverage).toEqual({
covered: 0,
total: 1,
uncoveredBranches: [
{ armIndex: 0, line: 33 },
],
})
})
it('should require all branch arms when the branch condition changes', () => {
const entry = {
b: {
0: [0, 0],
},
branchMap: {
0: {
line: 30,
loc: { start: { line: 30 }, end: { line: 35 } },
locations: [
{ start: { line: 31 }, end: { line: 34 } },
{ start: { line: 35 }, end: { line: 38 } },
],
type: 'if',
},
},
}
const coverage = getChangedBranchCoverage(entry, new Set([30]))
expect(coverage).toEqual({
covered: 0,
total: 2,
uncoveredBranches: [
{ armIndex: 0, line: 31 },
{ armIndex: 1, line: 35 },
],
})
})
it('should ignore changed lines with valid pragma reasons and report invalid pragmas', () => {
const sourceCode = [
'const a = 1',
'const b = 2 // diff-coverage-ignore-line: defensive fallback',
'const c = 3 // diff-coverage-ignore-line:',
'const d = 4 // diff-coverage-ignore-line: not changed',
].join('\n')
const result = getIgnoredChangedLinesFromSource(sourceCode, new Set([2, 3]))
expect([...result.effectiveChangedLines]).toEqual([3])
expect([...result.ignoredLines.entries()]).toEqual([
[2, 'defensive fallback'],
])
expect(result.invalidPragmas).toEqual([
{ line: 3, reason: 'missing ignore reason' },
])
})
})
@@ -0,0 +1,72 @@
import {
getCoverageStats,
isRelevantTestFile,
isTrackedComponentSourceFile,
loadTrackedCoverageEntries,
} from '../scripts/components-coverage-common.mjs'
describe('components coverage common helpers', () => {
it('should identify tracked component source files and relevant tests', () => {
const excludedComponentCoverageFiles = new Set([
'web/app/components/share/types.ts',
])
expect(isTrackedComponentSourceFile('web/app/components/share/index.tsx', excludedComponentCoverageFiles)).toBe(true)
expect(isTrackedComponentSourceFile('web/app/components/share/types.ts', excludedComponentCoverageFiles)).toBe(false)
expect(isTrackedComponentSourceFile('web/app/components/provider/index.tsx', excludedComponentCoverageFiles)).toBe(false)
expect(isRelevantTestFile('web/__tests__/share/text-generation-run-once-flow.test.tsx')).toBe(true)
expect(isRelevantTestFile('web/app/components/share/__tests__/index.spec.tsx')).toBe(true)
expect(isRelevantTestFile('web/utils/format.spec.ts')).toBe(false)
})
it('should load only tracked coverage entries from mixed coverage paths', () => {
const context = {
excludedComponentCoverageFiles: new Set([
'web/app/components/share/types.ts',
]),
repoRoot: '/repo',
webRoot: '/repo/web',
}
const coverage = {
'/repo/web/app/components/provider/index.tsx': {
path: '/repo/web/app/components/provider/index.tsx',
statementMap: { 0: { start: { line: 1 }, end: { line: 1 } } },
s: { 0: 1 },
},
'app/components/share/index.tsx': {
path: 'app/components/share/index.tsx',
statementMap: { 0: { start: { line: 2 }, end: { line: 2 } } },
s: { 0: 1 },
},
'app/components/share/types.ts': {
path: 'app/components/share/types.ts',
statementMap: { 0: { start: { line: 3 }, end: { line: 3 } } },
s: { 0: 1 },
},
}
expect([...loadTrackedCoverageEntries(coverage, context).keys()]).toEqual([
'web/app/components/share/index.tsx',
])
})
it('should calculate coverage stats using statement-derived line hits', () => {
const entry = {
b: { 0: [1, 0] },
f: { 0: 1, 1: 0 },
s: { 0: 1, 1: 0 },
statementMap: {
0: { start: { line: 10 }, end: { line: 10 } },
1: { start: { line: 12 }, end: { line: 13 } },
},
}
expect(getCoverageStats(entry)).toEqual({
branches: { covered: 1, total: 2 },
functions: { covered: 1, total: 2 },
lines: { covered: 1, total: 2 },
statements: { covered: 1, total: 2 },
})
})
})
@@ -13,7 +13,7 @@ import { DataSourceType } from '@/models/datasets'
import { renderHookWithNuqs } from '@/test/nuqs-testing'
const mockPush = vi.fn()
vi.mock('@/next/navigation', () => ({
vi.mock('next/navigation', () => ({
useSearchParams: () => new URLSearchParams(''),
useRouter: () => ({ push: mockPush }),
usePathname: () => '/datasets/ds-1/documents',
@@ -7,12 +7,12 @@ import type { Mock } from 'vitest'
*/
import { fireEvent, render, screen } from '@testing-library/react'
import { useRouter } from '@/next/navigation'
import { useRouter } from 'next/navigation'
import { useDocumentDetail, useDocumentMetadata } from '@/service/knowledge/use-document'
// Mock Next.js router
const mockPush = vi.fn()
vi.mock('@/next/navigation', () => ({
vi.mock('next/navigation', () => ({
useRouter: vi.fn(() => ({
push: mockPush,
})),
+1 -1
View File
@@ -8,7 +8,7 @@ const replaceMock = vi.fn()
const backMock = vi.fn()
const useSearchParamsMock = vi.fn(() => new URLSearchParams())
vi.mock('@/next/navigation', () => ({
vi.mock('next/navigation', () => ({
usePathname: vi.fn(() => '/chatbot/test-app'),
useRouter: vi.fn(() => ({
replace: replaceMock,
@@ -4,7 +4,7 @@ import WebAppStoreProvider, { useWebAppStore } from '@/context/web-app-context'
import { AccessMode } from '@/models/access-control'
vi.mock('@/next/navigation', () => ({
vi.mock('next/navigation', () => ({
usePathname: vi.fn(() => '/chatbot/sample-app'),
useSearchParams: vi.fn(() => {
const params = new URLSearchParams()
@@ -19,7 +19,7 @@ const mockUninstall = vi.fn()
const mockUpdatePinStatus = vi.fn()
let mockInstalledApps: InstalledApp[] = []
vi.mock('@/next/navigation', () => ({
vi.mock('next/navigation', () => ({
useSelectedLayoutSegments: () => mockSegments,
useRouter: () => ({
push: mockPush,
@@ -5,7 +5,7 @@ import TextGeneration from '@/app/components/share/text-generation'
const useSearchParamsMock = vi.fn(() => new URLSearchParams())
vi.mock('@/next/navigation', () => ({
vi.mock('next/navigation', () => ({
useSearchParams: () => useSearchParamsMock(),
}))
@@ -13,6 +13,8 @@ import {
RiTerminalWindowLine,
} from '@remixicon/react'
import { useUnmount } from 'ahooks'
import dynamic from 'next/dynamic'
import { usePathname, useRouter } from 'next/navigation'
import * as React from 'react'
import { useCallback, useEffect, useState } from 'react'
import { useTranslation } from 'react-i18next'
@@ -24,8 +26,6 @@ import { useStore as useTagStore } from '@/app/components/base/tag-management/st
import { useAppContext } from '@/context/app-context'
import useBreakpoints, { MediaType } from '@/hooks/use-breakpoints'
import useDocumentTitle from '@/hooks/use-document-title'
import dynamic from '@/next/dynamic'
import { usePathname, useRouter } from '@/next/navigation'
import { fetchAppDetailDirect } from '@/service/apps'
import { AppModeEnum } from '@/types/app'
import { cn } from '@/utils/classnames'
@@ -7,6 +7,7 @@ import {
RiEqualizer2Line,
} from '@remixicon/react'
import { useBoolean } from 'ahooks'
import { usePathname } from 'next/navigation'
import * as React from 'react'
import { useEffect, useState } from 'react'
import { useTranslation } from 'react-i18next'
@@ -16,7 +17,6 @@ import Loading from '@/app/components/base/loading'
import Toast from '@/app/components/base/toast'
import Indicator from '@/app/components/header/indicator'
import { useAppContext } from '@/context/app-context'
import { usePathname } from '@/next/navigation'
import { fetchTracingConfig as doFetchTracingConfig, fetchTracingStatus, updateTracingStatus } from '@/service/apps'
import { cn } from '@/utils/classnames'
import ConfigButton from './config-button'
@@ -254,7 +254,7 @@ const Panel: FC = () => {
)}
>
<TracingIcon size="md" />
<div className="mx-2 text-text-secondary system-sm-semibold">{t(`${I18N_PREFIX}.title`, { ns: 'app' })}</div>
<div className="system-sm-semibold mx-2 text-text-secondary">{t(`${I18N_PREFIX}.title`, { ns: 'app' })}</div>
<div className="rounded-md p-1">
<RiEqualizer2Line className="h-4 w-4 text-text-tertiary" />
</div>
@@ -294,7 +294,7 @@ const Panel: FC = () => {
>
<div className="ml-4 mr-1 flex items-center">
<Indicator color={enabled ? 'green' : 'gray'} />
<div className="ml-1.5 text-text-tertiary system-xs-semibold-uppercase">
<div className="system-xs-semibold-uppercase ml-1.5 text-text-tertiary">
{t(`${I18N_PREFIX}.${enabled ? 'enabled' : 'disabled'}`, { ns: 'app' })}
</div>
</div>
@@ -9,6 +9,7 @@ import {
RiFocus2Fill,
RiFocus2Line,
} from '@remixicon/react'
import { usePathname } from 'next/navigation'
import * as React from 'react'
import { useEffect, useMemo, useState } from 'react'
import { useTranslation } from 'react-i18next'
@@ -22,7 +23,6 @@ import DatasetDetailContext from '@/context/dataset-detail'
import { useEventEmitterContextContext } from '@/context/event-emitter'
import useBreakpoints, { MediaType } from '@/hooks/use-breakpoints'
import useDocumentTitle from '@/hooks/use-document-title'
import { usePathname } from '@/next/navigation'
import { useDatasetDetail, useDatasetRelatedApps } from '@/service/knowledge/use-dataset'
import { cn } from '@/utils/classnames'
@@ -6,7 +6,7 @@ import DatasetsLayout from './layout'
const mockReplace = vi.fn()
const mockUseAppContext = vi.fn()
vi.mock('@/next/navigation', () => ({
vi.mock('next/navigation', () => ({
useRouter: () => ({
replace: mockReplace,
}),
+1 -1
View File
@@ -1,11 +1,11 @@
'use client'
import { useRouter } from 'next/navigation'
import { useEffect } from 'react'
import Loading from '@/app/components/base/loading'
import { useAppContext } from '@/context/app-context'
import { ExternalApiPanelProvider } from '@/context/external-api-panel-context'
import { ExternalKnowledgeApiProvider } from '@/context/external-knowledge-api-context'
import { useRouter } from '@/next/navigation'
export default function DatasetsLayout({ children }: { children: React.ReactNode }) {
const { isCurrentWorkspaceEditor, isCurrentWorkspaceDatasetOperator, currentWorkspace, isLoadingCurrentWorkspace } = useAppContext()
@@ -1,15 +1,15 @@
'use client'
import {
useRouter,
useSearchParams,
} from 'next/navigation'
import {
useEffect,
useMemo,
} from 'react'
import EducationApplyPage from '@/app/education-apply/education-apply-page'
import { useProviderContext } from '@/context/provider-context'
import {
useRouter,
useSearchParams,
} from '@/next/navigation'
export default function EducationApply() {
const router = useRouter()
@@ -6,7 +6,7 @@ const mockReplace = vi.fn()
const mockUseAppContext = vi.fn()
let mockPathname = '/apps'
vi.mock('@/next/navigation', () => ({
vi.mock('next/navigation', () => ({
usePathname: () => mockPathname,
useRouter: () => ({
replace: mockReplace,
+1 -1
View File
@@ -1,10 +1,10 @@
'use client'
import type { ReactNode } from 'react'
import { usePathname, useRouter } from 'next/navigation'
import { useEffect } from 'react'
import Loading from '@/app/components/base/loading'
import { useAppContext } from '@/context/app-context'
import { usePathname, useRouter } from '@/next/navigation'
const datasetOperatorRedirectRoutes = ['/apps', '/app', '/explore', '/tools'] as const
@@ -9,6 +9,7 @@ import {
RiInformation2Fill,
} from '@remixicon/react'
import { produce } from 'immer'
import { useParams } from 'next/navigation'
import * as React from 'react'
import { useEffect, useMemo, useState } from 'react'
import { useTranslation } from 'react-i18next'
@@ -20,7 +21,6 @@ import { getButtonStyle } from '@/app/components/base/chat/chat/answer/human-inp
import Loading from '@/app/components/base/loading'
import DifyLogo from '@/app/components/base/logo/dify-logo'
import useDocumentTitle from '@/hooks/use-document-title'
import { useParams } from '@/next/navigation'
import { useGetHumanInputForm, useSubmitHumanInputForm } from '@/service/use-share'
import { cn } from '@/utils/classnames'
@@ -106,17 +106,17 @@ const FormContent = () => {
<RiCheckboxCircleFill className="h-8 w-8 text-text-success" />
</div>
<div className="grow">
<div className="text-text-primary title-4xl-semi-bold">{t('humanInput.thanks', { ns: 'share' })}</div>
<div className="text-text-primary title-4xl-semi-bold">{t('humanInput.recorded', { ns: 'share' })}</div>
<div className="title-4xl-semi-bold text-text-primary">{t('humanInput.thanks', { ns: 'share' })}</div>
<div className="title-4xl-semi-bold text-text-primary">{t('humanInput.recorded', { ns: 'share' })}</div>
</div>
<div className="shrink-0 text-text-tertiary system-2xs-regular-uppercase">{t('humanInput.submissionID', { id: token, ns: 'share' })}</div>
<div className="system-2xs-regular-uppercase shrink-0 text-text-tertiary">{t('humanInput.submissionID', { id: token, ns: 'share' })}</div>
</div>
<div className="flex flex-row-reverse px-2 py-3">
<div className={cn(
'flex shrink-0 items-center gap-1.5 px-1',
)}
>
<div className="text-text-tertiary system-2xs-medium-uppercase">{t('chat.poweredBy', { ns: 'share' })}</div>
<div className="system-2xs-medium-uppercase text-text-tertiary">{t('chat.poweredBy', { ns: 'share' })}</div>
<DifyLogo size="small" />
</div>
</div>
@@ -134,17 +134,17 @@ const FormContent = () => {
<RiInformation2Fill className="h-8 w-8 text-text-accent" />
</div>
<div className="grow">
<div className="text-text-primary title-4xl-semi-bold">{t('humanInput.sorry', { ns: 'share' })}</div>
<div className="text-text-primary title-4xl-semi-bold">{t('humanInput.expired', { ns: 'share' })}</div>
<div className="title-4xl-semi-bold text-text-primary">{t('humanInput.sorry', { ns: 'share' })}</div>
<div className="title-4xl-semi-bold text-text-primary">{t('humanInput.expired', { ns: 'share' })}</div>
</div>
<div className="shrink-0 text-text-tertiary system-2xs-regular-uppercase">{t('humanInput.submissionID', { id: token, ns: 'share' })}</div>
<div className="system-2xs-regular-uppercase shrink-0 text-text-tertiary">{t('humanInput.submissionID', { id: token, ns: 'share' })}</div>
</div>
<div className="flex flex-row-reverse px-2 py-3">
<div className={cn(
'flex shrink-0 items-center gap-1.5 px-1',
)}
>
<div className="text-text-tertiary system-2xs-medium-uppercase">{t('chat.poweredBy', { ns: 'share' })}</div>
<div className="system-2xs-medium-uppercase text-text-tertiary">{t('chat.poweredBy', { ns: 'share' })}</div>
<DifyLogo size="small" />
</div>
</div>
@@ -162,17 +162,17 @@ const FormContent = () => {
<RiInformation2Fill className="h-8 w-8 text-text-accent" />
</div>
<div className="grow">
<div className="text-text-primary title-4xl-semi-bold">{t('humanInput.sorry', { ns: 'share' })}</div>
<div className="text-text-primary title-4xl-semi-bold">{t('humanInput.completed', { ns: 'share' })}</div>
<div className="title-4xl-semi-bold text-text-primary">{t('humanInput.sorry', { ns: 'share' })}</div>
<div className="title-4xl-semi-bold text-text-primary">{t('humanInput.completed', { ns: 'share' })}</div>
</div>
<div className="shrink-0 text-text-tertiary system-2xs-regular-uppercase">{t('humanInput.submissionID', { id: token, ns: 'share' })}</div>
<div className="system-2xs-regular-uppercase shrink-0 text-text-tertiary">{t('humanInput.submissionID', { id: token, ns: 'share' })}</div>
</div>
<div className="flex flex-row-reverse px-2 py-3">
<div className={cn(
'flex shrink-0 items-center gap-1.5 px-1',
)}
>
<div className="text-text-tertiary system-2xs-medium-uppercase">{t('chat.poweredBy', { ns: 'share' })}</div>
<div className="system-2xs-medium-uppercase text-text-tertiary">{t('chat.poweredBy', { ns: 'share' })}</div>
<DifyLogo size="small" />
</div>
</div>
@@ -190,7 +190,7 @@ const FormContent = () => {
<RiErrorWarningFill className="h-8 w-8 text-text-destructive" />
</div>
<div className="grow">
<div className="text-text-primary title-4xl-semi-bold">{t('humanInput.rateLimitExceeded', { ns: 'share' })}</div>
<div className="title-4xl-semi-bold text-text-primary">{t('humanInput.rateLimitExceeded', { ns: 'share' })}</div>
</div>
</div>
<div className="flex flex-row-reverse px-2 py-3">
@@ -198,7 +198,7 @@ const FormContent = () => {
'flex shrink-0 items-center gap-1.5 px-1',
)}
>
<div className="text-text-tertiary system-2xs-medium-uppercase">{t('chat.poweredBy', { ns: 'share' })}</div>
<div className="system-2xs-medium-uppercase text-text-tertiary">{t('chat.poweredBy', { ns: 'share' })}</div>
<DifyLogo size="small" />
</div>
</div>
@@ -216,7 +216,7 @@ const FormContent = () => {
<RiErrorWarningFill className="h-8 w-8 text-text-destructive" />
</div>
<div className="grow">
<div className="text-text-primary title-4xl-semi-bold">{t('humanInput.formNotFound', { ns: 'share' })}</div>
<div className="title-4xl-semi-bold text-text-primary">{t('humanInput.formNotFound', { ns: 'share' })}</div>
</div>
</div>
<div className="flex flex-row-reverse px-2 py-3">
@@ -224,7 +224,7 @@ const FormContent = () => {
'flex shrink-0 items-center gap-1.5 px-1',
)}
>
<div className="text-text-tertiary system-2xs-medium-uppercase">{t('chat.poweredBy', { ns: 'share' })}</div>
<div className="system-2xs-medium-uppercase text-text-tertiary">{t('chat.poweredBy', { ns: 'share' })}</div>
<DifyLogo size="small" />
</div>
</div>
@@ -245,7 +245,7 @@ const FormContent = () => {
background={site.icon_background}
imageUrl={site.icon_url}
/>
<div className="grow text-text-primary system-xl-semibold">{site.title}</div>
<div className="system-xl-semibold grow text-text-primary">{site.title}</div>
</div>
<div className="h-0 w-full grow overflow-y-auto">
<div className="border-components-divider-subtle rounded-[20px] border bg-chat-bubble-bg p-4 shadow-lg backdrop-blur-sm">
@@ -277,7 +277,7 @@ const FormContent = () => {
'flex shrink-0 items-center gap-1.5 px-1',
)}
>
<div className="text-text-tertiary system-2xs-medium-uppercase">{t('chat.poweredBy', { ns: 'share' })}</div>
<div className="system-2xs-medium-uppercase text-text-tertiary">{t('chat.poweredBy', { ns: 'share' })}</div>
<DifyLogo size="small" />
</div>
</div>
@@ -1,12 +1,12 @@
'use client'
import { usePathname, useRouter, useSearchParams } from 'next/navigation'
import * as React from 'react'
import { useCallback, useEffect } from 'react'
import { useTranslation } from 'react-i18next'
import AppUnavailable from '@/app/components/base/app-unavailable'
import Loading from '@/app/components/base/loading'
import { useWebAppStore } from '@/context/web-app-context'
import { usePathname, useRouter, useSearchParams } from '@/next/navigation'
import { useGetUserCanAccessApp } from '@/service/access-control'
import { useGetWebAppInfo, useGetWebAppMeta, useGetWebAppParams } from '@/service/use-share'
import { webAppLogout } from '@/service/webapp-auth'
@@ -81,7 +81,7 @@ const AuthenticatedLayout = ({ children }: { children: React.ReactNode }) => {
return (
<div className="flex h-full flex-col items-center justify-center gap-y-2">
<AppUnavailable className="h-auto w-auto" code={403} unknownReason="no permission." />
<span className="cursor-pointer text-text-tertiary system-sm-regular" onClick={backToHome}>{t('userProfile.logout', { ns: 'common' })}</span>
<span className="system-sm-regular cursor-pointer text-text-tertiary" onClick={backToHome}>{t('userProfile.logout', { ns: 'common' })}</span>
</div>
)
}
+2 -2
View File
@@ -1,11 +1,11 @@
'use client'
import type { FC, PropsWithChildren } from 'react'
import { useRouter, useSearchParams } from 'next/navigation'
import { useCallback, useEffect, useState } from 'react'
import { useTranslation } from 'react-i18next'
import AppUnavailable from '@/app/components/base/app-unavailable'
import Loading from '@/app/components/base/loading'
import { useWebAppStore } from '@/context/web-app-context'
import { useRouter, useSearchParams } from '@/next/navigation'
import { fetchAccessToken } from '@/service/share'
import { setWebAppAccessToken, setWebAppPassport, webAppLoginStatus, webAppLogout } from '@/service/webapp-auth'
@@ -95,7 +95,7 @@ const Splash: FC<PropsWithChildren> = ({ children }) => {
return (
<div className="flex h-full flex-col items-center justify-center gap-y-4">
<AppUnavailable className="h-auto w-auto" code={code || t('common.appUnavailable', { ns: 'share' })} unknownReason={message} />
<span className="cursor-pointer text-text-tertiary system-sm-regular" onClick={backToHome}>{code === '403' ? t('userProfile.logout', { ns: 'common' }) : t('login.backToHome', { ns: 'share' })}</span>
<span className="system-sm-regular cursor-pointer text-text-tertiary" onClick={backToHome}>{code === '403' ? t('userProfile.logout', { ns: 'common' }) : t('login.backToHome', { ns: 'share' })}</span>
</div>
)
}
@@ -1,14 +1,14 @@
'use client'
import { RiArrowLeftLine, RiMailSendFill } from '@remixicon/react'
import { useRouter, useSearchParams } from 'next/navigation'
import { useState } from 'react'
import { useTranslation } from 'react-i18next'
import Button from '@/app/components/base/button'
import Input from '@/app/components/base/input'
import Toast from '@/app/components/base/toast'
import Countdown from '@/app/components/signin/countdown'
import { useLocale } from '@/context/i18n'
import { useRouter, useSearchParams } from '@/next/navigation'
import { useLocale } from '@/context/i18n'
import { sendWebAppResetPasswordCode, verifyWebAppResetPasswordCode } from '@/service/common'
export default function CheckCode() {
@@ -69,8 +69,8 @@ export default function CheckCode() {
<RiMailSendFill className="h-6 w-6 text-2xl" />
</div>
<div className="pb-4 pt-2">
<h2 className="text-text-primary title-4xl-semi-bold">{t('checkCode.checkYourEmail', { ns: 'login' })}</h2>
<p className="mt-2 text-text-secondary body-md-regular">
<h2 className="title-4xl-semi-bold text-text-primary">{t('checkCode.checkYourEmail', { ns: 'login' })}</h2>
<p className="body-md-regular mt-2 text-text-secondary">
<span>
{t('checkCode.tipsPrefix', { ns: 'login' })}
<strong>{email}</strong>
@@ -82,7 +82,7 @@ export default function CheckCode() {
<form action="">
<input type="text" className="hidden" />
<label htmlFor="code" className="mb-1 text-text-secondary system-md-semibold">{t('checkCode.verificationCode', { ns: 'login' })}</label>
<label htmlFor="code" className="system-md-semibold mb-1 text-text-secondary">{t('checkCode.verificationCode', { ns: 'login' })}</label>
<Input value={code} onChange={e => setVerifyCode(e.target.value)} maxLength={6} className="mt-1" placeholder={t('checkCode.verificationCodePlaceholder', { ns: 'login' }) || ''} />
<Button loading={loading} disabled={loading} className="my-3 w-full" variant="primary" onClick={verify}>{t('checkCode.verify', { ns: 'login' })}</Button>
<Countdown onResend={resendCode} />
@@ -94,7 +94,7 @@ export default function CheckCode() {
<div className="bg-background-default-dimm inline-block rounded-full p-1">
<RiArrowLeftLine size={12} />
</div>
<span className="ml-2 system-xs-regular">{t('back', { ns: 'login' })}</span>
<span className="system-xs-regular ml-2">{t('back', { ns: 'login' })}</span>
</div>
</div>
)
@@ -1,6 +1,8 @@
'use client'
import { RiArrowLeftLine, RiLockPasswordLine } from '@remixicon/react'
import { noop } from 'es-toolkit/function'
import Link from 'next/link'
import { useRouter, useSearchParams } from 'next/navigation'
import { useState } from 'react'
import { useTranslation } from 'react-i18next'
import Button from '@/app/components/base/button'
@@ -8,11 +10,9 @@ import Input from '@/app/components/base/input'
import Toast from '@/app/components/base/toast'
import { COUNT_DOWN_KEY, COUNT_DOWN_TIME_MS } from '@/app/components/signin/countdown'
import { emailRegex } from '@/config'
import { useLocale } from '@/context/i18n'
import useDocumentTitle from '@/hooks/use-document-title'
import Link from '@/next/link'
import { useRouter, useSearchParams } from '@/next/navigation'
import { sendResetPasswordCode } from '@/service/common'
export default function CheckCode() {
@@ -74,8 +74,8 @@ export default function CheckCode() {
<RiLockPasswordLine className="h-6 w-6 text-2xl text-text-accent-light-mode-only" />
</div>
<div className="pb-4 pt-2">
<h2 className="text-text-primary title-4xl-semi-bold">{t('resetPassword', { ns: 'login' })}</h2>
<p className="mt-2 text-text-secondary body-md-regular">
<h2 className="title-4xl-semi-bold text-text-primary">{t('resetPassword', { ns: 'login' })}</h2>
<p className="body-md-regular mt-2 text-text-secondary">
{t('resetPasswordDesc', { ns: 'login' })}
</p>
</div>
@@ -83,7 +83,7 @@ export default function CheckCode() {
<form onSubmit={noop}>
<input type="text" className="hidden" />
<div className="mb-2">
<label htmlFor="email" className="my-2 text-text-secondary system-md-semibold">{t('email', { ns: 'login' })}</label>
<label htmlFor="email" className="system-md-semibold my-2 text-text-secondary">{t('email', { ns: 'login' })}</label>
<div className="mt-1">
<Input id="email" type="email" disabled={loading} value={email} placeholder={t('emailPlaceholder', { ns: 'login' }) as string} onChange={e => setEmail(e.target.value)} />
</div>
@@ -99,7 +99,7 @@ export default function CheckCode() {
<div className="inline-block rounded-full bg-background-default-dimmed p-1">
<RiArrowLeftLine size={12} />
</div>
<span className="ml-2 system-xs-regular">{t('backToLogin', { ns: 'login' })}</span>
<span className="system-xs-regular ml-2">{t('backToLogin', { ns: 'login' })}</span>
</Link>
</div>
)

Some files were not shown because too many files have changed in this diff Show More