Compare commits

..
Author SHA1 Message Date
crazywoola 8ad90a673b chore: replace x-shared-env anchor with env_file, drop template generator
The 1677-line generated docker-compose.yaml was ~440 lines of YAML-anchor
boilerplate (`x-shared-env: &shared-api-worker-env` populated by
`generate_docker_compose` from `.env.example`). Native compose has had
`env_file:` for years and supports interpolation in env files since 2.24,
so the anchor + generator pair was reimplementing what compose does itself.

This commit:

- Removes the `x-shared-env: &shared-api-worker-env` anchor and all
  `<<: *shared-api-worker-env` merges. Services that need the shared
  config (`api`, `worker`, `worker_beat`, `plugin_daemon`) now use
    env_file:
      - .env.example
      - .env
- Deletes `docker/docker-compose-template.yaml` (no longer needed; the
  generated compose file becomes the single source of truth).
- Deletes `docker/generate_docker_compose` (no template, no generation).

`docker/docker-compose.yaml` now contains the full, hand-readable compose
config in 958 lines (was 1677 — a 43% reduction). The default
`docker compose up -d` brings up exactly the same set of services
(api/web/worker/redis/postgres/weaviate/...) via the existing
profile-gating in COMPOSE_PROFILES.

Suggestion from #35925.

Behavior note: shared-config keys are no longer interpolated through
compose, so passing `KEY=value docker compose up -d` from the shell
no longer overrides container env for those keys — set them in
`docker/.env` instead. Service-specific overrides via the
`environment:` block continue to work as before.
2026-05-08 17:11:54 +08:00
crazywoola d2404d0375 chore: remove dify-compose wrappers, use native docker compose
#35708 added 660+ lines of bash + PowerShell to wrap docker compose
and merge a new .env.default with the user's .env. Native Compose
already provides everything that script does:

- `${VAR:-default}` interpolation in compose.yaml gives per-key
  defaults without a separate .env.default file.
- Auto-loaded .env in the project dir interpolates ${VAR:-default}
  on the value side, so COMPOSE_PROFILES=${VECTOR_STORE:-weaviate},
  ${DB_TYPE:-postgresql} expands correctly without a wrapper.
- `profiles:` on services already gates DB / vector-store selection.
- Maintaining two scripts (bash + PowerShell) duplicates env-file
  parsing logic that compose owns natively, with subtle differences
  (e.g. PowerShell's Get-Content uses system encoding, the bash
  script's awk parser doesn't handle inline `# comment` after a
  value).

Drop the wrappers and `.env.default`. Restore the README to the
two-step `cp .env.example .env && docker compose up -d` flow.
The .env.example placeholder cleanups landed in #35708 are kept.

Net: -712 lines, same UX modulo one extra `cp` step.
2026-05-08 16:48:00 +08:00
非法操作andGitHub 927a17804b feat: support configurable explore app categories (#35723) 2026-05-08 06:04:07 +00:00
Xiyuan ChenandGitHub 29f34848cd fix(tools): scope builtin tool default-credential clear to tenant (#35887) 2026-05-08 05:08:23 +00:00
1b0d4637b3 fix(i18n): update Turkish translations for new strings (#35905)
Co-authored-by: Baki Burak Öğün <b.burak.ogun@goc.local>
Co-authored-by: Crazywoola <100913391+crazywoola@users.noreply.github.com>
2026-05-08 03:24:02 +00:00
EvanYaoGitHubautofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
936a09c704 fix: replace SimpleNamespace with MagicMock(spec=App) in _app_stub (#34636) (#35897)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-05-08 03:18:21 +00:00
kien duongandGitHub 5cc62fd1c9 fix(web): pass end_user.external_user_id string to transcript_asr (#35898) 2026-05-08 02:55:32 +00:00
kien duongandGitHub 7bc19d8251 fix(web): explicitly resume AudioContext and play audio on first TTS load (#35901) 2026-05-08 02:51:14 +00:00
Stephen ZhouandGitHub e845475408 chore: update deps (#35907) 2026-05-08 02:33:18 +00:00
yyhandGitHub 9a8aa6a0c3 refactor(web): inline tag query defaults (#35883)
Signed-off-by: yyh <yuanyouhuilyz@gmail.com>
2026-05-08 02:06:10 +00:00
Stephen ZhouandGitHub 76a7f5f4b9 chore: update deps (#35812) 2026-05-08 01:32:14 +00:00
yyhandGitHub 2ff50514c8 refactor: migrate app selector to combobox (#35896) 2026-05-08 01:23:32 +00:00
Asuka MinatoandGitHub 7901ac9a97 chore: example of isinstance to match case (#35903) 2026-05-08 01:07:19 +00:00
Asuka MinatoGitHubautofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
ecd830083a test: add type to test (#35871)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-05-08 01:06:25 +00:00
203b3a9499 refactor(api): migrate console apikey responses to BaseModel (#35218)
Co-authored-by: ai-hpc <ai-hpc@users.noreply.github.com>
2026-05-07 16:12:25 +00:00
Coding On StarGitHubCodingOnStarautofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>yyh
9331024d91 refactor(web): portal to follow elem migration (#35892)
Co-authored-by: CodingOnStar <hanxujiang@dify.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: yyh <yuanyouhuilyz@gmail.com>
2026-05-07 13:02:11 +00:00
Escape0707andGitHub c6a5de3c18 fix: delete redundant api/libs/typing.py (#35890) 2026-05-07 11:16:29 +00:00
Escape0707andGitHub cd3327013a chore(test): Move plugin permission tests to testcontainers (#35884) 2026-05-07 09:57:53 +00:00
yyhandGitHub cd66559ebf refactor(web): migrate tag controls to combobox (#35881) 2026-05-07 08:55:13 +00:00
JoelGitHubautofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
8b77ec7f31 fix: transfer workspace dropdown not show (#35876)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-05-07 07:46:47 +00:00
yyhandGitHub bb3de5dd32 refactor: improve model selector search (#35875) 2026-05-07 07:05:14 +00:00
StreamandGitHub 1e2d309122 chore: easier and simpler deploy (#35708) 2026-05-07 06:01:32 +00:00
yyhandGitHub a24ec60e51 feat: add dify-ui autocomplete and combobox (#35868) 2026-05-07 05:39:13 +00:00
Asuka MinatoGitHubgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>Copilotautofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
8fd616d27f refactor: add type to test (#30873)
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-05-07 03:46:23 +00:00
Stephen ZhouandGitHub e5bdc40dce feat(dev-proxy): init package (#35852) 2026-05-07 03:32:14 +00:00
shawnYJGitHub非法操作autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
376c43e5ac fix(workflow): use correct field ID in KB metadata filter selection (#34149)
Co-authored-by: 非法操作 <hjlarry@163.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-05-07 02:21:30 +00:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
3ebb449d25 chore(deps): bump gitpython from 3.1.47 to 3.1.49 in /api (#35864)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-05-07 02:12:20 +00:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
5297ac76ec chore(deps): bump mako from 1.3.11 to 1.3.12 in /api (#35863)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-05-07 02:11:55 +00:00
zyssyz123GitHubautofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
bbed1d4a7c fix: update endpoint when update tidb status (#35854)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-05-07 01:54:07 +00:00
Asuka MinatoGitHubautofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
c804dbed8c ci: bump tyck (#35862)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-05-07 01:49:40 +00:00
yyhGitHubautofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
00bf3f83f2 refactor: verticalize tag management and batch bindings (#35840)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-05-07 01:36:10 +00:00
github-actions[bot]GitHubclaude[bot] <41898282+claude[bot]@users.noreply.github.com>
7e6745e105 chore(i18n): sync translations with en-US (#35853)
Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
2026-05-06 12:50:46 +00:00
JoelGitHubautofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
d648ce6888 chore: improve the progress of education pay (#35851)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-05-06 10:42:03 +00:00
Stephen ZhouGitHubautofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
f3c3534e33 chore(web): add enterprise dev proxy support (#35842)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-05-06 09:26:34 +00:00
yyhGitHubautofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
8967ff34b3 fix(web): normalize dify-ui overlay control (#35832)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-05-06 07:28:13 +00:00
sawyer-shiandGitHub 57539792c1 fix(workflow): unblock plugin model selector tools on 1.14.0 (#35794) 2026-05-06 07:07:43 +00:00
yyhGitHubautofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
03e227f8f1 fix(web): align Tailwind v4 CSS migration (#35829)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-05-06 06:20:28 +00:00
非法操作GitHubautofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
506e1a8bc7 fix: migrate condition operator popover (#35828)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-05-06 06:16:28 +00:00
yyhGitHubautofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
f8873ec07b refactor(webapp): migrate partial overlays (#35825)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-05-06 04:52:06 +00:00
yyhGitHubautofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
b2dacf0718 chore(tailwind-css): migrate to css first (#35754)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
2026-05-06 04:28:11 +00:00
70eb98d6c5 fix(file_factory): drop doubled dot when standardizing datasource file extension (#35808)
Co-authored-by: Beandon13 <bzarnitz23@gmail.com>
2026-05-06 03:03:22 +00:00
非法操作GitHubautofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
b83f296634 fix: restore workflow versions via backend API (#35817)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-05-06 02:56:10 +00:00
wangxiaoleiGitHubautofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
5c68f12bb8 fix: fix Working outside of application context (#35819)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-05-06 02:40:19 +00:00
wangxiaoleiandGitHub 4df7c00859 fix: fix test_sharded_channel failed (#35814) 2026-05-06 01:47:32 +00:00
yyhGitHubautofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
995c43f3dd refactor: migrate workflow queries to contracts (#35799)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-05-05 14:53:38 +00:00
非法操作GitHubautofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
c0431ec843 fix: workflow online users polling for large app lists (#35786)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-05-05 10:48:14 +00:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
a0af10abc8 chore(deps): bump the storage group across 1 directory with 2 updates (#35791)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-05-05 10:00:42 +00:00
yyhGitHubautofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
8e2b8168be refactor(web): migrate HITL overlays to base dialog (#35792)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-05-05 08:50:49 +00:00
kien duongGitHubAsuka Minatoautofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
1f29565673 fix(rag): use doc_id dedup key for any provider, not only dify (#35759)
Co-authored-by: Asuka Minato <i@asukaminato.eu.org>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-05-05 06:42:18 +00:00
yyhandGitHub 90fe54ca9e refactor(web): migrate workflow panel context menu primitive (#35787) 2026-05-04 23:12:26 +00:00
JingyiandGitHub b43ebf539d fix: preserve single-run input variable types (#35710) 2026-05-04 15:07:21 +00:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
853b859032 chore(deps-dev): bump the dev group in /api with 6 updates (#35782)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-05-04 14:22:26 +00:00
yyhandGitHub 8f3e42e9c2 refactor(web): migrate workflow node actions menu (#35785) 2026-05-04 13:24:29 +00:00
yyhGitHubautofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
1359c03216 refactor(web): migrate legacy tooltip to infotip (#35774)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-05-04 13:17:09 +00:00
Prince PalGitHubAsuka Minatoautofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
4b7dc17546 fix: skip empty documents before vector embedding (#35763)
Co-authored-by: Asuka Minato <i@asukaminato.eu.org>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-05-04 13:06:58 +00:00
guangyang1206GitHubyeranyangautofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
81090effe2 refactor(web): convert ValidatedStatus enum to as-const in key-valida… (#35749)
Co-authored-by: yeranyang <yeranyang@tencent.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-05-04 09:36:35 +00:00
aliworksx08andGitHub d92c336394 fix(web): secure external form help links (#35751) 2026-05-04 09:18:28 +00:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
cd9daef564 chore(deps): bump anthropics/claude-code-action from 1.0.110 to 1.0.111 in the github-actions-dependencies group (#35781)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-05-04 07:33:38 +00:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2876839d7e chore(deps): bump the google group in /api with 2 updates (#35779)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-05-04 07:33:12 +00:00
NeatGuyCodingandGitHub 7ba408eebe fix: IDOR on console GET /account/avatar (#35771)
Signed-off-by: NeatGuyCoding <15627489+NeatGuyCoding@users.noreply.github.com>
2026-05-03 14:42:56 +00:00
guangyang1206GitHubyeranyangAsuka Minatoautofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
3708e3eef1 refactor(tests): use db_session_with_containers in test_storage_key_loader (#35766)
Co-authored-by: yeranyang <yeranyang@tencent.com>
Co-authored-by: Asuka Minato <i@asukaminato.eu.org>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-05-02 17:49:20 +00:00
Jonathan ChangGitHubautofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
ff5c2c57a1 fix: Clean upload filenames parsed from URLs (#35706)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-05-02 14:45:31 +00:00
Asuka MinatoandGitHub 955c25589d ci: Remove API contracts generation step from autofix workflow (#35768) 2026-05-02 09:30:54 +00:00
Prince PalandGitHub 54bde0bdf6 fix(api): preserve dataset metadata filters (#35700) 2026-05-01 09:20:14 +00:00
87add9a4f3 refactor: replace Any with [T] syntax (#35750)
Co-authored-by: Asuka Minato <i@asukaminato.eu.org>
2026-04-30 14:08:29 +00:00
Stephen ZhouandGitHub 574d5865f4 chore: generate contact from api (#35748) 2026-04-30 11:34:27 +00:00
994 changed files with 77315 additions and 24728 deletions
@@ -1,6 +1,6 @@
---
name: frontend-query-mutation
description: Guide for implementing Dify frontend query and mutation patterns with TanStack Query and oRPC. Trigger when creating or updating contracts in web/contract, wiring router composition, consuming consoleQuery or marketplaceQuery in components or services, deciding whether to call queryOptions() directly or extract a helper or use-* hook, handling conditional queries, cache invalidation, mutation error handling, or migrating legacy service calls to contract-first query and mutation helpers.
description: Guide for implementing Dify frontend query and mutation patterns with TanStack Query and oRPC. Trigger when creating or updating contracts in web/contract, wiring router composition, consuming consoleQuery or marketplaceQuery in components or services, deciding whether to call queryOptions()/mutationOptions() directly or extract a helper or use-* hook, configuring oRPC experimental_defaults/default options, handling conditional queries, cache updates/invalidation, mutation error handling, or migrating legacy service calls to contract-first query and mutation helpers.
---
# Frontend Query & Mutation
@@ -9,22 +9,24 @@ description: Guide for implementing Dify frontend query and mutation patterns wi
- Keep contract as the single source of truth in `web/contract/*`.
- Prefer contract-shaped `queryOptions()` and `mutationOptions()`.
- Keep invalidation and mutation flow knowledge in the service layer.
- Keep default cache behavior with `consoleQuery`/`marketplaceQuery` setup, and keep business orchestration in feature vertical hooks when direct contract calls are not enough.
- Treat `web/service/use-*` query or mutation wrappers as legacy migration targets, not the preferred destination.
- Keep abstractions minimal to preserve TypeScript inference.
## Workflow
1. Identify the change surface.
- Read `references/contract-patterns.md` for contract files, router composition, client helpers, and query or mutation call-site shape.
- Read `references/runtime-rules.md` for conditional queries, invalidation, error handling, and legacy migrations.
- Read `references/runtime-rules.md` for conditional queries, default options, cache updates/invalidation, error handling, and legacy migrations.
- Read both references when a task spans contract shape and runtime behavior.
2. Implement the smallest abstraction that fits the task.
- Default to direct `useQuery(...)` or `useMutation(...)` calls with oRPC helpers at the call site.
- Extract a small shared query helper only when multiple call sites share the same extra options.
- Create `web/service/use-{domain}.ts` only for orchestration or shared domain behavior.
- Create or keep feature hooks only for real orchestration or shared domain behavior.
- When touching thin `web/service/use-*` wrappers, migrate them away when feasible.
3. Preserve Dify conventions.
- Keep contract inputs in `{ params, query?, body? }` shape.
- Bind invalidation in the service-layer mutation definition.
- Bind default cache updates/invalidation in `createTanstackQueryUtils(...experimental_defaults...)`; use feature hooks only for workflows that cannot be expressed as default operation behavior.
- Prefer `mutate(...)`; use `mutateAsync(...)` only when Promise semantics are required.
## Files Commonly Touched
@@ -33,7 +35,7 @@ description: Guide for implementing Dify frontend query and mutation patterns wi
- `web/contract/marketplace.ts`
- `web/contract/router.ts`
- `web/service/client.ts`
- `web/service/use-*.ts`
- legacy `web/service/use-*.ts` files when migrating wrappers away
- component and hook call sites using `consoleQuery` or `marketplaceQuery`
## References
@@ -1,4 +1,4 @@
interface:
display_name: "Frontend Query & Mutation"
short_description: "Dify TanStack Query and oRPC patterns"
default_prompt: "Use this skill when implementing or reviewing Dify frontend contracts, query and mutation call sites, conditional queries, invalidation, or legacy query/mutation migrations."
short_description: "Dify TanStack Query, oRPC, and default option patterns"
default_prompt: "Use this skill when implementing or reviewing Dify frontend contracts, query and mutation call sites, oRPC default options, conditional queries, cache updates/invalidation, or legacy query/mutation migrations."
@@ -7,6 +7,7 @@
- Core workflow
- Query usage decision rule
- Mutation usage decision rule
- Thin hook decision rule
- Anti-patterns
- Contract rules
- Type export
@@ -55,9 +56,13 @@ const invoiceQuery = useQuery(consoleQuery.billing.invoices.queryOptions({
1. Default to direct `*.queryOptions(...)` usage at the call site.
2. If 3 or more call sites share the same extra options, extract a small query helper, not a `use-*` passthrough hook.
3. Create `web/service/use-{domain}.ts` only for orchestration.
3. Create or keep feature hooks only for orchestration.
- Combine multiple queries or mutations.
- Share domain-level derived state or invalidation helpers.
- Prefer `web/features/{domain}/hooks/*` for feature-owned workflows.
4. Treat `web/service/use-{domain}.ts` as legacy.
- Do not create new thin service wrappers for oRPC contracts.
- When touching existing wrappers, inline direct `consoleQuery` or `marketplaceQuery` consumption when the wrapper is only a passthrough.
```typescript
const invoicesBaseQueryOptions = () =>
@@ -74,11 +79,37 @@ const invoiceQuery = useQuery({
1. Default to mutation helpers from `consoleQuery` or `marketplaceQuery`, for example `useMutation(consoleQuery.billing.bindPartnerStack.mutationOptions(...))`.
2. If the mutation flow is heavily custom, use oRPC clients as `mutationFn`, for example `consoleClient.xxx` or `marketplaceClient.xxx`, instead of handwritten non-oRPC mutation logic.
```typescript
const createTagMutation = useMutation(consoleQuery.tags.create.mutationOptions())
```
## Thin Hook Decision Rule
Remove thin hooks when they only rename a single oRPC query or mutation helper.
Keep hooks when they orchestrate business behavior across multiple operations, own local workflow state, or normalize a feature-specific API.
Prefer feature vertical hooks for kept orchestration. Do not move new contract-first wrappers into `web/service/use-*`.
Use:
```typescript
const deleteTagMutation = useMutation(consoleQuery.tags.delete.mutationOptions())
```
Keep:
```typescript
const applyTagBindingsMutation = useApplyTagBindingsMutation()
```
`useApplyTagBindingsMutation` is acceptable because it coordinates bind and unbind requests, computes deltas, and exposes a feature-level workflow rather than a single endpoint passthrough.
## Anti-Patterns
- Do not wrap `useQuery` with `options?: Partial<UseQueryOptions>`.
- Do not split local `queryKey` and `queryFn` when oRPC `queryOptions` already exists and fits the use case.
- Do not create thin `use-*` passthrough hooks for a single endpoint.
- Do not create business-layer helpers whose only purpose is to call `consoleQuery.xxx.mutationOptions()` or `queryOptions()`.
- Do not introduce new `web/service/use-*` files for oRPC contract passthroughs.
- These patterns can degrade inference, especially around `throwOnError` and `select`, and add unnecessary indirection.
## Contract Rules
@@ -3,6 +3,7 @@
## Table of Contents
- Conditional queries
- oRPC default options
- Cache invalidation
- Key API guide
- `mutate` vs `mutateAsync`
@@ -35,9 +36,50 @@ function useBadAccessMode(appId: string | undefined) {
}
```
## oRPC Default Options
Use `experimental_defaults` in `createTanstackQueryUtils` when a contract operation should always carry shared TanStack Query behavior, such as default stale time, mutation cache writes, or invalidation.
Place defaults at the query utility creation point in `web/service/client.ts`:
```typescript
export const consoleQuery = createTanstackQueryUtils(consoleClient, {
path: ['console'],
experimental_defaults: {
tags: {
create: {
mutationOptions: {
onSuccess: (tag, _variables, _result, context) => {
context.client.setQueryData(
consoleQuery.tags.list.queryKey({
input: {
query: {
type: tag.type,
},
},
}),
(oldTags: Tag[] | undefined) => oldTags ? [tag, ...oldTags] : oldTags,
)
},
},
},
},
},
})
```
Rules:
- Keep defaults inline in the `consoleQuery` or `marketplaceQuery` initialization when they need sibling oRPC key builders.
- Do not create a wrapper function solely to host `createTanstackQueryUtils`.
- Do not split defaults into a vertical feature file if that forces handwritten operation paths such as `generateOperationKey(['console', ...])`.
- Keep feature-level orchestration in the feature vertical; keep query utility lifecycle defaults with the query utility.
- Prefer call-site callbacks for UI feedback only; shared cache behavior belongs in oRPC defaults when it is tied to a contract operation.
## Cache Invalidation
Bind invalidation in the service-layer mutation definition.
Bind shared invalidation in oRPC defaults when it is tied to a contract operation.
Use feature vertical hooks only for multi-operation workflows or domain orchestration that cannot live in a single operation default.
Components may add UI feedback in call-site callbacks, but they should not decide which queries to invalidate.
Use:
@@ -49,7 +91,7 @@ Use:
Do not use deprecated `useInvalid` from `use-base.ts`.
```typescript
// Service layer owns cache invalidation.
// Feature orchestration owns cache invalidation only when defaults are not enough.
export const useUpdateAccessMode = () => {
const queryClient = useQueryClient()
@@ -124,7 +166,7 @@ When touching old code, migrate it toward these rules:
| Old pattern | New pattern |
|---|---|
| `useInvalid(key)` in service layer | `queryClient.invalidateQueries(...)` inside mutation `onSuccess` |
| component-triggered invalidation after mutation | move invalidation into the service-layer mutation definition |
| `useInvalid(key)` in service wrappers | oRPC defaults, or a feature vertical hook for real orchestration |
| component-triggered invalidation after mutation | move invalidation into oRPC defaults or a feature vertical hook |
| imperative fetch plus manual invalidation | wrap it in `useMutation(...mutationOptions(...))` |
| `await mutateAsync()` without `try/catch` | switch to `mutate(...)` or add `try/catch` |
+1 -1
View File
@@ -113,7 +113,7 @@ jobs:
find . -name "*.py.bak" -type f -delete
- name: Setup web environment
if: github.event_name != 'merge_group' && steps.web-changes.outputs.any_changed == 'true'
if: github.event_name != 'merge_group'
uses: ./.github/actions/setup-web
- name: ESLint autofix
+2
View File
@@ -109,6 +109,8 @@ jobs:
- name: Web tsslint
if: steps.changed-files.outputs.any_changed == 'true'
working-directory: ./web
env:
NODE_OPTIONS: --max-old-space-size=4096
run: vp run lint:tss
- name: Web type check
+1 -1
View File
@@ -158,7 +158,7 @@ jobs:
- name: Run Claude Code for Translation Sync
if: steps.context.outputs.CHANGED_FILES != ''
uses: anthropics/claude-code-action@ef50f123a3a9be95b60040d042717517407c7256 # v1.0.110
uses: anthropics/claude-code-action@fefa07e9c665b7320f08c3b525980457f22f58aa # v1.0.111
with:
anthropic_api_key: ${{ secrets.ANTHROPIC_API_KEY }}
github_token: ${{ secrets.GITHUB_TOKEN }}
+3
View File
@@ -219,6 +219,9 @@ node_modules
# plugin migrate
plugins.jsonl
# generated API OpenAPI specs
packages/contracts/openapi/
# mise
mise.toml
+1 -2
View File
@@ -74,8 +74,7 @@ Dify is an open-source LLM app development platform. Its intuitive interface com
The easiest way to start the Dify server is through [Docker Compose](docker/docker-compose.yaml). Before running Dify with the following commands, make sure that [Docker](https://docs.docker.com/get-docker/) and [Docker Compose](https://docs.docker.com/compose/install/) are installed on your machine:
```bash
cd dify
cd docker
cd dify/docker
cp .env.example .env
docker compose up -d
```
+22 -22
View File
@@ -19,7 +19,7 @@
"name": "Website Generator"
},
"app_id": "b53545b1-79ea-4da3-b31a-c39391c6f041",
"category": "Programming",
"categories": ["Programming"],
"copyright": null,
"description": null,
"is_listed": true,
@@ -35,7 +35,7 @@
"name": "Investment Analysis Report Copilot"
},
"app_id": "a23b57fa-85da-49c0-a571-3aff375976c1",
"category": "Agent",
"categories": ["Agent"],
"copyright": "Dify.AI",
"description": "Welcome to your personalized Investment Analysis Copilot service, where we delve into the depths of stock analysis to provide you with comprehensive insights. \n",
"is_listed": true,
@@ -51,7 +51,7 @@
"name": "Workflow Planning Assistant "
},
"app_id": "f3303a7d-a81c-404e-b401-1f8711c998c1",
"category": "Workflow",
"categories": ["Workflow"],
"copyright": null,
"description": "An assistant that helps you plan and select the right node for a workflow (V0.6.0). ",
"is_listed": true,
@@ -67,7 +67,7 @@
"name": "Automated Email Reply "
},
"app_id": "e9d92058-7d20-4904-892f-75d90bef7587",
"category": "Workflow",
"categories": ["Workflow"],
"copyright": null,
"description": "Reply emails using Gmail API. It will automatically retrieve email in your inbox and create a response in Gmail. \nConfigure your Gmail API in Google Cloud Console. ",
"is_listed": true,
@@ -83,7 +83,7 @@
"name": "Book Translation "
},
"app_id": "98b87f88-bd22-4d86-8b74-86beba5e0ed4",
"category": "Workflow",
"categories": ["Workflow"],
"copyright": null,
"description": "A workflow designed to translate a full book up to 15000 tokens per run. Uses Code node to separate text into chunks and Iteration to translate each chunk. ",
"is_listed": true,
@@ -99,7 +99,7 @@
"name": "Python bug fixer"
},
"app_id": "cae337e6-aec5-4c7b-beca-d6f1a808bd5e",
"category": "Programming",
"categories": ["Programming"],
"copyright": null,
"description": null,
"is_listed": true,
@@ -115,7 +115,7 @@
"name": "Code Interpreter"
},
"app_id": "d077d587-b072-4f2c-b631-69ed1e7cdc0f",
"category": "Programming",
"categories": ["Programming"],
"copyright": "Copyright 2023 Dify",
"description": "Code interpreter, clarifying the syntax and semantics of the code.",
"is_listed": true,
@@ -131,7 +131,7 @@
"name": "SVG Logo Design "
},
"app_id": "73fbb5f1-c15d-4d74-9cc8-46d9db9b2cca",
"category": "Agent",
"categories": ["Agent"],
"copyright": "Dify.AI",
"description": "Hello, I am your creative partner in bringing ideas to vivid life! I can assist you in creating stunning designs by leveraging abilities of DALL·E 3. ",
"is_listed": true,
@@ -147,7 +147,7 @@
"name": "Long Story Generator (Iteration) "
},
"app_id": "5efb98d7-176b-419c-b6ef-50767391ab62",
"category": "Workflow",
"categories": ["Workflow"],
"copyright": null,
"description": "A workflow demonstrating how to use Iteration node to generate long article that is longer than the context length of LLMs. ",
"is_listed": true,
@@ -163,7 +163,7 @@
"name": "Text Summarization Workflow"
},
"app_id": "f00c4531-6551-45ee-808f-1d7903099515",
"category": "Workflow",
"categories": ["Workflow"],
"copyright": null,
"description": "Based on users' choice, retrieve external knowledge to more accurately summarize articles.",
"is_listed": true,
@@ -179,7 +179,7 @@
"name": "YouTube Channel Data Analysis"
},
"app_id": "be591209-2ca8-410f-8f3b-ca0e530dd638",
"category": "Agent",
"categories": ["Agent"],
"copyright": "Dify.AI",
"description": "I am a YouTube Channel Data Analysis Copilot, I am here to provide expert data analysis tailored to your needs. ",
"is_listed": true,
@@ -195,7 +195,7 @@
"name": "Article Grading Bot"
},
"app_id": "a747f7b4-c48b-40d6-b313-5e628232c05f",
"category": "Writing",
"categories": ["Writing"],
"copyright": null,
"description": "Assess the quality of articles and text based on user defined criteria. ",
"is_listed": true,
@@ -211,7 +211,7 @@
"name": "SEO Blog Generator"
},
"app_id": "18f3bd03-524d-4d7a-8374-b30dbe7c69d5",
"category": "Workflow",
"categories": ["Workflow"],
"copyright": null,
"description": "Workflow for retrieving information from the internet, followed by segmented generation of SEO blogs.",
"is_listed": true,
@@ -227,7 +227,7 @@
"name": "SQL Creator"
},
"app_id": "050ef42e-3e0c-40c1-a6b6-a64f2c49d744",
"category": "Programming",
"categories": ["Programming"],
"copyright": "Copyright 2023 Dify",
"description": "Write SQL from natural language by pasting in your schema with the request.Please describe your query requirements in natural language and select the target database type.",
"is_listed": true,
@@ -243,7 +243,7 @@
"name": "Sentiment Analysis "
},
"app_id": "f06bf86b-d50c-4895-a942-35112dbe4189",
"category": "Workflow",
"categories": ["Workflow"],
"copyright": null,
"description": "Batch sentiment analysis of text, followed by JSON output of sentiment classification along with scores.",
"is_listed": true,
@@ -259,7 +259,7 @@
"name": "Strategic Consulting Expert"
},
"app_id": "7e8ca1ae-02f2-4b5f-979e-62d19133bee2",
"category": "Assistant",
"categories": ["Assistant"],
"copyright": "Copyright 2023 Dify",
"description": "I can answer your questions related to strategic marketing.",
"is_listed": true,
@@ -275,7 +275,7 @@
"name": "Code Converter"
},
"app_id": "4006c4b2-0735-4f37-8dbb-fb1a8c5bd87a",
"category": "Programming",
"categories": ["Programming"],
"copyright": "Copyright 2023 Dify",
"description": "This is an application that provides the ability to convert code snippets in multiple programming languages. You can input the code you wish to convert, select the target programming language, and get the desired output.",
"is_listed": true,
@@ -291,7 +291,7 @@
"name": "Question Classifier + Knowledge + Chatbot "
},
"app_id": "d9f6b733-e35d-4a40-9f38-ca7bbfa009f7",
"category": "Workflow",
"categories": ["Workflow"],
"copyright": null,
"description": "Basic Workflow Template, a chatbot capable of identifying intents alongside with a knowledge base.",
"is_listed": true,
@@ -307,7 +307,7 @@
"name": "AI Front-end interviewer"
},
"app_id": "127efead-8944-4e20-ba9d-12402eb345e0",
"category": "HR",
"categories": ["HR"],
"copyright": "Copyright 2023 Dify",
"description": "A simulated front-end interviewer that tests the skill level of front-end development through questioning.",
"is_listed": true,
@@ -323,7 +323,7 @@
"name": "Knowledge Retrieval + Chatbot "
},
"app_id": "e9870913-dd01-4710-9f06-15d4180ca1ce",
"category": "Workflow",
"categories": ["Workflow"],
"copyright": null,
"description": "Basic Workflow Template, A chatbot with a knowledge base. ",
"is_listed": true,
@@ -339,7 +339,7 @@
"name": "Email Assistant Workflow "
},
"app_id": "dd5b6353-ae9b-4bce-be6a-a681a12cf709",
"category": "Workflow",
"categories": ["Workflow"],
"copyright": null,
"description": "A multifunctional email assistant capable of summarizing, replying, composing, proofreading, and checking grammar.",
"is_listed": true,
@@ -355,7 +355,7 @@
"name": "Customer Review Analysis Workflow "
},
"app_id": "9c0cd31f-4b62-4005-adf5-e3888d08654a",
"category": "Workflow",
"categories": ["Workflow"],
"copyright": null,
"description": "Utilize LLM (Large Language Models) to classify customer reviews and forward them to the internal system.",
"is_listed": true,
+2 -1
View File
@@ -41,7 +41,8 @@ def guess_file_info_from_response(response: httpx.Response):
# Try to extract filename from URL
parsed_url = urllib.parse.urlparse(url)
url_path = parsed_url.path
filename = os.path.basename(url_path)
# Decode percent-encoded characters in the path segment
filename = urllib.parse.unquote(os.path.basename(url_path))
# If filename couldn't be extracted, use Content-Disposition header
if not filename:
+29 -9
View File
@@ -1,4 +1,5 @@
import logging
import re
import uuid
from datetime import datetime
from typing import Any, Literal
@@ -8,6 +9,7 @@ from flask_restx import Resource
from pydantic import AliasChoices, BaseModel, Field, computed_field, field_validator
from sqlalchemy import select
from sqlalchemy.orm import Session
from werkzeug.datastructures import MultiDict
from werkzeug.exceptions import BadRequest
from controllers.common.helpers import FileInfo
@@ -57,6 +59,7 @@ ALLOW_CREATE_APP_MODES = ["chat", "agent-chat", "advanced-chat", "workflow", "co
register_enum_models(console_ns, IconType)
_logger = logging.getLogger(__name__)
_TAG_IDS_BRACKET_PATTERN = re.compile(r"^tag_ids\[(\d+)\]$")
class AppListQuery(BaseModel):
@@ -66,22 +69,19 @@ class AppListQuery(BaseModel):
default="all", description="App mode filter"
)
name: str | None = Field(default=None, description="Filter by app name")
tag_ids: list[str] | None = Field(default=None, description="Comma-separated tag IDs")
tag_ids: list[str] | None = Field(default=None, description="Filter by tag IDs")
is_created_by_me: bool | None = Field(default=None, description="Filter by creator")
@field_validator("tag_ids", mode="before")
@classmethod
def validate_tag_ids(cls, value: str | list[str] | None) -> list[str] | None:
def validate_tag_ids(cls, value: list[str] | None) -> list[str] | None:
if not value:
return None
if isinstance(value, str):
items = [item.strip() for item in value.split(",") if item.strip()]
elif isinstance(value, list):
items = [str(item).strip() for item in value if item and str(item).strip()]
else:
raise TypeError("Unsupported tag_ids type.")
if not isinstance(value, list):
raise ValueError("Unsupported tag_ids type.")
items = [str(item).strip() for item in value if item and str(item).strip()]
if not items:
return None
@@ -91,6 +91,26 @@ class AppListQuery(BaseModel):
raise ValueError("Invalid UUID format in tag_ids.") from exc
def _normalize_app_list_query_args(query_args: MultiDict[str, str]) -> dict[str, str | list[str]]:
normalized: dict[str, str | list[str]] = {}
indexed_tag_ids: list[tuple[int, str]] = []
for key in query_args:
match = _TAG_IDS_BRACKET_PATTERN.fullmatch(key)
if match:
indexed_tag_ids.extend((int(match.group(1)), value) for value in query_args.getlist(key))
continue
value = query_args.get(key)
if value is not None:
normalized[key] = value
if indexed_tag_ids:
normalized["tag_ids"] = [value for _, value in sorted(indexed_tag_ids)]
return normalized
class CreateAppPayload(BaseModel):
name: str = Field(..., min_length=1, description="App name")
description: str | None = Field(default=None, description="App description (max 400 chars)", max_length=400)
@@ -455,7 +475,7 @@ class AppListApi(Resource):
"""Get app list"""
current_user, current_tenant_id = current_account_with_tenant()
args = AppListQuery.model_validate(request.args.to_dict(flat=True)) # type: ignore
args = AppListQuery.model_validate(_normalize_app_list_query_args(request.args))
args_dict = args.model_dump()
# get app list
+32 -15
View File
@@ -60,7 +60,8 @@ _file_access_controller = DatabaseFileAccessController()
LISTENING_RETRY_IN = 2000
DEFAULT_REF_TEMPLATE_SWAGGER_2_0 = "#/definitions/{model}"
RESTORE_SOURCE_WORKFLOW_MUST_BE_PUBLISHED_MESSAGE = "source workflow must be published"
MAX_WORKFLOW_ONLINE_USERS_QUERY_IDS = 50
MAX_WORKFLOW_ONLINE_USERS_REQUEST_IDS = 1000
WORKFLOW_ONLINE_USERS_REDIS_BATCH_SIZE = 50
# Register models for flask_restx to avoid dict type issues in Swagger
# Register in dependency order: base models first, then dependent models
@@ -158,8 +159,13 @@ class WorkflowFeaturesPayload(BaseModel):
features: dict[str, Any] = Field(..., description="Workflow feature configuration")
class WorkflowOnlineUsersQuery(BaseModel):
app_ids: str = Field(..., description="Comma-separated app IDs")
class WorkflowOnlineUsersPayload(BaseModel):
app_ids: list[str] = Field(default_factory=list, description="App IDs")
@field_validator("app_ids")
@classmethod
def normalize_app_ids(cls, app_ids: list[str]) -> list[str]:
return list(dict.fromkeys(app_id.strip() for app_id in app_ids if app_id.strip()))
class DraftWorkflowTriggerRunPayload(BaseModel):
@@ -186,7 +192,7 @@ reg(ConvertToWorkflowPayload)
reg(WorkflowListQuery)
reg(WorkflowUpdatePayload)
reg(WorkflowFeaturesPayload)
reg(WorkflowOnlineUsersQuery)
reg(WorkflowOnlineUsersPayload)
reg(DraftWorkflowTriggerRunPayload)
reg(DraftWorkflowTriggerRunAllPayload)
@@ -1384,19 +1390,19 @@ class DraftWorkflowTriggerRunAllApi(Resource):
@console_ns.route("/apps/workflows/online-users")
class WorkflowOnlineUsersApi(Resource):
@console_ns.expect(console_ns.models[WorkflowOnlineUsersQuery.__name__])
@console_ns.expect(console_ns.models[WorkflowOnlineUsersPayload.__name__])
@console_ns.doc("get_workflow_online_users")
@console_ns.doc(description="Get workflow online users")
@setup_required
@login_required
@account_initialization_required
@marshal_with(online_user_list_fields)
def get(self):
args = WorkflowOnlineUsersQuery.model_validate(request.args.to_dict(flat=True)) # type: ignore
def post(self):
args = WorkflowOnlineUsersPayload.model_validate(console_ns.payload or {})
app_ids = list(dict.fromkeys(app_id.strip() for app_id in args.app_ids.split(",") if app_id.strip()))
if len(app_ids) > MAX_WORKFLOW_ONLINE_USERS_QUERY_IDS:
raise BadRequest(f"Maximum {MAX_WORKFLOW_ONLINE_USERS_QUERY_IDS} app_ids are allowed per request.")
app_ids = args.app_ids
if len(app_ids) > MAX_WORKFLOW_ONLINE_USERS_REQUEST_IDS:
raise BadRequest(f"Maximum {MAX_WORKFLOW_ONLINE_USERS_REQUEST_IDS} app_ids are allowed per request.")
if not app_ids:
return {"data": []}
@@ -1404,13 +1410,24 @@ class WorkflowOnlineUsersApi(Resource):
_, current_tenant_id = current_account_with_tenant()
workflow_service = WorkflowService()
accessible_app_ids = workflow_service.get_accessible_app_ids(app_ids, current_tenant_id)
ordered_accessible_app_ids = [app_id for app_id in app_ids if app_id in accessible_app_ids]
users_json_by_app_id: dict[str, Any] = {}
for start_index in range(0, len(ordered_accessible_app_ids), WORKFLOW_ONLINE_USERS_REDIS_BATCH_SIZE):
app_id_batch = ordered_accessible_app_ids[
start_index : start_index + WORKFLOW_ONLINE_USERS_REDIS_BATCH_SIZE
]
pipe = redis_client.pipeline(transaction=False)
for app_id in app_id_batch:
pipe.hgetall(f"{WORKFLOW_ONLINE_USERS_PREFIX}{app_id}")
users_json_batch = pipe.execute()
for app_id, users_json in zip(app_id_batch, users_json_batch):
users_json_by_app_id[app_id] = users_json
results = []
for app_id in app_ids:
if app_id not in accessible_app_ids:
continue
users_json = redis_client.hgetall(f"{WORKFLOW_ONLINE_USERS_PREFIX}{app_id}")
for app_id in ordered_accessible_app_ids:
users_json = users_json_by_app_id.get(app_id, {})
users = []
for _, user_info_json in users_json.items():
@@ -75,14 +75,15 @@ console_ns.schema_model(
def _convert_values_to_json_serializable_object(value: Segment):
if isinstance(value, FileSegment):
return value.value.model_dump()
elif isinstance(value, ArrayFileSegment):
return [i.model_dump() for i in value.value]
elif isinstance(value, SegmentGroup):
return [_convert_values_to_json_serializable_object(i) for i in value.value]
else:
return value.value
match value:
case FileSegment():
return value.value.model_dump()
case ArrayFileSegment():
return [i.model_dump() for i in value.value]
case SegmentGroup():
return [_convert_values_to_json_serializable_object(i) for i in value.value]
case _:
return value.value
def _serialize_var_value(variable: WorkflowDraftVariable):
@@ -52,7 +52,7 @@ class RecommendedAppResponse(ResponseModel):
copyright: str | None = None
privacy_policy: str | None = None
custom_disclaimer: str | None = None
category: str | None = None
categories: list[str] = Field(default_factory=list)
position: int | None = None
is_listed: bool | None = None
can_trial: bool | None = None
+8 -53
View File
@@ -32,12 +32,7 @@ class TagBindingPayload(BaseModel):
class TagBindingRemovePayload(BaseModel):
tag_id: str = Field(description="Tag ID to remove")
target_id: str = Field(description="Target ID to unbind tag from")
type: TagType = Field(description="Tag type")
class TagBindingItemDeletePayload(BaseModel):
tag_ids: list[str] = Field(description="Tag IDs to remove", min_length=1)
target_id: str = Field(description="Target ID to unbind tag from")
type: TagType = Field(description="Tag type")
@@ -75,7 +70,6 @@ register_schema_models(
TagBasePayload,
TagBindingPayload,
TagBindingRemovePayload,
TagBindingItemDeletePayload,
TagListQueryParam,
TagResponse,
)
@@ -184,13 +178,13 @@ def _create_tag_bindings() -> tuple[dict[str, str], int]:
return {"result": "success"}, 200
def _remove_tag_binding() -> tuple[dict[str, str], int]:
def _remove_tag_bindings() -> tuple[dict[str, str], int]:
_require_tag_binding_edit_permission()
payload = TagBindingRemovePayload.model_validate(console_ns.payload or {})
TagService.delete_tag_binding(
TagBindingDeletePayload(
tag_id=payload.tag_id,
tag_ids=payload.tag_ids,
target_id=payload.target_id,
type=payload.type,
)
@@ -211,54 +205,15 @@ class TagBindingCollectionApi(Resource):
return _create_tag_bindings()
@console_ns.route("/tag-bindings/<uuid:id>")
class TagBindingItemApi(Resource):
"""Canonical item resource for tag binding deletion."""
@console_ns.doc("delete_tag_binding")
@console_ns.doc(params={"id": "Tag ID"})
@console_ns.expect(console_ns.models[TagBindingItemDeletePayload.__name__])
@setup_required
@login_required
@account_initialization_required
def delete(self, id):
_require_tag_binding_edit_permission()
payload = TagBindingItemDeletePayload.model_validate(console_ns.payload or {})
TagService.delete_tag_binding(
TagBindingDeletePayload(
tag_id=str(id),
target_id=payload.target_id,
type=payload.type,
)
)
return {"result": "success"}, 200
@console_ns.route("/tag-bindings/create")
class DeprecatedTagBindingCreateApi(Resource):
"""Deprecated verb-based alias for tag binding creation."""
@console_ns.doc("create_tag_binding_deprecated")
@console_ns.doc(deprecated=True)
@console_ns.doc(description="Deprecated legacy alias. Use POST /tag-bindings instead.")
@console_ns.expect(console_ns.models[TagBindingPayload.__name__])
@setup_required
@login_required
@account_initialization_required
def post(self):
return _create_tag_bindings()
@console_ns.route("/tag-bindings/remove")
class DeprecatedTagBindingRemoveApi(Resource):
"""Deprecated verb-based alias for tag binding deletion."""
class TagBindingRemoveApi(Resource):
"""Batch resource for tag binding deletion."""
@console_ns.doc("delete_tag_binding_deprecated")
@console_ns.doc(deprecated=True)
@console_ns.doc(description="Deprecated legacy alias. Use DELETE /tag-bindings/{id} instead.")
@console_ns.doc("remove_tag_bindings")
@console_ns.doc(description="Remove one or more tag bindings from a target.")
@console_ns.expect(console_ns.models[TagBindingRemovePayload.__name__])
@setup_required
@login_required
@account_initialization_required
def post(self):
return _remove_tag_binding()
return _remove_tag_bindings()
+19 -1
View File
@@ -8,6 +8,7 @@ from flask import request
from flask_restx import Resource
from pydantic import BaseModel, Field, field_validator, model_validator
from sqlalchemy import select
from werkzeug.exceptions import NotFound
from configs import dify_config
from constants.languages import supported_language
@@ -45,6 +46,8 @@ from libs.helper import EmailStr, extract_remote_ip, timezone
from libs.login import current_account_with_tenant, login_required
from models import AccountIntegrate, InvitationCode
from models.account import AccountStatus, InvitationCodeStatus
from models.enums import CreatorUserRole
from models.model import UploadFile
from services.account_service import AccountService
from services.billing_service import BillingService
from services.errors.account import CurrentPasswordIncorrectError as ServiceCurrentPasswordIncorrectError
@@ -322,9 +325,24 @@ class AccountAvatarApi(Resource):
@login_required
@account_initialization_required
def get(self):
current_user, current_tenant_id = current_account_with_tenant()
args = AccountAvatarQuery.model_validate(request.args.to_dict(flat=True)) # type: ignore
avatar = args.avatar
avatar_url = file_helpers.get_signed_file_url(args.avatar)
if avatar.startswith(("http://", "https://")):
return {"avatar_url": avatar}
upload_file = db.session.scalar(select(UploadFile).where(UploadFile.id == avatar).limit(1))
if upload_file is None:
raise NotFound("Avatar file not found")
if upload_file.tenant_id != current_tenant_id:
raise NotFound("Avatar file not found")
if upload_file.created_by_role != CreatorUserRole.ACCOUNT or upload_file.created_by != current_user.id:
raise NotFound("Avatar file not found")
avatar_url = file_helpers.get_signed_file_url(upload_file_id=upload_file.id)
return {"avatar_url": avatar_url}
@console_ns.expect(console_ns.models[AccountAvatarPayload.__name__])
@@ -876,10 +876,10 @@ class ToolBuiltinProviderSetDefaultApi(Resource):
@login_required
@account_initialization_required
def post(self, provider):
current_user, current_tenant_id = current_account_with_tenant()
_, current_tenant_id = current_account_with_tenant()
payload = BuiltinProviderDefaultCredentialPayload.model_validate(console_ns.payload or {})
return BuiltinToolManageService.set_default_provider(
tenant_id=current_tenant_id, user_id=current_user.id, provider=provider, id=payload.id
tenant_id=current_tenant_id, provider=provider, id=payload.id
)
+24 -6
View File
@@ -2,7 +2,7 @@ from typing import Any, Literal, cast
from flask import request
from flask_restx import marshal
from pydantic import BaseModel, Field, TypeAdapter, field_validator
from pydantic import BaseModel, Field, TypeAdapter, field_validator, model_validator
from werkzeug.exceptions import Forbidden, NotFound
import services
@@ -100,9 +100,27 @@ class TagBindingPayload(BaseModel):
class TagUnbindingPayload(BaseModel):
tag_id: str
"""Accept the legacy single-tag Service API payload while exposing a normalized tag_ids list internally."""
tag_ids: list[str] = Field(default_factory=list)
tag_id: str | None = None
target_id: str
@model_validator(mode="before")
@classmethod
def normalize_legacy_tag_id(cls, data: object) -> object:
if not isinstance(data, dict):
return data
if not data.get("tag_ids") and data.get("tag_id"):
return {**data, "tag_ids": [data["tag_id"]]}
return data
@model_validator(mode="after")
def validate_tag_ids(self) -> "TagUnbindingPayload":
if not self.tag_ids:
raise ValueError("Tag IDs is required.")
return self
class DatasetListQuery(BaseModel):
page: int = Field(default=1, description="Page number")
@@ -601,11 +619,11 @@ class DatasetTagBindingApi(DatasetApiResource):
@service_api_ns.route("/datasets/tags/unbinding")
class DatasetTagUnbindingApi(DatasetApiResource):
@service_api_ns.expect(service_api_ns.models[TagUnbindingPayload.__name__])
@service_api_ns.doc("unbind_dataset_tag")
@service_api_ns.doc(description="Unbind a tag from a dataset")
@service_api_ns.doc("unbind_dataset_tags")
@service_api_ns.doc(description="Unbind tags from a dataset")
@service_api_ns.doc(
responses={
204: "Tag unbound successfully",
204: "Tags unbound successfully",
401: "Unauthorized - invalid API token",
403: "Forbidden - insufficient permissions",
}
@@ -618,7 +636,7 @@ class DatasetTagUnbindingApi(DatasetApiResource):
payload = TagUnbindingPayload.model_validate(service_api_ns.payload or {})
TagService.delete_tag_binding(
TagBindingDeletePayload(tag_id=payload.tag_id, target_id=payload.target_id, type=TagType.KNOWLEDGE)
TagBindingDeletePayload(tag_ids=payload.tag_ids, target_id=payload.target_id, type=TagType.KNOWLEDGE)
)
return "", 204
+4 -4
View File
@@ -23,7 +23,7 @@ from controllers.web.wraps import WebApiResource
from core.errors.error import ModelCurrentlyNotSupportError, ProviderTokenNotInitError, QuotaExceededError
from graphon.model_runtime.errors.invoke import InvokeError
from libs.helper import uuid_value
from models.model import App
from models.model import App, EndUser
from services.audio_service import AudioService
from services.errors.audio import (
AudioTooLargeServiceError,
@@ -69,12 +69,12 @@ class AudioApi(WebApiResource):
500: "Internal Server Error",
}
)
def post(self, app_model: App, end_user):
def post(self, app_model: App, end_user: EndUser):
"""Convert audio to text"""
file = request.files["file"]
try:
response = AudioService.transcript_asr(app_model=app_model, file=file, end_user=end_user)
response = AudioService.transcript_asr(app_model=app_model, file=file, end_user=end_user.external_user_id)
return response
except services.errors.app_model_config.AppModelConfigBrokenError:
@@ -117,7 +117,7 @@ class TextApi(WebApiResource):
500: "Internal Server Error",
}
)
def post(self, app_model: App, end_user):
def post(self, app_model: App, end_user: EndUser):
"""Convert text to audio"""
try:
payload = TextToAudioPayload.model_validate(web_ns.payload or {})
@@ -175,7 +175,7 @@ class AdvancedChatAppRunner(WorkflowBasedAppRunner):
# Create a variable pool.
# init variable pool
variable_pool = VariablePool.from_bootstrap()
variable_pool = VariablePool()
add_variables_to_pool(
variable_pool,
build_bootstrap_variables(
@@ -144,7 +144,7 @@ class PipelineRunner(WorkflowBasedAppRunner):
)
)
variable_pool = VariablePool.from_bootstrap()
variable_pool = VariablePool()
add_variables_to_pool(
variable_pool,
build_bootstrap_variables(
+1 -1
View File
@@ -106,7 +106,7 @@ class WorkflowAppRunner(WorkflowBasedAppRunner):
workflow_id=app_config.workflow_id,
workflow_execution_id=self.application_generate_entity.workflow_execution_id,
)
variable_pool = VariablePool.from_bootstrap()
variable_pool = VariablePool()
add_variables_to_pool(
variable_pool,
build_bootstrap_variables(
+1 -1
View File
@@ -188,7 +188,7 @@ class WorkflowBasedAppRunner:
ValueError: If neither single_iteration_run nor single_loop_run is specified
"""
# Create initial runtime state with variable pool containing environment variables
variable_pool = VariablePool.from_bootstrap()
variable_pool = VariablePool()
add_variables_to_pool(
variable_pool,
build_bootstrap_variables(
+1 -3
View File
@@ -16,7 +16,6 @@ from graphon.graph_engine.entities.commands import AbortCommand, CommandType
from graphon.graph_engine.layers import GraphEngineLayer
from graphon.graph_events import GraphEngineEvent, GraphNodeEventBase, NodeRunSucceededEvent
from graphon.nodes.base.node import Node
from graphon.nodes.llm.runtime_protocols import PreparedLLMProtocol
if TYPE_CHECKING:
from graphon.nodes.llm.node import LLMNode
@@ -117,8 +116,7 @@ class LLMQuotaLayer(GraphEngineLayer):
case BuiltinNodeTypes.PARAMETER_EXTRACTOR:
model_instance = cast("ParameterExtractorNode", node).model_instance
case BuiltinNodeTypes.QUESTION_CLASSIFIER:
typed_node: QuestionClassifierNode = cast("QuestionClassifierNode", node)
model_instance = cast(PreparedLLMProtocol, typed_node._model_instance)
model_instance = cast("QuestionClassifierNode", node).model_instance
case _:
return None
except AttributeError:
+5 -7
View File
@@ -24,7 +24,6 @@ from core.entities.provider_entities import (
from core.helper import encrypter
from core.helper.model_provider_cache import ProviderCredentialsCache, ProviderCredentialsCacheType
from core.plugin.impl.model_runtime_factory import create_plugin_model_provider_factory
from graphon.model_runtime import ModelRuntime
from graphon.model_runtime.entities.model_entities import AIModelEntity, FetchFrom, ModelType
from graphon.model_runtime.entities.provider_entities import (
ConfigurateMethod,
@@ -34,6 +33,7 @@ from graphon.model_runtime.entities.provider_entities import (
)
from graphon.model_runtime.model_providers.base.ai_model import AIModel
from graphon.model_runtime.model_providers.model_provider_factory import ModelProviderFactory
from graphon.model_runtime.runtime import ModelRuntime
from libs.datetime_utils import naive_utc_now
from models.engine import db
from models.enums import CredentialSourceType
@@ -109,7 +109,7 @@ class ProviderConfiguration(BaseModel):
def get_model_provider_factory(self) -> ModelProviderFactory:
"""Return a provider factory that preserves any request-bound runtime."""
if self._bound_model_runtime is not None:
return ModelProviderFactory(runtime=self._bound_model_runtime)
return ModelProviderFactory(model_runtime=self._bound_model_runtime)
return create_plugin_model_provider_factory(tenant_id=self.tenant_id)
def get_current_credentials(self, model_type: ModelType, model: str) -> dict[str, Any] | None:
@@ -1392,12 +1392,10 @@ class ProviderConfiguration(BaseModel):
:param model_type: model type
:return:
"""
from core.plugin.impl.model_runtime_factory import create_model_type_instance
model_provider_factory = self.get_model_provider_factory()
return create_model_type_instance(
factory=model_provider_factory, provider=self.provider.provider, model_type=model_type
)
# Get model instance of LLM
return model_provider_factory.get_model_type_instance(provider=self.provider.provider, model_type=model_type)
def get_model_schema(
self, model_type: ModelType, model: str, credentials: dict[str, Any] | None
+3 -3
View File
@@ -4,7 +4,7 @@ from typing import cast
from core.app.entities.app_invoke_entities import ModelConfigWithCredentialsEntity
from core.entities import DEFAULT_PLUGIN_ID
from core.plugin.impl.model_runtime_factory import create_model_type_instance, create_plugin_model_provider_factory
from core.plugin.impl.model_runtime_factory import create_plugin_model_provider_factory
from extensions.ext_hosting_provider import hosting_configuration
from graphon.model_runtime.entities.model_entities import ModelType
from graphon.model_runtime.errors.invoke import InvokeBadRequestError
@@ -44,8 +44,8 @@ def check_moderation(tenant_id: str, model_config: ModelConfigWithCredentialsEnt
model_provider_factory = create_plugin_model_provider_factory(tenant_id=tenant_id)
# Get model instance of LLM
model_type_instance = create_model_type_instance(
factory=model_provider_factory, provider=openai_provider_name, model_type=ModelType.MODERATION
model_type_instance = model_provider_factory.get_model_type_instance(
provider=openai_provider_name, model_type=ModelType.MODERATION
)
model_type_instance = cast(ModerationModel, model_type_instance)
moderation_result = model_type_instance.invoke(
+3 -80
View File
@@ -4,7 +4,7 @@ import hashlib
import logging
from collections.abc import Generator, Iterable, Sequence
from threading import Lock
from typing import IO, Any, Literal, Union, overload
from typing import IO, Any, Union
from pydantic import ValidationError
from redis import RedisError
@@ -14,18 +14,13 @@ from core.plugin.entities.plugin_daemon import PluginModelProviderEntity
from core.plugin.impl.asset import PluginAssetManager
from core.plugin.impl.model import PluginModelClient
from extensions.ext_redis import redis_client
from graphon.model_runtime import ModelRuntime
from graphon.model_runtime.entities.llm_entities import (
LLMResult,
LLMResultChunk,
LLMResultChunkWithStructuredOutput,
LLMResultWithStructuredOutput,
)
from graphon.model_runtime.entities.llm_entities import LLMResult, LLMResultChunk
from graphon.model_runtime.entities.message_entities import PromptMessage, PromptMessageTool
from graphon.model_runtime.entities.model_entities import AIModelEntity, ModelType
from graphon.model_runtime.entities.provider_entities import ProviderEntity
from graphon.model_runtime.entities.rerank_entities import MultimodalRerankInput, RerankResult
from graphon.model_runtime.entities.text_embedding_entities import EmbeddingInputType, EmbeddingResult
from graphon.model_runtime.runtime import ModelRuntime
from models.provider_ids import ModelProviderID
logger = logging.getLogger(__name__)
@@ -200,34 +195,6 @@ class PluginModelRuntime(ModelRuntime):
return schema
@overload
def invoke_llm(
self,
*,
provider: str,
model: str,
credentials: dict[str, Any],
model_parameters: dict[str, Any],
prompt_messages: Sequence[PromptMessage],
tools: list[PromptMessageTool] | None,
stop: Sequence[str] | None,
stream: Literal[False],
) -> LLMResult: ...
@overload
def invoke_llm(
self,
*,
provider: str,
model: str,
credentials: dict[str, Any],
model_parameters: dict[str, Any],
prompt_messages: Sequence[PromptMessage],
tools: list[PromptMessageTool] | None,
stop: Sequence[str] | None,
stream: Literal[True],
) -> Generator[LLMResultChunk, None, None]: ...
def invoke_llm(
self,
*,
@@ -255,50 +222,6 @@ class PluginModelRuntime(ModelRuntime):
stream=stream,
)
@overload
def invoke_llm_with_structured_output(
self,
*,
provider: str,
model: str,
credentials: dict[str, Any],
json_schema: dict[str, Any],
model_parameters: dict[str, Any],
prompt_messages: Sequence[PromptMessage],
stop: Sequence[str] | None,
stream: Literal[False],
) -> LLMResultWithStructuredOutput: ...
@overload
def invoke_llm_with_structured_output(
self,
*,
provider: str,
model: str,
credentials: dict[str, Any],
json_schema: dict[str, Any],
model_parameters: dict[str, Any],
prompt_messages: Sequence[PromptMessage],
stop: Sequence[str] | None,
stream: Literal[True],
) -> Generator[LLMResultChunkWithStructuredOutput, None, None]: ...
def invoke_llm_with_structured_output(
self,
*,
provider: str,
model: str,
credentials: dict[str, Any],
json_schema: dict[str, Any],
model_parameters: dict[str, Any],
prompt_messages: Sequence[PromptMessage],
stop: Sequence[str] | None,
stream: bool,
) -> LLMResultWithStructuredOutput | Generator[LLMResultChunkWithStructuredOutput, None, None]:
# TODO: added to pass type check.
# it is a new method from upstream that is not invoked at all.
raise NotImplementedError
def get_llm_num_tokens(
self,
*,
+1 -45
View File
@@ -3,14 +3,6 @@ from __future__ import annotations
from typing import TYPE_CHECKING
from core.plugin.impl.model import PluginModelClient
from graphon.model_runtime.entities.model_entities import ModelType
from graphon.model_runtime.model_providers.base.ai_model import AIModel
from graphon.model_runtime.model_providers.base.large_language_model import LargeLanguageModel
from graphon.model_runtime.model_providers.base.moderation_model import ModerationModel
from graphon.model_runtime.model_providers.base.rerank_model import RerankModel
from graphon.model_runtime.model_providers.base.speech2text_model import Speech2TextModel
from graphon.model_runtime.model_providers.base.text_embedding_model import TextEmbeddingModel
from graphon.model_runtime.model_providers.base.tts_model import TTSModel
from graphon.model_runtime.model_providers.model_provider_factory import ModelProviderFactory
if TYPE_CHECKING:
@@ -18,15 +10,6 @@ if TYPE_CHECKING:
from core.plugin.impl.model_runtime import PluginModelRuntime
from core.provider_manager import ProviderManager
_MODEL_TYPE_CLASS_MAP: dict[ModelType, type[AIModel]] = {
ModelType.LLM: LargeLanguageModel,
ModelType.TEXT_EMBEDDING: TextEmbeddingModel,
ModelType.RERANK: RerankModel,
ModelType.SPEECH2TEXT: Speech2TextModel,
ModelType.MODERATION: ModerationModel,
ModelType.TTS: TTSModel,
}
class PluginModelAssembly:
"""Compose request-scoped model views on top of a single plugin runtime."""
@@ -55,7 +38,7 @@ class PluginModelAssembly:
@property
def model_provider_factory(self) -> ModelProviderFactory:
if self._model_provider_factory is None:
self._model_provider_factory = ModelProviderFactory(runtime=self.model_runtime)
self._model_provider_factory = ModelProviderFactory(model_runtime=self.model_runtime)
return self._model_provider_factory
@property
@@ -104,30 +87,3 @@ def create_plugin_provider_manager(*, tenant_id: str, user_id: str | None = None
def create_plugin_model_manager(*, tenant_id: str, user_id: str | None = None) -> ModelManager:
"""Create a tenant-bound model manager for service flows."""
return create_plugin_model_assembly(tenant_id=tenant_id, user_id=user_id).model_manager
def create_model_type_instance(
factory: ModelProviderFactory,
provider: str,
model_type: ModelType,
) -> AIModel:
"""Instantiate the AIModel subclass for *model_type* backed by *factory*'s runtime.
This replaces ``ModelProviderFactory.get_model_type_instance`` which was
removed in graphon 0.3.0. The mapping from ModelType to concrete AIModel
subclass is maintained here so that callers do not need to know the
subclass constructors.
:param factory: factory whose ``runtime`` and provider resolution are used.
:param provider: provider identifier (canonical or short name).
:param model_type: the model type to instantiate.
:returns: an AIModel subclass instance wired to the factory's runtime.
:raises ValueError: if *model_type* is not supported.
"""
model_class = _MODEL_TYPE_CLASS_MAP.get(model_type)
if model_class is None:
msg = f"Unsupported model type: {model_type}"
raise ValueError(msg)
provider_entity = factory.get_model_provider(provider)
return model_class(provider_schema=provider_entity, model_runtime=factory.runtime)
+12 -12
View File
@@ -9,9 +9,9 @@ from typing import TYPE_CHECKING, Any
from pydantic import TypeAdapter
from sqlalchemy import select
from sqlalchemy.exc import IntegrityError
from sqlalchemy.orm import Session
from configs import dify_config
from core.db.session_factory import session_factory
from core.entities.model_entities import DefaultModelEntity, DefaultModelProviderEntity
from core.entities.provider_configuration import ProviderConfiguration, ProviderConfigurations, ProviderModelBundle
from core.entities.provider_entities import (
@@ -56,7 +56,7 @@ from models.provider_ids import ModelProviderID
from services.feature_service import FeatureService
if TYPE_CHECKING:
from graphon.model_runtime import ModelRuntime
from graphon.model_runtime.runtime import ModelRuntime
_credentials_adapter: TypeAdapter[dict[str, Any]] = TypeAdapter(dict[str, Any])
@@ -165,7 +165,7 @@ class ProviderManager:
)
# Get all provider entities
model_provider_factory = ModelProviderFactory(runtime=self._model_runtime)
model_provider_factory = ModelProviderFactory(model_runtime=self._model_runtime)
provider_entities = model_provider_factory.get_providers()
# Get All preferred provider types of the workspace
@@ -362,7 +362,7 @@ class ProviderManager:
if not default_model:
return None
model_provider_factory = ModelProviderFactory(runtime=self._model_runtime)
model_provider_factory = ModelProviderFactory(model_runtime=self._model_runtime)
provider_schema = model_provider_factory.get_provider_schema(provider=default_model.provider_name)
return DefaultModelEntity(
@@ -445,7 +445,7 @@ class ProviderManager:
@staticmethod
def _get_all_providers(tenant_id: str) -> dict[str, list[Provider]]:
provider_name_to_provider_records_dict = defaultdict(list)
with Session(db.engine, expire_on_commit=False) as session:
with session_factory.create_session() as session:
stmt = select(Provider).where(Provider.tenant_id == tenant_id, Provider.is_valid == True)
providers = session.scalars(stmt)
for provider in providers:
@@ -462,7 +462,7 @@ class ProviderManager:
:return:
"""
provider_name_to_provider_model_records_dict = defaultdict(list)
with Session(db.engine, expire_on_commit=False) as session:
with session_factory.create_session() as session:
stmt = select(ProviderModel).where(ProviderModel.tenant_id == tenant_id, ProviderModel.is_valid == True)
provider_models = session.scalars(stmt)
for provider_model in provider_models:
@@ -478,7 +478,7 @@ class ProviderManager:
:return:
"""
provider_name_to_preferred_provider_type_records_dict = {}
with Session(db.engine, expire_on_commit=False) as session:
with session_factory.create_session() as session:
stmt = select(TenantPreferredModelProvider).where(TenantPreferredModelProvider.tenant_id == tenant_id)
preferred_provider_types = session.scalars(stmt)
provider_name_to_preferred_provider_type_records_dict = {
@@ -496,7 +496,7 @@ class ProviderManager:
:return:
"""
provider_name_to_provider_model_settings_dict = defaultdict(list)
with Session(db.engine, expire_on_commit=False) as session:
with session_factory.create_session() as session:
stmt = select(ProviderModelSetting).where(ProviderModelSetting.tenant_id == tenant_id)
provider_model_settings = session.scalars(stmt)
for provider_model_setting in provider_model_settings:
@@ -514,7 +514,7 @@ class ProviderManager:
:return:
"""
provider_name_to_provider_model_credentials_dict = defaultdict(list)
with Session(db.engine, expire_on_commit=False) as session:
with session_factory.create_session() as session:
stmt = select(ProviderModelCredential).where(ProviderModelCredential.tenant_id == tenant_id)
provider_model_credentials = session.scalars(stmt)
for provider_model_credential in provider_model_credentials:
@@ -544,7 +544,7 @@ class ProviderManager:
return {}
provider_name_to_provider_load_balancing_model_configs_dict = defaultdict(list)
with Session(db.engine, expire_on_commit=False) as session:
with session_factory.create_session() as session:
stmt = select(LoadBalancingModelConfig).where(LoadBalancingModelConfig.tenant_id == tenant_id)
provider_load_balancing_configs = session.scalars(stmt)
for provider_load_balancing_config in provider_load_balancing_configs:
@@ -578,7 +578,7 @@ class ProviderManager:
:param provider_name: provider name
:return:
"""
with Session(db.engine, expire_on_commit=False) as session:
with session_factory.create_session() as session:
stmt = (
select(ProviderCredential)
.where(
@@ -608,7 +608,7 @@ class ProviderManager:
:param model_type: model type
:return:
"""
with Session(db.engine, expire_on_commit=False) as session:
with session_factory.create_session() as session:
stmt = (
select(ProviderModelCredential)
.where(
+8 -8
View File
@@ -217,10 +217,11 @@ class RetrievalService:
"""Deduplicate documents in O(n) while preserving first-seen order.
Rules:
- For provider == "dify" and metadata["doc_id"] exists: keep the doc with the highest
metadata["score"] among duplicates; if a later duplicate has no score, ignore it.
- For non-dify documents (or dify without doc_id): deduplicate by content key
(provider, page_content), keeping the first occurrence.
- If metadata["doc_id"] exists (any provider): deduplicate by (provider, doc_id) key;
keep the doc with the highest metadata["score"] among duplicates. If a later duplicate
has no score, ignore it.
- If metadata["doc_id"] is absent: deduplicate by content key (provider, page_content),
keeping the first occurrence.
"""
if not documents:
return documents
@@ -231,11 +232,10 @@ class RetrievalService:
order: list[tuple] = []
for doc in documents:
is_dify = doc.provider == "dify"
doc_id = (doc.metadata or {}).get("doc_id") if is_dify else None
doc_id = (doc.metadata or {}).get("doc_id")
if is_dify and doc_id:
key = ("dify", doc_id)
if doc_id:
key = (doc.provider or "dify", doc_id)
if key not in chosen:
chosen[key] = doc
order.append(key)
@@ -144,8 +144,20 @@ class Vector:
def get_vector_factory(vector_type: str) -> type[AbstractVectorFactory]:
return get_vector_factory_class(vector_type)
@staticmethod
def _filter_empty_text_documents(documents: list[Document]) -> list[Document]:
filtered_documents = [document for document in documents if document.page_content.strip()]
skipped_count = len(documents) - len(filtered_documents)
if skipped_count:
logger.warning("skip %d empty documents before vector embedding", skipped_count)
return filtered_documents
def create(self, texts: list | None = None, **kwargs):
if texts:
texts = self._filter_empty_text_documents(texts)
if not texts:
return
start = time.time()
logger.info("start embedding %s texts %s", len(texts), start)
batch_size = 1000
@@ -203,8 +215,14 @@ class Vector:
logger.info("Embedding %s files took %s s", len(file_documents), time.time() - start)
def add_texts(self, documents: list[Document], **kwargs):
documents = self._filter_empty_text_documents(documents)
if not documents:
return
if kwargs.get("duplicate_check", False):
documents = self._filter_duplicate_texts(documents)
if not documents:
return
embeddings = self._embeddings.embed_documents([document.page_content for document in documents])
self._vector_processor.create(texts=documents, embeddings=embeddings, **kwargs)
+41
View File
@@ -1078,6 +1078,13 @@ class ToolManager:
if parameter.form == ToolParameter.ToolParameterForm.FORM:
if variable_pool:
config = tool_configurations.get(parameter.name, {})
selector_value = cls._extract_runtime_selector_value(parameter, config)
if selector_value is not None:
# Selector parameters carry structured dictionaries, not scalar ToolInput values.
runtime_parameters[parameter.name] = selector_value
continue
if not (config and isinstance(config, dict) and config.get("value") is not None):
continue
tool_input = ToolNodeData.ToolInput.model_validate(tool_configurations.get(parameter.name, {}))
@@ -1105,5 +1112,39 @@ class ToolManager:
runtime_parameters[parameter.name] = value
return runtime_parameters
@classmethod
def _extract_runtime_selector_value(cls, parameter: ToolParameter, config: Any) -> dict[str, Any] | None:
if parameter.type not in {
ToolParameter.ToolParameterType.MODEL_SELECTOR,
ToolParameter.ToolParameterType.APP_SELECTOR,
}:
return None
if not isinstance(config, dict):
return None
input_value = config.get("value")
if isinstance(input_value, dict) and cls._is_selector_value(parameter, input_value):
return cast("dict[str, Any]", parameter.init_frontend_parameter(input_value))
if cls._is_selector_value(parameter, config):
selector_value = dict(config)
selector_value.pop("type", None)
selector_value.pop("value", None)
return cast("dict[str, Any]", parameter.init_frontend_parameter(selector_value))
return None
@classmethod
def _is_selector_value(cls, parameter: ToolParameter, value: Mapping[str, Any]) -> bool:
if parameter.type == ToolParameter.ToolParameterType.MODEL_SELECTOR:
return (
isinstance(value.get("provider"), str)
and isinstance(value.get("model"), str)
and isinstance(value.get("model_type"), str)
)
if parameter.type == ToolParameter.ToolParameterType.APP_SELECTOR:
return isinstance(value.get("app_id"), str)
return False
ToolManager.load_hardcoded_providers_cache()
+30
View File
@@ -272,6 +272,14 @@ def _adapt_tool_node_data_for_graph(node_data: Mapping[str, Any]) -> dict[str, A
normalized_tool_configurations[name] = value
continue
selector_value = _extract_selector_configuration(value)
if selector_value is not None:
# Model/app selectors are dictionaries even when they come through the legacy tool configuration path.
# Move them to tool_parameters so graph validation does not flatten them as primitive constants.
found_legacy_tool_inputs = True
normalized_tool_parameters.setdefault(name, {"type": "constant", "value": selector_value})
continue
input_type = value.get("type")
input_value = value.get("value")
if input_type not in {"mixed", "variable", "constant"}:
@@ -310,6 +318,28 @@ def _flatten_legacy_tool_configuration_value(*, input_type: Any, input_value: An
return None
def _extract_selector_configuration(value: Mapping[str, Any]) -> dict[str, Any] | None:
input_value = value.get("value")
if isinstance(input_value, Mapping) and _is_selector_configuration(input_value):
return dict(input_value)
if _is_selector_configuration(value):
selector_value = dict(value)
selector_value.pop("type", None)
selector_value.pop("value", None)
return selector_value
return None
def _is_selector_configuration(value: Mapping[str, Any]) -> bool:
return (
isinstance(value.get("provider"), str)
and isinstance(value.get("model"), str)
and isinstance(value.get("model_type"), str)
) or isinstance(value.get("app_id"), str)
def _normalize_email_recipients(recipients: Mapping[str, Any]) -> dict[str, Any]:
normalized = dict(recipients)
+1 -1
View File
@@ -448,7 +448,7 @@ class DifyNodeFactory(NodeFactory):
node_init_kwargs = node_init_kwargs_factories.get(node_type, lambda: {})()
return node_class(
node_id=node_id,
data=config_for_node_init,
config=config_for_node_init,
graph_init_params=self.graph_init_params,
graph_runtime_state=self.graph_runtime_state,
**node_init_kwargs,
+5 -1
View File
@@ -501,11 +501,15 @@ class DifyToolNodeRuntime(ToolNodeRuntimeProtocol):
@staticmethod
def _build_tool_runtime_spec(node_data: ToolNodeData) -> _WorkflowToolRuntimeSpec:
tool_configurations = dict(node_data.tool_configurations)
tool_configurations.update(
{name: tool_input.model_dump(mode="python") for name, tool_input in node_data.tool_parameters.items()}
)
return _WorkflowToolRuntimeSpec(
provider_type=CoreToolProviderType(node_data.provider_type.value),
provider_id=node_data.provider_id,
tool_name=node_data.tool_name,
tool_configurations=dict(node_data.tool_configurations),
tool_configurations=tool_configurations,
credential_id=node_data.credential_id,
)
+2 -2
View File
@@ -35,7 +35,7 @@ class AgentNode(Node[AgentNodeData]):
def __init__(
self,
node_id: str,
data: AgentNodeData,
config: AgentNodeData,
*,
graph_init_params: GraphInitParams,
graph_runtime_state: GraphRuntimeState,
@@ -46,7 +46,7 @@ class AgentNode(Node[AgentNodeData]):
) -> None:
super().__init__(
node_id=node_id,
data=data,
config=config,
graph_init_params=graph_init_params,
graph_runtime_state=graph_runtime_state,
)
@@ -36,14 +36,14 @@ class DatasourceNode(Node[DatasourceNodeData]):
def __init__(
self,
node_id: str,
data: DatasourceNodeData,
config: DatasourceNodeData,
*,
graph_init_params: "GraphInitParams",
graph_runtime_state: "GraphRuntimeState",
) -> None:
super().__init__(
node_id=node_id,
data=data,
config=config,
graph_init_params=graph_init_params,
graph_runtime_state=graph_runtime_state,
)
@@ -32,14 +32,14 @@ class KnowledgeIndexNode(Node[KnowledgeIndexNodeData]):
def __init__(
self,
node_id: str,
data: KnowledgeIndexNodeData,
config: KnowledgeIndexNodeData,
*,
graph_init_params: "GraphInitParams",
graph_runtime_state: "GraphRuntimeState",
) -> None:
super().__init__(
node_id=node_id,
data=data,
config=config,
graph_init_params=graph_init_params,
graph_runtime_state=graph_runtime_state,
)
@@ -71,14 +71,14 @@ class KnowledgeRetrievalNode(LLMUsageTrackingMixin, Node[KnowledgeRetrievalNodeD
def __init__(
self,
node_id: str,
data: KnowledgeRetrievalNodeData,
config: KnowledgeRetrievalNodeData,
*,
graph_init_params: "GraphInitParams",
graph_runtime_state: "GraphRuntimeState",
) -> None:
super().__init__(
node_id=node_id,
data=data,
config=config,
graph_init_params=graph_init_params,
graph_runtime_state=graph_runtime_state,
)
+15 -9
View File
@@ -3,7 +3,7 @@ from __future__ import annotations
from collections import defaultdict
from collections.abc import Mapping, Sequence
from enum import StrEnum
from typing import Any, Protocol
from typing import Any, Protocol, cast
from uuid import uuid4
from graphon.enums import BuiltinNodeTypes
@@ -82,10 +82,13 @@ def build_system_variables(values: Mapping[str, Any] | None = None, /, **kwargs:
normalized = _normalize_system_variable_values(values, **kwargs)
return [
segment_to_variable(
segment=build_segment(value),
selector=system_variable_selector(key),
name=key,
cast(
Variable,
segment_to_variable(
segment=build_segment(value),
selector=system_variable_selector(key),
name=key,
),
)
for key, value in normalized.items()
]
@@ -127,10 +130,13 @@ def build_bootstrap_variables(
for node_id, value in rag_pipeline_variables_map.items():
variables.append(
segment_to_variable(
segment=build_segment(value),
selector=(RAG_PIPELINE_VARIABLE_NODE_ID, node_id),
name=node_id,
cast(
Variable,
segment_to_variable(
segment=build_segment(value),
selector=(RAG_PIPELINE_VARIABLE_NODE_ID, node_id),
name=node_id,
),
)
)
+1 -1
View File
@@ -411,7 +411,7 @@ class WorkflowEntry:
raise ValueError(f"Node class not found for node type {node_type}")
# init variable pool
variable_pool = VariablePool.from_bootstrap()
variable_pool = VariablePool()
add_variables_to_pool(variable_pool, default_system_variables())
# init graph context and runtime state
+3 -1
View File
@@ -1,7 +1,9 @@
from flask import Flask
from core.db.session_factory import configure_session_factory
from extensions.ext_database import db
def init_app(app):
def init_app(app: Flask):
with app.app_context():
configure_session_factory(db.engine)
+1 -1
View File
@@ -298,7 +298,7 @@ def _build_from_datasource_file(
raise ValueError(f"DatasourceFile {mapping.get('datasource_file_id')} not found")
extension = "." + datasource_file.key.split(".")[-1] if "." in datasource_file.key else ".bin"
detected_file_type = standardize_file_type(extension="." + extension, mime_type=datasource_file.mime_type)
detected_file_type = standardize_file_type(extension=extension, mime_type=datasource_file.mime_type)
file_type = _resolve_file_type(
detected_file_type=detected_file_type,
specified_type=mapping.get("type"),
+14 -4
View File
@@ -19,8 +19,13 @@ from werkzeug.http import parse_options_header
from core.helper import ssrf_proxy
def extract_filename(url_path: str, content_disposition: str | None) -> str | None:
"""Extract a safe filename from Content-Disposition or the request URL path."""
def extract_filename(url_or_path: str, content_disposition: str | None) -> str | None:
"""Extract a safe filename from Content-Disposition or the request URL path.
Handles full URLs, paths with query strings, hash fragments, and percent-encoded segments.
Query strings and hash fragments are stripped from the URL before extracting the basename.
Percent-encoded characters in the path are decoded safely.
"""
filename: str | None = None
if content_disposition:
filename_star_match = re.search(r"filename\*=([^;]+)", content_disposition)
@@ -47,8 +52,13 @@ def extract_filename(url_path: str, content_disposition: str | None) -> str | No
filename = urllib.parse.unquote(raw)
if not filename:
candidate = os.path.basename(url_path)
filename = urllib.parse.unquote(candidate) if candidate else None
# Parse the URL to extract just the path, stripping query strings and fragments
# This handles both full URLs and bare paths
parsed = urllib.parse.urlparse(url_or_path)
path = parsed.path
candidate = os.path.basename(path)
# Decode percent-encoded characters, with safe fallback for malformed input
filename = urllib.parse.unquote(candidate, errors="replace") if candidate else None
if filename:
filename = os.path.basename(filename)
-9
View File
@@ -1,9 +0,0 @@
from typing import TypeGuard
def is_str_dict(v: object) -> TypeGuard[dict[str, object]]:
return isinstance(v, dict)
def is_str(v: object) -> TypeGuard[str]:
return isinstance(v, str)
@@ -0,0 +1,26 @@
"""add recommended app categories
Revision ID: a4f2d8c9b731
Revises: 227822d22895
Create Date: 2026-04-29 12:00:00.000000
"""
import sqlalchemy as sa
from alembic import op
# revision identifiers, used by Alembic.
revision = "a4f2d8c9b731"
down_revision = "227822d22895"
branch_labels = None
depends_on = None
def upgrade():
with op.batch_alter_table("recommended_apps", schema=None) as batch_op:
batch_op.add_column(sa.Column("categories", sa.JSON(), nullable=True))
def downgrade():
with op.batch_alter_table("recommended_apps", schema=None) as batch_op:
batch_op.drop_column("categories")
+1
View File
@@ -878,6 +878,7 @@ class RecommendedApp(TypeBase):
copyright: Mapped[str] = mapped_column(String(255), nullable=False)
privacy_policy: Mapped[str] = mapped_column(String(255), nullable=False)
category: Mapped[str] = mapped_column(String(255), nullable=False)
categories: Mapped[list[str] | None] = mapped_column(sa.JSON, nullable=True, default=None)
custom_disclaimer: Mapped[str] = mapped_column(LongText, default="")
position: Mapped[int] = mapped_column(sa.Integer, nullable=False, default=0)
is_listed: Mapped[bool] = mapped_column(sa.Boolean, nullable=False, default=True)
+7 -5
View File
@@ -9,11 +9,11 @@ import sqlalchemy as sa
from sqlalchemy import DateTime, String, func, select, text
from sqlalchemy.orm import Mapped, mapped_column
from core.db.session_factory import session_factory
from graphon.model_runtime.entities.model_entities import ModelType
from libs.uuid_utils import uuidv7
from .base import TypeBase
from .engine import db
from .enums import CredentialSourceType, PaymentStatus, ProviderQuotaType
from .types import EnumText, LongText, StringUUID
@@ -82,7 +82,8 @@ class Provider(TypeBase):
@cached_property
def credential(self):
if self.credential_id:
return db.session.scalar(select(ProviderCredential).where(ProviderCredential.id == self.credential_id))
with session_factory.create_session() as session:
return session.scalar(select(ProviderCredential).where(ProviderCredential.id == self.credential_id))
@property
def credential_name(self):
@@ -145,9 +146,10 @@ class ProviderModel(TypeBase):
@cached_property
def credential(self):
if self.credential_id:
return db.session.scalar(
select(ProviderModelCredential).where(ProviderModelCredential.id == self.credential_id)
)
with session_factory.create_session() as session:
return session.scalar(
select(ProviderModelCredential).where(ProviderModelCredential.id == self.credential_id)
)
@property
def credential_name(self):
@@ -246,8 +246,18 @@ class TidbService:
userPrefix = item["userPrefix"]
if state == "ACTIVE" and len(userPrefix) > 0:
cluster_info = tidb_serverless_list_map[item["clusterId"]]
cluster_info.status = TidbAuthBindingStatus.ACTIVE
cluster_info.account = f"{userPrefix}.root"
if not cluster_info.qdrant_endpoint:
cluster_info.qdrant_endpoint = TidbService.extract_qdrant_endpoint(
item
) or TidbService.fetch_qdrant_endpoint(api_url, public_key, private_key, item["clusterId"])
if cluster_info.qdrant_endpoint:
cluster_info.status = TidbAuthBindingStatus.ACTIVE
else:
logger.warning(
"Cluster %s is ACTIVE but qdrant endpoint is not ready; will retry later",
item["clusterId"],
)
db.session.add(cluster_info)
db.session.commit()
else:
@@ -1,8 +1,11 @@
from types import SimpleNamespace
from unittest.mock import MagicMock, patch
import pytest
from dify_vdb_tidb_on_qdrant.tidb_service import TidbService
from models.enums import TidbAuthBindingStatus
class TestExtractQdrantEndpoint:
"""Unit tests for TidbService.extract_qdrant_endpoint."""
@@ -216,3 +219,86 @@ class TestBatchCreateEdgeCases:
private_key="priv",
region="us-east-1",
)
class TestBatchUpdateTidbServerlessClusterStatus:
"""Verify that status updates only expose clusters after qdrant endpoint is ready."""
@patch("dify_vdb_tidb_on_qdrant.tidb_service.db")
@patch("dify_vdb_tidb_on_qdrant.tidb_service._tidb_http_client")
def test_sets_active_when_batch_response_contains_endpoint(self, mock_http, mock_db):
binding = SimpleNamespace(
cluster_id="c-1",
status=TidbAuthBindingStatus.CREATING,
account="root",
qdrant_endpoint=None,
)
mock_http.get.return_value = MagicMock(
status_code=200,
json=lambda: {
"clusters": [
{
"clusterId": "c-1",
"state": "ACTIVE",
"userPrefix": "pfx",
"endpoints": {"public": {"host": "gw.tidbcloud.com"}},
}
]
},
)
TidbService.batch_update_tidb_serverless_cluster_status([binding], "proj", "url", "iam", "pub", "priv")
assert binding.account == "pfx.root"
assert binding.qdrant_endpoint == "https://qdrant-gw.tidbcloud.com"
assert binding.status == TidbAuthBindingStatus.ACTIVE
mock_db.session.add.assert_called_once_with(binding)
mock_db.session.commit.assert_called_once()
@patch.object(TidbService, "fetch_qdrant_endpoint", return_value="https://qdrant-gw.tidbcloud.com")
@patch("dify_vdb_tidb_on_qdrant.tidb_service.db")
@patch("dify_vdb_tidb_on_qdrant.tidb_service._tidb_http_client")
def test_fetches_endpoint_when_batch_response_omits_it(self, mock_http, mock_db, mock_fetch_endpoint):
binding = SimpleNamespace(
cluster_id="c-1",
status=TidbAuthBindingStatus.CREATING,
account="root",
qdrant_endpoint=None,
)
mock_http.get.return_value = MagicMock(
status_code=200,
json=lambda: {"clusters": [{"clusterId": "c-1", "state": "ACTIVE", "userPrefix": "pfx", "endpoints": {}}]},
)
TidbService.batch_update_tidb_serverless_cluster_status([binding], "proj", "url", "iam", "pub", "priv")
assert binding.account == "pfx.root"
assert binding.qdrant_endpoint == "https://qdrant-gw.tidbcloud.com"
assert binding.status == TidbAuthBindingStatus.ACTIVE
mock_fetch_endpoint.assert_called_once_with("url", "pub", "priv", "c-1")
mock_db.session.add.assert_called_once_with(binding)
mock_db.session.commit.assert_called_once()
@patch.object(TidbService, "fetch_qdrant_endpoint", return_value=None)
@patch("dify_vdb_tidb_on_qdrant.tidb_service.db")
@patch("dify_vdb_tidb_on_qdrant.tidb_service._tidb_http_client")
def test_keeps_creating_when_endpoint_is_not_ready(self, mock_http, mock_db, mock_fetch_endpoint):
binding = SimpleNamespace(
cluster_id="c-1",
status=TidbAuthBindingStatus.CREATING,
account="root",
qdrant_endpoint=None,
)
mock_http.get.return_value = MagicMock(
status_code=200,
json=lambda: {"clusters": [{"clusterId": "c-1", "state": "ACTIVE", "userPrefix": "pfx", "endpoints": {}}]},
)
TidbService.batch_update_tidb_serverless_cluster_status([binding], "proj", "url", "iam", "pub", "priv")
assert binding.account == "pfx.root"
assert binding.qdrant_endpoint is None
assert binding.status == TidbAuthBindingStatus.CREATING
mock_fetch_endpoint.assert_called_once_with("url", "pub", "priv", "c-1")
mock_db.session.add.assert_called_once_with(binding)
mock_db.session.commit.assert_called_once()
+12 -12
View File
@@ -6,7 +6,7 @@ requires-python = "~=3.12.0"
dependencies = [
# Legacy: mature and widely deployed
"bleach>=6.3.0",
"boto3>=1.42.96",
"boto3>=1.43.3",
"celery>=5.6.3",
"croniter>=6.2.2",
"flask>=3.1.3,<4.0.0",
@@ -14,7 +14,7 @@ dependencies = [
"gevent>=26.4.0",
"gevent-websocket>=0.10.1",
"gmpy2>=2.3.0",
"google-api-python-client>=2.194.0",
"google-api-python-client>=2.195.0",
"gunicorn>=25.3.0",
"psycogreen>=1.0.2",
"psycopg2-binary>=2.9.12",
@@ -31,7 +31,7 @@ dependencies = [
"flask-migrate>=4.1.0,<5.0.0",
"flask-orjson>=2.0.0,<3.0.0",
"flask-restx>=1.3.2,<2.0.0",
"google-cloud-aiplatform>=1.148.1,<2.0.0",
"google-cloud-aiplatform>=1.149.0,<2.0.0",
"httpx[socks]>=0.28.1,<1.0.0",
"opentelemetry-distro>=0.62b1,<1.0.0",
"opentelemetry-instrumentation-celery>=0.62b0,<1.0.0",
@@ -45,7 +45,7 @@ dependencies = [
# Emerging: newer and fast-moving, use compatible pins
"fastopenapi[flask]~=0.7.0",
"graphon~=0.3.0",
"graphon~=0.2.2",
"httpx-sse~=0.4.0",
"json-repair~=0.59.4",
]
@@ -127,7 +127,7 @@ dev = [
"testcontainers>=4.14.2",
"types-aiofiles>=25.1.0",
"types-beautifulsoup4>=4.12.0",
"types-cachetools>=6.2.0",
"types-cachetools>=7.0.0.20260503",
"types-colorama>=0.4.15",
"types-defusedxml>=0.7.0",
"types-deprecated>=1.3.1",
@@ -135,7 +135,7 @@ dev = [
"types-flask-cors>=6.0.0",
"types-flask-migrate>=4.1.0",
"types-gevent>=26.4.0",
"types-greenlet>=3.4.0",
"types-greenlet>=3.5.0.20260428",
"types-html5lib>=1.1.11",
"types-markdown>=3.10.2",
"types-oauthlib>=3.3.0",
@@ -143,7 +143,7 @@ dev = [
"types-olefile>=0.47.0",
"types-openpyxl>=3.1.5",
"types-pexpect>=4.9.0",
"types-protobuf>=7.34.1",
"types-protobuf>=7.34.1.20260503",
"types-psutil>=7.2.2",
"types-psycopg2>=2.9.21.20260422",
"types-pygments>=2.20.0",
@@ -158,11 +158,11 @@ dev = [
"types-tensorflow>=2.18.0.20260408",
"types-tqdm>=4.67.3.20260408",
"types-ujson>=5.10.0",
"boto3-stubs>=1.42.96",
"boto3-stubs>=1.43.2",
"types-jmespath>=1.1.0.20260408",
"hypothesis>=6.152.3",
"hypothesis>=6.152.4",
"types_pyOpenSSL>=24.1.0",
"types_cffi>=2.0.0.20260408",
"types_cffi>=2.0.0.20260429",
"types_setuptools>=82.0.0.20260408",
"pandas-stubs>=3.0.0",
"scipy-stubs>=1.17.1.4",
@@ -174,7 +174,7 @@ dev = [
# "locust>=2.40.4", # Temporarily removed due to compatibility issues. Uncomment when resolved.
"pytest-timeout>=2.4.0",
"pytest-xdist>=3.8.0",
"pyrefly>=0.62.0",
"pyrefly>=0.64.0",
"xinference-client>=2.7.0",
]
@@ -184,7 +184,7 @@ dev = [
############################################################
storage = [
"azure-storage-blob>=12.28.0",
"bce-python-sdk>=0.9.70",
"bce-python-sdk>=0.9.71",
"cos-python-sdk-v5>=1.9.42",
"esdk-obs-python>=3.22.2",
"google-cloud-storage>=3.10.1",
@@ -3,6 +3,7 @@ from typing import Any, Literal
from pydantic import BaseModel, field_validator
from core.rag.entities import Rule
from core.rag.entities.metadata_entities import MetadataFilteringCondition
from core.rag.index_processor.constant.index_type import IndexStructureType
from core.rag.retrieval.retrieval_methods import RetrievalMethod
@@ -83,6 +84,7 @@ class RetrievalModel(BaseModel):
score_threshold_enabled: bool
score_threshold: float | None = None
weights: WeightModel | None = None
metadata_filtering_conditions: MetadataFilteringCondition | None = None
class MetaDataConfig(BaseModel):
+1 -1
View File
@@ -88,7 +88,7 @@ logger = logging.getLogger(__name__)
def _build_seeded_variable_pool(variables: Sequence[Variable]) -> VariablePool:
variable_pool = VariablePool.from_bootstrap()
variable_pool = VariablePool()
add_variables_to_pool(variable_pool, variables)
return variable_pool
@@ -0,0 +1,49 @@
"""Apply Redis-backed category ordering for DB-backed Explore apps."""
import json
import logging
from collections.abc import Collection
from typing import Any
from extensions.ext_redis import redis_client
logger = logging.getLogger(__name__)
EXPLORE_APP_CATEGORY_ORDER_KEY_PREFIX = "explore:apps:category_order"
def _category_order_key(language: str) -> str:
return f"{EXPLORE_APP_CATEGORY_ORDER_KEY_PREFIX}:{language}"
def get_explore_app_category_order(language: str) -> list[str]:
try:
raw_categories = redis_client.get(_category_order_key(language))
except Exception:
logger.exception("Failed to read explore app category order from Redis.")
return []
if not raw_categories:
return []
if isinstance(raw_categories, bytes):
raw_categories = raw_categories.decode("utf-8")
try:
categories: Any = json.loads(raw_categories)
except (TypeError, json.JSONDecodeError):
logger.warning("Invalid explore app category order payload for language %s.", language)
return []
if not isinstance(categories, list):
return []
return [category for category in categories if isinstance(category, str)]
def order_categories(categories: Collection[str], language: str) -> list[str]:
configured_order = get_explore_app_category_order(language)
if configured_order:
return configured_order
return sorted(categories)
@@ -6,6 +6,7 @@ from constants.languages import languages
from extensions.ext_database import db
from models.model import App, RecommendedApp
from services.app_dsl_service import AppDslService
from services.recommend_app.category_order import order_categories
from services.recommend_app.recommend_app_base import RecommendAppRetrievalBase
from services.recommend_app.recommend_app_type import RecommendAppType
@@ -18,7 +19,7 @@ class RecommendedAppItemDict(TypedDict):
copyright: Any
privacy_policy: Any
custom_disclaimer: str
category: str
categories: list[str]
position: int
is_listed: bool
@@ -80,6 +81,7 @@ class DatabaseRecommendAppRetrieval(RecommendAppRetrievalBase):
if not site:
continue
app_categories = recommended_app.categories or []
recommended_app_result: RecommendedAppItemDict = {
"id": recommended_app.id,
"app": recommended_app.app,
@@ -88,15 +90,18 @@ class DatabaseRecommendAppRetrieval(RecommendAppRetrievalBase):
"copyright": site.copyright,
"privacy_policy": site.privacy_policy,
"custom_disclaimer": site.custom_disclaimer,
"category": recommended_app.category,
"categories": app_categories,
"position": recommended_app.position,
"is_listed": recommended_app.is_listed,
}
recommended_apps_result.append(recommended_app_result)
categories.add(recommended_app.category)
categories.update(app_categories)
return RecommendedAppsResultDict(recommended_apps=recommended_apps_result, categories=sorted(categories))
return RecommendedAppsResultDict(
recommended_apps=recommended_apps_result,
categories=order_categories(categories, language),
)
@classmethod
def fetch_recommended_app_detail_from_db(cls, app_id: str) -> RecommendedAppDetailDict | None:
+15 -8
View File
@@ -1,9 +1,11 @@
import uuid
from typing import cast
import sqlalchemy as sa
from flask_login import current_user
from pydantic import BaseModel, Field
from sqlalchemy import func, select
from sqlalchemy import delete, func, select
from sqlalchemy.engine import CursorResult
from werkzeug.exceptions import NotFound
from extensions.ext_database import db
@@ -29,7 +31,7 @@ class TagBindingCreatePayload(BaseModel):
class TagBindingDeletePayload(BaseModel):
tag_id: str
tag_ids: list[str] = Field(min_length=1)
target_id: str
type: TagType
@@ -178,13 +180,18 @@ class TagService:
@staticmethod
def delete_tag_binding(payload: TagBindingDeletePayload):
TagService.check_target_exists(payload.type, payload.target_id)
tag_binding = db.session.scalar(
select(TagBinding)
.where(TagBinding.target_id == payload.target_id, TagBinding.tag_id == payload.tag_id)
.limit(1)
result = cast(
CursorResult,
db.session.execute(
delete(TagBinding).where(
TagBinding.target_id == payload.target_id,
TagBinding.tag_id.in_(payload.tag_ids),
TagBinding.tenant_id == current_user.current_tenant_id,
)
),
)
if tag_binding:
db.session.delete(tag_binding)
if result.rowcount:
db.session.commit()
@staticmethod
@@ -408,7 +408,7 @@ class BuiltinToolManageService:
return {"result": "success"}
@staticmethod
def set_default_provider(tenant_id: str, user_id: str, provider: str, id: str):
def set_default_provider(tenant_id: str, provider: str, id: str):
"""
set default provider
"""
@@ -422,12 +422,11 @@ class BuiltinToolManageService:
if target_provider is None:
raise ValueError("provider not found")
# clear default provider
# clear default provider (tenant-scoped: only one default per provider per workspace)
session.execute(
update(BuiltinToolProvider)
.where(
BuiltinToolProvider.tenant_id == tenant_id,
BuiltinToolProvider.user_id == user_id,
BuiltinToolProvider.provider == provider,
BuiltinToolProvider.is_default.is_(True),
)
@@ -32,7 +32,7 @@ from graphon.file import File
from graphon.nodes import BuiltinNodeTypes
from graphon.nodes.variable_assigner.common.helpers import get_updated_variables
from graphon.variable_loader import VariableLoader
from graphon.variables import Segment, StringSegment, Variable, VariableBase
from graphon.variables import Segment, StringSegment, VariableBase
from graphon.variables.consts import SELECTORS_LENGTH
from graphon.variables.segments import (
ArrayFileSegment,
@@ -162,7 +162,7 @@ class DraftVarLoader(VariableLoader):
return list(variable_by_selector.values())
def _load_offloaded_variable(self, draft_var: WorkflowDraftVariable) -> tuple[tuple[str, str], Variable]:
def _load_offloaded_variable(self, draft_var: WorkflowDraftVariable) -> tuple[tuple[str, str], VariableBase]:
# This logic is closely tied to `WorkflowDraftVaribleService._try_offload_large_variable`
# and must remain synchronized with it.
# Ideally, these should be co-located for better maintainability.
+4 -4
View File
@@ -877,7 +877,7 @@ class WorkflowService:
)
else:
variable_pool = VariablePool.from_bootstrap()
variable_pool = VariablePool()
add_variables_to_pool(
variable_pool,
build_bootstrap_variables(
@@ -1251,7 +1251,7 @@ class WorkflowService:
node_data = HumanInputNode.validate_node_data(adapt_human_input_node_data_for_graph(node_config["data"]))
node = HumanInputNode(
node_id=node_config["id"],
data=node_data,
config=node_data,
graph_init_params=graph_init_params,
graph_runtime_state=graph_runtime_state,
runtime=DifyHumanInputNodeRuntime(run_context),
@@ -1271,7 +1271,7 @@ class WorkflowService:
draft_var_srv = WorkflowDraftVariableService(session)
draft_var_srv.prefill_conversation_variable_default_values(workflow, user_id=user_id)
variable_pool = VariablePool.from_bootstrap()
variable_pool = VariablePool()
add_variables_to_pool(
variable_pool,
build_bootstrap_variables(
@@ -1659,7 +1659,7 @@ def _setup_variable_pool(
system_variable = default_system_variables()
# init variable pool
variable_pool = VariablePool.from_bootstrap()
variable_pool = VariablePool()
add_variables_to_pool(
variable_pool,
build_bootstrap_variables(
@@ -71,7 +71,7 @@ def test_node_integration_minimal_stream(mocker):
node = DatasourceNode(
node_id="n",
data=DatasourceNodeData(
config=DatasourceNodeData(
type="datasource",
version="1",
title="Datasource",
@@ -4,7 +4,7 @@ from core.app.entities.app_invoke_entities import ModelConfigWithCredentialsEnti
from core.entities.provider_configuration import ProviderConfiguration, ProviderModelBundle
from core.entities.provider_entities import CustomConfiguration, CustomProviderConfiguration, SystemConfiguration
from core.model_manager import ModelInstance
from core.plugin.impl.model_runtime_factory import create_model_type_instance, create_plugin_model_provider_factory
from core.plugin.impl.model_runtime_factory import create_plugin_model_provider_factory
from graphon.model_runtime.entities.model_entities import ModelType
from models.provider import ProviderType
@@ -16,11 +16,7 @@ def get_mocked_fetch_model_config(
credentials: dict,
):
model_provider_factory = create_plugin_model_provider_factory(tenant_id="9d2074fc-6f86-45a9-b09d-6ecc63b9056b")
model_type_instance = create_model_type_instance(
factory=model_provider_factory,
provider=provider,
model_type=ModelType.LLM,
)
model_type_instance = model_provider_factory.get_model_type_instance(provider, ModelType.LLM)
provider_model_bundle = ProviderModelBundle(
configuration=ProviderConfiguration(
tenant_id="1",
@@ -45,7 +45,7 @@ def init_code_node(code_config: dict):
)
# construct variable pool
variable_pool = VariablePool.from_bootstrap(
variable_pool = VariablePool(
system_variables=build_system_variables(user_id="aaa", files=[]),
user_inputs={},
environment_variables=[],
@@ -66,7 +66,7 @@ def init_code_node(code_config: dict):
node = CodeNode(
node_id=str(uuid.uuid4()),
data=CodeNodeData.model_validate(code_config["data"]),
config=CodeNodeData.model_validate(code_config["data"]),
graph_init_params=init_params,
graph_runtime_state=graph_runtime_state,
code_executor=node_factory._code_executor,
@@ -55,7 +55,7 @@ def init_http_node(config: dict):
)
# construct variable pool
variable_pool = VariablePool.from_bootstrap(
variable_pool = VariablePool(
system_variables=build_system_variables(user_id="aaa", files=[]),
user_inputs={},
environment_variables=[],
@@ -76,7 +76,7 @@ def init_http_node(config: dict):
node = HttpRequestNode(
node_id=str(uuid.uuid4()),
data=HttpRequestNodeData.model_validate(config["data"]),
config=HttpRequestNodeData.model_validate(config["data"]),
graph_init_params=init_params,
graph_runtime_state=graph_runtime_state,
http_request_config=HTTP_REQUEST_CONFIG,
@@ -204,7 +204,7 @@ def test_custom_auth_with_empty_api_key_raises_error(setup_http_mock):
from graphon.runtime import VariablePool
# Create variable pool
variable_pool = VariablePool.from_bootstrap(
variable_pool = VariablePool(
system_variables=build_system_variables(user_id="test", files=[]),
user_inputs={},
environment_variables=[],
@@ -702,7 +702,7 @@ def test_nested_object_variable_selector(setup_http_mock):
)
# Create independent variable pool for this test only
variable_pool = VariablePool.from_bootstrap(
variable_pool = VariablePool(
system_variables=build_system_variables(user_id="aaa", files=[]),
user_inputs={},
environment_variables=[],
@@ -724,7 +724,7 @@ def test_nested_object_variable_selector(setup_http_mock):
node = HttpRequestNode(
node_id=str(uuid.uuid4()),
data=HttpRequestNodeData.model_validate(graph_config["nodes"][1]["data"]),
config=HttpRequestNodeData.model_validate(graph_config["nodes"][1]["data"]),
graph_init_params=init_params,
graph_runtime_state=graph_runtime_state,
http_request_config=HTTP_REQUEST_CONFIG,
@@ -53,7 +53,7 @@ def init_llm_node(config: dict) -> LLMNode:
)
# construct variable pool
variable_pool = VariablePool.from_bootstrap(
variable_pool = VariablePool(
system_variables=build_system_variables(
user_id="aaa",
app_id=app_id,
@@ -77,7 +77,7 @@ def init_llm_node(config: dict) -> LLMNode:
node = LLMNode(
node_id=str(uuid.uuid4()),
data=LLMNodeData.model_validate(config["data"]),
config=LLMNodeData.model_validate(config["data"]),
graph_init_params=init_params,
graph_runtime_state=graph_runtime_state,
credentials_provider=MagicMock(spec=CredentialsProvider),
@@ -56,7 +56,7 @@ def init_parameter_extractor_node(config: dict, memory=None):
)
# construct variable pool
variable_pool = VariablePool.from_bootstrap(
variable_pool = VariablePool(
system_variables=build_system_variables(
user_id="aaa", files=[], query="what's the weather in SF", conversation_id="abababa"
),
@@ -71,7 +71,7 @@ def init_parameter_extractor_node(config: dict, memory=None):
node = ParameterExtractorNode(
node_id=str(uuid.uuid4()),
data=ParameterExtractorNodeData.model_validate(config["data"]),
config=ParameterExtractorNodeData.model_validate(config["data"]),
graph_init_params=init_params,
graph_runtime_state=graph_runtime_state,
credentials_provider=MagicMock(spec=CredentialsProvider),
@@ -66,7 +66,7 @@ def test_execute_template_transform():
)
# construct variable pool
variable_pool = VariablePool.from_bootstrap(
variable_pool = VariablePool(
system_variables=build_system_variables(user_id="aaa", files=[]),
user_inputs={},
environment_variables=[],
@@ -88,7 +88,7 @@ def test_execute_template_transform():
node = TemplateTransformNode(
node_id=str(uuid.uuid4()),
data=TemplateTransformNodeData.model_validate(config["data"]),
config=TemplateTransformNodeData.model_validate(config["data"]),
graph_init_params=init_params,
graph_runtime_state=graph_runtime_state,
jinja2_template_renderer=_SimpleJinja2Renderer(),
@@ -41,7 +41,7 @@ def init_tool_node(config: dict):
)
# construct variable pool
variable_pool = VariablePool.from_bootstrap(
variable_pool = VariablePool(
system_variables=build_system_variables(user_id="aaa", files=[]),
user_inputs={},
environment_variables=[],
@@ -62,7 +62,7 @@ def init_tool_node(config: dict):
node = ToolNode(
node_id=str(uuid.uuid4()),
data=ToolNodeData.model_validate(config["data"]),
config=ToolNodeData.model_validate(config["data"]),
graph_init_params=init_params,
graph_runtime_state=graph_runtime_state,
tool_file_manager_factory=tool_file_manager_factory,
@@ -433,7 +433,7 @@ def flask_app_with_containers(set_up_containers_and_env) -> Flask:
@pytest.fixture
def flask_req_ctx_with_containers(flask_app_with_containers) -> Generator[None, None, None]:
def flask_req_ctx_with_containers(flask_app_with_containers: Flask) -> Generator[None, None, None]:
"""
Request context fixture for containerized Flask application.
@@ -454,7 +454,7 @@ def flask_req_ctx_with_containers(flask_app_with_containers) -> Generator[None,
@pytest.fixture
def test_client_with_containers(flask_app_with_containers) -> Generator[FlaskClient, None, None]:
def test_client_with_containers(flask_app_with_containers: Flask) -> Generator[FlaskClient, None, None]:
"""
Test client fixture for containerized Flask application.
@@ -475,7 +475,7 @@ def test_client_with_containers(flask_app_with_containers) -> Generator[FlaskCli
@pytest.fixture
def db_session_with_containers(flask_app_with_containers) -> Generator[Session, None, None]:
def db_session_with_containers(flask_app_with_containers: Flask) -> Generator[Session, None, None]:
"""
Database session fixture for containerized testing.
@@ -7,6 +7,7 @@ from types import SimpleNamespace
from unittest.mock import MagicMock, patch
import pytest
from flask import Flask
from pydantic import ValidationError
from werkzeug.exceptions import BadRequest, NotFound
@@ -69,7 +70,7 @@ def _unwrap(func):
class TestCompletionEndpoints:
@pytest.fixture
def app(self, flask_app_with_containers):
def app(self, flask_app_with_containers: Flask):
return flask_app_with_containers
def test_completion_create_payload(self):
@@ -86,7 +87,7 @@ class TestCompletionEndpoints:
)
assert payload.query == "hi"
def test_completion_api_success(self, app, monkeypatch):
def test_completion_api_success(self, app: Flask, monkeypatch: pytest.MonkeyPatch):
api = completion_module.CompletionMessageApi()
method = _unwrap(api.post)
@@ -116,7 +117,7 @@ class TestCompletionEndpoints:
assert resp == {"result": {"text": "ok"}}
def test_completion_api_conversation_not_exists(self, app, monkeypatch):
def test_completion_api_conversation_not_exists(self, app: Flask, monkeypatch: pytest.MonkeyPatch):
api = completion_module.CompletionMessageApi()
method = _unwrap(api.post)
@@ -142,7 +143,7 @@ class TestCompletionEndpoints:
with pytest.raises(NotFound):
method(app_model=MagicMock(id="app-1"))
def test_completion_api_provider_not_initialized(self, app, monkeypatch):
def test_completion_api_provider_not_initialized(self, app: Flask, monkeypatch: pytest.MonkeyPatch):
api = completion_module.CompletionMessageApi()
method = _unwrap(api.post)
@@ -166,7 +167,7 @@ class TestCompletionEndpoints:
with pytest.raises(completion_module.ProviderNotInitializeError):
method(app_model=MagicMock(id="app-1"))
def test_completion_api_quota_exceeded(self, app, monkeypatch):
def test_completion_api_quota_exceeded(self, app: Flask, monkeypatch: pytest.MonkeyPatch):
api = completion_module.CompletionMessageApi()
method = _unwrap(api.post)
@@ -193,10 +194,10 @@ class TestCompletionEndpoints:
class TestAppEndpoints:
@pytest.fixture
def app(self, flask_app_with_containers):
def app(self, flask_app_with_containers: Flask):
return flask_app_with_containers
def test_app_put_should_preserve_icon_type_when_payload_omits_it(self, app, monkeypatch):
def test_app_put_should_preserve_icon_type_when_payload_omits_it(self, app: Flask, monkeypatch: pytest.MonkeyPatch):
api = app_module.AppApi()
method = _unwrap(api.put)
payload = {
@@ -234,7 +235,7 @@ class TestAppEndpoints:
}
)
def test_app_icon_post_should_forward_icon_type(self, app, monkeypatch):
def test_app_icon_post_should_forward_icon_type(self, app: Flask, monkeypatch: pytest.MonkeyPatch):
api = app_module.AppIconApi()
method = _unwrap(api.post)
payload = {
@@ -266,7 +267,7 @@ class TestAppEndpoints:
class TestOpsTraceEndpoints:
@pytest.fixture
def app(self, flask_app_with_containers):
def app(self, flask_app_with_containers: Flask):
return flask_app_with_containers
def test_ops_trace_query_basic(self):
@@ -277,7 +278,7 @@ class TestOpsTraceEndpoints:
payload = TraceConfigPayload(tracing_provider="langfuse", tracing_config={"api_key": "k"})
assert payload.tracing_config["api_key"] == "k"
def test_trace_app_config_get_empty(self, app, monkeypatch):
def test_trace_app_config_get_empty(self, app: Flask, monkeypatch: pytest.MonkeyPatch):
api = ops_trace_module.TraceAppConfigApi()
method = _unwrap(api.get)
@@ -292,7 +293,7 @@ class TestOpsTraceEndpoints:
assert result == {"has_not_configured": True}
def test_trace_app_config_post_invalid(self, app, monkeypatch):
def test_trace_app_config_post_invalid(self, app: Flask, monkeypatch: pytest.MonkeyPatch):
api = ops_trace_module.TraceAppConfigApi()
method = _unwrap(api.post)
@@ -309,7 +310,7 @@ class TestOpsTraceEndpoints:
with pytest.raises(BadRequest):
method(app_id="app-1")
def test_trace_app_config_delete_not_found(self, app, monkeypatch):
def test_trace_app_config_delete_not_found(self, app: Flask, monkeypatch: pytest.MonkeyPatch):
api = ops_trace_module.TraceAppConfigApi()
method = _unwrap(api.delete)
@@ -326,7 +327,7 @@ class TestOpsTraceEndpoints:
class TestSiteEndpoints:
@pytest.fixture
def app(self, flask_app_with_containers):
def app(self, flask_app_with_containers: Flask):
return flask_app_with_containers
def test_site_response_structure(self):
@@ -337,7 +338,7 @@ class TestSiteEndpoints:
payload = AppSiteUpdatePayload(default_language="en-US")
assert payload.default_language == "en-US"
def test_app_site_update_post(self, app, monkeypatch):
def test_app_site_update_post(self, app: Flask, monkeypatch: pytest.MonkeyPatch):
api = site_module.AppSite()
method = _unwrap(api.post)
@@ -375,7 +376,7 @@ class TestSiteEndpoints:
assert isinstance(result, dict)
assert result["title"] == "My Site"
def test_app_site_access_token_reset(self, app, monkeypatch):
def test_app_site_access_token_reset(self, app: Flask, monkeypatch: pytest.MonkeyPatch):
api = site_module.AppSiteAccessTokenReset()
method = _unwrap(api.post)
@@ -427,7 +428,7 @@ class TestWorkflowEndpoints:
class TestWorkflowAppLogEndpoints:
@pytest.fixture
def app(self, flask_app_with_containers):
def app(self, flask_app_with_containers: Flask):
return flask_app_with_containers
def test_workflow_app_log_query(self):
@@ -438,7 +439,7 @@ class TestWorkflowAppLogEndpoints:
query = WorkflowAppLogQuery(detail="true")
assert query.detail is True
def test_workflow_app_log_api_get(self, app, monkeypatch):
def test_workflow_app_log_api_get(self, app: Flask, monkeypatch: pytest.MonkeyPatch):
api = workflow_app_log_module.WorkflowAppLogApi()
method = _unwrap(api.get)
@@ -477,14 +478,14 @@ class TestWorkflowAppLogEndpoints:
class TestWorkflowDraftVariableEndpoints:
@pytest.fixture
def app(self, flask_app_with_containers):
def app(self, flask_app_with_containers: Flask):
return flask_app_with_containers
def test_workflow_variable_creation(self):
payload = WorkflowDraftVariableUpdatePayload(name="var1", value="test")
assert payload.name == "var1"
def test_workflow_variable_collection_get(self, app, monkeypatch):
def test_workflow_variable_collection_get(self, app: Flask, monkeypatch: pytest.MonkeyPatch):
api = workflow_draft_variable_module.WorkflowVariableCollectionApi()
method = _unwrap(api.get)
@@ -529,7 +530,7 @@ class TestWorkflowDraftVariableEndpoints:
class TestWorkflowStatisticEndpoints:
@pytest.fixture
def app(self, flask_app_with_containers):
def app(self, flask_app_with_containers: Flask):
return flask_app_with_containers
def test_workflow_statistic_time_range(self):
@@ -541,7 +542,7 @@ class TestWorkflowStatisticEndpoints:
assert query.start is None
assert query.end is None
def test_workflow_daily_runs_statistic(self, app, monkeypatch):
def test_workflow_daily_runs_statistic(self, app: Flask, monkeypatch: pytest.MonkeyPatch):
monkeypatch.setattr(workflow_statistic_module, "db", SimpleNamespace(engine=MagicMock()))
monkeypatch.setattr(
workflow_statistic_module.DifyAPIRepositoryFactory,
@@ -567,7 +568,7 @@ class TestWorkflowStatisticEndpoints:
assert response.get_json() == {"data": [{"date": "2024-01-01"}]}
def test_workflow_daily_terminals_statistic(self, app, monkeypatch):
def test_workflow_daily_terminals_statistic(self, app: Flask, monkeypatch: pytest.MonkeyPatch):
monkeypatch.setattr(workflow_statistic_module, "db", SimpleNamespace(engine=MagicMock()))
monkeypatch.setattr(
workflow_statistic_module.DifyAPIRepositoryFactory,
@@ -598,7 +599,7 @@ class TestWorkflowStatisticEndpoints:
class TestWorkflowTriggerEndpoints:
@pytest.fixture
def app(self, flask_app_with_containers):
def app(self, flask_app_with_containers: Flask):
return flask_app_with_containers
def test_webhook_trigger_payload(self):
@@ -608,7 +609,7 @@ class TestWorkflowTriggerEndpoints:
enable_payload = ParserEnable(trigger_id="trigger-1", enable_trigger=True)
assert enable_payload.enable_trigger is True
def test_webhook_trigger_api_get(self, app, monkeypatch):
def test_webhook_trigger_api_get(self, app: Flask, monkeypatch: pytest.MonkeyPatch):
api = workflow_trigger_module.WebhookTriggerApi()
method = _unwrap(api.get)
@@ -6,6 +6,7 @@ from types import SimpleNamespace
from unittest.mock import MagicMock
import pytest
from flask import Flask
from controllers.console.app import app_import as app_import_module
from services.app_dsl_service import ImportStatus
@@ -36,10 +37,10 @@ def _install_features(monkeypatch: pytest.MonkeyPatch, enabled: bool) -> None:
class TestAppImportApi:
@pytest.fixture
def app(self, flask_app_with_containers):
def app(self, flask_app_with_containers: Flask):
return flask_app_with_containers
def test_import_post_returns_failed_status(self, app, monkeypatch: pytest.MonkeyPatch) -> None:
def test_import_post_returns_failed_status(self, app: Flask, monkeypatch: pytest.MonkeyPatch) -> None:
api = app_import_module.AppImportApi()
method = _unwrap(api.post)
@@ -57,7 +58,7 @@ class TestAppImportApi:
assert status == 400
assert response["status"] == ImportStatus.FAILED
def test_import_post_returns_pending_status(self, app, monkeypatch: pytest.MonkeyPatch) -> None:
def test_import_post_returns_pending_status(self, app: Flask, monkeypatch: pytest.MonkeyPatch) -> None:
api = app_import_module.AppImportApi()
method = _unwrap(api.post)
@@ -75,7 +76,7 @@ class TestAppImportApi:
assert status == 202
assert response["status"] == ImportStatus.PENDING
def test_import_post_updates_webapp_auth_when_enabled(self, app, monkeypatch: pytest.MonkeyPatch) -> None:
def test_import_post_updates_webapp_auth_when_enabled(self, app: Flask, monkeypatch: pytest.MonkeyPatch) -> None:
api = app_import_module.AppImportApi()
method = _unwrap(api.post)
@@ -96,7 +97,7 @@ class TestAppImportApi:
assert status == 200
assert response["status"] == ImportStatus.COMPLETED
def test_import_post_commits_session_on_success(self, app, monkeypatch: pytest.MonkeyPatch) -> None:
def test_import_post_commits_session_on_success(self, app: Flask, monkeypatch: pytest.MonkeyPatch) -> None:
api = app_import_module.AppImportApi()
method = _unwrap(api.post)
@@ -121,7 +122,7 @@ class TestAppImportApi:
assert status == 200
assert response["status"] == ImportStatus.COMPLETED
def test_import_post_rolls_back_session_on_failure(self, app, monkeypatch: pytest.MonkeyPatch) -> None:
def test_import_post_rolls_back_session_on_failure(self, app: Flask, monkeypatch: pytest.MonkeyPatch) -> None:
api = app_import_module.AppImportApi()
method = _unwrap(api.post)
@@ -149,10 +150,10 @@ class TestAppImportApi:
class TestAppImportConfirmApi:
@pytest.fixture
def app(self, flask_app_with_containers):
def app(self, flask_app_with_containers: Flask):
return flask_app_with_containers
def test_import_confirm_returns_failed_status(self, app, monkeypatch: pytest.MonkeyPatch) -> None:
def test_import_confirm_returns_failed_status(self, app: Flask, monkeypatch: pytest.MonkeyPatch) -> None:
api = app_import_module.AppImportConfirmApi()
method = _unwrap(api.post)
@@ -172,10 +173,10 @@ class TestAppImportConfirmApi:
class TestAppImportCheckDependenciesApi:
@pytest.fixture
def app(self, flask_app_with_containers):
def app(self, flask_app_with_containers: Flask):
return flask_app_with_containers
def test_import_check_dependencies_returns_result(self, app, monkeypatch: pytest.MonkeyPatch) -> None:
def test_import_check_dependencies_returns_result(self, app: Flask, monkeypatch: pytest.MonkeyPatch) -> None:
api = app_import_module.AppImportCheckDependenciesApi()
method = _unwrap(api.get)
@@ -6,6 +6,7 @@ from types import SimpleNamespace
from unittest.mock import MagicMock, patch
import pytest
from flask import Flask
from controllers.console.auth.email_register import (
EmailRegisterCheckApi,
@@ -16,7 +17,7 @@ from services.account_service import AccountService
@pytest.fixture
def app(flask_app_with_containers):
def app(flask_app_with_containers: Flask):
return flask_app_with_containers
@@ -33,7 +34,7 @@ class TestEmailRegisterSendEmailApi:
mock_is_freeze,
mock_send_mail,
mock_get_account,
app,
app: Flask,
):
mock_send_mail.return_value = "token-123"
mock_is_freeze.return_value = False
@@ -75,7 +76,7 @@ class TestEmailRegisterCheckApi:
mock_revoke,
mock_generate_token,
mock_reset_rate,
app,
app: Flask,
):
mock_rate_limit_check.return_value = False
mock_get_data.return_value = {"email": "User@Example.com", "code": "4321"}
@@ -120,7 +121,7 @@ class TestEmailRegisterResetApi:
mock_create_account,
mock_login,
mock_reset_login_rate,
app,
app: Flask,
):
mock_get_data.return_value = {"phase": "register", "email": "Invitee@Example.com"}
mock_create_account.return_value = MagicMock()
@@ -6,6 +6,7 @@ from types import SimpleNamespace
from unittest.mock import MagicMock, patch
import pytest
from flask import Flask
from controllers.console.auth.forgot_password import (
ForgotPasswordCheckApi,
@@ -16,7 +17,7 @@ from services.account_service import AccountService
@pytest.fixture
def app(flask_app_with_containers):
def app(flask_app_with_containers: Flask):
return flask_app_with_containers
@@ -31,7 +32,7 @@ class TestForgotPasswordSendEmailApi:
mock_is_ip_limit,
mock_send_email,
mock_get_account,
app,
app: Flask,
):
mock_account = MagicMock()
mock_get_account.return_value = mock_account
@@ -80,7 +81,7 @@ class TestForgotPasswordCheckApi:
mock_revoke_token,
mock_generate_token,
mock_reset_rate,
app,
app: Flask,
):
mock_rate_limit_check.return_value = False
mock_get_data.return_value = {"email": "Admin@Example.com", "code": "4321"}
@@ -123,7 +124,7 @@ class TestForgotPasswordResetApi:
mock_db,
mock_get_account,
mock_update_account,
app,
app: Flask,
):
mock_get_reset_data.return_value = {"phase": "reset", "email": "User@Example.com"}
mock_account = MagicMock()
@@ -5,6 +5,7 @@ from __future__ import annotations
from unittest.mock import MagicMock, patch
import pytest
from flask import Flask
from controllers.console.auth.oauth import (
OAuthCallback,
@@ -21,7 +22,7 @@ from services.errors.account import AccountRegisterError
class TestGetOAuthProviders:
@pytest.fixture
def app(self, flask_app_with_containers):
def app(self, flask_app_with_containers: Flask):
return flask_app_with_containers
@pytest.mark.parametrize(
@@ -65,7 +66,7 @@ class TestOAuthLogin:
return OAuthLogin()
@pytest.fixture
def app(self, flask_app_with_containers):
def app(self, flask_app_with_containers: Flask):
return flask_app_with_containers
@pytest.fixture
@@ -89,7 +90,7 @@ class TestOAuthLogin:
mock_redirect,
mock_get_providers,
resource,
app,
app: Flask,
mock_oauth_provider,
invite_token,
expected_token,
@@ -130,7 +131,7 @@ class TestOAuthCallback:
return OAuthCallback()
@pytest.fixture
def app(self, flask_app_with_containers):
def app(self, flask_app_with_containers: Flask):
return flask_app_with_containers
@pytest.fixture
@@ -164,7 +165,7 @@ class TestOAuthCallback:
mock_get_providers,
mock_config,
resource,
app,
app: Flask,
oauth_setup,
):
mock_config.CONSOLE_WEB_URL = "http://localhost:3000"
@@ -217,7 +218,7 @@ class TestOAuthCallback:
mock_get_providers,
mock_config,
resource,
app,
app: Flask,
oauth_setup,
):
mock_config.CONSOLE_WEB_URL = "http://localhost:3000"
@@ -261,7 +262,7 @@ class TestOAuthCallback:
mock_tenant_service,
mock_account_service,
resource,
app,
app: Flask,
oauth_setup,
account_status,
expected_redirect,
@@ -300,7 +301,7 @@ class TestOAuthCallback:
mock_get_providers,
mock_config,
resource,
app,
app: Flask,
oauth_setup,
):
mock_get_providers.return_value = {"github": oauth_setup["provider"]}
@@ -336,7 +337,7 @@ class TestOAuthCallback:
mock_get_providers,
mock_config,
resource,
app,
app: Flask,
oauth_setup,
):
"""Defensive test for CLOSED account status handling in OAuth callback.
@@ -394,7 +395,7 @@ class TestOAuthCallback:
class TestAccountGeneration:
@pytest.fixture
def app(self, flask_app_with_containers):
def app(self, flask_app_with_containers: Flask):
return flask_app_with_containers
@pytest.fixture
@@ -465,7 +466,7 @@ class TestAccountGeneration:
mock_register_service,
mock_feature_service,
mock_get_account,
app,
app: Flask,
user_info,
mock_account,
allow_register,
@@ -504,7 +505,7 @@ class TestAccountGeneration:
mock_register_service,
mock_feature_service,
mock_get_account,
app,
app: Flask,
):
user_info = OAuthUserInfo(id="123", name="Test User", email="Upper@Example.com")
mock_feature_service.get_system_features.return_value.is_allow_register = True
@@ -529,7 +530,7 @@ class TestAccountGeneration:
mock_feature_service,
mock_tenant_service,
mock_get_account,
app,
app: Flask,
user_info,
mock_account,
):
@@ -5,6 +5,7 @@ from __future__ import annotations
from unittest.mock import MagicMock, patch
import pytest
from flask import Flask
from controllers.console.auth.error import (
EmailCodeError,
@@ -25,7 +26,7 @@ class TestForgotPasswordSendEmailApi:
"""Test cases for sending password reset emails."""
@pytest.fixture
def app(self, flask_app_with_containers):
def app(self, flask_app_with_containers: Flask):
return flask_app_with_containers
@pytest.fixture
@@ -46,7 +47,7 @@ class TestForgotPasswordSendEmailApi:
mock_send_email,
mock_get_account,
mock_is_ip_limit,
app,
app: Flask,
mock_account,
):
# Arrange
@@ -68,7 +69,7 @@ class TestForgotPasswordSendEmailApi:
mock_send_email.assert_called_once()
@patch("controllers.console.auth.forgot_password.AccountService.is_email_send_ip_limit")
def test_send_reset_email_ip_rate_limited(self, mock_is_ip_limit, app):
def test_send_reset_email_ip_rate_limited(self, mock_is_ip_limit, app: Flask):
"""
Test password reset email blocked by IP rate limit.
@@ -104,7 +105,7 @@ class TestForgotPasswordSendEmailApi:
mock_send_email,
mock_get_account,
mock_is_ip_limit,
app,
app: Flask,
mock_account,
language_input,
expected_language,
@@ -138,7 +139,7 @@ class TestForgotPasswordCheckApi:
"""Test cases for verifying password reset codes."""
@pytest.fixture
def app(self, flask_app_with_containers):
def app(self, flask_app_with_containers: Flask):
return flask_app_with_containers
@patch("controllers.console.auth.forgot_password.AccountService.is_forgot_password_error_rate_limit")
@@ -153,7 +154,7 @@ class TestForgotPasswordCheckApi:
mock_revoke_token,
mock_get_data,
mock_is_rate_limit,
app,
app: Flask,
):
"""
Test successful verification code validation.
@@ -200,7 +201,7 @@ class TestForgotPasswordCheckApi:
mock_revoke_token,
mock_get_data,
mock_is_rate_limit,
app,
app: Flask,
):
mock_is_rate_limit.return_value = False
mock_get_data.return_value = {"email": "User@Example.com", "code": "999888"}
@@ -221,7 +222,7 @@ class TestForgotPasswordCheckApi:
mock_reset_rate_limit.assert_called_once_with("user@example.com")
@patch("controllers.console.auth.forgot_password.AccountService.is_forgot_password_error_rate_limit")
def test_verify_code_rate_limited(self, mock_is_rate_limit, app):
def test_verify_code_rate_limited(self, mock_is_rate_limit, app: Flask):
"""
Test code verification blocked by rate limit.
@@ -244,7 +245,7 @@ class TestForgotPasswordCheckApi:
@patch("controllers.console.auth.forgot_password.AccountService.is_forgot_password_error_rate_limit")
@patch("controllers.console.auth.forgot_password.AccountService.get_reset_password_data")
def test_verify_code_invalid_token(self, mock_get_data, mock_is_rate_limit, app):
def test_verify_code_invalid_token(self, mock_get_data, mock_is_rate_limit, app: Flask):
"""
Test code verification with invalid token.
@@ -267,7 +268,7 @@ class TestForgotPasswordCheckApi:
@patch("controllers.console.auth.forgot_password.AccountService.is_forgot_password_error_rate_limit")
@patch("controllers.console.auth.forgot_password.AccountService.get_reset_password_data")
def test_verify_code_email_mismatch(self, mock_get_data, mock_is_rate_limit, app):
def test_verify_code_email_mismatch(self, mock_get_data, mock_is_rate_limit, app: Flask):
"""
Test code verification with mismatched email.
@@ -292,7 +293,7 @@ class TestForgotPasswordCheckApi:
@patch("controllers.console.auth.forgot_password.AccountService.is_forgot_password_error_rate_limit")
@patch("controllers.console.auth.forgot_password.AccountService.get_reset_password_data")
@patch("controllers.console.auth.forgot_password.AccountService.add_forgot_password_error_rate_limit")
def test_verify_code_wrong_code(self, mock_add_rate_limit, mock_get_data, mock_is_rate_limit, app):
def test_verify_code_wrong_code(self, mock_add_rate_limit, mock_get_data, mock_is_rate_limit, app: Flask):
"""
Test code verification with incorrect code.
@@ -321,7 +322,7 @@ class TestForgotPasswordResetApi:
"""Test cases for resetting password with verified token."""
@pytest.fixture
def app(self, flask_app_with_containers):
def app(self, flask_app_with_containers: Flask):
return flask_app_with_containers
@pytest.fixture
@@ -344,7 +345,7 @@ class TestForgotPasswordResetApi:
mock_get_account,
mock_revoke_token,
mock_get_data,
app,
app: Flask,
mock_account,
):
"""
@@ -375,7 +376,7 @@ class TestForgotPasswordResetApi:
mock_revoke_token.assert_called_once_with("valid_token")
@patch("controllers.console.auth.forgot_password.AccountService.get_reset_password_data")
def test_reset_password_mismatch(self, mock_get_data, app):
def test_reset_password_mismatch(self, mock_get_data, app: Flask):
"""
Test password reset with mismatched passwords.
@@ -397,7 +398,7 @@ class TestForgotPasswordResetApi:
api.post()
@patch("controllers.console.auth.forgot_password.AccountService.get_reset_password_data")
def test_reset_password_invalid_token(self, mock_get_data, app):
def test_reset_password_invalid_token(self, mock_get_data, app: Flask):
"""
Test password reset with invalid token.
@@ -418,7 +419,7 @@ class TestForgotPasswordResetApi:
api.post()
@patch("controllers.console.auth.forgot_password.AccountService.get_reset_password_data")
def test_reset_password_wrong_phase(self, mock_get_data, app):
def test_reset_password_wrong_phase(self, mock_get_data, app: Flask):
"""
Test password reset with token not in reset phase.
@@ -442,7 +443,7 @@ class TestForgotPasswordResetApi:
@patch("controllers.console.auth.forgot_password.AccountService.get_reset_password_data")
@patch("controllers.console.auth.forgot_password.AccountService.revoke_reset_password_token")
@patch("controllers.console.auth.forgot_password.AccountService.get_account_by_email_with_case_fallback")
def test_reset_password_account_not_found(self, mock_get_account, mock_revoke_token, mock_get_data, app):
def test_reset_password_account_not_found(self, mock_get_account, mock_revoke_token, mock_get_data, app: Flask):
"""
Test password reset for non-existent account.
@@ -6,6 +6,7 @@ from unittest.mock import MagicMock, patch
from uuid import uuid4
import pytest
from flask import Flask
from sqlalchemy.orm import Session
from controllers.console import console_ns
@@ -26,10 +27,10 @@ def unwrap(func):
class TestPipelineTemplateListApi:
@pytest.fixture
def app(self, flask_app_with_containers):
def app(self, flask_app_with_containers: Flask):
return flask_app_with_containers
def test_get_success(self, app):
def test_get_success(self, app: Flask):
api = PipelineTemplateListApi()
method = unwrap(api.get)
@@ -50,10 +51,10 @@ class TestPipelineTemplateListApi:
class TestPipelineTemplateDetailApi:
@pytest.fixture
def app(self, flask_app_with_containers):
def app(self, flask_app_with_containers: Flask):
return flask_app_with_containers
def test_get_success(self, app):
def test_get_success(self, app: Flask):
api = PipelineTemplateDetailApi()
method = unwrap(api.get)
@@ -74,7 +75,7 @@ class TestPipelineTemplateDetailApi:
assert status == 200
assert response == template
def test_get_returns_404_when_template_not_found(self, app):
def test_get_returns_404_when_template_not_found(self, app: Flask):
api = PipelineTemplateDetailApi()
method = unwrap(api.get)
@@ -93,7 +94,7 @@ class TestPipelineTemplateDetailApi:
assert status == 404
assert "error" in response
def test_get_returns_404_for_customized_type_not_found(self, app):
def test_get_returns_404_for_customized_type_not_found(self, app: Flask):
api = PipelineTemplateDetailApi()
method = unwrap(api.get)
@@ -115,10 +116,10 @@ class TestPipelineTemplateDetailApi:
class TestCustomizedPipelineTemplateApi:
@pytest.fixture
def app(self, flask_app_with_containers):
def app(self, flask_app_with_containers: Flask):
return flask_app_with_containers
def test_patch_success(self, app):
def test_patch_success(self, app: Flask):
api = CustomizedPipelineTemplateApi()
method = unwrap(api.patch)
@@ -140,7 +141,7 @@ class TestCustomizedPipelineTemplateApi:
update_mock.assert_called_once()
assert response == 200
def test_delete_success(self, app):
def test_delete_success(self, app: Flask):
api = CustomizedPipelineTemplateApi()
method = unwrap(api.delete)
@@ -155,7 +156,7 @@ class TestCustomizedPipelineTemplateApi:
delete_mock.assert_called_once_with("tpl-1")
assert response == 200
def test_post_success(self, app, db_session_with_containers: Session):
def test_post_success(self, app: Flask, db_session_with_containers: Session):
api = CustomizedPipelineTemplateApi()
method = unwrap(api.post)
@@ -182,7 +183,7 @@ class TestCustomizedPipelineTemplateApi:
assert status == 200
assert response == {"data": "yaml-data"}
def test_post_template_not_found(self, app):
def test_post_template_not_found(self, app: Flask):
api = CustomizedPipelineTemplateApi()
method = unwrap(api.post)
@@ -193,10 +194,10 @@ class TestCustomizedPipelineTemplateApi:
class TestPublishCustomizedPipelineTemplateApi:
@pytest.fixture
def app(self, flask_app_with_containers):
def app(self, flask_app_with_containers: Flask):
return flask_app_with_containers
def test_post_success(self, app):
def test_post_success(self, app: Flask):
api = PublishCustomizedPipelineTemplateApi()
method = unwrap(api.post)
@@ -5,6 +5,7 @@ from __future__ import annotations
from unittest.mock import MagicMock, patch
import pytest
from flask import Flask
from werkzeug.exceptions import Forbidden
import services
@@ -24,13 +25,13 @@ def unwrap(func):
class TestCreateRagPipelineDatasetApi:
@pytest.fixture
def app(self, flask_app_with_containers):
def app(self, flask_app_with_containers: Flask):
return flask_app_with_containers
def _valid_payload(self):
return {"yaml_content": "name: test"}
def test_post_success(self, app):
def test_post_success(self, app: Flask):
api = CreateRagPipelineDatasetApi()
method = unwrap(api.post)
@@ -58,7 +59,7 @@ class TestCreateRagPipelineDatasetApi:
assert status == 201
assert response == import_info
def test_post_forbidden_non_editor(self, app):
def test_post_forbidden_non_editor(self, app: Flask):
api = CreateRagPipelineDatasetApi()
method = unwrap(api.post)
@@ -76,7 +77,7 @@ class TestCreateRagPipelineDatasetApi:
with pytest.raises(Forbidden):
method(api)
def test_post_dataset_name_duplicate(self, app):
def test_post_dataset_name_duplicate(self, app: Flask):
api = CreateRagPipelineDatasetApi()
method = unwrap(api.post)
@@ -101,7 +102,7 @@ class TestCreateRagPipelineDatasetApi:
with pytest.raises(DatasetNameDuplicateError):
method(api)
def test_post_invalid_payload(self, app):
def test_post_invalid_payload(self, app: Flask):
api = CreateRagPipelineDatasetApi()
method = unwrap(api.post)
@@ -122,10 +123,10 @@ class TestCreateRagPipelineDatasetApi:
class TestCreateEmptyRagPipelineDatasetApi:
@pytest.fixture
def app(self, flask_app_with_containers):
def app(self, flask_app_with_containers: Flask):
return flask_app_with_containers
def test_post_success(self, app):
def test_post_success(self, app: Flask):
api = CreateEmptyRagPipelineDatasetApi()
method = unwrap(api.post)
@@ -152,7 +153,7 @@ class TestCreateEmptyRagPipelineDatasetApi:
assert status == 201
assert response == {"id": "ds-1"}
def test_post_forbidden_non_editor(self, app):
def test_post_forbidden_non_editor(self, app: Flask):
api = CreateEmptyRagPipelineDatasetApi()
method = unwrap(api.post)
@@ -5,6 +5,7 @@ from __future__ import annotations
from unittest.mock import MagicMock, patch
import pytest
from flask import Flask
from controllers.console import console_ns
from controllers.console.datasets.rag_pipeline.rag_pipeline_import import (
@@ -25,7 +26,7 @@ def unwrap(func):
class TestRagPipelineImportApi:
@pytest.fixture
def app(self, flask_app_with_containers):
def app(self, flask_app_with_containers: Flask):
return flask_app_with_containers
def _payload(self, mode="create"):
@@ -35,7 +36,7 @@ class TestRagPipelineImportApi:
"name": "Test",
}
def test_post_success_200(self, app):
def test_post_success_200(self, app: Flask):
api = RagPipelineImportApi()
method = unwrap(api.post)
@@ -65,7 +66,7 @@ class TestRagPipelineImportApi:
assert status == 200
assert response == {"status": "success"}
def test_post_failed_400(self, app):
def test_post_failed_400(self, app: Flask):
api = RagPipelineImportApi()
method = unwrap(api.post)
@@ -95,7 +96,7 @@ class TestRagPipelineImportApi:
assert status == 400
assert response == {"status": "failed"}
def test_post_pending_202(self, app):
def test_post_pending_202(self, app: Flask):
api = RagPipelineImportApi()
method = unwrap(api.post)
@@ -128,10 +129,10 @@ class TestRagPipelineImportApi:
class TestRagPipelineImportConfirmApi:
@pytest.fixture
def app(self, flask_app_with_containers):
def app(self, flask_app_with_containers: Flask):
return flask_app_with_containers
def test_confirm_success(self, app):
def test_confirm_success(self, app: Flask):
api = RagPipelineImportConfirmApi()
method = unwrap(api.post)
@@ -159,7 +160,7 @@ class TestRagPipelineImportConfirmApi:
assert status == 200
assert response == {"ok": True}
def test_confirm_failed(self, app):
def test_confirm_failed(self, app: Flask):
api = RagPipelineImportConfirmApi()
method = unwrap(api.post)
@@ -190,10 +191,10 @@ class TestRagPipelineImportConfirmApi:
class TestRagPipelineImportCheckDependenciesApi:
@pytest.fixture
def app(self, flask_app_with_containers):
def app(self, flask_app_with_containers: Flask):
return flask_app_with_containers
def test_get_success(self, app):
def test_get_success(self, app: Flask):
api = RagPipelineImportCheckDependenciesApi()
method = unwrap(api.get)
@@ -219,10 +220,10 @@ class TestRagPipelineImportCheckDependenciesApi:
class TestRagPipelineExportApi:
@pytest.fixture
def app(self, flask_app_with_containers):
def app(self, flask_app_with_containers: Flask):
return flask_app_with_containers
def test_get_with_include_secret(self, app):
def test_get_with_include_secret(self, app: Flask):
api = RagPipelineExportApi()
method = unwrap(api.get)
@@ -7,6 +7,7 @@ from unittest.mock import MagicMock, patch
from uuid import uuid4
import pytest
from flask import Flask
from sqlalchemy.orm import Session
from werkzeug.exceptions import BadRequest, Forbidden, HTTPException, NotFound
@@ -45,10 +46,10 @@ def unwrap(func):
class TestDraftWorkflowApi:
@pytest.fixture
def app(self, flask_app_with_containers):
def app(self, flask_app_with_containers: Flask):
return flask_app_with_containers
def test_get_draft_success(self, app):
def test_get_draft_success(self, app: Flask):
api = DraftRagPipelineApi()
method = unwrap(api.get)
@@ -68,7 +69,7 @@ class TestDraftWorkflowApi:
result = method(api, pipeline)
assert result == workflow
def test_get_draft_not_exist(self, app):
def test_get_draft_not_exist(self, app: Flask):
api = DraftRagPipelineApi()
method = unwrap(api.get)
@@ -86,7 +87,7 @@ class TestDraftWorkflowApi:
with pytest.raises(DraftWorkflowNotExist):
method(api, pipeline)
def test_sync_hash_not_match(self, app):
def test_sync_hash_not_match(self, app: Flask):
api = DraftRagPipelineApi()
method = unwrap(api.post)
@@ -111,7 +112,7 @@ class TestDraftWorkflowApi:
with pytest.raises(DraftWorkflowNotSync):
method(api, pipeline)
def test_sync_invalid_text_plain(self, app):
def test_sync_invalid_text_plain(self, app: Flask):
api = DraftRagPipelineApi()
method = unwrap(api.post)
@@ -128,7 +129,7 @@ class TestDraftWorkflowApi:
response, status = method(api, pipeline)
assert status == 400
def test_restore_published_workflow_to_draft_success(self, app):
def test_restore_published_workflow_to_draft_success(self, app: Flask):
api = RagPipelineDraftWorkflowRestoreApi()
method = unwrap(api.post)
@@ -155,7 +156,7 @@ class TestDraftWorkflowApi:
assert result["result"] == "success"
assert result["hash"] == "restored-hash"
def test_restore_published_workflow_to_draft_not_found(self, app):
def test_restore_published_workflow_to_draft_not_found(self, app: Flask):
api = RagPipelineDraftWorkflowRestoreApi()
method = unwrap(api.post)
@@ -179,7 +180,7 @@ class TestDraftWorkflowApi:
with pytest.raises(NotFound):
method(api, pipeline, "published-workflow")
def test_restore_published_workflow_to_draft_returns_400_for_draft_source(self, app):
def test_restore_published_workflow_to_draft_returns_400_for_draft_source(self, app: Flask):
api = RagPipelineDraftWorkflowRestoreApi()
method = unwrap(api.post)
@@ -211,10 +212,10 @@ class TestDraftWorkflowApi:
class TestDraftRunNodes:
@pytest.fixture
def app(self, flask_app_with_containers):
def app(self, flask_app_with_containers: Flask):
return flask_app_with_containers
def test_iteration_node_success(self, app):
def test_iteration_node_success(self, app: Flask):
api = RagPipelineDraftRunIterationNodeApi()
method = unwrap(api.post)
@@ -240,7 +241,7 @@ class TestDraftRunNodes:
result = method(api, pipeline, "node")
assert result == {"ok": True}
def test_iteration_node_conversation_not_exists(self, app):
def test_iteration_node_conversation_not_exists(self, app: Flask):
api = RagPipelineDraftRunIterationNodeApi()
method = unwrap(api.post)
@@ -262,7 +263,7 @@ class TestDraftRunNodes:
with pytest.raises(NotFound):
method(api, pipeline, "node")
def test_loop_node_success(self, app):
def test_loop_node_success(self, app: Flask):
api = RagPipelineDraftRunLoopNodeApi()
method = unwrap(api.post)
@@ -290,10 +291,10 @@ class TestDraftRunNodes:
class TestPipelineRunApis:
@pytest.fixture
def app(self, flask_app_with_containers):
def app(self, flask_app_with_containers: Flask):
return flask_app_with_containers
def test_draft_run_success(self, app):
def test_draft_run_success(self, app: Flask):
api = DraftRagPipelineRunApi()
method = unwrap(api.post)
@@ -325,7 +326,7 @@ class TestPipelineRunApis:
):
assert method(api, pipeline) == {"ok": True}
def test_draft_run_rate_limit(self, app):
def test_draft_run_rate_limit(self, app: Flask):
api = DraftRagPipelineRunApi()
method = unwrap(api.post)
@@ -356,10 +357,10 @@ class TestPipelineRunApis:
class TestDraftNodeRun:
@pytest.fixture
def app(self, flask_app_with_containers):
def app(self, flask_app_with_containers: Flask):
return flask_app_with_containers
def test_execution_not_found(self, app):
def test_execution_not_found(self, app: Flask):
api = RagPipelineDraftNodeRunApi()
method = unwrap(api.post)
@@ -387,10 +388,10 @@ class TestDraftNodeRun:
class TestPublishedPipelineApis:
@pytest.fixture
def app(self, flask_app_with_containers):
def app(self, flask_app_with_containers: Flask):
return flask_app_with_containers
def test_publish_success(self, app, db_session_with_containers: Session):
def test_publish_success(self, app: Flask, db_session_with_containers: Session):
from models.dataset import Pipeline
api = PublishedRagPipelineApi()
@@ -436,10 +437,10 @@ class TestPublishedPipelineApis:
class TestMiscApis:
@pytest.fixture
def app(self, flask_app_with_containers):
def app(self, flask_app_with_containers: Flask):
return flask_app_with_containers
def test_task_stop(self, app):
def test_task_stop(self, app: Flask):
api = RagPipelineTaskStopApi()
method = unwrap(api.post)
@@ -460,7 +461,7 @@ class TestMiscApis:
stop_mock.assert_called_once()
assert result["result"] == "success"
def test_transform_forbidden(self, app):
def test_transform_forbidden(self, app: Flask):
api = RagPipelineTransformApi()
method = unwrap(api.post)
@@ -476,7 +477,7 @@ class TestMiscApis:
with pytest.raises(Forbidden):
method(api, "ds1")
def test_recommended_plugins(self, app):
def test_recommended_plugins(self, app: Flask):
api = RagPipelineRecommendedPluginApi()
method = unwrap(api.get)
@@ -496,10 +497,10 @@ class TestMiscApis:
class TestPublishedRagPipelineRunApi:
@pytest.fixture
def app(self, flask_app_with_containers):
def app(self, flask_app_with_containers: Flask):
return flask_app_with_containers
def test_published_run_success(self, app):
def test_published_run_success(self, app: Flask):
api = PublishedRagPipelineRunApi()
method = unwrap(api.post)
@@ -533,7 +534,7 @@ class TestPublishedRagPipelineRunApi:
result = method(api, pipeline)
assert result == {"ok": True}
def test_published_run_rate_limit(self, app):
def test_published_run_rate_limit(self, app: Flask):
api = PublishedRagPipelineRunApi()
method = unwrap(api.post)
@@ -565,10 +566,10 @@ class TestPublishedRagPipelineRunApi:
class TestDefaultBlockConfigApi:
@pytest.fixture
def app(self, flask_app_with_containers):
def app(self, flask_app_with_containers: Flask):
return flask_app_with_containers
def test_get_block_config_success(self, app):
def test_get_block_config_success(self, app: Flask):
api = DefaultRagPipelineBlockConfigApi()
method = unwrap(api.get)
@@ -587,7 +588,7 @@ class TestDefaultBlockConfigApi:
result = method(api, pipeline, "llm")
assert result == {"k": "v"}
def test_get_block_config_invalid_json(self, app):
def test_get_block_config_invalid_json(self, app: Flask):
api = DefaultRagPipelineBlockConfigApi()
method = unwrap(api.get)
@@ -600,10 +601,10 @@ class TestDefaultBlockConfigApi:
class TestPublishedAllRagPipelineApi:
@pytest.fixture
def app(self, flask_app_with_containers):
def app(self, flask_app_with_containers: Flask):
return flask_app_with_containers
def test_get_published_workflows_success(self, app):
def test_get_published_workflows_success(self, app: Flask):
api = PublishedAllRagPipelineApi()
method = unwrap(api.get)
@@ -629,7 +630,7 @@ class TestPublishedAllRagPipelineApi:
assert result["items"] == [{"id": "w1"}]
assert result["has_more"] is False
def test_get_published_workflows_forbidden(self, app):
def test_get_published_workflows_forbidden(self, app: Flask):
api = PublishedAllRagPipelineApi()
method = unwrap(api.get)
@@ -649,10 +650,10 @@ class TestPublishedAllRagPipelineApi:
class TestRagPipelineByIdApi:
@pytest.fixture
def app(self, flask_app_with_containers):
def app(self, flask_app_with_containers: Flask):
return flask_app_with_containers
def test_patch_success(self, app):
def test_patch_success(self, app: Flask):
api = RagPipelineByIdApi()
method = unwrap(api.patch)
@@ -682,7 +683,7 @@ class TestRagPipelineByIdApi:
assert result == workflow
def test_patch_no_fields(self, app):
def test_patch_no_fields(self, app: Flask):
api = RagPipelineByIdApi()
method = unwrap(api.patch)
@@ -700,7 +701,7 @@ class TestRagPipelineByIdApi:
result, status = method(api, pipeline, "w1")
assert status == 400
def test_delete_success(self, app):
def test_delete_success(self, app: Flask):
api = RagPipelineByIdApi()
method = unwrap(api.delete)
@@ -720,7 +721,7 @@ class TestRagPipelineByIdApi:
workflow_service.delete_workflow.assert_called_once()
assert result == (None, 204)
def test_delete_active_workflow_rejected(self, app):
def test_delete_active_workflow_rejected(self, app: Flask):
api = RagPipelineByIdApi()
method = unwrap(api.delete)
@@ -733,10 +734,10 @@ class TestRagPipelineByIdApi:
class TestRagPipelineWorkflowLastRunApi:
@pytest.fixture
def app(self, flask_app_with_containers):
def app(self, flask_app_with_containers: Flask):
return flask_app_with_containers
def test_last_run_success(self, app):
def test_last_run_success(self, app: Flask):
api = RagPipelineWorkflowLastRunApi()
method = unwrap(api.get)
@@ -758,7 +759,7 @@ class TestRagPipelineWorkflowLastRunApi:
result = method(api, pipeline, "node1")
assert result == node_exec
def test_last_run_not_found(self, app):
def test_last_run_not_found(self, app: Flask):
api = RagPipelineWorkflowLastRunApi()
method = unwrap(api.get)
@@ -780,10 +781,10 @@ class TestRagPipelineWorkflowLastRunApi:
class TestRagPipelineDatasourceVariableApi:
@pytest.fixture
def app(self, flask_app_with_containers):
def app(self, flask_app_with_containers: Flask):
return flask_app_with_containers
def test_set_datasource_variables_success(self, app):
def test_set_datasource_variables_success(self, app: Flask):
api = RagPipelineDatasourceVariableApi()
method = unwrap(api.post)
@@ -5,6 +5,7 @@ from __future__ import annotations
from unittest.mock import MagicMock, PropertyMock, patch
import pytest
from flask import Flask
from werkzeug.exceptions import NotFound
from controllers.console.datasets import data_source
@@ -51,10 +52,10 @@ def mock_engine():
class TestDataSourceApi:
@pytest.fixture
def app(self, flask_app_with_containers):
def app(self, flask_app_with_containers: Flask):
return flask_app_with_containers
def test_get_success(self, app, patch_tenant):
def test_get_success(self, app: Flask, patch_tenant):
api = DataSourceApi()
method = unwrap(api.get)
@@ -78,7 +79,7 @@ class TestDataSourceApi:
assert status == 200
assert response["data"][0]["is_bound"] is True
def test_get_no_bindings(self, app, patch_tenant):
def test_get_no_bindings(self, app: Flask, patch_tenant):
api = DataSourceApi()
method = unwrap(api.get)
@@ -94,7 +95,7 @@ class TestDataSourceApi:
assert status == 200
assert response["data"] == []
def test_patch_enable_binding(self, app, patch_tenant, mock_engine):
def test_patch_enable_binding(self, app: Flask, patch_tenant, mock_engine):
api = DataSourceApi()
method = unwrap(api.patch)
@@ -115,7 +116,7 @@ class TestDataSourceApi:
assert status == 200
assert binding.disabled is False
def test_patch_disable_binding(self, app, patch_tenant, mock_engine):
def test_patch_disable_binding(self, app: Flask, patch_tenant, mock_engine):
api = DataSourceApi()
method = unwrap(api.patch)
@@ -136,7 +137,7 @@ class TestDataSourceApi:
assert status == 200
assert binding.disabled is True
def test_patch_binding_not_found(self, app, patch_tenant, mock_engine):
def test_patch_binding_not_found(self, app: Flask, patch_tenant, mock_engine):
api = DataSourceApi()
method = unwrap(api.patch)
@@ -151,7 +152,7 @@ class TestDataSourceApi:
with pytest.raises(NotFound):
method(api, "b1", "enable")
def test_patch_enable_already_enabled(self, app, patch_tenant, mock_engine):
def test_patch_enable_already_enabled(self, app: Flask, patch_tenant, mock_engine):
api = DataSourceApi()
method = unwrap(api.patch)
@@ -168,7 +169,7 @@ class TestDataSourceApi:
with pytest.raises(ValueError):
method(api, "b1", "enable")
def test_patch_disable_already_disabled(self, app, patch_tenant, mock_engine):
def test_patch_disable_already_disabled(self, app: Flask, patch_tenant, mock_engine):
api = DataSourceApi()
method = unwrap(api.patch)
@@ -188,10 +189,10 @@ class TestDataSourceApi:
class TestDataSourceNotionListApi:
@pytest.fixture
def app(self, flask_app_with_containers):
def app(self, flask_app_with_containers: Flask):
return flask_app_with_containers
def test_get_credential_not_found(self, app, patch_tenant):
def test_get_credential_not_found(self, app: Flask, patch_tenant):
api = DataSourceNotionListApi()
method = unwrap(api.get)
@@ -205,7 +206,7 @@ class TestDataSourceNotionListApi:
with pytest.raises(NotFound):
method(api)
def test_get_success_no_dataset_id(self, app, patch_tenant, mock_engine):
def test_get_success_no_dataset_id(self, app: Flask, patch_tenant, mock_engine):
api = DataSourceNotionListApi()
method = unwrap(api.get)
@@ -246,7 +247,7 @@ class TestDataSourceNotionListApi:
assert status == 200
def test_get_success_with_dataset_id(self, app, patch_tenant, mock_engine):
def test_get_success_with_dataset_id(self, app: Flask, patch_tenant, mock_engine):
api = DataSourceNotionListApi()
method = unwrap(api.get)
@@ -299,7 +300,7 @@ class TestDataSourceNotionListApi:
assert status == 200
def test_get_invalid_dataset_type(self, app, patch_tenant, mock_engine):
def test_get_invalid_dataset_type(self, app: Flask, patch_tenant, mock_engine):
api = DataSourceNotionListApi()
method = unwrap(api.get)
@@ -323,10 +324,10 @@ class TestDataSourceNotionListApi:
class TestDataSourceNotionApi:
@pytest.fixture
def app(self, flask_app_with_containers):
def app(self, flask_app_with_containers: Flask):
return flask_app_with_containers
def test_get_preview_success(self, app, patch_tenant):
def test_get_preview_success(self, app: Flask, patch_tenant):
api = DataSourceNotionApi()
method = unwrap(api.get)
@@ -347,7 +348,7 @@ class TestDataSourceNotionApi:
assert status == 200
def test_post_indexing_estimate_success(self, app, patch_tenant):
def test_post_indexing_estimate_success(self, app: Flask, patch_tenant):
api = DataSourceNotionApi()
method = unwrap(api.post)
@@ -381,10 +382,10 @@ class TestDataSourceNotionApi:
class TestDataSourceNotionDatasetSyncApi:
@pytest.fixture
def app(self, flask_app_with_containers):
def app(self, flask_app_with_containers: Flask):
return flask_app_with_containers
def test_get_success(self, app, patch_tenant):
def test_get_success(self, app: Flask, patch_tenant):
api = DataSourceNotionDatasetSyncApi()
method = unwrap(api.get)
@@ -407,7 +408,7 @@ class TestDataSourceNotionDatasetSyncApi:
assert status == 200
def test_get_dataset_not_found(self, app, patch_tenant):
def test_get_dataset_not_found(self, app: Flask, patch_tenant):
api = DataSourceNotionDatasetSyncApi()
method = unwrap(api.get)
@@ -424,10 +425,10 @@ class TestDataSourceNotionDatasetSyncApi:
class TestDataSourceNotionDocumentSyncApi:
@pytest.fixture
def app(self, flask_app_with_containers):
def app(self, flask_app_with_containers: Flask):
return flask_app_with_containers
def test_get_success(self, app, patch_tenant):
def test_get_success(self, app: Flask, patch_tenant):
api = DataSourceNotionDocumentSyncApi()
method = unwrap(api.get)
@@ -450,7 +451,7 @@ class TestDataSourceNotionDocumentSyncApi:
assert status == 200
def test_get_document_not_found(self, app, patch_tenant):
def test_get_document_not_found(self, app: Flask, patch_tenant):
api = DataSourceNotionDocumentSyncApi()
method = unwrap(api.get)
@@ -5,6 +5,7 @@ from __future__ import annotations
from unittest.mock import MagicMock, patch
import pytest
from flask import Flask
from werkzeug.exceptions import NotFound
import controllers.console.explore.conversation as conversation_module
@@ -53,10 +54,10 @@ def user():
class TestConversationListApi:
@pytest.fixture
def app(self, flask_app_with_containers):
def app(self, flask_app_with_containers: Flask):
return flask_app_with_containers
def test_get_success(self, app, chat_app, user):
def test_get_success(self, app: Flask, chat_app, user):
api = conversation_module.ConversationListApi()
method = unwrap(api.get)
@@ -81,7 +82,7 @@ class TestConversationListApi:
assert result["has_more"] is False
assert len(result["data"]) == 2
def test_last_conversation_not_exists(self, app, chat_app, user):
def test_last_conversation_not_exists(self, app: Flask, chat_app, user):
api = conversation_module.ConversationListApi()
method = unwrap(api.get)
@@ -97,7 +98,7 @@ class TestConversationListApi:
with pytest.raises(NotFound):
method(chat_app)
def test_wrong_app_mode(self, app, non_chat_app):
def test_wrong_app_mode(self, app: Flask, non_chat_app):
api = conversation_module.ConversationListApi()
method = unwrap(api.get)
@@ -108,10 +109,10 @@ class TestConversationListApi:
class TestConversationApi:
@pytest.fixture
def app(self, flask_app_with_containers):
def app(self, flask_app_with_containers: Flask):
return flask_app_with_containers
def test_delete_success(self, app, chat_app, user):
def test_delete_success(self, app: Flask, chat_app, user):
api = conversation_module.ConversationApi()
method = unwrap(api.delete)
@@ -129,7 +130,7 @@ class TestConversationApi:
assert status == 204
assert body["result"] == "success"
def test_delete_not_found(self, app, chat_app, user):
def test_delete_not_found(self, app: Flask, chat_app, user):
api = conversation_module.ConversationApi()
method = unwrap(api.delete)
@@ -145,7 +146,7 @@ class TestConversationApi:
with pytest.raises(NotFound):
method(chat_app, "cid")
def test_delete_wrong_app_mode(self, app, non_chat_app):
def test_delete_wrong_app_mode(self, app: Flask, non_chat_app):
api = conversation_module.ConversationApi()
method = unwrap(api.delete)
@@ -156,10 +157,10 @@ class TestConversationApi:
class TestConversationRenameApi:
@pytest.fixture
def app(self, flask_app_with_containers):
def app(self, flask_app_with_containers: Flask):
return flask_app_with_containers
def test_rename_success(self, app, chat_app, user):
def test_rename_success(self, app: Flask, chat_app, user):
api = conversation_module.ConversationRenameApi()
method = unwrap(api.post)
@@ -178,7 +179,7 @@ class TestConversationRenameApi:
assert result["id"] == "cid"
def test_rename_not_found(self, app, chat_app, user):
def test_rename_not_found(self, app: Flask, chat_app, user):
api = conversation_module.ConversationRenameApi()
method = unwrap(api.post)
@@ -197,10 +198,10 @@ class TestConversationRenameApi:
class TestConversationPinApi:
@pytest.fixture
def app(self, flask_app_with_containers):
def app(self, flask_app_with_containers: Flask):
return flask_app_with_containers
def test_pin_success(self, app, chat_app, user):
def test_pin_success(self, app: Flask, chat_app, user):
api = conversation_module.ConversationPinApi()
method = unwrap(api.patch)
@@ -219,10 +220,10 @@ class TestConversationPinApi:
class TestConversationUnPinApi:
@pytest.fixture
def app(self, flask_app_with_containers):
def app(self, flask_app_with_containers: Flask):
return flask_app_with_containers
def test_unpin_success(self, app, chat_app, user):
def test_unpin_success(self, app: Flask, chat_app, user):
api = conversation_module.ConversationUnPinApi()
method = unwrap(api.patch)
@@ -6,6 +6,7 @@ import json
from unittest.mock import MagicMock, patch
import pytest
from flask import Flask
from werkzeug.exceptions import Forbidden
from controllers.console.workspace.tool_providers import (
@@ -60,7 +61,7 @@ def _mock_user_tenant():
@pytest.fixture
def client(flask_app_with_containers):
def client(flask_app_with_containers: Flask):
return flask_app_with_containers.test_client()
@@ -147,10 +148,10 @@ class TestUtils:
class TestToolProviderListApi:
@pytest.fixture
def app(self, flask_app_with_containers):
def app(self, flask_app_with_containers: Flask):
return flask_app_with_containers
def test_get_success(self, app):
def test_get_success(self, app: Flask):
api = ToolProviderListApi()
method = unwrap(api.get)
@@ -170,10 +171,10 @@ class TestToolProviderListApi:
class TestBuiltinProviderApis:
@pytest.fixture
def app(self, flask_app_with_containers):
def app(self, flask_app_with_containers: Flask):
return flask_app_with_containers
def test_list_tools(self, app):
def test_list_tools(self, app: Flask):
api = ToolBuiltinProviderListToolsApi()
method = unwrap(api.get)
@@ -190,7 +191,7 @@ class TestBuiltinProviderApis:
):
assert method(api, "provider") == [{"a": 1}]
def test_info(self, app):
def test_info(self, app: Flask):
api = ToolBuiltinProviderInfoApi()
method = unwrap(api.get)
@@ -207,7 +208,7 @@ class TestBuiltinProviderApis:
):
assert method(api, "provider") == {"x": 1}
def test_delete(self, app):
def test_delete(self, app: Flask):
api = ToolBuiltinProviderDeleteApi()
method = unwrap(api.post)
@@ -224,7 +225,7 @@ class TestBuiltinProviderApis:
):
assert method(api, "provider")["result"] == "success"
def test_add_invalid_type(self, app):
def test_add_invalid_type(self, app: Flask):
api = ToolBuiltinProviderAddApi()
method = unwrap(api.post)
@@ -238,7 +239,7 @@ class TestBuiltinProviderApis:
with pytest.raises(ValueError):
method(api, "provider")
def test_add_success(self, app):
def test_add_success(self, app: Flask):
api = ToolBuiltinProviderAddApi()
method = unwrap(api.post)
@@ -257,7 +258,7 @@ class TestBuiltinProviderApis:
):
assert method(api, "provider")["id"] == 1
def test_update(self, app):
def test_update(self, app: Flask):
api = ToolBuiltinProviderUpdateApi()
method = unwrap(api.post)
@@ -276,7 +277,7 @@ class TestBuiltinProviderApis:
):
assert method(api, "provider")["ok"]
def test_get_credentials(self, app):
def test_get_credentials(self, app: Flask):
api = ToolBuiltinProviderGetCredentialsApi()
method = unwrap(api.get)
@@ -293,7 +294,7 @@ class TestBuiltinProviderApis:
):
assert method(api, "provider") == {"k": "v"}
def test_icon(self, app):
def test_icon(self, app: Flask):
api = ToolBuiltinProviderIconApi()
method = unwrap(api.get)
@@ -307,7 +308,7 @@ class TestBuiltinProviderApis:
response = method(api, "provider")
assert response.mimetype == "image/png"
def test_credentials_schema(self, app):
def test_credentials_schema(self, app: Flask):
api = ToolBuiltinProviderCredentialsSchemaApi()
method = unwrap(api.get)
@@ -324,7 +325,7 @@ class TestBuiltinProviderApis:
):
assert method(api, "provider", "oauth2") == {"schema": {}}
def test_set_default_credential(self, app):
def test_set_default_credential(self, app: Flask):
api = ToolBuiltinProviderSetDefaultApi()
method = unwrap(api.post)
@@ -341,7 +342,7 @@ class TestBuiltinProviderApis:
):
assert method(api, "provider")["ok"]
def test_get_credential_info(self, app):
def test_get_credential_info(self, app: Flask):
api = ToolBuiltinProviderGetCredentialInfoApi()
method = unwrap(api.get)
@@ -358,7 +359,7 @@ class TestBuiltinProviderApis:
):
assert method(api, "provider") == {"info": "x"}
def test_get_oauth_client_schema(self, app):
def test_get_oauth_client_schema(self, app: Flask):
api = ToolBuiltinProviderGetOauthClientSchemaApi()
method = unwrap(api.get)
@@ -378,10 +379,10 @@ class TestBuiltinProviderApis:
class TestApiProviderApis:
@pytest.fixture
def app(self, flask_app_with_containers):
def app(self, flask_app_with_containers: Flask):
return flask_app_with_containers
def test_add(self, app):
def test_add(self, app: Flask):
api = ToolApiProviderAddApi()
method = unwrap(api.post)
@@ -406,7 +407,7 @@ class TestApiProviderApis:
):
assert method(api)["id"] == 1
def test_remote_schema(self, app):
def test_remote_schema(self, app: Flask):
api = ToolApiProviderGetRemoteSchemaApi()
method = unwrap(api.get)
@@ -423,7 +424,7 @@ class TestApiProviderApis:
):
assert method(api)["schema"] == "x"
def test_list_tools(self, app):
def test_list_tools(self, app: Flask):
api = ToolApiProviderListToolsApi()
method = unwrap(api.get)
@@ -440,7 +441,7 @@ class TestApiProviderApis:
):
assert method(api) == [{"tool": 1}]
def test_update(self, app):
def test_update(self, app: Flask):
api = ToolApiProviderUpdateApi()
method = unwrap(api.post)
@@ -468,7 +469,7 @@ class TestApiProviderApis:
):
assert method(api)["ok"]
def test_delete(self, app):
def test_delete(self, app: Flask):
api = ToolApiProviderDeleteApi()
method = unwrap(api.post)
@@ -485,7 +486,7 @@ class TestApiProviderApis:
):
assert method(api)["result"] == "success"
def test_get(self, app):
def test_get(self, app: Flask):
api = ToolApiProviderGetApi()
method = unwrap(api.get)
@@ -505,10 +506,10 @@ class TestApiProviderApis:
class TestWorkflowApis:
@pytest.fixture
def app(self, flask_app_with_containers):
def app(self, flask_app_with_containers: Flask):
return flask_app_with_containers
def test_create(self, app):
def test_create(self, app: Flask):
api = ToolWorkflowProviderCreateApi()
method = unwrap(api.post)
@@ -534,7 +535,7 @@ class TestWorkflowApis:
):
assert method(api)["id"] == 1
def test_update_invalid(self, app):
def test_update_invalid(self, app: Flask):
api = ToolWorkflowProviderUpdateApi()
method = unwrap(api.post)
@@ -560,7 +561,7 @@ class TestWorkflowApis:
result = method(api)
assert result["ok"]
def test_delete(self, app):
def test_delete(self, app: Flask):
api = ToolWorkflowProviderDeleteApi()
method = unwrap(api.post)
@@ -577,7 +578,7 @@ class TestWorkflowApis:
):
assert method(api)["ok"]
def test_get_error(self, app):
def test_get_error(self, app: Flask):
api = ToolWorkflowProviderGetApi()
method = unwrap(api.get)
@@ -594,10 +595,10 @@ class TestWorkflowApis:
class TestLists:
@pytest.fixture
def app(self, flask_app_with_containers):
def app(self, flask_app_with_containers: Flask):
return flask_app_with_containers
def test_builtin_list(self, app):
def test_builtin_list(self, app: Flask):
api = ToolBuiltinListApi()
method = unwrap(api.get)
@@ -617,7 +618,7 @@ class TestLists:
):
assert method(api) == [{"x": 1}]
def test_api_list(self, app):
def test_api_list(self, app: Flask):
api = ToolApiListApi()
method = unwrap(api.get)
@@ -637,7 +638,7 @@ class TestLists:
):
assert method(api) == [{"x": 1}]
def test_workflow_list(self, app):
def test_workflow_list(self, app: Flask):
api = ToolWorkflowListApi()
method = unwrap(api.get)
@@ -660,10 +661,10 @@ class TestLists:
class TestLabels:
@pytest.fixture
def app(self, flask_app_with_containers):
def app(self, flask_app_with_containers: Flask):
return flask_app_with_containers
def test_labels(self, app):
def test_labels(self, app: Flask):
api = ToolLabelsApi()
method = unwrap(api.get)
@@ -679,10 +680,10 @@ class TestLabels:
class TestOAuth:
@pytest.fixture
def app(self, flask_app_with_containers):
def app(self, flask_app_with_containers: Flask):
return flask_app_with_containers
def test_oauth_no_client(self, app):
def test_oauth_no_client(self, app: Flask):
api = ToolPluginOAuthApi()
method = unwrap(api.get)
@@ -700,7 +701,7 @@ class TestOAuth:
with pytest.raises(Forbidden):
method(api, "provider")
def test_oauth_callback_no_cookie(self, app):
def test_oauth_callback_no_cookie(self, app: Flask):
api = ToolOAuthCallback()
method = unwrap(api.get)
@@ -711,10 +712,10 @@ class TestOAuth:
class TestOAuthCustomClient:
@pytest.fixture
def app(self, flask_app_with_containers):
def app(self, flask_app_with_containers: Flask):
return flask_app_with_containers
def test_save_custom_client(self, app):
def test_save_custom_client(self, app: Flask):
api = ToolOAuthCustomClient()
method = unwrap(api.post)
@@ -731,7 +732,7 @@ class TestOAuthCustomClient:
):
assert method(api, "provider")["ok"]
def test_get_custom_client(self, app):
def test_get_custom_client(self, app: Flask):
api = ToolOAuthCustomClient()
method = unwrap(api.get)
@@ -748,7 +749,7 @@ class TestOAuthCustomClient:
):
assert method(api, "provider") == {"client_id": "x"}
def test_delete_custom_client(self, app):
def test_delete_custom_client(self, app: Flask):
api = ToolOAuthCustomClient()
method = unwrap(api.delete)
@@ -5,6 +5,7 @@ from __future__ import annotations
from unittest.mock import MagicMock, patch
import pytest
from flask import Flask
from werkzeug.exceptions import BadRequest, Forbidden
from controllers.console.workspace.trigger_providers import (
@@ -45,10 +46,10 @@ def mock_user():
class TestTriggerProviderApis:
@pytest.fixture
def app(self, flask_app_with_containers):
def app(self, flask_app_with_containers: Flask):
return flask_app_with_containers
def test_icon_success(self, app):
def test_icon_success(self, app: Flask):
api = TriggerProviderIconApi()
method = unwrap(api.get)
@@ -62,7 +63,7 @@ class TestTriggerProviderApis:
):
assert method(api, "github") == "icon"
def test_list_providers(self, app):
def test_list_providers(self, app: Flask):
api = TriggerProviderListApi()
method = unwrap(api.get)
@@ -76,7 +77,7 @@ class TestTriggerProviderApis:
):
assert method(api) == []
def test_provider_info(self, app):
def test_provider_info(self, app: Flask):
api = TriggerProviderInfoApi()
method = unwrap(api.get)
@@ -93,10 +94,10 @@ class TestTriggerProviderApis:
class TestTriggerSubscriptionListApi:
@pytest.fixture
def app(self, flask_app_with_containers):
def app(self, flask_app_with_containers: Flask):
return flask_app_with_containers
def test_list_success(self, app):
def test_list_success(self, app: Flask):
api = TriggerSubscriptionListApi()
method = unwrap(api.get)
@@ -110,7 +111,7 @@ class TestTriggerSubscriptionListApi:
):
assert method(api, "github") == []
def test_list_invalid_provider(self, app):
def test_list_invalid_provider(self, app: Flask):
api = TriggerSubscriptionListApi()
method = unwrap(api.get)
@@ -128,10 +129,10 @@ class TestTriggerSubscriptionListApi:
class TestTriggerSubscriptionBuilderApis:
@pytest.fixture
def app(self, flask_app_with_containers):
def app(self, flask_app_with_containers: Flask):
return flask_app_with_containers
def test_create_builder(self, app):
def test_create_builder(self, app: Flask):
api = TriggerSubscriptionBuilderCreateApi()
method = unwrap(api.post)
@@ -146,7 +147,7 @@ class TestTriggerSubscriptionBuilderApis:
result = method(api, "github")
assert "subscription_builder" in result
def test_get_builder(self, app):
def test_get_builder(self, app: Flask):
api = TriggerSubscriptionBuilderGetApi()
method = unwrap(api.get)
@@ -159,7 +160,7 @@ class TestTriggerSubscriptionBuilderApis:
):
assert method(api, "github", "b1") == {"id": "b1"}
def test_verify_builder(self, app):
def test_verify_builder(self, app: Flask):
api = TriggerSubscriptionBuilderVerifyApi()
method = unwrap(api.post)
@@ -173,7 +174,7 @@ class TestTriggerSubscriptionBuilderApis:
):
assert method(api, "github", "b1") == {"ok": True}
def test_verify_builder_error(self, app):
def test_verify_builder_error(self, app: Flask):
api = TriggerSubscriptionBuilderVerifyApi()
method = unwrap(api.post)
@@ -188,7 +189,7 @@ class TestTriggerSubscriptionBuilderApis:
with pytest.raises(ValueError):
method(api, "github", "b1")
def test_update_builder(self, app):
def test_update_builder(self, app: Flask):
api = TriggerSubscriptionBuilderUpdateApi()
method = unwrap(api.post)
@@ -202,7 +203,7 @@ class TestTriggerSubscriptionBuilderApis:
):
assert method(api, "github", "b1") == {"id": "b1"}
def test_logs(self, app):
def test_logs(self, app: Flask):
api = TriggerSubscriptionBuilderLogsApi()
method = unwrap(api.get)
@@ -219,7 +220,7 @@ class TestTriggerSubscriptionBuilderApis:
):
assert "logs" in method(api, "github", "b1")
def test_build(self, app):
def test_build(self, app: Flask):
api = TriggerSubscriptionBuilderBuildApi()
method = unwrap(api.post)
@@ -236,10 +237,10 @@ class TestTriggerSubscriptionBuilderApis:
class TestTriggerSubscriptionCrud:
@pytest.fixture
def app(self, flask_app_with_containers):
def app(self, flask_app_with_containers: Flask):
return flask_app_with_containers
def test_update_rename_only(self, app):
def test_update_rename_only(self, app: Flask):
api = TriggerSubscriptionUpdateApi()
method = unwrap(api.post)
@@ -258,7 +259,7 @@ class TestTriggerSubscriptionCrud:
):
assert method(api, "s1") == 200
def test_update_not_found(self, app):
def test_update_not_found(self, app: Flask):
api = TriggerSubscriptionUpdateApi()
method = unwrap(api.post)
@@ -273,7 +274,7 @@ class TestTriggerSubscriptionCrud:
with pytest.raises(NotFoundError):
method(api, "x")
def test_update_rebuild(self, app):
def test_update_rebuild(self, app: Flask):
api = TriggerSubscriptionUpdateApi()
method = unwrap(api.post)
@@ -296,7 +297,7 @@ class TestTriggerSubscriptionCrud:
):
assert method(api, "s1") == 200
def test_delete_subscription(self, app):
def test_delete_subscription(self, app: Flask):
api = TriggerSubscriptionDeleteApi()
method = unwrap(api.post)
@@ -319,7 +320,7 @@ class TestTriggerSubscriptionCrud:
assert result["result"] == "success"
def test_delete_subscription_value_error(self, app):
def test_delete_subscription_value_error(self, app: Flask):
api = TriggerSubscriptionDeleteApi()
method = unwrap(api.post)
@@ -342,10 +343,10 @@ class TestTriggerSubscriptionCrud:
class TestTriggerOAuthApis:
@pytest.fixture
def app(self, flask_app_with_containers):
def app(self, flask_app_with_containers: Flask):
return flask_app_with_containers
def test_oauth_authorize_success(self, app):
def test_oauth_authorize_success(self, app: Flask):
api = TriggerOAuthAuthorizeApi()
method = unwrap(api.get)
@@ -372,7 +373,7 @@ class TestTriggerOAuthApis:
resp = method(api, "github")
assert resp.status_code == 200
def test_oauth_authorize_no_client(self, app):
def test_oauth_authorize_no_client(self, app: Flask):
api = TriggerOAuthAuthorizeApi()
method = unwrap(api.get)
@@ -387,7 +388,7 @@ class TestTriggerOAuthApis:
with pytest.raises(NotFoundError):
method(api, "github")
def test_oauth_callback_forbidden(self, app):
def test_oauth_callback_forbidden(self, app: Flask):
api = TriggerOAuthCallbackApi()
method = unwrap(api.get)
@@ -395,7 +396,7 @@ class TestTriggerOAuthApis:
with pytest.raises(Forbidden):
method(api, "github")
def test_oauth_callback_success(self, app):
def test_oauth_callback_success(self, app: Flask):
api = TriggerOAuthCallbackApi()
method = unwrap(api.get)
@@ -425,7 +426,7 @@ class TestTriggerOAuthApis:
resp = method(api, "github")
assert resp.status_code == 302
def test_oauth_callback_no_oauth_client(self, app):
def test_oauth_callback_no_oauth_client(self, app: Flask):
api = TriggerOAuthCallbackApi()
method = unwrap(api.get)
@@ -449,7 +450,7 @@ class TestTriggerOAuthApis:
with pytest.raises(Forbidden):
method(api, "github")
def test_oauth_callback_empty_credentials(self, app):
def test_oauth_callback_empty_credentials(self, app: Flask):
api = TriggerOAuthCallbackApi()
method = unwrap(api.get)
@@ -480,10 +481,10 @@ class TestTriggerOAuthApis:
class TestTriggerOAuthClientManageApi:
@pytest.fixture
def app(self, flask_app_with_containers):
def app(self, flask_app_with_containers: Flask):
return flask_app_with_containers
def test_get_client(self, app):
def test_get_client(self, app: Flask):
api = TriggerOAuthClientManageApi()
method = unwrap(api.get)
@@ -510,7 +511,7 @@ class TestTriggerOAuthClientManageApi:
result = method(api, "github")
assert "configured" in result
def test_post_client(self, app):
def test_post_client(self, app: Flask):
api = TriggerOAuthClientManageApi()
method = unwrap(api.post)
@@ -524,7 +525,7 @@ class TestTriggerOAuthClientManageApi:
):
assert method(api, "github") == {"ok": True}
def test_delete_client(self, app):
def test_delete_client(self, app: Flask):
api = TriggerOAuthClientManageApi()
method = unwrap(api.delete)
@@ -538,7 +539,7 @@ class TestTriggerOAuthClientManageApi:
):
assert method(api, "github") == {"ok": True}
def test_oauth_client_post_value_error(self, app):
def test_oauth_client_post_value_error(self, app: Flask):
api = TriggerOAuthClientManageApi()
method = unwrap(api.post)
@@ -556,10 +557,10 @@ class TestTriggerOAuthClientManageApi:
class TestTriggerSubscriptionVerifyApi:
@pytest.fixture
def app(self, flask_app_with_containers):
def app(self, flask_app_with_containers: Flask):
return flask_app_with_containers
def test_verify_success(self, app):
def test_verify_success(self, app: Flask):
api = TriggerSubscriptionVerifyApi()
method = unwrap(api.post)
@@ -18,6 +18,7 @@ from types import SimpleNamespace
from unittest.mock import Mock, patch
import pytest
from flask import Flask
from sqlalchemy.orm import Session
from werkzeug.exceptions import Forbidden, NotFound
@@ -217,10 +218,20 @@ class TestTagUnbindingPayload:
"""Test suite for TagUnbindingPayload Pydantic model."""
def test_payload_with_valid_data(self):
payload = TagUnbindingPayload(tag_id="tag_123", target_id="dataset_456")
assert payload.tag_id == "tag_123"
payload = TagUnbindingPayload(tag_ids=["tag_123"], target_id="dataset_456")
assert payload.tag_ids == ["tag_123"]
assert payload.target_id == "dataset_456"
def test_payload_normalizes_legacy_tag_id(self):
payload = TagUnbindingPayload(tag_id="tag_123", target_id="dataset_456")
assert payload.tag_ids == ["tag_123"]
assert payload.target_id == "dataset_456"
def test_payload_rejects_empty_tag_ids(self):
with pytest.raises(ValueError) as exc_info:
TagUnbindingPayload(tag_ids=[], target_id="dataset_456")
assert "Tag IDs is required" in str(exc_info.value)
# ---------------------------------------------------------------------------
# Helpers
@@ -236,7 +247,7 @@ def _unwrap(method):
@pytest.fixture
def app(flask_app_with_containers):
def app(flask_app_with_containers: Flask):
# Uses the full containerised app so that Flask config, extensions, and
# blueprint registrations match production. Most tests mock the service
# layer to isolate controller logic; a few (e.g. test_list_tags_from_db)
@@ -280,7 +291,7 @@ class TestDatasetListApiGet:
mock_current_user,
mock_provider_mgr,
mock_marshal,
app,
app: Flask,
mock_tenant,
):
from controllers.service_api.dataset.dataset import DatasetListApi
@@ -315,7 +326,7 @@ class TestDatasetListApiPost:
mock_dataset_svc,
mock_current_user,
mock_marshal,
app,
app: Flask,
mock_tenant,
):
from controllers.service_api.dataset.dataset import DatasetListApi
@@ -341,7 +352,7 @@ class TestDatasetListApiPost:
self,
mock_dataset_svc,
mock_current_user,
app,
app: Flask,
mock_tenant,
):
from controllers.service_api.dataset.dataset import DatasetListApi
@@ -379,7 +390,7 @@ class TestDatasetApiGet:
mock_provider_mgr,
mock_marshal,
mock_perm_svc,
app,
app: Flask,
mock_dataset,
):
from controllers.service_api.dataset.dataset import DatasetApi
@@ -429,7 +440,7 @@ class TestDatasetApiGet:
self,
mock_dataset_svc,
mock_current_user,
app,
app: Flask,
mock_dataset,
):
from controllers.service_api.dataset.dataset import DatasetApi
@@ -457,7 +468,7 @@ class TestDatasetApiDelete:
mock_dataset_svc,
mock_current_user,
mock_perm_svc,
app,
app: Flask,
mock_dataset,
):
from controllers.service_api.dataset.dataset import DatasetApi
@@ -479,7 +490,7 @@ class TestDatasetApiDelete:
self,
mock_dataset_svc,
mock_current_user,
app,
app: Flask,
mock_dataset,
):
from controllers.service_api.dataset.dataset import DatasetApi
@@ -500,7 +511,7 @@ class TestDatasetApiDelete:
self,
mock_dataset_svc,
mock_current_user,
app,
app: Flask,
mock_dataset,
):
from controllers.service_api.dataset.dataset import DatasetApi
@@ -532,7 +543,7 @@ class TestDocumentStatusApiPatch:
mock_dataset_svc,
mock_current_user,
mock_doc_svc,
app,
app: Flask,
mock_tenant,
mock_dataset,
):
@@ -563,7 +574,7 @@ class TestDocumentStatusApiPatch:
def test_batch_update_status_dataset_not_found(
self,
mock_dataset_svc,
app,
app: Flask,
mock_tenant,
mock_dataset,
):
@@ -592,7 +603,7 @@ class TestDocumentStatusApiPatch:
mock_dataset_svc,
mock_current_user,
mock_doc_svc,
app,
app: Flask,
mock_tenant,
mock_dataset,
):
@@ -625,7 +636,7 @@ class TestDocumentStatusApiPatch:
mock_dataset_svc,
mock_current_user,
mock_doc_svc,
app,
app: Flask,
mock_tenant,
mock_dataset,
):
@@ -658,7 +669,7 @@ class TestDocumentStatusApiPatch:
mock_dataset_svc,
mock_current_user,
mock_doc_svc,
app,
app: Flask,
mock_tenant,
mock_dataset,
):
@@ -698,7 +709,7 @@ class TestDatasetTagsApiGet:
self,
mock_current_user,
mock_tag_svc,
app,
app: Flask,
):
from controllers.service_api.dataset.dataset import DatasetTagsApi
@@ -720,7 +731,7 @@ class TestDatasetTagsApiGet:
def test_list_tags_from_db(
self,
mock_current_user,
app,
app: Flask,
db_session_with_containers: Session,
):
"""Integration test: creates real Tag rows and retrieves them
@@ -763,7 +774,7 @@ class TestDatasetTagsApiPost:
self,
mock_current_user,
mock_tag_svc,
app,
app: Flask,
):
from controllers.service_api.dataset.dataset import DatasetTagsApi
@@ -786,7 +797,7 @@ class TestDatasetTagsApiPost:
mock_tag_svc.save_tags.assert_called_once()
@patch("controllers.service_api.dataset.dataset.current_user")
def test_create_tag_forbidden(self, mock_current_user, app):
def test_create_tag_forbidden(self, mock_current_user, app: Flask):
from controllers.service_api.dataset.dataset import DatasetTagsApi
mock_current_user.__class__ = Account
@@ -815,7 +826,7 @@ class TestDatasetTagsApiPatch:
mock_current_user,
mock_service_api_ns,
mock_tag_svc,
app,
app: Flask,
):
from controllers.service_api.dataset.dataset import DatasetTagsApi
@@ -841,7 +852,7 @@ class TestDatasetTagsApiPatch:
mock_tag_svc.update_tags.assert_called_once_with({"name": "Updated Tag", "type": "knowledge"}, "tag-1")
@patch("controllers.service_api.dataset.dataset.current_user")
def test_update_tag_forbidden(self, mock_current_user, app):
def test_update_tag_forbidden(self, mock_current_user, app: Flask):
from controllers.service_api.dataset.dataset import DatasetTagsApi
mock_current_user.__class__ = Account
@@ -869,7 +880,7 @@ class TestDatasetTagsApiDelete:
mock_current_user,
mock_service_api_ns,
mock_tag_svc,
app,
app: Flask,
):
from controllers.service_api.dataset.dataset import DatasetTagsApi
@@ -894,7 +905,7 @@ class TestDatasetTagsApiDelete:
mock_tag_svc.delete_tag.assert_called_once_with("tag-1")
@patch("libs.login.current_user")
def test_delete_tag_forbidden(self, mock_current_user, app):
def test_delete_tag_forbidden(self, mock_current_user, app: Flask):
from controllers.service_api.dataset.dataset import DatasetTagsApi
user_obj = Mock(spec=Account)
@@ -922,7 +933,7 @@ class TestDatasetTagsBindingStatusApi:
self,
mock_current_user,
mock_tag_svc,
app,
app: Flask,
):
from controllers.service_api.dataset.dataset import DatasetTagsBindingStatusApi
@@ -952,7 +963,7 @@ class TestDatasetTagBindingApiPost:
self,
mock_current_user,
mock_tag_svc,
app,
app: Flask,
):
from controllers.service_api.dataset.dataset import DatasetTagBindingApi
@@ -977,7 +988,7 @@ class TestDatasetTagBindingApiPost:
)
@patch("controllers.service_api.dataset.dataset.current_user")
def test_bind_tags_forbidden(self, mock_current_user, app):
def test_bind_tags_forbidden(self, mock_current_user, app: Flask):
from controllers.service_api.dataset.dataset import DatasetTagBindingApi
mock_current_user.__class__ = Account
@@ -1003,7 +1014,37 @@ class TestDatasetTagUnbindingApiPost:
self,
mock_current_user,
mock_tag_svc,
app,
app: Flask,
):
from controllers.service_api.dataset.dataset import DatasetTagUnbindingApi
mock_current_user.__class__ = Account
mock_current_user.has_edit_permission = True
mock_current_user.is_dataset_editor = True
mock_tag_svc.delete_tag_binding.return_value = None
with app.test_request_context(
"/datasets/tags/unbinding",
method="POST",
json={"tag_ids": ["tag-1"], "target_id": "ds-1"},
):
api = DatasetTagUnbindingApi()
result = api.post(_=None)
assert result == ("", 204)
from services.tag_service import TagBindingDeletePayload
mock_tag_svc.delete_tag_binding.assert_called_once_with(
TagBindingDeletePayload(tag_ids=["tag-1"], target_id="ds-1", type="knowledge")
)
@patch("controllers.service_api.dataset.dataset.TagService")
@patch("controllers.service_api.dataset.dataset.current_user")
def test_unbind_legacy_tag_id_success(
self,
mock_current_user,
mock_tag_svc,
app: Flask,
):
from controllers.service_api.dataset.dataset import DatasetTagUnbindingApi
@@ -1024,11 +1065,11 @@ class TestDatasetTagUnbindingApiPost:
from services.tag_service import TagBindingDeletePayload
mock_tag_svc.delete_tag_binding.assert_called_once_with(
TagBindingDeletePayload(tag_id="tag-1", target_id="ds-1", type="knowledge")
TagBindingDeletePayload(tag_ids=["tag-1"], target_id="ds-1", type="knowledge")
)
@patch("controllers.service_api.dataset.dataset.current_user")
def test_unbind_tag_forbidden(self, mock_current_user, app):
def test_unbind_tag_forbidden(self, mock_current_user, app: Flask):
from controllers.service_api.dataset.dataset import DatasetTagUnbindingApi
mock_current_user.__class__ = Account
@@ -1038,7 +1079,7 @@ class TestDatasetTagUnbindingApiPost:
with app.test_request_context(
"/datasets/tags/unbinding",
method="POST",
json={"tag_id": "tag-1", "target_id": "ds-1"},
json={"tag_ids": ["tag-1"], "target_id": "ds-1"},
):
api = DatasetTagUnbindingApi()
with pytest.raises(Forbidden):
@@ -7,6 +7,7 @@ from unittest.mock import MagicMock, patch
from uuid import uuid4
import pytest
from flask import Flask
from werkzeug.exceptions import NotFound
from controllers.web.conversation import (
@@ -34,16 +35,16 @@ def _end_user() -> SimpleNamespace:
class TestConversationListApi:
@pytest.fixture
def app(self, flask_app_with_containers):
def app(self, flask_app_with_containers: Flask):
return flask_app_with_containers
def test_non_chat_mode_raises(self, app) -> None:
def test_non_chat_mode_raises(self, app: Flask) -> None:
with app.test_request_context("/conversations"):
with pytest.raises(NotChatAppError):
ConversationListApi().get(_completion_app(), _end_user())
@patch("controllers.web.conversation.WebConversationService.pagination_by_last_id")
def test_happy_path(self, mock_paginate: MagicMock, app) -> None:
def test_happy_path(self, mock_paginate: MagicMock, app: Flask) -> None:
conv_id = str(uuid4())
conv = SimpleNamespace(
id=conv_id,
@@ -65,16 +66,16 @@ class TestConversationListApi:
class TestConversationApi:
@pytest.fixture
def app(self, flask_app_with_containers):
def app(self, flask_app_with_containers: Flask):
return flask_app_with_containers
def test_non_chat_mode_raises(self, app) -> None:
def test_non_chat_mode_raises(self, app: Flask) -> None:
with app.test_request_context(f"/conversations/{uuid4()}"):
with pytest.raises(NotChatAppError):
ConversationApi().delete(_completion_app(), _end_user(), uuid4())
@patch("controllers.web.conversation.ConversationService.delete")
def test_delete_success(self, mock_delete: MagicMock, app) -> None:
def test_delete_success(self, mock_delete: MagicMock, app: Flask) -> None:
c_id = uuid4()
with app.test_request_context(f"/conversations/{c_id}"):
result, status = ConversationApi().delete(_chat_app(), _end_user(), c_id)
@@ -83,7 +84,7 @@ class TestConversationApi:
assert result["result"] == "success"
@patch("controllers.web.conversation.ConversationService.delete", side_effect=ConversationNotExistsError())
def test_delete_not_found(self, mock_delete: MagicMock, app) -> None:
def test_delete_not_found(self, mock_delete: MagicMock, app: Flask) -> None:
c_id = uuid4()
with app.test_request_context(f"/conversations/{c_id}"):
with pytest.raises(NotFound, match="Conversation Not Exists"):
@@ -92,17 +93,17 @@ class TestConversationApi:
class TestConversationRenameApi:
@pytest.fixture
def app(self, flask_app_with_containers):
def app(self, flask_app_with_containers: Flask):
return flask_app_with_containers
def test_non_chat_mode_raises(self, app) -> None:
def test_non_chat_mode_raises(self, app: Flask) -> None:
with app.test_request_context(f"/conversations/{uuid4()}/name", method="POST", json={"name": "x"}):
with pytest.raises(NotChatAppError):
ConversationRenameApi().post(_completion_app(), _end_user(), uuid4())
@patch("controllers.web.conversation.ConversationService.rename")
@patch("controllers.web.conversation.web_ns")
def test_rename_success(self, mock_ns: MagicMock, mock_rename: MagicMock, app) -> None:
def test_rename_success(self, mock_ns: MagicMock, mock_rename: MagicMock, app: Flask) -> None:
c_id = uuid4()
mock_ns.payload = {"name": "New Name", "auto_generate": False}
conv = SimpleNamespace(
@@ -126,7 +127,7 @@ class TestConversationRenameApi:
side_effect=ConversationNotExistsError(),
)
@patch("controllers.web.conversation.web_ns")
def test_rename_not_found(self, mock_ns: MagicMock, mock_rename: MagicMock, app) -> None:
def test_rename_not_found(self, mock_ns: MagicMock, mock_rename: MagicMock, app: Flask) -> None:
c_id = uuid4()
mock_ns.payload = {"name": "X", "auto_generate": False}
@@ -137,16 +138,16 @@ class TestConversationRenameApi:
class TestConversationPinApi:
@pytest.fixture
def app(self, flask_app_with_containers):
def app(self, flask_app_with_containers: Flask):
return flask_app_with_containers
def test_non_chat_mode_raises(self, app) -> None:
def test_non_chat_mode_raises(self, app: Flask) -> None:
with app.test_request_context(f"/conversations/{uuid4()}/pin", method="PATCH"):
with pytest.raises(NotChatAppError):
ConversationPinApi().patch(_completion_app(), _end_user(), uuid4())
@patch("controllers.web.conversation.WebConversationService.pin")
def test_pin_success(self, mock_pin: MagicMock, app) -> None:
def test_pin_success(self, mock_pin: MagicMock, app: Flask) -> None:
c_id = uuid4()
with app.test_request_context(f"/conversations/{c_id}/pin", method="PATCH"):
result = ConversationPinApi().patch(_chat_app(), _end_user(), c_id)
@@ -154,7 +155,7 @@ class TestConversationPinApi:
assert result["result"] == "success"
@patch("controllers.web.conversation.WebConversationService.pin", side_effect=ConversationNotExistsError())
def test_pin_not_found(self, mock_pin: MagicMock, app) -> None:
def test_pin_not_found(self, mock_pin: MagicMock, app: Flask) -> None:
c_id = uuid4()
with app.test_request_context(f"/conversations/{c_id}/pin", method="PATCH"):
with pytest.raises(NotFound):
@@ -163,16 +164,16 @@ class TestConversationPinApi:
class TestConversationUnPinApi:
@pytest.fixture
def app(self, flask_app_with_containers):
def app(self, flask_app_with_containers: Flask):
return flask_app_with_containers
def test_non_chat_mode_raises(self, app) -> None:
def test_non_chat_mode_raises(self, app: Flask) -> None:
with app.test_request_context(f"/conversations/{uuid4()}/unpin", method="PATCH"):
with pytest.raises(NotChatAppError):
ConversationUnPinApi().patch(_completion_app(), _end_user(), uuid4())
@patch("controllers.web.conversation.WebConversationService.unpin")
def test_unpin_success(self, mock_unpin: MagicMock, app) -> None:
def test_unpin_success(self, mock_unpin: MagicMock, app: Flask) -> None:
c_id = uuid4()
with app.test_request_context(f"/conversations/{c_id}/unpin", method="PATCH"):
result = ConversationUnPinApi().patch(_chat_app(), _end_user(), c_id)
@@ -7,6 +7,7 @@ from types import SimpleNamespace
from unittest.mock import MagicMock, patch
import pytest
from flask import Flask
from controllers.web.forgot_password import (
ForgotPasswordCheckApi,
@@ -29,7 +30,7 @@ def _patch_wraps():
class TestForgotPasswordSendEmailApi:
@pytest.fixture
def app(self, flask_app_with_containers):
def app(self, flask_app_with_containers: Flask):
return flask_app_with_containers
@patch("controllers.web.forgot_password.AccountService.send_reset_password_email")
@@ -42,7 +43,7 @@ class TestForgotPasswordSendEmailApi:
mock_rate_limit,
mock_get_account,
mock_send_mail,
app,
app: Flask,
):
mock_account = MagicMock()
mock_get_account.return_value = mock_account
@@ -64,7 +65,7 @@ class TestForgotPasswordSendEmailApi:
class TestForgotPasswordCheckApi:
@pytest.fixture
def app(self, flask_app_with_containers):
def app(self, flask_app_with_containers: Flask):
return flask_app_with_containers
@patch("controllers.web.forgot_password.AccountService.reset_forgot_password_error_rate_limit")
@@ -81,7 +82,7 @@ class TestForgotPasswordCheckApi:
mock_revoke_token,
mock_generate_token,
mock_reset_rate,
app,
app: Flask,
):
mock_is_rate_limit.return_value = False
mock_get_data.return_value = {"email": "User@Example.com", "code": "1234"}
@@ -117,7 +118,7 @@ class TestForgotPasswordCheckApi:
mock_revoke_token,
mock_generate_token,
mock_reset_rate,
app,
app: Flask,
):
mock_is_rate_limit.return_value = False
mock_get_data.return_value = {"email": "MixedCase@Example.com", "code": "5678"}
@@ -142,7 +143,7 @@ class TestForgotPasswordCheckApi:
class TestForgotPasswordResetApi:
@pytest.fixture
def app(self, flask_app_with_containers):
def app(self, flask_app_with_containers: Flask):
return flask_app_with_containers
@patch("controllers.web.forgot_password.ForgotPasswordResetApi._update_existing_account")
@@ -157,7 +158,7 @@ class TestForgotPasswordResetApi:
mock_db,
mock_get_account,
mock_update_account,
app,
app: Flask,
):
mock_get_reset_data.return_value = {"phase": "reset", "email": "User@Example.com", "code": "1234"}
mock_account = MagicMock()
@@ -194,7 +195,7 @@ class TestForgotPasswordResetApi:
mock_db,
mock_token_bytes,
mock_hash_password,
app,
app: Flask,
):
mock_get_reset_data.return_value = {"phase": "reset", "email": "user@example.com"}
account = MagicMock()
@@ -8,6 +8,7 @@ from unittest.mock import MagicMock, patch
from uuid import uuid4
import pytest
from flask import Flask
from sqlalchemy.orm import Session
from werkzeug.exceptions import BadRequest, NotFound, Unauthorized
@@ -182,7 +183,7 @@ class TestValidateUserAccessibility:
class TestDecodeJwtToken:
@pytest.fixture
def app(self, flask_app_with_containers):
def app(self, flask_app_with_containers: Flask):
return flask_app_with_containers
def _create_app_site_enduser(self, db_session: Session, *, enable_site: bool = True):
@@ -239,7 +240,7 @@ class TestDecodeJwtToken:
mock_access_mode: MagicMock,
mock_validate_token: MagicMock,
mock_validate_user: MagicMock,
app,
app: Flask,
db_session_with_containers: Session,
) -> None:
app_model, site, end_user = self._create_app_site_enduser(db_session_with_containers)
@@ -299,7 +300,7 @@ class TestDecodeJwtToken:
mock_extract: MagicMock,
mock_passport_cls: MagicMock,
mock_features: MagicMock,
app,
app: Flask,
db_session_with_containers: Session,
) -> None:
app_model, site, end_user = self._create_app_site_enduser(db_session_with_containers, enable_site=False)
@@ -324,7 +325,7 @@ class TestDecodeJwtToken:
mock_extract: MagicMock,
mock_passport_cls: MagicMock,
mock_features: MagicMock,
app,
app: Flask,
db_session_with_containers: Session,
) -> None:
app_model, site, _ = self._create_app_site_enduser(db_session_with_containers)
@@ -350,7 +351,7 @@ class TestDecodeJwtToken:
mock_extract: MagicMock,
mock_passport_cls: MagicMock,
mock_features: MagicMock,
app,
app: Flask,
db_session_with_containers: Session,
) -> None:
app_model, site, end_user = self._create_app_site_enduser(db_session_with_containers)
@@ -85,7 +85,7 @@ class TestPauseStatePersistenceLayerTestContainers:
return WorkflowRunService(engine)
@pytest.fixture(autouse=True)
def setup_test_data(self, db_session_with_containers, file_service, workflow_run_service):
def setup_test_data(self, db_session_with_containers: Session, file_service, workflow_run_service):
"""Set up test data for each test method using TestContainers."""
# Create test tenant and account
from models.account import AccountStatus, Tenant, TenantAccountJoin, TenantAccountRole, TenantStatus
@@ -210,9 +210,7 @@ class TestPauseStatePersistenceLayerTestContainers:
execution_id = workflow_run_id or getattr(self, "test_workflow_run_id", None) or str(uuid.uuid4())
# Create variable pool
variable_pool = VariablePool.from_bootstrap(
system_variables=build_system_variables(workflow_execution_id=execution_id),
)
variable_pool = VariablePool(system_variables=build_system_variables(workflow_execution_id=execution_id))
if variables:
for (node_id, var_key), value in variables.items():
variable_pool.add([node_id, var_key], value)
@@ -297,7 +295,7 @@ class TestPauseStatePersistenceLayerTestContainers:
generate_entity=entity,
)
def test_complete_pause_flow_with_real_dependencies(self, db_session_with_containers):
def test_complete_pause_flow_with_real_dependencies(self, db_session_with_containers: Session):
"""Test complete pause flow: event -> state serialization -> database save -> storage save."""
# Arrange
layer = self._create_pause_state_persistence_layer()
@@ -354,7 +352,7 @@ class TestPauseStatePersistenceLayerTestContainers:
assert isinstance(persisted_entity, WorkflowAppGenerateEntity)
assert persisted_entity.workflow_execution_id == self.test_workflow_run_id
def test_state_persistence_and_retrieval(self, db_session_with_containers):
def test_state_persistence_and_retrieval(self, db_session_with_containers: Session):
"""Test that pause state can be persisted and retrieved correctly."""
# Arrange
layer = self._create_pause_state_persistence_layer()
@@ -404,7 +402,7 @@ class TestPauseStatePersistenceLayerTestContainers:
assert retrieved_state["node_run_steps"] == 10
assert resumption_context.get_generate_entity().workflow_execution_id == self.test_workflow_run_id
def test_database_transaction_handling(self, db_session_with_containers):
def test_database_transaction_handling(self, db_session_with_containers: Session):
"""Test that database transactions are handled correctly."""
# Arrange
layer = self._create_pause_state_persistence_layer()
@@ -435,7 +433,7 @@ class TestPauseStatePersistenceLayerTestContainers:
assert pause_model.resumed_at is None
assert pause_model.state_object_key != ""
def test_file_storage_integration(self, db_session_with_containers):
def test_file_storage_integration(self, db_session_with_containers: Session):
"""Test integration with file storage system."""
# Arrange
layer = self._create_pause_state_persistence_layer()
@@ -469,7 +467,7 @@ class TestPauseStatePersistenceLayerTestContainers:
assert resumption_context.serialized_graph_runtime_state == graph_runtime_state.dumps()
assert resumption_context.get_generate_entity().workflow_execution_id == self.test_workflow_run_id
def test_workflow_with_different_creators(self, db_session_with_containers):
def test_workflow_with_different_creators(self, db_session_with_containers: Session):
"""Test pause state with workflows created by different users."""
# Arrange - Create workflow with different creator
different_user_id = str(uuid.uuid4())
@@ -534,7 +532,7 @@ class TestPauseStatePersistenceLayerTestContainers:
resumption_context = WorkflowResumptionContext.loads(pause_entity.get_state().decode())
assert resumption_context.get_generate_entity().workflow_execution_id == different_workflow_run.id
def test_layer_ignores_non_pause_events(self, db_session_with_containers):
def test_layer_ignores_non_pause_events(self, db_session_with_containers: Session):
"""Test that layer ignores non-pause events."""
# Arrange
layer = self._create_pause_state_persistence_layer()
@@ -564,7 +562,7 @@ class TestPauseStatePersistenceLayerTestContainers:
).all()
assert len(pause_states) == 0
def test_layer_requires_initialization(self, db_session_with_containers):
def test_layer_requires_initialization(self, db_session_with_containers: Session):
"""Test that layer requires proper initialization before handling events."""
# Arrange
layer = self._create_pause_state_persistence_layer()
@@ -15,11 +15,14 @@ from uuid import uuid4
import pytest
from faker import Faker
from sqlalchemy.orm import Session
from core.rag.pipeline.queue import TaskWrapper, TenantIsolatedTaskQueue
from extensions.ext_redis import redis_client
from models import Account, AccountStatus, Tenant, TenantAccountJoin, TenantAccountRole, TenantStatus
TenantAndAccount = tuple[Tenant, Account]
@dataclass
class TestTask:
@@ -40,7 +43,7 @@ class TestTenantIsolatedTaskQueueIntegration:
return Faker()
@pytest.fixture
def test_tenant_and_account(self, db_session_with_containers, fake):
def test_tenant_and_account(self, db_session_with_containers: Session, fake: Faker):
"""Create test tenant and account for testing."""
# Create account
account = Account(
@@ -73,18 +76,18 @@ class TestTenantIsolatedTaskQueueIntegration:
return tenant, account
@pytest.fixture
def test_queue(self, test_tenant_and_account):
def test_queue(self, test_tenant_and_account: TenantAndAccount):
"""Create a generic test queue for testing."""
tenant, _ = test_tenant_and_account
return TenantIsolatedTaskQueue(tenant.id, "test_queue")
@pytest.fixture
def secondary_queue(self, test_tenant_and_account):
def secondary_queue(self, test_tenant_and_account: TenantAndAccount):
"""Create a secondary test queue for testing isolation."""
tenant, _ = test_tenant_and_account
return TenantIsolatedTaskQueue(tenant.id, "secondary_queue")
def test_queue_initialization(self, test_tenant_and_account):
def test_queue_initialization(self, test_tenant_and_account: TenantAndAccount):
"""Test queue initialization with correct key generation."""
tenant, _ = test_tenant_and_account
queue = TenantIsolatedTaskQueue(tenant.id, "test-key")
@@ -94,7 +97,9 @@ class TestTenantIsolatedTaskQueueIntegration:
assert queue._queue == f"tenant_self_test-key_task_queue:{tenant.id}"
assert queue._task_key == f"tenant_test-key_task:{tenant.id}"
def test_tenant_isolation(self, test_tenant_and_account, db_session_with_containers, fake):
def test_tenant_isolation(
self, test_tenant_and_account: TenantAndAccount, db_session_with_containers: Session, fake: Faker
):
"""Test that different tenants have isolated queues."""
tenant1, _ = test_tenant_and_account
@@ -114,7 +119,7 @@ class TestTenantIsolatedTaskQueueIntegration:
assert queue1._queue == f"tenant_self_same-key_task_queue:{tenant1.id}"
assert queue2._queue == f"tenant_self_same-key_task_queue:{tenant2.id}"
def test_key_isolation(self, test_tenant_and_account):
def test_key_isolation(self, test_tenant_and_account: TenantAndAccount):
"""Test that different keys have isolated queues."""
tenant, _ = test_tenant_and_account
queue1 = TenantIsolatedTaskQueue(tenant.id, "key1")
@@ -176,7 +181,7 @@ class TestTenantIsolatedTaskQueueIntegration:
assert len(remaining_tasks) == 2
assert remaining_tasks == ["task4", "task5"]
def test_push_and_pull_complex_objects(self, test_queue, fake):
def test_push_and_pull_complex_objects(self, test_queue, fake: Faker):
"""Test pushing and pulling complex object tasks."""
# Create complex task objects as dictionaries (not dataclass instances)
tasks = [
@@ -218,7 +223,7 @@ class TestTenantIsolatedTaskQueueIntegration:
assert pulled_task["data"] == original_task["data"]
assert pulled_task["metadata"] == original_task["metadata"]
def test_mixed_task_types(self, test_queue, fake):
def test_mixed_task_types(self, test_queue, fake: Faker):
"""Test pushing and pulling mixed string and object tasks."""
string_task = "simple_string_task"
object_task = {
@@ -267,7 +272,7 @@ class TestTenantIsolatedTaskQueueIntegration:
# Verify task key has expired
assert test_queue.get_task_key() is None
def test_large_task_batch(self, test_queue, fake):
def test_large_task_batch(self, test_queue, fake: Faker):
"""Test handling large batches of tasks."""
# Create large batch of tasks
large_batch = []
@@ -292,7 +297,7 @@ class TestTenantIsolatedTaskQueueIntegration:
assert isinstance(task, dict)
assert task["index"] == i # FIFO order
def test_queue_operations_isolation(self, test_tenant_and_account, fake):
def test_queue_operations_isolation(self, test_tenant_and_account: TenantAndAccount, fake: Faker):
"""Test concurrent operations on different queues."""
tenant, _ = test_tenant_and_account
@@ -312,7 +317,7 @@ class TestTenantIsolatedTaskQueueIntegration:
assert tasks2 == ["task1_queue2", "task2_queue2"]
assert tasks1 != tasks2
def test_task_wrapper_serialization_roundtrip(self, test_queue, fake):
def test_task_wrapper_serialization_roundtrip(self, test_queue, fake: Faker):
"""Test TaskWrapper serialization and deserialization roundtrip."""
# Create complex nested data
complex_data = {
@@ -346,7 +351,7 @@ class TestTenantIsolatedTaskQueueIntegration:
task = test_queue.pull_tasks(1)
assert task[0] == invalid_json_task
def test_real_world_batch_processing_scenario(self, test_queue, fake):
def test_real_world_batch_processing_scenario(self, test_queue, fake: Faker):
"""Test realistic batch processing scenario."""
# Simulate batch processing tasks
batch_tasks = []
@@ -403,7 +408,7 @@ class TestTenantIsolatedTaskQueueCompatibility:
return Faker()
@pytest.fixture
def test_tenant_and_account(self, db_session_with_containers, fake):
def test_tenant_and_account(self, db_session_with_containers: Session, fake: Faker):
"""Create test tenant and account for testing."""
# Create account
account = Account(
@@ -435,7 +440,7 @@ class TestTenantIsolatedTaskQueueCompatibility:
return tenant, account
def test_legacy_string_queue_compatibility(self, test_tenant_and_account, fake):
def test_legacy_string_queue_compatibility(self, test_tenant_and_account: TenantAndAccount, fake: Faker):
"""
Test compatibility with legacy queues containing only string data.
@@ -465,7 +470,7 @@ class TestTenantIsolatedTaskQueueCompatibility:
expected_order = ["legacy_task_1", "legacy_task_2", "legacy_task_3", "legacy_task_4", "legacy_task_5"]
assert pulled_tasks == expected_order
def test_legacy_queue_migration_scenario(self, test_tenant_and_account, fake):
def test_legacy_queue_migration_scenario(self, test_tenant_and_account: TenantAndAccount, fake: Faker):
"""
Test complete migration scenario from legacy to new system.
@@ -546,7 +551,7 @@ class TestTenantIsolatedTaskQueueCompatibility:
assert task["tenant_id"] == tenant.id
assert task["processing_type"] == "new_system"
def test_legacy_queue_error_recovery(self, test_tenant_and_account, fake):
def test_legacy_queue_error_recovery(self, test_tenant_and_account: TenantAndAccount, fake: Faker):
"""
Test error recovery when legacy queue contains malformed data.
@@ -3,6 +3,7 @@ from unittest.mock import patch
import pytest
from faker import Faker
from sqlalchemy.orm import Session
from core.rag.index_processor.constant.index_type import IndexStructureType, IndexTechniqueType
from core.rag.retrieval.dataset_retrieval import DatasetRetrieval
@@ -15,7 +16,7 @@ from tests.test_containers_integration_tests.helpers import generate_valid_passw
class TestGetAvailableDatasetsIntegration:
def test_returns_datasets_with_available_documents(
self, db_session_with_containers, mock_external_service_dependencies
self, db_session_with_containers: Session, mock_external_service_dependencies
):
# Arrange
fake = Faker()
@@ -77,7 +78,7 @@ class TestGetAvailableDatasetsIntegration:
assert result[0].name == dataset.name
def test_filters_out_datasets_with_only_archived_documents(
self, db_session_with_containers, mock_external_service_dependencies
self, db_session_with_containers: Session, mock_external_service_dependencies
):
# Arrange
fake = Faker()
@@ -130,7 +131,7 @@ class TestGetAvailableDatasetsIntegration:
assert len(result) == 0
def test_filters_out_datasets_with_only_disabled_documents(
self, db_session_with_containers, mock_external_service_dependencies
self, db_session_with_containers: Session, mock_external_service_dependencies
):
# Arrange
fake = Faker()
@@ -183,7 +184,7 @@ class TestGetAvailableDatasetsIntegration:
assert len(result) == 0
def test_filters_out_datasets_with_non_completed_documents(
self, db_session_with_containers, mock_external_service_dependencies
self, db_session_with_containers: Session, mock_external_service_dependencies
):
# Arrange
fake = Faker()
@@ -236,7 +237,7 @@ class TestGetAvailableDatasetsIntegration:
assert len(result) == 0
def test_includes_external_datasets_without_documents(
self, db_session_with_containers, mock_external_service_dependencies
self, db_session_with_containers: Session, mock_external_service_dependencies
):
"""
Test that external datasets are returned even with no available documents.
@@ -280,7 +281,7 @@ class TestGetAvailableDatasetsIntegration:
assert result[0].id == dataset.id
assert result[0].provider == "external"
def test_filters_by_tenant_id(self, db_session_with_containers, mock_external_service_dependencies):
def test_filters_by_tenant_id(self, db_session_with_containers: Session, mock_external_service_dependencies):
# Arrange
fake = Faker()
@@ -356,7 +357,7 @@ class TestGetAvailableDatasetsIntegration:
assert result[0].tenant_id == tenant1.id
def test_returns_empty_list_when_no_datasets_found(
self, db_session_with_containers, mock_external_service_dependencies
self, db_session_with_containers: Session, mock_external_service_dependencies
):
# Arrange
fake = Faker()
@@ -379,7 +380,9 @@ class TestGetAvailableDatasetsIntegration:
# Assert
assert result == []
def test_returns_only_requested_dataset_ids(self, db_session_with_containers, mock_external_service_dependencies):
def test_returns_only_requested_dataset_ids(
self, db_session_with_containers: Session, mock_external_service_dependencies
):
# Arrange
fake = Faker()
@@ -439,7 +442,7 @@ class TestGetAvailableDatasetsIntegration:
class TestKnowledgeRetrievalIntegration:
def test_knowledge_retrieval_with_available_datasets(
self, db_session_with_containers, mock_external_service_dependencies
self, db_session_with_containers: Session, mock_external_service_dependencies
):
# Arrange
fake = Faker()
@@ -507,7 +510,7 @@ class TestKnowledgeRetrievalIntegration:
assert isinstance(result, list)
def test_knowledge_retrieval_no_available_datasets(
self, db_session_with_containers, mock_external_service_dependencies
self, db_session_with_containers: Session, mock_external_service_dependencies
):
# Arrange
fake = Faker()
@@ -555,7 +558,7 @@ class TestKnowledgeRetrievalIntegration:
assert result == []
def test_knowledge_retrieval_rate_limit_exceeded(
self, db_session_with_containers, mock_external_service_dependencies
self, db_session_with_containers: Session, mock_external_service_dependencies
):
# Arrange
fake = Faker()
@@ -66,7 +66,7 @@ def _mock_form_repository_with_submission(action_id: str) -> HumanInputFormRepos
def _build_runtime_state(workflow_execution_id: str, app_id: str, workflow_id: str, user_id: str) -> GraphRuntimeState:
variable_pool = VariablePool.from_bootstrap(
variable_pool = VariablePool(
system_variables=build_system_variables(
workflow_execution_id=workflow_execution_id,
app_id=app_id,
@@ -102,7 +102,7 @@ def _build_graph(
start_data = StartNodeData(title="start", variables=[])
start_node = StartNode(
node_id="start",
data=start_data,
config=start_data,
graph_init_params=params,
graph_runtime_state=runtime_state,
)
@@ -117,7 +117,7 @@ def _build_graph(
)
human_node = HumanInputNode(
node_id="human",
data=human_data,
config=human_data,
graph_init_params=params,
graph_runtime_state=runtime_state,
form_repository=form_repository,
@@ -131,7 +131,7 @@ def _build_graph(
)
end_node = EndNode(
node_id="end",
data=end_data,
config=end_data,
graph_init_params=params,
graph_runtime_state=runtime_state,
)
@@ -1,4 +1,5 @@
import unittest
from __future__ import annotations
from datetime import UTC, datetime
from unittest.mock import patch
from uuid import uuid4
@@ -16,7 +17,7 @@ from models.enums import CreatorUserRole
@pytest.mark.usefixtures("flask_req_ctx_with_containers")
class TestStorageKeyLoader(unittest.TestCase):
class TestStorageKeyLoader:
"""
Integration tests for StorageKeyLoader class.
@@ -24,110 +25,82 @@ class TestStorageKeyLoader(unittest.TestCase):
with different transfer methods: LOCAL_FILE, REMOTE_URL, and TOOL_FILE.
"""
def setUp(self):
"""Set up test data before each test method."""
self.session = db.session()
self.tenant_id = str(uuid4())
self.user_id = str(uuid4())
self.conversation_id = str(uuid4())
# Create test data that will be cleaned up after each test
self.test_upload_files = []
self.test_tool_files = []
# Create StorageKeyLoader instance
self.loader = StorageKeyLoader(
self.session,
self.tenant_id,
access_controller=DatabaseFileAccessController(),
)
def tearDown(self):
"""Clean up test data after each test method."""
self.session.rollback()
# ------------------------------------------------------------------
# Per-test helpers (use db_session_with_containers as parameter)
# ------------------------------------------------------------------
@staticmethod
def _create_upload_file(
self, file_id: str | None = None, storage_key: str | None = None, tenant_id: str | None = None
session: Session,
tenant_id: str,
user_id: str,
*,
file_id: str | None = None,
storage_key: str | None = None,
override_tenant_id: str | None = None,
) -> UploadFile:
"""Helper method to create an UploadFile record for testing."""
if file_id is None:
file_id = str(uuid4())
if storage_key is None:
storage_key = f"test_storage_key_{uuid4()}"
if tenant_id is None:
tenant_id = self.tenant_id
"""Create and flush an UploadFile record for testing."""
upload_file = UploadFile(
tenant_id=tenant_id,
tenant_id=override_tenant_id if override_tenant_id is not None else tenant_id,
storage_type=StorageType.LOCAL,
key=storage_key,
key=storage_key or f"test_storage_key_{uuid4()}",
name="test_file.txt",
size=1024,
extension=".txt",
mime_type="text/plain",
created_by_role=CreatorUserRole.ACCOUNT,
created_by=self.user_id,
created_by=user_id,
created_at=datetime.now(UTC),
used=False,
)
upload_file.id = file_id
self.session.add(upload_file)
self.session.flush()
self.test_upload_files.append(upload_file)
upload_file.id = file_id or str(uuid4())
session.add(upload_file)
session.flush()
return upload_file
@staticmethod
def _create_tool_file(
self, file_id: str | None = None, file_key: str | None = None, tenant_id: str | None = None
session: Session,
tenant_id: str,
user_id: str,
conversation_id: str,
*,
file_id: str | None = None,
file_key: str | None = None,
override_tenant_id: str | None = None,
) -> ToolFile:
"""Helper method to create a ToolFile record for testing."""
if file_id is None:
file_id = str(uuid4())
if file_key is None:
file_key = f"test_file_key_{uuid4()}"
if tenant_id is None:
tenant_id = self.tenant_id
"""Create and flush a ToolFile record for testing."""
tool_file = ToolFile(
user_id=self.user_id,
tenant_id=tenant_id,
conversation_id=self.conversation_id,
file_key=file_key,
user_id=user_id,
tenant_id=override_tenant_id if override_tenant_id is not None else tenant_id,
conversation_id=conversation_id,
file_key=file_key or f"test_file_key_{uuid4()}",
mimetype="text/plain",
original_url="http://example.com/file.txt",
name="test_tool_file.txt",
size=2048,
)
tool_file.id = file_id
self.session.add(tool_file)
self.session.flush()
self.test_tool_files.append(tool_file)
tool_file.id = file_id or str(uuid4())
session.add(tool_file)
session.flush()
return tool_file
def _create_file(self, related_id: str, transfer_method: FileTransferMethod, tenant_id: str | None = None) -> File:
"""Helper method to create a File object for testing."""
if tenant_id is None:
tenant_id = self.tenant_id
# Set related_id for LOCAL_FILE and TOOL_FILE transfer methods
file_related_id = None
remote_url = None
if transfer_method in (FileTransferMethod.LOCAL_FILE, FileTransferMethod.TOOL_FILE):
file_related_id = related_id
elif transfer_method == FileTransferMethod.REMOTE_URL:
remote_url = "https://example.com/test_file.txt"
file_related_id = related_id
@staticmethod
def _create_file(
tenant_id: str,
related_id: str,
transfer_method: FileTransferMethod,
*,
override_tenant_id: str | None = None,
) -> File:
"""Build a File value-object for testing."""
remote_url = "https://example.com/test_file.txt" if transfer_method == FileTransferMethod.REMOTE_URL else None
return File(
file_id=str(uuid4()), # Generate new UUID for File.id
tenant_id=tenant_id,
file_id=str(uuid4()),
tenant_id=override_tenant_id if override_tenant_id is not None else tenant_id,
file_type=FileType.DOCUMENT,
transfer_method=transfer_method,
related_id=file_related_id,
related_id=related_id,
remote_url=remote_url,
filename="test_file.txt",
extension=".txt",
@@ -136,240 +109,280 @@ class TestStorageKeyLoader(unittest.TestCase):
storage_key="initial_key",
)
def test_load_storage_keys_local_file(self):
# ------------------------------------------------------------------
# Tests
# ------------------------------------------------------------------
def test_load_storage_keys_local_file(self, db_session_with_containers: Session):
"""Test loading storage keys for LOCAL_FILE transfer method."""
# Create test data
upload_file = self._create_upload_file()
file = self._create_file(related_id=upload_file.id, transfer_method=FileTransferMethod.LOCAL_FILE)
tenant_id = str(uuid4())
user_id = str(uuid4())
# Load storage keys
self.loader.load_storage_keys([file])
upload_file = self._create_upload_file(db_session_with_containers, tenant_id, user_id)
file = self._create_file(tenant_id, related_id=upload_file.id, transfer_method=FileTransferMethod.LOCAL_FILE)
loader = StorageKeyLoader(
db_session_with_containers, tenant_id, access_controller=DatabaseFileAccessController()
)
loader.load_storage_keys([file])
# Verify storage key was loaded correctly
assert file._storage_key == upload_file.key
def test_load_storage_keys_remote_url(self):
def test_load_storage_keys_remote_url(self, db_session_with_containers: Session):
"""Test loading storage keys for REMOTE_URL transfer method."""
# Create test data
upload_file = self._create_upload_file()
file = self._create_file(related_id=upload_file.id, transfer_method=FileTransferMethod.REMOTE_URL)
tenant_id = str(uuid4())
user_id = str(uuid4())
# Load storage keys
self.loader.load_storage_keys([file])
upload_file = self._create_upload_file(db_session_with_containers, tenant_id, user_id)
file = self._create_file(tenant_id, related_id=upload_file.id, transfer_method=FileTransferMethod.REMOTE_URL)
loader = StorageKeyLoader(
db_session_with_containers, tenant_id, access_controller=DatabaseFileAccessController()
)
loader.load_storage_keys([file])
# Verify storage key was loaded correctly
assert file._storage_key == upload_file.key
def test_load_storage_keys_tool_file(self):
def test_load_storage_keys_tool_file(self, db_session_with_containers: Session):
"""Test loading storage keys for TOOL_FILE transfer method."""
# Create test data
tool_file = self._create_tool_file()
file = self._create_file(related_id=tool_file.id, transfer_method=FileTransferMethod.TOOL_FILE)
tenant_id = str(uuid4())
user_id = str(uuid4())
conversation_id = str(uuid4())
# Load storage keys
self.loader.load_storage_keys([file])
tool_file = self._create_tool_file(db_session_with_containers, tenant_id, user_id, conversation_id)
file = self._create_file(tenant_id, related_id=tool_file.id, transfer_method=FileTransferMethod.TOOL_FILE)
loader = StorageKeyLoader(
db_session_with_containers, tenant_id, access_controller=DatabaseFileAccessController()
)
loader.load_storage_keys([file])
# Verify storage key was loaded correctly
assert file._storage_key == tool_file.file_key
def test_load_storage_keys_mixed_methods(self):
def test_load_storage_keys_mixed_methods(self, db_session_with_containers: Session):
"""Test batch loading with mixed transfer methods."""
# Create test data for different transfer methods
upload_file1 = self._create_upload_file()
upload_file2 = self._create_upload_file()
tool_file = self._create_tool_file()
tenant_id = str(uuid4())
user_id = str(uuid4())
conversation_id = str(uuid4())
file1 = self._create_file(related_id=upload_file1.id, transfer_method=FileTransferMethod.LOCAL_FILE)
file2 = self._create_file(related_id=upload_file2.id, transfer_method=FileTransferMethod.REMOTE_URL)
file3 = self._create_file(related_id=tool_file.id, transfer_method=FileTransferMethod.TOOL_FILE)
upload_file1 = self._create_upload_file(db_session_with_containers, tenant_id, user_id)
upload_file2 = self._create_upload_file(db_session_with_containers, tenant_id, user_id)
tool_file = self._create_tool_file(db_session_with_containers, tenant_id, user_id, conversation_id)
files = [file1, file2, file3]
file1 = self._create_file(tenant_id, related_id=upload_file1.id, transfer_method=FileTransferMethod.LOCAL_FILE)
file2 = self._create_file(tenant_id, related_id=upload_file2.id, transfer_method=FileTransferMethod.REMOTE_URL)
file3 = self._create_file(tenant_id, related_id=tool_file.id, transfer_method=FileTransferMethod.TOOL_FILE)
# Load storage keys
self.loader.load_storage_keys(files)
loader = StorageKeyLoader(
db_session_with_containers, tenant_id, access_controller=DatabaseFileAccessController()
)
loader.load_storage_keys([file1, file2, file3])
# Verify all storage keys were loaded correctly
assert file1._storage_key == upload_file1.key
assert file2._storage_key == upload_file2.key
assert file3._storage_key == tool_file.file_key
def test_load_storage_keys_empty_list(self):
"""Test with empty file list."""
# Should not raise any exceptions
self.loader.load_storage_keys([])
def test_load_storage_keys_empty_list(self, db_session_with_containers: Session):
"""Test with empty file list — should not raise."""
tenant_id = str(uuid4())
loader = StorageKeyLoader(
db_session_with_containers, tenant_id, access_controller=DatabaseFileAccessController()
)
loader.load_storage_keys([])
def test_load_storage_keys_ignores_legacy_file_tenant_id(self):
def test_load_storage_keys_ignores_legacy_file_tenant_id(self, db_session_with_containers: Session):
"""Legacy file tenant_id should not override the loader tenant scope."""
upload_file = self._create_upload_file()
tenant_id = str(uuid4())
user_id = str(uuid4())
upload_file = self._create_upload_file(db_session_with_containers, tenant_id, user_id)
file = self._create_file(
related_id=upload_file.id, transfer_method=FileTransferMethod.LOCAL_FILE, tenant_id=str(uuid4())
tenant_id,
related_id=upload_file.id,
transfer_method=FileTransferMethod.LOCAL_FILE,
override_tenant_id=str(uuid4()),
)
self.loader.load_storage_keys([file])
loader = StorageKeyLoader(
db_session_with_containers, tenant_id, access_controller=DatabaseFileAccessController()
)
loader.load_storage_keys([file])
assert file._storage_key == upload_file.key
def test_load_storage_keys_missing_file_id(self):
"""Test with None file.related_id."""
# Create a file with valid parameters first, then manually set related_id to None
file = self._create_file(related_id=str(uuid4()), transfer_method=FileTransferMethod.LOCAL_FILE)
def test_load_storage_keys_missing_file_id(self, db_session_with_containers: Session):
"""Test with None file.related_id — should raise ValueError."""
tenant_id = str(uuid4())
user_id = str(uuid4())
upload_file = self._create_upload_file(db_session_with_containers, tenant_id, user_id)
file = self._create_file(tenant_id, related_id=upload_file.id, transfer_method=FileTransferMethod.LOCAL_FILE)
file.related_id = None
# Should raise ValueError for None file related_id
with pytest.raises(ValueError) as context:
self.loader.load_storage_keys([file])
loader = StorageKeyLoader(
db_session_with_containers, tenant_id, access_controller=DatabaseFileAccessController()
)
with pytest.raises(ValueError, match="file id should not be None."):
loader.load_storage_keys([file])
assert str(context.value) == "file id should not be None."
def test_load_storage_keys_nonexistent_upload_file_records(self, db_session_with_containers: Session):
"""Test with missing UploadFile database records — should raise ValueError."""
tenant_id = str(uuid4())
file = self._create_file(tenant_id, related_id=str(uuid4()), transfer_method=FileTransferMethod.LOCAL_FILE)
def test_load_storage_keys_nonexistent_upload_file_records(self):
"""Test with missing UploadFile database records."""
# Create file with non-existent upload file id
non_existent_id = str(uuid4())
file = self._create_file(related_id=non_existent_id, transfer_method=FileTransferMethod.LOCAL_FILE)
# Should raise ValueError for missing record
loader = StorageKeyLoader(
db_session_with_containers, tenant_id, access_controller=DatabaseFileAccessController()
)
with pytest.raises(ValueError):
self.loader.load_storage_keys([file])
loader.load_storage_keys([file])
def test_load_storage_keys_nonexistent_tool_file_records(self):
"""Test with missing ToolFile database records."""
# Create file with non-existent tool file id
non_existent_id = str(uuid4())
file = self._create_file(related_id=non_existent_id, transfer_method=FileTransferMethod.TOOL_FILE)
def test_load_storage_keys_nonexistent_tool_file_records(self, db_session_with_containers: Session):
"""Test with missing ToolFile database records — should raise ValueError."""
tenant_id = str(uuid4())
file = self._create_file(tenant_id, related_id=str(uuid4()), transfer_method=FileTransferMethod.TOOL_FILE)
# Should raise ValueError for missing record
loader = StorageKeyLoader(
db_session_with_containers, tenant_id, access_controller=DatabaseFileAccessController()
)
with pytest.raises(ValueError):
self.loader.load_storage_keys([file])
loader.load_storage_keys([file])
def test_load_storage_keys_invalid_uuid(self):
"""Test with invalid UUID format."""
# Create a file with valid parameters first, then manually set invalid related_id
file = self._create_file(related_id=str(uuid4()), transfer_method=FileTransferMethod.LOCAL_FILE)
def test_load_storage_keys_invalid_uuid(self, db_session_with_containers: Session):
"""Test with invalid UUID format — should raise ValueError."""
tenant_id = str(uuid4())
user_id = str(uuid4())
upload_file = self._create_upload_file(db_session_with_containers, tenant_id, user_id)
file = self._create_file(tenant_id, related_id=upload_file.id, transfer_method=FileTransferMethod.LOCAL_FILE)
file.related_id = "invalid-uuid-format"
# Should raise ValueError for invalid UUID
loader = StorageKeyLoader(
db_session_with_containers, tenant_id, access_controller=DatabaseFileAccessController()
)
with pytest.raises(ValueError):
self.loader.load_storage_keys([file])
loader.load_storage_keys([file])
def test_load_storage_keys_batch_efficiency(self):
"""Test batched operations use efficient queries."""
# Create multiple files of different types
upload_files = [self._create_upload_file() for _ in range(3)]
tool_files = [self._create_tool_file() for _ in range(2)]
def test_load_storage_keys_batch_efficiency(self, db_session_with_containers: Session):
"""Batched operations should issue exactly 2 queries for mixed file types."""
tenant_id = str(uuid4())
user_id = str(uuid4())
conversation_id = str(uuid4())
files = []
files.extend(
[self._create_file(related_id=uf.id, transfer_method=FileTransferMethod.LOCAL_FILE) for uf in upload_files]
upload_files = [self._create_upload_file(db_session_with_containers, tenant_id, user_id) for _ in range(3)]
tool_files = [
self._create_tool_file(db_session_with_containers, tenant_id, user_id, conversation_id) for _ in range(2)
]
files = [
self._create_file(tenant_id, related_id=uf.id, transfer_method=FileTransferMethod.LOCAL_FILE)
for uf in upload_files
] + [
self._create_file(tenant_id, related_id=tf.id, transfer_method=FileTransferMethod.TOOL_FILE)
for tf in tool_files
]
loader = StorageKeyLoader(
db_session_with_containers, tenant_id, access_controller=DatabaseFileAccessController()
)
files.extend(
[self._create_file(related_id=tf.id, transfer_method=FileTransferMethod.TOOL_FILE) for tf in tool_files]
)
# Mock the session to count queries
with patch.object(self.session, "scalars", wraps=self.session.scalars) as mock_scalars:
self.loader.load_storage_keys(files)
# Should make exactly 2 queries (one for upload_files, one for tool_files)
with patch.object(
db_session_with_containers, "scalars", wraps=db_session_with_containers.scalars
) as mock_scalars:
loader.load_storage_keys(files)
# Exactly 2 DB round-trips: one for UploadFile, one for ToolFile
assert mock_scalars.call_count == 2
# Verify all storage keys were loaded correctly
for i, file in enumerate(files[:3]):
assert file._storage_key == upload_files[i].key
for i, file in enumerate(files[3:]):
assert file._storage_key == tool_files[i].file_key
def test_load_storage_keys_tenant_isolation(self):
"""Test that tenant isolation works correctly."""
# Create files for different tenants
def test_load_storage_keys_tenant_isolation(self, db_session_with_containers: Session):
"""Loader should not surface records belonging to a different tenant."""
tenant_id = str(uuid4())
other_tenant_id = str(uuid4())
user_id = str(uuid4())
# Create upload file for current tenant
upload_file_current = self._create_upload_file()
upload_file_current = self._create_upload_file(db_session_with_containers, tenant_id, user_id)
file_current = self._create_file(
related_id=upload_file_current.id, transfer_method=FileTransferMethod.LOCAL_FILE
tenant_id, related_id=upload_file_current.id, transfer_method=FileTransferMethod.LOCAL_FILE
)
# Create upload file for other tenant (but don't add to cleanup list)
upload_file_other = UploadFile(
tenant_id=other_tenant_id,
storage_type=StorageType.LOCAL,
key="other_tenant_key",
name="other_file.txt",
size=1024,
extension=".txt",
mime_type="text/plain",
created_by_role=CreatorUserRole.ACCOUNT,
created_by=self.user_id,
created_at=datetime.now(UTC),
used=False,
upload_file_other = self._create_upload_file(
db_session_with_containers,
tenant_id,
user_id,
override_tenant_id=other_tenant_id,
)
upload_file_other.id = str(uuid4())
self.session.add(upload_file_other)
self.session.flush()
# Create file for other tenant but try to load with current tenant's loader
file_other = self._create_file(
related_id=upload_file_other.id, transfer_method=FileTransferMethod.LOCAL_FILE, tenant_id=other_tenant_id
tenant_id,
related_id=upload_file_other.id,
transfer_method=FileTransferMethod.LOCAL_FILE,
override_tenant_id=other_tenant_id,
)
# Should raise ValueError due to tenant mismatch
with pytest.raises(ValueError) as context:
self.loader.load_storage_keys([file_other])
loader = StorageKeyLoader(
db_session_with_containers, tenant_id, access_controller=DatabaseFileAccessController()
)
assert "Upload file not found for id:" in str(context.value)
with pytest.raises(ValueError, match="Upload file not found for id:"):
loader.load_storage_keys([file_other])
# Current tenant's file should still work
self.loader.load_storage_keys([file_current])
# Current-tenant file still resolves correctly
loader.load_storage_keys([file_current])
assert file_current._storage_key == upload_file_current.key
def test_load_storage_keys_mixed_tenant_batch(self):
"""Test batch with mixed tenant files (should fail on first mismatch)."""
# Create files for current tenant
upload_file_current = self._create_upload_file()
def test_load_storage_keys_mixed_tenant_batch(self, db_session_with_containers: Session):
"""A batch containing a foreign-tenant file should fail on the mismatch."""
tenant_id = str(uuid4())
user_id = str(uuid4())
upload_file_current = self._create_upload_file(db_session_with_containers, tenant_id, user_id)
file_current = self._create_file(
related_id=upload_file_current.id, transfer_method=FileTransferMethod.LOCAL_FILE
tenant_id, related_id=upload_file_current.id, transfer_method=FileTransferMethod.LOCAL_FILE
)
# Create file for different tenant
other_tenant_id = str(uuid4())
file_other = self._create_file(
related_id=str(uuid4()), transfer_method=FileTransferMethod.LOCAL_FILE, tenant_id=other_tenant_id
tenant_id,
related_id=str(uuid4()),
transfer_method=FileTransferMethod.LOCAL_FILE,
override_tenant_id=str(uuid4()),
)
# Should raise ValueError on tenant mismatch
with pytest.raises(ValueError) as context:
self.loader.load_storage_keys([file_current, file_other])
loader = StorageKeyLoader(
db_session_with_containers, tenant_id, access_controller=DatabaseFileAccessController()
)
with pytest.raises(ValueError, match="Upload file not found for id:"):
loader.load_storage_keys([file_current, file_other])
assert "Upload file not found for id:" in str(context.value)
def test_load_storage_keys_duplicate_file_ids(self, db_session_with_containers: Session):
"""Duplicate file IDs in the batch should be handled gracefully."""
tenant_id = str(uuid4())
user_id = str(uuid4())
def test_load_storage_keys_duplicate_file_ids(self):
"""Test handling of duplicate file IDs in the batch."""
# Create upload file
upload_file = self._create_upload_file()
upload_file = self._create_upload_file(db_session_with_containers, tenant_id, user_id)
file1 = self._create_file(tenant_id, related_id=upload_file.id, transfer_method=FileTransferMethod.LOCAL_FILE)
file2 = self._create_file(tenant_id, related_id=upload_file.id, transfer_method=FileTransferMethod.LOCAL_FILE)
# Create two File objects with same related_id
file1 = self._create_file(related_id=upload_file.id, transfer_method=FileTransferMethod.LOCAL_FILE)
file2 = self._create_file(related_id=upload_file.id, transfer_method=FileTransferMethod.LOCAL_FILE)
loader = StorageKeyLoader(
db_session_with_containers, tenant_id, access_controller=DatabaseFileAccessController()
)
loader.load_storage_keys([file1, file2])
# Should handle duplicates gracefully
self.loader.load_storage_keys([file1, file2])
# Both files should have the same storage key
assert file1._storage_key == upload_file.key
assert file2._storage_key == upload_file.key
def test_load_storage_keys_session_isolation(self):
"""Test that the loader uses the provided session correctly."""
# Create test data
upload_file = self._create_upload_file()
file = self._create_file(related_id=upload_file.id, transfer_method=FileTransferMethod.LOCAL_FILE)
def test_load_storage_keys_session_isolation(self, db_session_with_containers: Session):
"""A loader backed by an uncommitted session should not see data from another session."""
tenant_id = str(uuid4())
user_id = str(uuid4())
# Create loader with different session (same underlying connection)
upload_file = self._create_upload_file(db_session_with_containers, tenant_id, user_id)
file = self._create_file(tenant_id, related_id=upload_file.id, transfer_method=FileTransferMethod.LOCAL_FILE)
# A loader with a fresh, separate session cannot see uncommitted rows from db_session_with_containers
with Session(bind=db.engine) as other_session:
other_loader = StorageKeyLoader(
other_session,
self.tenant_id,
tenant_id,
access_controller=DatabaseFileAccessController(),
)
with pytest.raises(ValueError):
@@ -8,6 +8,7 @@ Covers real Redis 7+ sharded pub/sub interactions including:
- Resource cleanup accounting via PUBSUB SHARDNUMSUB
"""
import socket
import threading
import time
import uuid
@@ -356,10 +357,17 @@ class TestShardedRedisBroadcastChannelClusterIntegration:
def _get_test_topic_name(cls) -> str:
return f"test_sharded_cluster_topic_{uuid.uuid4()}"
@staticmethod
def _resolve_announced_ip(host: str) -> str:
"""Resolve the container host name to a literal IP accepted by Redis cluster config."""
return socket.getaddrinfo(host, None, type=socket.SOCK_STREAM)[0][4][0]
@staticmethod
def _ensure_single_node_cluster(host: str, port: int) -> None:
"""Bootstrap a single-node cluster using a literal IP for Redis node advertisement."""
client = redis.Redis(host=host, port=port, decode_responses=False)
client.config_set("cluster-announce-ip", host)
announced_ip = TestShardedRedisBroadcastChannelClusterIntegration._resolve_announced_ip(host)
client.config_set("cluster-announce-ip", announced_ip)
client.config_set("cluster-announce-port", port)
slots = client.execute_command("CLUSTER", "SLOTS")
if not slots:
@@ -5,6 +5,7 @@ from unittest.mock import Mock, patch
from uuid import uuid4
import pytest
from sqlalchemy.orm import Session
from models.source import DataSourceApiKeyAuthBinding
from services.auth.api_key_auth_service import ApiKeyAuthService
@@ -31,7 +32,7 @@ class TestApiKeyAuthService:
def mock_args(self, category, provider, mock_credentials) -> dict:
return {"category": category, "provider": provider, "credentials": mock_credentials}
def _create_binding(self, db_session, *, tenant_id, category, provider, credentials=None, disabled=False):
def _create_binding(self, db_session: Session, *, tenant_id, category, provider, credentials=None, disabled=False):
binding = DataSourceApiKeyAuthBinding(
tenant_id=tenant_id,
category=category,
@@ -44,7 +45,7 @@ class TestApiKeyAuthService:
return binding
def test_get_provider_auth_list_success(
self, flask_app_with_containers, db_session_with_containers, tenant_id, category, provider
self, flask_app_with_containers, db_session_with_containers: Session, tenant_id, category, provider
):
self._create_binding(db_session_with_containers, tenant_id=tenant_id, category=category, provider=provider)
db_session_with_containers.expire_all()
@@ -56,14 +57,16 @@ class TestApiKeyAuthService:
assert len(tenant_results) == 1
assert tenant_results[0].provider == provider
def test_get_provider_auth_list_empty(self, flask_app_with_containers, db_session_with_containers, tenant_id):
def test_get_provider_auth_list_empty(
self, flask_app_with_containers, db_session_with_containers: Session, tenant_id
):
result = ApiKeyAuthService.get_provider_auth_list(tenant_id)
tenant_results = [r for r in result if r.tenant_id == tenant_id]
assert tenant_results == []
def test_get_provider_auth_list_filters_disabled(
self, flask_app_with_containers, db_session_with_containers, tenant_id, category, provider
self, flask_app_with_containers, db_session_with_containers: Session, tenant_id, category, provider
):
self._create_binding(
db_session_with_containers, tenant_id=tenant_id, category=category, provider=provider, disabled=True
@@ -78,7 +81,13 @@ class TestApiKeyAuthService:
@patch("services.auth.api_key_auth_service.ApiKeyAuthFactory")
@patch("services.auth.api_key_auth_service.encrypter")
def test_create_provider_auth_success(
self, mock_encrypter, mock_factory, flask_app_with_containers, db_session_with_containers, tenant_id, mock_args
self,
mock_encrypter,
mock_factory,
flask_app_with_containers,
db_session_with_containers: Session,
tenant_id,
mock_args,
):
mock_auth_instance = Mock()
mock_auth_instance.validate_credentials.return_value = True
@@ -97,7 +106,7 @@ class TestApiKeyAuthService:
@patch("services.auth.api_key_auth_service.ApiKeyAuthFactory")
def test_create_provider_auth_validation_failed(
self, mock_factory, flask_app_with_containers, db_session_with_containers, tenant_id, mock_args
self, mock_factory, flask_app_with_containers, db_session_with_containers: Session, tenant_id, mock_args
):
mock_auth_instance = Mock()
mock_auth_instance.validate_credentials.return_value = False
@@ -112,7 +121,13 @@ class TestApiKeyAuthService:
@patch("services.auth.api_key_auth_service.ApiKeyAuthFactory")
@patch("services.auth.api_key_auth_service.encrypter")
def test_create_provider_auth_encrypts_api_key(
self, mock_encrypter, mock_factory, flask_app_with_containers, db_session_with_containers, tenant_id, mock_args
self,
mock_encrypter,
mock_factory,
flask_app_with_containers,
db_session_with_containers: Session,
tenant_id,
mock_args,
):
mock_auth_instance = Mock()
mock_auth_instance.validate_credentials.return_value = True
@@ -128,7 +143,13 @@ class TestApiKeyAuthService:
mock_encrypter.encrypt_token.assert_called_once_with(tenant_id, original_key)
def test_get_auth_credentials_success(
self, flask_app_with_containers, db_session_with_containers, tenant_id, category, provider, mock_credentials
self,
flask_app_with_containers,
db_session_with_containers: Session,
tenant_id,
category,
provider,
mock_credentials,
):
self._create_binding(
db_session_with_containers,
@@ -144,14 +165,14 @@ class TestApiKeyAuthService:
assert result == mock_credentials
def test_get_auth_credentials_not_found(
self, flask_app_with_containers, db_session_with_containers, tenant_id, category, provider
self, flask_app_with_containers, db_session_with_containers: Session, tenant_id, category, provider
):
result = ApiKeyAuthService.get_auth_credentials(tenant_id, category, provider)
assert result is None
def test_get_auth_credentials_json_parsing(
self, flask_app_with_containers, db_session_with_containers, tenant_id, category, provider
self, flask_app_with_containers, db_session_with_containers: Session, tenant_id, category, provider
):
special_credentials = {"auth_type": "api_key", "config": {"api_key": "key_with_中文_and_special_chars_!@#$%"}}
self._create_binding(
@@ -169,7 +190,7 @@ class TestApiKeyAuthService:
assert result["config"]["api_key"] == "key_with_中文_and_special_chars_!@#$%"
def test_delete_provider_auth_success(
self, flask_app_with_containers, db_session_with_containers, tenant_id, category, provider
self, flask_app_with_containers, db_session_with_containers: Session, tenant_id, category, provider
):
binding = self._create_binding(
db_session_with_containers, tenant_id=tenant_id, category=category, provider=provider
@@ -183,7 +204,9 @@ class TestApiKeyAuthService:
remaining = db_session_with_containers.query(DataSourceApiKeyAuthBinding).filter_by(id=binding_id).first()
assert remaining is None
def test_delete_provider_auth_not_found(self, flask_app_with_containers, db_session_with_containers, tenant_id):
def test_delete_provider_auth_not_found(
self, flask_app_with_containers, db_session_with_containers: Session, tenant_id
):
# Should not raise when binding not found
ApiKeyAuthService.delete_provider_auth(tenant_id, str(uuid4()))
@@ -10,6 +10,7 @@ from uuid import uuid4
import httpx
import pytest
from sqlalchemy.orm import Session
from models.source import DataSourceApiKeyAuthBinding
from services.auth.api_key_auth_factory import ApiKeyAuthFactory
@@ -114,7 +115,7 @@ class TestAuthIntegration:
assert result2[0].tenant_id == tenant_id_2
def test_cross_tenant_access_prevention(
self, flask_app_with_containers, db_session_with_containers, tenant_id_2, category
self, flask_app_with_containers, db_session_with_containers: Session, tenant_id_2, category
):
result = ApiKeyAuthService.get_auth_credentials(tenant_id_2, category, AuthType.FIRECRAWL)
@@ -12,6 +12,7 @@ from unittest.mock import create_autospec, patch
from uuid import uuid4
import pytest
from sqlalchemy.orm import Session
from core.rag.index_processor.constant.index_type import IndexStructureType
from extensions.storage.storage_type import StorageType
@@ -273,7 +274,9 @@ class TestDocumentServicePauseDocument:
"user_id": user_id,
}
def test_pause_document_waiting_state_success(self, db_session_with_containers, mock_document_service_dependencies):
def test_pause_document_waiting_state_success(
self, db_session_with_containers: Session, mock_document_service_dependencies
):
"""
Test successful pause of document in waiting state.
@@ -310,7 +313,7 @@ class TestDocumentServicePauseDocument:
mock_document_service_dependencies["redis_client"].setnx.assert_called_once_with(expected_cache_key, "True")
def test_pause_document_indexing_state_success(
self, db_session_with_containers, mock_document_service_dependencies
self, db_session_with_containers: Session, mock_document_service_dependencies
):
"""
Test successful pause of document in indexing state.
@@ -340,7 +343,9 @@ class TestDocumentServicePauseDocument:
assert document.is_paused is True
assert document.paused_by == mock_document_service_dependencies["user_id"]
def test_pause_document_parsing_state_success(self, db_session_with_containers, mock_document_service_dependencies):
def test_pause_document_parsing_state_success(
self, db_session_with_containers: Session, mock_document_service_dependencies
):
"""
Test successful pause of document in parsing state.
@@ -367,7 +372,9 @@ class TestDocumentServicePauseDocument:
db_session_with_containers.refresh(document)
assert document.is_paused is True
def test_pause_document_completed_state_error(self, db_session_with_containers, mock_document_service_dependencies):
def test_pause_document_completed_state_error(
self, db_session_with_containers: Session, mock_document_service_dependencies
):
"""
Test error when trying to pause completed document.
@@ -396,7 +403,9 @@ class TestDocumentServicePauseDocument:
db_session_with_containers.refresh(document)
assert document.is_paused is False
def test_pause_document_error_state_error(self, db_session_with_containers, mock_document_service_dependencies):
def test_pause_document_error_state_error(
self, db_session_with_containers: Session, mock_document_service_dependencies
):
"""
Test error when trying to pause document in error state.
@@ -467,7 +476,9 @@ class TestDocumentServiceRecoverDocument:
"recover_task": mock_task,
}
def test_recover_document_paused_success(self, db_session_with_containers, mock_document_service_dependencies):
def test_recover_document_paused_success(
self, db_session_with_containers: Session, mock_document_service_dependencies
):
"""
Test successful recovery of paused document.
@@ -510,7 +521,9 @@ class TestDocumentServiceRecoverDocument:
document.dataset_id, document.id
)
def test_recover_document_not_paused_error(self, db_session_with_containers, mock_document_service_dependencies):
def test_recover_document_not_paused_error(
self, db_session_with_containers: Session, mock_document_service_dependencies
):
"""
Test error when trying to recover non-paused document.
@@ -590,7 +603,9 @@ class TestDocumentServiceRetryDocument:
"user_id": user_id,
}
def test_retry_document_single_success(self, db_session_with_containers, mock_document_service_dependencies):
def test_retry_document_single_success(
self, db_session_with_containers: Session, mock_document_service_dependencies
):
"""
Test successful retry of single document.
@@ -629,7 +644,9 @@ class TestDocumentServiceRetryDocument:
dataset.id, [document.id], mock_document_service_dependencies["user_id"]
)
def test_retry_document_multiple_success(self, db_session_with_containers, mock_document_service_dependencies):
def test_retry_document_multiple_success(
self, db_session_with_containers: Session, mock_document_service_dependencies
):
"""
Test successful retry of multiple documents.
@@ -675,7 +692,7 @@ class TestDocumentServiceRetryDocument:
)
def test_retry_document_concurrent_retry_error(
self, db_session_with_containers, mock_document_service_dependencies
self, db_session_with_containers: Session, mock_document_service_dependencies
):
"""
Test error when document is already being retried.
@@ -708,7 +725,7 @@ class TestDocumentServiceRetryDocument:
assert document.indexing_status == IndexingStatus.ERROR
def test_retry_document_missing_current_user_error(
self, db_session_with_containers, mock_document_service_dependencies
self, db_session_with_containers: Session, mock_document_service_dependencies
):
"""
Test error when current_user is missing.
@@ -794,7 +811,7 @@ class TestDocumentServiceBatchUpdateDocumentStatus:
}
def test_batch_update_document_status_enable_success(
self, db_session_with_containers, mock_document_service_dependencies
self, db_session_with_containers: Session, mock_document_service_dependencies
):
"""
Test successful batch enabling of documents.
@@ -844,7 +861,7 @@ class TestDocumentServiceBatchUpdateDocumentStatus:
assert mock_document_service_dependencies["add_task"].delay.call_count == 2
def test_batch_update_document_status_disable_success(
self, db_session_with_containers, mock_document_service_dependencies
self, db_session_with_containers: Session, mock_document_service_dependencies
):
"""
Test successful batch disabling of documents.
@@ -886,7 +903,7 @@ class TestDocumentServiceBatchUpdateDocumentStatus:
mock_document_service_dependencies["remove_task"].delay.assert_called_once_with(document.id)
def test_batch_update_document_status_archive_success(
self, db_session_with_containers, mock_document_service_dependencies
self, db_session_with_containers: Session, mock_document_service_dependencies
):
"""
Test successful batch archiving of documents.
@@ -928,7 +945,7 @@ class TestDocumentServiceBatchUpdateDocumentStatus:
mock_document_service_dependencies["remove_task"].delay.assert_called_once_with(document.id)
def test_batch_update_document_status_unarchive_success(
self, db_session_with_containers, mock_document_service_dependencies
self, db_session_with_containers: Session, mock_document_service_dependencies
):
"""
Test successful batch unarchiving of documents.
@@ -970,7 +987,7 @@ class TestDocumentServiceBatchUpdateDocumentStatus:
mock_document_service_dependencies["add_task"].delay.assert_called_once_with(document.id)
def test_batch_update_document_status_empty_list(
self, db_session_with_containers, mock_document_service_dependencies
self, db_session_with_containers: Session, mock_document_service_dependencies
):
"""
Test handling of empty document list.
@@ -996,7 +1013,7 @@ class TestDocumentServiceBatchUpdateDocumentStatus:
mock_document_service_dependencies["remove_task"].delay.assert_not_called()
def test_batch_update_document_status_document_indexing_error(
self, db_session_with_containers, mock_document_service_dependencies
self, db_session_with_containers: Session, mock_document_service_dependencies
):
"""
Test error when document is being indexed.
@@ -1073,7 +1090,7 @@ class TestDocumentServiceRenameDocument:
"current_user": mock_current_user,
}
def test_rename_document_success(self, db_session_with_containers, mock_document_service_dependencies):
def test_rename_document_success(self, db_session_with_containers: Session, mock_document_service_dependencies):
"""
Test successful document renaming.
@@ -1111,7 +1128,9 @@ class TestDocumentServiceRenameDocument:
assert result == document
assert document.name == new_name
def test_rename_document_with_built_in_fields(self, db_session_with_containers, mock_document_service_dependencies):
def test_rename_document_with_built_in_fields(
self, db_session_with_containers: Session, mock_document_service_dependencies
):
"""
Test document renaming with built-in fields enabled.
@@ -1154,7 +1173,9 @@ class TestDocumentServiceRenameDocument:
assert document.doc_metadata["document_name"] == new_name
assert document.doc_metadata["existing_key"] == "existing_value"
def test_rename_document_with_upload_file(self, db_session_with_containers, mock_document_service_dependencies):
def test_rename_document_with_upload_file(
self, db_session_with_containers: Session, mock_document_service_dependencies
):
"""
Test document renaming with associated upload file.
@@ -1202,7 +1223,7 @@ class TestDocumentServiceRenameDocument:
assert upload_file.name == new_name
def test_rename_document_dataset_not_found_error(
self, db_session_with_containers, mock_document_service_dependencies
self, db_session_with_containers: Session, mock_document_service_dependencies
):
"""
Test error when dataset is not found.
@@ -1224,7 +1245,9 @@ class TestDocumentServiceRenameDocument:
with pytest.raises(ValueError, match="Dataset not found"):
DocumentService.rename_document(dataset_id, document_id, new_name)
def test_rename_document_not_found_error(self, db_session_with_containers, mock_document_service_dependencies):
def test_rename_document_not_found_error(
self, db_session_with_containers: Session, mock_document_service_dependencies
):
"""
Test error when document is not found.
@@ -1251,7 +1274,9 @@ class TestDocumentServiceRenameDocument:
with pytest.raises(ValueError, match="Document not found"):
DocumentService.rename_document(dataset.id, document_id, new_name)
def test_rename_document_permission_error(self, db_session_with_containers, mock_document_service_dependencies):
def test_rename_document_permission_error(
self, db_session_with_containers: Session, mock_document_service_dependencies
):
"""
Test error when user lacks permission.
@@ -11,6 +11,7 @@ from uuid import uuid4
import pytest
from redis import RedisError
from sqlalchemy.orm import Session
from extensions.ext_redis import redis_client
from models.account import TenantAccountJoin
@@ -122,7 +123,7 @@ class TestSyncAccountDeletion:
mock_queue_task.assert_not_called()
def test_sync_account_deletion_multiple_workspaces(
self, flask_app_with_containers, db_session_with_containers, mock_queue_task
self, flask_app_with_containers, db_session_with_containers: Session, mock_queue_task
):
account_id = str(uuid4())
tenant_ids = [str(uuid4()) for _ in range(3)]
@@ -144,7 +145,7 @@ class TestSyncAccountDeletion:
assert queued_workspace_ids == set(tenant_ids)
def test_sync_account_deletion_no_workspaces(
self, flask_app_with_containers, db_session_with_containers, mock_queue_task
self, flask_app_with_containers, db_session_with_containers: Session, mock_queue_task
):
with patch("services.enterprise.account_deletion_sync.dify_config") as mock_config:
mock_config.ENTERPRISE_ENABLED = True
@@ -155,7 +156,7 @@ class TestSyncAccountDeletion:
mock_queue_task.assert_not_called()
def test_sync_account_deletion_partial_failure(
self, flask_app_with_containers, db_session_with_containers, mock_queue_task
self, flask_app_with_containers, db_session_with_containers: Session, mock_queue_task
):
account_id = str(uuid4())
tenant_ids = [str(uuid4()) for _ in range(3)]
@@ -180,7 +181,7 @@ class TestSyncAccountDeletion:
assert mock_queue_task.call_count == 3
def test_sync_account_deletion_all_failures(
self, flask_app_with_containers, db_session_with_containers, mock_queue_task
self, flask_app_with_containers, db_session_with_containers: Session, mock_queue_task
):
account_id = str(uuid4())
tenant_id = str(uuid4())
@@ -0,0 +1,94 @@
from __future__ import annotations
from uuid import uuid4
from sqlalchemy import func, select
from sqlalchemy.orm import Session
from models.account import TenantPluginPermission
from services.plugin.plugin_permission_service import PluginPermissionService
def _tenant_id() -> str:
return str(uuid4())
def _get_permission(session: Session, tenant_id: str) -> TenantPluginPermission | None:
session.expire_all()
stmt = select(TenantPluginPermission).where(TenantPluginPermission.tenant_id == tenant_id)
return session.scalars(stmt).one_or_none()
def _count_permissions(session: Session, tenant_id: str) -> int:
stmt = select(func.count()).select_from(TenantPluginPermission).where(TenantPluginPermission.tenant_id == tenant_id)
return session.scalar(stmt) or 0
class TestGetPermission:
"""Integration tests for PluginPermissionService.get_permission using testcontainers."""
def test_returns_permission_when_found(self, db_session_with_containers: Session):
tenant_id = _tenant_id()
permission = TenantPluginPermission(
tenant_id=tenant_id,
install_permission=TenantPluginPermission.InstallPermission.ADMINS,
debug_permission=TenantPluginPermission.DebugPermission.EVERYONE,
)
db_session_with_containers.add(permission)
db_session_with_containers.commit()
result = PluginPermissionService.get_permission(tenant_id)
assert result is not None
assert result.id == permission.id
assert result.tenant_id == tenant_id
assert result.install_permission == TenantPluginPermission.InstallPermission.ADMINS
assert result.debug_permission == TenantPluginPermission.DebugPermission.EVERYONE
def test_returns_none_when_not_found(self, db_session_with_containers: Session):
result = PluginPermissionService.get_permission(_tenant_id())
assert result is None
class TestChangePermission:
"""Integration tests for PluginPermissionService.change_permission using testcontainers."""
def test_creates_new_permission_when_not_exists(self, db_session_with_containers: Session):
tenant_id = _tenant_id()
result = PluginPermissionService.change_permission(
tenant_id,
TenantPluginPermission.InstallPermission.EVERYONE,
TenantPluginPermission.DebugPermission.EVERYONE,
)
permission = _get_permission(db_session_with_containers, tenant_id)
assert result is True
assert permission is not None
assert permission.install_permission == TenantPluginPermission.InstallPermission.EVERYONE
assert permission.debug_permission == TenantPluginPermission.DebugPermission.EVERYONE
def test_updates_existing_permission(self, db_session_with_containers: Session):
tenant_id = _tenant_id()
existing = TenantPluginPermission(
tenant_id=tenant_id,
install_permission=TenantPluginPermission.InstallPermission.EVERYONE,
debug_permission=TenantPluginPermission.DebugPermission.EVERYONE,
)
db_session_with_containers.add(existing)
db_session_with_containers.commit()
result = PluginPermissionService.change_permission(
tenant_id,
TenantPluginPermission.InstallPermission.ADMINS,
TenantPluginPermission.DebugPermission.ADMINS,
)
permission = _get_permission(db_session_with_containers, tenant_id)
assert result is True
assert permission is not None
assert permission.id == existing.id
assert permission.install_permission == TenantPluginPermission.InstallPermission.ADMINS
assert permission.debug_permission == TenantPluginPermission.DebugPermission.ADMINS
assert _count_permissions(db_session_with_containers, tenant_id) == 1
@@ -3,6 +3,8 @@ from __future__ import annotations
from unittest.mock import patch
from uuid import uuid4
from sqlalchemy.orm import Session
from models.model import App, RecommendedApp, Site
from services.recommend_app.database.database_retrieval import DatabaseRecommendAppRetrieval
from services.recommend_app.recommend_app_type import RecommendAppType
@@ -45,6 +47,7 @@ def _create_recommended_app(
*,
app_id: str,
category: str = "chat",
categories: list[str] | None = None,
language: str = "en-US",
is_listed: bool = True,
position: int = 1,
@@ -55,6 +58,7 @@ def _create_recommended_app(
copyright="copy",
privacy_policy="pp",
category=category,
categories=[category] if categories is None else categories,
language=language,
is_listed=is_listed,
position=position,
@@ -91,7 +95,7 @@ class TestDatabaseRecommendAppRetrieval:
class TestFetchRecommendedAppsFromDb:
def test_returns_apps_and_sorted_categories(self, flask_app_with_containers, db_session_with_containers):
def test_returns_apps_and_sorted_categories(self, flask_app_with_containers, db_session_with_containers: Session):
tenant_id = str(uuid4())
app1 = _create_app(db_session_with_containers, tenant_id=tenant_id)
_create_site(db_session_with_containers, app_id=app1.id)
@@ -111,7 +115,56 @@ class TestFetchRecommendedAppsFromDb:
assert "assistant" in result["categories"]
assert "writing" in result["categories"]
def test_falls_back_to_default_language_when_empty(self, flask_app_with_containers, db_session_with_containers):
def test_returns_multiple_categories_for_one_app(
self, flask_app_with_containers, db_session_with_containers: Session
):
tenant_id = str(uuid4())
created_app = _create_app(db_session_with_containers, tenant_id=tenant_id)
_create_site(db_session_with_containers, app_id=created_app.id)
_create_recommended_app(
db_session_with_containers,
app_id=created_app.id,
category="writing",
categories=["writing", "assistant"],
)
db_session_with_containers.expire_all()
result = DatabaseRecommendAppRetrieval.fetch_recommended_apps_from_db("en-US")
recommended_app = next(item for item in result["recommended_apps"] if item["app_id"] == created_app.id)
assert recommended_app["categories"] == ["writing", "assistant"]
assert "writing" in result["categories"]
assert "assistant" in result["categories"]
def test_ignores_legacy_category_when_categories_are_empty(
self,
flask_app_with_containers,
db_session_with_containers: Session,
):
legacy_category = f"legacy-empty-{uuid4()}"
tenant_id = str(uuid4())
created_app = _create_app(db_session_with_containers, tenant_id=tenant_id)
_create_site(db_session_with_containers, app_id=created_app.id)
_create_recommended_app(
db_session_with_containers,
app_id=created_app.id,
category=legacy_category,
categories=[],
)
db_session_with_containers.expire_all()
result = DatabaseRecommendAppRetrieval.fetch_recommended_apps_from_db("en-US")
recommended_app = next(item for item in result["recommended_apps"] if item["app_id"] == created_app.id)
assert "category" not in recommended_app
assert recommended_app["categories"] == []
assert legacy_category not in result["categories"]
def test_falls_back_to_default_language_when_empty(
self, flask_app_with_containers, db_session_with_containers: Session
):
tenant_id = str(uuid4())
app1 = _create_app(db_session_with_containers, tenant_id=tenant_id)
_create_site(db_session_with_containers, app_id=app1.id)
@@ -124,7 +177,7 @@ class TestFetchRecommendedAppsFromDb:
app_ids = {r["app_id"] for r in result["recommended_apps"]}
assert app1.id in app_ids
def test_skips_non_public_apps(self, flask_app_with_containers, db_session_with_containers):
def test_skips_non_public_apps(self, flask_app_with_containers, db_session_with_containers: Session):
tenant_id = str(uuid4())
app1 = _create_app(db_session_with_containers, tenant_id=tenant_id, is_public=False)
_create_site(db_session_with_containers, app_id=app1.id)
@@ -137,7 +190,7 @@ class TestFetchRecommendedAppsFromDb:
app_ids = {r["app_id"] for r in result["recommended_apps"]}
assert app1.id not in app_ids
def test_skips_apps_without_site(self, flask_app_with_containers, db_session_with_containers):
def test_skips_apps_without_site(self, flask_app_with_containers, db_session_with_containers: Session):
tenant_id = str(uuid4())
app1 = _create_app(db_session_with_containers, tenant_id=tenant_id)
_create_recommended_app(db_session_with_containers, app_id=app1.id)
@@ -151,12 +204,12 @@ class TestFetchRecommendedAppsFromDb:
class TestFetchRecommendedAppDetailFromDb:
def test_returns_none_when_not_listed(self, flask_app_with_containers, db_session_with_containers):
def test_returns_none_when_not_listed(self, flask_app_with_containers, db_session_with_containers: Session):
result = DatabaseRecommendAppRetrieval.fetch_recommended_app_detail_from_db(str(uuid4()))
assert result is None
def test_returns_none_when_app_not_public(self, flask_app_with_containers, db_session_with_containers):
def test_returns_none_when_app_not_public(self, flask_app_with_containers, db_session_with_containers: Session):
tenant_id = str(uuid4())
app1 = _create_app(db_session_with_containers, tenant_id=tenant_id, is_public=False)
_create_recommended_app(db_session_with_containers, app_id=app1.id)
@@ -168,7 +221,7 @@ class TestFetchRecommendedAppDetailFromDb:
assert result is None
@patch("services.recommend_app.database.database_retrieval.AppDslService")
def test_returns_detail_on_success(self, mock_dsl, flask_app_with_containers, db_session_with_containers):
def test_returns_detail_on_success(self, mock_dsl, flask_app_with_containers, db_session_with_containers: Session):
tenant_id = str(uuid4())
app1 = _create_app(db_session_with_containers, tenant_id=tenant_id)
_create_site(db_session_with_containers, app_id=app1.id)
@@ -2,6 +2,7 @@ import copy
import pytest
from faker import Faker
from sqlalchemy.orm import Session
from core.prompt.prompt_templates.advanced_prompt_templates import (
BAICHUAN_CHAT_APP_CHAT_PROMPT_CONFIG,
@@ -29,7 +30,9 @@ class TestAdvancedPromptTemplateService:
# for consistency with other test files
return {}
def test_get_prompt_baichuan_model_success(self, db_session_with_containers, mock_external_service_dependencies):
def test_get_prompt_baichuan_model_success(
self, db_session_with_containers: Session, mock_external_service_dependencies
):
"""
Test successful prompt generation for Baichuan model.
@@ -64,7 +67,9 @@ class TestAdvancedPromptTemplateService:
assert "{{#histories#}}" in prompt_text
assert "{{#query#}}" in prompt_text
def test_get_prompt_common_model_success(self, db_session_with_containers, mock_external_service_dependencies):
def test_get_prompt_common_model_success(
self, db_session_with_containers: Session, mock_external_service_dependencies
):
"""
Test successful prompt generation for common models.
@@ -100,7 +105,7 @@ class TestAdvancedPromptTemplateService:
assert "{{#query#}}" in prompt_text
def test_get_prompt_case_insensitive_baichuan_detection(
self, db_session_with_containers, mock_external_service_dependencies
self, db_session_with_containers: Session, mock_external_service_dependencies
):
"""
Test Baichuan model detection is case insensitive.
@@ -131,7 +136,7 @@ class TestAdvancedPromptTemplateService:
assert BAICHUAN_CONTEXT in prompt_text
def test_get_common_prompt_chat_app_completion_mode(
self, db_session_with_containers, mock_external_service_dependencies
self, db_session_with_containers: Session, mock_external_service_dependencies
):
"""
Test common prompt generation for chat app with completion mode.
@@ -161,7 +166,9 @@ class TestAdvancedPromptTemplateService:
assert "{{#histories#}}" in prompt_text
assert "{{#query#}}" in prompt_text
def test_get_common_prompt_chat_app_chat_mode(self, db_session_with_containers, mock_external_service_dependencies):
def test_get_common_prompt_chat_app_chat_mode(
self, db_session_with_containers: Session, mock_external_service_dependencies
):
"""
Test common prompt generation for chat app with chat mode.
@@ -189,7 +196,7 @@ class TestAdvancedPromptTemplateService:
assert "{{#pre_prompt#}}" in prompt_text
def test_get_common_prompt_completion_app_completion_mode(
self, db_session_with_containers, mock_external_service_dependencies
self, db_session_with_containers: Session, mock_external_service_dependencies
):
"""
Test common prompt generation for completion app with completion mode.
@@ -217,7 +224,7 @@ class TestAdvancedPromptTemplateService:
assert "{{#pre_prompt#}}" in prompt_text
def test_get_common_prompt_completion_app_chat_mode(
self, db_session_with_containers, mock_external_service_dependencies
self, db_session_with_containers: Session, mock_external_service_dependencies
):
"""
Test common prompt generation for completion app with chat mode.
@@ -245,7 +252,9 @@ class TestAdvancedPromptTemplateService:
assert CONTEXT in prompt_text
assert "{{#pre_prompt#}}" in prompt_text
def test_get_common_prompt_no_context(self, db_session_with_containers, mock_external_service_dependencies):
def test_get_common_prompt_no_context(
self, db_session_with_containers: Session, mock_external_service_dependencies
):
"""
Test common prompt generation without context.
@@ -273,7 +282,7 @@ class TestAdvancedPromptTemplateService:
assert "{{#query#}}" in prompt_text
def test_get_common_prompt_unsupported_app_mode(
self, db_session_with_containers, mock_external_service_dependencies
self, db_session_with_containers: Session, mock_external_service_dependencies
):
"""
Test common prompt generation with unsupported app mode.
@@ -291,7 +300,7 @@ class TestAdvancedPromptTemplateService:
assert result == {}
def test_get_common_prompt_unsupported_model_mode(
self, db_session_with_containers, mock_external_service_dependencies
self, db_session_with_containers: Session, mock_external_service_dependencies
):
"""
Test common prompt generation with unsupported model mode.
@@ -308,7 +317,9 @@ class TestAdvancedPromptTemplateService:
# Assert: Verify empty dict is returned
assert result == {}
def test_get_completion_prompt_with_context(self, db_session_with_containers, mock_external_service_dependencies):
def test_get_completion_prompt_with_context(
self, db_session_with_containers: Session, mock_external_service_dependencies
):
"""
Test completion prompt generation with context.
@@ -339,7 +350,7 @@ class TestAdvancedPromptTemplateService:
assert result_text == CONTEXT + original_text
def test_get_completion_prompt_without_context(
self, db_session_with_containers, mock_external_service_dependencies
self, db_session_with_containers: Session, mock_external_service_dependencies
):
"""
Test completion prompt generation without context.
@@ -368,7 +379,9 @@ class TestAdvancedPromptTemplateService:
assert result_text == original_text
assert CONTEXT not in result_text
def test_get_chat_prompt_with_context(self, db_session_with_containers, mock_external_service_dependencies):
def test_get_chat_prompt_with_context(
self, db_session_with_containers: Session, mock_external_service_dependencies
):
"""
Test chat prompt generation with context.
@@ -399,7 +412,9 @@ class TestAdvancedPromptTemplateService:
assert original_text in result_text
assert result_text == CONTEXT + original_text
def test_get_chat_prompt_without_context(self, db_session_with_containers, mock_external_service_dependencies):
def test_get_chat_prompt_without_context(
self, db_session_with_containers: Session, mock_external_service_dependencies
):
"""
Test chat prompt generation without context.
@@ -429,7 +444,7 @@ class TestAdvancedPromptTemplateService:
assert CONTEXT not in result_text
def test_get_baichuan_prompt_chat_app_completion_mode(
self, db_session_with_containers, mock_external_service_dependencies
self, db_session_with_containers: Session, mock_external_service_dependencies
):
"""
Test Baichuan prompt generation for chat app with completion mode.
@@ -460,7 +475,7 @@ class TestAdvancedPromptTemplateService:
assert "{{#query#}}" in prompt_text
def test_get_baichuan_prompt_chat_app_chat_mode(
self, db_session_with_containers, mock_external_service_dependencies
self, db_session_with_containers: Session, mock_external_service_dependencies
):
"""
Test Baichuan prompt generation for chat app with chat mode.
@@ -489,7 +504,7 @@ class TestAdvancedPromptTemplateService:
assert "{{#pre_prompt#}}" in prompt_text
def test_get_baichuan_prompt_completion_app_completion_mode(
self, db_session_with_containers, mock_external_service_dependencies
self, db_session_with_containers: Session, mock_external_service_dependencies
):
"""
Test Baichuan prompt generation for completion app with completion mode.
@@ -517,7 +532,7 @@ class TestAdvancedPromptTemplateService:
assert "{{#pre_prompt#}}" in prompt_text
def test_get_baichuan_prompt_completion_app_chat_mode(
self, db_session_with_containers, mock_external_service_dependencies
self, db_session_with_containers: Session, mock_external_service_dependencies
):
"""
Test Baichuan prompt generation for completion app with chat mode.
@@ -545,7 +560,9 @@ class TestAdvancedPromptTemplateService:
assert BAICHUAN_CONTEXT in prompt_text
assert "{{#pre_prompt#}}" in prompt_text
def test_get_baichuan_prompt_no_context(self, db_session_with_containers, mock_external_service_dependencies):
def test_get_baichuan_prompt_no_context(
self, db_session_with_containers: Session, mock_external_service_dependencies
):
"""
Test Baichuan prompt generation without context.
@@ -573,7 +590,7 @@ class TestAdvancedPromptTemplateService:
assert "{{#query#}}" in prompt_text
def test_get_baichuan_prompt_unsupported_app_mode(
self, db_session_with_containers, mock_external_service_dependencies
self, db_session_with_containers: Session, mock_external_service_dependencies
):
"""
Test Baichuan prompt generation with unsupported app mode.
@@ -591,7 +608,7 @@ class TestAdvancedPromptTemplateService:
assert result == {}
def test_get_baichuan_prompt_unsupported_model_mode(
self, db_session_with_containers, mock_external_service_dependencies
self, db_session_with_containers: Session, mock_external_service_dependencies
):
"""
Test Baichuan prompt generation with unsupported model mode.
@@ -609,7 +626,7 @@ class TestAdvancedPromptTemplateService:
assert result == {}
def test_get_prompt_all_app_modes_common_model(
self, db_session_with_containers, mock_external_service_dependencies
self, db_session_with_containers: Session, mock_external_service_dependencies
):
"""
Test prompt generation for all app modes with common model.
@@ -641,7 +658,7 @@ class TestAdvancedPromptTemplateService:
assert result != {}
def test_get_prompt_all_app_modes_baichuan_model(
self, db_session_with_containers, mock_external_service_dependencies
self, db_session_with_containers: Session, mock_external_service_dependencies
):
"""
Test prompt generation for all app modes with Baichuan model.
@@ -672,7 +689,7 @@ class TestAdvancedPromptTemplateService:
assert result is not None
assert result != {}
def test_get_prompt_edge_cases(self, db_session_with_containers, mock_external_service_dependencies):
def test_get_prompt_edge_cases(self, db_session_with_containers: Session, mock_external_service_dependencies):
"""
Test prompt generation with edge cases.
@@ -704,7 +721,7 @@ class TestAdvancedPromptTemplateService:
# Should either return a valid result or empty dict, but not crash
assert result is not None
def test_template_immutability(self, db_session_with_containers, mock_external_service_dependencies):
def test_template_immutability(self, db_session_with_containers: Session, mock_external_service_dependencies):
"""
Test that original templates are not modified.
@@ -738,7 +755,9 @@ class TestAdvancedPromptTemplateService:
assert original_completion_completion == COMPLETION_APP_COMPLETION_PROMPT_CONFIG
assert original_completion_chat == COMPLETION_APP_CHAT_PROMPT_CONFIG
def test_baichuan_template_immutability(self, db_session_with_containers, mock_external_service_dependencies):
def test_baichuan_template_immutability(
self, db_session_with_containers: Session, mock_external_service_dependencies
):
"""
Test that original Baichuan templates are not modified.
@@ -772,7 +791,9 @@ class TestAdvancedPromptTemplateService:
assert original_baichuan_completion_completion == BAICHUAN_COMPLETION_APP_COMPLETION_PROMPT_CONFIG
assert original_baichuan_completion_chat == BAICHUAN_COMPLETION_APP_CHAT_PROMPT_CONFIG
def test_context_integration_consistency(self, db_session_with_containers, mock_external_service_dependencies):
def test_context_integration_consistency(
self, db_session_with_containers: Session, mock_external_service_dependencies
):
"""
Test consistency of context integration across different scenarios.
@@ -828,7 +849,7 @@ class TestAdvancedPromptTemplateService:
assert prompt_text.startswith(CONTEXT)
def test_baichuan_context_integration_consistency(
self, db_session_with_containers, mock_external_service_dependencies
self, db_session_with_containers: Session, mock_external_service_dependencies
):
"""
Test consistency of Baichuan context integration across different scenarios.

Some files were not shown because too many files have changed in this diff Show More