Compare commits

..
Author SHA1 Message Date
GareArc f729bc87a2 feat(inner_api): add endpoint to invalidate plugin model providers cache
Enterprise installs plugins into a workspace by talking to the plugin
daemon directly, so Dify never learns that a tenant's cached model
provider list is stale and serves it for the full 24h TTL.

Expose the existing PluginService.invalidate_plugin_model_providers_cache
over the inner API so enterprise can drop the cache for the tenants it
just changed. Batching is left to the caller.
2026-07-23 20:51:52 -07:00
GareArc 160a1bc405 Merge branch 'gareth/esq1-170-rbac-for-agent' into deploy/enterprise 2026-07-22 22:51:51 -07:00
GareArc 6b8937f3e1 Merge remote-tracking branch 'origin/gareth/esq1-170-rbac-for-agent' into gareth/esq1-170-rbac-for-agent 2026-07-22 22:50:36 -07:00
GareArc d673cc96ae fix(agent): drop agent.manage gate from workflow copy-from-roster 2026-07-22 22:50:14 -07:00
GareArc 553c38b1f2 Merge branch 'gareth/esq1-170-rbac-for-agent' into deploy/enterprise 2026-07-22 22:17:03 -07:00
Xiyuan ChenandGitHub 9e74520b50 Merge branch 'main' into gareth/esq1-170-rbac-for-agent 2026-07-22 22:01:46 -07:00
GareArc c56b46a318 fix(agent): gate creators platform publish for agent apps 2026-07-22 21:58:28 -07:00
GareArc d16d2d1efa Merge remote-tracking branch 'origin/gareth/esq1-170-rbac-for-agent' into gareth/esq1-170-rbac-for-agent 2026-07-22 21:32:30 -07:00
GareArc 91126a5fda fix(agent): stack agent.manage onto agent app management and import paths
Roster composer writes keep their app_edit / snippets gate and require
workspace agent.manage in addition, instead of replacing it. Generic app
management routes (update/delete/copy/export/name/icon/site/site-enable/
api-enable/api-keys) enforce agent.manage when the target is a roster
Agent App and reject hidden workflow-only backing apps outright, which
stay usable only through the Agent runtime surfaces. Importing an Agent
DSL requires agent.manage and the denial surfaces as 403 instead of a
generic failed import.
2026-07-22 21:31:56 -07:00
GareArc 8482167f1c fix(agent): authorize workflow-only agents against their parent workflow app
The RBAC resource resolver returned the hidden runtime backing app for
workflow-only agents, which never receives a resource access policy, so
per-resource overrides on the parent workflow app were ignored. Resolve
agent_id to the policy-owning agent.app_id instead and keep
backing_app_id strictly runtime-only.
2026-07-22 21:31:43 -07:00
1835e8ef31 fix: use re.fullmatch in email validator to reject trailing newlines (#39234) (#39320)
Co-authored-by: Asuka Minato <i@asukaminato.eu.org>
2026-07-23 04:19:44 +00:00
yyhandGitHub 6cd10f5e04 Merge branch 'main' into gareth/esq1-170-rbac-for-agent 2026-07-23 12:07:51 +08:00
zyssyz123GitHubautofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2b8e40af32 fix(auth): validate account before accepting invitations (#39438)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-07-23 03:56:32 +00:00
yyhandGitHub 625ed2a0e7 fix(workflow): hide Agent V2 in chatflow (#39439) 2026-07-23 03:37:13 +00:00
林玮 (Jade Lin)GitHubautofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
45711eb6dd chore(api): cloud use presigned download url on api site (#39436)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-07-23 03:34:49 +00:00
Asuka MinatoandGitHub fd703739b5 test: use sqlite3 session in test_async_workflow_service (#38692) 2026-07-23 03:34:10 +00:00
Asuka MinatoandGitHub 82c8741c73 test: use sqlite3 session in test_hitl_service_api (#38685) 2026-07-23 03:33:01 +00:00
Asuka MinatoandGitHub 78023bf50d test: use sqlite3 session in test_conversation (#38684) 2026-07-23 03:32:17 +00:00
GareArc 1b5fbb2142 Merge remote-tracking branch 'origin/main' into gareth/esq1-170-rbac-for-agent 2026-07-22 20:24:53 -07:00
Asuka MinatoGitHubautofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
0ad985515b test: move password reset coverage to unit tests (#38936)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-07-23 03:16:06 +00:00
yunlu.wen 60885db938 Merge branch 'feat/redis-azure-managed-identity' into deploy/enterprise 2026-07-23 11:13:31 +08:00
autofix-ci[bot]andGitHub 1ba65a3431 [autofix.ci] apply automated fixes 2026-07-23 02:53:05 +00:00
David ParkandGitHub da2177c7b2 fix(api): send valid body in external knowledge validation probe (#39410) 2026-07-23 02:52:52 +00:00
GareArc 09d7e44bd1 fix(agent): gate roster composer writes with agent.manage
Require agent.manage on the composer endpoints that create or write shared
roster/normal drafts (copy-from-roster, save-to-roster, and the agent
composer PUT) for both app and snippet variants, leaving the local node and
snippet draft saves on their existing edit permissions. Hide the Make Copy
and Save to Roster controls when the user cannot manage agents.
2026-07-22 19:49:05 -07:00
8d293023cc feat: gate cloud analytics behind cookie consent (#39408)
Co-authored-by: CodingOnStar <hanxujiang@dify.com>
2026-07-23 02:40:54 +00:00
yyhGitHubautofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
6736f4005b fix(web): improve import DSL dialog states (#39432)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-07-23 02:26:59 +00:00
GareArc 67451b0ecc Merge branch 'gareth/esq1-170-rbac-for-agent' into deploy/enterprise 2026-07-22 19:08:36 -07:00
yunlu.wen cd62373421 valid pubsub url when mi is enabled 2026-07-23 09:50:08 +08:00
Stephen ZhouGitHubautofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
bd8f0104d6 feat(dataset): expose New RAG KnowledgeFS contracts (#39314)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-07-23 01:48:36 +00:00
yyh 245c6c3525 fix(web): merge agent console link styles 2026-07-23 09:37:59 +08:00
yyh 4e95e73d86 fix(web): align enterprise edition configuration 2026-07-23 09:37:51 +08:00
yyhandGitHub 018fe7a9d3 fix(dify-ui): preserve link semantics in button story (#39430) 2026-07-23 01:07:08 +00:00
GareArc fd88da9743 refactor(web): gate community-edition tip on deployment edition
Replace the enterprise_enabled system feature with an env-derived edition
signal. enterprise_enabled expanded both the Console and Web API contracts,
and via cloudSystemFeatures it made Cloud render the Community Edition
warning.

- Drop enterprise_enabled from SystemFeatureModel; regenerate contracts and
  openapi docs.
- Add NEXT_PUBLIC_ENTERPRISE_ENABLED (forwarding the existing backend
  ENTERPRISE_ENABLED) and derive IS_ENTERPRISE_EDITION / IS_COMMUNITY_EDITION.
- CommunityEditionTip renders only on community edition, so enterprise and
  cloud both correctly hide it.

Review fixes:
- edit-in-console-link: render the enabled control as an anchor instead of a
  Button-rendered link, which forced type="button" onto the <a>; export
  buttonVariants for the styling.
- agents-access-guard: named export; redirect to '/'.
- Remove the redundant @detailSidebar/agents access-guard layout.
2026-07-22 17:44:27 -07:00
非法操作GitHubautofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>WH-2099
d5788cc019 feat: support tool multi-select input (#39346)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: WH-2099 <wh2099@pm.me>
2026-07-23 00:38:54 +00:00
zyssyz123andGitHub 4da4fa72cd fix(agent): refresh stale inline preview drafts (#39415) 2026-07-22 12:54:09 +00:00
平衡世界的BOYGitHubautofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
8853ed99a5 fix: prevent RBAC role names overflowing settings panel (#39412)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-07-22 12:44:29 +00:00
Stephen ZhouGitHubautofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
34a5ca191b feat(dataset): add New Knowledge list and empty state (#39300)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-07-22 12:00:50 +00:00
Asuka MinatoandGitHub 1271cfcf3f test: use sqlite3 session in test_workflow_events_openapi (#38683) 2026-07-22 11:25:14 +00:00
Asuka MinatoandGitHub 2a0b1a8a63 test: use sqlite3 session in test_runtime_credentials (#38682) 2026-07-22 09:50:31 +00:00
Asuka MinatoandGitHub 4a2e92f43f test: use sqlite3 session in test_plugin_wraps (#38681) 2026-07-22 09:49:54 +00:00
Asuka MinatoandGitHub 84889b88ec test: use sqlite3 session in test_init_validate (#38680) 2026-07-22 09:44:29 +00:00
Asuka MinatoandGitHub 6f034d2c93 test: use sqlite3 session in test_tags (#38679) 2026-07-22 09:43:41 +00:00
f91ea49414 test: use sqlite3 session in test_app_import_api (#38674)
Co-authored-by: Byron Wang <byron@dify.ai>
2026-07-22 09:43:30 +00:00
yyhGitHubautofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
62db37c403 fix(web): improve DSL export feedback (#39409)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-07-22 09:40:47 +00:00
Asuka MinatoandGitHub de07730548 test: use sqlite3 session in test_snippet_workflow_draft_variable (#38678) 2026-07-22 09:20:51 +00:00
Asuka MinatoandGitHub 9b2aa8b216 test: use sqlite3 session in test_snippet_workflow (#38677) 2026-07-22 09:20:21 +00:00
Asuka MinatoandGitHub 8ee3a7eabc test: use sqlite3 session in test_rag_pipeline_workflow (#38676) 2026-07-22 09:19:25 +00:00
zyssyz123GitHubautofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
0f9fffb7a2 fix(agent): isolate build and preview chat conversations (#39405)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-07-22 09:17:55 +00:00
Asuka MinatoandGitHub 187501f53e test: use sqlite3 session in test_reset_encrypt_key_pair (#38673) 2026-07-22 09:17:34 +00:00
yunlu.wen c5b345636d support custom pub/sub url 2026-07-22 17:01:55 +08:00
Wu TianweiandGitHub 03fad2e041 chore: Update permission tip across i18n locales (#39406) 2026-07-22 08:54:37 +00:00
yunlu.wen 2e30c59652 fix: ssl cert configs default to None 2026-07-22 16:07:50 +08:00
yyhGitHubautofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
acb5ee29e1 test(e2e): validate generated console contracts (#39400)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-07-22 07:48:37 +00:00
autofix-ci[bot]andGitHub 6cde1271aa [autofix.ci] apply automated fixes 2026-07-22 07:39:00 +00:00
yunlu.wen 1ff2d022ac fix typing 2026-07-22 15:35:33 +08:00
林玮 (Jade Lin)andGitHub c5aadfe557 fix(api,billing): invalidate vector space cache after cleanup (#39404) 2026-07-22 07:30:16 +00:00
autofix-ci[bot]andGitHub 0064a2d8ff [autofix.ci] apply automated fixes 2026-07-22 07:23:27 +00:00
c76ff4c38c test: use sqlite3 session in test_base_app_runner (#38740)
Co-authored-by: Byron.wang <byron@dify.ai>
2026-07-22 07:21:22 +00:00
yunlu.wen 8dd62b6cef delegate get_credentials directly 2026-07-22 15:19:52 +08:00
yunlu.wen 09338631bc move validation to root 2026-07-22 15:15:36 +08:00
yyhandGitHub 4b355b7039 fix(ci): avoid duplicate post-merge e2e runs (#39401) 2026-07-22 06:45:18 +00:00
yunlu.wen 1c58bd0f15 move db validation to config layer 2026-07-22 14:37:03 +08:00
林玮 (Jade Lin)andGitHub a3654d6a7d fix(oauth): reauth after accepting invitation (#39366) 2026-07-22 05:59:04 +00:00
yyhandGitHub 855464e7b4 chore: clean up CODEOWNERS (#39398) 2026-07-22 05:34:54 +00:00
yunlu.wen 8ce3063ece add to env example 2026-07-22 13:31:52 +08:00
yunlu.wen 28a7c247b9 feat: support entraid for azure managed redis 2026-07-22 13:27:08 +08:00
yyhGitHubautofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
05a25943e7 fix(e2e): align fixtures with generated contracts (#39394)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-07-22 05:09:23 +00:00
Asuka MinatoandGitHub c5d471dbdd test: use sqlite3 session in test_model_config_api (#38675) 2026-07-22 04:53:13 +00:00
Asuka MinatoandGitHub ba449890a9 test: use sqlite3 session in test_app_runner_conversation_variables (#38687) 2026-07-22 04:53:00 +00:00
Asuka MinatoandGitHub 072953a83a test: use sqlite3 session in test_export_service (#38718) 2026-07-22 04:52:19 +00:00
Asuka MinatoandGitHub 7b95e2a75f test: use sqlite3 session in test_generator_api_missing (#38719) 2026-07-22 04:52:06 +00:00
Asuka MinatoGitHubautofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
01efc6eecb test: use SQLite sessions in service API fixtures (#38784)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-07-22 04:51:49 +00:00
Asuka MinatoandGitHub a4c7261bf9 test: use sqlite3 session in test_workflow (#38686) 2026-07-22 04:51:33 +00:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
0aa04f610e chore: bump pillow from 12.2.0 to 12.3.0 in /dify-agent (#39328)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-07-22 04:23:00 +00:00
Asuka MinatoandGitHub 66afa3bf4a test: use sqlite3 session in test_clear_free_plan_tenant_expired_logs (#38693) 2026-07-22 04:16:21 +00:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
c0b991c5f9 chore: bump google.golang.org/grpc from 1.82.0 to 1.82.1 in /dify-agent-runtime (#39386)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-07-22 04:06:51 +00:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
d2aea76dec chore: bump setuptools from 82.0.1 to 83.0.0 in /dify-agent (#39387)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-07-22 04:06:30 +00:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
bc21972878 chore: bump pyasn1 from 0.6.3 to 0.6.4 in /dify-agent (#39388)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-07-22 04:06:08 +00:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
389a7608bf chore: bump gitpython from 3.1.50 to 3.1.52 in /api (#39389)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-07-22 04:05:53 +00:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
3abbab4798 chore: bump pyasn1 from 0.6.3 to 0.6.4 in /api (#39390)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-07-22 04:05:40 +00:00
Wu TianweiandGitHub 84f3c410c4 fix(datasets): update empty state handling and permissions messaging (#39360) 2026-07-22 03:57:35 +00:00
yyhGitHubautofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
04f9026724 fix(web): show unavailable state for disabled Web Apps (#39392)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-07-22 03:32:40 +00:00
Xiyuan ChenandGitHub 3f0f57c594 fix: apply configured timeout to enterprise inner API requests (#39335) 2026-07-22 02:56:08 +00:00
Asuka MinatoandGitHub 79a5b31bc6 test: move console OAuth coverage to unit tests (#38923) 2026-07-22 02:27:55 +00:00
Asuka MinatoandGitHub f35655a5bb test: use sqlite3 session in test_human_input_forms (#38777) 2026-07-22 02:27:30 +00:00
97489d5a44 docs: note minimum Docker Compose version (#39374)
Co-authored-by: zq <zhouquan1511@163.com>
2026-07-22 02:27:30 +00:00
Asuka MinatoandGitHub e9bd8741f4 test: use sqlite3 session in test_auth_wraps (#38760) 2026-07-22 02:23:37 +00:00
Asuka MinatoandGitHub 33acaa558e test: use SQLite sessions in core workflow (#39109) 2026-07-22 02:18:56 +00:00
Asuka MinatoandGitHub b38caf3cdb test: use SQLite sessions in services core (#39111) 2026-07-22 02:15:01 +00:00
Asuka MinatoandGitHub cb2b36f1aa test: use SQLite sessions in services enterprise (#39113) 2026-07-22 02:13:52 +00:00
Asuka MinatoandGitHub 2d15743b96 test: use SQLite sessions in services misc (#39114) 2026-07-22 02:11:52 +00:00
yyhandGitHub d5e5227c6e chore(deps): upgrade Next.js to 16.2.11 (#39384) 2026-07-22 02:04:05 +00:00
yyhGitHubautofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
9b432d0d29 fix: enforce tool provider type contracts (#39380)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-07-22 01:30:55 +00:00
Wu TianweiandGitHub 5216ee1d20 feat: add WebSocket support to dev proxy with cookie rewriting (#39370) 2026-07-22 01:14:27 +00:00
ba8ce362d7 fix(api): PaginatedResult.pages should be 0 for an empty result set (#39229)
Co-authored-by: Loi Nguyen <1948922+lntutor@users.noreply.github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 01:11:14 +00:00
GareArc bddf574994 Merge branch 'gareth/esq1-170-rbac-for-agent' into deploy/enterprise 2026-07-21 16:51:42 -07:00
GareArc bfd4de159e Merge remote-tracking branch 'origin/gareth/esq1-170-rbac-for-agent' into gareth/esq1-170-rbac-for-agent 2026-07-21 16:50:02 -07:00
GareArc 2279298c1a Merge remote-tracking branch 'origin/main' into gareth/esq1-170-rbac-for-agent 2026-07-21 16:38:49 -07:00
autofix-ci[bot]andGitHub 6056646eea [autofix.ci] apply automated fixes 2026-07-21 10:06:36 +00:00
Xiyuan ChenandGitHub e2d0528e29 Merge branch 'main' into gareth/esq1-170-rbac-for-agent 2026-07-21 03:02:56 -07:00
GareArc ac87341d69 Merge remote-tracking branch 'origin/main' into gareth/esq1-170-rbac-for-agent
# Conflicts:
#	api/services/enterprise/rbac_service.py
2026-07-21 03:02:11 -07:00
GareArc 0a2ae4ba1d Merge remote-tracking branch 'origin/gareth/esq1-170-rbac-for-agent' into deploy/enterprise 2026-07-21 01:35:59 -07:00
GareArc 058fa95517 feat: expose enterprise_enabled and gate the community edition tip on it
The sandbox isolation caveat was suppressed by inspecting license.status,
which answers a different question: a lapsed enterprise licence would start
showing a community edition warning that still does not apply, because sandbox
isolation is a property of how the deployment was built, not of billing state.

ENTERPRISE_ENABLED already drives branding, webapp auth and the plugin manager
in get_system_features, but was never surfaced, leaving the frontend to infer
the edition from side effects. Expose it alongside rbac_enabled and read it
directly, which drops the license status set entirely.
2026-07-21 01:34:28 -07:00
GareArc 75cb8ea408 refactor(api): reuse the roster service's agent resolution for RBAC checks
The app-scoped RBAC gate hand-rolled a query to turn an agent_id into the id
of the App backing it, duplicating AgentRosterService.runtime_backing_app_id.
Worse, it filtered on id and tenant only, so the auth check accepted archived
Agents and sources the runtime resolver rejects -- a weaker notion of a valid
Agent than the layer it guards.

Add peek_runtime_backing_app_id, a read-only counterpart to
get_agent_runtime_app_model that never materializes the hidden backing App,
and share the eligibility filter between the two so they cannot drift.

The tenant filter was also untested: deleting it left all 50 tests passing,
since the specs stubbed the db module wholesale. Assert the resolver is called
with the caller's tenant instead, and cover the precedence and read-only
guarantees against the service directly.
2026-07-21 01:34:16 -07:00
GareArc 457e1e9187 Merge remote-tracking branch 'origin/gareth/esq1-170-rbac-for-agent' into deploy/enterprise 2026-07-21 00:22:42 -07:00
GareArc edbaaeae23 fix(api): resolve agent routes to their backing app for app-scoped RBAC 2026-07-21 00:22:24 -07:00
GareArc 79370e8c75 fix(web): hide the Agent Console link without agent.manage 2026-07-21 00:22:23 -07:00
GareArc c039a15014 Merge remote-tracking branch 'origin/gareth/esq1-170-rbac-for-agent' into deploy/enterprise 2026-07-20 22:55:37 -07:00
GareArc 66473d5d51 refactor(web): extract CommunityEditionTip and gate the build-mode caveat 2026-07-20 22:55:16 -07:00
GareArc 024ba2571e Merge remote-tracking branch 'origin/gareth/esq1-170-rbac-for-agent' into deploy/enterprise 2026-07-20 21:57:26 -07:00
GareArc 4626a9bc59 fix(web): hide the sandbox isolation disclaimer under an enterprise license 2026-07-20 21:57:09 -07:00
GareArc 51e985665f Merge remote-tracking branch 'origin/gareth/esq1-170-rbac-for-agent' into deploy/enterprise 2026-07-20 20:46:31 -07:00
GareArc 7037f2acc5 fix(web): add agent.manage permission label to all locales 2026-07-20 20:45:20 -07:00
GareArc 13063dbc10 Merge remote-tracking branch 'origin/gareth/esq1-170-rbac-for-agent' into deploy/enterprise 2026-07-20 19:27:50 -07:00
GareArc 5e48e39d10 fix(web): stop exporting the unused agent.manage key constant 2026-07-20 18:28:21 -07:00
GareArc e606156580 Merge branch 'main' into gareth/esq1-170-rbac-for-agent 2026-07-20 03:47:12 -07:00
GareArc c56ff84565 Merge branch 'gareth/esq1-170-rbac-for-agent' into deploy/enterprise 2026-07-20 02:42:28 -07:00
GareArc 525e76b577 test: port agent RBAC specs to the console state fixtures
main replaced the dataset-access atom mocks with @/test/console
fixtures (#39239), which landed after this branch was cut. Move the
agents access guard spec onto createWorkspaceStateModuleMock /
createPermissionStateModuleMock and update the main nav additions to
mockConsoleState.
2026-07-20 02:42:01 -07:00
GareArc 181f58ed9c Merge branch 'main' into gareth/esq1-170-rbac-for-agent 2026-07-20 02:31:18 -07:00
GareArc f947cbfffb test: cover the agents access guard and unbreak the roster layout suite
AgentsAccessGuard calls useRouter, which threw "invariant expected app
router to be mounted" in the existing roster layout suite. Mock the guard
there so that suite stays scoped to the feature flag, and give the guard
its own spec covering loading, redirect and pass-through.

Also assert the disabled edit-in-console control through its accessible
role, and move the legacy-key test into a class like its neighbours.
2026-07-20 02:09:31 -07:00
GareArc 87e7b5660a refactor(web): build edit-in-console control from the Button primitive
The enabled and disabled states each carried a hand-written copy of the
secondary-button class list, and the disabled state faked a button with
an aria-disabled span. Render both from dify-ui Button instead, which
already ships the secondary variant and its disabled styling.
2026-07-20 01:49:16 -07:00
GareArc 8440564007 Merge branch 'feat/agentv2-webapp-access-control' into deploy/enterprise 2026-07-20 01:19:17 -07:00
GareArc 582ddbce43 refactor(web): make main nav visibility a predicate
The visibility string union resolved through an if-ladder whose final
branch was an unlabeled fallthrough, so an unrecognized value silently
resolved to the app-deploy check. Replace it with a predicate on the
route config plus named shared predicates, and drop the unused
notDatasetOperator case.
2026-07-20 00:59:55 -07:00
yyh 222169e8d5 Merge remote-tracking branch 'origin/main' into feat/agentv2-webapp-access-control 2026-07-20 15:41:04 +08:00
yyh ac63b8c671 test(web): use complete system feature fixture 2026-07-20 15:36:56 +08:00
yyh ffe318906b fix: lint 2026-07-20 15:30:30 +08:00
yyh 787cda9ee1 fix(web): use radio semantics for app access modes 2026-07-20 15:22:38 +08:00
yyh 6b8b27962d fix(web): preserve app access selection on rerender 2026-07-20 15:19:54 +08:00
yyh 1e5a07ba21 perf(web): lazy load agent access control dialog 2026-07-20 15:19:20 +08:00
yyh 509f92ef2d refactor(web): keep app ACL state reads local 2026-07-20 15:18:37 +08:00
yyh ebe83f3976 test(web): seed agent access system features 2026-07-20 15:17:03 +08:00
yyh f2ba0a7c08 refactor(web): suspend on agent system features 2026-07-20 15:15:39 +08:00
yyh 816c6cad82 refactor(web): centralize web app access cache updates 2026-07-20 15:14:19 +08:00
yyh 4ae9028d4a fix(web): target agent backing app access control 2026-07-20 15:06:00 +08:00
yyh 2ee957bf49 fix(web): guard invalid agent access modes 2026-07-20 15:05:28 +08:00
GareArc 1f9ffc1680 Merge branch 'feat/agentv2-webapp-access-control' into deploy/enterprise 2026-07-19 23:51:58 -07:00
GareArc 96f63dfb1e refactor(web): drop access-mode map extraction no longer needed by agent card 2026-07-19 23:51:53 -07:00
GareArc 37b720fd1d Merge branch 'feat/agentv2-webapp-access-control' into deploy/enterprise 2026-07-19 23:40:42 -07:00
GareArc 2ef1b05afc Revert "fix(web): split agent access cards by container width instead of viewport breakpoint"
This reverts commit 3c460a3bec.
2026-07-19 23:40:29 -07:00
GareArc 1215fb7c25 Merge branch 'feat/agentv2-webapp-access-control' into deploy/enterprise 2026-07-19 23:38:29 -07:00
GareArc 3c460a3bec fix(web): split agent access cards by container width instead of viewport breakpoint 2026-07-19 23:38:23 -07:00
GareArc a14a746931 Merge branch 'feat/agentv2-webapp-access-control' into deploy/enterprise 2026-07-19 23:19:02 -07:00
GareArc 03a005ce2c feat(web): move agent webapp access control entry into action row button 2026-07-19 23:18:48 -07:00
GareArc dac931b5f1 feat(web): disable edit-in-console link without agent.manage 2026-07-19 23:16:55 -07:00
GareArc e284ee15a1 feat(web): redirect non-holders away from agents routes 2026-07-19 23:16:54 -07:00
GareArc 9a3968aba0 feat(web): gate Agents nav entry behind agent.manage 2026-07-19 23:10:09 -07:00
GareArc 3e8b14f8e0 feat(api): require agent.manage on module-exclusive agent endpoints 2026-07-19 23:07:21 -07:00
GareArc 5c42d0928b feat(api): add agent.manage RBAC permission and legacy fallback keys 2026-07-19 23:04:12 -07:00
GareArc d34f3af9c3 Merge remote-tracking branch 'origin/main' into deploy/enterprise 2026-07-19 22:57:47 -07:00
GareArc ae41e8d267 Merge branch 'feat/agentv2-webapp-access-control' into deploy/enterprise 2026-07-19 21:25:39 -07:00
GareArc 48c347eeb7 feat(web): add webapp access control to agent access card 2026-07-19 21:16:18 -07:00
GareArc afc9109148 refactor(web): export access mode maps, narrow AccessControl app prop 2026-07-19 20:54:34 -07:00
515 changed files with 24356 additions and 7395 deletions
@@ -32,12 +32,11 @@ Keep this skill focused on Cucumber, Playwright, and package-level E2E guidance.
- `e2e/` uses Cucumber for scenarios and Playwright as the browser layer.
- `DifyWorld` is the per-scenario context object. Type `this` as `DifyWorld` and use `async function`, not arrow functions.
- Keep glue organized by capability under `e2e/features/step-definitions/`; use `common/` only for broadly reusable steps.
- Browser session behavior comes from `features/support/hooks.ts`:
- default: authenticated session with shared storage state
- `@unauthenticated`: clean browser context
- `@authenticated`: readability/selective-run tag only unless implementation changes
- `@fresh`: only for `e2e:full*` flows
- Treat `e2e/AGENTS.md`, `features/support/hooks.ts`, and the Cucumber configuration as the owners of current session and tag semantics. Verify them when behavior depends on session state instead of copying a tag inventory into this skill.
- Do not import Playwright Test runner patterns that bypass the current Cucumber + `DifyWorld` architecture unless the task is explicitly about changing that architecture.
- Perform the behavior under test through Playwright. APIs are allowed for setup, seed preparation, persistence polling, and cleanup, but ordinary Console JSON and representable multipart operations must use the scenario- or process-owned generated oRPC client with request and response validation enabled. Keep the setup/cleanup API identity independent from an unauthenticated or logged-out behavior browser.
- Consume generated operations directly. Do not add one-to-one API wrappers, handwritten endpoint URLs, response DTO casts, duplicate schemas, global mutable clients, or TanStack Query caching in Cucumber. Keep helpers only for real fixture construction, multi-operation orchestration, invariants, polling, derived test views, or protocol adapters.
- Keep SSE, binary, redirect-only, external-service, and readiness exceptions centralized under their protocol owner. A contract mismatch must fail and be fixed at the backend schema owner followed by regeneration; never weaken validation to make E2E pass.
## Workflow
@@ -66,7 +65,7 @@ Keep this skill focused on Cucumber, Playwright, and package-level E2E guidance.
- If a product element has real user-facing semantics but no accessible name, prefer fixing that accessible contract over adding a test id.
5. Validate narrowly.
- Run the narrowest tagged scenario or flow that exercises the change.
- Run `vpr lint --fix --quiet` from the repository root and `pnpm -C e2e type-check`.
- Run the package-required static checks documented in `e2e/AGENTS.md`.
- Broaden verification only when the change affects hooks, tags, setup, or shared step semantics.
## Review Checklist
@@ -77,6 +76,8 @@ Keep this skill focused on Cucumber, Playwright, and package-level E2E guidance.
- Are locators user-facing and assertions web-first?
- Does the change introduce hidden coupling across scenarios, tags, or instance state?
- Does it document or implement behavior that differs from the real hooks or configuration?
- Does setup/cleanup use the generated client directly, with any remaining helper owning more than a one-to-one endpoint forward?
- Is every raw HTTP call a documented protocol or infrastructure exception rather than an ordinary Console operation?
Lead findings with correctness, flake risk, and architecture drift.
+17 -44
View File
@@ -8,7 +8,6 @@
# Lint bulk suppression baselines.
/oxlint-suppressions.json
/eslint-suppressions.json
# CODEOWNERS file
/.github/CODEOWNERS @laipz8200 @crazywoola
@@ -33,31 +32,9 @@
# Backend (default owner, more specific rules below will override)
/api/ @QuantumGhost
# Backend - MCP
/api/core/mcp/ @Nov1c444
/api/core/entities/mcp_provider.py @Nov1c444
/api/services/tools/mcp_tools_manage_service.py @Nov1c444
/api/controllers/mcp/ @Nov1c444
/api/controllers/console/app/mcp_server.py @Nov1c444
# Backend - Tests
/api/tests/ @laipz8200 @QuantumGhost
/api/tests/**/*mcp* @Nov1c444
# Backend - Workflow - Engine (Core graph execution engine)
/api/core/workflow/graph_engine/ @laipz8200 @QuantumGhost
/api/core/workflow/runtime/ @laipz8200 @QuantumGhost
/api/core/workflow/graph/ @laipz8200 @QuantumGhost
/api/core/workflow/graph_events/ @laipz8200 @QuantumGhost
/api/core/workflow/node_events/ @laipz8200 @QuantumGhost
# Backend - Workflow - Nodes (Agent, Iteration, Loop, LLM)
/api/core/workflow/nodes/agent/ @Nov1c444
/api/core/workflow/nodes/iteration/ @Nov1c444
/api/core/workflow/nodes/loop/ @Nov1c444
/api/core/workflow/nodes/llm/ @Nov1c444
# Backend - RAG (Retrieval Augmented Generation)
/api/core/rag/ @JohnJyong
/api/services/rag_pipeline/ @JohnJyong
@@ -111,7 +88,6 @@
/api/core/app/layers/trigger_post_layer.py @CourTeous33
/api/services/trigger/ @CourTeous33
/api/models/trigger.py @CourTeous33
/api/fields/workflow_trigger_fields.py @CourTeous33
/api/repositories/workflow_trigger_log_repository.py @CourTeous33
/api/repositories/sqlalchemy_workflow_trigger_log_repository.py @CourTeous33
/api/libs/schedule_utils.py @CourTeous33
@@ -136,11 +112,11 @@
/api/controllers/console/billing/ @hj24 @zyssyz123
# Backend - Enterprise
/api/configs/enterprise/ @GarfieldDai @GareArc
/api/services/enterprise/ @GarfieldDai @GareArc
/api/services/feature_service.py @GarfieldDai @GareArc
/api/controllers/console/feature.py @GarfieldDai @GareArc
/api/controllers/web/feature.py @GarfieldDai @GareArc
/api/configs/enterprise/ @GareArc
/api/services/enterprise/ @GareArc
/api/services/feature_service.py @GareArc
/api/controllers/console/feature.py @GareArc
/api/controllers/web/feature.py @GareArc
# Backend - Database Migrations
/api/migrations/ @snakevash @laipz8200 @MRZHUH
@@ -153,7 +129,6 @@
# Frontend - Platform and Features
/web/config/ @lyzno1
/web/contract/ @lyzno1
/web/env.ts @lyzno1
/web/features/ @lyzno1
/web/hooks/ @lyzno1
@@ -212,7 +187,6 @@
/web/app/components/rag-pipeline/store/ @iamjoel @zxhlyh
# Frontend - RAG - Documents List
/web/app/components/datasets/documents/list.tsx @iamjoel @WTW0313
/web/app/components/datasets/documents/create-from-pipeline/ @iamjoel @WTW0313
# Frontend - RAG - Segments List
@@ -231,22 +205,22 @@
/web/app/components/plugins/marketplace/ @iamjoel @Yessenia-d
# Frontend - Login and Registration
/web/app/signin/ @douxc @iamjoel
/web/app/signup/ @douxc @iamjoel
/web/app/reset-password/ @douxc @iamjoel
/web/app/install/ @douxc @iamjoel
/web/app/init/ @douxc @iamjoel
/web/app/forgot-password/ @douxc @iamjoel
/web/app/account/ @douxc @iamjoel
/web/app/signin/ @iamjoel
/web/app/signup/ @iamjoel
/web/app/reset-password/ @iamjoel
/web/app/install/ @iamjoel
/web/app/init/ @iamjoel
/web/app/forgot-password/ @iamjoel
/web/app/account/ @iamjoel
# Frontend - Service Authentication
/web/service/base.ts @douxc @iamjoel
/web/service/base.ts @iamjoel
# Frontend - WebApp Authentication and Access Control
/web/app/(shareLayout)/components/ @douxc @iamjoel
/web/app/(shareLayout)/webapp-signin/ @douxc @iamjoel
/web/app/(shareLayout)/webapp-reset-password/ @douxc @iamjoel
/web/app/components/app/app-access-control/ @douxc @iamjoel
/web/app/(shareLayout)/components/ @iamjoel
/web/app/(shareLayout)/webapp-signin/ @iamjoel
/web/app/(shareLayout)/webapp-reset-password/ @iamjoel
/web/app/components/app/app-access-control/ @iamjoel
# Frontend - Explore Page
/web/app/components/explore/ @CodingOnStar @iamjoel
@@ -265,7 +239,6 @@
/web/app/components/base/**/*.spec.tsx @hyoban @CodingOnStar
# Frontend - Utils and Hooks
/web/utils/classnames.ts @iamjoel @zxhlyh
/web/utils/time.ts @iamjoel @zxhlyh
/web/utils/format.ts @iamjoel @zxhlyh
/web/utils/clipboard.ts @iamjoel @zxhlyh
+2
View File
@@ -335,6 +335,8 @@ jobs:
- check-changes
if: needs.pre_job.outputs.should_skip != 'true' && needs.check-changes.outputs.e2e-changed == 'true'
uses: ./.github/workflows/web-e2e.yml
with:
run-external-runtime: false
secrets: inherit
web-e2e-skip:
+16
View File
@@ -26,19 +26,30 @@ jobs:
external_e2e:
- 'e2e/features/agent-v2/**'
- 'e2e/features/step-definitions/agent-v2/**'
- 'e2e/features/step-definitions/common/**'
- 'e2e/features/support/**'
- 'e2e/fixtures/auth.ts'
- 'e2e/fixtures/test-materials/**'
- 'e2e/scripts/**'
- 'e2e/support/**'
- 'e2e/cucumber.config.ts'
- 'e2e/package.json'
- 'e2e/test-env.ts'
- 'e2e/tsconfig.json'
- 'e2e/tsx-register.js'
- 'package.json'
- 'pnpm-lock.yaml'
- '.nvmrc'
- '.github/workflows/post-merge.yml'
- '.github/workflows/web-e2e.yml'
- '.github/actions/setup-web/**'
- 'docker/docker-compose.middleware.yaml'
- 'docker/envs/middleware.env.example'
- 'dify-agent/**'
- 'dify-agent-runtime/**'
- 'api/pyproject.toml'
- 'api/uv.lock'
- 'api/tests/integration_tests/.env.example'
- 'api/clients/agent_backend/**'
- 'api/core/app/apps/agent_app/**'
- 'api/core/workflow/nodes/agent_v2/**'
@@ -48,8 +59,13 @@ jobs:
- 'api/services/plugin/**'
- 'api/core/tools/**'
- 'api/services/tools/**'
- 'packages/contracts/package.json'
- 'packages/contracts/generated/api/console/agent/**'
- 'packages/contracts/generated/api/console/apps/**'
- 'packages/contracts/generated/api/console/datasets/**'
- 'packages/contracts/generated/api/console/orpc.gen.ts'
- 'packages/contracts/generated/api/console/workspaces/**'
- 'packages/contracts/generated/api/service/**'
- 'web/features/agent-v2/**'
- 'web/app/(commonLayout)/agents/**'
- 'web/app/(commonLayout)/@detailSidebar/agents/**'
+7 -4
View File
@@ -4,9 +4,9 @@ on:
workflow_call:
inputs:
run-external-runtime:
required: false
description: Run only the prepared and external runtime suite instead of the core suites.
required: true
type: boolean
default: false
permissions:
contents: read
@@ -46,6 +46,7 @@ jobs:
run: uv sync --project api --dev
- name: Run E2E support unit tests
if: ${{ !inputs.run-external-runtime }}
working-directory: ./e2e
run: vp run test:unit
@@ -54,6 +55,7 @@ jobs:
run: vp run e2e:install
- name: Run isolated source-api and built-web Cucumber E2E tests
if: ${{ !inputs.run-external-runtime }}
working-directory: ./e2e
env:
E2E_ADMIN_EMAIL: e2e-admin@example.com
@@ -64,7 +66,7 @@ jobs:
run: vp run e2e:full
- name: Preserve Chromium E2E report and logs
if: ${{ !cancelled() }}
if: ${{ !cancelled() && !inputs.run-external-runtime }}
run: |
if [[ -d e2e/cucumber-report ]]; then
mv e2e/cucumber-report e2e/cucumber-report-non-external
@@ -74,6 +76,7 @@ jobs:
fi
- name: Run WebKit keyboard and browser smoke tests
if: ${{ !inputs.run-external-runtime }}
working-directory: ./e2e
env:
E2E_ADMIN_EMAIL: e2e-admin@example.com
@@ -99,7 +102,7 @@ jobs:
vp run e2e -- --tags '@browser-smoke'
- name: Preserve WebKit E2E report and logs
if: ${{ !cancelled() }}
if: ${{ !cancelled() && !inputs.run-external-runtime }}
run: |
if [[ -d e2e/cucumber-report ]]; then
mv e2e/cucumber-report e2e/cucumber-report-webkit
+1 -1
View File
@@ -71,7 +71,7 @@ Dify is an open-source LLM app development platform. Its intuitive interface com
<br/>
The easiest way to start the Dify server is through [Docker Compose](docker/docker-compose.yaml). Before running Dify with the following commands, make sure that [Docker](https://docs.docker.com/get-docker/) and [Docker Compose](https://docs.docker.com/compose/install/) are installed on your machine:
The easiest way to start the Dify server is through [Docker Compose](docker/docker-compose.yaml). Before running Dify with the following commands, make sure that [Docker](https://docs.docker.com/get-docker/) and Docker Compose v2.24.0 or later are installed on your machine:
```bash
cd dify
+29 -2
View File
@@ -1,8 +1,17 @@
import os
from typing import Any, Literal, TypedDict, cast
from urllib.parse import parse_qsl, quote_plus
from urllib.parse import urlparse as _urlparse
from pydantic import Field, NonNegativeFloat, NonNegativeInt, PositiveFloat, PositiveInt, computed_field
from pydantic import (
Field,
NonNegativeFloat,
NonNegativeInt,
PositiveFloat,
PositiveInt,
computed_field,
model_validator,
)
from pydantic_settings import BaseSettings
from .cache.redis_config import RedisConfig
@@ -408,4 +417,22 @@ class MiddlewareConfig(
DatasetQueueMonitorConfig,
MatrixoneConfig,
):
pass
@model_validator(mode="after")
def _validate_redis_urls_db_for_azure(self):
"""Azure Managed Redis only supports db 0; reject non-zero db in Redis URLs."""
if not self.REDIS_USE_AZURE_MANAGED_IDENTITY:
return self
for url, name in (
(self.CELERY_BROKER_URL, "CELERY_BROKER_URL"),
(self.PUBSUB_REDIS_URL, "PUBSUB_REDIS_URL"),
):
if not url:
continue
db: str = _urlparse(url).path.lstrip("/") or "0"
if db != "0":
raise ValueError(
f"Azure Managed Redis only supports db 0, but {name} uses db {db}. "
"Please set the db index to 0 in your URL."
)
return self
+32 -4
View File
@@ -1,4 +1,4 @@
from pydantic import Field, NonNegativeInt, PositiveFloat, PositiveInt, field_validator
from pydantic import Field, NonNegativeInt, PositiveFloat, PositiveInt, field_validator, model_validator
from pydantic_settings import BaseSettings
@@ -42,6 +42,13 @@ class RedisConfig(BaseSettings):
default=False,
)
REDIS_USE_AZURE_MANAGED_IDENTITY: bool = Field(
description="Use Azure Managed Identity (Entra ID) for Redis authentication."
" When enabled, username/password are ignored and a token is acquired via DefaultAzureCredential."
" Requires azure-identity and redis-entraid packages.",
default=False,
)
REDIS_SSL_CERT_REQS: str = Field(
description="SSL certificate requirements (CERT_NONE, CERT_OPTIONAL, CERT_REQUIRED)",
default="CERT_NONE",
@@ -158,12 +165,33 @@ class RedisConfig(BaseSettings):
REDIS_KEEPALIVE_INTERVAL: PositiveInt = Field(default=10, description="redis keepalive interval")
REDIS_KEEPALIVE_COUNT: PositiveInt = Field(default=10, description="redis keepalive count")
@field_validator("REDIS_MAX_CONNECTIONS", mode="before")
@field_validator(
"REDIS_SSL_CA_CERTS",
"REDIS_SSL_CERTFILE",
"REDIS_SSL_KEYFILE",
"REDIS_MAX_CONNECTIONS",
mode="before",
)
@classmethod
def _empty_string_to_none_for_max_conns(cls, v):
"""Allow empty string in env/.env to mean 'unset' (None)."""
def _empty_string_to_none(cls, v):
"""Allow empty string in env/.env to mean 'unset' (None).
Particularly important for SSL file paths: an empty string would cause
redis-py to call ``ssl.SSLContext.load_verify_locations(cafile="")``
which raises ``FileNotFoundError``.
"""
if v is None:
return None
if isinstance(v, str) and v.strip() == "":
return None
return v
@model_validator(mode="after")
def _validate_azure_managed_identity(self):
"""Azure Managed Redis only supports db 0."""
if self.REDIS_USE_AZURE_MANAGED_IDENTITY and self.REDIS_DB != 0:
raise ValueError(
f"Azure Managed Redis only supports db 0, but REDIS_DB is set to {self.REDIS_DB}. "
"Please set REDIS_DB=0 when REDIS_USE_AZURE_MANAGED_IDENTITY is enabled."
)
return self
+20 -7
View File
@@ -10,6 +10,7 @@ from extensions.ext_database import db
from libs.login import current_account_with_tenant
from models.dataset import Dataset
from models.model import App
from services.agent.roster_service import AgentRosterService
from services.enterprise.rbac_service import RBACService
__all__ = ["RBACPermission", "RBACResourceScope", "enforce_rbac_access", "rbac_permission_required"]
@@ -51,7 +52,7 @@ def enforce_rbac_access(
check_resource_type = None if resource_type == RBACResourceScope.WORKSPACE else resource_type
resource_id = None
if resource_required and check_resource_type:
resource_id = _extract_resource_id(resource_type, path_args)
resource_id = _extract_resource_id(resource_type, tenant_id, path_args)
if _is_resource_owned_by_current_user(tenant_id, account_id, resource_type, resource_id):
return
allowed = RBACService.CheckAccess.check(
@@ -131,11 +132,14 @@ def _is_resource_owned_by_current_user(
return False
def _extract_resource_id(resource_type: RBACResourceScope, path_args: dict[str, object] | None = None) -> str:
def _extract_resource_id(
resource_type: RBACResourceScope, tenant_id: str, path_args: dict[str, object] | None = None
) -> str:
"""Extract the resource ID from matched path arguments.
Some legacy route classes use neutral names such as ``resource_id`` for
app/dataset resources, and Agent App routes use ``agent_id`` as the app id.
app/dataset resources, and Agent routes carry ``agent_id``, which is
resolved to the App backing that Agent.
Dataset endpoints behind a rag-pipeline route contain ``pipeline_id``
instead of ``dataset_id``. In that case we look up the associated
``Dataset`` row via ``Dataset.pipeline_id``.
@@ -146,10 +150,19 @@ def _extract_resource_id(resource_type: RBACResourceScope, path_args: dict[str,
matched_args = {**view_args, **(path_args or {})}
if resource_type == RBACResourceScope.APP:
app_id = matched_args.get("app_id") or matched_args.get("agent_id") or matched_args.get("resource_id")
if not app_id:
raise ValueError("Missing app_id in request path")
return str(app_id)
app_id = matched_args.get("app_id")
if app_id:
return str(app_id)
agent_id = matched_args.get("agent_id")
if agent_id:
authz_app_id = AgentRosterService(db.session).peek_authz_app_id(tenant_id=tenant_id, agent_id=str(agent_id))
return authz_app_id or str(agent_id)
resource_id = matched_args.get("resource_id")
if resource_id:
return str(resource_id)
raise ValueError("Missing app_id in request path")
if resource_type == RBACResourceScope.DATASET:
dataset_id = matched_args.get("dataset_id") or matched_args.get("resource_id")
@@ -230,6 +230,7 @@ class WorkflowAgentComposerSaveToRosterApi(Resource):
@account_initialization_required
@edit_permission_required
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_EDIT)
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.AGENT_MANAGE, resource_required=False)
@with_current_user_id
@with_current_tenant_id
@with_session
@@ -439,6 +440,7 @@ class SnippetAgentComposerSaveToRosterApi(Resource):
@rbac_permission_required(
RBACResourceScope.WORKSPACE, RBACPermission.SNIPPETS_CREATE_AND_MODIFY, resource_required=False
)
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.AGENT_MANAGE, resource_required=False)
@with_current_user_id
@with_current_tenant_id
@with_session
@@ -478,6 +480,7 @@ class AgentComposerApi(Resource):
@account_initialization_required
@edit_permission_required
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_EDIT)
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.AGENT_MANAGE, resource_required=False)
@with_current_user_id
@with_current_tenant_id
@with_session
+44 -1
View File
@@ -62,7 +62,7 @@ from libs.datetime_utils import parse_time_range
from libs.helper import dump_response
from libs.login import login_required
from models import Account
from models.agent import Agent, AgentStatus
from models.agent import Agent, AgentConfigDraftType, AgentStatus
from models.agent_config_entities import AgentSoulConfig
from models.enums import ApiTokenType
from models.model import ApiToken, App, IconType
@@ -266,6 +266,13 @@ class AgentDebugConversationRefreshResponse(BaseModel):
debug_conversation_message_count: int = 0
class AgentDebugConversationRefreshPayload(BaseModel):
draft_type: AgentConfigDraftType = Field(
default=AgentConfigDraftType.DEBUG_BUILD,
description="Agent draft surface whose conversation should be refreshed",
)
class AgentPublishPayload(BaseModel):
version_note: str | None = Field(default=None, description="Optional note for this published Agent version")
@@ -309,6 +316,7 @@ register_schema_models(
AgentAppCopyPayload,
AgentPublishPayload,
AgentBuildDraftCheckoutPayload,
AgentDebugConversationRefreshPayload,
ComposerSavePayload,
AgentApiStatusPayload,
AgentInviteOptionsQuery,
@@ -392,6 +400,7 @@ def _serialize_agent_app_detail(
tenant_id=app_model.tenant_id,
agent_id=agent.id,
account_id=current_user.id,
draft_type=AgentConfigDraftType.DEBUG_BUILD,
commit=False,
)
message_count = roster_service.count_agent_app_debug_conversation_messages(
@@ -439,6 +448,7 @@ def _serialize_agent_app_pagination(session: Session, app_pagination, *, tenant_
tenant_id=tenant_id,
agents=list(agents_by_app_id.values()),
account_id=current_user.id,
draft_type=AgentConfigDraftType.DEBUG_BUILD,
)
payload = AgentAppPagination.model_validate(
app_pagination,
@@ -542,6 +552,7 @@ class AgentAppListApi(Resource):
@setup_required
@login_required
@account_initialization_required
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.AGENT_MANAGE, resource_required=False)
@with_current_user
@with_current_tenant_id
@with_session
@@ -579,6 +590,7 @@ class AgentAppListApi(Resource):
@login_required
@account_initialization_required
@edit_permission_required
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.AGENT_MANAGE, resource_required=False)
@with_current_user
@with_current_tenant_id
@with_session
@@ -620,6 +632,7 @@ class AgentAppApi(Resource):
@login_required
@account_initialization_required
@edit_permission_required
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.AGENT_MANAGE, resource_required=False)
@with_current_user
@with_current_tenant_id
@with_session
@@ -645,6 +658,7 @@ class AgentAppApi(Resource):
@login_required
@account_initialization_required
@edit_permission_required
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.AGENT_MANAGE, resource_required=False)
@with_current_tenant_id
@with_session
def delete(self, session: Session, tenant_id: str, agent_id: UUID):
@@ -655,6 +669,16 @@ class AgentAppApi(Resource):
@console_ns.route("/agent/<uuid:agent_id>/debug-conversation/refresh")
class AgentDebugConversationRefreshApi(Resource):
@console_ns.expect(console_ns.models[AgentDebugConversationRefreshPayload.__name__])
@console_ns.doc(
params={
"payload": {
"in": "body",
"required": False,
"schema": {"$ref": f"#/components/schemas/{AgentDebugConversationRefreshPayload.__name__}"},
}
}
)
@console_ns.response(
200,
"Agent debug conversation refreshed",
@@ -669,10 +693,12 @@ class AgentDebugConversationRefreshApi(Resource):
@with_current_tenant_id
@with_session
def post(self, session: Session, tenant_id: str, current_user: Account, agent_id: UUID):
args = AgentDebugConversationRefreshPayload.model_validate(request.get_json(silent=True) or {})
debug_conversation_id = _agent_roster_service(session).refresh_agent_app_debug_conversation_id(
tenant_id=tenant_id,
agent_id=str(agent_id),
account_id=current_user.id,
draft_type=args.draft_type,
)
return AgentDebugConversationRefreshResponse(
debug_conversation_id=debug_conversation_id,
@@ -690,6 +716,7 @@ class AgentPublishApi(Resource):
@login_required
@account_initialization_required
@edit_permission_required
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.AGENT_MANAGE, resource_required=False)
@with_current_user
@with_current_tenant_id
@with_session
@@ -712,6 +739,7 @@ class AgentBuildDraftCheckoutApi(Resource):
@login_required
@account_initialization_required
@edit_permission_required
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.AGENT_MANAGE, resource_required=False)
@with_current_user
@with_current_tenant_id
@with_session
@@ -729,6 +757,7 @@ class AgentBuildDraftCheckoutApi(Resource):
@console_ns.route("/agent/<uuid:agent_id>/build-draft")
class AgentBuildDraftApi(Resource):
@console_ns.response(200, "Agent build draft", console_ns.models[AgentBuildDraftResponse.__name__])
@console_ns.response(404, "Agent build draft not found")
@setup_required
@login_required
@account_initialization_required
@@ -787,6 +816,7 @@ class AgentBuildDraftApplyApi(Resource):
@login_required
@account_initialization_required
@edit_permission_required
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.AGENT_MANAGE, resource_required=False)
@with_current_user
@with_current_tenant_id
@with_session
@@ -809,6 +839,7 @@ class AgentAppCopyApi(Resource):
@login_required
@account_initialization_required
@edit_permission_required
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.AGENT_MANAGE, resource_required=False)
@with_current_user
@with_current_tenant_id
@with_session
@@ -834,6 +865,7 @@ class AgentApiAccessApi(Resource):
@setup_required
@login_required
@account_initialization_required
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.AGENT_MANAGE, resource_required=False)
@with_current_tenant_id
@with_session(write=False)
def get(self, session: Session, tenant_id: str, agent_id: UUID):
@@ -850,6 +882,7 @@ class AgentApiStatusApi(Resource):
@login_required
@is_admin_or_owner_required
@account_initialization_required
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.AGENT_MANAGE, resource_required=False)
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_RELEASE_AND_VERSION)
@with_current_tenant_id
@with_session
@@ -868,6 +901,7 @@ class AgentApiKeyListApi(BaseApiKeyListResource):
token_prefix = "app-"
@console_ns.response(200, "Agent service API keys", console_ns.models[ApiKeyList.__name__])
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.AGENT_MANAGE, resource_required=False)
@with_current_tenant_id
@with_session(write=False)
def get(self, session: Session, tenant_id: str, agent_id: UUID) -> dict[str, object]:
@@ -878,6 +912,7 @@ class AgentApiKeyListApi(BaseApiKeyListResource):
@console_ns.response(400, "Maximum keys exceeded")
@with_current_tenant_id
@edit_permission_required
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.AGENT_MANAGE, resource_required=False)
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_RELEASE_AND_VERSION)
@with_session
def post(self, session: Session, tenant_id: str, agent_id: UUID) -> tuple[dict[str, object], int]:
@@ -897,6 +932,7 @@ class AgentApiKeyApi(BaseApiKeyResource):
@console_ns.response(204, "Agent service API key deleted")
@with_current_user
@with_current_tenant_id
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.AGENT_MANAGE, resource_required=False)
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_RELEASE_AND_VERSION)
@with_session
def delete(
@@ -942,6 +978,7 @@ class AgentLogsApi(Resource):
@setup_required
@login_required
@account_initialization_required
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.AGENT_MANAGE, resource_required=False)
@with_current_user
@with_current_tenant_id
@with_session(write=False)
@@ -980,6 +1017,7 @@ class AgentLogMessagesApi(Resource):
@setup_required
@login_required
@account_initialization_required
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.AGENT_MANAGE, resource_required=False)
@with_current_user
@with_current_tenant_id
@with_session(write=False)
@@ -1018,6 +1056,7 @@ class AgentLogSourcesApi(Resource):
@setup_required
@login_required
@account_initialization_required
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.AGENT_MANAGE, resource_required=False)
@with_current_user
@with_current_tenant_id
@with_session(write=False)
@@ -1038,6 +1077,7 @@ class AgentStatisticsSummaryApi(Resource):
@setup_required
@login_required
@account_initialization_required
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.AGENT_MANAGE, resource_required=False)
@with_current_user
@with_current_tenant_id
@with_session(write=False)
@@ -1063,6 +1103,7 @@ class AgentRosterVersionsApi(Resource):
@setup_required
@login_required
@account_initialization_required
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.AGENT_MANAGE, resource_required=False)
@with_current_tenant_id
@with_session(write=False)
def get(self, session: Session, tenant_id: str, agent_id: UUID):
@@ -1078,6 +1119,7 @@ class AgentRosterVersionDetailApi(Resource):
@setup_required
@login_required
@account_initialization_required
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.AGENT_MANAGE, resource_required=False)
@with_current_tenant_id
@with_session(write=False)
def get(self, session: Session, tenant_id: str, agent_id: UUID, version_id: UUID):
@@ -1098,6 +1140,7 @@ class AgentRosterVersionRestoreApi(Resource):
@login_required
@account_initialization_required
@edit_permission_required
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.AGENT_MANAGE, resource_required=False)
@with_current_user
@with_current_tenant_id
@with_session
+4
View File
@@ -12,6 +12,7 @@ from werkzeug.exceptions import Forbidden
from configs import dify_config
from controllers.common.schema import register_response_schema_models
from controllers.common.session import with_session
from controllers.console.app.wraps import agent_manage_required_for_agent_app
from fields.base import ResponseModel
from libs.helper import dump_response, to_timestamp
from libs.login import login_required
@@ -194,6 +195,7 @@ class AppApiKeyListResource(BaseApiKeyListResource):
@console_ns.doc(params={"resource_id": "App ID"})
@console_ns.response(200, "API keys retrieved successfully", console_ns.models[ApiKeyList.__name__])
@with_current_tenant_id
@agent_manage_required_for_agent_app
@with_session(write=False)
def get(self, session: Session, current_tenant_id: str, resource_id: UUID) -> dict[str, object]:
"""Get all API keys for an app"""
@@ -210,6 +212,7 @@ class AppApiKeyListResource(BaseApiKeyListResource):
@with_current_tenant_id
@edit_permission_required
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_RELEASE_AND_VERSION)
@agent_manage_required_for_agent_app
@with_session
def post(self, session: Session, current_tenant_id: str, resource_id: UUID) -> tuple[dict[str, object], int]:
"""Create a new API key for an app"""
@@ -233,6 +236,7 @@ class AppApiKeyResource(BaseApiKeyResource):
@with_current_user
@with_current_tenant_id
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_RELEASE_AND_VERSION)
@agent_manage_required_for_agent_app
@with_session
def delete(
self,
+34 -17
View File
@@ -9,7 +9,7 @@ from flask_restx import Resource
from pydantic import AliasChoices, BaseModel, Field, ValidationInfo, computed_field, field_validator, model_validator
from sqlalchemy import select
from sqlalchemy.orm import Session
from werkzeug.exceptions import BadRequest, NotFound
from werkzeug.exceptions import BadRequest, Forbidden, NotFound
from configs import dify_config
from controllers.common.app_access import resolve_app_access_filter
@@ -23,7 +23,7 @@ from controllers.common.schema import (
register_schema_models,
)
from controllers.console import console_ns
from controllers.console.app.wraps import get_app_model, with_session
from controllers.console.app.wraps import agent_manage_required_for_agent_app, get_app_model, with_session
from controllers.console.workspace.models import LoadBalancingPayload
from controllers.console.wraps import (
RBACPermission,
@@ -75,6 +75,7 @@ from services.entities.knowledge_entities.knowledge_entities import (
WeightModel,
WeightVectorSetting,
)
from services.errors.account import NoPermissionError
from services.feature_service import FeatureService
from tasks.initialize_created_app_rbac_access_task import initialize_created_app_rbac_access_task
@@ -246,7 +247,7 @@ class ModelConfigPartial(ResponseModel):
return to_timestamp(value)
class ModelConfig(ResponseModel):
class AppModelConfigResponse(ResponseModel):
opening_statement: str | None = None
suggested_questions: Any | None = Field(
default=None, validation_alias=AliasChoices("suggested_questions_list", "suggested_questions")
@@ -419,7 +420,7 @@ class AppDetail(AppResponseModel):
icon_background: str | None = None
enable_site: bool
enable_api: bool
model_config_: ModelConfig | None = Field(
model_config_: AppModelConfigResponse | None = Field(
default=None,
validation_alias=AliasChoices("app_model_config", "model_config"),
alias="model_config",
@@ -525,7 +526,13 @@ def _enrich_app_list_items(session: Session, *, apps: Sequence[App], tenant_id:
register_enum_models(console_ns, RetrievalMethod, WorkflowExecutionStatus, DatasetPermissionEnum)
register_response_schema_models(
console_ns, RedirectUrlResponse, SimpleResultResponse, AppImportResponse, AppTraceResponse
console_ns,
RedirectUrlResponse,
SimpleResultResponse,
AppImportResponse,
AppTraceResponse,
AppModelConfigResponse,
AppDetail,
)
register_schema_models(
@@ -544,10 +551,8 @@ register_schema_models(
Tag,
WorkflowPartial,
ModelConfigPartial,
ModelConfig,
AppDetailSiteResponse,
DeletedTool,
AppDetail,
AppExportResponse,
Segmentation,
PreProcessingRule,
@@ -823,6 +828,7 @@ class AppApi(Resource):
@account_initialization_required
@edit_permission_required
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_EDIT)
@agent_manage_required_for_agent_app
@with_session
@get_app_model(mode=None)
def put(self, session: Session, app_model: App):
@@ -857,6 +863,7 @@ class AppApi(Resource):
@account_initialization_required
@edit_permission_required
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_DELETE)
@agent_manage_required_for_agent_app
@with_session
@get_app_model
def delete(self, session: Session, app_model: App):
@@ -881,6 +888,7 @@ class AppCopyApi(Resource):
@account_initialization_required
@edit_permission_required
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_CREATE_AND_MANAGEMENT)
@agent_manage_required_for_agent_app
@with_current_user
@with_current_tenant_id
@get_app_model(mode=None)
@@ -892,16 +900,19 @@ class AppCopyApi(Resource):
with Session(db.engine, expire_on_commit=False) as session:
import_service = AppDslService(session)
yaml_content = import_service.export_dsl(app_model=app_model, session=session, include_secret=True)
result = import_service.import_app(
account=current_user,
import_mode=ImportMode.YAML_CONTENT,
yaml_content=yaml_content,
name=args.name,
description=args.description,
icon_type=args.icon_type,
icon=args.icon,
icon_background=args.icon_background,
)
try:
result = import_service.import_app(
account=current_user,
import_mode=ImportMode.YAML_CONTENT,
yaml_content=yaml_content,
name=args.name,
description=args.description,
icon_type=args.icon_type,
icon=args.icon,
icon_background=args.icon_background,
)
except NoPermissionError as e:
raise Forbidden(str(e))
if result.status == ImportStatus.FAILED:
session.rollback()
return dump_response(AppImportResponse, result), 400
@@ -955,6 +966,7 @@ class AppExportApi(Resource):
@account_initialization_required
@edit_permission_required
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_IMPORT_EXPORT_DSL)
@agent_manage_required_for_agent_app
@get_app_model
def get(self, app_model: App):
"""Export app"""
@@ -979,6 +991,7 @@ class AppPublishToCreatorsPlatformApi(Resource):
@account_initialization_required
@edit_permission_required
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_IMPORT_EXPORT_DSL)
@agent_manage_required_for_agent_app
@with_current_user_id
@get_app_model(mode=None)
def post(self, current_user_id: str, app_model: App):
@@ -1009,6 +1022,7 @@ class AppNameApi(Resource):
@account_initialization_required
@edit_permission_required
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_EDIT)
@agent_manage_required_for_agent_app
@with_session
@get_app_model(mode=None)
def post(self, session: Session, app_model: App):
@@ -1036,6 +1050,7 @@ class AppIconApi(Resource):
@account_initialization_required
@edit_permission_required
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_EDIT)
@agent_manage_required_for_agent_app
@with_session
@get_app_model(mode=None)
def post(self, session: Session, app_model: App):
@@ -1069,6 +1084,7 @@ class AppSiteStatus(Resource):
@account_initialization_required
@edit_permission_required
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_RELEASE_AND_VERSION)
@agent_manage_required_for_agent_app
@with_session
@get_app_model(mode=None)
def post(self, session: Session, app_model: App):
@@ -1096,6 +1112,7 @@ class AppApiStatus(Resource):
@is_admin_or_owner_required
@account_initialization_required
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_RELEASE_AND_VERSION)
@agent_manage_required_for_agent_app
@with_session
@get_app_model(mode=None)
def post(self, session: Session, app_model: App):
+21 -13
View File
@@ -1,6 +1,7 @@
from flask_restx import Resource
from pydantic import BaseModel, Field
from sqlalchemy.orm import Session
from werkzeug.exceptions import Forbidden
from configs import dify_config
from controllers.common.schema import register_enum_models, register_schema_models
@@ -28,6 +29,7 @@ from services.app_dsl_service import (
)
from services.enterprise.enterprise_service import EnterpriseService
from services.entities.dsl_entities import CheckDependenciesResult, ImportStatus
from services.errors.account import NoPermissionError
from services.feature_service import FeatureService
from .. import console_ns
@@ -91,18 +93,21 @@ class AppImportApi(Resource):
import_service = AppDslService(session)
# Import app
account = current_user
result = import_service.import_app(
account=account,
import_mode=args.mode,
yaml_content=args.yaml_content,
yaml_url=args.yaml_url,
name=args.name,
description=args.description,
icon_type=args.icon_type,
icon=args.icon,
icon_background=args.icon_background,
app_id=args.app_id,
)
try:
result = import_service.import_app(
account=account,
import_mode=args.mode,
yaml_content=args.yaml_content,
yaml_url=args.yaml_url,
name=args.name,
description=args.description,
icon_type=args.icon_type,
icon=args.icon,
icon_background=args.icon_background,
app_id=args.app_id,
)
except NoPermissionError as e:
raise Forbidden(str(e))
if result.status == ImportStatus.FAILED:
session.rollback()
else:
@@ -157,7 +162,10 @@ class AppImportConfirmApi(Resource):
import_service = AppDslService(session)
# Confirm import
account = current_user
result = import_service.confirm_import(import_id=import_id, account=account)
try:
result = import_service.confirm_import(import_id=import_id, account=account)
except NoPermissionError as e:
raise Forbidden(str(e))
if result.status == ImportStatus.FAILED:
session.rollback()
else:
+14 -1
View File
@@ -49,6 +49,7 @@ from libs import helper
from libs.helper import uuid_value
from libs.login import login_required
from models import Account
from models.agent import AgentConfigDraftType
from models.model import App, AppMode
from services.agent.errors import AgentNotFoundError
from services.agent.roster_service import AgentRosterService
@@ -343,14 +344,23 @@ class AgentChatMessageStopApi(Resource):
def _resolve_current_user_agent_debug_conversation_id(
*, session: Session, current_tenant_id: str, current_user: Account, app_model: App, agent_id: str | None
*,
session: Session,
current_tenant_id: str,
current_user: Account,
app_model: App,
agent_id: str | None,
draft_type: AgentConfigDraftType,
) -> str:
"""Resolve the current editor's conversation without crossing draft surfaces."""
roster_service = AgentRosterService(session)
if agent_id:
return roster_service.get_or_create_agent_app_debug_conversation_id(
tenant_id=current_tenant_id,
agent_id=agent_id,
account_id=current_user.id,
draft_type=draft_type,
)
agent = roster_service.get_app_backing_agent(tenant_id=current_tenant_id, app_id=str(app_model.id))
@@ -360,6 +370,7 @@ def _resolve_current_user_agent_debug_conversation_id(
tenant_id=current_tenant_id,
agent_id=agent.id,
account_id=current_user.id,
draft_type=draft_type,
)
@@ -382,6 +393,7 @@ def _create_chat_message(
current_user=current_user,
app_model=app_model,
agent_id=agent_id,
draft_type=AgentConfigDraftType(args_model.draft_type),
)
if args_model.conversation_id and args_model.conversation_id != debug_conversation_id:
raise NotFound("Conversation Not Exists.")
@@ -418,6 +430,7 @@ def _create_build_chat_finalization_message(
current_user=current_user,
app_model=app_model,
agent_id=agent_id,
draft_type=AgentConfigDraftType.DEBUG_BUILD,
)
args: dict[str, Any] = {
"query": _BUILD_CHAT_FINALIZATION_QUERY,
+3 -1
View File
@@ -10,7 +10,7 @@ from constants.languages import supported_language
from controllers.common.schema import register_schema_models
from controllers.common.session import with_session
from controllers.console import console_ns
from controllers.console.app.wraps import get_app_model
from controllers.console.app.wraps import agent_manage_required_for_agent_app, get_app_model
from controllers.console.wraps import (
RBACPermission,
RBACResourceScope,
@@ -93,6 +93,7 @@ class AppSite(Resource):
@login_required
@edit_permission_required
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_RELEASE_AND_VERSION)
@agent_manage_required_for_agent_app
@account_initialization_required
@with_current_user
@with_session
@@ -145,6 +146,7 @@ class AppSiteAccessTokenReset(Resource):
@login_required
@is_admin_or_owner_required
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_RELEASE_AND_VERSION)
@agent_manage_required_for_agent_app
@account_initialization_required
@with_current_user
@with_session
+16 -13
View File
@@ -359,6 +359,12 @@ class WorkflowPublishResponse(ResponseModel):
created_at: int
class SyncDraftWorkflowResponse(ResponseModel):
result: str
hash: str
updated_at: int
class WorkflowRestoreResponse(ResponseModel):
result: str
hash: str
@@ -441,6 +447,7 @@ register_response_schema_models(
WorkflowOnlineUsersByApp,
WorkflowOnlineUsersResponse,
WorkflowPublishResponse,
SyncDraftWorkflowResponse,
WorkflowRestoreResponse,
DefaultBlockConfigsResponse,
DefaultBlockConfigResponse,
@@ -556,14 +563,7 @@ class DraftWorkflowApi(Resource):
@console_ns.response(
200,
"Draft workflow synced successfully",
console_ns.model(
"SyncDraftWorkflowResponse",
{
"result": fields.String,
"hash": fields.String,
"updated_at": fields.String,
},
),
console_ns.models[SyncDraftWorkflowResponse.__name__],
)
@console_ns.response(400, "Invalid workflow configuration")
@console_ns.response(403, "Permission denied")
@@ -618,11 +618,14 @@ class DraftWorkflowApi(Resource):
except VariableError as e:
raise InvalidArgumentError(description=str(e))
return {
"result": "success",
"hash": workflow.unique_hash,
"updated_at": TimestampField().format(workflow.updated_at or workflow.created_at),
}
return dump_response(
SyncDraftWorkflowResponse,
{
"result": "success",
"hash": workflow.unique_hash,
"updated_at": TimestampField().format(workflow.updated_at or workflow.created_at),
},
)
@console_ns.route("/apps/<uuid:app_id>/advanced-chat/workflows/draft/run")
+48 -1
View File
@@ -12,14 +12,22 @@ from typing import cast, overload
from sqlalchemy import select
from sqlalchemy.orm import Session
from configs import dify_config
from controllers.common.session import with_session
from controllers.common.wraps import RBACPermission, RBACResourceScope, enforce_rbac_access
from controllers.console.app.error import AppNotFoundError
from extensions.ext_database import db
from libs.login import current_account_with_tenant
from models import App, AppMode, TrialApp
from models.agent import AgentScope
from services.recommended_app_service import RecommendedAppService
__all__ = ["get_app_model", "get_app_model_with_trial", "with_session"]
__all__ = [
"agent_manage_required_for_agent_app",
"get_app_model",
"get_app_model_with_trial",
"with_session",
]
def _load_app_model(session: Session, app_id: str) -> App | None:
@@ -48,6 +56,45 @@ def _load_app_model_with_trial(session: Session, app_id: str) -> App | None:
return app_model
def agent_manage_required_for_agent_app[**P, R](view: Callable[P, R]) -> Callable[P, R]:
"""Gate generic app management routes that target an Agent App.
A hidden workflow-only backing App only reuses the App runtime and is not
part of the general app management plane, so generic routes reject it
outright. Managing a roster Agent App mutates the roster Agent behind it
(rename/icon sync, archive, API enablement), so it additionally requires
workspace ``agent.manage`` on top of the route's existing App permission
checks when RBAC is enabled. A no-op for non-agent Apps. Must be placed
above ``get_app_model`` so the ``app_id`` path parameter is still present.
"""
@wraps(view)
def decorated(*args: P.args, **kwargs: P.kwargs) -> R:
raw_app_id = kwargs.get("app_id") or kwargs.get("resource_id")
if raw_app_id is not None:
app_model = _load_app_model_from_scoped_session(str(raw_app_id))
binding = (
app_model.agent_app_binding_with_session(session=db.session(), include_archived=True)
if app_model is not None
else None
)
if binding is not None:
if binding.scope == AgentScope.WORKFLOW_ONLY:
raise AppNotFoundError()
if dify_config.RBAC_ENABLED:
current_user, current_tenant_id = current_account_with_tenant()
enforce_rbac_access(
tenant_id=current_tenant_id,
account_id=current_user.id,
resource_type=RBACResourceScope.WORKSPACE,
scene=RBACPermission.AGENT_MANAGE,
resource_required=False,
)
return view(*args, **kwargs)
return decorated
def _get_injected_session(args: tuple[object, ...]) -> Session | None:
"""Return the request session inserted by `with_session`, if this handler has been migrated."""
if len(args) < 2:
+14
View File
@@ -7,10 +7,13 @@ from configs import dify_config
from constants.languages import supported_language
from controllers.common.schema import query_params_from_model, register_schema_models
from controllers.console import console_ns
from controllers.console.auth.error import InvitationAccountMismatchError
from controllers.console.error import AccountInFreezeError, AlreadyActivateError
from extensions.ext_database import db
from libs.datetime_utils import naive_utc_now
from libs.helper import EmailStr, timezone
from libs.login import current_account_with_tenant
from libs.token import extract_access_token
from models import AccountStatus
from models.account import TenantAccountJoin, TenantAccountRole
from services.account_service import RegisterService, TenantService
@@ -136,6 +139,12 @@ class ActivateApi(Resource):
)
@console_ns.response(400, "Already activated or invalid token")
def post(self):
"""Accept an invitation without letting an existing session act for another account.
Token-only activation remains available for legacy clients. When the request already
carries a console session, that session must belong to the account encoded in the
invitation before the token is consumed or tenant membership is changed.
"""
args = ActivatePayload.model_validate(console_ns.payload)
normalized_request_email = args.email.lower() if args.email else None
@@ -146,6 +155,11 @@ class ActivateApi(Resource):
raise AlreadyActivateError()
account = invitation["account"]
if extract_access_token(request):
current_account, _ = current_account_with_tenant()
if current_account.id != account.id:
raise InvitationAccountMismatchError()
if dify_config.BILLING_ENABLED and BillingService.is_email_in_freeze(account.email):
raise AccountInFreezeError()
+6
View File
@@ -13,6 +13,12 @@ class InvalidEmailError(BaseHTTPException):
code = 400
class InvitationAccountMismatchError(BaseHTTPException):
error_code = "invitation_account_mismatch"
description = "This invitation was sent to another account. Please sign in with the invited account."
code = 403
class PasswordMismatchError(BaseHTTPException):
error_code = "password_mismatch"
description = "The passwords do not match."
+35 -21
View File
@@ -6,6 +6,7 @@ from flask import current_app, redirect, request
from flask_restx import Resource
from pydantic import BaseModel, Field
from werkzeug.exceptions import Unauthorized
from werkzeug.wrappers import Response
from configs import dify_config
from constants.languages import languages
@@ -127,6 +128,20 @@ def _preferred_interface_language(language: str | None = None) -> str:
return languages[0]
def _redirect_with_console_session(account: Account, target_url: str) -> Response:
"""Create a console session and attach its cookies to a redirect response."""
token_pair = AccountService.login(
account=account,
session=db.session(),
ip_address=extract_remote_ip(request),
)
response = redirect(target_url)
set_access_token_to_cookie(request, response, token_pair.access_token)
set_refresh_token_to_cookie(request, response, token_pair.refresh_token)
set_csrf_token_to_cookie(request, response, token_pair.csrf_token)
return response
@console_ns.route("/oauth/login/<provider>")
class OAuthLogin(Resource):
@console_ns.doc("oauth_login")
@@ -195,16 +210,26 @@ class OAuthCallback(Resource):
return redirect(f"{dify_config.CONSOLE_WEB_URL}/signin?message={urllib.parse.quote(str(e))}")
if invite_token and RegisterService.is_valid_invite_token(invite_token):
invitation = RegisterService.get_invitation_by_token(token=invite_token)
if invitation:
invitation_email = invitation.get("email", None)
invitation_email_normalized = (
invitation_email.lower() if isinstance(invitation_email, str) else invitation_email
)
if invitation_email_normalized != user_info.email.lower():
return redirect(f"{dify_config.CONSOLE_WEB_URL}/signin?message=Invalid invitation token.")
invitation = RegisterService.get_invitation_if_token_valid(
None,
None,
invite_token,
session=db.session(),
)
if not invitation:
return redirect(f"{dify_config.CONSOLE_WEB_URL}/signin?message=Invalid invitation token.")
if invitation["data"]["email"].lower() != user_info.email.lower():
message = "This invitation was sent to another account. Please sign in with the invited account."
query = urllib.parse.urlencode({"message": message, "invite_token": invite_token})
return redirect(f"{dify_config.CONSOLE_WEB_URL}/signin?{query}")
return redirect(f"{dify_config.CONSOLE_WEB_URL}/signin/invite-settings?invite_token={invite_token}")
account = invitation["account"]
if account.status == AccountStatus.BANNED:
return redirect(f"{dify_config.CONSOLE_WEB_URL}/signin?message=Account is banned.")
AccountService.link_account_integrate(provider, user_info.id, account, session=db.session())
target_url = f"{dify_config.CONSOLE_WEB_URL}/signin/invite-settings?invite_token={invite_token}"
return _redirect_with_console_session(account, target_url)
try:
account, oauth_new_user = _generate_account(provider, user_info, timezone=timezone, language=language)
@@ -239,21 +264,10 @@ class OAuthCallback(Resource):
"?message=Workspace not found, please contact system admin to invite you to join in a workspace."
)
token_pair = AccountService.login(
account=account,
session=db.session(),
ip_address=extract_remote_ip(request),
)
target_url = _get_redirect_target(redirect_url)
query_char = "&" if "?" in target_url else "?"
target_url = f"{target_url}{query_char}oauth_new_user={str(oauth_new_user).lower()}"
response = redirect(target_url)
set_access_token_to_cookie(request, response, token_pair.access_token)
set_refresh_token_to_cookie(request, response, token_pair.refresh_token)
set_csrf_token_to_cookie(request, response, token_pair.csrf_token)
return response
return _redirect_with_console_session(account, target_url)
def _get_account_by_openid_or_email(provider: str, user_info: OAuthUserInfo) -> Account | None:
+132 -45
View File
@@ -8,8 +8,9 @@ can be validated explicitly against the pinned KnowledgeFS contract during devel
Console auth and contract-specific dataset RBAC run before forwarding. Request
bodies are capped at 64 MiB, JSON and binary responses have separate bounds,
SSE responses remain streaming with a bounded idle read timeout, and only safe
response headers are exposed. Upstream 401 responses become 502 so they cannot
trigger Dify browser-session recovery; resource-level 403 responses remain 403.
response headers are exposed. Operation-specific upstream error mappings are
applied before Console JSON error handling; the default maps 401 to 502 so it
cannot trigger browser-session recovery and preserves resource-level 403.
"""
from __future__ import annotations
@@ -27,9 +28,11 @@ from werkzeug.exceptions import (
BadGateway,
Forbidden,
GatewayTimeout,
HTTPException,
NotFound,
RequestEntityTooLarge,
ServiceUnavailable,
default_exceptions,
)
from configs import dify_config
@@ -41,21 +44,28 @@ from controllers.console.wraps import (
)
from core.helper import ssrf_proxy
from libs.login import current_account_with_tenant, login_required
from services.knowledge_fs_operations import KnowledgeFSMethod
from services.knowledge_fs_proxy import (
KnowledgeFSAccessDeniedError,
KnowledgeFSAuthorization,
KnowledgeFSConfigurationError,
KnowledgeFSMethod,
KnowledgeFSRouteNotAllowedError,
KnowledgeFSTimeoutError,
KnowledgeFSTransportError,
KnowledgeFSUpstreamResponse,
authorize_knowledge_fs_request,
get_knowledge_fs_operation,
proxy_authorized_knowledge_fs_request,
proxy_knowledge_fs_request,
)
logger = logging.getLogger(__name__)
type _KnowledgeFSRequestForwarder = Callable[
[str | None, str | None, bytes | None, bytes | None],
KnowledgeFSUpstreamResponse,
]
_MAX_PROXY_BODY_BYTES = 64 * 1024 * 1024
_RESPONSE_HEADER_ALLOWLIST = (
"Cache-Control",
@@ -128,27 +138,25 @@ def _translate_proxy_error(exc: Exception, *, tenant_id: str) -> NoReturn:
def _knowledge_fs_operation_access_required(
view: Callable[[KnowledgeFSMethod, str], ResponseReturnValue],
view: Callable[[KnowledgeFSAuthorization], ResponseReturnValue],
) -> Callable[[KnowledgeFSMethod, str], ResponseReturnValue]:
"""Authorize one declared operation before billing and request-body work."""
@wraps(view)
def decorated(method: KnowledgeFSMethod, upstream_path: str) -> ResponseReturnValue:
try:
operation = get_knowledge_fs_operation(method, upstream_path)
except KnowledgeFSRouteNotAllowedError as exc:
raise NotFound() from exc
current_user, tenant_id = current_account_with_tenant()
try:
authorize_knowledge_fs_request(
authorization = authorize_knowledge_fs_request(
account=current_user,
tenant_id=tenant_id,
operation=operation,
method=method,
path=upstream_path,
)
except KnowledgeFSRouteNotAllowedError as exc:
raise NotFound() from exc
except KnowledgeFSAccessDeniedError as exc:
_translate_proxy_error(exc, tenant_id=tenant_id)
return view(method, upstream_path)
return view(authorization)
return decorated
@@ -190,21 +198,26 @@ def _proxy_response(
"""Expose raw content, status, and allowlisted headers from KnowledgeFS.
Raises:
BadGateway: KnowledgeFS rejects the configured server credential.
Forbidden: KnowledgeFS denies the account access to the requested resource.
HTTPException: KnowledgeFS returns a status normalized by the operation contract.
"""
upstream = upstream_result.response
if upstream.status_code == HTTPStatus.UNAUTHORIZED:
mapped_status = dict(upstream_result.operation.error_status_map).get(upstream.status_code)
if mapped_status is not None:
upstream.close()
logger.error(
"KnowledgeFS rejected the Dify server credential with HTTP %s for tenant_id=%s",
upstream.status_code,
tenant_id,
)
raise BadGateway("KnowledgeFS authentication failed")
if upstream.status_code == HTTPStatus.FORBIDDEN:
upstream.close()
raise Forbidden()
description = "KnowledgeFS upstream request failed"
if upstream.status_code == HTTPStatus.UNAUTHORIZED:
description = "KnowledgeFS authentication failed"
logger.error(
"KnowledgeFS rejected the Dify server credential with HTTP %s for tenant_id=%s",
upstream.status_code,
tenant_id,
)
exception_type = default_exceptions.get(mapped_status)
if exception_type is None:
exception = HTTPException(description)
exception.code = mapped_status
raise exception
raise exception_type(description)
allowed_header_names = dict.fromkeys(
name.lower() for name in (*_RESPONSE_HEADER_ALLOWLIST, *contract_response_headers)
@@ -237,26 +250,21 @@ def _proxy_response(
return Response(content, status=upstream.status_code, headers=headers)
def _proxy_request(method: KnowledgeFSMethod, upstream_path: str) -> Response:
"""Forward the current raw request and return its filtered upstream response.
The call performs one outbound KnowledgeFS request. Integration failures are
converted to Console HTTP exceptions for the outer JSON error adapter.
"""
def _proxy_current_request(
*,
method: KnowledgeFSMethod,
tenant_id: str,
forward: _KnowledgeFSRequestForwarder,
) -> Response:
"""Forward the current raw request through one preconfigured service entry."""
if not dify_config.KNOWLEDGE_FS_ENABLED:
raise NotFound()
current_user, tenant_id = current_account_with_tenant()
try:
proxy_result = proxy_knowledge_fs_request(
account=current_user,
method=method,
path=upstream_path,
tenant_id=tenant_id,
accept=request.headers.get("Accept"),
content_type=request.content_type,
query=request.query_string or None,
body=_request_body() if method != "GET" else None,
request_headers=request.headers,
proxy_result = forward(
request.headers.get("Accept"),
request.content_type,
request.query_string or None,
_request_body() if method != "GET" else None,
)
except (
KnowledgeFSConfigurationError,
@@ -274,20 +282,99 @@ def _proxy_request(method: KnowledgeFSMethod, upstream_path: str) -> Response:
)
def _proxy_request(
method: KnowledgeFSMethod,
upstream_path: str,
) -> Response:
"""Authorize and forward the current request through the combined service use case."""
if not dify_config.KNOWLEDGE_FS_ENABLED:
raise NotFound()
current_user, tenant_id = current_account_with_tenant()
def forward(
accept: str | None,
content_type: str | None,
query: bytes | None,
body: bytes | None,
) -> KnowledgeFSUpstreamResponse:
return proxy_knowledge_fs_request(
account=current_user,
method=method,
path=upstream_path,
tenant_id=tenant_id,
accept=accept,
content_type=content_type,
query=query,
body=body,
request_headers=request.headers,
)
return _proxy_current_request(method=method, tenant_id=tenant_id, forward=forward)
def _proxy_authorized_request(authorization: KnowledgeFSAuthorization) -> Response:
"""Forward the current request using one previously authorized operation capability.
Args:
authorization: Request-scoped capability produced before billing and body parsing.
Returns:
The filtered response returned by KnowledgeFS.
Raises:
HTTPException: The integration is disabled or forwarding fails.
"""
operation = authorization.operation
tenant_id = authorization.tenant_id
def forward(
accept: str | None,
content_type: str | None,
query: bytes | None,
body: bytes | None,
) -> KnowledgeFSUpstreamResponse:
return proxy_authorized_knowledge_fs_request(
authorization=authorization,
accept=accept,
content_type=content_type,
query=query,
body=body,
request_headers=request.headers,
)
return _proxy_current_request(method=operation.method, tenant_id=tenant_id, forward=forward)
@_knowledge_fs_enabled
@_knowledge_fs_operation_access_required
@cloud_edition_billing_rate_limit_check("knowledge")
def _proxy_knowledge_fs_non_get(
method: KnowledgeFSMethod,
upstream_path: str,
authorization: KnowledgeFSAuthorization,
) -> ResponseReturnValue:
"""Apply knowledge billing checks to one allowlisted non-GET operation."""
return _proxy_request(method, upstream_path)
return _proxy_authorized_request(authorization)
@bp.route(
"/knowledge-fs/<path:upstream_path>",
methods=["GET", "OPTIONS"],
methods=["OPTIONS"],
provide_automatic_options=False,
)
@_console_api_errors
@_knowledge_fs_enabled
def proxy_knowledge_fs_options(upstream_path: str) -> ResponseReturnValue:
"""Complete a CORS preflight only for an enabled Console operation."""
requested_method = cast(KnowledgeFSMethod, request.headers.get("Access-Control-Request-Method", "").upper())
try:
get_knowledge_fs_operation(requested_method, upstream_path)
except KnowledgeFSRouteNotAllowedError as exc:
raise NotFound() from exc
return Response(status=HTTPStatus.NO_CONTENT)
@bp.route(
"/knowledge-fs/<path:upstream_path>",
methods=["GET"],
provide_automatic_options=False,
)
@_console_api_errors
@@ -67,6 +67,15 @@ from services.plugin.plugin_parameter_service import PluginParameterService
from services.plugin.plugin_permission_service import PluginPermissionService
from services.tools.tools_transform_service import ToolTransformService
_PLUGIN_PACKAGE_UPLOAD_PARAMS = {
"pkg": {
"description": "Plugin package to upload",
"in": "formData",
"type": "file",
"required": True,
}
}
class AutoUpgradeSettingsResponse(TypedDict):
strategy_setting: TenantPluginAutoUpgradeStrategySetting
@@ -645,6 +654,7 @@ class PluginAssetApi(Resource):
@console_ns.route("/workspaces/current/plugin/upload/pkg")
class PluginUploadFromPkgApi(Resource):
@console_ns.doc(consumes=["multipart/form-data"], params=_PLUGIN_PACKAGE_UPLOAD_PARAMS)
@console_ns.response(200, "Success", console_ns.models[PluginDecodeResponse.__name__])
@setup_required
@login_required
+2
View File
@@ -23,6 +23,7 @@ from .knowledge import retrieval as _knowledge_retrieval
from .plugin import agent_config as _agent_config
from .plugin import agent_drive as _agent_drive
from .plugin import plugin as _plugin
from .workspace import plugin_model_providers as _plugin_model_providers
from .workspace import workspace as _workspace
api.add_namespace(inner_api_ns)
@@ -35,6 +36,7 @@ __all__ = [
"_knowledge_retrieval",
"_mail",
"_plugin",
"_plugin_model_providers",
"_runtime_credentials",
"_workspace",
"api",
@@ -0,0 +1,39 @@
from flask_restx import Resource
from pydantic import BaseModel, ConfigDict, Field
from controllers.common.schema import register_schema_model
from controllers.console.wraps import setup_required
from controllers.inner_api import inner_api_ns
from controllers.inner_api.wraps import enterprise_inner_api_only
from core.plugin.plugin_service import PluginService
class InvalidatePluginModelProvidersCachePayload(BaseModel):
model_config = ConfigDict(extra="forbid")
tenant_ids: list[str] = Field(default_factory=list, description="Workspace ids whose cache should be invalidated")
register_schema_model(inner_api_ns, InvalidatePluginModelProvidersCachePayload)
@inner_api_ns.route("/enterprise/workspace/plugin-model-providers/invalidate")
class EnterprisePluginModelProvidersCacheInvalidate(Resource):
@setup_required
@enterprise_inner_api_only
@inner_api_ns.doc(
"enterprise_invalidate_plugin_model_providers_cache",
responses={
200: "Cache invalidated",
400: "Invalid request",
401: "Unauthorized - invalid API key",
},
)
@inner_api_ns.expect(inner_api_ns.models[InvalidatePluginModelProvidersCachePayload.__name__])
def post(self):
args = InvalidatePluginModelProvidersCachePayload.model_validate(inner_api_ns.payload or {})
for tenant_id in args.tenant_ids:
PluginService.invalidate_plugin_model_providers_cache(tenant_id)
return {"result": "success"}, 200
+17 -7
View File
@@ -1,6 +1,6 @@
from typing import Any, Self
from pydantic import AliasChoices, Field, computed_field
from pydantic import AliasChoices, Field
from sqlalchemy import select
from werkzeug.exceptions import Forbidden
@@ -9,11 +9,13 @@ from controllers.common.schema import register_response_schema_models
from controllers.web import web_ns
from controllers.web.wraps import WebApiResource
from extensions.ext_database import db
from extensions.storage.storage_type import StorageType
from fields.base import ResponseModel
from libs.helper import build_icon_url
from models.account import Tenant, TenantStatus
from models.model import App, EndUser, Site
from models.model import App, EndUser, IconType, Site
from services.feature_service import FeatureModel, FeatureService
from services.file_service import FileService
class WebSiteResponse(ResponseModel):
@@ -32,11 +34,7 @@ class WebSiteResponse(ResponseModel):
prompt_public: bool | None = None
show_workflow_steps: bool | None = None
use_icon_as_answer_icon: bool | None = None
@computed_field(return_type=str | None) # type: ignore[prop-decorator]
@property
def icon_url(self) -> str | None:
return build_icon_url(self.icon_type, self.icon)
icon_url: str | None = None
class WebModelConfigResponse(ResponseModel):
@@ -88,6 +86,7 @@ class WebAppSiteResponse(ResponseModel):
end_user_id: str | None,
features: FeatureModel,
can_replace_logo: bool,
icon_url: str | None = None,
) -> Self:
custom_config = None
if can_replace_logo:
@@ -102,6 +101,7 @@ class WebAppSiteResponse(ResponseModel):
)
site_response = WebSiteResponse.model_validate(site, from_attributes=True)
site_response.icon_url = icon_url if icon_url is not None else build_icon_url(site.icon_type, site.icon)
if features.billing.enabled and not features.webapp_copyright_enabled:
site_response.copyright = None
site_response.input_placeholder = None
@@ -123,6 +123,15 @@ register_response_schema_models(
)
def _build_site_icon_url(*, site: Site, tenant_id: str) -> str | None:
"""Use direct S3 URLs only in Cloud Mode and preserve preview URLs elsewhere."""
if site.icon_type != IconType.IMAGE or not site.icon:
return None
if dify_config.EDITION == "CLOUD" and StorageType(dify_config.STORAGE_TYPE) == StorageType.S3:
return FileService(db.engine).get_file_presigned_url(file_id=site.icon, tenant_id=tenant_id)
return build_icon_url(site.icon_type, site.icon)
@web_ns.route("/site")
class AppSiteApi(WebApiResource):
@web_ns.doc("Get App Site Info")
@@ -159,4 +168,5 @@ class AppSiteApi(WebApiResource):
end_user_id=end_user.id,
features=features,
can_replace_logo=features.can_replace_logo,
icon_url=_build_site_icon_url(site=site, tenant_id=tenant.id),
).model_dump(mode="json")
+14 -29
View File
@@ -682,42 +682,27 @@ class AgentAppGenerator(MessageBasedAppGenerator):
if draft_type == AgentConfigDraftType.DEBUG_BUILD.value
else AgentConfigDraftType.DRAFT
)
if effective_draft_type == AgentConfigDraftType.DRAFT:
from services.agent.composer_service import AgentComposerService
return AgentComposerService.get_or_create_normal_agent_draft(
session=session,
tenant_id=tenant_id,
agent=agent,
created_by=agent.updated_by or agent.created_by,
)
if not account_id:
raise AgentAppGeneratorError("Build draft requires an account user")
stmt = select(AgentConfigDraft).where(
AgentConfigDraft.tenant_id == tenant_id,
AgentConfigDraft.agent_id == agent.id,
AgentConfigDraft.draft_type == effective_draft_type,
AgentConfigDraft.draft_type == AgentConfigDraftType.DEBUG_BUILD,
AgentConfigDraft.account_id == account_id,
)
if effective_draft_type == AgentConfigDraftType.DEBUG_BUILD:
if not account_id:
raise AgentAppGeneratorError("Build draft requires an account user")
stmt = stmt.where(AgentConfigDraft.account_id == account_id)
else:
stmt = stmt.where(AgentConfigDraft.account_id.is_(None))
draft = session.scalar(stmt.order_by(AgentConfigDraft.updated_at.desc()).limit(1))
if draft is not None:
return draft
if effective_draft_type == AgentConfigDraftType.DEBUG_BUILD:
raise AgentAppGeneratorError("Agent build draft not found")
_, snapshot, agent_soul = AgentAppGenerator._resolve_agent_by_id(
tenant_id=tenant_id,
agent_id=agent.id,
snapshot_id=agent.active_config_snapshot_id,
session=session,
)
draft = AgentConfigDraft(
tenant_id=tenant_id,
agent_id=agent.id,
draft_type=AgentConfigDraftType.DRAFT,
account_id=None,
draft_owner_key="",
base_snapshot_id=snapshot.id,
config_snapshot=agent_soul,
created_by=agent.created_by,
updated_by=agent.updated_by,
)
session.add(draft)
session.flush()
return draft
raise AgentAppGeneratorError("Agent build draft not found")
@staticmethod
def _resolve_agent_by_id(
+1
View File
@@ -61,6 +61,7 @@ class RBACPermission(StrEnum):
WORKSPACE_ROLE_MANAGE = "workspace_role_manage"
API_EXTENSION_MANAGE = "api_extension_manage"
CUSTOMIZATION_MANAGE = "customization_manage"
AGENT_MANAGE = "agent_manage"
SNIPPETS_CREATE_AND_MODIFY = "snippets_create_and_modify"
SNIPPETS_MANAGE = "snippets_management"
+28 -15
View File
@@ -58,7 +58,6 @@ class Tool(ABC):
if self.runtime and self.runtime.runtime_parameters:
tool_parameters.update(self.runtime.runtime_parameters)
# try parse tool parameters into the correct type
tool_parameters = self._transform_tool_parameters_type(tool_parameters)
result = self._invoke(
@@ -87,14 +86,14 @@ class Tool(ABC):
return result
def _transform_tool_parameters_type(self, tool_parameters: dict[str, Any]) -> dict[str, Any]:
"""
Transform tool parameters type
"""
# Temp fix for the issue that the tool parameters will be converted to empty while validating the credentials
"""Transform declared tool parameter values without resolving runtime schemas."""
result = deepcopy(tool_parameters)
for parameter in self.entity.parameters or []:
if parameter.name in tool_parameters:
result[parameter.name] = parameter.type.cast_value(tool_parameters[parameter.name])
if parameter.multiple:
result[parameter.name] = parameter.init_frontend_parameter(result.get(parameter.name))
else:
result[parameter.name] = parameter.type.cast_value(tool_parameters[parameter.name])
return result
@@ -196,17 +195,31 @@ class Tool(ABC):
}:
continue
parameter_schema: dict[str, Any] = (
{
"type": parameter.type.as_normal_type(),
"description": parameter.llm_description or "",
}
if parameter.input_schema is None
else deepcopy(parameter.input_schema)
)
is_multiple_select = parameter.multiple and parameter.type in {
ToolParameter.ToolParameterType.SELECT,
ToolParameter.ToolParameterType.DYNAMIC_SELECT,
}
if is_multiple_select:
item_schema: dict[str, Any] = {"type": "string"}
if parameter.type == ToolParameter.ToolParameterType.SELECT and parameter.options:
item_schema["enum"] = [option.value for option in parameter.options]
parameter_schema: dict[str, Any] = {"type": "array", "items": item_schema}
else:
parameter_schema = (
{
"type": parameter.type.as_normal_type(),
"description": parameter.llm_description or "",
}
if parameter.input_schema is None
else deepcopy(parameter.input_schema)
)
parameter_schema.setdefault("description", parameter.llm_description or "")
if parameter.type == ToolParameter.ToolParameterType.SELECT and parameter.options:
if (
not is_multiple_select
and parameter.type == ToolParameter.ToolParameterType.SELECT
and parameter.options
):
parameter_schema["enum"] = [option.value for option in parameter.options]
schema["properties"][parameter.name] = parameter_schema
+36 -5
View File
@@ -292,9 +292,7 @@ class ToolInvokeMessageBinary(BaseModel):
class ToolParameter(PluginParameter):
"""
Overrides type
"""
"""Tool-specific parameter declaration and invocation-value normalization."""
class ToolParameterType(StrEnum):
"""
@@ -333,12 +331,28 @@ class ToolParameter(PluginParameter):
LLM = auto() # will be set by LLM
type: ToolParameterType = Field(..., description="The type of the parameter")
multiple: bool = Field(
default=False,
description="Whether the parameter is multiple select, only valid for select or dynamic-select type",
)
human_description: I18nObject | None = Field(default=None, description="The description presented to the user")
form: ToolParameterForm = Field(..., description="The form of the parameter, schema/form/llm")
llm_description: str | None = None
# MCP object and array type parameters use this field to store the schema
input_schema: dict[str, Any] | None = None
@model_validator(mode="after")
def validate_multiple(self) -> ToolParameter:
supports_multiple = self.type in {
self.ToolParameterType.SELECT,
self.ToolParameterType.DYNAMIC_SELECT,
}
if self.multiple and not supports_multiple:
raise ValueError("multiple is only valid for select and dynamic-select parameters")
if supports_multiple and self.default is not None and (isinstance(self.default, list) != self.multiple):
raise ValueError("default must be a list exactly when multiple is true")
return self
@classmethod
def get_simple_instance(
cls,
@@ -378,8 +392,25 @@ class ToolParameter(PluginParameter):
options=option_objs,
)
def init_frontend_parameter(self, value: Any):
return init_frontend_parameter(self, self.type, value)
def init_frontend_parameter(self, value: Any) -> Any:
"""Normalize a value against this tool parameter's full declaration."""
if not self.multiple:
return init_frontend_parameter(self, self.type, value)
parameter_value = self.default if value is None else value
if parameter_value is None:
parameter_value = []
if not isinstance(parameter_value, list):
raise ValueError(f"tool parameter {self.name} must be a list when multiple is true")
if not all(isinstance(item, str) for item in parameter_value):
raise ValueError(f"tool parameter {self.name} must contain only strings")
if self.required and not parameter_value:
raise ValueError(f"tool parameter {self.name} not found in tool config")
if self.type == self.ToolParameterType.SELECT:
options = [option.value for option in self.options]
if any(item not in options for item in parameter_value):
raise ValueError(f"tool parameter {self.name} value {parameter_value} not in options {options}")
return parameter_value
class ToolProviderIdentity(BaseModel):
@@ -2,7 +2,7 @@ from __future__ import annotations
from collections.abc import Mapping
from dataclasses import dataclass
from typing import Any, Literal, Protocol, cast
from typing import Any, Final, Literal, Protocol
from dify_agent.layers.dify_core_tools import DifyCoreToolConfig, DifyCoreToolProviderType, DifyCoreToolsLayerConfig
from dify_agent.layers.dify_plugin import (
@@ -29,6 +29,13 @@ from models.provider_ids import ToolProviderID
from models.tools import WorkflowToolProvider
from services.tools.mcp_tools_manage_service import MCPToolManageService
_CORE_TOOL_PROVIDER_TYPES: Final[dict[ToolProviderType, DifyCoreToolProviderType]] = {
ToolProviderType.BUILT_IN: "builtin",
ToolProviderType.API: "api",
ToolProviderType.WORKFLOW: "workflow",
ToolProviderType.MCP: "mcp",
}
class WorkflowAgentDifyToolsBuildError(ValueError):
"""Raised when Agent Soul tools cannot be prepared for Agent backend."""
@@ -235,7 +242,7 @@ class WorkflowAgentDifyToolsBuilder:
if tool_config.tool_name is not None:
expanded.append(tool_config)
continue
provider_type = ToolProviderType.value_of(tool_config.provider_type)
provider_type = tool_config.provider_type
provider_id = self._provider_id(tool_config)
try:
tool_names = self._provider_declared_tool_names(
@@ -279,7 +286,7 @@ class WorkflowAgentDifyToolsBuilder:
tenant_id: str,
tool_config: AgentSoulDifyToolConfig,
) -> AgentSoulDifyToolConfig:
if tool_config.provider_type != ToolProviderType.MCP.value:
if tool_config.provider_type is not ToolProviderType.MCP:
return tool_config
provider_id = self._mcp_provider_id_resolver(tenant_id=tenant_id, provider_id=self._provider_id(tool_config))
return tool_config.model_copy(update={"provider_id": provider_id, "plugin_id": None, "provider": None})
@@ -326,7 +333,7 @@ class WorkflowAgentDifyToolsBuilder:
def _to_agent_tool_entity(tool_config: AgentSoulDifyToolConfig) -> AgentToolEntity:
assert tool_config.tool_name is not None
return AgentToolEntity(
provider_type=ToolProviderType.value_of(tool_config.provider_type),
provider_type=tool_config.provider_type,
provider_id=WorkflowAgentDifyToolsBuilder._provider_id(tool_config),
tool_name=tool_config.tool_name,
tool_parameters=dict(tool_config.runtime_parameters),
@@ -343,24 +350,16 @@ class WorkflowAgentDifyToolsBuilder:
@staticmethod
def _provider_key(tool_config: AgentSoulDifyToolConfig) -> tuple[ToolProviderType, str]:
return (
ToolProviderType.value_of(tool_config.provider_type),
WorkflowAgentDifyToolsBuilder._provider_id(tool_config),
)
return (tool_config.provider_type, WorkflowAgentDifyToolsBuilder._provider_id(tool_config))
@staticmethod
def _tool_layer_destination(tool_config: AgentSoulDifyToolConfig) -> Literal["plugin", "core"]:
provider_type = ToolProviderType.value_of(tool_config.provider_type)
provider_type = tool_config.provider_type
if provider_type is ToolProviderType.PLUGIN or (
provider_type is ToolProviderType.BUILT_IN and _is_plugin_provider_id(tool_config.provider_id)
):
return "plugin"
if provider_type in {
ToolProviderType.BUILT_IN,
ToolProviderType.API,
ToolProviderType.WORKFLOW,
ToolProviderType.MCP,
}:
if provider_type in _CORE_TOOL_PROVIDER_TYPES:
return "core"
if provider_type is ToolProviderType.DATASET_RETRIEVAL:
raise WorkflowAgentDifyToolsBuildError(
@@ -417,7 +416,7 @@ class WorkflowAgentDifyToolsBuilder:
) -> DifyCoreToolConfig:
parameters = self._prepared_parameters(tool_runtime)
return DifyCoreToolConfig(
provider_type=cast(DifyCoreToolProviderType, tool_config.provider_type),
provider_type=_CORE_TOOL_PROVIDER_TYPES[tool_config.provider_type],
provider_id=self._provider_id(tool_config),
tool_name=tool_config.tool_name or exposed_name,
credential_id=tool_config.credential_ref.id if tool_config.credential_ref else None,
+134 -4
View File
@@ -12,6 +12,7 @@ import json
import subprocess
import sys
import tempfile
from copy import deepcopy
from pathlib import Path
from typing import Any, Literal, TypedDict
@@ -24,6 +25,16 @@ LOCK_PATH = API_ROOT / "knowledge-fs-contract.lock.json"
DEFAULT_REPOSITORY = WORKSPACE_ROOT.parent / "knowledge-fs"
OPENAPI_METHODS = ("delete", "get", "head", "options", "patch", "post", "put", "trace")
PROXY_METHODS = frozenset({"delete", "get", "patch", "post", "put"})
CONSOLE_PROXY_ERROR_SCHEMA_NAME = "ConsoleProxyError"
CONSOLE_PROXY_ERROR_SCHEMA: dict[str, Any] = {
"type": "object",
"required": ["code", "message", "status"],
"properties": {
"code": {"type": "string"},
"message": {"type": "string"},
"status": {"type": "integer"},
},
}
class ContractDeclaration(TypedDict):
@@ -38,6 +49,7 @@ class ContractDeclaration(TypedDict):
request_headers: tuple[str, ...]
response_headers: tuple[str, ...]
response_media_types: tuple[str, ...]
error_status_map: tuple[tuple[int, int], ...]
type DeclarationField = Literal[
@@ -70,6 +82,7 @@ def main() -> None:
mode.add_argument("--check", action="store_true")
mode.add_argument("--update-lock", action="store_true")
parser.add_argument("--repository", type=Path, default=DEFAULT_REPOSITORY)
parser.add_argument("--output-openapi", type=Path)
args = parser.parse_args()
repository = args.repository.resolve()
@@ -101,7 +114,15 @@ def main() -> None:
)
document: dict[str, Any] = json.loads(openapi_content)
validate_declarations(document, console_contract_declarations())
declarations = console_contract_declarations()
validate_declarations(document, declarations)
if args.output_openapi:
filtered_document = filter_openapi_document(document, declarations)
filtered_document["x-dify-source-openapi-sha256"] = openapi_sha256
filtered_document["x-dify-console-declarations-sha256"] = contract_declarations_sha256(declarations)
args.output_openapi.parent.mkdir(parents=True, exist_ok=True)
args.output_openapi.write_text(json.dumps(filtered_document, indent=2) + "\n")
if args.update_lock:
LOCK_PATH.write_text(
@@ -147,8 +168,7 @@ def validate_declarations(document: dict[str, Any], declarations: tuple[Contract
raise ValueError(f"KnowledgeFS OpenAPI path must be absolute: {path}")
if method not in PROXY_METHODS:
raise ValueError(f"KnowledgeFS proxy does not support {method.upper()} {path}")
expected: ContractDeclaration = {
"operation_id": operation_id,
expected: dict[DeclarationField, object] = {
"method": method.upper(),
"path": path[1:],
"required_scope": required_scope(operation),
@@ -166,11 +186,114 @@ def validate_declarations(document: dict[str, Any], declarations: tuple[Contract
f"KnowledgeFS operation {operation_id} field {field} drifted: "
f"expected {expected_value!r}, received {received_value!r}"
)
validate_error_status_map(operation_id, declaration["error_status_map"])
def filter_openapi_document(
document: dict[str, Any],
declarations: tuple[ContractDeclaration, ...],
) -> dict[str, Any]:
"""Return a code-generation document containing only Console-allowlisted operations."""
filtered_document: dict[str, Any] = {
key: value for key, value in document.items() if key not in {"components", "paths"}
}
source_paths = document.get("paths", {})
filtered_paths: dict[str, Any] = {}
for declaration in declarations:
path = f"/{declaration['path']}"
method = declaration["method"].lower()
source_path_item = source_paths[path]
path_metadata = {key: value for key, value in source_path_item.items() if key not in OPENAPI_METHODS}
filtered_path_item = filtered_paths.setdefault(path, path_metadata)
filtered_operation = deepcopy(source_path_item[method])
_rewrite_proxy_error_responses(filtered_operation, declaration["error_status_map"])
filtered_path_item[method] = filtered_operation
filtered_document["paths"] = filtered_paths
source_components = document.get("components", {})
filtered_components = {key: value for key, value in source_components.items() if key != "schemas"}
source_schemas = source_components.get("schemas", {})
available_schemas = {**source_schemas, CONSOLE_PROXY_ERROR_SCHEMA_NAME: CONSOLE_PROXY_ERROR_SCHEMA}
schema_names = _referenced_schema_names(filtered_paths, available_schemas)
filtered_components["schemas"] = {
name: schema for name, schema in available_schemas.items() if name in schema_names
}
filtered_document["components"] = filtered_components
return filtered_document
def validate_error_status_map(operation_id: str, error_status_map: tuple[tuple[int, int], ...]) -> None:
"""Validate the status normalization advertised by one Console operation."""
upstream_statuses: set[int] = set()
for upstream_status, console_status in error_status_map:
if upstream_status in upstream_statuses:
raise ValueError(f"KnowledgeFS operation {operation_id} has duplicate error status: {upstream_status}")
if not 400 <= upstream_status <= 599 or not 400 <= console_status <= 599:
raise ValueError(f"KnowledgeFS operation {operation_id} has invalid error status mapping")
upstream_statuses.add(upstream_status)
def _rewrite_proxy_error_responses(
operation: dict[str, Any],
error_status_map: tuple[tuple[int, int], ...],
) -> None:
responses = operation.setdefault("responses", {})
proxy_error_response = {
"description": "Error normalized by the Dify Console KnowledgeFS proxy.",
"content": {
"application/json": {"schema": {"$ref": f"#/components/schemas/{CONSOLE_PROXY_ERROR_SCHEMA_NAME}"}}
},
}
for upstream_status, console_status in error_status_map:
existing_target = responses.get(str(console_status)) if upstream_status != console_status else None
responses.pop(str(upstream_status), None)
normalized_response: dict[str, Any] = deepcopy(proxy_error_response)
existing_schema = (
existing_target.get("content", {}).get("application/json", {}).get("schema")
if isinstance(existing_target, dict)
else None
)
if existing_schema is not None:
normalized_response["content"]["application/json"]["schema"] = {
"oneOf": [
deepcopy(existing_schema),
{"$ref": f"#/components/schemas/{CONSOLE_PROXY_ERROR_SCHEMA_NAME}"},
]
}
responses[str(console_status)] = normalized_response
def _referenced_schema_names(value: Any, schemas: dict[str, Any]) -> set[str]:
reference_prefix = "#/components/schemas/"
selected: set[str] = set()
pending: list[Any] = [value]
while pending:
current = pending.pop()
if isinstance(current, list):
pending.extend(current)
continue
if not isinstance(current, dict):
continue
reference = current.get("$ref")
if isinstance(reference, str) and reference.startswith(reference_prefix):
name = reference.removeprefix(reference_prefix)
if name not in selected:
if name not in schemas:
raise ValueError(f"KnowledgeFS OpenAPI references missing schema: {name}")
selected.add(name)
pending.append(schemas[name])
pending.extend(current.values())
return selected
def console_contract_declarations() -> tuple[ContractDeclaration, ...]:
"""Return transport declarations from the runtime Console operation registry."""
from services.knowledge_fs_proxy import KNOWLEDGE_FS_CONSOLE_OPERATIONS
from services.knowledge_fs_operations import KNOWLEDGE_FS_CONSOLE_OPERATIONS
return tuple(
{
@@ -183,11 +306,18 @@ def console_contract_declarations() -> tuple[ContractDeclaration, ...]:
"request_headers": operation.request_headers,
"response_headers": operation.response_headers,
"response_media_types": operation.response_media_types,
"error_status_map": operation.error_status_map,
}
for operation in KNOWLEDGE_FS_CONSOLE_OPERATIONS
)
def contract_declarations_sha256(declarations: tuple[ContractDeclaration, ...]) -> str:
"""Return a stable digest for the runtime Console operation declarations."""
content = json.dumps(declarations, separators=(",", ":"), sort_keys=True).encode()
return sha256(content)
def response_kind(operation: dict[str, Any]) -> str:
media_types = response_media_types(operation)
if "text/event-stream" in media_types:
+58
View File
@@ -0,0 +1,58 @@
"""Azure-specific helpers for Redis authentication via Entra ID (Managed Identity)."""
from typing import Union, override
from redis import CredentialProvider
AZURE_REDIS_SCOPE = "https://redis.azure.com/.default"
class AzureEntraIdCredentialProvider(CredentialProvider):
"""Redis credential provider for Azure Entra ID (Managed Identity) authentication.
Wraps ``redis-entraid``'s provider so that it can be instantiated with no
arguments required by kombu's URL-based ``credential_provider`` resolution.
"""
_inner: CredentialProvider
def __init__(self) -> None:
from redis_entraid.cred_provider import create_from_default_azure_credential
self._inner = create_from_default_azure_credential(
scopes=(AZURE_REDIS_SCOPE,),
)
@override
def get_credentials(self) -> Union[tuple[str], tuple[str, str]]:
return self._inner.get_credentials()
def get_azure_credential_provider() -> CredentialProvider:
"""Create a redis-py credential provider for Azure Entra ID authentication."""
from redis_entraid.cred_provider import create_from_default_azure_credential
return create_from_default_azure_credential(
scopes=(AZURE_REDIS_SCOPE,),
)
def apply_azure_redis_auth(params: dict) -> None:
"""Apply Azure Entra ID authentication to a Redis connection params dict.
Removes static username/password and injects a credential_provider instead.
"""
params.pop("username", None)
params.pop("password", None)
params["credential_provider"] = get_azure_credential_provider()
def apply_azure_celery_broker_auth(celery_app, broker_url: str) -> None:
"""Configure Celery broker to authenticate via Azure Entra ID credential provider."""
cred_param = "credential_provider=extensions.azure.AzureEntraIdCredentialProvider"
sep = "&" if "?" in broker_url else "?"
broker_url_with_cred = f"{broker_url}{sep}{cred_param}"
celery_app.conf.update(
broker_read_url=broker_url_with_cred,
broker_write_url=broker_url_with_cred,
)
+18 -4
View File
@@ -1,3 +1,4 @@
import logging
import ssl
from datetime import timedelta
from typing import Any
@@ -9,9 +10,12 @@ from typing_extensions import TypedDict
from configs import dify_config
from dify_app import DifyApp
from extensions.azure import AzureEntraIdCredentialProvider, apply_azure_celery_broker_auth
from extensions.redis_names import normalize_redis_key_prefix
from extensions.workflow_warm_shutdown import setup_workflow_warm_shutdown_handler
logger = logging.getLogger(__name__)
class _CelerySentinelKwargsDict(TypedDict):
socket_timeout: float | None
@@ -108,10 +112,12 @@ def init_app(app: DifyApp) -> Celery:
broker_transport_options = get_celery_broker_transport_options()
broker_url = dify_config.CELERY_BROKER_URL
celery_app = Celery(
app.name,
task_cls=FlaskTask,
broker=dify_config.CELERY_BROKER_URL,
broker=broker_url,
backend=dify_config.CELERY_BACKEND,
)
@@ -128,9 +134,12 @@ def init_app(app: DifyApp) -> Celery:
)
if dify_config.CELERY_BACKEND == "redis":
celery_app.conf.update(
result_backend_transport_options=broker_transport_options,
)
redis_backend_conf: dict[str, Any] = {
"result_backend_transport_options": broker_transport_options,
}
if dify_config.REDIS_USE_AZURE_MANAGED_IDENTITY:
redis_backend_conf["redis_backend_credential_provider"] = AzureEntraIdCredentialProvider()
celery_app.conf.update(**redis_backend_conf)
# Apply SSL configuration if enabled
ssl_options = get_celery_ssl_options()
@@ -141,6 +150,10 @@ def init_app(app: DifyApp) -> Celery:
redis_backend_use_ssl=ssl_options if dify_config.CELERY_BACKEND == "redis" else None,
)
if dify_config.REDIS_USE_AZURE_MANAGED_IDENTITY and broker_url:
apply_azure_celery_broker_auth(celery_app, broker_url)
logger.info("Celery broker: using Azure Managed Identity (Entra ID) authentication")
if dify_config.LOG_FILE:
celery_app.conf.update(
worker_logfile=dify_config.LOG_FILE,
@@ -157,6 +170,7 @@ def init_app(app: DifyApp) -> Celery:
"tasks.regenerate_summary_index_task", # summary index regeneration
"tasks.initialize_created_app_rbac_access_task", # app access initialization
"tasks.install_default_plugins_task", # tenant default plugin installation
"tasks.refresh_billing_vector_space_task", # billing vector-space cache refresh
"tasks.app_generate.resume_agent_app_task", # ENG-635: Agent v2 chat ask_human resume
"tasks.workflow_run_archive_download_tasks", # workflow-run archive download preparation
]
+14 -1
View File
@@ -21,6 +21,7 @@ from typing_extensions import TypedDict
from configs import dify_config
from dify_app import DifyApp
from extensions.azure import apply_azure_redis_auth, get_azure_credential_provider
from extensions.redis_names import (
normalize_redis_key_prefix,
serialize_redis_name,
@@ -438,6 +439,10 @@ def _create_standalone_client(redis_params: RedisBaseParamsDict) -> Union[redis.
"connection_class": connection_class,
}
if dify_config.REDIS_USE_AZURE_MANAGED_IDENTITY:
apply_azure_redis_auth(params)
logger.info("Redis: using Azure Managed Identity (Entra ID) authentication")
if dify_config.REDIS_MAX_CONNECTIONS:
params["max_connections"] = dify_config.REDIS_MAX_CONNECTIONS
@@ -457,12 +462,20 @@ def _create_pubsub_client(pubsub_url: str, use_clusters: bool) -> redis.Redis |
kwargs: dict[str, Any] = {**health_params}
if max_conns:
kwargs["max_connections"] = max_conns
if dify_config.REDIS_USE_AZURE_MANAGED_IDENTITY:
kwargs["credential_provider"] = get_azure_credential_provider()
kwargs["ssl_cert_reqs"] = ssl.CERT_NONE
logger.info("PubSub Redis (cluster): using Azure Managed Identity (Entra ID) authentication")
return RedisCluster.from_url(pubsub_url, **kwargs)
standalone_health_params: dict[str, Any] = dict(_get_connection_health_params())
kwargs = {**standalone_health_params}
if max_conns:
kwargs["max_connections"] = max_conns
if dify_config.REDIS_USE_AZURE_MANAGED_IDENTITY:
kwargs["credential_provider"] = get_azure_credential_provider()
kwargs["ssl_cert_reqs"] = ssl.CERT_NONE
logger.info("PubSub Redis: using Azure Managed Identity (Entra ID) authentication")
return redis.Redis.from_url(pubsub_url, **kwargs)
@@ -486,7 +499,7 @@ def init_app(app: DifyApp):
global _pubsub_redis_client
_pubsub_redis_client = client
if dify_config.normalized_pubsub_redis_url:
if dify_config.PUBSUB_REDIS_URL:
_pubsub_redis_client = _create_pubsub_client(
dify_config.normalized_pubsub_redis_url, dify_config.PUBSUB_REDIS_USE_CLUSTERS
)
+13
View File
@@ -119,6 +119,19 @@ class Storage:
def delete(self, filename: str):
return self.storage_runner.delete(filename)
def generate_presigned_url(
self,
filename: str,
*,
expires_in: int,
content_type: str | None = None,
) -> str:
return self.storage_runner.generate_presigned_url(
filename,
expires_in=expires_in,
content_type=content_type,
)
def scan(self, path: str, files: bool = True, directories: bool = False) -> list[str]:
return self.storage_runner.scan(path, files=files, directories=directories)
+18
View File
@@ -92,3 +92,21 @@ class AwsS3Storage(BaseStorage):
@override
def delete(self, filename: str):
self.client.delete_object(Bucket=self.bucket_name, Key=filename)
@override
def generate_presigned_url(
self,
filename: str,
*,
expires_in: int,
content_type: str | None = None,
) -> str:
params = {"Bucket": self.bucket_name, "Key": filename}
if content_type:
params["ResponseContentType"] = content_type
return self.client.generate_presigned_url(
"get_object",
Params=params,
ExpiresIn=expires_in,
)
+10
View File
@@ -31,6 +31,16 @@ class BaseStorage(ABC):
def delete(self, filename: str):
raise NotImplementedError
def generate_presigned_url(
self,
filename: str,
*,
expires_in: int,
content_type: str | None = None,
) -> str:
"""Generate a temporary direct-download URL when the backend supports it."""
raise NotImplementedError("This storage backend doesn't support presigned URLs")
def scan(self, path, files=True, directories=False) -> list[str]:
"""
Scan files and directories in the given path.
+2 -2
View File
@@ -1,5 +1,5 @@
{
"commit": "4310e2d582d25e7de58183f27720afab01e123cf",
"openapiSha256": "5827ca930ce38462bfd1b2bef387efbf37eb7ffcaedde4558af2fbaeccbfbc4b",
"commit": "a0f50470612cc0b3656f89e4f2435aaf412e6e3b",
"openapiSha256": "f18910e9c45a64f0855e0643a7a626fb2889021b4f943458de86c6bd2469facb",
"repository": "https://github.com/langgenius/knowledge-fs"
}
+5 -2
View File
@@ -221,8 +221,11 @@ def current_timestamp() -> int:
def email(email):
# Define a regex pattern for email addresses
pattern = r"^[\w\.!#$%&'*+\-/=?^_`{|}~]+@([\w-]+\.)+[\w-]{2,}$"
# Check if the email matches the pattern
if re.match(pattern, email) is not None:
# Use re.fullmatch instead of re.match to reject trailing newlines.
# In Python, '$' matches at end-of-string OR just before a trailing newline,
# so re.match accepts "user@example.com\n". re.fullmatch requires the entire
# string to match, closing the mail header-injection vector. (#39234)
if re.fullmatch(pattern, email) is not None:
return email
error = f"{email} is not a valid email."
+2 -2
View File
@@ -23,9 +23,9 @@ class PaginatedResult[T]:
@property
def pages(self) -> int:
if self.per_page == 0:
if self.total == 0 or self.per_page == 0:
return 0
return max(1, math.ceil(self.total / self.per_page))
return math.ceil(self.total / self.per_page)
@property
def has_next(self) -> bool:
@@ -0,0 +1,77 @@
"""scope agent debug conversations by draft type
Revision ID: d2825e7b9c10
Revises: b8c9d0e1f2a3
Create Date: 2026-07-22 15:00:00.000000
"""
import sqlalchemy as sa
from alembic import op
import models
# revision identifiers, used by Alembic.
revision = "d2825e7b9c10"
down_revision = "b8c9d0e1f2a3"
branch_labels = None
depends_on = None
def upgrade():
# Existing pointers have always represented Build chat because the Agent
# detail API exposes them as ``debug_conversation_id`` for that surface.
op.add_column(
"agent_debug_conversations",
sa.Column(
"draft_type",
sa.String(length=32),
nullable=False,
server_default=sa.text("'debug_build'"),
),
)
op.drop_constraint(
"agent_debug_conversation_agent_account_unique",
"agent_debug_conversations",
type_="unique",
)
op.create_unique_constraint(
"agent_debug_conversation_agent_account_draft_type_unique",
"agent_debug_conversations",
["tenant_id", "agent_id", "account_id", "draft_type"],
)
def downgrade():
debug_conversations = sa.table(
"agent_debug_conversations",
sa.column("tenant_id", models.types.StringUUID()),
sa.column("agent_id", models.types.StringUUID()),
sa.column("account_id", models.types.StringUUID()),
sa.column("draft_type", sa.String(length=32)),
)
build_conversations = debug_conversations.alias("build_conversations")
op.get_bind().execute(
sa.delete(debug_conversations).where(
debug_conversations.c.draft_type == "draft",
sa.exists(
sa.select(sa.literal(1)).where(
build_conversations.c.tenant_id == debug_conversations.c.tenant_id,
build_conversations.c.agent_id == debug_conversations.c.agent_id,
build_conversations.c.account_id == debug_conversations.c.account_id,
build_conversations.c.draft_type == "debug_build",
)
),
)
)
op.drop_constraint(
"agent_debug_conversation_agent_account_draft_type_unique",
"agent_debug_conversations",
type_="unique",
)
op.create_unique_constraint(
"agent_debug_conversation_agent_account_unique",
"agent_debug_conversations",
["tenant_id", "agent_id", "account_id"],
)
op.drop_column("agent_debug_conversations", "draft_type")
+12 -3
View File
@@ -222,11 +222,13 @@ class Agent(DefaultFieldsMixin, Base):
class AgentDebugConversation(DefaultFieldsMixin, Base):
"""Per-account console debug conversation for an Agent App.
"""Per-account, per-draft console debug conversation for an Agent App.
Agent App preview state must be isolated by editor account. The Agent row is
shared by everyone in the workspace, so this table owns the user-specific
conversation pointer used by console debug chat.
conversation pointers used by console debug chat. ``draft`` is the Preview
conversation and ``debug_build`` is the Build conversation; they must never
share persisted messages or runtime sessions.
"""
__tablename__ = "agent_debug_conversations"
@@ -236,7 +238,8 @@ class AgentDebugConversation(DefaultFieldsMixin, Base):
"tenant_id",
"agent_id",
"account_id",
name="agent_debug_conversation_agent_account_unique",
"draft_type",
name="agent_debug_conversation_agent_account_draft_type_unique",
),
Index("agent_debug_conversation_conversation_idx", "conversation_id"),
Index("agent_debug_conversation_account_idx", "tenant_id", "account_id"),
@@ -246,6 +249,12 @@ class AgentDebugConversation(DefaultFieldsMixin, Base):
agent_id: Mapped[str] = mapped_column(StringUUID, nullable=False)
app_id: Mapped[str] = mapped_column(StringUUID, nullable=False)
account_id: Mapped[str] = mapped_column(StringUUID, nullable=False)
draft_type: Mapped[AgentConfigDraftType] = mapped_column(
EnumText(AgentConfigDraftType, length=32),
nullable=False,
default=AgentConfigDraftType.DEBUG_BUILD,
server_default=sa.text("'debug_build'"),
)
conversation_id: Mapped[str] = mapped_column(StringUUID, nullable=False)
+22 -15
View File
@@ -7,6 +7,7 @@ from typing import Annotated, Any, Final, Literal, Self
from pydantic import BaseModel, ConfigDict, Field, WithJsonSchema, field_validator, model_validator
from core.rag.entities.metadata_entities import ConditionValue, SupportedComparisonOperator
from core.tools.entities.tool_entities import ToolProviderType
from core.workflow.file_reference import is_canonical_file_reference
from graphon.file import FileTransferMethod, FileType
@@ -43,18 +44,28 @@ _DECLARED_OUTPUT_CHILDREN_JSON_SCHEMA = {
},
"description": {"anyOf": [{"type": "string"}, {"type": "null"}]},
"required": {"type": "boolean"},
"file": {"type": "object", "additionalProperties": True},
"file": {
"anyOf": [
{"type": "object", "additionalProperties": True},
{"type": "null"},
]
},
"array_item": {
"type": "object",
"additionalProperties": True,
"properties": {
"type": {
"type": "string",
"enum": [item.value for item in DeclaredOutputType],
"anyOf": [
{
"type": "object",
"additionalProperties": True,
"properties": {
"type": {
"type": "string",
"enum": [item.value for item in DeclaredOutputType],
},
"description": {"anyOf": [{"type": "string"}, {"type": "null"}]},
"children": {"type": "array", "items": {"type": "object", "additionalProperties": True}},
},
},
"description": {"anyOf": [{"type": "string"}, {"type": "null"}]},
"children": {"type": "array", "items": {"type": "object", "additionalProperties": True}},
},
{"type": "null"},
]
},
"children": {"type": "array", "items": {"type": "object", "additionalProperties": True}},
},
@@ -653,11 +664,7 @@ class AgentSoulDifyToolConfig(BaseModel):
model_config = ConfigDict(extra="ignore")
enabled: bool = True
# ``plugin`` remains the default for legacy Agent Soul payloads. The runtime
# now also accepts ``builtin`` / ``api`` / ``workflow`` / ``mcp`` here and
# routes them through ``dify.core.tools``; keeping the default narrow still
# makes a missing field resolve against the plugin provider table.
provider_type: str = "plugin"
provider_type: ToolProviderType
provider_id: str | None = Field(default=None, max_length=255)
plugin_id: str | None = Field(default=None, max_length=255)
provider: str | None = Field(default=None, max_length=255)
+32 -14
View File
@@ -56,6 +56,7 @@ from .provider_ids import GenericProviderID
from .types import EnumText, LongText, StringUUID
if TYPE_CHECKING:
from .agent import Agent
from .workflow import Workflow
@@ -501,25 +502,42 @@ class App(Base):
Resolved via ``Agent.app_id`` so the console can open the Composer in
roster-detail mode from the app id. ``None`` for non-agent apps.
"""
agent = self.agent_app_binding_with_session(session=session)
return agent.id if agent else None
def agent_app_binding_with_session(self, *, session: Session, include_archived: bool = False) -> Agent | None:
"""For an Agent App (mode=agent), the Agent bound to it.
A roster Agent is bound through ``Agent.app_id``; a workflow-only Agent
is bound to its hidden runtime backing App through
``Agent.backing_app_id``. Callers branch on ``Agent.scope`` to tell the
public roster Agent App apart from the hidden backing App. Archived
Agents are excluded unless ``include_archived`` is set (authorization
gates must keep covering an Agent App after its Agent is archived).
``None`` for non-agent apps and unbound agent apps.
"""
if self.mode != AppMode.AGENT:
return None
from .agent import APP_BACKED_AGENT_SOURCES, Agent, AgentScope, AgentStatus
agent = session.scalar(
select(Agent).where(
Agent.tenant_id == self.tenant_id,
sa.or_(
sa.and_(
Agent.app_id == self.id,
Agent.scope == AgentScope.ROSTER,
Agent.source.in_(APP_BACKED_AGENT_SOURCES),
),
Agent.backing_app_id == self.id,
conditions = [
Agent.tenant_id == self.tenant_id,
sa.or_(
sa.and_(
Agent.app_id == self.id,
Agent.scope == AgentScope.ROSTER,
Agent.source.in_(APP_BACKED_AGENT_SOURCES),
),
Agent.status == AgentStatus.ACTIVE,
)
)
return agent.id if agent else None
sa.and_(
Agent.backing_app_id == self.id,
Agent.scope == AgentScope.WORKFLOW_ONLY,
),
),
]
if not include_archived:
conditions.append(Agent.status == AgentStatus.ACTIVE)
return session.scalar(select(Agent).where(*conditions).limit(1))
@property
def api_base_url(self) -> str:
+66 -11
View File
@@ -260,7 +260,12 @@ Get account avatar url
| 200 | Success | **application/json**: [AccountResponse](#accountresponse)<br> |
### [POST] /activate
**Accept an invitation without letting an existing session act for another account**
Activate account with invitation token
Token-only activation remains available for legacy clients. When the request already
carries a console session, that session must belong to the account encoded in the
invitation before the token is consumed or tenant membership is changed.
#### Request Body
@@ -531,6 +536,7 @@ Run a build-draft Agent App turn that asks the agent to push config updates
| Code | Description | Schema |
| ---- | ----------- | ------ |
| 200 | Agent build draft | **application/json**: [AgentBuildDraftResponse](#agentbuilddraftresponse)<br> |
| 404 | Agent build draft not found | |
### [PUT] /agent/{agent_id}/build-draft
#### Parameters
@@ -958,6 +964,12 @@ Stop a running Agent App chat message generation
| ---- | ---------- | ----------- | -------- | ------ |
| agent_id | path | | Yes | string (uuid) |
#### Request Body
| Required | Schema |
| -------- | ------ |
| No | **application/json**: [AgentDebugConversationRefreshPayload](#agentdebugconversationrefreshpayload)<br> |
#### Responses
| Code | Description | Schema |
@@ -11332,6 +11344,12 @@ Returns permission flags that control workspace features like member invitations
| 200 | Success | **application/json**: [PluginDecodeResponse](#plugindecoderesponse)<br> |
### [POST] /workspaces/current/plugin/upload/pkg
#### Request Body
| Required | Schema |
| -------- | ------ |
| Yes | **multipart/form-data**: { **"pkg"**: binary }<br> |
#### Responses
| Code | Description | Schema |
@@ -13278,7 +13296,7 @@ Model class for AI model.
| maintainer | string | | No |
| max_active_requests | integer | | No |
| mode | string | | Yes |
| model_config | [ModelConfig](#modelconfig) | | No |
| model_config | [AppModelConfigResponse](#appmodelconfigresponse) | | No |
| name | string | | Yes |
| permission_keys | [ string ] | | No |
| role | string | | No |
@@ -13899,6 +13917,12 @@ Stable Agent Soul reference to one normalized skill archive.
| date | string | | Yes |
| message_count | integer | | Yes |
#### AgentDebugConversationRefreshPayload
| Name | Type | Description | Required |
| ---- | ---- | ----------- | -------- |
| draft_type | [AgentConfigDraftType](#agentconfigdrafttype) | Agent draft surface whose conversation should be refreshed | No |
#### AgentDebugConversationRefreshResponse
| Name | Type | Description | Required |
@@ -14737,7 +14761,7 @@ should send ``plugin_id`` + ``provider`` when available.
| plugin_id | string | | No |
| provider | string | | No |
| provider_id | string | | No |
| provider_type | string, <br>**Default:** plugin | | No |
| provider_type | [ToolProviderType](#toolprovidertype) | | Yes |
| runtime_parameters | object | | No |
| tool_name | string | | No |
@@ -15348,7 +15372,6 @@ This class is used to store the schema information of an api based tool.
| Name | Type | Description | Required |
| ---- | ---- | ----------- | -------- |
| access_mode | string | | No |
| app_model_config | [ModelConfig](#modelconfig) | | No |
| created_at | integer | | No |
| created_by | string | | No |
| description | string | | No |
@@ -15358,7 +15381,8 @@ This class is used to store the schema information of an api based tool.
| icon_background | string | | No |
| id | string | | Yes |
| maintainer | string | | No |
| mode_compatible_with_agent | string | | Yes |
| mode | string | | Yes |
| model_config | [AppModelConfigResponse](#appmodelconfigresponse) | | No |
| name | string | | Yes |
| permission_keys | [ string ] | | No |
| tags | [ [Tag](#tag) ] | | No |
@@ -15420,7 +15444,7 @@ This class is used to store the schema information of an api based tool.
| maintainer | string | | No |
| max_active_requests | integer | | No |
| mode | string | | Yes |
| model_config | [ModelConfig](#modelconfig) | | No |
| model_config | [AppModelConfigResponse](#appmodelconfigresponse) | | No |
| name | string | | Yes |
| permission_keys | [ string ] | | No |
| site | [AppDetailSiteResponse](#appdetailsiteresponse) | | No |
@@ -15519,6 +15543,35 @@ AppMCPServer Status Enum
| ---- | ---- | ----------- | -------- |
| AppMCPServerStatus | string | AppMCPServer Status Enum | |
#### AppModelConfigResponse
| Name | Type | Description | Required |
| ---- | ---- | ----------- | -------- |
| agent_mode | | | No |
| annotation_reply | | | No |
| chat_prompt_config | | | No |
| completion_prompt_config | | | No |
| created_at | integer | | No |
| created_by | string | | No |
| dataset_configs | | | No |
| dataset_query_variable | string | | No |
| external_data_tools | | | No |
| file_upload | | | No |
| model | | | No |
| more_like_this | | | No |
| opening_statement | string | | No |
| pre_prompt | string | | No |
| prompt_type | string | | No |
| retriever_resource | | | No |
| sensitive_word_avoidance | | | No |
| speech_to_text | | | No |
| suggested_questions | | | No |
| suggested_questions_after_answer | | | No |
| text_to_speech | | | No |
| updated_at | integer | | No |
| updated_by | string | | No |
| user_input_form | | | No |
#### AppNamePayload
| Name | Type | Description | Required |
@@ -17147,7 +17200,7 @@ about. Stage 4 §4.2.
| Name | Type | Description | Required |
| ---- | ---- | ----------- | -------- |
| children | [ { **"array_item"**: { **"children"**: [ object ], **"description"**: , **"type"**: string, <br>**Available values:** "array", "boolean", "file", "number", "object", "string" }, **"children"**: [ object ], **"description"**: , **"file"**: object, **"name"**: string, **"required"**: boolean, **"type"**: string, <br>**Available values:** "array", "boolean", "file", "number", "object", "string" } ] | | No |
| children | [ { **"array_item"**: , **"children"**: [ object ], **"description"**: , **"file"**: , **"name"**: string, **"required"**: boolean, **"type"**: string, <br>**Available values:** "array", "boolean", "file", "number", "object", "string" } ] | | No |
| description | string | | No |
| type | [DeclaredOutputType](#declaredoutputtype) | | Yes |
@@ -17176,7 +17229,7 @@ code can call ``output.failure_strategy.on_failure`` without None-guards.
| ---- | ---- | ----------- | -------- |
| array_item | [DeclaredArrayItem](#declaredarrayitem) | | No |
| check | [DeclaredOutputCheckConfig](#declaredoutputcheckconfig) | | No |
| children | [ { **"array_item"**: { **"children"**: [ object ], **"description"**: , **"type"**: string, <br>**Available values:** "array", "boolean", "file", "number", "object", "string" }, **"children"**: [ object ], **"description"**: , **"file"**: object, **"name"**: string, **"required"**: boolean, **"type"**: string, <br>**Available values:** "array", "boolean", "file", "number", "object", "string" } ] | | No |
| children | [ { **"array_item"**: , **"children"**: [ object ], **"description"**: , **"file"**: , **"name"**: string, **"required"**: boolean, **"type"**: string, <br>**Available values:** "array", "boolean", "file", "number", "object", "string" } ] | | No |
| description | string | | No |
| failure_strategy | [DeclaredOutputFailureStrategy](#declaredoutputfailurestrategy) | | No |
| file | [DeclaredOutputFileConfig](#declaredoutputfileconfig) | | No |
@@ -21954,9 +22007,9 @@ The subscription constructor of the trigger provider
| Name | Type | Description | Required |
| ---- | ---- | ----------- | -------- |
| hash | string | | No |
| result | string | | No |
| updated_at | string | | No |
| hash | string | | Yes |
| result | string | | Yes |
| updated_at | integer | | Yes |
#### SystemConfigurationResponse
@@ -21988,6 +22041,7 @@ Model class for provider system configuration response.
| is_allow_create_workspace | boolean | | Yes |
| is_allow_register | boolean | | Yes |
| is_email_setup | boolean | | Yes |
| knowledge_fs_enabled | boolean | | Yes |
| license | [LicenseModel](#licensemodel) | | Yes |
| max_plugin_package_size | integer, <br>**Default:** 15728640 | | Yes |
| plugin_installation_permission | [PluginInstallationPermissionModel](#plugininstallationpermissionmodel) | | Yes |
@@ -22281,7 +22335,7 @@ Tool label
#### ToolParameter
Overrides type
Tool-specific parameter declaration and invocation-value normalization.
| Name | Type | Description | Required |
| ---- | ---- | ----------- | -------- |
@@ -22294,6 +22348,7 @@ Overrides type
| llm_description | string | | No |
| max | number<br>integer | | No |
| min | number<br>integer | | No |
| multiple | boolean | Whether the parameter is multiple select, only valid for select or dynamic-select type | No |
| name | string | The name of the parameter | Yes |
| options | [ [PluginParameterOption](#pluginparameteroption) ] | | No |
| placeholder | [I18nObject](#i18nobject) | The placeholder presented to the user | No |
+2 -1
View File
@@ -1582,6 +1582,7 @@ Default configuration for form inputs.
| is_allow_create_workspace | boolean | | Yes |
| is_allow_register | boolean | | Yes |
| is_email_setup | boolean | | Yes |
| knowledge_fs_enabled | boolean | | Yes |
| license | [LicenseModel](#licensemodel) | | Yes |
| max_plugin_package_size | integer, <br>**Default:** 15728640 | | Yes |
| plugin_installation_permission | [PluginInstallationPermissionModel](#plugininstallationpermissionmodel) | | Yes |
@@ -1733,7 +1734,7 @@ in form definiton, or a variable while the workflow is running.
| icon | string | | No |
| icon_background | string | | No |
| icon_type | string | | No |
| icon_url | string | | Yes |
| icon_url | string | | No |
| input_placeholder | string | | No |
| privacy_policy | string | | No |
| prompt_public | boolean | | No |
+1
View File
@@ -21,6 +21,7 @@ dependencies = [
"psycopg2-binary>=2.9.12,<3.0.0",
"python-socketio>=5.13.0,<6.0.0",
"redis[hiredis]>=7.4.0,<8.0.0",
"redis-entraid>=1.2.0,<2.0.0",
"sendgrid>=6.12.5,<7.0.0",
"sseclient-py>=1.8.0,<2.0.0",
# Stable: production-proven, cap below the next major
+75
View File
@@ -1164,6 +1164,20 @@ class AgentComposerService:
agent.active_config_is_published = True
agent.updated_by = account_id
binding.current_snapshot_id = version.id
normal_draft = cls._get_agent_draft(
session=session,
tenant_id=tenant_id,
agent_id=agent.id,
draft_type=AgentConfigDraftType.DRAFT,
account_id=None,
)
if normal_draft is not None and cls._rebase_workflow_only_normal_draft(
agent=agent,
draft=normal_draft,
snapshot=version,
updated_by=account_id,
):
session.flush()
binding.updated_by = account_id
return binding
@@ -1748,6 +1762,47 @@ class AgentComposerService:
stmt = stmt.where(AgentConfigDraft.account_id.is_(None))
return session.scalar(stmt.order_by(AgentConfigDraft.updated_at.desc()).limit(1))
@classmethod
def get_or_create_normal_agent_draft(
cls,
*,
session: Session,
tenant_id: str,
agent: Agent,
created_by: str | None,
) -> AgentConfigDraft:
"""Resolve the shared Preview draft, rebasing inline agents when needed."""
return cls._get_or_create_agent_draft(
session=session,
tenant_id=tenant_id,
agent=agent,
draft_type=AgentConfigDraftType.DRAFT,
account_id=None,
created_by=created_by,
)
@staticmethod
def _rebase_workflow_only_normal_draft(
*,
agent: Agent,
draft: AgentConfigDraft,
snapshot: AgentConfigSnapshot,
updated_by: str | None,
) -> bool:
if (
agent.scope != AgentScope.WORKFLOW_ONLY
or draft.draft_type != AgentConfigDraftType.DRAFT
or draft.account_id is not None
or not agent.active_config_snapshot_id
or draft.base_snapshot_id == agent.active_config_snapshot_id
or snapshot.id != agent.active_config_snapshot_id
):
return False
draft.base_snapshot_id = snapshot.id
draft.config_snapshot = AgentSoulConfig.model_validate(snapshot.config_snapshot_dict)
draft.updated_by = updated_by
return True
@classmethod
def _get_or_create_agent_draft(
cls,
@@ -1767,6 +1822,26 @@ class AgentComposerService:
account_id=account_id,
)
if draft is not None:
if (
agent.scope == AgentScope.WORKFLOW_ONLY
and draft_type == AgentConfigDraftType.DRAFT
and draft.account_id is None
and agent.active_config_snapshot_id
and draft.base_snapshot_id != agent.active_config_snapshot_id
):
active_snapshot = cls._get_version_if_present(
session=session,
tenant_id=tenant_id,
agent_id=agent.id,
version_id=agent.active_config_snapshot_id,
)
if active_snapshot is not None and cls._rebase_workflow_only_normal_draft(
agent=agent,
draft=draft,
snapshot=active_snapshot,
updated_by=agent.updated_by or agent.created_by,
):
session.flush()
return draft
base_snapshot = cls._get_version_if_present(
session=session,
+94 -23
View File
@@ -430,7 +430,11 @@ class AgentRosterService:
agent.active_config_has_model = agent_soul_has_model(soul)
agent.active_config_is_published = False
self._session.flush()
self._get_or_create_agent_app_debug_conversation(agent=agent, account_id=account_id)
self._get_or_create_agent_app_debug_conversation(
agent=agent,
account_id=account_id,
draft_type=AgentConfigDraftType.DEBUG_BUILD,
)
return agent
def create_hidden_backing_app_for_workflow_agent(
@@ -527,7 +531,11 @@ class AgentRosterService:
self._session.flush()
return backing_app.id
def _get_or_create_agent_app_debug_conversation(self, *, agent: Agent, account_id: str) -> str:
def _get_or_create_agent_app_debug_conversation(
self, *, agent: Agent, account_id: str, draft_type: AgentConfigDraftType
) -> str:
"""Return the editor's conversation for one Agent draft surface."""
backing_app_id = self._ensure_workflow_agent_backing_app(agent=agent, account_id=account_id)
if not backing_app_id:
raise AgentNotFoundError()
@@ -537,6 +545,7 @@ class AgentRosterService:
AgentDebugConversation.tenant_id == agent.tenant_id,
AgentDebugConversation.agent_id == agent.id,
AgentDebugConversation.account_id == account_id,
AgentDebugConversation.draft_type == draft_type,
)
)
if mapping is not None:
@@ -570,6 +579,7 @@ class AgentRosterService:
agent_id=agent.id,
app_id=backing_app_id,
account_id=account_id,
draft_type=draft_type,
conversation_id=conversation_id,
)
)
@@ -577,9 +587,15 @@ class AgentRosterService:
return conversation_id
def get_or_create_agent_app_debug_conversation_id(
self, *, tenant_id: str, agent_id: str, account_id: str, commit: bool = True
self,
*,
tenant_id: str,
agent_id: str,
account_id: str,
draft_type: AgentConfigDraftType = AgentConfigDraftType.DEBUG_BUILD,
commit: bool = True,
) -> str:
"""Return the current editor's debug conversation for an Agent App."""
"""Return the current editor's Build or Preview conversation for an Agent App."""
agent = self._session.scalar(
select(Agent).where(
@@ -591,13 +607,24 @@ class AgentRosterService:
if agent is None:
raise AgentNotFoundError()
conversation_id = self._get_or_create_agent_app_debug_conversation(agent=agent, account_id=account_id)
conversation_id = self._get_or_create_agent_app_debug_conversation(
agent=agent,
account_id=account_id,
draft_type=draft_type,
)
if commit:
self._session.commit()
return conversation_id
def load_agent_app_debug_conversation_id(self, *, tenant_id: str, agent_id: str, account_id: str) -> str | None:
"""Return the current editor's existing debug conversation without creating or repairing rows."""
def load_agent_app_debug_conversation_id(
self,
*,
tenant_id: str,
agent_id: str,
account_id: str,
draft_type: AgentConfigDraftType = AgentConfigDraftType.DEBUG_BUILD,
) -> str | None:
"""Return the editor's existing scoped conversation without creating or repairing rows."""
return self._session.scalar(
select(Conversation.id)
@@ -606,6 +633,7 @@ class AgentRosterService:
AgentDebugConversation.tenant_id == tenant_id,
AgentDebugConversation.agent_id == agent_id,
AgentDebugConversation.account_id == account_id,
AgentDebugConversation.draft_type == draft_type,
AgentDebugConversation.app_id == Conversation.app_id,
Conversation.from_source == ConversationFromSource.CONSOLE,
Conversation.from_account_id == account_id,
@@ -626,16 +654,21 @@ class AgentRosterService:
)
def refresh_agent_app_debug_conversation_id(
self, *, tenant_id: str, agent_id: str, account_id: str, commit: bool = True
self,
*,
tenant_id: str,
agent_id: str,
account_id: str,
draft_type: AgentConfigDraftType = AgentConfigDraftType.DEBUG_BUILD,
commit: bool = True,
) -> str:
"""Start a new console debug conversation for the current Agent App editor.
"""Start a new scoped console conversation for the current Agent App editor.
If this account already has a debug conversation mapping, the previous
If this account already has a mapping for the requested draft surface, the previous
conversation is abandoned first: any ACTIVE conversation-owned Agent
runtime sessions for that old conversation are sent through best-effort
backend cleanup and then retired locally even when enqueueing fails.
The debug mapping is then repointed to the freshly created
conversation.
backend cleanup and then retired locally even when enqueueing fails. The
other draft surface is left untouched.
"""
agent = self._session.scalar(
@@ -663,6 +696,7 @@ class AgentRosterService:
AgentDebugConversation.tenant_id == tenant_id,
AgentDebugConversation.agent_id == agent_id,
AgentDebugConversation.account_id == account_id,
AgentDebugConversation.draft_type == draft_type,
)
)
if mapping is None:
@@ -672,6 +706,7 @@ class AgentRosterService:
agent_id=agent_id,
app_id=backing_app_id,
account_id=account_id,
draft_type=draft_type,
conversation_id=conversation_id,
)
)
@@ -683,6 +718,7 @@ class AgentRosterService:
tenant_id=tenant_id,
agent_id=agent_id,
account_id=account_id,
draft_type=draft_type,
app_id=previous_app_id or backing_app_id,
conversation_id=previous_conversation_id,
)
@@ -699,6 +735,7 @@ class AgentRosterService:
tenant_id: str,
agent_id: str,
account_id: str,
draft_type: AgentConfigDraftType,
app_id: str,
conversation_id: str,
) -> None:
@@ -727,7 +764,8 @@ class AgentRosterService:
session_snapshot=stored_session.session_snapshot,
runtime_layer_specs=stored_session.runtime_layer_specs,
idempotency_key=(
f"{tenant_id}:{agent_id}:{account_id}:{conversation_id}:debug-session-cleanup:"
f"{tenant_id}:{agent_id}:{account_id}:{draft_type.value}:{conversation_id}:"
"debug-session-cleanup:"
f"{stored_session.scope.agent_id}:"
f"{stored_session.scope.agent_config_snapshot_id or 'no-config'}:"
f"{stored_session.backend_run_id or 'no-run'}"
@@ -738,6 +776,7 @@ class AgentRosterService:
"conversation_id": stored_session.scope.conversation_id,
"agent_id": stored_session.scope.agent_id,
"agent_config_snapshot_id": stored_session.scope.agent_config_snapshot_id,
"draft_type": draft_type.value,
"previous_agent_backend_run_id": stored_session.backend_run_id,
},
)
@@ -772,9 +811,14 @@ class AgentRosterService:
)
def load_or_create_agent_app_debug_conversation_ids_by_agent_id(
self, *, tenant_id: str, agents: list[Agent], account_id: str
self,
*,
tenant_id: str,
agents: list[Agent],
account_id: str,
draft_type: AgentConfigDraftType = AgentConfigDraftType.DEBUG_BUILD,
) -> dict[str, str]:
"""Return per-account debug conversations for a page of Agent Apps."""
"""Return per-account scoped conversations for a page of Agent Apps."""
conversation_ids_by_agent_id: dict[str, str] = {}
changed = False
@@ -784,6 +828,7 @@ class AgentRosterService:
conversation_ids_by_agent_id[agent.id] = self._get_or_create_agent_app_debug_conversation(
agent=agent,
account_id=account_id,
draft_type=draft_type,
)
changed = True
if changed:
@@ -866,16 +911,14 @@ class AgentRosterService:
raise AgentNotFoundError()
return app
def get_agent_runtime_app_model(self, *, tenant_id: str, agent_id: str) -> App:
"""Resolve the App that backs an Agent runtime surface.
def _get_runtime_resolvable_agent(self, *, tenant_id: str, agent_id: str) -> Agent | None:
"""Load an Agent that is eligible to resolve to a runtime backing App.
Roster Agents use their public Agent App. Workflow-only Agents use a
hidden Agent App stored in ``backing_app_id`` so console chat/logs can
reuse the app runtime without exposing the resource in workspace app
lists.
Shared by the runtime resolver and the read-only authorization resolver
so both agree on what counts as a resolvable Agent.
"""
agent = self._session.scalar(
return self._session.scalar(
select(Agent)
.where(
Agent.tenant_id == tenant_id,
@@ -893,6 +936,34 @@ class AgentRosterService:
)
.limit(1)
)
def peek_authz_app_id(self, *, tenant_id: str, agent_id: str) -> str | None:
"""Resolve the App id whose access policy governs an Agent.
Roster Agents are governed by their own Agent App, while workflow-only
Agents are governed by their parent workflow App: the hidden runtime
backing App never receives a resource access policy, so it must not be
used for authorization. Stays read-only unlike
:meth:`get_agent_runtime_app_model`, this never materializes the hidden
backing App. Returns ``None`` when the Agent does not resolve, leaving
the caller to decide how to treat it.
"""
agent = self._get_runtime_resolvable_agent(tenant_id=tenant_id, agent_id=agent_id)
if agent is None:
return None
return agent.app_id
def get_agent_runtime_app_model(self, *, tenant_id: str, agent_id: str) -> App:
"""Resolve the App that backs an Agent runtime surface.
Roster Agents use their public Agent App. Workflow-only Agents use a
hidden Agent App stored in ``backing_app_id`` so console chat/logs can
reuse the app runtime without exposing the resource in workspace app
lists.
"""
agent = self._get_runtime_resolvable_agent(tenant_id=tenant_id, agent_id=agent_id)
if agent is None:
raise AgentNotFoundError()
should_commit_backing_app = agent.scope == AgentScope.WORKFLOW_ONLY and not agent.backing_app_id
+26
View File
@@ -19,6 +19,7 @@ from configs import dify_config
from constants.dsl_version import CURRENT_APP_DSL_VERSION
from core.file import remote_fetcher
from core.plugin.entities.plugin import PluginDependency
from core.rbac import RBACPermission
from core.trigger.constants import (
TRIGGER_PLUGIN_NODE_TYPE,
TRIGGER_SCHEDULE_NODE_TYPE,
@@ -43,7 +44,9 @@ from services.agent.dsl_service import AgentDslService, AgentPackage
from services.agent.workflow_publish_service import WorkflowAgentPublishService
from services.dsl_content import DSL_MAX_SIZE, dsl_content_size
from services.dsl_version import check_version_compatibility
from services.enterprise.rbac_service import RBACService
from services.entities.dsl_entities import CheckDependenciesResult, DslImportWarning, ImportMode, ImportStatus
from services.errors.account import NoPermissionError
from services.errors.app import WorkflowNotFoundError
from services.plugin.dependencies_analysis import DependenciesAnalysisService
from services.workflow_draft_variable_service import WorkflowDraftVariableService
@@ -301,6 +304,9 @@ class AppDslService:
error=f"Invalid YAML format: {str(e)}",
)
except NoPermissionError:
raise
except Exception as e:
logger.exception("Failed to import app")
return Import(
@@ -364,6 +370,9 @@ class AppDslService:
warnings=self._warnings,
)
except NoPermissionError:
raise
except Exception as e:
logger.exception("Error confirming import")
return Import(
@@ -395,6 +404,21 @@ class AppDslService:
leaked_dependencies=leaked_dependencies,
)
@staticmethod
def _ensure_agent_manage_permission(account: Account) -> None:
"""Importing an Agent DSL creates a roster Agent, which requires ``agent.manage``."""
if not dify_config.RBAC_ENABLED:
return
if account.current_tenant_id is None:
raise ValueError("Current tenant is not set")
allowed = RBACService.CheckAccess.check(
account.current_tenant_id,
account.id,
scene=RBACPermission.AGENT_MANAGE,
)
if not allowed:
raise NoPermissionError("Agent management permission is required to import an Agent App")
def _create_or_update_app(
self,
*,
@@ -415,6 +439,8 @@ class AppDslService:
if not app_mode:
raise ValueError("loss app mode")
app_mode = AppMode(app_mode)
if app_mode == AppMode.AGENT:
self._ensure_agent_manage_permission(account)
# Set icon type
icon_type_value = icon_type or app_data.get("icon_type")
+7 -1
View File
@@ -217,12 +217,18 @@ class BillingService:
return _billing_info_adapter.validate_python(billing_info)
@classmethod
def get_vector_space(cls, tenant_id: str) -> _VectorSpaceQuota:
def get_vector_space(cls, tenant_id: str, bypass_cache: bool = False) -> _VectorSpaceQuota:
params = {"tenant_id": tenant_id}
if bypass_cache:
params["bypass_cache"] = "true"
return _vector_space_quota_adapter.validate_python(
cls._send_request("GET", "/subscription/vector-space", params=params)
)
@classmethod
def invalidate_vector_space_cache(cls, tenant_id: str) -> None:
cls.get_vector_space(tenant_id, bypass_cache=True)
@classmethod
def get_tenant_feature_plan_usage_info(cls, tenant_id: str):
"""Deprecated: Use get_quota_info instead."""
+10 -2
View File
@@ -1974,7 +1974,10 @@ class DocumentService:
if data_source_info and "upload_file_id" in data_source_info:
file_id = data_source_info["upload_file_id"]
document_was_deleted.send(
document.id, dataset_id=document.dataset_id, doc_form=document.doc_form, file_id=file_id
document.id,
dataset_id=document.dataset_id,
doc_form=document.doc_form,
file_id=file_id,
)
session.delete(document)
@@ -2013,7 +2016,12 @@ class DocumentService:
# Dispatch cleanup task after commit to avoid lock contention
# Task cleans up segments, files, and vector indexes
if deleted_document_ids and doc_form is not None:
batch_clean_document_task.delay(deleted_document_ids, dataset_ref.dataset_id, doc_form, file_ids)
batch_clean_document_task.delay(
deleted_document_ids,
dataset_ref.dataset_id,
doc_form,
file_ids,
)
@staticmethod
def rename_document(dataset_id: str, document_id: str, name: str, session: Session) -> Document:
+10 -8
View File
@@ -5,6 +5,7 @@ from typing import Any
import httpx
from configs import dify_config
from core.helper.trace_id_helper import generate_traceparent_header
from services.errors.enterprise import (
EnterpriseAPIBadRequestError,
@@ -96,12 +97,14 @@ class BaseRequest:
logger.debug("Failed to generate traceparent header", exc_info=True)
with httpx.Client(mounts=mounts) as client:
# IMPORTANT:
# - In httpx, passing timeout=None disables timeouts (infinite) and overrides the library default.
# - To preserve httpx's default timeout behavior for existing call sites, only pass the kwarg when set.
request_kwargs: dict[str, Any] = {"json": json, "params": params, "headers": headers}
if timeout is not None:
request_kwargs["timeout"] = timeout
# Callers that pass an explicit timeout keep it; everyone else gets the
# configured budget rather than httpx's implicit 5s default.
request_kwargs: dict[str, Any] = {
"json": json,
"params": params,
"headers": headers,
"timeout": timeout if timeout is not None else dify_config.ENTERPRISE_REQUEST_TIMEOUT,
}
response = client.request(method, url, **request_kwargs)
@@ -206,9 +209,8 @@ class EnterpriseRequest(BaseRequest):
"json": json,
"params": params,
"headers": {"Content-Type": "application/json", cls.secret_key_header: cls.secret_key, **inner_headers},
"timeout": timeout if timeout is not None else dify_config.ENTERPRISE_RBAC_REQUEST_TIMEOUT,
}
if timeout is not None:
request_kwargs["timeout"] = timeout
response = client.request(method, url, **request_kwargs)
if not response.is_success:
cls._handle_error_response(response)
+3
View File
@@ -330,6 +330,7 @@ _LEGACY_WORKSPACE_OWNER_KEYS: list[str] = [
"snippets.management",
"tool.manage",
"mcp.manage",
"agent.manage",
]
_LEGACY_WORKSPACE_ADMIN_KEYS: list[str] = [
@@ -357,6 +358,7 @@ _LEGACY_WORKSPACE_ADMIN_KEYS: list[str] = [
"snippets.management",
"tool.manage",
"mcp.manage",
"agent.manage",
]
_LEGACY_WORKSPACE_EDITOR_KEYS: list[str] = [
@@ -371,6 +373,7 @@ _LEGACY_WORKSPACE_EDITOR_KEYS: list[str] = [
"dataset.external.connect",
"snippets.create_and_modify",
"tool.manage",
"agent.manage",
]
_LEGACY_WORKSPACE_NORMAL_KEYS: list[str] = [
+13 -1
View File
@@ -93,8 +93,20 @@ class ExternalDatasetService:
raise ValueError(f"invalid endpoint: {endpoint} must start with http:// or https://")
else:
raise ValueError(f"invalid endpoint: {endpoint}")
# Send a minimal body shaped like the External Knowledge API retrieval contract so providers
# that require a JSON payload (e.g. RAGFlow) accept the validation probe instead of rejecting
# a body-less POST. Mirrors the request built in fetch_external_knowledge_retrieval.
validation_payload = {
"knowledge_id": "",
"query": "",
"retrieval_setting": {"top_k": 1, "score_threshold": 0.0},
}
try:
response = ssrf_proxy.post(endpoint, headers={"Authorization": f"Bearer {api_key}"})
response = ssrf_proxy.post(
endpoint,
headers={"Authorization": f"Bearer {api_key}", "Content-Type": "application/json"},
data=json.dumps(validation_payload),
)
except Exception as e:
raise ValueError(f"failed to connect to the endpoint: {endpoint}") from e
if response.status_code == 502:
+2
View File
@@ -186,6 +186,7 @@ class SystemFeatureModel(FeatureResponseModel):
enable_learn_app: bool = True
enable_step_by_step_tour: bool = False
rbac_enabled: bool = False
knowledge_fs_enabled: bool = False
class FeatureService:
@@ -289,6 +290,7 @@ class FeatureService:
system_features.enable_learn_app = dify_config.ENABLE_LEARN_APP
system_features.webapp_auth.allow_public_access = dify_config.WEBAPP_PUBLIC_ACCESS_ENABLED
system_features.enable_step_by_step_tour = dify_config.ENABLE_STEP_BY_STEP_TOUR
system_features.knowledge_fs_enabled = dify_config.KNOWLEDGE_FS_ENABLED
@classmethod
def _fulfill_trial_models_from_env(cls) -> list[str]:
+23
View File
@@ -141,6 +141,29 @@ class FileService:
blob = storage.load_once(upload_file_key)
return base64.b64encode(blob).decode()
def get_file_presigned_url(self, *, file_id: str, tenant_id: str) -> str:
"""Generate a direct storage URL for a tenant-owned upload file."""
with self._session_maker(expire_on_commit=False) as session:
upload_file = session.scalar(
select(UploadFile)
.where(
UploadFile.id == file_id,
UploadFile.tenant_id == tenant_id,
)
.limit(1)
)
if upload_file is None:
raise NotFound("File not found")
file_key = upload_file.key
content_type = upload_file.mime_type
return storage.generate_presigned_url(
file_key,
expires_in=dify_config.FILES_ACCESS_TIMEOUT,
content_type=content_type,
)
def upload_text(self, text: str, text_name: str, user_id: str, tenant_id: str) -> UploadFile:
if len(text_name) > 200:
text_name = text_name[:200]
+502
View File
@@ -0,0 +1,502 @@
"""Product-facing KnowledgeFS operation and authorization declarations.
This registry is Dify's explicit Console surface. Transport concerns live in
knowledge_fs_proxy so contract review does not require reading proxy mechanics.
"""
from __future__ import annotations
from typing import Final, Literal, NamedTuple
from core.rbac import RBACPermission
type KnowledgeFSMethod = Literal["DELETE", "GET", "PATCH", "POST", "PUT"]
type KnowledgeFSResponseKind = Literal["binary", "buffered", "stream"]
type KnowledgeFSRequiredScope = Literal["knowledge-spaces:read", "knowledge-spaces:write"]
type KnowledgeFSLegacyRole = Literal["reader", "dataset_editor", "admin"]
type KnowledgeFSErrorStatusMap = tuple[tuple[int, int], ...]
class KnowledgeFSOperation(NamedTuple):
operation_id: str
method: KnowledgeFSMethod
path: str
response_kind: KnowledgeFSResponseKind
required_scope: KnowledgeFSRequiredScope
rbac_permission: RBACPermission
legacy_role: KnowledgeFSLegacyRole
max_response_bytes: int
request_headers: tuple[str, ...]
response_headers: tuple[str, ...]
response_media_types: tuple[str, ...]
error_status_map: KnowledgeFSErrorStatusMap
def _console_operation(
operation_id: str,
method: KnowledgeFSMethod,
path: str,
*,
rbac_permission: RBACPermission,
legacy_role: KnowledgeFSLegacyRole,
max_response_bytes: int = 1_048_576,
request_headers: tuple[str, ...] = ("x-trace-id",),
response_kind: KnowledgeFSResponseKind = "buffered",
response_media_types: tuple[str, ...] = ("application/json",),
error_status_map: KnowledgeFSErrorStatusMap = ((401, 502), (403, 403)),
) -> KnowledgeFSOperation:
"""Declare one contract-pinned operation with an explicit Dify authorization policy."""
is_read = method == "GET"
return KnowledgeFSOperation(
operation_id=operation_id,
method=method,
path=path,
response_kind=response_kind,
required_scope="knowledge-spaces:read" if is_read else "knowledge-spaces:write",
rbac_permission=rbac_permission,
legacy_role=legacy_role,
max_response_bytes=max_response_bytes,
request_headers=request_headers,
response_headers=("x-trace-id",),
response_media_types=response_media_types,
error_status_map=error_status_map,
)
def _dataset_read_operation(operation_id: str, path: str) -> KnowledgeFSOperation:
"""Declare a dataset-readable buffered JSON operation."""
return _console_operation(
operation_id,
"GET",
path,
rbac_permission=RBACPermission.DATASET_READONLY,
legacy_role="reader",
)
def _dataset_edit_operation(
operation_id: str,
method: KnowledgeFSMethod,
path: str,
*,
request_headers: tuple[str, ...] = ("x-trace-id",),
) -> KnowledgeFSOperation:
"""Declare a dataset-editable buffered JSON operation."""
return _console_operation(
operation_id,
method,
path,
rbac_permission=RBACPermission.DATASET_EDIT,
legacy_role="dataset_editor",
request_headers=request_headers,
)
def _external_source_operation(
operation_id: str,
method: KnowledgeFSMethod,
path: str,
*,
request_headers: tuple[str, ...] = ("x-trace-id",),
) -> KnowledgeFSOperation:
"""Declare a source-connection operation restricted to dataset editors."""
return _console_operation(
operation_id,
method,
path,
rbac_permission=RBACPermission.DATASET_EXTERNAL_CONNECT,
legacy_role="dataset_editor",
request_headers=request_headers,
)
KNOWLEDGE_FS_CONSOLE_OPERATIONS: Final[tuple[KnowledgeFSOperation, ...]] = (
_console_operation(
operation_id="listKnowledgeSpaces",
method="GET",
path="knowledge-spaces",
rbac_permission=RBACPermission.DATASET_READONLY,
legacy_role="reader",
),
_console_operation(
operation_id="createKnowledgeSpace",
method="POST",
path="knowledge-spaces",
rbac_permission=RBACPermission.DATASET_CREATE_AND_MANAGEMENT,
legacy_role="dataset_editor",
),
_console_operation(
operation_id="getKnowledgeSpacesById",
method="GET",
path="knowledge-spaces/{id}",
rbac_permission=RBACPermission.DATASET_READONLY,
legacy_role="reader",
),
_dataset_edit_operation("patchKnowledgeSpacesById", "PATCH", "knowledge-spaces/{id}"),
_dataset_edit_operation(
"deleteKnowledgeSpacesById",
"DELETE",
"knowledge-spaces/{id}",
request_headers=("idempotency-key", "x-trace-id"),
),
_dataset_read_operation("getKnowledgeSpacesByIdStats", "knowledge-spaces/{id}/stats"),
_console_operation(
operation_id="getKnowledgeSpacesByIdAccessPolicy",
method="GET",
path="knowledge-spaces/{id}/access-policy",
rbac_permission=RBACPermission.DATASET_READONLY,
legacy_role="reader",
),
_console_operation(
operation_id="patchKnowledgeSpacesByIdAccessPolicy",
method="PATCH",
path="knowledge-spaces/{id}/access-policy",
rbac_permission=RBACPermission.DATASET_ACCESS_CONFIG,
legacy_role="admin",
),
_console_operation(
operation_id="getSourceProviders",
method="GET",
path="source-providers",
rbac_permission=RBACPermission.DATASET_EXTERNAL_CONNECT,
legacy_role="dataset_editor",
),
_console_operation(
operation_id="getKnowledgeSpacesByIdSourceConnections",
method="GET",
path="knowledge-spaces/{id}/source-connections",
rbac_permission=RBACPermission.DATASET_EXTERNAL_CONNECT,
legacy_role="dataset_editor",
),
_console_operation(
operation_id="postKnowledgeSpacesByIdSourceConnections",
method="POST",
path="knowledge-spaces/{id}/source-connections",
rbac_permission=RBACPermission.DATASET_EXTERNAL_CONNECT,
legacy_role="dataset_editor",
),
_external_source_operation(
"postKnowledgeSpacesByIdSourceConnectionsOauth",
"POST",
"knowledge-spaces/{id}/source-connections/oauth",
),
_external_source_operation("postSourceOauthCallback", "POST", "source-oauth/callback"),
_external_source_operation(
"getKnowledgeSpacesByIdSourceConnectionsByConnectionId",
"GET",
"knowledge-spaces/{id}/source-connections/{connectionId}",
),
_external_source_operation(
"deleteKnowledgeSpacesByIdSourceConnectionsByConnectionId",
"DELETE",
"knowledge-spaces/{id}/source-connections/{connectionId}",
),
_console_operation(
operation_id="postKnowledgeSpacesByIdSourceConnectionsByConnectionIdRefresh",
method="POST",
path="knowledge-spaces/{id}/source-connections/{connectionId}/refresh",
rbac_permission=RBACPermission.DATASET_EXTERNAL_CONNECT,
legacy_role="dataset_editor",
),
_console_operation(
operation_id="getKnowledgeSpacesByIdSources",
method="GET",
path="knowledge-spaces/{id}/sources",
rbac_permission=RBACPermission.DATASET_READONLY,
legacy_role="reader",
),
_console_operation(
operation_id="postKnowledgeSpacesByIdSources",
method="POST",
path="knowledge-spaces/{id}/sources",
rbac_permission=RBACPermission.DATASET_EXTERNAL_CONNECT,
legacy_role="dataset_editor",
),
_external_source_operation(
"getKnowledgeSpacesByIdSourcesBySourceId",
"GET",
"knowledge-spaces/{id}/sources/{sourceId}",
),
_external_source_operation(
"patchKnowledgeSpacesByIdSourcesBySourceId",
"PATCH",
"knowledge-spaces/{id}/sources/{sourceId}",
),
_external_source_operation(
"deleteKnowledgeSpacesByIdSourcesBySourceId",
"DELETE",
"knowledge-spaces/{id}/sources/{sourceId}",
request_headers=("idempotency-key", "x-trace-id"),
),
_external_source_operation(
"putKnowledgeSpacesByIdSourcesBySourceIdCredentials",
"PUT",
"knowledge-spaces/{id}/sources/{sourceId}/credentials",
),
_external_source_operation(
"deleteKnowledgeSpacesByIdSourcesBySourceIdCredentials",
"DELETE",
"knowledge-spaces/{id}/sources/{sourceId}/credentials",
),
_external_source_operation(
"postKnowledgeSpacesByIdSourcesBySourceIdSync",
"POST",
"knowledge-spaces/{id}/sources/{sourceId}/sync",
request_headers=("idempotency-key", "x-trace-id"),
),
_console_operation(
operation_id="postKnowledgeSpacesByIdSourcesBySourceIdCrawlPreview",
method="POST",
path="knowledge-spaces/{id}/sources/{sourceId}/crawl-preview",
rbac_permission=RBACPermission.DATASET_EXTERNAL_CONNECT,
legacy_role="dataset_editor",
request_headers=("idempotency-key", "x-trace-id"),
),
_external_source_operation(
"postKnowledgeSpacesByIdSourcesBySourceIdWorkflowImports",
"POST",
"knowledge-spaces/{id}/sources/{sourceId}/workflow-imports",
request_headers=("idempotency-key", "x-trace-id"),
),
_external_source_operation(
"getKnowledgeSpacesByIdSourcesBySourceIdPages",
"GET",
"knowledge-spaces/{id}/sources/{sourceId}/pages",
),
_external_source_operation(
"getKnowledgeSpacesByIdSourcesBySourceIdFiles",
"GET",
"knowledge-spaces/{id}/sources/{sourceId}/files",
),
_external_source_operation(
"postKnowledgeSpacesByIdSourcesBySourceIdCrawl",
"POST",
"knowledge-spaces/{id}/sources/{sourceId}/crawl",
),
_external_source_operation(
"postKnowledgeSpacesByIdSourcesBySourceIdImport",
"POST",
"knowledge-spaces/{id}/sources/{sourceId}/import",
),
_external_source_operation(
"postKnowledgeSpacesByIdSourcesBySourceIdTest",
"POST",
"knowledge-spaces/{id}/sources/{sourceId}/test",
),
_external_source_operation(
"postKnowledgeSpacesByIdSourcesBySourceIdImportFiles",
"POST",
"knowledge-spaces/{id}/sources/{sourceId}/import-files",
),
_external_source_operation(
"postKnowledgeSpacesByIdSourcesBulk",
"POST",
"knowledge-spaces/{id}/sources/bulk",
request_headers=("idempotency-key", "x-trace-id"),
),
_external_source_operation(
"getKnowledgeSpacesByIdSourceWorkflows",
"GET",
"knowledge-spaces/{id}/source-workflows",
),
_console_operation(
operation_id="getKnowledgeSpacesByIdSourceWorkflowsByRunId",
method="GET",
path="knowledge-spaces/{id}/source-workflows/{runId}",
rbac_permission=RBACPermission.DATASET_EXTERNAL_CONNECT,
legacy_role="dataset_editor",
),
_external_source_operation(
"getKnowledgeSpacesByIdSourceWorkflowsByRunIdBulkItems",
"GET",
"knowledge-spaces/{id}/source-workflows/{runId}/bulk-items",
),
_console_operation(
operation_id="getKnowledgeSpacesByIdSourceWorkflowsByRunIdPages",
method="GET",
path="knowledge-spaces/{id}/source-workflows/{runId}/pages",
rbac_permission=RBACPermission.DATASET_EXTERNAL_CONNECT,
legacy_role="dataset_editor",
),
_console_operation(
operation_id="postKnowledgeSpacesByIdSourceWorkflowsByRunIdCancel",
method="POST",
path="knowledge-spaces/{id}/source-workflows/{runId}/cancel",
rbac_permission=RBACPermission.DATASET_EXTERNAL_CONNECT,
legacy_role="dataset_editor",
),
_console_operation(
operation_id="postKnowledgeSpacesByIdSourceWorkflowsByRunIdRetry",
method="POST",
path="knowledge-spaces/{id}/source-workflows/{runId}/retry",
rbac_permission=RBACPermission.DATASET_EXTERNAL_CONNECT,
legacy_role="dataset_editor",
),
_console_operation(
operation_id="postKnowledgeSpacesByIdSourceWorkflowsByRunIdSelection",
method="POST",
path="knowledge-spaces/{id}/source-workflows/{runId}/selection",
rbac_permission=RBACPermission.DATASET_EXTERNAL_CONNECT,
legacy_role="dataset_editor",
request_headers=("idempotency-key", "x-trace-id"),
),
_console_operation(
operation_id="getKnowledgeSpacesByIdSourcesBySourceIdSyncPolicy",
method="GET",
path="knowledge-spaces/{id}/sources/{sourceId}/sync-policy",
rbac_permission=RBACPermission.DATASET_READONLY,
legacy_role="reader",
),
_console_operation(
operation_id="putKnowledgeSpacesByIdSourcesBySourceIdSyncPolicy",
method="PUT",
path="knowledge-spaces/{id}/sources/{sourceId}/sync-policy",
rbac_permission=RBACPermission.DATASET_EDIT,
legacy_role="dataset_editor",
),
_dataset_read_operation("getKnowledgeSpacesByIdDocuments", "knowledge-spaces/{id}/documents"),
_dataset_edit_operation("postKnowledgeSpacesByIdDocuments", "POST", "knowledge-spaces/{id}/documents"),
_dataset_edit_operation(
"deleteKnowledgeSpacesByIdDocumentsBulk",
"DELETE",
"knowledge-spaces/{id}/documents/bulk",
request_headers=("idempotency-key", "x-trace-id"),
),
_dataset_edit_operation(
"postKnowledgeSpacesByIdDocumentsBulk",
"POST",
"knowledge-spaces/{id}/documents/bulk",
),
_dataset_edit_operation(
"postKnowledgeSpacesByIdDocumentsBulkReindex",
"POST",
"knowledge-spaces/{id}/documents/bulk/reindex",
),
_dataset_read_operation(
"getKnowledgeSpacesByIdDocumentsByDocumentId",
"knowledge-spaces/{id}/documents/{documentId}",
),
_dataset_edit_operation(
"deleteKnowledgeSpacesByIdDocumentsByDocumentId",
"DELETE",
"knowledge-spaces/{id}/documents/{documentId}",
request_headers=("idempotency-key", "x-trace-id"),
),
_console_operation(
operation_id="getKnowledgeSpacesByIdLogicalDocuments",
method="GET",
path="knowledge-spaces/{id}/logical-documents",
rbac_permission=RBACPermission.DATASET_READONLY,
legacy_role="reader",
),
_dataset_edit_operation(
"deleteKnowledgeSpacesByIdLogicalDocumentsByDocumentId",
"DELETE",
"knowledge-spaces/{id}/logical-documents/{documentId}",
request_headers=("idempotency-key", "x-trace-id"),
),
_dataset_read_operation(
"getKnowledgeSpacesByIdDocumentsByDocumentIdOutline",
"knowledge-spaces/{id}/documents/{documentId}/outline",
),
_console_operation(
operation_id="getKnowledgeSpacesByIdLogicalDocumentsByDocumentId",
method="GET",
path="knowledge-spaces/{id}/logical-documents/{documentId}",
rbac_permission=RBACPermission.DATASET_READONLY,
legacy_role="reader",
),
_console_operation(
operation_id="getKnowledgeSpacesByIdDocumentsByDocumentIdRevisions",
method="GET",
path="knowledge-spaces/{id}/documents/{documentId}/revisions",
rbac_permission=RBACPermission.DATASET_READONLY,
legacy_role="reader",
),
_dataset_edit_operation(
"postKnowledgeSpacesByIdDocumentsByDocumentIdRevisionsByRevisionRollback",
"POST",
"knowledge-spaces/{id}/documents/{documentId}/revisions/{revision}/rollback",
),
_dataset_edit_operation(
"patchKnowledgeSpacesByIdDocumentsByDocumentIdMetadata",
"PATCH",
"knowledge-spaces/{id}/documents/{documentId}/metadata",
),
_console_operation(
operation_id="getKnowledgeSpacesByIdDocumentsByDocumentIdRevisionsByRevisionChunks",
method="GET",
path="knowledge-spaces/{id}/documents/{documentId}/revisions/{revision}/chunks",
rbac_permission=RBACPermission.DATASET_READONLY,
legacy_role="reader",
),
_dataset_read_operation(
"getKnowledgeSpacesByIdDocumentsByDocumentIdRevisionsByRevisionChunksByChunkId",
"knowledge-spaces/{id}/documents/{documentId}/revisions/{revision}/chunks/{chunkId}",
),
_dataset_edit_operation(
"postKnowledgeSpacesByIdDocumentsByDocumentIdRevisionsByRevisionChunksByChunkIdState",
"POST",
"knowledge-spaces/{id}/documents/{documentId}/revisions/{revision}/chunks/{chunkId}/state",
),
_console_operation(
operation_id="getKnowledgeSpacesByIdProcessingTasks",
method="GET",
path="knowledge-spaces/{id}/processing-tasks",
rbac_permission=RBACPermission.DATASET_READONLY,
legacy_role="reader",
),
_dataset_read_operation(
"getKnowledgeSpacesByIdDocumentsByDocumentIdProcessingTasks",
"knowledge-spaces/{id}/documents/{documentId}/processing-tasks",
),
_dataset_read_operation(
"getKnowledgeSpacesByIdDocumentsByDocumentIdProcessingTasksByTaskId",
"knowledge-spaces/{id}/documents/{documentId}/processing-tasks/{taskId}",
),
_console_operation(
operation_id="getKnowledgeSpacesByIdDocumentsByDocumentIdProcessingTasksByTaskIdEvents",
method="GET",
path="knowledge-spaces/{id}/documents/{documentId}/processing-tasks/{taskId}/events",
rbac_permission=RBACPermission.DATASET_READONLY,
legacy_role="reader",
max_response_bytes=67_108_864,
request_headers=("last-event-id", "x-trace-id"),
response_kind="stream",
response_media_types=("text/event-stream",),
),
_console_operation(
operation_id="deleteKnowledgeSpacesByIdDocumentsByDocumentIdProcessingTasksByTaskId",
method="DELETE",
path="knowledge-spaces/{id}/documents/{documentId}/processing-tasks/{taskId}",
rbac_permission=RBACPermission.DATASET_EDIT,
legacy_role="dataset_editor",
),
_console_operation(
operation_id="postKnowledgeSpacesByIdDocumentsByDocumentIdProcessingTasksByTaskIdRetry",
method="POST",
path="knowledge-spaces/{id}/documents/{documentId}/processing-tasks/{taskId}/retry",
rbac_permission=RBACPermission.DATASET_EDIT,
legacy_role="dataset_editor",
),
_dataset_read_operation(
"getKnowledgeSpacesByIdDocumentsByDocumentIdSettings",
"knowledge-spaces/{id}/documents/{documentId}/settings",
),
_dataset_edit_operation(
"putKnowledgeSpacesByIdDocumentsByDocumentIdSettings",
"PUT",
"knowledge-spaces/{id}/documents/{documentId}/settings",
),
_dataset_read_operation("getJobsById", "jobs/{id}"),
_dataset_edit_operation("deleteJobsById", "DELETE", "jobs/{id}"),
_dataset_edit_operation("postJobsByIdRetry", "POST", "jobs/{id}/retry"),
_dataset_read_operation("getDeletionJobsByJobId", "deletion-jobs/{jobId}"),
_dataset_edit_operation(
"postDeletionJobsByJobIdRetry",
"POST",
"deletion-jobs/{jobId}/retry",
request_headers=("idempotency-key", "x-trace-id"),
),
_dataset_read_operation("getBulkJobsById", "bulk-jobs/{id}"),
)
+118 -68
View File
@@ -1,33 +1,32 @@
"""Transport-only forwarding for the explicitly enabled KnowledgeFS Console operations.
"""Authorize and forward the explicitly enabled KnowledgeFS Console operations.
KnowledgeFS owns the request and response contract. This module binds short-lived
account and workspace identities, enforces Dify's coarse workspace policy, and
normalizes transport failures. Dify deliberately maintains a small product-facing
operation registry instead of exposing the full upstream OpenAPI surface. The
dedicated request path uses Dify's shared SSRF policy, never follows redirects,
bounds buffered responses, and rejects compressed responses.
The dedicated request path uses Dify's shared SSRF policy, never follows redirects,
bounds buffered responses, and rejects compressed streaming responses.
"""
from __future__ import annotations
from collections.abc import Iterable, Mapping
from dataclasses import dataclass
from datetime import UTC, datetime, timedelta
from http import HTTPStatus
from typing import Final, Literal, NamedTuple, Protocol
from typing import NamedTuple, Protocol
import httpx
import jwt
from configs import dify_config
from core.helper import ssrf_proxy
from core.rbac import RBACPermission, RBACResourceScope
from core.rbac import RBACResourceScope
from core.tools.errors import ToolSSRFError
from models import Account
from services.enterprise.rbac_service import RBACService
type KnowledgeFSMethod = Literal["DELETE", "GET", "PATCH", "POST", "PUT"]
type KnowledgeFSResponseKind = Literal["binary", "buffered", "stream"]
type KnowledgeFSRequiredScope = Literal["knowledge-spaces:read", "knowledge-spaces:write"]
from services.knowledge_fs_operations import (
KNOWLEDGE_FS_CONSOLE_OPERATIONS,
KnowledgeFSMethod,
KnowledgeFSOperation,
KnowledgeFSResponseKind,
)
_JWT_AUDIENCE = "knowledge-fs"
_JWT_ISSUER = "dify"
@@ -35,56 +34,55 @@ _JWT_TTL_SECONDS = 60
_MAX_BUFFERED_RESPONSE_BYTES = 1024 * 1024
class KnowledgeFSOperation(NamedTuple):
operation_id: str
method: KnowledgeFSMethod
path: str
response_kind: KnowledgeFSResponseKind
required_scope: KnowledgeFSRequiredScope
rbac_permission: RBACPermission
requires_dataset_editor: bool
max_response_bytes: int
request_headers: tuple[str, ...]
response_headers: tuple[str, ...]
response_media_types: tuple[str, ...]
KNOWLEDGE_FS_CONSOLE_OPERATIONS: Final[tuple[KnowledgeFSOperation, ...]] = (
KnowledgeFSOperation(
operation_id="listKnowledgeSpaces",
method="GET",
path="knowledge-spaces",
response_kind="buffered",
required_scope="knowledge-spaces:read",
rbac_permission=RBACPermission.DATASET_READONLY,
requires_dataset_editor=False,
max_response_bytes=1_048_576,
request_headers=("x-trace-id",),
response_headers=("x-trace-id",),
response_media_types=("application/json",),
),
KnowledgeFSOperation(
operation_id="createKnowledgeSpace",
method="POST",
path="knowledge-spaces",
response_kind="buffered",
required_scope="knowledge-spaces:write",
rbac_permission=RBACPermission.DATASET_CREATE_AND_MANAGEMENT,
requires_dataset_editor=True,
max_response_bytes=1_048_576,
request_headers=("x-trace-id",),
response_headers=("x-trace-id",),
response_media_types=("application/json",),
),
)
class KnowledgeFSUpstreamResponse(NamedTuple):
response: httpx.Response
response_kind: KnowledgeFSResponseKind
operation: KnowledgeFSOperation
_AUTHORIZATION_MARKER = object()
@dataclass(eq=False, frozen=True, init=False, slots=True)
class KnowledgeFSAuthorization:
"""Single-use forwarding capability created after Dify workspace policy checks.
Callers obtain this value from :func:`authorize_knowledge_fs_request`. Direct
construction and repeated forwarding are rejected before outbound I/O.
"""
account_id: str
tenant_id: str
operation: KnowledgeFSOperation
_used: bool
def __init__(
self,
account_id: str,
tenant_id: str,
operation: KnowledgeFSOperation,
*,
_marker: object | None = None,
) -> None:
if _marker is not _AUTHORIZATION_MARKER:
raise KnowledgeFSAccessDeniedError("KnowledgeFS authorization must be created by workspace authorization")
object.__setattr__(self, "account_id", account_id)
object.__setattr__(self, "tenant_id", tenant_id)
object.__setattr__(self, "operation", operation)
object.__setattr__(self, "_used", False)
def consume(self) -> tuple[str, str, KnowledgeFSOperation]:
"""Return the authorized principals and canonical operation exactly once.
Raises:
KnowledgeFSAccessDeniedError: The capability was already consumed.
"""
if self._used:
raise KnowledgeFSAccessDeniedError("KnowledgeFS authorization has already been used")
object.__setattr__(self, "_used", True)
return self.account_id, self.tenant_id, self.operation
class _RequestHeaders(Protocol):
def items(self) -> Iterable[tuple[str, str]]: ...
@@ -113,20 +111,29 @@ def authorize_knowledge_fs_request(
*,
account: Account,
tenant_id: str,
operation: KnowledgeFSOperation,
) -> None:
method: KnowledgeFSMethod,
path: str,
) -> KnowledgeFSAuthorization:
"""Enforce Dify's workspace policy before KFS performs resource authorization.
Args:
account: Authenticated Dify account with its current workspace role.
tenant_id: Current Dify workspace identifier.
operation: Dify-maintained KnowledgeFS operation and policy metadata.
method: Requested upstream HTTP method.
path: Requested relative KnowledgeFS path.
Raises:
KnowledgeFSRouteNotAllowedError: The method and path do not resolve to a declared operation.
KnowledgeFSAccessDeniedError: The account lacks a required legacy or enterprise permission.
Returns:
A request-scoped capability binding the authorized account, workspace, and operation.
"""
if operation.requires_dataset_editor and not account.is_dataset_editor:
raise KnowledgeFSAccessDeniedError("KnowledgeFS mutations require dataset edit access")
operation = get_knowledge_fs_operation(method, path)
if operation.legacy_role == "dataset_editor" and not account.is_dataset_editor:
raise KnowledgeFSAccessDeniedError("KnowledgeFS operation requires dataset edit access")
if operation.legacy_role == "admin" and not account.is_admin_or_owner:
raise KnowledgeFSAccessDeniedError("KnowledgeFS operation requires workspace administration access")
if not RBACService.CheckAccess.check(
tenant_id,
account.id,
@@ -134,6 +141,7 @@ def authorize_knowledge_fs_request(
resource_type=RBACResourceScope.DATASET.value,
):
raise KnowledgeFSAccessDeniedError("KnowledgeFS operation is denied by workspace RBAC")
return KnowledgeFSAuthorization(account.id, tenant_id, operation, _marker=_AUTHORIZATION_MARKER)
def proxy_knowledge_fs_request(
@@ -149,20 +157,62 @@ def proxy_knowledge_fs_request(
request_headers: _RequestHeaders | None = None,
) -> KnowledgeFSUpstreamResponse:
"""Authorize and forward one allowlisted KnowledgeFS request as a single use case."""
operation = get_knowledge_fs_operation(method, path)
authorize_knowledge_fs_request(
authorization = authorize_knowledge_fs_request(
account=account,
tenant_id=tenant_id,
operation=operation,
method=method,
path=path,
)
return proxy_authorized_knowledge_fs_request(
authorization=authorization,
accept=accept,
content_type=content_type,
query=query,
body=body,
request_headers=request_headers,
)
def proxy_authorized_knowledge_fs_request(
*,
authorization: KnowledgeFSAuthorization,
accept: str | None = None,
content_type: str | None = None,
query: bytes | None = None,
body: bytes | None = None,
request_headers: _RequestHeaders | None = None,
) -> KnowledgeFSUpstreamResponse:
"""Forward one request whose operation and workspace policy were already authorized.
This performs one outbound KnowledgeFS request and does not repeat Dify RBAC checks.
Args:
authorization: Request-scoped capability returned by :func:`authorize_knowledge_fs_request`.
accept: Original Accept header, when present.
content_type: Original request Content-Type header, when present.
query: Original encoded query string from the Console request.
body: Original request body, when present.
request_headers: Incoming headers; only names declared by the operation are forwarded.
Returns:
The bounded KnowledgeFS response together with its transport metadata.
Raises:
KnowledgeFSConfigurationError: The connection is incomplete or blocked by outbound policy.
KnowledgeFSRouteNotAllowedError: A forwarded request header is outside the operation contract.
KnowledgeFSTimeoutError: KnowledgeFS exceeds the configured timeout.
KnowledgeFSTransportError: The request fails or its response violates transport bounds.
"""
account_id, tenant_id, operation = authorization.consume()
incoming_request_headers = {name.lower(): value for name, value in (request_headers or {}).items()}
contract_request_headers = {
name: incoming_request_headers[name] for name in operation.request_headers if name in incoming_request_headers
}
return _forward_knowledge_fs_request(
account_id=account.id,
method=method,
path=path,
account_id=account_id,
method=operation.method,
path=operation.path,
tenant_id=tenant_id,
accept=accept,
content_type=content_type,
+12 -1
View File
@@ -13,6 +13,7 @@ from core.tools.utils.web_reader_tool import get_image_upload_file_ids
from extensions.ext_storage import storage
from models.dataset import Dataset, DatasetMetadataBinding, DocumentSegment
from models.model import UploadFile
from tasks.refresh_billing_vector_space_task import schedule_billing_vector_space_refresh
logger = logging.getLogger(__name__)
@@ -21,7 +22,12 @@ BATCH_SIZE = 1000
@shared_task(queue="dataset")
def batch_clean_document_task(document_ids: list[str], dataset_id: str, doc_form: str | None, file_ids: list[str]):
def batch_clean_document_task(
document_ids: list[str],
dataset_id: str,
doc_form: str | None,
file_ids: list[str],
) -> None:
"""
Clean document when document deleted.
:param document_ids: document ids
@@ -40,6 +46,7 @@ def batch_clean_document_task(document_ids: list[str], dataset_id: str, doc_form
index_node_ids: list[str] = []
segment_ids: list[str] = []
total_image_upload_file_ids: list[str] = []
dataset_tenant_id: str | None = None
try:
# ============ Step 1: Query segment and file data (short read-only transaction) ============
@@ -88,6 +95,7 @@ def batch_clean_document_task(document_ids: list[str], dataset_id: str, doc_form
delete_summaries=True,
session=session,
)
dataset_tenant_id = dataset.tenant_id
except Exception:
logger.exception(
"Failed to clean vector index for dataset_id: %s, document_ids: %s, index_node_ids count: %d",
@@ -203,6 +211,9 @@ def batch_clean_document_task(document_ids: list[str], dataset_id: str, doc_form
dataset_id,
)
if dataset_tenant_id is not None:
schedule_billing_vector_space_refresh(dataset_tenant_id)
end_at = time.perf_counter()
logger.info(
click.style(
+5
View File
@@ -24,6 +24,7 @@ from models.dataset import (
)
from models.model import UploadFile
from models.workflow import Workflow
from tasks.refresh_billing_vector_space_task import schedule_billing_vector_space_refresh
logger = logging.getLogger(__name__)
@@ -52,6 +53,7 @@ def clean_dataset_task(
"""
logger.info(click.style(f"Start clean dataset when dataset deleted: {dataset_id}", fg="green"))
start_at = time.perf_counter()
vector_cleanup_succeeded = False
with session_factory.create_session() as session:
try:
@@ -93,6 +95,7 @@ def clean_dataset_task(
try:
index_processor = IndexProcessorFactory(doc_form).init_index_processor()
index_processor.clean(dataset, None, with_keywords=True, delete_child_chunks=True, session=session)
vector_cleanup_succeeded = True
logger.info(click.style(f"Successfully cleaned vector database for dataset: {dataset_id}", fg="green"))
except Exception:
logger.exception(click.style(f"Failed to clean vector database for dataset {dataset_id}", fg="red"))
@@ -186,6 +189,8 @@ def clean_dataset_task(
session.execute(file_delete_stmt)
session.commit()
if vector_cleanup_succeeded:
schedule_billing_vector_space_refresh(dataset.tenant_id)
end_at = time.perf_counter()
logger.info(
click.style(
+13 -1
View File
@@ -11,12 +11,18 @@ from core.tools.utils.web_reader_tool import get_image_upload_file_ids
from extensions.ext_storage import storage
from models.dataset import Dataset, DatasetMetadataBinding, DocumentSegment, SegmentAttachmentBinding
from models.model import UploadFile
from tasks.refresh_billing_vector_space_task import schedule_billing_vector_space_refresh
logger = logging.getLogger(__name__)
@shared_task(queue="dataset")
def clean_document_task(document_id: str, dataset_id: str, doc_form: str, file_id: str | None):
def clean_document_task(
document_id: str,
dataset_id: str,
doc_form: str,
file_id: str | None,
) -> None:
"""
Clean document when document deleted.
:param document_id: document id
@@ -29,6 +35,7 @@ def clean_document_task(document_id: str, dataset_id: str, doc_form: str, file_i
logger.info(click.style(f"Start clean document when document deleted: {document_id}", fg="green"))
start_at = time.perf_counter()
total_attachment_files = []
vector_cleanup_succeeded = False
with session_factory.create_session() as session:
try:
@@ -37,6 +44,7 @@ def clean_document_task(document_id: str, dataset_id: str, doc_form: str, file_i
if not dataset:
raise Exception("Document has no dataset")
dataset_tenant_id = dataset.tenant_id
segments = session.scalars(select(DocumentSegment).where(DocumentSegment.document_id == document_id)).all()
# Use JOIN to fetch attachments with bindings in a single query
attachments_with_bindings = session.execute(
@@ -82,6 +90,7 @@ def clean_document_task(document_id: str, dataset_id: str, doc_form: str, file_i
delete_summaries=True,
session=session,
)
vector_cleanup_succeeded = True
except Exception:
logger.exception(
"Failed to clean vector / keyword index in clean_document_task, "
@@ -154,6 +163,9 @@ def clean_document_task(document_id: str, dataset_id: str, doc_form: str, file_i
)
)
if vector_cleanup_succeeded:
schedule_billing_vector_space_refresh(dataset_tenant_id)
end_at = time.perf_counter()
logger.info(
click.style(
@@ -0,0 +1,59 @@
import logging
from celery import shared_task
from opentelemetry import metrics
from configs import dify_config
from services.billing_service import BillingService
logger = logging.getLogger(__name__)
_MAX_RETRIES = 3
_RETRY_DELAY_SECONDS = 30
_refresh_counter = metrics.get_meter(__name__).create_counter(
"billing.vector_space_cache_refresh.count",
description="Number of billing vector-space cache refresh outcomes",
)
@shared_task(queue="dataset", bind=True, max_retries=_MAX_RETRIES, default_retry_delay=_RETRY_DELAY_SECONDS)
def refresh_billing_vector_space_task(self, tenant_id: str) -> None:
"""Refresh billing vector-space usage after vector cleanup has completed."""
if not dify_config.BILLING_ENABLED:
return
try:
BillingService.invalidate_vector_space_cache(tenant_id)
except Exception as exc:
if self.request.retries >= _MAX_RETRIES:
_refresh_counter.add(1, {"outcome": "exhausted"})
logger.exception(
"Billing vector-space cache refresh retry budget exhausted, tenant_id=%s",
tenant_id,
)
raise
_refresh_counter.add(1, {"outcome": "retry"})
logger.warning(
"Billing vector-space cache refresh failed, scheduling retry %d/%d, tenant_id=%s",
self.request.retries + 1,
_MAX_RETRIES,
tenant_id,
exc_info=True,
)
raise self.retry(exc=exc, countdown=_RETRY_DELAY_SECONDS * (2**self.request.retries))
_refresh_counter.add(1, {"outcome": "success"})
logger.info("Billing vector-space cache refreshed, tenant_id=%s", tenant_id)
def schedule_billing_vector_space_refresh(tenant_id: str) -> None:
"""Dispatch a best-effort billing refresh without changing cleanup status."""
if not dify_config.BILLING_ENABLED:
return
try:
refresh_billing_vector_space_task.delay(tenant_id)
except Exception:
_refresh_counter.add(1, {"outcome": "dispatch_failure"})
logger.exception("Failed to dispatch billing vector-space cache refresh, tenant_id=%s", tenant_id)
@@ -9,7 +9,9 @@ from flask import Flask
from sqlalchemy.orm import Session
from werkzeug.exceptions import Forbidden
from configs import dify_config
from controllers.web.site import AppSiteApi, WebAppSiteResponse, WebModelConfigResponse
from extensions.storage.storage_type import StorageType
from models import Tenant, TenantStatus
from models.account import TenantCustomConfigDict
from models.model import App, AppMode, AppModelConfig, CustomizeTokenStrategy, EndUser, Site
@@ -96,6 +98,39 @@ class TestAppSiteApi:
assert result["plan"] == "basic"
assert result["enable_site"] is True
@patch("controllers.web.site.FileService.get_file_presigned_url")
@patch("controllers.web.site.FeatureService.get_features")
def test_image_icon_uses_s3_presigned_url(
self,
mock_features: MagicMock,
mock_get_file_presigned_url: MagicMock,
app: Flask,
db_session_with_containers: Session,
) -> None:
app.config["RESTX_MASK_HEADER"] = "X-Fields"
tenant = _create_tenant(db_session_with_containers)
app_model = _create_app(db_session_with_containers, tenant.id)
site = _create_site(db_session_with_containers, app_model.id)
site.icon_type = "image"
site.icon = "11111111-1111-4111-8111-111111111111"
db_session_with_containers.commit()
end_user = _end_user(tenant.id, app_model.id)
mock_features.return_value = FeatureModel(can_replace_logo=False)
mock_get_file_presigned_url.return_value = "https://s3.example.com/icon.png?signature=test"
with (
patch.object(dify_config, "EDITION", "CLOUD"),
patch.object(dify_config, "STORAGE_TYPE", StorageType.S3),
app.test_request_context("/site"),
):
result = AppSiteApi().get(app_model, end_user)
assert result["site"]["icon_url"] == "https://s3.example.com/icon.png?signature=test"
mock_get_file_presigned_url.assert_called_once_with(
file_id="11111111-1111-4111-8111-111111111111",
tenant_id=tenant.id,
)
def test_missing_site_raises_forbidden(self, app: Flask, db_session_with_containers: Session) -> None:
app.config["RESTX_MASK_HEADER"] = "X-Fields"
tenant = _create_tenant(db_session_with_containers)
@@ -212,6 +212,17 @@ def test_generate_specs_include_console_contract_shapes_for_schema_migration(tmp
assert {"type": "null"} in app_detail_nullable_schema["anyOf"]
assert schemas["RecommendedAppInfoResponse"]["properties"]["icon_url"]["readOnly"] is True
assert schemas["InstalledAppInfoResponse"]["properties"]["icon_url"]["readOnly"] is True
assert _response_schema(paths["/apps/{app_id}"]["get"])["$ref"] == "#/components/schemas/AppDetailWithSite"
app_model_config = schemas["AppDetailWithSite"]["properties"]["model_config"]
assert {"$ref": "#/components/schemas/AppModelConfigResponse"} in app_model_config["anyOf"]
app_detail = schemas["AppDetail"]
assert "mode" in app_detail["properties"]
assert "mode_compatible_with_agent" not in app_detail["properties"]
sync_draft_workflow = schemas["SyncDraftWorkflowResponse"]
assert _response_schema(paths["/apps/{app_id}/workflows/draft"]["post"])["$ref"] == (
"#/components/schemas/SyncDraftWorkflowResponse"
)
assert sync_draft_workflow["properties"]["updated_at"]["type"] == "integer"
tool_icon_schema = schemas["ExploreAppMetaResponse"]["properties"]["tool_icons"]["additionalProperties"]
assert {"type": "string"} in tool_icon_schema["anyOf"]
assert {"additionalProperties": True, "type": "object"} in tool_icon_schema["anyOf"]
@@ -1,15 +1,24 @@
"""Unit tests for the reset-encrypt-key-pair CLI command (#35396).
"""SQLite-backed tests for the reset-encrypt-key-pair CLI command (#35396).
The command must purge every table that stores ciphertext encrypted with the
tenant's asymmetric key, otherwise stale rows cause downstream API failures
such as `/console/api/workspaces/current/tool-providers` returning 500.
Tests bind the command-owned transaction to the fixture engine and assert the
committed state rather than inspecting fabricated ``Session.execute`` calls.
"""
from unittest.mock import MagicMock, patch
from types import SimpleNamespace
import pytest
from sqlalchemy import select
from sqlalchemy.orm import Session
import commands
from commands import system as system_commands
from models.provider import Provider, ProviderModel
from core.tools.entities.tool_entities import ApiProviderSchemaType
from graphon.model_runtime.entities.model_entities import ModelType
from models import Tenant
from models.provider import Provider, ProviderModel, ProviderType
from models.tools import ApiToolProvider, BuiltinToolProvider, MCPToolProvider
@@ -21,17 +30,60 @@ def _invoke_reset() -> int:
return 0
def _delete_targets(session_mock: MagicMock) -> list:
"""Extract the model class targeted by each `delete(...)` call on the session."""
targets = []
for call in session_mock.execute.call_args_list:
stmt = call.args[0]
# `delete(Foo)` constructs a `Delete` statement whose entity is `Foo`.
try:
targets.append(stmt.table.name)
except AttributeError:
targets.append(repr(stmt))
return targets
TENANT_ID = "11111111-1111-1111-1111-111111111111"
OTHER_TENANT_ID = "11111111-1111-1111-1111-111111111112"
USER_ID = "22222222-2222-2222-2222-222222222222"
def _tenant(tenant_id: str, *, name: str = "Test tenant") -> Tenant:
tenant = Tenant(name=name, encrypt_public_key="old-key")
tenant.id = tenant_id
return tenant
def _encrypted_rows(tenant_id: str, *, suffix: str = "1") -> tuple[object, ...]:
"""Build one persisted credential-bearing row for every purge target."""
return (
Provider(tenant_id=tenant_id, provider_name=f"provider-{suffix}"),
ProviderModel(
tenant_id=tenant_id,
provider_name=f"provider-{suffix}",
model_name=f"model-{suffix}",
model_type=ModelType.LLM,
),
BuiltinToolProvider(
name=f"builtin-credential-{suffix}",
tenant_id=tenant_id,
user_id=USER_ID,
provider=f"builtin-{suffix}",
encrypted_credentials="ciphertext",
),
ApiToolProvider(
name=f"api-{suffix}",
icon="icon",
schema="{}",
schema_type_str=ApiProviderSchemaType.OPENAPI,
user_id=USER_ID,
tenant_id=tenant_id,
description="description",
tools_str="[]",
credentials_str="{}",
),
MCPToolProvider(
name=f"mcp-{suffix}",
server_identifier=f"server-{suffix}",
server_url="ciphertext",
server_url_hash=f"hash-{suffix}",
icon=None,
tenant_id=tenant_id,
user_id=USER_ID,
encrypted_credentials="ciphertext",
),
)
def _bind_command_to_sqlite(monkeypatch: pytest.MonkeyPatch, session: Session) -> None:
monkeypatch.setattr(system_commands, "db", SimpleNamespace(engine=session.get_bind()))
def test_reset_aborts_when_not_self_hosted(monkeypatch, capsys):
@@ -44,65 +96,73 @@ def test_reset_aborts_when_not_self_hosted(monkeypatch, capsys):
assert "only for SELF_HOSTED" in captured.out
def test_reset_purges_provider_and_tool_tables_for_each_tenant(monkeypatch, capsys):
@pytest.mark.parametrize(
"sqlite_session",
[(Tenant, Provider, ProviderModel, BuiltinToolProvider, ApiToolProvider, MCPToolProvider)],
indirect=True,
)
def test_reset_purges_provider_and_tool_tables_for_each_tenant(
monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str], sqlite_session: Session
) -> None:
"""The command must purge LLM provider rows AND every tool provider table
that stores ciphertext encrypted under the tenant key (#35396)."""
monkeypatch.setattr(system_commands.dify_config, "EDITION", "SELF_HOSTED")
monkeypatch.setattr(system_commands, "generate_key_pair", lambda tenant_id: f"new-key-{tenant_id}")
_bind_command_to_sqlite(monkeypatch, sqlite_session)
fake_tenant = MagicMock(id="tenant-abc", encrypt_public_key="old-key")
session = MagicMock()
session.scalars.return_value.all.return_value = [fake_tenant]
tenant = _tenant(TENANT_ID)
other_tenant = _tenant(OTHER_TENANT_ID, name="Other tenant")
system_provider = Provider(
tenant_id=TENANT_ID,
provider_name="system-provider",
provider_type=ProviderType.SYSTEM,
)
sqlite_session.add_all((tenant, other_tenant, system_provider, *_encrypted_rows(TENANT_ID)))
sqlite_session.commit()
fake_sessionmaker = MagicMock()
fake_sessionmaker.begin.return_value.__enter__.return_value = session
fake_sessionmaker.begin.return_value.__exit__.return_value = False
with (
patch.object(system_commands, "db", MagicMock()),
patch.object(system_commands, "sessionmaker", return_value=fake_sessionmaker),
):
exit_code = _invoke_reset()
exit_code = _invoke_reset()
captured = capsys.readouterr()
assert exit_code == 0
assert "tenant-abc" in captured.out
assert TENANT_ID in captured.out
# New key pair generated and assigned.
assert fake_tenant.encrypt_public_key == "new-key-tenant-abc"
# Every encrypted-credential table should have been purged for this tenant.
table_names = _delete_targets(session)
expected = {
Provider.__tablename__,
ProviderModel.__tablename__,
BuiltinToolProvider.__tablename__,
ApiToolProvider.__tablename__,
MCPToolProvider.__tablename__,
}
assert expected.issubset(set(table_names)), f"missing purges: expected {expected}, got {table_names}"
sqlite_session.expire_all()
assert sqlite_session.get(Tenant, TENANT_ID).encrypt_public_key == f"new-key-{TENANT_ID}"
assert sqlite_session.get(Tenant, OTHER_TENANT_ID).encrypt_public_key == f"new-key-{OTHER_TENANT_ID}"
assert sqlite_session.scalars(select(Provider).where(Provider.provider_type == ProviderType.CUSTOM)).all() == []
assert sqlite_session.scalars(select(ProviderModel)).all() == []
assert sqlite_session.scalars(select(BuiltinToolProvider)).all() == []
assert sqlite_session.scalars(select(ApiToolProvider)).all() == []
assert sqlite_session.scalars(select(MCPToolProvider)).all() == []
assert (
sqlite_session.scalar(select(Provider).where(Provider.provider_type == ProviderType.SYSTEM)) is system_provider
)
def test_reset_iterates_all_tenants(monkeypatch, capsys):
@pytest.mark.parametrize(
"sqlite_session",
[(Tenant, Provider, ProviderModel, BuiltinToolProvider, ApiToolProvider, MCPToolProvider)],
indirect=True,
)
def test_reset_iterates_all_tenants(monkeypatch: pytest.MonkeyPatch, sqlite_session: Session) -> None:
"""Multi-tenant deployments must purge every tenant, not just the first."""
monkeypatch.setattr(system_commands.dify_config, "EDITION", "SELF_HOSTED")
monkeypatch.setattr(system_commands, "generate_key_pair", lambda tenant_id: f"new-key-{tenant_id}")
tenants = [MagicMock(id=f"tenant-{i}", encrypt_public_key="old") for i in range(3)]
session = MagicMock()
session.scalars.return_value.all.return_value = tenants
_bind_command_to_sqlite(monkeypatch, sqlite_session)
tenant_ids = [f"11111111-1111-1111-1111-{index:012d}" for index in range(3)]
tenants = [_tenant(tenant_id, name=f"Tenant {index}") for index, tenant_id in enumerate(tenant_ids)]
for index, tenant in enumerate(tenants):
sqlite_session.add(tenant)
sqlite_session.add_all(_encrypted_rows(tenant.id, suffix=str(index)))
sqlite_session.commit()
fake_sessionmaker = MagicMock()
fake_sessionmaker.begin.return_value.__enter__.return_value = session
fake_sessionmaker.begin.return_value.__exit__.return_value = False
assert _invoke_reset() == 0
with (
patch.object(system_commands, "db", MagicMock()),
patch.object(system_commands, "sessionmaker", return_value=fake_sessionmaker),
):
_invoke_reset()
# Five purges per tenant × 3 tenants = 15 execute calls.
assert session.execute.call_count == 15
for tenant in tenants:
assert tenant.encrypt_public_key == f"new-key-{tenant.id}"
sqlite_session.expire_all()
persisted_tenants = sqlite_session.scalars(select(Tenant).order_by(Tenant.id)).all()
assert [tenant.encrypt_public_key for tenant in persisted_tenants] == [
f"new-key-{tenant_id}" for tenant_id in tenant_ids
]
for model in (Provider, ProviderModel, BuiltinToolProvider, ApiToolProvider, MCPToolProvider):
assert sqlite_session.scalars(select(model)).all() == []
+38 -24
View File
@@ -37,6 +37,7 @@ os.environ.setdefault("STORAGE_TYPE", "opendal")
from core.db.session_factory import configure_session_factory, session_factory
from extensions import ext_redis
from models.account import Account, Tenant, TenantAccountJoin, TenantAccountRole
from models.base import TypeBase
@@ -148,32 +149,45 @@ def _configure_session_factory(_unit_test_engine):
configure_session_factory(_unit_test_engine, expire_on_commit=False)
def setup_mock_tenant_owner_execute_result(mock_db, mock_tenant, mock_owner):
"""
Helper to stub the tenant-owner execute result for service API app authentication.
def persist_service_api_tenant_owner(session: Session, tenant: Tenant, owner: Account) -> TenantAccountJoin:
"""Persist the owner identity resolved by service-API app authentication.
The validate_app_token decorator currently resolves the active tenant owner
via db.session.execute(select(Tenant, Account)...).one_or_none().
Args:
mock_db: The mocked db object
mock_tenant: Mock tenant object to return
mock_owner: Mock owner object to return from the execute result
The legacy name is retained temporarily for consumers on independent
conversion branches, but this helper no longer fabricates an execute result.
"""
membership = TenantAccountJoin(
tenant_id=tenant.id,
account_id=owner.id,
role=TenantAccountRole.OWNER,
)
owner._current_tenant = tenant
session.add_all([tenant, owner, membership])
session.commit()
return membership
def persist_service_api_dataset_owner(
session: Session,
tenant: Tenant,
tenant_account_join: TenantAccountJoin,
) -> None:
"""Persist the tenant-owner mapping resolved by dataset-token authentication."""
session.add_all([tenant, tenant_account_join])
session.commit()
def setup_mock_tenant_owner_execute_result(mock_db: MagicMock, mock_tenant: object, mock_owner: object) -> None:
"""Stub the legacy owner query; SQLite-backed tests use ``persist_service_api_tenant_owner``."""
mock_db.session.execute.return_value.one_or_none.return_value = (mock_tenant, mock_owner)
def setup_mock_dataset_owner_execute_result(mock_db, mock_tenant, mock_tenant_account_join):
"""
Helper to stub the tenant-owner execute result for dataset token authentication.
The validate_dataset_token decorator currently resolves the owner mapping via
db.session.execute(select(Tenant, TenantAccountJoin)...).one_or_none(), and
then loads the Account separately via db.session.get(...).
Args:
mock_db: The mocked db object
mock_tenant: Mock tenant object to return
mock_tenant_account_join: Mock tenant-account join object to return
"""
mock_db.session.execute.return_value.one_or_none.return_value = (mock_tenant, mock_tenant_account_join)
def setup_mock_dataset_owner_execute_result(
mock_db: MagicMock,
mock_tenant: object,
mock_tenant_account_join: object,
) -> None:
"""Stub the legacy dataset-owner query; SQLite tests use ``persist_service_api_dataset_owner``."""
mock_db.session.execute.return_value.one_or_none.return_value = (
mock_tenant,
mock_tenant_account_join,
)
@@ -53,6 +53,7 @@ from controllers.console.app.message import (
AgentMessageFeedbackApi,
AgentMessageSuggestedQuestionApi,
)
from models.agent import AgentConfigDraftType
from services.entities.agent_entities import ComposerSaveStrategy, ComposerVariant
@@ -371,6 +372,7 @@ def test_agent_app_list_and_create_use_agent_route(
"tenant_id": "tenant-1",
"agent_id": "agent-created",
"account_id": account_id,
"draft_type": AgentConfigDraftType.DEBUG_BUILD,
"commit": False,
}
@@ -544,8 +546,19 @@ def test_agent_app_copy_uses_agent_id_and_returns_agent_detail(
}
def test_agent_debug_conversation_refresh_uses_current_user(
app: Flask, monkeypatch: pytest.MonkeyPatch, account_id: str
@pytest.mark.parametrize(
("payload", "expected_draft_type"),
[
(None, AgentConfigDraftType.DEBUG_BUILD),
({"draft_type": "draft"}, AgentConfigDraftType.DRAFT),
],
)
def test_agent_debug_conversation_refresh_uses_current_user_and_draft_type(
app: Flask,
monkeypatch: pytest.MonkeyPatch,
account_id: str,
payload: dict[str, str] | None,
expected_draft_type: AgentConfigDraftType,
) -> None:
agent_id = "00000000-0000-0000-0000-000000000001"
captured: dict[str, object] = {}
@@ -557,7 +570,9 @@ def test_agent_debug_conversation_refresh_uses_current_user(
monkeypatch.setattr(roster_controller, "_agent_roster_service", lambda *_args: FakeRosterService())
with app.test_request_context(
"/console/api/agent/00000000-0000-0000-0000-000000000001/debug-conversation/refresh", method="POST"
"/console/api/agent/00000000-0000-0000-0000-000000000001/debug-conversation/refresh",
method="POST",
json=payload,
):
response = unwrap(AgentDebugConversationRefreshApi.post)(
AgentDebugConversationRefreshApi(), MagicMock(), "tenant-1", SimpleNamespace(id=account_id), agent_id
@@ -567,7 +582,12 @@ def test_agent_debug_conversation_refresh_uses_current_user(
"debug_conversation_has_messages": False,
"debug_conversation_message_count": 0,
}
assert captured == {"tenant_id": "tenant-1", "agent_id": agent_id, "account_id": account_id}
assert captured == {
"tenant_id": "tenant-1",
"agent_id": agent_id,
"account_id": account_id,
"draft_type": expected_draft_type,
}
def test_agent_publish_and_build_draft_routes_call_composer_service(
@@ -1456,6 +1476,7 @@ def test_build_chat_finalization_helper_forces_debug_build_and_push_prompt(
"current_user": SimpleNamespace(id=account_id),
"app_model": app_model,
"agent_id": "agent-1",
"draft_type": AgentConfigDraftType.DEBUG_BUILD,
}
generate_call = cast(dict[str, object], captured["generate"])
assert generate_call["app_model"] is app_model
@@ -1520,11 +1541,15 @@ def test_agent_chat_helper_forces_agent_streaming_and_external_trace(
captured.update(kwargs)
return {"answer": "ok"}
def resolve_debug_conversation(**kwargs: object) -> str:
captured["resolve_debug_conversation"] = kwargs
return "debug-conversation-1"
monkeypatch.setattr(completion_controller.AppGenerateService, "generate", generate)
monkeypatch.setattr(
completion_controller,
"_resolve_current_user_agent_debug_conversation_id",
lambda **kwargs: "debug-conversation-1",
resolve_debug_conversation,
)
monkeypatch.setattr(
completion_controller.helper, "compact_generate_response", lambda response: {"response": response}
@@ -1544,6 +1569,7 @@ def test_agent_chat_helper_forces_agent_streaming_and_external_trace(
assert args["conversation_id"] == "debug-conversation-1"
assert args["auto_generate_name"] is False
assert args["external_trace_id"] == "trace-1"
assert cast(dict[str, object], captured["resolve_debug_conversation"])["draft_type"] == AgentConfigDraftType.DRAFT
def test_agent_chat_helper_ignores_private_exit_intent_payload_key(
@@ -1642,6 +1668,7 @@ def test_resolve_current_user_agent_debug_conversation_uses_agent_or_backing_app
current_user=SimpleNamespace(id="account-1"),
app_model=SimpleNamespace(id="app-1"),
agent_id="agent-1",
draft_type=AgentConfigDraftType.DRAFT,
)
fallback_id = completion_controller._resolve_current_user_agent_debug_conversation_id(
session="session-1", # type: ignore[arg-type]
@@ -1649,13 +1676,26 @@ def test_resolve_current_user_agent_debug_conversation_uses_agent_or_backing_app
current_user=SimpleNamespace(id="account-1"),
app_model=SimpleNamespace(id="app-1"),
agent_id=None,
draft_type=AgentConfigDraftType.DEBUG_BUILD,
)
assert explicit_id == "debug-agent-1"
assert fallback_id == "debug-backing-agent"
assert calls[1] == {"get_or_create": {"tenant_id": "tenant-1", "agent_id": "agent-1", "account_id": "account-1"}}
assert calls[1] == {
"get_or_create": {
"tenant_id": "tenant-1",
"agent_id": "agent-1",
"account_id": "account-1",
"draft_type": AgentConfigDraftType.DRAFT,
}
}
assert calls[3] == {"get_app_backing_agent": {"tenant_id": "tenant-1", "app_id": "app-1"}}
assert calls[4] == {
"get_or_create": {"tenant_id": "tenant-1", "agent_id": "backing-agent", "account_id": "account-1"}
"get_or_create": {
"tenant_id": "tenant-1",
"agent_id": "backing-agent",
"account_id": "account-1",
"draft_type": AgentConfigDraftType.DEBUG_BUILD,
}
}
@@ -0,0 +1,150 @@
from types import SimpleNamespace
from unittest.mock import MagicMock, patch
import pytest
from werkzeug.exceptions import Forbidden
from controllers.console.app.error import AppNotFoundError
from controllers.console.app.wraps import agent_manage_required_for_agent_app
from core.rbac import RBACPermission, RBACResourceScope
from models.agent import AgentScope
TENANT_ID = "tenant-1"
ACCOUNT = SimpleNamespace(id="account-1")
def _guarded_view():
calls: list[dict[str, object]] = []
@agent_manage_required_for_agent_app
def view(*args, **kwargs):
calls.append(kwargs)
return "ok"
return view, calls
def _app_with_binding(binding):
app_model = MagicMock()
app_model.agent_app_binding_with_session.return_value = binding
return app_model
def _patch_guard(app_model, rbac_enabled: bool):
mock_db = MagicMock()
mock_db.session.scalar.return_value = app_model
return (
patch("controllers.console.app.wraps.db", mock_db),
patch("controllers.console.app.wraps.current_account_with_tenant", return_value=(ACCOUNT, TENANT_ID)),
patch("controllers.console.app.wraps.dify_config.RBAC_ENABLED", rbac_enabled),
)
class TestAgentManageRequiredForAgentApp:
def test_non_agent_app_passes_through_without_workspace_check(self):
view, calls = _guarded_view()
patches = _patch_guard(_app_with_binding(None), rbac_enabled=True)
with patches[0], patches[1], patches[2], patch("controllers.console.app.wraps.enforce_rbac_access") as gate:
assert view(app_id="app-1") == "ok"
gate.assert_not_called()
assert calls == [{"app_id": "app-1"}]
def test_roster_agent_app_requires_agent_manage_when_rbac_enabled(self):
view, _ = _guarded_view()
binding = SimpleNamespace(scope=AgentScope.ROSTER)
patches = _patch_guard(_app_with_binding(binding), rbac_enabled=True)
with patches[0], patches[1], patches[2], patch("controllers.console.app.wraps.enforce_rbac_access") as gate:
assert view(app_id="app-1") == "ok"
gate.assert_called_once_with(
tenant_id=TENANT_ID,
account_id=ACCOUNT.id,
resource_type=RBACResourceScope.WORKSPACE,
scene=RBACPermission.AGENT_MANAGE,
resource_required=False,
)
def test_roster_agent_app_denied_without_agent_manage(self):
view, calls = _guarded_view()
binding = SimpleNamespace(scope=AgentScope.ROSTER)
patches = _patch_guard(_app_with_binding(binding), rbac_enabled=True)
with (
patches[0],
patches[1],
patches[2],
patch("controllers.console.app.wraps.enforce_rbac_access", side_effect=Forbidden()),
):
with pytest.raises(Forbidden):
view(app_id="app-1")
assert calls == []
def test_roster_agent_app_skips_workspace_check_when_rbac_disabled(self):
view, _ = _guarded_view()
binding = SimpleNamespace(scope=AgentScope.ROSTER)
patches = _patch_guard(_app_with_binding(binding), rbac_enabled=False)
with patches[0], patches[1], patches[2], patch("controllers.console.app.wraps.enforce_rbac_access") as gate:
assert view(app_id="app-1") == "ok"
gate.assert_not_called()
def test_hidden_backing_app_is_rejected_even_without_rbac(self):
"""A workflow-only backing App is not part of the general app management plane."""
view, calls = _guarded_view()
binding = SimpleNamespace(scope=AgentScope.WORKFLOW_ONLY)
patches = _patch_guard(_app_with_binding(binding), rbac_enabled=False)
with patches[0], patches[1], patches[2]:
with pytest.raises(AppNotFoundError):
view(app_id="app-1")
assert calls == []
def test_hidden_backing_app_is_rejected_before_workspace_check(self):
view, calls = _guarded_view()
binding = SimpleNamespace(scope=AgentScope.WORKFLOW_ONLY)
patches = _patch_guard(_app_with_binding(binding), rbac_enabled=True)
with patches[0], patches[1], patches[2], patch("controllers.console.app.wraps.enforce_rbac_access") as gate:
with pytest.raises(AppNotFoundError):
view(app_id="app-1")
gate.assert_not_called()
assert calls == []
def test_binding_lookup_covers_archived_agents(self):
"""An Agent App stays gated after its roster Agent is archived."""
view, _ = _guarded_view()
app_model = _app_with_binding(SimpleNamespace(scope=AgentScope.ROSTER))
patches = _patch_guard(app_model, rbac_enabled=True)
with patches[0], patches[1], patches[2], patch("controllers.console.app.wraps.enforce_rbac_access"):
view(app_id="app-1")
_, call_kwargs = app_model.agent_app_binding_with_session.call_args
assert call_kwargs["include_archived"] is True
def test_resource_id_path_alias_is_resolved(self):
view, _ = _guarded_view()
binding = SimpleNamespace(scope=AgentScope.ROSTER)
patches = _patch_guard(_app_with_binding(binding), rbac_enabled=True)
with patches[0], patches[1], patches[2], patch("controllers.console.app.wraps.enforce_rbac_access") as gate:
assert view(resource_id="app-1") == "ok"
gate.assert_called_once()
def test_unknown_app_passes_through_for_downstream_handling(self):
view, calls = _guarded_view()
patches = _patch_guard(None, rbac_enabled=True)
with patches[0], patches[1], patches[2], patch("controllers.console.app.wraps.enforce_rbac_access") as gate:
assert view(app_id="app-1") == "ok"
gate.assert_not_called()
assert calls == [{"app_id": "app-1"}]
@@ -9,13 +9,14 @@ from unittest.mock import MagicMock
import pytest
from flask import Flask
from sqlalchemy import event
from sqlalchemy import Engine, event
from sqlalchemy.orm import Session
from controllers.console.app import app_import as app_import_module
from models.account import Account
from models.base import TypeBase
from models.engine import db
from models.model import App
from models.model import App, AppMode
from services.app_dsl_service import ImportStatus
from services.entities.dsl_entities import CheckDependenciesResult
from services.feature_service import SystemFeatureModel, WebAppAuthModel
@@ -66,6 +67,13 @@ def app() -> Iterator[Flask]:
yield app
@pytest.fixture
def sqlite_app_engine(app: Flask) -> Engine:
engine = db.engine
TypeBase.metadata.create_all(engine, tables=[TypeBase.metadata.tables[App.__tablename__]])
return engine
@dataclass
class TransactionEvents:
commits: int = 0
@@ -93,11 +101,34 @@ def transaction_events() -> TransactionEvents:
event.remove(Session, "after_rollback", record_rollback)
def _failed_result_after_starting_transaction(
service: app_import_module.AppDslService, *, app_id: str | None = None
) -> _Result:
service._session.begin()
return _Result(ImportStatus.FAILED, app_id=app_id)
def _install_persisting_service_result(
monkeypatch: pytest.MonkeyPatch,
*,
method_name: str,
result: _Result,
) -> str:
app_id = result.app_id or "rolled-back-app"
def _return_result(import_service: app_import_module.AppDslService, *_args, **_kwargs):
import_service._session.add(
App(
id=app_id,
tenant_id="tenant-1",
name="Imported App",
mode=AppMode.WORKFLOW,
enable_site=True,
enable_api=True,
)
)
return result
monkeypatch.setattr(app_import_module.AppDslService, method_name, _return_result)
return app_id
def _assert_app_persistence(sqlite_app_engine: Engine, app_id: str, *, persisted: bool) -> None:
with Session(sqlite_app_engine) as session:
assert (session.get(App, app_id) is not None) is persisted
class TestAppImportApi:
@@ -110,15 +141,16 @@ class TestAppImportApi:
api,
app: Flask,
monkeypatch: pytest.MonkeyPatch,
sqlite_app_engine: Engine,
transaction_events: TransactionEvents,
) -> None:
method = unwrap(api.post)
_install_features(monkeypatch, enabled=False)
monkeypatch.setattr(
app_import_module.AppDslService,
"import_app",
lambda service, *_args, **_kwargs: _failed_result_after_starting_transaction(service, app_id=None),
app_id = _install_persisting_service_result(
monkeypatch,
method_name="import_app",
result=_Result(ImportStatus.FAILED, app_id=None),
)
with app.test_request_context("/console/api/apps/imports", method="POST", json={"mode": "yaml-content"}):
@@ -126,6 +158,7 @@ class TestAppImportApi:
assert transaction_events.rollbacks == 1
assert transaction_events.commits == 0
_assert_app_persistence(sqlite_app_engine, app_id, persisted=False)
assert status == 400
assert response["status"] == ImportStatus.FAILED
@@ -134,15 +167,16 @@ class TestAppImportApi:
api,
app: Flask,
monkeypatch: pytest.MonkeyPatch,
sqlite_app_engine: Engine,
transaction_events: TransactionEvents,
) -> None:
method = unwrap(api.post)
_install_features(monkeypatch, enabled=False)
monkeypatch.setattr(
app_import_module.AppDslService,
"import_app",
lambda *_args, **_kwargs: _Result(ImportStatus.PENDING),
app_id = _install_persisting_service_result(
monkeypatch,
method_name="import_app",
result=_Result(ImportStatus.PENDING),
)
with app.test_request_context("/console/api/apps/imports", method="POST", json={"mode": "yaml-content"}):
@@ -150,6 +184,7 @@ class TestAppImportApi:
assert transaction_events.commits == 1
assert transaction_events.rollbacks == 0
_assert_app_persistence(sqlite_app_engine, app_id, persisted=True)
assert status == 202
assert response["status"] == ImportStatus.PENDING
@@ -158,15 +193,16 @@ class TestAppImportApi:
api,
app: Flask,
monkeypatch: pytest.MonkeyPatch,
sqlite_app_engine: Engine,
transaction_events: TransactionEvents,
) -> None:
method = unwrap(api.post)
_install_features(monkeypatch, enabled=True)
monkeypatch.setattr(
app_import_module.AppDslService,
"import_app",
lambda *_args, **_kwargs: _Result(ImportStatus.COMPLETED, app_id="app-123"),
app_id = _install_persisting_service_result(
monkeypatch,
method_name="import_app",
result=_Result(ImportStatus.COMPLETED, app_id="app-123"),
)
update_access = MagicMock()
monkeypatch.setattr(app_import_module.EnterpriseService.WebAppAuth, "update_app_access_mode", update_access)
@@ -176,6 +212,7 @@ class TestAppImportApi:
assert transaction_events.commits == 1
assert transaction_events.rollbacks == 0
_assert_app_persistence(sqlite_app_engine, app_id, persisted=True)
update_access.assert_called_once_with("app-123", "private")
assert status == 200
assert response["status"] == ImportStatus.COMPLETED
@@ -185,6 +222,7 @@ class TestAppImportApi:
api,
app: Flask,
monkeypatch: pytest.MonkeyPatch,
sqlite_app_engine: Engine,
transaction_events: TransactionEvents,
) -> None:
method = _unwrap(api.post)
@@ -196,10 +234,10 @@ class TestAppImportApi:
lambda: (_make_account(), "tenant-1"),
)
monkeypatch.setattr(app_import_module.dify_config, "RBAC_ENABLED", True)
monkeypatch.setattr(
app_import_module.AppDslService,
"import_app",
lambda *_args, **_kwargs: _Result(ImportStatus.COMPLETED, app_id="app-123"),
app_id = _install_persisting_service_result(
monkeypatch,
method_name="import_app",
result=_Result(ImportStatus.COMPLETED, app_id="app-123"),
)
monkeypatch.setattr(
app_import_module,
@@ -211,6 +249,7 @@ class TestAppImportApi:
response, status = method()
assert transaction_events.commits == 1
_assert_app_persistence(sqlite_app_engine, app_id, persisted=True)
assert status == 200
assert response["permission_keys"] == ["app.acl.view_layout", "app.acl.edit"]
@@ -219,6 +258,7 @@ class TestAppImportApi:
api,
app: Flask,
monkeypatch: pytest.MonkeyPatch,
sqlite_app_engine: Engine,
transaction_events: TransactionEvents,
) -> None:
method = _unwrap(api.post)
@@ -230,10 +270,10 @@ class TestAppImportApi:
lambda: (_make_account(), "tenant-1"),
)
monkeypatch.setattr(app_import_module.dify_config, "RBAC_ENABLED", True)
monkeypatch.setattr(
app_import_module.AppDslService,
"import_app",
lambda *_args, **_kwargs: _Result(ImportStatus.COMPLETED, app_id="app-123"),
app_id = _install_persisting_service_result(
monkeypatch,
method_name="import_app",
result=_Result(ImportStatus.COMPLETED, app_id="app-123"),
)
monkeypatch.setattr(
app_import_module,
@@ -249,6 +289,7 @@ class TestAppImportApi:
response, status = method()
assert transaction_events.commits == 1
_assert_app_persistence(sqlite_app_engine, app_id, persisted=True)
assert status == 200
assert response["permission_keys"] == []
@@ -263,14 +304,15 @@ class TestAppImportConfirmApi:
api,
app: Flask,
monkeypatch: pytest.MonkeyPatch,
sqlite_app_engine: Engine,
transaction_events: TransactionEvents,
) -> None:
method = unwrap(api.post)
monkeypatch.setattr(
app_import_module.AppDslService,
"confirm_import",
lambda service, *_args, **_kwargs: _failed_result_after_starting_transaction(service),
app_id = _install_persisting_service_result(
monkeypatch,
method_name="confirm_import",
result=_Result(ImportStatus.FAILED),
)
with app.test_request_context("/console/api/apps/imports/import-1/confirm", method="POST"):
@@ -278,6 +320,7 @@ class TestAppImportConfirmApi:
assert transaction_events.rollbacks == 1
assert transaction_events.commits == 0
_assert_app_persistence(sqlite_app_engine, app_id, persisted=False)
assert status == 400
assert response["status"] == ImportStatus.FAILED
@@ -286,6 +329,7 @@ class TestAppImportConfirmApi:
api,
app: Flask,
monkeypatch: pytest.MonkeyPatch,
sqlite_app_engine: Engine,
transaction_events: TransactionEvents,
) -> None:
method = _unwrap(api.post)
@@ -304,10 +348,10 @@ class TestAppImportConfirmApi:
),
)
monkeypatch.setattr(app_import_module.dify_config, "RBAC_ENABLED", True)
monkeypatch.setattr(
app_import_module.AppDslService,
"confirm_import",
lambda *_args, **_kwargs: _Result(ImportStatus.COMPLETED, app_id="app-456"),
app_id = _install_persisting_service_result(
monkeypatch,
method_name="confirm_import",
result=_Result(ImportStatus.COMPLETED, app_id="app-456"),
)
monkeypatch.setattr(
app_import_module,
@@ -319,6 +363,7 @@ class TestAppImportConfirmApi:
response, status = method(import_id="import-1")
assert transaction_events.commits == 1
_assert_app_persistence(sqlite_app_engine, app_id, persisted=True)
assert status == 200
assert response["permission_keys"] == ["app.acl.view_layout", "app.acl.edit"]
@@ -327,6 +372,7 @@ class TestAppImportConfirmApi:
api,
app: Flask,
monkeypatch: pytest.MonkeyPatch,
sqlite_app_engine: Engine,
transaction_events: TransactionEvents,
) -> None:
method = _unwrap(api.post)
@@ -345,10 +391,10 @@ class TestAppImportConfirmApi:
),
)
monkeypatch.setattr(app_import_module.dify_config, "RBAC_ENABLED", True)
monkeypatch.setattr(
app_import_module.AppDslService,
"confirm_import",
lambda *_args, **_kwargs: _Result(ImportStatus.COMPLETED, app_id="app-456"),
app_id = _install_persisting_service_result(
monkeypatch,
method_name="confirm_import",
result=_Result(ImportStatus.COMPLETED, app_id="app-456"),
)
monkeypatch.setattr(
app_import_module,
@@ -360,6 +406,7 @@ class TestAppImportConfirmApi:
response, status = method(import_id="import-1")
assert transaction_events.commits == 1
_assert_app_persistence(sqlite_app_engine, app_id, persisted=True)
assert status == 200
assert response["permission_keys"] == []
@@ -1,5 +1,6 @@
import pytest
from flask import Flask
from sqlalchemy.orm import Session
from controllers.console.app import generator as generator_module
from core.errors.error import ModelCurrentlyNotSupportError, ProviderTokenNotInitError, QuotaExceededError
@@ -102,12 +103,14 @@ def test_structured_output_generate_exceptions(app: Flask, monkeypatch: pytest.M
method(api, "t1")
def test_instruction_generate_exceptions(app: Flask, monkeypatch: pytest.MonkeyPatch) -> None:
@pytest.mark.parametrize("sqlite_session", [()], indirect=True)
def test_instruction_generate_exceptions(
app: Flask,
monkeypatch: pytest.MonkeyPatch,
sqlite_session: Session,
) -> None:
api = generator_module.InstructionGenerateApi()
method = unwrap(api.post)
from types import SimpleNamespace
session = SimpleNamespace()
exceptions_to_test = [
(ProviderTokenNotInitError("token error"), generator_module.ProviderNotInitializeError),
@@ -135,4 +138,4 @@ def test_instruction_generate_exceptions(app: Flask, monkeypatch: pytest.MonkeyP
},
):
with pytest.raises(expected_exception):
method(api, session, "t1")
method(api, sqlite_session, "t1")
@@ -11,7 +11,7 @@ import pytest
from flask import Flask
from sqlalchemy import func, select
from sqlalchemy.engine import Engine
from sqlalchemy.orm import object_session, sessionmaker
from sqlalchemy.orm import Session, object_session, sessionmaker
from controllers.common import session as controller_session
from controllers.console.app import model_config as model_config_module
@@ -29,11 +29,14 @@ def _poison_implicit_app_config_properties(monkeypatch: pytest.MonkeyPatch) -> N
@pytest.mark.parametrize("app_mode", [AppMode.CHAT, AppMode.COMPLETION])
@pytest.mark.parametrize("sqlite_session", [(AppModelConfig,)], indirect=True)
def test_post_updates_non_agent_model_config_without_implicit_properties(
app: Flask,
monkeypatch: pytest.MonkeyPatch,
app_mode: AppMode,
sqlite_session: Session,
) -> None:
"""Flush a non-agent config through the injected session without legacy model properties."""
api = model_config_module.ModelConfigResource()
method = unwrap(api.post)
@@ -45,14 +48,16 @@ def test_post_updates_non_agent_model_config_without_implicit_properties(
updated_at=None,
)
original_config = AppModelConfig(app_id="app-1", created_by="u1", updated_by="u1")
original_config.id = "config-0"
original_config.agent_mode = None
sqlite_session.add(original_config)
sqlite_session.commit()
_poison_implicit_app_config_properties(monkeypatch)
monkeypatch.setattr(
model_config_module.AppModelConfigService,
"validate_configuration",
lambda **_kwargs: {"pre_prompt": "hi"},
)
session = MagicMock()
def _from_model_config_dict(self, model_config):
self.pre_prompt = model_config["pre_prompt"]
@@ -62,18 +67,16 @@ def test_post_updates_non_agent_model_config_without_implicit_properties(
monkeypatch.setattr(AppModelConfig, "from_model_config_dict", _from_model_config_dict)
send_mock = MagicMock()
monkeypatch.setattr(model_config_module.app_model_config_was_updated, "send", send_mock)
session.get.return_value = original_config
with app.test_request_context("/console/api/apps/app-1/model-config", method="POST", json={"pre_prompt": "hi"}):
response = method(api, session, "t1", "u1", app_model=app_model)
response = method(api, sqlite_session, "t1", "u1", app_model=app_model)
session.get.assert_called_once_with(AppModelConfig, "config-0")
session.add.assert_called_once()
session.flush.assert_called_once()
session.commit.assert_not_called()
assert send_mock.call_args.kwargs["session"] is session
assert send_mock.call_args.kwargs["session"] is sqlite_session
assert app_model.app_model_config_id == "config-1"
assert app_model.mode == app_mode
persisted_config = sqlite_session.get(AppModelConfig, "config-1")
assert persisted_config is not None
assert persisted_config.pre_prompt == "hi"
assert response["result"] == "success"
@@ -160,7 +163,11 @@ def test_post_uses_one_session_and_rolls_back_when_signal_fails(
assert config_count == 1
def test_post_encrypts_agent_tool_parameters(app: Flask, monkeypatch: pytest.MonkeyPatch) -> None:
@pytest.mark.parametrize("sqlite_session", [(AppModelConfig,)], indirect=True)
def test_post_encrypts_agent_tool_parameters(
app: Flask, monkeypatch: pytest.MonkeyPatch, sqlite_session: Session
) -> None:
"""Agent parameter encryption reads and writes persisted model configurations."""
api = model_config_module.ModelConfigResource()
method = unwrap(api.post)
@@ -174,6 +181,7 @@ def test_post_encrypts_agent_tool_parameters(app: Flask, monkeypatch: pytest.Mon
_poison_implicit_app_config_properties(monkeypatch)
original_config = AppModelConfig(app_id="app-1", created_by="u1", updated_by="u1")
original_config.id = "config-0"
original_config.agent_mode = json.dumps(
{
"enabled": True,
@@ -190,8 +198,8 @@ def test_post_encrypts_agent_tool_parameters(app: Flask, monkeypatch: pytest.Mon
}
)
session = MagicMock()
session.scalar.return_value = original_config
sqlite_session.add(original_config)
sqlite_session.commit()
monkeypatch.setattr(
model_config_module.AppModelConfigService,
@@ -236,11 +244,11 @@ def test_post_encrypts_agent_tool_parameters(app: Flask, monkeypatch: pytest.Mon
monkeypatch.setattr(model_config_module.app_model_config_was_updated, "send", send_mock)
with app.test_request_context("/console/api/apps/app-1/model-config", method="POST", json={"pre_prompt": "hi"}):
response = method(api, session, "t1", "u1", app_model=app_model)
response = method(api, sqlite_session, "t1", "u1", app_model=app_model)
stored_config = session.add.call_args[0][0]
stored_config = sqlite_session.get(AppModelConfig, app_model.app_model_config_id)
assert stored_config is not None
stored_agent_mode = json.loads(stored_config.agent_mode)
session.scalar.assert_called_once()
assert app_model.mode == AppMode.AGENT_CHAT
assert stored_agent_mode["tools"][0]["tool_parameters"]["secret"] == "encrypted"
assert response["result"] == "success"
@@ -14,6 +14,7 @@ import pytest
from flask import Flask
from controllers.console.auth.activate import ActivateApi, ActivateCheckApi
from controllers.console.auth.error import InvitationAccountMismatchError
from controllers.console.error import AccountInFreezeError, AlreadyActivateError
from models.account import AccountStatus, TenantAccountRole
@@ -202,6 +203,50 @@ class TestActivateApi:
with patch("controllers.console.auth.activate.TenantService.switch_tenant") as mock:
yield mock
@patch("controllers.console.auth.activate.TenantService.create_tenant_member")
@patch("controllers.console.auth.activate.RegisterService.get_invitation_with_case_fallback")
@patch("controllers.console.auth.activate.RegisterService.revoke_token")
@patch("controllers.console.auth.activate.current_account_with_tenant")
@patch("controllers.console.auth.activate.extract_access_token", return_value="access-token")
@patch("controllers.console.auth.activate.db")
def test_activation_rejects_invitation_for_different_authenticated_account(
self,
mock_db: MagicMock,
mock_extract_access_token: MagicMock,
mock_current_account_with_tenant: MagicMock,
mock_revoke_token: MagicMock,
mock_get_invitation: MagicMock,
mock_create_tenant_member: MagicMock,
app: Flask,
mock_invitation: MagicMock,
mock_account: MagicMock,
mock_switch_tenant: MagicMock,
):
"""A logged-in account cannot consume another account's invitation token."""
current_account = MagicMock()
current_account.id = "current-account-id"
mock_account.id = "invited-account-id"
mock_account.status = AccountStatus.ACTIVE
mock_invitation["data"]["requires_setup"] = False
mock_get_invitation.return_value = mock_invitation
mock_current_account_with_tenant.return_value = (current_account, "current-workspace-id")
with app.test_request_context(
"/activate",
method="POST",
json={
"token": "valid_token",
},
):
with pytest.raises(InvitationAccountMismatchError):
ActivateApi().post()
mock_extract_access_token.assert_called_once()
mock_revoke_token.assert_not_called()
mock_create_tenant_member.assert_not_called()
mock_switch_tenant.assert_not_called()
mock_db.session.scalar.assert_not_called()
@patch("controllers.console.auth.activate.RegisterService.get_invitation_if_token_valid")
@patch("controllers.console.auth.activate.RegisterService.revoke_token")
@patch("controllers.console.auth.activate.db")
@@ -1,4 +1,4 @@
"""Testcontainers integration tests for OAuth controller endpoints."""
"""Unit tests for OAuth controller endpoints."""
from __future__ import annotations
@@ -16,15 +16,10 @@ from controllers.console.auth.oauth import (
)
from libs.oauth import OAuthUserInfo, encode_oauth_state
from models.account import AccountStatus
from services.account_service import AccountService
from services.errors.account import AccountRegisterError
class TestGetOAuthProviders:
@pytest.fixture
def app(self, flask_app_with_containers: Flask):
return flask_app_with_containers
@pytest.mark.parametrize(
("github_config", "google_config", "expected_github", "expected_google"),
[
@@ -65,10 +60,6 @@ class TestOAuthLogin:
def resource(self):
return OAuthLogin()
@pytest.fixture
def app(self, flask_app_with_containers: Flask):
return flask_app_with_containers
@pytest.fixture
def mock_oauth_provider(self):
provider = MagicMock()
@@ -181,10 +172,6 @@ class TestOAuthCallback:
def resource(self):
return OAuthCallback()
@pytest.fixture
def app(self, flask_app_with_containers: Flask):
return flask_app_with_containers
@pytest.fixture
def oauth_setup(self):
"""Common OAuth setup for callback tests"""
@@ -263,10 +250,12 @@ class TestOAuthCallback:
@patch("controllers.console.auth.oauth.dify_config")
@patch("controllers.console.auth.oauth.get_oauth_providers")
@patch("controllers.console.auth.oauth.RegisterService")
@patch("controllers.console.auth.oauth.AccountService")
@patch("controllers.console.auth.oauth.redirect")
def test_invitation_comparison_is_case_insensitive(
self,
mock_redirect,
mock_account_service,
mock_register_service,
mock_get_providers,
mock_config,
@@ -280,13 +269,20 @@ class TestOAuthCallback:
)
mock_get_providers.return_value = {"github": oauth_setup["provider"]}
mock_register_service.is_valid_invite_token.return_value = True
mock_register_service.get_invitation_by_token.return_value = {"email": "user@example.com"}
mock_register_service.get_invitation_if_token_valid.return_value = {
"account": oauth_setup["account"],
"data": {"email": "user@example.com"},
"tenant": MagicMock(),
}
mock_account_service.login.return_value = oauth_setup["token_pair"]
state = encode_oauth_state(invite_token="invite123", timezone="Asia/Shanghai")
with app.test_request_context(f"/auth/oauth/github/callback?code=test_code&state={state}"):
resource.get("github")
mock_register_service.get_invitation_by_token.assert_called_once_with(token="invite123")
mock_register_service.get_invitation_if_token_valid.assert_called_once_with(
None, None, "invite123", session=ANY
)
mock_redirect.assert_called_once_with("http://localhost:3000/signin/invite-settings?invite_token=invite123")
@pytest.mark.parametrize(
@@ -448,10 +444,6 @@ class TestOAuthCallback:
class TestAccountGeneration:
@pytest.fixture
def app(self, flask_app_with_containers: Flask):
return flask_app_with_containers
@pytest.fixture
def user_info(self):
return OAuthUserInfo(id="123", name="Test User", email="test@example.com")
@@ -468,39 +460,25 @@ class TestAccountGeneration:
self,
mock_account_model,
mock_get_account,
flask_req_ctx_with_containers,
app: Flask,
user_info: OAuthUserInfo,
mock_account,
):
# Test OpenID found
mock_account_model.get_by_openid.return_value = mock_account
result = _get_account_by_openid_or_email("github", user_info)
assert result == mock_account
mock_account_model.get_by_openid.assert_called_once_with("github", "123")
mock_get_account.assert_not_called()
with app.test_request_context("/"):
# Test OpenID found
mock_account_model.get_by_openid.return_value = mock_account
result = _get_account_by_openid_or_email("github", user_info)
assert result == mock_account
mock_account_model.get_by_openid.assert_called_once_with("github", "123")
mock_get_account.assert_not_called()
# Test fallback to email lookup
mock_account_model.get_by_openid.return_value = None
mock_get_account.return_value = mock_account
# Test fallback to email lookup
mock_account_model.get_by_openid.return_value = None
mock_get_account.return_value = mock_account
result = _get_account_by_openid_or_email("github", user_info)
assert result == mock_account
mock_get_account.assert_called_once()
def test_get_account_by_email_with_case_fallback_falls_back_to_lowercase(self):
"""Test that case fallback tries lowercase when exact match fails."""
mock_session = MagicMock()
first_result = MagicMock()
first_result.scalar_one_or_none.return_value = None
expected_account = MagicMock()
second_result = MagicMock()
second_result.scalar_one_or_none.return_value = expected_account
mock_session.execute.side_effect = [first_result, second_result]
result = AccountService.get_account_by_email_with_case_fallback("Case@Test.com", session=mock_session)
assert result is expected_account
assert mock_session.execute.call_count == 2
result = _get_account_by_openid_or_email("github", user_info)
assert result == mock_account
mock_get_account.assert_called_once()
@pytest.mark.parametrize(
("allow_register", "existing_account", "should_create"),
@@ -1,5 +1,5 @@
import urllib.parse
from unittest.mock import MagicMock, patch
from unittest.mock import ANY, MagicMock, patch
import pytest
from flask import Flask
@@ -91,3 +91,102 @@ def test_oauth_callback_validates_redirect_url_and_appends_new_user_flag(
assert response.headers["Location"] == (
f"{expected_target_url}{query_char}oauth_new_user={str(oauth_new_user).lower()}"
)
def test_oauth_callback_with_invitation_establishes_console_session(app: Flask) -> None:
oauth_provider = MagicMock()
oauth_provider.get_access_token.return_value = "google-access-token"
oauth_provider.get_user_info.return_value = OAuthUserInfo(
id="google-user-123",
name="Test User",
email="Invitee@Example.com",
)
account = MagicMock()
account.status = AccountStatus.ACTIVE
token_pair = MagicMock()
token_pair.access_token = "dify-access-token"
token_pair.refresh_token = "dify-refresh-token"
token_pair.csrf_token = "dify-csrf-token"
state = encode_oauth_state(invite_token="invite-token")
with (
patch("controllers.console.auth.oauth.get_oauth_providers", return_value={"google": oauth_provider}),
patch("controllers.console.auth.oauth.dify_config.CONSOLE_WEB_URL", CONSOLE_WEB_URL),
patch("controllers.console.auth.oauth.RegisterService") as register_service,
patch("controllers.console.auth.oauth.AccountService.link_account_integrate") as link_account,
patch("controllers.console.auth.oauth.AccountService.login", return_value=token_pair) as login,
patch("controllers.console.auth.oauth.TenantService.create_owner_tenant_if_not_exist") as create_workspace,
patch("controllers.console.auth.oauth.set_access_token_to_cookie") as set_access_cookie,
patch("controllers.console.auth.oauth.set_refresh_token_to_cookie") as set_refresh_cookie,
patch("controllers.console.auth.oauth.set_csrf_token_to_cookie") as set_csrf_cookie,
app.test_request_context(f"/oauth/authorize/google?code=test-code&state={state}"),
):
register_service.is_valid_invite_token.return_value = True
register_service.get_invitation_if_token_valid.return_value = {
"account": account,
"data": {
"account_id": "account-id",
"email": "invitee@example.com",
"workspace_id": "workspace-id",
},
"tenant": MagicMock(),
}
response = OAuthCallback().get("google")
assert response.status_code == 302
assert response.headers["Location"] == (f"{CONSOLE_WEB_URL}/signin/invite-settings?invite_token=invite-token")
link_account.assert_called_once_with("google", "google-user-123", account, session=ANY)
login.assert_called_once_with(account=account, session=ANY, ip_address=ANY)
create_workspace.assert_not_called()
set_access_cookie.assert_called_once_with(ANY, response, "dify-access-token")
set_refresh_cookie.assert_called_once_with(ANY, response, "dify-refresh-token")
set_csrf_cookie.assert_called_once_with(ANY, response, "dify-csrf-token")
def test_oauth_callback_with_invitation_rejects_another_account(app: Flask) -> None:
oauth_provider = MagicMock()
oauth_provider.get_access_token.return_value = "google-access-token"
oauth_provider.get_user_info.return_value = OAuthUserInfo(
id="google-user-123",
name="Test User",
email="another@example.com",
)
account = MagicMock()
account.status = AccountStatus.ACTIVE
state = encode_oauth_state(invite_token="invite-token")
with (
patch("controllers.console.auth.oauth.get_oauth_providers", return_value={"google": oauth_provider}),
patch("controllers.console.auth.oauth.dify_config.CONSOLE_WEB_URL", CONSOLE_WEB_URL),
patch("controllers.console.auth.oauth.RegisterService") as register_service,
patch("controllers.console.auth.oauth.AccountService.link_account_integrate") as link_account,
patch("controllers.console.auth.oauth.AccountService.login") as login,
patch("controllers.console.auth.oauth.set_access_token_to_cookie") as set_access_cookie,
patch("controllers.console.auth.oauth.set_refresh_token_to_cookie") as set_refresh_cookie,
patch("controllers.console.auth.oauth.set_csrf_token_to_cookie") as set_csrf_cookie,
app.test_request_context(f"/oauth/authorize/google?code=test-code&state={state}"),
):
register_service.is_valid_invite_token.return_value = True
register_service.get_invitation_if_token_valid.return_value = {
"account": account,
"data": {
"account_id": "account-id",
"email": "invitee@example.com",
"workspace_id": "workspace-id",
},
"tenant": MagicMock(),
}
response = OAuthCallback().get("google")
query = urllib.parse.parse_qs(urllib.parse.urlparse(response.headers["Location"]).query)
assert response.status_code == 302
assert query["message"] == ["This invitation was sent to another account. Please sign in with the invited account."]
assert query["invite_token"] == ["invite-token"]
link_account.assert_not_called()
login.assert_not_called()
register_service.revoke_token.assert_not_called()
set_access_cookie.assert_not_called()
set_refresh_cookie.assert_not_called()
set_csrf_cookie.assert_not_called()
@@ -1,12 +1,14 @@
"""Testcontainers integration tests for password reset authentication flows."""
"""Unit tests for password reset controller flows."""
from __future__ import annotations
from collections.abc import Generator
from contextlib import contextmanager
from unittest.mock import MagicMock, patch
import pytest
from flask import Flask
from sqlalchemy.orm import Session
from sqlalchemy.orm import Session, scoped_session, sessionmaker
from controllers.console.auth.error import (
EmailCodeError,
@@ -21,47 +23,58 @@ from controllers.console.auth.forgot_password import (
ForgotPasswordSendEmailApi,
)
from controllers.console.error import AccountNotFound, EmailSendIpLimitError
from tests.test_containers_integration_tests.controllers.console.helpers import ensure_dify_setup
from models.account import Account, Tenant, TenantAccountJoin
from services.feature_service import SystemFeatureModel
SQLITE_MODELS = (Account, Tenant, TenantAccountJoin)
@contextmanager
def _bind_database_session(session: Session) -> Generator[scoped_session[Session]]:
"""Bind the controller's session proxy to the SQLite test engine."""
database_session = scoped_session(sessionmaker(bind=session.get_bind(), expire_on_commit=False))
try:
with patch("controllers.console.auth.forgot_password.db.session", database_session):
yield database_session
finally:
database_session.remove()
@pytest.fixture(autouse=True)
def enable_password_login_wrappers(monkeypatch: pytest.MonkeyPatch) -> None:
"""Keep endpoint decorators deterministic without requiring the configured app database."""
monkeypatch.setattr("controllers.console.wraps.dify_config.EDITION", "CLOUD")
monkeypatch.setattr(
"controllers.console.wraps.FeatureService.get_system_features",
lambda: SystemFeatureModel(enable_email_password_login=True),
)
class TestForgotPasswordSendEmailApi:
"""Test cases for sending password reset emails."""
@pytest.fixture
def app(self, flask_app_with_containers: Flask, db_session_with_containers: Session):
ensure_dify_setup(db_session_with_containers)
return flask_app_with_containers
@pytest.fixture
def mock_account(self):
"""Create mock account object."""
account = MagicMock()
account.email = "test@example.com"
account.name = "Test User"
return account
@pytest.mark.parametrize("sqlite_session", [SQLITE_MODELS], indirect=True)
@patch("controllers.console.auth.forgot_password.AccountService.is_email_send_ip_limit")
@patch("controllers.console.auth.forgot_password.AccountService.get_account_by_email_with_case_fallback")
@patch("controllers.console.auth.forgot_password.AccountService.send_reset_password_email")
@patch("controllers.console.auth.forgot_password.FeatureService.get_system_features")
def test_send_reset_email_success(
self,
mock_get_features,
mock_send_email,
mock_get_account,
mock_is_ip_limit,
app: Flask,
mock_account,
sqlite_session: Session,
):
# Arrange
mock_is_ip_limit.return_value = False
mock_get_account.return_value = mock_account
mock_send_email.return_value = "reset_token_123"
mock_get_features.return_value.is_allow_register = True
# Act
with app.test_request_context(
"/forgot-password", method="POST", json={"email": "test@example.com", "language": "en-US"}
with (
_bind_database_session(sqlite_session),
app.test_request_context(
"/forgot-password", method="POST", json={"email": "test@example.com", "language": "en-US"}
),
):
api = ForgotPasswordSendEmailApi()
response = api.post()
@@ -98,20 +111,17 @@ class TestForgotPasswordSendEmailApi:
(None, "en-US"), # Defaults to en-US when not provided
],
)
@pytest.mark.parametrize("sqlite_session", [SQLITE_MODELS], indirect=True)
@patch("controllers.console.auth.forgot_password.AccountService.is_email_send_ip_limit")
@patch("controllers.console.auth.forgot_password.AccountService.get_account_by_email_with_case_fallback")
@patch("controllers.console.auth.forgot_password.AccountService.send_reset_password_email")
@patch("controllers.console.auth.forgot_password.FeatureService.get_system_features")
def test_send_reset_email_language_handling(
self,
mock_get_features,
mock_send_email,
mock_get_account,
mock_is_ip_limit,
app: Flask,
mock_account,
language_input,
expected_language,
app: Flask,
sqlite_session: Session,
):
"""
Test password reset email with different language preferences.
@@ -122,13 +132,14 @@ class TestForgotPasswordSendEmailApi:
"""
# Arrange
mock_is_ip_limit.return_value = False
mock_get_account.return_value = mock_account
mock_send_email.return_value = "token"
mock_get_features.return_value.is_allow_register = True
# Act
with app.test_request_context(
"/forgot-password", method="POST", json={"email": "test@example.com", "language": language_input}
with (
_bind_database_session(sqlite_session),
app.test_request_context(
"/forgot-password", method="POST", json={"email": "test@example.com", "language": language_input}
),
):
api = ForgotPasswordSendEmailApi()
api.post()
@@ -141,11 +152,6 @@ class TestForgotPasswordSendEmailApi:
class TestForgotPasswordCheckApi:
"""Test cases for verifying password reset codes."""
@pytest.fixture
def app(self, flask_app_with_containers: Flask, db_session_with_containers: Session):
ensure_dify_setup(db_session_with_containers)
return flask_app_with_containers
@patch("controllers.console.auth.forgot_password.AccountService.is_forgot_password_error_rate_limit")
@patch("controllers.console.auth.forgot_password.AccountService.get_reset_password_data")
@patch("controllers.console.auth.forgot_password.AccountService.revoke_reset_password_token")
@@ -153,10 +159,10 @@ class TestForgotPasswordCheckApi:
@patch("controllers.console.auth.forgot_password.AccountService.reset_forgot_password_error_rate_limit")
def test_verify_code_success(
self,
mock_reset_rate_limit,
mock_generate_token,
mock_revoke_token,
mock_get_data,
mock_reset_rate_limit: MagicMock,
mock_generate_token: MagicMock,
mock_revoke_token: MagicMock,
mock_get_data: MagicMock,
mock_is_rate_limit,
app: Flask,
):
@@ -200,10 +206,10 @@ class TestForgotPasswordCheckApi:
@patch("controllers.console.auth.forgot_password.AccountService.reset_forgot_password_error_rate_limit")
def test_verify_code_preserves_token_email_case(
self,
mock_reset_rate_limit,
mock_generate_token,
mock_revoke_token,
mock_get_data,
mock_reset_rate_limit: MagicMock,
mock_generate_token: MagicMock,
mock_revoke_token: MagicMock,
mock_get_data: MagicMock,
mock_is_rate_limit,
app: Flask,
):
@@ -325,33 +331,15 @@ class TestForgotPasswordCheckApi:
class TestForgotPasswordResetApi:
"""Test cases for resetting password with verified token."""
@pytest.fixture
def app(self, flask_app_with_containers: Flask, db_session_with_containers: Session):
ensure_dify_setup(db_session_with_containers)
return flask_app_with_containers
@pytest.fixture
def mock_account(self):
"""Create mock account object."""
account = MagicMock()
account.email = "test@example.com"
account.name = "Test User"
return account
@pytest.mark.parametrize("sqlite_session", [SQLITE_MODELS], indirect=True)
@patch("controllers.console.auth.forgot_password.AccountService.get_reset_password_data")
@patch("controllers.console.auth.forgot_password.AccountService.revoke_reset_password_token")
@patch("controllers.console.auth.forgot_password.AccountService.get_account_by_email_with_case_fallback")
@patch("controllers.console.auth.forgot_password.db")
@patch("controllers.console.auth.forgot_password.TenantService.get_join_tenants")
def test_reset_password_success(
self,
mock_get_tenants,
mock_db,
mock_get_account,
mock_revoke_token,
mock_get_data,
mock_revoke_token: MagicMock,
mock_get_data: MagicMock,
app: Flask,
mock_account,
sqlite_session: Session,
):
"""
Test successful password reset.
@@ -363,25 +351,39 @@ class TestForgotPasswordResetApi:
"""
# Arrange
mock_get_data.return_value = {"email": "test@example.com", "phase": "reset"}
mock_get_account.return_value = mock_account
mock_db.session.merge.return_value = mock_account
mock_get_tenants.return_value = [MagicMock()]
# Act
with app.test_request_context(
"/forgot-password/resets",
method="POST",
json={"token": "valid_token", "new_password": "NewPass123!", "password_confirm": "NewPass123!"},
):
api = ForgotPasswordResetApi()
response = api.post()
with _bind_database_session(sqlite_session) as database_session:
account = Account(name="Test User", email="test@example.com")
tenant = Tenant(name="Test Workspace")
database_session.add_all([account, tenant])
database_session.flush()
database_session.add(TenantAccountJoin(tenant_id=tenant.id, account_id=account.id))
database_session.commit()
account_id = account.id
with app.test_request_context(
"/forgot-password/resets",
method="POST",
json={
"token": "valid_token",
"new_password": "NewPass123!",
"password_confirm": "NewPass123!",
},
):
api = ForgotPasswordResetApi()
response = api.post()
updated_account = database_session.get(Account, account_id)
# Assert
assert response["result"] == "success"
mock_revoke_token.assert_called_once_with("valid_token")
assert updated_account is not None
assert updated_account.password is not None
assert updated_account.password_salt is not None
@patch("controllers.console.auth.forgot_password.AccountService.get_reset_password_data")
def test_reset_password_mismatch(self, mock_get_data, app: Flask):
def test_reset_password_mismatch(self, app: Flask):
"""
Test password reset with mismatched passwords.
@@ -389,9 +391,6 @@ class TestForgotPasswordResetApi:
- PasswordMismatchError is raised when passwords don't match
- No password update occurs
"""
# Arrange
mock_get_data.return_value = {"email": "test@example.com", "phase": "reset"}
# Act & Assert
with app.test_request_context(
"/forgot-password/resets",
@@ -445,10 +444,12 @@ class TestForgotPasswordResetApi:
with pytest.raises(InvalidTokenError):
api.post()
@pytest.mark.parametrize("sqlite_session", [SQLITE_MODELS], indirect=True)
@patch("controllers.console.auth.forgot_password.AccountService.get_reset_password_data")
@patch("controllers.console.auth.forgot_password.AccountService.revoke_reset_password_token")
@patch("controllers.console.auth.forgot_password.AccountService.get_account_by_email_with_case_fallback")
def test_reset_password_account_not_found(self, mock_get_account, mock_revoke_token, mock_get_data, app: Flask):
def test_reset_password_account_not_found(
self, mock_revoke_token, mock_get_data, app: Flask, sqlite_session: Session
):
"""
Test password reset for non-existent account.
@@ -457,13 +458,15 @@ class TestForgotPasswordResetApi:
"""
# Arrange
mock_get_data.return_value = {"email": "nonexistent@example.com", "phase": "reset"}
mock_get_account.return_value = None
# Act & Assert
with app.test_request_context(
"/forgot-password/resets",
method="POST",
json={"token": "token", "new_password": "NewPass123!", "password_confirm": "NewPass123!"},
with (
_bind_database_session(sqlite_session),
app.test_request_context(
"/forgot-password/resets",
method="POST",
json={"token": "token", "new_password": "NewPass123!", "password_confirm": "NewPass123!"},
),
):
api = ForgotPasswordResetApi()
with pytest.raises(AccountNotFound):
@@ -1,3 +1,9 @@
"""RAG pipeline workflow controller serialization tests.
Handlers that own transactions run against real SQLite sessions so response
DTOs must be materialized before those transaction contexts close.
"""
from __future__ import annotations
from datetime import datetime
@@ -7,6 +13,8 @@ from unittest.mock import PropertyMock, patch
import pytest
from flask import Flask
from sqlalchemy.engine import Engine
from sqlalchemy.orm import Session
from controllers.console.datasets.rag_pipeline import rag_pipeline_workflow as module
from models.account import Account, TenantAccountRole
@@ -73,90 +81,80 @@ def test_draft_rag_pipeline_workflow_get_serializes_response_model(monkeypatch:
def test_published_rag_pipeline_workflows_serialize_items_before_session_closes(
app, monkeypatch: pytest.MonkeyPatch
app, monkeypatch: pytest.MonkeyPatch, sqlite_engine: Engine
) -> None:
api = module.PublishedAllRagPipelineApi()
handler = unwrap_all(api.get)
session_state = {"open": False}
class _SessionContext:
def __enter__(self):
session_state["open"] = True
return object()
def __exit__(self, exc_type, exc, tb):
session_state["open"] = False
return False
class _SessionMaker:
def begin(self):
return _SessionContext()
session_state: dict[str, Session] = {}
base_workflow = _make_workflow()
class _Workflow:
def __getattr__(self, name: str):
assert session_state["open"] is True
assert session_state["session"].in_transaction() is True
return getattr(base_workflow, name)
monkeypatch.setattr(module, "db", SimpleNamespace(engine=object(), session=lambda: object()))
monkeypatch.setattr(module, "sessionmaker", lambda *_args, **_kwargs: _SessionMaker())
def _get_all_published_workflow(**kwargs):
session_state["session"] = kwargs["session"]
return [_Workflow()], False
monkeypatch.setattr(
module,
"RagPipelineService",
lambda *_args, **_kwargs: SimpleNamespace(get_all_published_workflow=lambda **_kwargs: ([_Workflow()], False)),
lambda *_args, **_kwargs: SimpleNamespace(get_all_published_workflow=_get_all_published_workflow),
)
with app.test_request_context(
"/rag/pipelines/pipeline-1/workflows",
method="GET",
query_string={"page": 1, "limit": 10, "user_id": "", "named_only": "false"},
):
response = handler(api, _account(), pipeline=_pipeline())
with Session(sqlite_engine) as request_session:
monkeypatch.setattr(module, "db", SimpleNamespace(engine=sqlite_engine, session=lambda: request_session))
with app.test_request_context(
"/rag/pipelines/pipeline-1/workflows",
method="GET",
query_string={"page": 1, "limit": 10, "user_id": "", "named_only": "false"},
):
response = handler(api, _account(), pipeline=_pipeline())
assert session_state["session"].in_transaction() is False
assert response["items"][0]["id"] == "workflow-1"
assert response["page"] == 1
assert response["limit"] == 10
assert response["has_more"] is False
def test_rag_pipeline_workflow_patch_serializes_response_model(app: Flask, monkeypatch: pytest.MonkeyPatch) -> None:
def test_rag_pipeline_workflow_patch_serializes_response_model(
app: Flask, monkeypatch: pytest.MonkeyPatch, sqlite_engine: Engine
) -> None:
workflow = _make_workflow(marked_name="Updated release")
captured_session: dict[str, Session] = {}
class _SessionContext:
def __enter__(self):
return object()
def _update_workflow(**kwargs):
captured_session["session"] = kwargs["session"]
assert kwargs["session"].in_transaction() is True
return workflow
def __exit__(self, exc_type, exc, tb):
return False
class _SessionMaker:
def begin(self):
return _SessionContext()
monkeypatch.setattr(module, "db", SimpleNamespace(engine=object(), session=lambda: object()))
monkeypatch.setattr(module, "sessionmaker", lambda *_args, **_kwargs: _SessionMaker())
monkeypatch.setattr(
module,
"RagPipelineService",
lambda *_args, **_kwargs: SimpleNamespace(update_workflow=lambda **_kwargs: workflow),
lambda *_args, **_kwargs: SimpleNamespace(update_workflow=_update_workflow),
)
payload: dict[str, object] = {"marked_name": "Updated release"}
api = module.RagPipelineByIdApi()
handler = unwrap_all(api.patch)
with (
app.test_request_context("/rag/pipelines/pipeline-1/workflows/workflow-1", method="PATCH", json=payload),
patch.object(type(module.console_ns), "payload", new_callable=PropertyMock, return_value=payload),
):
response = handler(
api,
_account(),
pipeline=_pipeline(),
workflow_id="workflow-1",
)
with Session(sqlite_engine) as request_session:
monkeypatch.setattr(module, "db", SimpleNamespace(engine=sqlite_engine, session=lambda: request_session))
with (
app.test_request_context("/rag/pipelines/pipeline-1/workflows/workflow-1", method="PATCH", json=payload),
patch.object(type(module.console_ns), "payload", new_callable=PropertyMock, return_value=payload),
):
response = handler(
api,
_account(),
pipeline=_pipeline(),
workflow_id="workflow-1",
)
assert captured_session["session"].in_transaction() is False
assert response["id"] == "workflow-1"
assert response["marked_name"] == "Updated release"
assert response["hash"] == "hash-1"
@@ -8,6 +8,8 @@ from unittest.mock import Mock
import pytest
from flask import Flask
from sqlalchemy.engine import Engine
from sqlalchemy.orm import Session, sessionmaker
from werkzeug.exceptions import HTTPException, NotFound
from controllers.console.snippets import snippet_workflow as snippet_workflow_module
@@ -36,7 +38,9 @@ def _snippet(**overrides) -> CustomizedSnippet:
@pytest.fixture(autouse=True)
def _patch_snippet_service_factory(monkeypatch: pytest.MonkeyPatch) -> None:
def _patch_snippet_service_factory(monkeypatch: pytest.MonkeyPatch, sqlite_engine: Engine) -> None:
snippet_session_maker = sessionmaker(bind=sqlite_engine, expire_on_commit=False)
def factory():
try:
return snippet_workflow_module.SnippetService(snippet_workflow_module._snippet_session_maker())
@@ -44,7 +48,7 @@ def _patch_snippet_service_factory(monkeypatch: pytest.MonkeyPatch) -> None:
return snippet_workflow_module.SnippetService()
monkeypatch.setattr(snippet_workflow_module, "_snippet_service", factory)
monkeypatch.setattr(snippet_workflow_module, "_snippet_session_maker", Mock(return_value=Mock()))
monkeypatch.setattr(snippet_workflow_module, "_snippet_session_maker", lambda: snippet_session_maker)
def test_get_snippet_requires_snippet_id(app):
@@ -150,28 +154,28 @@ def test_published_workflow_get_returns_none_when_not_published(app) -> None:
assert handler(api, snippet=SimpleNamespace(id="snippet-1", is_published=False)) is None
def test_published_workflow_post_returns_400_when_publish_fails(app: Flask, monkeypatch: pytest.MonkeyPatch) -> None:
@pytest.mark.parametrize("sqlite_session", [(CustomizedSnippet,)], indirect=True)
def test_published_workflow_post_returns_400_when_publish_fails(
app: Flask,
monkeypatch: pytest.MonkeyPatch,
sqlite_engine: Engine,
sqlite_session: Session,
) -> None:
user = _account("account-1")
snippet = _snippet()
merged_snippet = _snippet()
session = SimpleNamespace(merge=Mock(return_value=merged_snippet), commit=Mock())
sqlite_session.add(snippet)
sqlite_session.commit()
class SessionContext:
def __init__(self, engine):
self.engine = engine
def fail_publish(*, session: Session, snippet: CustomizedSnippet, account: Account):
snippet.name = "Uncommitted name"
session.add(snippet)
raise ValueError("No valid workflow found.")
def __enter__(self):
return session
def __exit__(self, exc_type, exc, tb):
return False
monkeypatch.setattr(snippet_workflow_module, "Session", SessionContext)
monkeypatch.setattr(snippet_workflow_module, "db", SimpleNamespace(engine=object()))
monkeypatch.setattr(snippet_workflow_module, "db", SimpleNamespace(engine=sqlite_engine))
monkeypatch.setattr(
snippet_workflow_module,
"SnippetService",
lambda: SimpleNamespace(publish_workflow=Mock(side_effect=ValueError("No valid workflow found."))),
lambda: SimpleNamespace(publish_workflow=Mock(side_effect=fail_publish)),
)
api = snippet_workflow_module.SnippetPublishedWorkflowApi()
@@ -182,7 +186,8 @@ def test_published_workflow_post_returns_400_when_publish_fails(app: Flask, monk
assert status_code == 400
assert response == {"message": "No valid workflow found."}
session.commit.assert_not_called()
sqlite_session.refresh(snippet)
assert snippet.name == "Snippet"
def test_default_block_configs_delegates_to_service(app: Flask, monkeypatch: pytest.MonkeyPatch) -> None:
@@ -203,7 +208,11 @@ def test_default_block_configs_delegates_to_service(app: Flask, monkeypatch: pyt
get_default_block_configs.assert_called_once()
def test_list_published_snippet_workflows_includes_input_fields(app: Flask, monkeypatch: pytest.MonkeyPatch) -> None:
def test_list_published_snippet_workflows_includes_input_fields(
app: Flask,
monkeypatch: pytest.MonkeyPatch,
sqlite_engine: Engine,
) -> None:
workflow = SimpleNamespace(
id="workflow-1",
graph_dict={"nodes": [], "edges": []},
@@ -224,18 +233,7 @@ def test_list_published_snippet_workflows_includes_input_fields(app: Flask, monk
input_fields = [{"variable": "query", "type": "text"}]
snippet = _snippet(input_fields=json.dumps(input_fields))
class SessionContext:
def __init__(self, engine):
self.engine = engine
def __enter__(self):
return Mock()
def __exit__(self, exc_type, exc, tb):
return False
monkeypatch.setattr(snippet_workflow_module, "Session", SessionContext)
monkeypatch.setattr(snippet_workflow_module, "db", SimpleNamespace(engine=object()))
monkeypatch.setattr(snippet_workflow_module, "db", SimpleNamespace(engine=sqlite_engine))
monkeypatch.setattr(
snippet_workflow_module,
"SnippetService",
@@ -364,8 +362,11 @@ def test_restore_published_snippet_workflow_to_draft_returns_400_for_invalid_gra
assert exc.value.description == "invalid snippet workflow graph"
@pytest.mark.parametrize("sqlite_session", [(CustomizedSnippet,)], indirect=True)
def test_update_published_snippet_workflow_returns_updated_workflow(
app: Flask, monkeypatch: pytest.MonkeyPatch
app: Flask,
monkeypatch: pytest.MonkeyPatch,
sqlite_session: Session,
) -> None:
workflow = SimpleNamespace(
id="workflow-1",
@@ -387,21 +388,15 @@ def test_update_published_snippet_workflow_returns_updated_workflow(
user = _account("account-1")
input_fields = [{"variable": "query", "type": "text"}]
snippet = _snippet(input_fields=json.dumps(input_fields))
session = SimpleNamespace()
update_workflow = Mock(return_value=workflow)
sqlite_session.add(snippet)
sqlite_session.commit()
class TransactionContext:
def __enter__(self):
return session
def update_persisted_snippet(*, session: Session, snippet: CustomizedSnippet, **_kwargs):
merged_snippet = session.merge(snippet)
merged_snippet.description = "Updated in transaction"
return workflow
def __exit__(self, exc_type, exc, tb):
return False
class SessionMaker:
def begin(self):
return TransactionContext()
monkeypatch.setattr(snippet_workflow_module, "_snippet_session_maker", Mock(return_value=SessionMaker()))
update_workflow = Mock(side_effect=update_persisted_snippet)
monkeypatch.setattr(
snippet_workflow_module,
"SnippetService",
@@ -418,16 +413,18 @@ def test_update_published_snippet_workflow_returns_updated_workflow(
):
response = handler(api, user, snippet, workflow_id="workflow-1")
update_workflow.assert_called_once_with(
session=session,
snippet=snippet,
workflow_id="workflow-1",
account=user,
data={"marked_name": "v1", "marked_comment": "first version"},
)
update_workflow.assert_called_once()
update_call = update_workflow.call_args.kwargs
assert isinstance(update_call["session"], Session)
assert update_call["snippet"] is snippet
assert update_call["workflow_id"] == "workflow-1"
assert update_call["account"] is user
assert update_call["data"] == {"marked_name": "v1", "marked_comment": "first version"}
assert response["marked_name"] == "v1"
assert response["marked_comment"] == "first version"
assert response["input_fields"] == input_fields
sqlite_session.refresh(snippet)
assert snippet.description == "Updated in transaction"
def test_update_published_snippet_workflow_returns_400_when_no_fields(app: Flask) -> None:
@@ -441,26 +438,25 @@ def test_update_published_snippet_workflow_returns_400_when_no_fields(app: Flask
assert response == {"message": "No valid fields to update"}
def test_update_published_snippet_workflow_raises_not_found(app: Flask, monkeypatch: pytest.MonkeyPatch) -> None:
@pytest.mark.parametrize("sqlite_session", [(CustomizedSnippet,)], indirect=True)
def test_update_published_snippet_workflow_raises_not_found(
app: Flask,
monkeypatch: pytest.MonkeyPatch,
sqlite_session: Session,
) -> None:
user = _account("account-1")
snippet = _snippet()
sqlite_session.add(snippet)
sqlite_session.commit()
class TransactionContext:
def __enter__(self):
return SimpleNamespace()
def update_missing_workflow(*, session: Session, snippet: CustomizedSnippet, **_kwargs):
merged_snippet = session.merge(snippet)
merged_snippet.name = "Rolled back name"
def __exit__(self, exc_type, exc, tb):
return False
class SessionMaker:
def begin(self):
return TransactionContext()
monkeypatch.setattr(snippet_workflow_module, "_snippet_session_maker", Mock(return_value=SessionMaker()))
monkeypatch.setattr(
snippet_workflow_module,
"SnippetService",
lambda: SimpleNamespace(update_workflow=Mock(return_value=None)),
lambda: SimpleNamespace(update_workflow=Mock(side_effect=update_missing_workflow)),
)
api = snippet_workflow_module.SnippetWorkflowByIdApi()
@@ -474,6 +470,9 @@ def test_update_published_snippet_workflow_raises_not_found(app: Flask, monkeypa
with pytest.raises(NotFound, match="Workflow not found"):
handler(api, user, snippet, workflow_id="missing-workflow")
sqlite_session.refresh(snippet)
assert snippet.name == "Snippet"
def test_workflow_run_detail_raises_not_found_when_run_missing(app: Flask, monkeypatch: pytest.MonkeyPatch) -> None:
snippet = _snippet()
@@ -1,15 +1,29 @@
from collections.abc import Iterator
from inspect import unwrap
from types import SimpleNamespace
from unittest.mock import Mock
import pytest
from flask import Flask
from sqlalchemy import event, select
from sqlalchemy.engine import Engine
from sqlalchemy.orm import Session, scoped_session, sessionmaker
from controllers.console.snippets import snippet_workflow_draft_variable as module
from core.workflow.variable_prefixes import CONVERSATION_VARIABLE_NODE_ID, SYSTEM_VARIABLE_NODE_ID
from graphon.variables import StringSegment
from models.account import Account, AccountStatus
from models.workflow import WorkflowDraftVariable, WorkflowDraftVariableFile
from services.workflow_draft_variable_service import WorkflowDraftVariableList
pytestmark = [
pytest.mark.usefixtures("sqlite_session"),
pytest.mark.parametrize(
"sqlite_session",
[(WorkflowDraftVariable, WorkflowDraftVariableFile)],
indirect=True,
),
]
def _make_account() -> Account:
account = Account(
@@ -21,8 +35,31 @@ def _make_account() -> Account:
return account
def _make_node_variable(
variable_id: str,
*,
app_id: str = "snippet-1",
user_id: str = "user-1",
node_id: str = "llm-1",
name: str | None = None,
node_execution_id: str | None = "execution-1",
) -> WorkflowDraftVariable:
"""Create a valid node variable for persisted controller tests."""
variable = WorkflowDraftVariable.new_node_variable(
app_id=app_id,
user_id=user_id,
node_id=node_id,
name=name or variable_id,
value=StringSegment(value=f"value-{variable_id}"),
node_execution_id=node_execution_id or "execution-1",
)
variable.id = variable_id
variable.node_execution_id = node_execution_id
return variable
@pytest.fixture(autouse=True)
def _patch_snippet_service_factory(monkeypatch: pytest.MonkeyPatch):
def _patch_snippet_service_factory(monkeypatch: pytest.MonkeyPatch) -> None:
def factory():
service_factory = module.SnippetService
if isinstance(service_factory, type):
@@ -33,33 +70,69 @@ def _patch_snippet_service_factory(monkeypatch: pytest.MonkeyPatch):
@pytest.fixture
def app():
def app() -> Flask:
app = Flask("test_snippet_workflow_draft_variable")
app.config["TESTING"] = True
return app
def test_ensure_snippet_draft_variable_row_allowed_rejects_system_variable():
variable = SimpleNamespace(node_id=SYSTEM_VARIABLE_NODE_ID)
@pytest.fixture
def controller_sessions(
monkeypatch: pytest.MonkeyPatch,
sqlite_engine: Engine,
) -> Iterator[scoped_session[Session]]:
"""Bind both controller session styles to the isolated SQLite engine."""
sessions = scoped_session(sessionmaker(bind=sqlite_engine, expire_on_commit=False))
monkeypatch.setattr(module, "db", SimpleNamespace(engine=sqlite_engine, session=sessions))
try:
yield sessions
finally:
sessions.remove()
def _persist_variables(sqlite_session: Session, *variables: WorkflowDraftVariable) -> None:
sqlite_session.add_all(variables)
sqlite_session.commit()
def _variable_ids(sqlite_engine: Engine) -> set[str]:
with Session(sqlite_engine) as session:
return set(session.scalars(select(WorkflowDraftVariable.id)))
def test_ensure_snippet_draft_variable_row_allowed_rejects_system_variable() -> None:
variable = WorkflowDraftVariable.new_sys_variable(
app_id="snippet-1",
user_id="user-1",
name="query",
value=StringSegment(value="query"),
node_execution_id="execution-1",
editable=True,
)
with pytest.raises(module.NotFoundError, match="variable not found"):
module._ensure_snippet_draft_variable_row_allowed(variable=variable, variable_id="var-1")
def test_ensure_snippet_draft_variable_row_allowed_rejects_conversation_variable():
variable = SimpleNamespace(node_id=CONVERSATION_VARIABLE_NODE_ID)
def test_ensure_snippet_draft_variable_row_allowed_rejects_conversation_variable() -> None:
variable = WorkflowDraftVariable.new_conversation_variable(
app_id="snippet-1",
user_id="user-1",
name="conversation-name",
value=StringSegment(value="value"),
)
with pytest.raises(module.NotFoundError, match="variable not found"):
module._ensure_snippet_draft_variable_row_allowed(variable=variable, variable_id="var-1")
def test_ensure_snippet_draft_variable_row_allowed_accepts_canvas_node_variable():
variable = SimpleNamespace(node_id="llm-1")
def test_ensure_snippet_draft_variable_row_allowed_accepts_canvas_node_variable() -> None:
variable = _make_node_variable("var-1")
module._ensure_snippet_draft_variable_row_allowed(variable=variable, variable_id="var-1")
def test_conversation_variables_returns_empty_list(app: Flask):
def test_conversation_variables_returns_empty_list(app: Flask) -> None:
api = module.SnippetConversationVariableCollectionApi()
handler = unwrap(api.get)
@@ -69,7 +142,7 @@ def test_conversation_variables_returns_empty_list(app: Flask):
assert result == WorkflowDraftVariableList(variables=[])
def test_system_variables_returns_empty_list(app: Flask):
def test_system_variables_returns_empty_list(app: Flask) -> None:
api = module.SnippetSystemVariableCollectionApi()
handler = unwrap(api.get)
@@ -79,12 +152,17 @@ def test_system_variables_returns_empty_list(app: Flask):
assert result == WorkflowDraftVariableList(variables=[])
def test_delete_variable_collection_deletes_current_user_variables(app: Flask, monkeypatch: pytest.MonkeyPatch):
draft_var_service = SimpleNamespace(delete_user_workflow_variables=Mock())
monkeypatch.setattr(module, "WorkflowDraftVariableService", Mock(return_value=draft_var_service))
db_session = Mock()
db_session.return_value = SimpleNamespace()
monkeypatch.setattr(module.db, "session", db_session)
def test_delete_variable_collection_deletes_only_current_user_variables(
app: Flask,
sqlite_session: Session,
sqlite_engine: Engine,
controller_sessions: scoped_session[Session],
) -> None:
matching = _make_node_variable("matching", name="matching")
matching_second = _make_node_variable("matching-second", node_id="tool-1", name="matching-second")
other_user = _make_node_variable("other-user", user_id="user-2", name="other-user")
other_snippet = _make_node_variable("other-snippet", app_id="snippet-2", name="other-snippet")
_persist_variables(sqlite_session, matching, matching_second, other_user, other_snippet)
api = module.SnippetWorkflowVariableCollectionApi()
handler = unwrap(api.delete)
@@ -92,11 +170,14 @@ def test_delete_variable_collection_deletes_current_user_variables(app: Flask, m
response = handler(api, _make_account(), snippet=SimpleNamespace(id="snippet-1"))
assert response.status_code == 204
draft_var_service.delete_user_workflow_variables.assert_called_once_with("snippet-1", user_id="user-1")
db_session.commit.assert_called_once()
assert _variable_ids(sqlite_engine) == {other_user.id, other_snippet.id}
assert not controller_sessions().in_transaction()
def test_variable_collection_get_raises_when_draft_workflow_missing(app: Flask, monkeypatch: pytest.MonkeyPatch):
def test_variable_collection_get_raises_when_draft_workflow_missing(
app: Flask,
monkeypatch: pytest.MonkeyPatch,
) -> None:
monkeypatch.setattr(
module,
"SnippetService",
@@ -111,47 +192,37 @@ def test_variable_collection_get_raises_when_draft_workflow_missing(app: Flask,
handler(api, _make_account(), snippet=SimpleNamespace(id="snippet-1"))
def test_node_variable_collection_get_lists_node_variables(app: Flask, monkeypatch: pytest.MonkeyPatch):
variables = WorkflowDraftVariableList(variables=[SimpleNamespace(id="var-1")])
list_node_variables = Mock(return_value=variables)
class SessionContext:
def __init__(self, bind, expire_on_commit=False):
self.bind = bind
self.expire_on_commit = expire_on_commit
def __enter__(self):
return SimpleNamespace()
def __exit__(self, exc_type, exc, tb):
return False
monkeypatch.setattr(module, "Session", SessionContext)
monkeypatch.setattr(module, "db", SimpleNamespace(engine=object()))
monkeypatch.setattr(
module,
"WorkflowDraftVariableService",
Mock(return_value=SimpleNamespace(list_node_variables=list_node_variables)),
)
def test_node_variable_collection_get_lists_persisted_node_variables(
app: Flask,
sqlite_session: Session,
controller_sessions: scoped_session[Session],
) -> None:
matching = _make_node_variable("matching", name="matching")
other_node = _make_node_variable("other-node", node_id="tool-1", name="other-node")
other_user = _make_node_variable("other-user", user_id="user-2", name="other-user")
other_snippet = _make_node_variable("other-snippet", app_id="snippet-2", name="other-snippet")
_persist_variables(sqlite_session, matching, other_node, other_user, other_snippet)
api = module.SnippetNodeVariableCollectionApi()
handler = unwrap(api.get)
with app.test_request_context("/"):
result = handler(api, _make_account(), snippet=SimpleNamespace(id="snippet-1"), node_id="llm-1")
assert result is variables
list_node_variables.assert_called_once_with("snippet-1", "llm-1", user_id="user-1")
assert [variable.id for variable in result.variables] == [matching.id]
assert controller_sessions().get_bind() is not None
def test_node_variable_collection_delete_deletes_node_variables(app: Flask, monkeypatch: pytest.MonkeyPatch):
delete_node_variables = Mock()
draft_var_service = SimpleNamespace(delete_node_variables=delete_node_variables)
monkeypatch.setattr(module, "WorkflowDraftVariableService", Mock(return_value=draft_var_service))
db_session = Mock()
db_session.return_value = SimpleNamespace()
monkeypatch.setattr(module.db, "session", db_session)
def test_node_variable_collection_delete_deletes_only_requested_node_variables(
app: Flask,
sqlite_session: Session,
sqlite_engine: Engine,
controller_sessions: scoped_session[Session],
) -> None:
matching = _make_node_variable("matching", name="matching")
matching_second = _make_node_variable("matching-second", name="matching-second")
other_node = _make_node_variable("other-node", node_id="tool-1", name="other-node")
other_user = _make_node_variable("other-user", user_id="user-2", name="other-user")
_persist_variables(sqlite_session, matching, matching_second, other_node, other_user)
api = module.SnippetNodeVariableCollectionApi()
handler = unwrap(api.delete)
@@ -159,83 +230,102 @@ def test_node_variable_collection_delete_deletes_node_variables(app: Flask, monk
response = handler(api, _make_account(), snippet=SimpleNamespace(id="snippet-1"), node_id="llm-1")
assert response.status_code == 204
delete_node_variables.assert_called_once_with("snippet-1", "llm-1", user_id="user-1")
db_session.commit.assert_called_once()
assert _variable_ids(sqlite_engine) == {other_node.id, other_user.id}
assert not controller_sessions().in_transaction()
def test_variable_patch_returns_variable_when_no_changes(app: Flask, monkeypatch: pytest.MonkeyPatch):
variable = SimpleNamespace(id="var-1", app_id="snippet-1", user_id="user-1", node_id="llm-1")
draft_var_service = SimpleNamespace(get_variable=Mock(return_value=variable), update_variable=Mock())
db_session = Mock()
db_session.return_value = SimpleNamespace()
monkeypatch.setattr(module.db, "session", db_session)
monkeypatch.setattr(module, "WorkflowDraftVariableService", Mock(return_value=draft_var_service))
def test_variable_patch_returns_persisted_variable_without_committing_when_no_changes(
app: Flask,
sqlite_session: Session,
controller_sessions: scoped_session[Session],
) -> None:
variable = _make_node_variable("var-1")
_persist_variables(sqlite_session, variable)
session = controller_sessions()
commits: list[bool] = []
def record_commit(_session: Session) -> None:
commits.append(True)
event.listen(session, "after_commit", record_commit)
api = module.SnippetVariableApi()
handler = unwrap(api.patch)
try:
with app.test_request_context("/", method="PATCH", json={}):
result = handler(
api,
_make_account(),
snippet=SimpleNamespace(id="snippet-1", tenant_id="tenant-1"),
variable_id="var-1",
)
finally:
event.remove(session, "after_commit", record_commit)
with app.test_request_context("/", method="PATCH", json={}):
result = handler(
api,
_make_account(),
snippet=SimpleNamespace(id="snippet-1", tenant_id="tenant-1"),
variable_id="var-1",
)
assert result is variable
draft_var_service.update_variable.assert_not_called()
db_session.commit.assert_not_called()
assert result.id == variable.id
assert result.app_id == "snippet-1"
assert commits == []
assert session.in_transaction()
def test_variable_delete_deletes_variable(app: Flask, monkeypatch: pytest.MonkeyPatch):
variable = SimpleNamespace(id="var-1", app_id="snippet-1", user_id="user-1", node_id="llm-1")
delete_variable = Mock()
draft_var_service = SimpleNamespace(get_variable=Mock(return_value=variable), delete_variable=delete_variable)
db_session = Mock()
db_session.return_value = SimpleNamespace()
monkeypatch.setattr(module.db, "session", db_session)
monkeypatch.setattr(module, "WorkflowDraftVariableService", Mock(return_value=draft_var_service))
def test_variable_delete_deletes_persisted_variable(
app: Flask,
sqlite_session: Session,
sqlite_engine: Engine,
controller_sessions: scoped_session[Session],
) -> None:
variable = _make_node_variable("var-1")
retained = _make_node_variable("var-2", name="retained")
_persist_variables(sqlite_session, variable, retained)
api = module.SnippetVariableApi()
handler = unwrap(api.delete)
with app.test_request_context("/", method="DELETE"):
response = handler(api, _make_account(), snippet=SimpleNamespace(id="snippet-1"), variable_id="var-1")
response = handler(
api,
_make_account(),
snippet=SimpleNamespace(id="snippet-1"),
variable_id=variable.id,
)
assert response.status_code == 204
delete_variable.assert_called_once_with(variable)
db_session.commit.assert_called_once()
assert _variable_ids(sqlite_engine) == {retained.id}
assert not controller_sessions().in_transaction()
def test_variable_reset_returns_no_content_when_reset_result_is_none(app: Flask, monkeypatch: pytest.MonkeyPatch):
variable = SimpleNamespace(id="var-1", app_id="snippet-1", user_id="user-1", node_id="llm-1")
draft_workflow = SimpleNamespace(id="workflow-1")
draft_var_service = SimpleNamespace(
get_variable=Mock(return_value=variable),
reset_variable=Mock(return_value=None),
)
db_session = Mock()
db_session.return_value = SimpleNamespace()
monkeypatch.setattr(module.db, "session", db_session)
monkeypatch.setattr(module, "WorkflowDraftVariableService", Mock(return_value=draft_var_service))
def test_variable_reset_deletes_variable_without_node_execution(
app: Flask,
monkeypatch: pytest.MonkeyPatch,
sqlite_session: Session,
sqlite_engine: Engine,
controller_sessions: scoped_session[Session],
) -> None:
variable = _make_node_variable("var-1", node_execution_id=None)
_persist_variables(sqlite_session, variable)
monkeypatch.setattr(
module,
"SnippetService",
Mock(return_value=SimpleNamespace(get_draft_workflow=Mock(return_value=draft_workflow))),
Mock(return_value=SimpleNamespace(get_draft_workflow=Mock(return_value=SimpleNamespace(id="workflow-1")))),
)
api = module.SnippetVariableResetApi()
handler = unwrap(api.put)
with app.test_request_context("/", method="PUT"):
response = handler(api, _make_account(), snippet=SimpleNamespace(id="snippet-1"), variable_id="var-1")
response = handler(
api,
_make_account(),
snippet=SimpleNamespace(id="snippet-1"),
variable_id=variable.id,
)
assert response.status_code == 204
draft_var_service.reset_variable.assert_called_once_with(draft_workflow, variable)
db_session.commit.assert_called_once()
assert _variable_ids(sqlite_engine) == set()
assert not controller_sessions().in_transaction()
def test_environment_variables_returns_workflow_environment_variables(app: Flask, monkeypatch: pytest.MonkeyPatch):
def test_environment_variables_returns_workflow_environment_variables(
app: Flask,
monkeypatch: pytest.MonkeyPatch,
) -> None:
env_var = SimpleNamespace(
id="env-1",
name="API_KEY",
@@ -1,9 +1,11 @@
from collections.abc import Iterator
from types import SimpleNamespace
from unittest.mock import MagicMock, PropertyMock, patch
from unittest.mock import PropertyMock, patch
import pytest
from flask import Flask
from sqlalchemy.orm import Session
from sqlalchemy import Engine
from sqlalchemy.orm import Session, scoped_session, sessionmaker
from werkzeug.exceptions import Forbidden
import controllers.console.tag.tags as module
@@ -16,15 +18,12 @@ from controllers.console.tag.tags import (
)
from models import Account
from models.account import AccountStatus, TenantAccountRole
from models.base import TypeBase
from models.enums import TagType
from models.model import Tag
from services.tag_service import UpdateTagPayload
class SessionMatcher:
def __eq__(self, other):
return isinstance(other, Session)
def unwrap(func):
"""
Recursively unwrap decorated functions.
@@ -41,6 +40,26 @@ def app():
return app
@pytest.fixture(autouse=True)
def sqlite_db_session(
sqlite_engine: Engine,
monkeypatch: pytest.MonkeyPatch,
) -> Iterator[scoped_session[Session]]:
TypeBase.metadata.create_all(sqlite_engine, tables=[TypeBase.metadata.tables[Tag.__tablename__]])
session_registry = scoped_session(sessionmaker(bind=sqlite_engine, expire_on_commit=False))
monkeypatch.setattr(module.db, "session", session_registry)
try:
yield session_registry
finally:
session_registry.remove()
def _assert_sqlite_session(session: object, sqlite_engine: Engine) -> None:
assert isinstance(session, Session)
assert session.get_bind() is sqlite_engine
assert session.is_active
@pytest.fixture
def admin_user():
account = Account(
@@ -66,11 +85,16 @@ def readonly_user():
@pytest.fixture
def tag():
tag = MagicMock()
def tag(sqlite_db_session: scoped_session[Session]):
tag = Tag(
tenant_id="tenant-1",
name="test-tag",
type=TagType.KNOWLEDGE,
created_by="user-1",
)
tag.id = "tag-1"
tag.name = "test-tag"
tag.type = TagType.KNOWLEDGE
sqlite_db_session.add(tag)
sqlite_db_session.commit()
return tag
@@ -111,7 +135,7 @@ class TestTagListApi:
assert status == 200
assert result == [{"id": "1", "name": "tag", "type": "knowledge", "binding_count": "1"}]
def test_get_snippet_tags(self, app: Flask):
def test_get_snippet_tags(self, app: Flask, sqlite_engine: Engine):
api = TagListApi()
method = unwrap(api.get)
@@ -131,7 +155,9 @@ class TestTagListApi:
):
result, status = method(api, "tenant-1")
get_tags_mock.assert_called_once_with("snippet", "tenant-1", None, session=SessionMatcher())
get_tags_mock.assert_called_once()
assert get_tags_mock.call_args.args == ("snippet", "tenant-1", None)
_assert_sqlite_session(get_tags_mock.call_args.kwargs["session"], sqlite_engine)
assert status == 200
assert result == [{"id": "1", "name": "snippet-tag", "type": "snippet", "binding_count": "1"}]
@@ -200,7 +226,7 @@ class TestTagListApi:
class TestTagUpdateDeleteApi:
def test_patch_success(self, app: Flask, admin_user, tag, payload_patch):
def test_patch_success(self, app: Flask, admin_user, tag, payload_patch, sqlite_engine: Engine):
api = TagUpdateDeleteApi()
method = unwrap(api.patch)
@@ -224,7 +250,7 @@ class TestTagUpdateDeleteApi:
update_payload, tag_id, session = update_tags_mock.call_args.args
assert update_payload == UpdateTagPayload(name="updated")
assert tag_id == "tag-1"
assert session == SessionMatcher()
_assert_sqlite_session(session, sqlite_engine)
assert result["binding_count"] == "3"
def test_patch_forbidden(self, app: Flask, readonly_user, payload_patch):
@@ -240,7 +266,7 @@ class TestTagUpdateDeleteApi:
with pytest.raises(Forbidden):
method(api, readonly_user, "tag-1")
def test_delete_success(self, app: Flask, admin_user):
def test_delete_success(self, app: Flask, admin_user, sqlite_engine: Engine):
api = TagUpdateDeleteApi()
method = unwrap(api.delete)
@@ -250,12 +276,30 @@ class TestTagUpdateDeleteApi:
):
result, status = method(api, "tag-1")
delete_mock.assert_called_once_with("tag-1", SessionMatcher())
delete_mock.assert_called_once()
tag_id, session = delete_mock.call_args.args
assert tag_id == "tag-1"
_assert_sqlite_session(session, sqlite_engine)
assert status == 204
def test_delete_snippet_tag_checks_type_in_current_tenant(self, app: Flask, admin_user):
def test_delete_snippet_tag_checks_type_in_current_tenant(
self,
app: Flask,
admin_user,
sqlite_db_session: scoped_session[Session],
sqlite_engine: Engine,
):
api = TagUpdateDeleteApi()
method = unwrap(api.delete)
tag = Tag(
tenant_id="tenant-1",
name="snippet-tag",
type=TagType.SNIPPET,
created_by="user-1",
)
tag.id = "tag-1"
sqlite_db_session.add(tag)
sqlite_db_session.commit()
with (
app.test_request_context("/"),
@@ -264,13 +308,11 @@ class TestTagUpdateDeleteApi:
"controllers.console.tag.tags.current_account_with_tenant",
return_value=(SimpleNamespace(id="user-1"), "tenant-1"),
),
patch.object(module.db.session, "scalar", return_value=TagType.SNIPPET) as scalar_mock,
patch("controllers.console.tag.tags.enforce_rbac_access") as enforce_mock,
patch("controllers.console.tag.tags.TagService.delete_tag") as delete_mock,
):
result, status = method(api, "tag-1")
scalar_mock.assert_called_once()
enforce_mock.assert_called_once_with(
tenant_id="tenant-1",
account_id="user-1",
@@ -278,7 +320,49 @@ class TestTagUpdateDeleteApi:
scene=module.RBACPermission.SNIPPETS_CREATE_AND_MODIFY,
resource_required=False,
)
delete_mock.assert_called_once_with("tag-1", SessionMatcher())
delete_mock.assert_called_once()
tag_id, session = delete_mock.call_args.args
assert tag_id == "tag-1"
_assert_sqlite_session(session, sqlite_engine)
assert result == ""
assert status == 204
def test_delete_does_not_apply_snippet_rbac_to_tag_from_another_tenant(
self,
app: Flask,
admin_user,
sqlite_db_session: scoped_session[Session],
sqlite_engine: Engine,
):
api = TagUpdateDeleteApi()
method = unwrap(api.delete)
tag = Tag(
tenant_id="other-tenant",
name="other-tenant-snippet-tag",
type=TagType.SNIPPET,
created_by="other-user",
)
tag.id = "tag-1"
sqlite_db_session.add(tag)
sqlite_db_session.commit()
with (
app.test_request_context("/"),
patch("controllers.console.tag.tags.dify_config.RBAC_ENABLED", True),
patch(
"controllers.console.tag.tags.current_account_with_tenant",
return_value=(SimpleNamespace(id="user-1"), "tenant-1"),
),
patch("controllers.console.tag.tags.enforce_rbac_access") as enforce_mock,
patch("controllers.console.tag.tags.TagService.delete_tag") as delete_mock,
):
result, status = method(api, "tag-1")
enforce_mock.assert_not_called()
delete_mock.assert_called_once()
tag_id, session = delete_mock.call_args.args
assert tag_id == "tag-1"
_assert_sqlite_session(session, sqlite_engine)
assert result == ""
assert status == 204
@@ -1,27 +1,16 @@
"""Initialization validation tests with real setup-state persistence in SQLite."""
from __future__ import annotations
from types import SimpleNamespace
from unittest.mock import Mock
import pytest
from flask import Flask
from sqlalchemy.orm import Session
from controllers.console import init_validate
from controllers.console.error import AlreadySetupError, InitValidateFailedError
class _SessionStub:
def __init__(self, has_setup: bool):
self._has_setup = has_setup
def __enter__(self):
return self
def __exit__(self, exc_type, exc, tb):
return False
def execute(self, *_args, **_kwargs):
return SimpleNamespace(scalar_one_or_none=lambda: Mock() if self._has_setup else None)
from models.model import DifySetup
def test_get_init_status_finished(monkeypatch: pytest.MonkeyPatch) -> None:
@@ -85,11 +74,15 @@ def test_get_init_validate_status_validated_session(app: Flask, monkeypatch: pyt
assert init_validate.get_init_validate_status() is True
def test_get_init_validate_status_setup_exists(app: Flask, monkeypatch: pytest.MonkeyPatch) -> None:
@pytest.mark.parametrize("sqlite_session", [(DifySetup,)], indirect=True)
def test_get_init_validate_status_setup_exists(
app: Flask, monkeypatch: pytest.MonkeyPatch, sqlite_session: Session
) -> None:
monkeypatch.setattr(init_validate.dify_config, "EDITION", "SELF_HOSTED")
monkeypatch.setenv("INIT_PASSWORD", "expected")
monkeypatch.setattr(init_validate, "Session", lambda *_args, **_kwargs: _SessionStub(True))
monkeypatch.setattr(init_validate, "db", SimpleNamespace(engine=object()))
monkeypatch.setattr(init_validate, "db", SimpleNamespace(engine=sqlite_session.get_bind()))
sqlite_session.add(DifySetup(version="test-version"))
sqlite_session.commit()
app.secret_key = "test-secret"
with app.test_request_context("/console/api/init", method="GET"):
@@ -97,11 +90,13 @@ def test_get_init_validate_status_setup_exists(app: Flask, monkeypatch: pytest.M
assert init_validate.get_init_validate_status() is True
def test_get_init_validate_status_not_validated(app: Flask, monkeypatch: pytest.MonkeyPatch) -> None:
@pytest.mark.parametrize("sqlite_session", [(DifySetup,)], indirect=True)
def test_get_init_validate_status_not_validated(
app: Flask, monkeypatch: pytest.MonkeyPatch, sqlite_session: Session
) -> None:
monkeypatch.setattr(init_validate.dify_config, "EDITION", "SELF_HOSTED")
monkeypatch.setenv("INIT_PASSWORD", "expected")
monkeypatch.setattr(init_validate, "Session", lambda *_args, **_kwargs: _SessionStub(False))
monkeypatch.setattr(init_validate, "db", SimpleNamespace(engine=object()))
monkeypatch.setattr(init_validate, "db", SimpleNamespace(engine=sqlite_session.get_bind()))
app.secret_key = "test-secret"
with app.test_request_context("/console/api/init", method="GET"):
@@ -7,9 +7,11 @@ from unittest.mock import MagicMock
import httpx
import pytest
from flask import Flask, Response
from pydantic import SecretStr
from werkzeug.exceptions import (
BadGateway,
Forbidden,
HTTPException,
NotFound,
RequestEntityTooLarge,
ServiceUnavailable,
@@ -22,15 +24,18 @@ from controllers.console.knowledge_fs_proxy import (
_proxy_request,
_proxy_response,
proxy_knowledge_fs_get,
proxy_knowledge_fs_options,
proxy_knowledge_fs_write,
)
from controllers.console.wraps import RBACPermission
from services.knowledge_fs_proxy import (
KnowledgeFSAccessDeniedError,
KnowledgeFSConfigurationError,
from services.knowledge_fs_operations import (
KnowledgeFSMethod,
KnowledgeFSOperation,
KnowledgeFSResponseKind,
)
from services.knowledge_fs_proxy import (
KnowledgeFSAccessDeniedError,
KnowledgeFSConfigurationError,
KnowledgeFSRouteNotAllowedError,
KnowledgeFSUpstreamResponse,
get_knowledge_fs_operation,
@@ -46,6 +51,7 @@ def _upstream(
response: httpx.Response,
kind: KnowledgeFSResponseKind = "buffered",
*,
error_status_map: tuple[tuple[int, int], ...] = ((401, 502), (403, 403)),
max_response_bytes: int | None = None,
) -> KnowledgeFSUpstreamResponse:
operation = KnowledgeFSOperation(
@@ -55,7 +61,7 @@ def _upstream(
response_kind=kind,
required_scope="knowledge-spaces:read",
rbac_permission=RBACPermission.DATASET_READONLY,
requires_dataset_editor=False,
legacy_role="reader",
max_response_bytes=max_response_bytes
or (64 * 1024 * 1024 if kind == "stream" else 25 * 1024 * 1024 if kind == "binary" else 1024 * 1024),
request_headers=(),
@@ -66,6 +72,7 @@ def _upstream(
"x-session-id",
),
response_media_types=(),
error_status_map=error_status_map,
)
return KnowledgeFSUpstreamResponse(response, kind, operation)
@@ -92,7 +99,7 @@ def _set_current_workspace(
def _bypass_policy_wrappers(monkeypatch: pytest.MonkeyPatch) -> None:
monkeypatch.setattr(
"controllers.console.knowledge_fs_proxy._proxy_knowledge_fs_non_get",
unwrap(_proxy_knowledge_fs_non_get),
lambda method, path: _proxy_request(method, path),
)
@@ -118,10 +125,61 @@ def test_console_blueprint_registers_generic_knowledge_fs_routes() -> None:
"/console/api/knowledge-fs/knowledge-spaces",
method="OPTIONS",
)
assert options_endpoint.endswith("proxy_knowledge_fs_get")
assert options_endpoint.endswith("proxy_knowledge_fs_options")
assert options_values == {"upstream_path": "knowledge-spaces"}
def test_proxy_options_does_not_require_an_authenticated_account(app: Flask) -> None:
with app.test_request_context(
"/console/api/knowledge-fs/knowledge-spaces",
method="OPTIONS",
headers={"Access-Control-Request-Method": "GET"},
):
response = app.make_response(proxy_knowledge_fs_options("knowledge-spaces"))
assert response.status_code == 204
def test_proxy_options_is_hidden_when_knowledge_fs_is_disabled(
app: Flask,
monkeypatch: pytest.MonkeyPatch,
) -> None:
monkeypatch.setattr("controllers.console.knowledge_fs_proxy.dify_config.KNOWLEDGE_FS_ENABLED", False)
with app.test_request_context(
"/console/api/knowledge-fs/knowledge-spaces",
method="OPTIONS",
headers={"Access-Control-Request-Method": "GET"},
):
response = app.make_response(proxy_knowledge_fs_options("knowledge-spaces"))
assert response.status_code == 404
@pytest.mark.parametrize(
("upstream_path", "requested_method"),
[
("unregistered", "GET"),
("knowledge-spaces", "DELETE"),
("knowledge-spaces", ""),
],
)
def test_proxy_options_hides_unregistered_operations(
app: Flask,
upstream_path: str,
requested_method: str,
) -> None:
headers = {"Access-Control-Request-Method": requested_method} if requested_method else None
with app.test_request_context(
f"/console/api/knowledge-fs/{upstream_path}",
method="OPTIONS",
headers=headers,
):
response = app.make_response(proxy_knowledge_fs_options(upstream_path))
assert response.status_code == 404
def test_proxy_is_hidden_when_knowledge_fs_is_disabled(app: Flask, monkeypatch: pytest.MonkeyPatch) -> None:
monkeypatch.setattr("controllers.console.knowledge_fs_proxy.dify_config.KNOWLEDGE_FS_ENABLED", False)
@@ -235,10 +293,8 @@ def test_read_post_applies_knowledge_rate_limit_once(
monkeypatch.setattr("controllers.console.knowledge_fs_proxy.current_account_with_tenant", current_workspace)
monkeypatch.setattr("controllers.console.wraps.current_account_with_tenant", current_workspace)
monkeypatch.setattr(
"services.knowledge_fs_proxy.RBACService.CheckAccess.check",
MagicMock(return_value=True),
)
check_access = MagicMock(return_value=True)
monkeypatch.setattr("services.knowledge_fs_proxy.RBACService.CheckAccess.check", check_access)
monkeypatch.setattr(
"controllers.console.wraps.FeatureService.get_knowledge_rate_limit",
MagicMock(return_value=MagicMock(enabled=True, limit=10)),
@@ -248,14 +304,19 @@ def test_read_post_applies_knowledge_rate_limit_once(
monkeypatch.setattr("controllers.console.wraps.redis_client.zremrangebyscore", MagicMock())
monkeypatch.setattr("controllers.console.wraps.redis_client.zcard", MagicMock(return_value=1))
proxy = MagicMock(return_value=Response(status=200))
monkeypatch.setattr("controllers.console.knowledge_fs_proxy._proxy_request", proxy)
monkeypatch.setattr("controllers.console.knowledge_fs_proxy._proxy_authorized_request", proxy)
with app.test_request_context("/console/api/knowledge-fs/knowledge-spaces", method="POST"):
response = _proxy_knowledge_fs_non_get("POST", "knowledge-spaces")
assert isinstance(response, Response)
zadd.assert_called_once()
proxy.assert_called_once_with("POST", "knowledge-spaces")
proxy.assert_called_once()
authorization = proxy.call_args.args[0]
assert authorization.account_id == "account-1"
assert authorization.tenant_id == "tenant-1"
assert authorization.operation.operation_id == "createKnowledgeSpace"
check_access.assert_called_once()
def test_denied_write_does_not_consume_the_workspace_rate_limit(
@@ -395,6 +456,65 @@ def test_generic_write_forwards_path_raw_body_and_current_tenant(
assert response.get_json()["tenantId"] == "tenant-1"
def test_generic_write_forwards_through_the_authorized_production_path(
app: Flask,
monkeypatch: pytest.MonkeyPatch,
) -> None:
account = MagicMock(id="account-1", is_dataset_editor=True)
def current_workspace() -> tuple[MagicMock, str]:
return account, "tenant-1"
monkeypatch.setattr("controllers.console.knowledge_fs_proxy.current_account_with_tenant", current_workspace)
monkeypatch.setattr("controllers.console.wraps.current_account_with_tenant", current_workspace)
check_access = MagicMock(return_value=True)
monkeypatch.setattr("services.knowledge_fs_proxy.RBACService.CheckAccess.check", check_access)
monkeypatch.setattr(
"controllers.console.wraps.FeatureService.get_knowledge_rate_limit",
MagicMock(return_value=MagicMock(enabled=False)),
)
monkeypatch.setattr(
"services.knowledge_fs_proxy.dify_config.KNOWLEDGE_FS_BASE_URL",
"http://knowledge-fs.test",
raising=False,
)
monkeypatch.setattr(
"services.knowledge_fs_proxy.dify_config.KNOWLEDGE_FS_JWT_SECRET",
SecretStr("production-secret-with-at-least-32-bytes"),
raising=False,
)
upstream_request = MagicMock(
return_value=httpx.Response(
201,
content=b'{"id":"space-1","tenantId":"tenant-1"}',
headers={"Content-Type": "application/json"},
)
)
monkeypatch.setattr("services.knowledge_fs_proxy.ssrf_proxy.make_request", upstream_request)
route = unwrap(proxy_knowledge_fs_write)
body = b'{"idempotencyKey":"create-product-docs","name":"Product docs"}'
with app.test_request_context(
"/console/api/knowledge-fs/knowledge-spaces",
method="POST",
query_string={"source": "console"},
data=body,
content_type="application/json",
headers={"X-Trace-Id": "trace-1"},
):
response = route("knowledge-spaces")
assert isinstance(response, Response)
assert response.status_code == 201
assert response.get_json() == {"id": "space-1", "tenantId": "tenant-1"}
check_access.assert_called_once()
assert upstream_request.call_args.kwargs["method"] == "POST"
assert upstream_request.call_args.kwargs["url"] == "http://knowledge-fs.test/knowledge-spaces"
assert upstream_request.call_args.kwargs["params"] == b"source=console"
assert upstream_request.call_args.kwargs["content"] == body
assert upstream_request.call_args.kwargs["headers"]["x-trace-id"] == "trace-1"
def test_generic_write_forwards_contract_declared_request_headers(
app: Flask,
monkeypatch: pytest.MonkeyPatch,
@@ -565,6 +685,43 @@ def test_resource_authorization_rejection_is_exposed_as_forbidden(
route("knowledge-spaces")
def test_proxy_response_applies_operation_specific_error_status_mapping() -> None:
upstream = httpx.Response(
429,
content=b'{"error":"rate limited"}',
headers={"Content-Type": "application/json"},
)
with pytest.raises(ServiceUnavailable):
_proxy_response(
_upstream(upstream, error_status_map=((429, 503),)),
tenant_id="tenant-1",
contract_response_headers=(),
max_response_bytes=1024 * 1024,
)
assert upstream.is_closed
def test_proxy_response_preserves_nonstandard_mapped_error_status() -> None:
upstream = httpx.Response(
429,
content=b'{"error":"rate limited"}',
headers={"Content-Type": "application/json"},
)
with pytest.raises(HTTPException) as exc_info:
_proxy_response(
_upstream(upstream, error_status_map=((429, 499),)),
tenant_id="tenant-1",
contract_response_headers=(),
max_response_bytes=1024 * 1024,
)
assert exc_info.value.code == 499
assert upstream.is_closed
def test_contract_response_headers_are_deduplicated_case_insensitively() -> None:
upstream = httpx.Response(
200,
@@ -619,6 +776,7 @@ def test_disallowed_non_get_route_is_hidden_as_not_found(
monkeypatch: pytest.MonkeyPatch,
method: KnowledgeFSMethod,
) -> None:
_set_current_workspace(monkeypatch)
route = unwrap(proxy_knowledge_fs_write)
with app.test_request_context("/console/api/knowledge-fs/not-a-route", method=method):
@@ -201,7 +201,7 @@ class TestRbacPermissionRequired:
):
assert protected_view(app_id="app-123") == "ok"
mock_extract.assert_called_once_with("app", {"app_id": "app-123"})
mock_extract.assert_called_once_with(RBACResourceScope.APP, "tenant-1", {"app_id": "app-123"})
mock_owned.assert_called_once_with("tenant-1", "account-1", "app", "app-123")
mock_check.assert_called_once_with(
"tenant-1",
@@ -307,7 +307,7 @@ class TestRbacPermissionRequired:
with app.test_request_context("/"):
request.view_args = {"app_id": "view-app"}
assert _extract_resource_id("app", {"app_id": "path-app"}) == "path-app"
assert _extract_resource_id("app", "tenant-1", {"app_id": "path-app"}) == "path-app"
def test_extract_resource_id_falls_back_to_request_view_args(self):
app = Flask(__name__)
@@ -315,22 +315,59 @@ class TestRbacPermissionRequired:
with app.test_request_context("/"):
request.view_args = {"app_id": "view-app"}
assert _extract_resource_id("app") == "view-app"
assert _extract_resource_id("app", "tenant-1") == "view-app"
def test_extract_resource_id_supports_legacy_route_aliases(self):
app = Flask(__name__)
with app.test_request_context("/apps/app-1/api-keys"):
request.view_args = {"resource_id": "app-1"}
assert _extract_resource_id(RBACResourceScope.APP) == "app-1"
with app.test_request_context("/agent/agent-1/features"):
request.view_args = {"agent_id": "agent-1"}
assert _extract_resource_id(RBACResourceScope.APP) == "agent-1"
assert _extract_resource_id(RBACResourceScope.APP, "tenant-1") == "app-1"
with app.test_request_context("/datasets/dataset-1/api-keys"):
request.view_args = {"resource_id": "dataset-1"}
assert _extract_resource_id(RBACResourceScope.DATASET) == "dataset-1"
assert _extract_resource_id(RBACResourceScope.DATASET, "tenant-1") == "dataset-1"
def test_extract_resource_id_resolves_agent_to_its_authz_app(self):
app = Flask(__name__)
with (
app.test_request_context("/agent/agent-1/chat-messages"),
patch("controllers.common.wraps.AgentRosterService") as mock_service,
):
request.view_args = {"agent_id": "agent-1"}
mock_service.return_value.peek_authz_app_id.return_value = "parent-app-1"
assert _extract_resource_id(RBACResourceScope.APP, "tenant-1") == "parent-app-1"
def test_extract_resource_id_scopes_agent_resolution_to_the_calling_tenant(self):
"""The tenant must reach the resolver, or an Agent id from any tenant resolves."""
app = Flask(__name__)
with (
app.test_request_context("/agent/agent-1/chat-messages"),
patch("controllers.common.wraps.AgentRosterService") as mock_service,
):
request.view_args = {"agent_id": "agent-1"}
mock_service.return_value.peek_authz_app_id.return_value = "parent-app-1"
_extract_resource_id(RBACResourceScope.APP, "tenant-9")
mock_service.return_value.peek_authz_app_id.assert_called_once_with(
tenant_id="tenant-9", agent_id="agent-1"
)
def test_extract_resource_id_keeps_agent_id_when_the_agent_does_not_resolve(self):
app = Flask(__name__)
with (
app.test_request_context("/agent/agent-1/chat-messages"),
patch("controllers.common.wraps.AgentRosterService") as mock_service,
):
request.view_args = {"agent_id": "agent-1"}
mock_service.return_value.peek_authz_app_id.return_value = None
assert _extract_resource_id(RBACResourceScope.APP, "tenant-1") == "agent-1"
def test_legacy_admin_decorator_noops_when_rbac_enabled(self):
@is_admin_or_owner_required
@@ -2,19 +2,75 @@
Unit tests for inner_api plugin decorators
"""
from collections.abc import Iterator
from types import SimpleNamespace
from typing import Any
from unittest.mock import MagicMock, patch
import pytest
from flask import Flask
from pydantic import ValidationError
from sqlalchemy import Engine, event, select
from sqlalchemy.orm import Session, scoped_session, sessionmaker
from controllers.inner_api.plugin import wraps as wraps_module
from controllers.inner_api.plugin.wraps import (
TenantUserPayload,
get_user,
get_user_tenant,
plugin_data,
)
from models.account import Tenant
from models.base import TypeBase
from models.enums import EndUserType
from models.model import DefaultEndUserSessionID, EndUser
@pytest.fixture
def sqlite_plugin_engine(
sqlite_engine: Engine,
monkeypatch: pytest.MonkeyPatch,
) -> Iterator[Engine]:
tables = [TypeBase.metadata.tables[model.__tablename__] for model in (Tenant, EndUser)]
TypeBase.metadata.create_all(sqlite_engine, tables=tables)
session_registry = scoped_session(sessionmaker(bind=sqlite_engine, expire_on_commit=False))
monkeypatch.setattr(
wraps_module,
"db",
SimpleNamespace(engine=sqlite_engine, session=session_registry),
)
try:
yield sqlite_engine
finally:
session_registry.remove()
def _persist_tenant(sqlite_engine: Engine, *, tenant_id: str = "tenant123") -> Tenant:
tenant = Tenant(name=f"Tenant {tenant_id}")
tenant.id = tenant_id
with Session(sqlite_engine) as session, session.begin():
session.add(tenant)
return tenant
def _persist_end_user(
sqlite_engine: Engine,
*,
tenant_id: str = "tenant123",
user_id: str,
session_id: str,
is_anonymous: bool = False,
) -> EndUser:
user = EndUser(
id=user_id,
tenant_id=tenant_id,
type=EndUserType.SERVICE_API,
is_anonymous=is_anonymous,
session_id=session_id,
)
with Session(sqlite_engine) as session, session.begin():
session.add(user)
return user
class TestTenantUserPayload:
@@ -41,185 +97,143 @@ class TestTenantUserPayload:
class TestGetUser:
"""Test get_user function"""
@patch("controllers.inner_api.plugin.wraps.select")
@patch("controllers.inner_api.plugin.wraps.EndUser")
@patch("controllers.inner_api.plugin.wraps.sessionmaker")
@patch("controllers.inner_api.plugin.wraps.db")
def test_should_return_existing_user_by_id(
self, mock_db, mock_sessionmaker, mock_enduser_class, mock_select, app: Flask
):
def test_should_return_existing_user_by_id(self, sqlite_plugin_engine: Engine, app: Flask):
"""Test returning existing user when found by ID"""
# Arrange
mock_user = MagicMock()
mock_user.id = "user123"
mock_session = MagicMock()
mock_sessionmaker.return_value.begin.return_value.__enter__.return_value = mock_session
mock_session.scalar.return_value = mock_user
mock_query = MagicMock()
mock_select.return_value.where.return_value.limit.return_value = mock_query
_persist_end_user(
sqlite_plugin_engine,
user_id="user123",
session_id="existing-session",
)
# Act
with app.app_context():
result = get_user("tenant123", "user123")
# Assert
assert result == mock_user
mock_session.scalar.assert_called_once()
assert result.id == "user123"
assert result.tenant_id == "tenant123"
@patch("controllers.inner_api.plugin.wraps.select")
@patch("controllers.inner_api.plugin.wraps.EndUser")
@patch("controllers.inner_api.plugin.wraps.sessionmaker")
@patch("controllers.inner_api.plugin.wraps.db")
def test_should_not_resolve_non_anonymous_users_across_tenants(
self,
mock_db,
mock_sessionmaker,
mock_enduser_class,
mock_select,
sqlite_plugin_engine: Engine,
app: Flask,
):
"""Test that explicit user IDs remain scoped to the current tenant."""
# Arrange
mock_session = MagicMock()
mock_sessionmaker.return_value.begin.return_value.__enter__.return_value = mock_session
mock_session.scalar.return_value = None
mock_new_user = MagicMock()
mock_new_user.tenant_id = "tenant-current"
mock_enduser_class.return_value = mock_new_user
_persist_end_user(
sqlite_plugin_engine,
tenant_id="tenant-foreign",
user_id="foreign-user-id",
session_id="foreign-session",
)
# Act
with app.app_context():
result = get_user("tenant-current", "foreign-user-id")
# Assert
assert result == mock_new_user
mock_session.get.assert_not_called()
# Non-anonymous miss now tries id, then session_id fallback (see
# #36736); both miss in this tenant → fall through to create.
assert mock_session.scalar.call_count == 2
mock_session.add.assert_called_once_with(mock_new_user)
assert result.id != "foreign-user-id"
assert result.tenant_id == "tenant-current"
assert result.session_id == "foreign-user-id"
with Session(sqlite_plugin_engine) as session:
current_tenant_users = session.scalars(select(EndUser).where(EndUser.tenant_id == "tenant-current")).all()
assert [user.id for user in current_tenant_users] == [result.id]
@patch("controllers.inner_api.plugin.wraps.select")
@patch("controllers.inner_api.plugin.wraps.EndUser")
@patch("controllers.inner_api.plugin.wraps.sessionmaker")
@patch("controllers.inner_api.plugin.wraps.db")
def test_should_return_existing_user_by_session_id_fallback_for_non_anonymous(
self, mock_db, mock_sessionmaker, mock_enduser_class, mock_select, app: Flask
self,
sqlite_plugin_engine: Engine,
app: Flask,
):
"""Non-anonymous user_id misses on EndUser.id but hits on
EndUser.session_id this is the plugin-daemon Reverse Invocation
case where the daemon sends a stable session-derived UUID that
was written into session_id on the first call. See #36736.
"""
# Arrange
mock_user = MagicMock()
mock_session = MagicMock()
mock_sessionmaker.return_value.begin.return_value.__enter__.return_value = mock_session
# First scalar (id lookup) returns None, second (session_id fallback) hits.
mock_session.scalar.side_effect = [None, mock_user]
_persist_end_user(
sqlite_plugin_engine,
user_id="persisted-user-id",
session_id="daemon-session-uuid",
)
# Act
with app.app_context():
result = get_user("tenant123", "daemon-session-uuid")
# Assert
assert result == mock_user
assert mock_session.scalar.call_count == 2
mock_session.add.assert_not_called()
assert result.id == "persisted-user-id"
with Session(sqlite_plugin_engine) as session:
users = session.scalars(select(EndUser)).all()
assert [user.id for user in users] == ["persisted-user-id"]
@patch("controllers.inner_api.plugin.wraps.select")
@patch("controllers.inner_api.plugin.wraps.EndUser")
@patch("controllers.inner_api.plugin.wraps.sessionmaker")
@patch("controllers.inner_api.plugin.wraps.db")
def test_should_return_existing_anonymous_user_by_session_id(
self, mock_db, mock_sessionmaker, mock_enduser_class, mock_select, app: Flask
self,
sqlite_plugin_engine: Engine,
app: Flask,
):
"""Test returning existing anonymous user by session_id"""
# Arrange
mock_user = MagicMock()
mock_user.session_id = "anonymous_session"
mock_session = MagicMock()
mock_sessionmaker.return_value.begin.return_value.__enter__.return_value = mock_session
mock_session.scalar.return_value = mock_user
mock_query = MagicMock()
mock_select.return_value.where.return_value.limit.return_value = mock_query
_persist_end_user(
sqlite_plugin_engine,
user_id="anonymous-user-id",
session_id="anonymous_session",
is_anonymous=True,
)
# Act
with app.app_context():
result = get_user("tenant123", "anonymous_session")
# Assert
assert result == mock_user
assert result.id == "anonymous-user-id"
@patch("controllers.inner_api.plugin.wraps.select")
@patch("controllers.inner_api.plugin.wraps.EndUser")
@patch("controllers.inner_api.plugin.wraps.sessionmaker")
@patch("controllers.inner_api.plugin.wraps.db")
def test_should_create_new_user_when_not_found(
self, mock_db, mock_sessionmaker, mock_enduser_class, mock_select, app: Flask
self,
sqlite_plugin_engine: Engine,
app: Flask,
):
"""Test creating new user when not found in database"""
# Arrange
mock_session = MagicMock()
mock_sessionmaker.return_value.begin.return_value.__enter__.return_value = mock_session
mock_session.scalar.return_value = None
mock_new_user = MagicMock()
mock_enduser_class.return_value = mock_new_user
mock_query = MagicMock()
mock_select.return_value.where.return_value.limit.return_value = mock_query
# Act
with app.app_context():
result = get_user("tenant123", "user123")
# Assert
assert result == mock_new_user
mock_session.add.assert_called_once()
mock_session.refresh.assert_called_once()
assert result.tenant_id == "tenant123"
assert result.session_id == "user123"
with Session(sqlite_plugin_engine) as session:
persisted_user = session.get(EndUser, result.id)
assert persisted_user is not None
assert persisted_user.session_id == "user123"
@patch("controllers.inner_api.plugin.wraps.select")
@patch("controllers.inner_api.plugin.wraps.EndUser")
@patch("controllers.inner_api.plugin.wraps.sessionmaker")
@patch("controllers.inner_api.plugin.wraps.db")
def test_should_use_default_session_id_when_user_id_none(
self, mock_db, mock_sessionmaker, mock_enduser_class, mock_select, app: Flask
self,
sqlite_plugin_engine: Engine,
app: Flask,
):
"""Test using default session ID when user_id is None"""
# Arrange
mock_user = MagicMock()
mock_session = MagicMock()
mock_sessionmaker.return_value.begin.return_value.__enter__.return_value = mock_session
# When user_id is None, is_anonymous=True, so session.scalar() is used
mock_session.scalar.return_value = mock_user
_persist_end_user(
sqlite_plugin_engine,
user_id="default-user-id",
session_id=DefaultEndUserSessionID.DEFAULT_SESSION_ID,
is_anonymous=True,
)
# Act
with app.app_context():
result = get_user("tenant123", None)
# Assert
assert result == mock_user
assert result.id == "default-user-id"
assert result.session_id == DefaultEndUserSessionID.DEFAULT_SESSION_ID
@patch("controllers.inner_api.plugin.wraps.EndUser")
@patch("controllers.inner_api.plugin.wraps.sessionmaker")
@patch("controllers.inner_api.plugin.wraps.db")
def test_should_raise_error_on_database_exception(self, mock_db, mock_sessionmaker, mock_enduser_class, app: Flask):
def test_should_raise_error_on_database_exception(self, sqlite_plugin_engine: Engine, app: Flask):
"""Test raising ValueError when database operation fails"""
# Arrange
mock_session = MagicMock()
mock_sessionmaker.return_value.begin.return_value.__enter__.return_value = mock_session
mock_session.scalar.side_effect = Exception("Database error")
# Act & Assert
with app.app_context():
with pytest.raises(ValueError, match="user not found"):
def _raise_database_error(*_args, **_kwargs):
raise RuntimeError("Database error")
event.listen(sqlite_plugin_engine, "before_cursor_execute", _raise_database_error)
try:
with app.app_context(), pytest.raises(ValueError, match="user not found"):
get_user("tenant123", "user123")
finally:
event.remove(sqlite_plugin_engine, "before_cursor_execute", _raise_database_error)
class TestGetUserTenant:
"""Test get_user_tenant decorator"""
@patch("controllers.inner_api.plugin.wraps.Tenant")
def test_should_inject_tenant_and_user_models(self, mock_tenant_class, app: Flask, monkeypatch: pytest.MonkeyPatch):
def test_should_inject_tenant_and_user_models(
self,
sqlite_plugin_engine: Engine,
app: Flask,
monkeypatch: pytest.MonkeyPatch,
):
"""Test that decorator injects tenant_model and user_model into kwargs"""
# Arrange
@@ -227,24 +241,20 @@ class TestGetUserTenant:
def protected_view(tenant_model, user_model, **kwargs):
return {"tenant": tenant_model, "user": user_model}
mock_tenant = MagicMock()
mock_tenant.id = "tenant123"
mock_user = MagicMock()
mock_user.id = "user456"
_persist_tenant(sqlite_plugin_engine)
_persist_end_user(
sqlite_plugin_engine,
user_id="user456",
session_id="user-session",
)
# Act
with app.test_request_context(json={"tenant_id": "tenant123", "user_id": "user456"}):
monkeypatch.setattr(app, "login_manager", MagicMock(), raising=False)
with patch("controllers.inner_api.plugin.wraps.db.session.get") as mock_get:
with patch("controllers.inner_api.plugin.wraps.get_user") as mock_get_user:
with patch("controllers.inner_api.plugin.wraps.user_logged_in"):
mock_get.return_value = mock_tenant
mock_get_user.return_value = mock_user
result = protected_view()
with patch("controllers.inner_api.plugin.wraps.user_logged_in"):
result = protected_view()
# Assert
assert result["tenant"] == mock_tenant
assert result["user"] == mock_user
assert result["tenant"].id == "tenant123"
assert result["user"].id == "user456"
def test_should_raise_error_when_tenant_id_missing(self, app: Flask):
"""Test that Pydantic ValidationError is raised when tenant_id is missing from payload"""
@@ -259,7 +269,7 @@ class TestGetUserTenant:
with pytest.raises(ValidationError):
protected_view()
def test_should_raise_error_when_tenant_not_found(self, app: Flask):
def test_should_raise_error_when_tenant_not_found(self, sqlite_plugin_engine: Engine, app: Flask):
"""Test that ValueError is raised when tenant is not found"""
# Arrange
@@ -267,16 +277,15 @@ class TestGetUserTenant:
def protected_view(tenant_model, user_model, **kwargs):
return "success"
# Act & Assert
with app.test_request_context(json={"tenant_id": "nonexistent", "user_id": "user456"}):
with patch("controllers.inner_api.plugin.wraps.db.session.get") as mock_get:
mock_get.return_value = None
with pytest.raises(ValueError, match="tenant not found"):
protected_view()
with pytest.raises(ValueError, match="tenant not found"):
protected_view()
@patch("controllers.inner_api.plugin.wraps.Tenant")
def test_should_use_default_session_id_when_user_id_empty(
self, mock_tenant_class, app: Flask, monkeypatch: pytest.MonkeyPatch
self,
sqlite_plugin_engine: Engine,
app: Flask,
monkeypatch: pytest.MonkeyPatch,
):
"""Test that default session ID is used when user_id is empty string"""
@@ -285,26 +294,22 @@ class TestGetUserTenant:
def protected_view(tenant_model, user_model, **kwargs):
return {"tenant": tenant_model, "user": user_model}
mock_tenant = MagicMock()
mock_tenant.id = "tenant123"
mock_user = MagicMock()
_persist_tenant(sqlite_plugin_engine)
_persist_end_user(
sqlite_plugin_engine,
user_id="default-user-id",
session_id=DefaultEndUserSessionID.DEFAULT_SESSION_ID,
is_anonymous=True,
)
# Act - use empty string for user_id to trigger default logic
with app.test_request_context(json={"tenant_id": "tenant123", "user_id": ""}):
monkeypatch.setattr(app, "login_manager", MagicMock(), raising=False)
with patch("controllers.inner_api.plugin.wraps.db.session.get") as mock_get:
with patch("controllers.inner_api.plugin.wraps.get_user") as mock_get_user:
with patch("controllers.inner_api.plugin.wraps.user_logged_in"):
mock_get.return_value = mock_tenant
mock_get_user.return_value = mock_user
result = protected_view()
with patch("controllers.inner_api.plugin.wraps.user_logged_in"):
result = protected_view()
# Assert
assert result["tenant"] == mock_tenant
assert result["user"] == mock_user
from models.model import DefaultEndUserSessionID
mock_get_user.assert_called_once_with("tenant123", DefaultEndUserSessionID.DEFAULT_SESSION_ID)
assert result["tenant"].id == "tenant123"
assert result["user"].id == "default-user-id"
assert result["user"].session_id == DefaultEndUserSessionID.DEFAULT_SESSION_ID
class PluginTestPayload:
@@ -2,10 +2,12 @@
Unit tests for inner_api auth decorators
"""
from unittest.mock import MagicMock, patch
from unittest.mock import patch
from uuid import NAMESPACE_URL, uuid5
import pytest
from flask import Flask
from sqlalchemy.orm import Session, sessionmaker
from werkzeug.exceptions import HTTPException
from configs import dify_config
@@ -16,9 +18,14 @@ from controllers.inner_api.wraps import (
inner_api_only,
plugin_inner_api_only,
)
from models.enums import EndUserType
from models.model import EndUser
def _stable_uuid(value: str) -> str:
return str(uuid5(NAMESPACE_URL, value))
class TestBillingInnerApiOnly:
"""Test billing_inner_api_only decorator"""
@@ -258,7 +265,7 @@ class TestEnterpriseInnerApiUserAuth:
assert result == "no_user"
def test_should_pass_through_when_hmac_signature_invalid(self, app: Flask):
"""Test that request passes through when HMAC signature is invalid"""
"""Invalid HMAC auth passes through without opening a database session."""
# Arrange
@enterprise_inner_api_user_auth
@@ -277,7 +284,8 @@ class TestEnterpriseInnerApiUserAuth:
assert result == "no_user"
mock_create_session.assert_not_called()
def test_should_inject_user_when_hmac_signature_valid(self, app: Flask):
@pytest.mark.parametrize("sqlite_session", [(EndUser,)], indirect=True)
def test_should_inject_user_when_hmac_signature_valid(self, app: Flask, sqlite_session: Session):
"""Test that user is injected when HMAC signature is valid"""
# Arrange
from base64 import b64encode
@@ -289,19 +297,25 @@ class TestEnterpriseInnerApiUserAuth:
return kwargs.get("user")
# Calculate valid HMAC signature
user_id = "user123"
user_id = _stable_uuid("end-user:user123")
inner_api_key = "valid_key"
data_to_sign = f"DIFY {user_id}"
signature = hmac_new(inner_api_key.encode("utf-8"), data_to_sign.encode("utf-8"), sha1)
valid_signature = b64encode(signature.digest()).decode("utf-8")
# Create mock user
mock_user = MagicMock()
mock_user.id = user_id
mock_session = MagicMock()
mock_session.get.return_value = mock_user
mock_session_context = MagicMock()
mock_session_context.__enter__.return_value = mock_session
end_user = EndUser(
id=user_id,
tenant_id=_stable_uuid("tenant:inner-api"),
type=EndUserType.BROWSER,
name="Inner API User",
session_id="inner-api-session",
)
sqlite_session.add(end_user)
sqlite_session.commit()
database_session_factory = sessionmaker(
bind=sqlite_session.get_bind(),
expire_on_commit=False,
)
# Act
with app.test_request_context(
@@ -310,14 +324,15 @@ class TestEnterpriseInnerApiUserAuth:
with patch.object(dify_config, "INNER_API", True):
with patch(
"controllers.inner_api.wraps.session_factory.create_session",
return_value=mock_session_context,
) as mock_create_session:
database_session_factory,
):
result = protected_view()
# Assert
assert result == mock_user
mock_create_session.assert_called_once_with()
mock_session.get.assert_called_once_with(EndUser, user_id)
assert isinstance(result, EndUser)
assert result.id == end_user.id
assert result.tenant_id == end_user.tenant_id
assert result.session_id == "inner-api-session"
class TestPluginInnerApiOnly:
@@ -1,14 +1,20 @@
"""Unit tests for runtime credential inner API."""
import inspect
import json
from unittest.mock import MagicMock, patch
import pytest
from flask import Flask
from sqlalchemy.engine import Engine
from sqlalchemy.orm import Session
from controllers.inner_api.runtime_credentials import (
EnterpriseRuntimeCredentialsResolve,
InnerRuntimeCredentialsResolvePayload,
)
from models.provider import ProviderCredential
from models.tools import BuiltinToolProvider
def test_runtime_credentials_payload_accepts_items():
@@ -32,14 +38,15 @@ def test_runtime_credentials_payload_accepts_items():
@patch("controllers.inner_api.runtime_credentials.encrypter.decrypt_token")
@patch("controllers.inner_api.runtime_credentials.db")
@patch("controllers.inner_api.runtime_credentials.Session")
@patch("controllers.inner_api.runtime_credentials.create_plugin_provider_manager")
@pytest.mark.parametrize("sqlite_session", [(ProviderCredential,)], indirect=True)
def test_runtime_model_credentials_resolve_returns_decrypted_values(
mock_provider_manager_factory,
mock_session_cls,
mock_db,
mock_decrypt_token,
app: Flask,
sqlite_engine: Engine,
sqlite_session: Session,
):
provider_configuration = MagicMock()
provider_configuration.provider.provider_credential_schema.credential_form_schemas = []
@@ -52,14 +59,16 @@ def test_runtime_model_credentials_resolve_returns_decrypted_values(
provider_manager.get_configurations.return_value = provider_configurations
mock_provider_manager_factory.return_value = provider_manager
credential = MagicMock()
credential.encrypted_config = '{"openai_api_key":"encrypted","api_base":"https://api.openai.com/v1"}'
session = MagicMock()
session.__enter__.return_value = session
session.__exit__.return_value = False
session.execute.return_value.scalar_one_or_none.return_value = credential
mock_session_cls.return_value = session
mock_db.engine = MagicMock()
credential = ProviderCredential(
tenant_id="tenant-1",
provider_name="langgenius/openai/openai",
credential_name="OpenAI",
encrypted_config='{"openai_api_key":"encrypted","api_base":"https://api.openai.com/v1"}',
)
credential.id = "credential-1"
sqlite_session.add(credential)
sqlite_session.commit()
mock_db.engine = sqlite_engine
mock_decrypt_token.return_value = "sk-test"
handler = EnterpriseRuntimeCredentialsResolve()
@@ -110,28 +119,32 @@ def test_runtime_model_credentials_resolve_rejects_unknown_provider(mock_provide
@patch("controllers.inner_api.runtime_credentials.create_provider_encrypter")
@patch("controllers.inner_api.runtime_credentials.ToolProviderCredentialsCache")
@patch("controllers.inner_api.runtime_credentials.db")
@patch("controllers.inner_api.runtime_credentials.Session")
@patch("controllers.inner_api.runtime_credentials.ToolManager")
@pytest.mark.parametrize("sqlite_session", [(BuiltinToolProvider,)], indirect=True)
def test_runtime_tool_credentials_resolve_returns_decrypted_values(
mock_tool_manager,
mock_session_cls,
mock_db,
mock_cache_cls,
mock_create_encrypter,
app: Flask,
sqlite_engine: Engine,
sqlite_session: Session,
):
provider_controller = MagicMock()
provider_controller.get_credentials_schema_by_type.return_value = []
mock_tool_manager.get_builtin_provider.return_value = provider_controller
builtin_provider = MagicMock()
builtin_provider = BuiltinToolProvider(
tenant_id="tenant-1",
user_id="user-1",
provider="langgenius/tavily/tavily",
name="Tavily",
encrypted_credentials=json.dumps({"tavily_api_key": "encrypted"}),
)
builtin_provider.id = "credential-1"
session = MagicMock()
session.__enter__.return_value = session
session.__exit__.return_value = False
session.execute.return_value.scalar_one_or_none.return_value = builtin_provider
mock_session_cls.return_value = session
mock_db.engine = MagicMock()
sqlite_session.add(builtin_provider)
sqlite_session.commit()
mock_db.engine = sqlite_engine
provider_encrypter = MagicMock()
provider_encrypter.decrypt.return_value = {"tavily_api_key": "tvly-secret"}
@@ -157,27 +170,34 @@ def test_runtime_tool_credentials_resolve_returns_decrypted_values(
assert body["credentials"][0]["kind"] == "tool"
assert body["credentials"][0]["provider"] == "langgenius/tavily/tavily"
assert body["credentials"][0]["values"]["tavily_api_key"] == "tvly-secret"
compiled = str(session.execute.call_args.args[0].compile(compile_kwargs={"literal_binds": True}))
assert "tool_builtin_providers.provider = 'langgenius/tavily/tavily'" in compiled
provider_encrypter.decrypt.assert_called_once_with({"tavily_api_key": "encrypted"})
@patch("controllers.inner_api.runtime_credentials.db")
@patch("controllers.inner_api.runtime_credentials.Session")
@patch("controllers.inner_api.runtime_credentials.ToolManager")
@pytest.mark.parametrize("sqlite_session", [(BuiltinToolProvider,)], indirect=True)
def test_runtime_tool_credentials_resolve_rejects_unknown_credential(
mock_tool_manager,
mock_session_cls,
mock_db,
app: Flask,
sqlite_engine: Engine,
sqlite_session: Session,
):
mock_tool_manager.get_builtin_provider.return_value = MagicMock()
session = MagicMock()
session.__enter__.return_value = session
session.__exit__.return_value = False
session.execute.return_value.scalar_one_or_none.return_value = None
mock_session_cls.return_value = session
mock_db.engine = MagicMock()
# The requested id exists for another tenant, proving the resolver does not
# expose a credential across workspace boundaries.
builtin_provider = BuiltinToolProvider(
tenant_id="tenant-2",
user_id="user-2",
provider="langgenius/tavily/tavily",
name="Other workspace Tavily",
encrypted_credentials=json.dumps({"tavily_api_key": "encrypted"}),
)
builtin_provider.id = "missing"
sqlite_session.add(builtin_provider)
sqlite_session.commit()
mock_db.engine = sqlite_engine
handler = EnterpriseRuntimeCredentialsResolve()
unwrapped = inspect.unwrap(handler.post)
@@ -0,0 +1,64 @@
import inspect
from unittest.mock import call, patch
import pytest
from flask import Flask
from pydantic import ValidationError
from controllers.inner_api.workspace.plugin_model_providers import (
EnterprisePluginModelProvidersCacheInvalidate,
InvalidatePluginModelProvidersCachePayload,
)
class TestInvalidatePluginModelProvidersCachePayload:
def test_valid_payload(self):
payload = InvalidatePluginModelProvidersCachePayload.model_validate(
{"tenant_ids": ["tenant-alpha", "tenant-beta"]}
)
assert payload.tenant_ids == ["tenant-alpha", "tenant-beta"]
def test_missing_tenant_ids_defaults_to_empty(self):
payload = InvalidatePluginModelProvidersCachePayload.model_validate({})
assert payload.tenant_ids == []
def test_unknown_field_rejected(self):
with pytest.raises(ValidationError):
InvalidatePluginModelProvidersCachePayload.model_validate({"tenant_ids": ["tenant-alpha"], "generation": 7})
class TestEnterprisePluginModelProvidersCacheInvalidate:
@pytest.fixture
def api_instance(self):
return EnterprisePluginModelProvidersCacheInvalidate()
def _post(self, api_instance, app: Flask, payload):
unwrapped_post = inspect.unwrap(api_instance.post)
with app.test_request_context():
with patch("controllers.inner_api.workspace.plugin_model_providers.inner_api_ns") as mock_ns:
mock_ns.payload = payload
return unwrapped_post(api_instance)
@patch("controllers.inner_api.workspace.plugin_model_providers.PluginService")
def test_post_invalidates_once_per_tenant(self, mock_plugin_service, api_instance, app: Flask):
result = self._post(api_instance, app, {"tenant_ids": ["tenant-alpha", "tenant-beta"]})
assert result == ({"result": "success"}, 200)
assert mock_plugin_service.invalidate_plugin_model_providers_cache.call_args_list == [
call("tenant-alpha"),
call("tenant-beta"),
]
@patch("controllers.inner_api.workspace.plugin_model_providers.PluginService")
def test_post_with_empty_list_is_a_no_op(self, mock_plugin_service, api_instance, app: Flask):
result = self._post(api_instance, app, {"tenant_ids": []})
assert result == ({"result": "success"}, 200)
mock_plugin_service.invalidate_plugin_model_providers_cache.assert_not_called()
@patch("controllers.inner_api.workspace.plugin_model_providers.PluginService")
def test_post_with_missing_payload_is_a_no_op(self, mock_plugin_service, api_instance, app: Flask):
result = self._post(api_instance, app, None)
assert result == ({"result": "success"}, 200)
mock_plugin_service.invalidate_plugin_model_providers_cache.assert_not_called()
@@ -1,4 +1,9 @@
"""Tests for openapi workflow events reconnect endpoint."""
"""Tests for the OpenAPI workflow-events reconnect endpoint.
The controller constructs a repository session factory, so every case binds
that real SQLAlchemy factory to an isolated SQLite engine. Repository behavior
remains mocked because these tests focus on authorization and SSE responses.
"""
from __future__ import annotations
@@ -9,6 +14,8 @@ from unittest.mock import Mock
import pytest
from flask import Flask
from sqlalchemy.engine import Engine
from sqlalchemy.orm import sessionmaker
from werkzeug.exceptions import NotFound
from controllers.openapi.auth.data import AuthData
@@ -47,6 +54,11 @@ def _make_workflow_run(
class TestOpenApiWorkflowEventsApi:
@pytest.fixture(autouse=True)
def _bind_sqlite_engine(self, monkeypatch: pytest.MonkeyPatch, sqlite_engine: Engine) -> None:
module = sys.modules["controllers.openapi.workflow_events"]
monkeypatch.setattr(module, "db", SimpleNamespace(engine=sqlite_engine))
def _get_api(self):
from controllers.openapi.workflow_events import OpenApiWorkflowEventsApi
@@ -59,8 +71,6 @@ class TestOpenApiWorkflowEventsApi:
factory_mock = Mock()
factory_mock.create_api_workflow_run_repository.return_value = repo_mock
monkeypatch.setattr(module, "DifyAPIRepositoryFactory", factory_mock)
monkeypatch.setattr(module, "sessionmaker", Mock(return_value=object()))
monkeypatch.setattr(module, "db", SimpleNamespace(engine=object()))
api = self._get_api()
from models.model import AppMode
@@ -77,6 +87,10 @@ class TestOpenApiWorkflowEventsApi:
auth_data=_make_auth_data(app_model, caller, "account"),
)
session_maker = factory_mock.create_api_workflow_run_repository.call_args.args[0]
assert isinstance(session_maker, sessionmaker)
assert session_maker.kw["bind"] is module.db.engine
def test_not_found_when_run_belongs_to_different_app(
self, app: Flask, bypass_pipeline, monkeypatch: pytest.MonkeyPatch
):
@@ -87,8 +101,6 @@ class TestOpenApiWorkflowEventsApi:
factory_mock = Mock()
factory_mock.create_api_workflow_run_repository.return_value = repo_mock
monkeypatch.setattr(module, "DifyAPIRepositoryFactory", factory_mock)
monkeypatch.setattr(module, "sessionmaker", Mock(return_value=object()))
monkeypatch.setattr(module, "db", SimpleNamespace(engine=object()))
api = self._get_api()
from models.model import AppMode
@@ -116,8 +128,6 @@ class TestOpenApiWorkflowEventsApi:
factory_mock = Mock()
factory_mock.create_api_workflow_run_repository.return_value = repo_mock
monkeypatch.setattr(module, "DifyAPIRepositoryFactory", factory_mock)
monkeypatch.setattr(module, "sessionmaker", Mock(return_value=object()))
monkeypatch.setattr(module, "db", SimpleNamespace(engine=object()))
snapshot_builder = Mock(return_value=iter([]))
monkeypatch.setattr(module, "build_workflow_event_stream", snapshot_builder)
@@ -156,8 +166,6 @@ class TestOpenApiWorkflowEventsApi:
factory_mock = Mock()
factory_mock.create_api_workflow_run_repository.return_value = repo_mock
monkeypatch.setattr(module, "DifyAPIRepositoryFactory", factory_mock)
monkeypatch.setattr(module, "sessionmaker", Mock(return_value=object()))
monkeypatch.setattr(module, "db", SimpleNamespace(engine=object()))
from models.model import AppMode
@@ -185,8 +193,6 @@ class TestOpenApiWorkflowEventsApi:
factory_mock = Mock()
factory_mock.create_api_workflow_run_repository.return_value = repo_mock
monkeypatch.setattr(module, "DifyAPIRepositoryFactory", factory_mock)
monkeypatch.setattr(module, "sessionmaker", Mock(return_value=object()))
monkeypatch.setattr(module, "db", SimpleNamespace(engine=object()))
msg_gen_mock = Mock()
msg_gen_mock.retrieve_events.return_value = iter([])
@@ -227,8 +233,6 @@ class TestOpenApiWorkflowEventsApi:
factory_mock = Mock()
factory_mock.create_api_workflow_run_repository.return_value = repo_mock
monkeypatch.setattr(module, "DifyAPIRepositoryFactory", factory_mock)
monkeypatch.setattr(module, "sessionmaker", Mock(return_value=object()))
monkeypatch.setattr(module, "db", SimpleNamespace(engine=object()))
finish_response = SimpleNamespace(
event=SimpleNamespace(value="workflow_finished"),
@@ -22,6 +22,8 @@ from unittest.mock import Mock, patch
import pytest
from flask import Flask
from sqlalchemy.engine import Engine
from sqlalchemy.orm import Session
from werkzeug.exceptions import BadRequest, NotFound
import services
@@ -39,19 +41,45 @@ from controllers.service_api.app.conversation import (
ConversationVariableUpdatePayload,
)
from controllers.service_api.app.error import NotChatAppError
from core.app.entities.app_invoke_entities import InvokeFrom
from fields._value_type_serializer import serialize_value_type
from graphon.variables import StringSegment
from graphon.variables.types import SegmentType
from models.model import App, AppMode, EndUser
from models.enums import ConversationFromSource
from models.model import App, AppMode, Conversation, EndUser
from services.conversation_service import ConversationService
from services.errors.conversation import (
ConversationNotExistsError,
ConversationVariableNotExistsError,
ConversationVariableTypeMismatchError,
LastConversationNotExistsError,
)
def _end_user(user_id: str = "end-user-1") -> EndUser:
end_user = EndUser()
end_user.id = user_id
return end_user
def _conversation(
*,
conversation_id: str,
app_id: str = "app-1",
end_user_id: str = "end-user-1",
) -> Conversation:
conversation = Conversation(
app_id=app_id,
mode=AppMode.CHAT,
name="Original Name",
from_source=ConversationFromSource.API,
from_end_user_id=end_user_id,
invoke_from=InvokeFrom.SERVICE_API,
)
conversation.id = conversation_id
conversation.inputs = {}
return conversation
class TestConversationListQuery:
"""Test suite for ConversationListQuery Pydantic model."""
@@ -462,23 +490,30 @@ class TestConversationService:
assert hasattr(result, "limit")
assert hasattr(result, "has_more")
@patch.object(ConversationService, "rename")
def test_rename_returns_conversation(self, mock_rename):
@pytest.mark.parametrize("sqlite_session", [(Conversation,)], indirect=True)
def test_rename_returns_conversation(self, sqlite_session: Session):
"""Test rename returns updated conversation."""
mock_conversation = Mock()
mock_conversation.name = "New Name"
mock_rename.return_value = mock_conversation
conversation_id = "00000000-0000-0000-0000-000000000001"
conversation = _conversation(conversation_id=conversation_id)
sqlite_session.add(conversation)
sqlite_session.commit()
app_model = App()
app_model.id = "app-1"
end_user = _end_user()
result = ConversationService.rename(
app_model=Mock(spec=App),
conversation_id="conv_123",
user=Mock(spec=EndUser),
app_model=app_model,
conversation_id=conversation_id,
user=end_user,
name="New Name",
auto_generate=False,
session=Mock(),
session=sqlite_session,
)
assert result.name == "New Name"
sqlite_session.refresh(conversation)
assert conversation.name == "New Name"
class TestConversationPayloadsController:
@@ -502,37 +537,29 @@ class TestConversationApiController:
with pytest.raises(NotChatAppError):
handler(api, app_model=app_model, end_user=end_user)
def test_list_last_not_found(self, app: Flask, monkeypatch: pytest.MonkeyPatch) -> None:
class _BeginStub:
def __enter__(self):
return SimpleNamespace()
@pytest.mark.parametrize("sqlite_session", [(Conversation,)], indirect=True)
def test_list_last_not_found(
self,
app: Flask,
monkeypatch: pytest.MonkeyPatch,
sqlite_engine: Engine,
sqlite_session: Session,
) -> None:
last_id = "00000000-0000-0000-0000-000000000001"
# The id exists for a different app, proving pagination cannot cross app boundaries.
sqlite_session.add(_conversation(conversation_id=last_id, app_id="other-app"))
sqlite_session.commit()
def __exit__(self, exc_type, exc, tb):
return False
class _SessionMakerStub:
def __init__(self, *args, **kwargs):
pass
def begin(self):
return _BeginStub()
monkeypatch.setattr(
ConversationService,
"pagination_by_last_id",
lambda *_args, **_kwargs: (_ for _ in ()).throw(LastConversationNotExistsError()),
)
conversation_module = sys.modules["controllers.service_api.app.conversation"]
monkeypatch.setattr(conversation_module, "db", SimpleNamespace(engine=object()))
monkeypatch.setattr(conversation_module, "sessionmaker", _SessionMakerStub)
monkeypatch.setattr(conversation_module, "db", SimpleNamespace(engine=sqlite_engine))
api = ConversationApi()
handler = unwrap(api.get)
app_model = SimpleNamespace(mode=AppMode.CHAT)
end_user = SimpleNamespace()
app_model = SimpleNamespace(id="app-1", mode=AppMode.CHAT)
end_user = _end_user()
with app.test_request_context(
"/conversations?last_id=00000000-0000-0000-0000-000000000001&limit=20",
f"/conversations?last_id={last_id}&limit=20",
method="GET",
):
with pytest.raises(NotFound):
@@ -14,6 +14,8 @@ from unittest.mock import ANY, MagicMock, Mock
import pytest
from flask import Flask
from sqlalchemy.engine import Engine
from sqlalchemy.orm import Session, sessionmaker
import services.app_generate_service as ags_module
from controllers.service_api.app.workflow_events import WorkflowEventsApi
@@ -31,7 +33,7 @@ from core.app.entities.task_entities import (
from core.app.layers.pause_state_persist_layer import WorkflowResumptionContext, _WorkflowGenerateEntityWrapper
from core.workflow.human_input_policy import FormDisposition, HumanInputSurface
from core.workflow.nodes.human_input.entities import ParagraphInputConfig, UserActionConfig
from core.workflow.nodes.human_input.enums import FormInputType
from core.workflow.nodes.human_input.enums import FormInputType, HumanInputFormKind, HumanInputFormStatus
from core.workflow.nodes.human_input.pause_reason import DifyHITLEventType, HumanInputRequired
from core.workflow.system_variables import build_system_variables
from graphon.entities import WorkflowStartReason
@@ -39,6 +41,7 @@ from graphon.enums import WorkflowExecutionStatus, WorkflowNodeExecutionStatus
from graphon.runtime import GraphRuntimeState, VariablePool
from models.account import Account
from models.enums import CreatorUserRole
from models.human_input import HumanInputForm
from models.model import AppMode
from models.workflow import WorkflowRun
from repositories.api_workflow_node_execution_repository import WorkflowNodeExecutionSnapshot
@@ -66,7 +69,7 @@ class _DummyRateLimit:
return generator
def _mock_repo_for_run(monkeypatch: pytest.MonkeyPatch, workflow_run):
def _mock_repo_for_run(monkeypatch: pytest.MonkeyPatch, workflow_run, sqlite_engine: Engine):
workflow_events_module = sys.modules["controllers.service_api.app.workflow_events"]
repo = SimpleNamespace(get_workflow_run_by_id_and_tenant_id=lambda **_kwargs: workflow_run)
monkeypatch.setattr(
@@ -74,10 +77,33 @@ def _mock_repo_for_run(monkeypatch: pytest.MonkeyPatch, workflow_run):
"create_api_workflow_run_repository",
lambda *_args, **_kwargs: repo,
)
monkeypatch.setattr(workflow_events_module, "db", SimpleNamespace(engine=object()))
monkeypatch.setattr(workflow_events_module, "db", SimpleNamespace(engine=sqlite_engine))
return workflow_events_module
def _persist_human_input_form(
sqlite_session: Session,
*,
expiration_time: datetime,
) -> HumanInputForm:
form = HumanInputForm(
id="form-1",
tenant_id="tenant-1",
app_id="app-1",
workflow_run_id="run-1",
conversation_id=None,
form_kind=HumanInputFormKind.RUNTIME,
node_id="node-1",
form_definition=json.dumps({"display_in_ui": True}),
rendered_content="Rendered",
status=HumanInputFormStatus.WAITING,
expiration_time=expiration_time,
)
sqlite_session.add(form)
sqlite_session.commit()
return form
def _build_service_api_pause_converter() -> WorkflowResponseConverter:
application_generate_entity = SimpleNamespace(
inputs={},
@@ -257,7 +283,10 @@ def _build_resumption_context(task_id: str) -> WorkflowResumptionContext:
class TestHitlServiceApi:
# Service API event-stream continuation
def test_workflow_events_continue_on_pause_keeps_stream_open(
self, app: Flask, monkeypatch: pytest.MonkeyPatch
self,
app: Flask,
monkeypatch: pytest.MonkeyPatch,
sqlite_engine: Engine,
) -> None:
workflow_run = SimpleNamespace(
id="run-1",
@@ -266,7 +295,11 @@ class TestHitlServiceApi:
created_by="end-user-1",
finished_at=None,
)
workflow_events_module = _mock_repo_for_run(monkeypatch, workflow_run=workflow_run)
workflow_events_module = _mock_repo_for_run(
monkeypatch,
workflow_run=workflow_run,
sqlite_engine=sqlite_engine,
)
msg_generator = Mock()
msg_generator.retrieve_events.return_value = ["raw-event"]
workflow_generator = Mock()
@@ -291,7 +324,10 @@ class TestHitlServiceApi:
workflow_generator.convert_to_event_stream.assert_called_once_with(["raw-event"])
def test_workflow_events_snapshot_continue_on_pause_keeps_pause_open(
self, app: Flask, monkeypatch: pytest.MonkeyPatch
self,
app: Flask,
monkeypatch: pytest.MonkeyPatch,
sqlite_engine: Engine,
) -> None:
workflow_run = SimpleNamespace(
id="run-1",
@@ -300,7 +336,11 @@ class TestHitlServiceApi:
created_by="end-user-1",
finished_at=None,
)
workflow_events_module = _mock_repo_for_run(monkeypatch, workflow_run=workflow_run)
workflow_events_module = _mock_repo_for_run(
monkeypatch,
workflow_run=workflow_run,
sqlite_engine=sqlite_engine,
)
msg_generator = Mock()
workflow_generator = Mock()
workflow_generator.convert_to_event_stream.return_value = iter(["data: snapshot\n\n"])
@@ -331,16 +371,24 @@ class TestHitlServiceApi:
human_input_surface=HumanInputSurface.SERVICE_API,
close_on_pause=False,
)
snapshot_session_maker = snapshot_builder.call_args.kwargs["session_maker"]
assert isinstance(snapshot_session_maker, sessionmaker)
assert snapshot_session_maker.kw["bind"] is sqlite_engine
workflow_generator.convert_to_event_stream.assert_called_once_with(["snapshot-events"])
def test_advanced_chat_blocking_injects_pause_state_config(self, monkeypatch: pytest.MonkeyPatch) -> None:
def test_advanced_chat_blocking_injects_pause_state_config(
self,
monkeypatch: pytest.MonkeyPatch,
sqlite_engine: Engine,
) -> None:
monkeypatch.setattr(ags_module.dify_config, "BILLING_ENABLED", False)
monkeypatch.setattr(ags_module, "RateLimit", _DummyRateLimit)
workflow = MagicMock()
workflow.created_by = "owner-id"
monkeypatch.setattr(AppGenerateService, "_get_workflow", lambda *args, **kwargs: workflow)
monkeypatch.setattr(ags_module.session_factory, "get_session_maker", lambda: "session-maker")
sqlite_session_maker = sessionmaker(bind=sqlite_engine, expire_on_commit=False)
monkeypatch.setattr(ags_module.session_factory, "get_session_maker", lambda: sqlite_session_maker)
generator_instance = MagicMock()
generator_instance.generate.return_value = {"result": "advanced-blocking"}
@@ -358,20 +406,21 @@ class TestHitlServiceApi:
user = MagicMock()
user.id = "user-id"
result = AppGenerateService.generate(
session=Mock(),
app_model=app_model,
user=user,
args={"workflow_id": None, "query": "hi", "inputs": {}},
invoke_from=InvokeFrom.SERVICE_API,
streaming=False,
)
with sqlite_session_maker() as session:
result = AppGenerateService.generate(
session=session,
app_model=app_model,
user=user,
args={"workflow_id": None, "query": "hi", "inputs": {}},
invoke_from=InvokeFrom.SERVICE_API,
streaming=False,
)
assert result == {"result": "advanced-blocking"}
call_kwargs = generator_instance.generate.call_args.kwargs
assert call_kwargs["streaming"] is False
assert call_kwargs["pause_state_config"] is not None
assert call_kwargs["pause_state_config"].session_factory == "session-maker"
assert call_kwargs["pause_state_config"].session_factory is sqlite_session_maker
assert call_kwargs["pause_state_config"].state_owner_user_id == "owner-id"
# Blocking payload contract
@@ -569,7 +618,13 @@ class TestHitlServiceApi:
assert response.data.paused_nodes == ["node-1"]
assert response.data.reasons == [{"TYPE": "human_input_required", "form_id": "form-1", "expiration_time": 1}]
def test_service_api_pause_event_serializes_hitl_reason(self, monkeypatch: pytest.MonkeyPatch) -> None:
@pytest.mark.parametrize("sqlite_session", [(HumanInputForm,)], indirect=True)
def test_service_api_pause_event_serializes_hitl_reason(
self,
monkeypatch: pytest.MonkeyPatch,
sqlite_engine: Engine,
sqlite_session: Session,
) -> None:
converter = _build_service_api_pause_converter()
converter.workflow_start_to_stream_response(
task_id="task",
@@ -578,20 +633,10 @@ class TestHitlServiceApi:
reason=WorkflowStartReason.INITIAL,
)
expiration_time = datetime(2024, 1, 1, tzinfo=UTC)
expiration_time = datetime(2024, 1, 1)
_persist_human_input_form(sqlite_session, expiration_time=expiration_time)
class _FakeSession:
def execute(self, _stmt):
return [("form-1", expiration_time, '{"display_in_ui": true}')]
def __enter__(self):
return self
def __exit__(self, exc_type, exc, tb):
return False
monkeypatch.setattr(workflow_response_converter, "Session", lambda **_: _FakeSession())
monkeypatch.setattr(workflow_response_converter, "db", SimpleNamespace(engine=object()))
monkeypatch.setattr(workflow_response_converter, "db", SimpleNamespace(engine=sqlite_engine))
monkeypatch.setattr(
workflow_response_converter,
"load_form_dispositions_by_form_id",
@@ -651,10 +696,18 @@ class TestHitlServiceApi:
assert hi_resp.data.expiration_time == int(expiration_time.timestamp())
# Snapshot payload contract
def test_snapshot_events_include_pause_payload_contract(self, monkeypatch: pytest.MonkeyPatch) -> None:
@pytest.mark.parametrize("sqlite_session", [(HumanInputForm,)], indirect=True)
def test_snapshot_events_include_pause_payload_contract(
self,
monkeypatch: pytest.MonkeyPatch,
sqlite_engine: Engine,
sqlite_session: Session,
) -> None:
workflow_run = _build_workflow_run(WorkflowExecutionStatus.PAUSED)
snapshot = _build_snapshot(WorkflowNodeExecutionStatus.PAUSED)
resumption_context = _build_resumption_context("task-ctx")
expiration_time = datetime(2024, 1, 1)
_persist_human_input_form(sqlite_session, expiration_time=expiration_time)
monkeypatch.setattr(
"services.workflow_event_snapshot_service.load_form_dispositions_by_form_id",
lambda form_ids, session=None, surface=None: {
@@ -662,22 +715,7 @@ class TestHitlServiceApi:
},
)
class _SessionContext:
def __init__(self, session):
self._session = session
def __enter__(self):
return self._session
def __exit__(self, exc_type, exc, tb):
return False
def session_maker() -> _SessionContext:
return _SessionContext(
SimpleNamespace(
execute=lambda _stmt: [("form-1", datetime(2024, 1, 1, tzinfo=UTC), '{"display_in_ui": true}')],
)
)
sqlite_session_maker = sessionmaker(bind=sqlite_engine, expire_on_commit=False)
pause_entity = _FakePauseEntity(
pause_id="pause-1",
@@ -701,7 +739,7 @@ class TestHitlServiceApi:
message_context=None,
pause_entity=pause_entity,
resumption_context=resumption_context,
session_maker=session_maker,
session_maker=sqlite_session_maker,
)
assert [event["event"] for event in events] == [
@@ -713,13 +751,13 @@ class TestHitlServiceApi:
]
assert events[2]["data"]["status"] == WorkflowNodeExecutionStatus.PAUSED.value
assert events[3]["data"]["form_token"] == "wtok"
assert events[3]["data"]["expiration_time"] == int(datetime(2024, 1, 1, tzinfo=UTC).timestamp())
assert events[3]["data"]["expiration_time"] == int(expiration_time.timestamp())
pause_data = events[-1]["data"]
assert pause_data["paused_nodes"] == ["node-1"]
assert pause_data["outputs"] == {"result": "value"}
assert pause_data["reasons"][0]["TYPE"] == "human_input_required"
assert pause_data["reasons"][0]["form_token"] == "wtok"
assert pause_data["reasons"][0]["expiration_time"] == int(datetime(2024, 1, 1, tzinfo=UTC).timestamp())
assert pause_data["reasons"][0]["expiration_time"] == int(expiration_time.timestamp())
assert pause_data["status"] == WorkflowExecutionStatus.PAUSED.value
assert pause_data["created_at"] == int(workflow_run.created_at.timestamp())
assert pause_data["elapsed_time"] == workflow_run.elapsed_time
@@ -16,20 +16,20 @@ Focus on:
import json
import sys
import uuid
from dataclasses import dataclass, field
from datetime import UTC, datetime
from inspect import unwrap
from types import SimpleNamespace
from unittest.mock import MagicMock, Mock, patch
import pytest
from flask import Flask
from sqlalchemy.orm import sessionmaker
from sqlalchemy.engine import Engine
from sqlalchemy.orm import Session, sessionmaker
from werkzeug.exceptions import BadRequest, NotFound
from controllers.service_api.app.error import NotWorkflowAppError, WorkflowVersionExecutionNotAllowedError
from controllers.service_api.app.workflow import (
AppQueueManager,
DifyAPIRepositoryFactory,
GraphEngineManager,
WorkflowAppLogApi,
WorkflowLogQuery,
@@ -44,6 +44,7 @@ from controllers.web.error import InvokeRateLimitError as InvokeRateLimitHttpErr
from core.app.entities.app_invoke_entities import InvokeFrom
from enums.cloud_plan import CloudPlan
from graphon.enums import WorkflowExecutionStatus
from models import Account
from models.enums import CreatorUserRole, WorkflowRunTriggeredFrom
from models.model import App, AppMode, EndUser
from models.workflow import WorkflowAppLog, WorkflowAppLogCreatedFrom, WorkflowRun, WorkflowType
@@ -51,58 +52,18 @@ from services.app_generate_service import AppGenerateService
from services.billing_service import BillingService
from services.errors.app import IsDraftWorkflowError, WorkflowNotFoundError
from services.errors.llm import InvokeRateLimitError
from services.workflow_app_service import LogView, LogViewDetails, WorkflowAppService
from services.workflow_app_service import WorkflowAppService
def _default_workflow_inputs() -> dict[str, object]:
return {"input": "value"}
def _default_log_details() -> LogViewDetails:
return {"trigger_metadata": {"node": "answer", "latency": 1.25}}
class _DbSessionStub:
def get(self, *args: object, **kwargs: object) -> None:
return None
@dataclass
class _DbStub:
engine: object = field(default_factory=object)
session: _DbSessionStub = field(default_factory=_DbSessionStub)
@dataclass
class _WorkflowRunRepositoryStub:
run: WorkflowRun | None
def get_workflow_run_by_id(self, *, tenant_id: str, app_id: str, run_id: str) -> WorkflowRun | None:
return self.run if tenant_id and app_id and run_id else None
def get_workflow_run_by_id_without_tenant(self, *, run_id: str) -> WorkflowRun | None:
return self.run if run_id else None
class _BeginStub:
def __enter__(self) -> object:
return object()
def __exit__(self, exc_type: object, exc: object, tb: object) -> bool:
return False
class _SessionMakerStub:
def __init__(self, *args: object, **kwargs: object) -> None:
pass
def begin(self) -> _BeginStub:
return _BeginStub()
def _make_workflow_run(
run_id: str = "run-1",
*,
tenant_id: str = "tenant-1",
app_id: str = "app-1",
workflow_id: str = "wf-1",
inputs: dict[str, object] | None = None,
outputs: dict[str, object] | None = None,
@@ -111,8 +72,8 @@ def _make_workflow_run(
) -> WorkflowRun:
return WorkflowRun(
id=run_id,
tenant_id="tenant-1",
app_id="app-1",
tenant_id=tenant_id,
app_id=app_id,
workflow_id=workflow_id,
type=WorkflowType.WORKFLOW,
triggered_from=WorkflowRunTriggeredFrom.APP_RUN,
@@ -133,12 +94,17 @@ def _make_workflow_run(
)
def _make_workflow_app_log() -> WorkflowAppLog:
def _make_workflow_app_log(
*,
tenant_id: str = "tenant-1",
app_id: str = "app-1",
workflow_run_id: str = "log-run-1",
) -> WorkflowAppLog:
log = WorkflowAppLog(
tenant_id="tenant-1",
app_id="app-1",
tenant_id=tenant_id,
app_id=app_id,
workflow_id="wf-1",
workflow_run_id="log-run-1",
workflow_run_id=workflow_run_id,
created_from=WorkflowAppLogCreatedFrom.SERVICE_API,
created_by_role=CreatorUserRole.ACCOUNT,
created_by="account-1",
@@ -148,16 +114,6 @@ def _make_workflow_app_log() -> WorkflowAppLog:
return log
def _make_workflow_log_page() -> dict[str, object]:
return {
"page": 1,
"limit": 20,
"total": 1,
"has_more": False,
"data": [LogView(_make_workflow_app_log(), _default_log_details())],
}
def _make_app_model(
*,
app_id: str = "app-1",
@@ -177,6 +133,43 @@ def _make_end_user(user_id: str = "end-user-1") -> EndUser:
return end_user
def _bind_sqlite_database(
monkeypatch: pytest.MonkeyPatch,
sqlite_engine: Engine,
sqlite_session: Session,
) -> None:
"""Bind controller- and model-owned database access to the test engine."""
database = SimpleNamespace(engine=sqlite_engine, session=sqlite_session)
monkeypatch.setattr(sys.modules["controllers.service_api.app.workflow"], "db", database)
monkeypatch.setattr(sys.modules["models.workflow"], "db", database)
def _persist_workflow_log(
sqlite_session: Session,
*,
tenant_id: str,
app_id: str,
) -> None:
workflow_run_id = "log-run-1"
sqlite_session.add_all(
[
_make_workflow_run(
run_id=workflow_run_id,
tenant_id=tenant_id,
app_id=app_id,
created_at=datetime(2026, 1, 1, 1, tzinfo=UTC),
finished_at=datetime(2026, 1, 1, 1, 0, 2, tzinfo=UTC),
),
_make_workflow_app_log(
tenant_id=tenant_id,
app_id=app_id,
workflow_run_id=workflow_run_id,
),
]
)
sqlite_session.commit()
def _expected_workflow_log_pagination_payload() -> dict[str, object]:
return {
"page": 1,
@@ -195,16 +188,16 @@ def _expected_workflow_log_pagination_payload() -> dict[str, object]:
"elapsed_time": 0.1,
"total_tokens": 10,
"total_steps": 1,
"created_at": 1767229200,
"finished_at": 1767229202,
"created_at": int(datetime(2026, 1, 1, 1).timestamp()),
"finished_at": int(datetime(2026, 1, 1, 1, 0, 2).timestamp()),
"exceptions_count": 0,
},
"details": {"trigger_metadata": {"node": "answer", "latency": 1.25}},
"details": None,
"created_from": "service-api",
"created_by_role": "account",
"created_by_account": None,
"created_by_end_user": None,
"created_at": 1767229203,
"created_at": int(datetime(2026, 1, 1, 1, 0, 3).timestamp()),
}
],
}
@@ -364,15 +357,15 @@ class TestWorkflowAppService:
assert hasattr(WorkflowAppService, "get_paginate_workflow_app_logs")
assert callable(WorkflowAppService.get_paginate_workflow_app_logs)
@patch.object(WorkflowAppService, "get_paginate_workflow_app_logs")
def test_get_paginate_workflow_app_logs_returns_pagination(self, mock_get_logs):
"""Test get_paginate_workflow_app_logs returns paginated result."""
pagination = _make_workflow_log_page()
mock_get_logs.return_value = pagination
@pytest.mark.parametrize("sqlite_session", [(WorkflowAppLog,)], indirect=True)
def test_get_paginate_workflow_app_logs_returns_pagination(self, sqlite_session: Session):
"""Test pagination returns committed logs scoped to the requested app."""
log = _make_workflow_app_log()
sqlite_session.add(log)
sqlite_session.commit()
service = WorkflowAppService()
result = service.get_paginate_workflow_app_logs(
session=Mock(),
session=sqlite_session,
app_model=_make_app_model(),
keyword=None,
status=None,
@@ -384,7 +377,11 @@ class TestWorkflowAppService:
created_by_account=None,
)
assert result == pagination
assert result["page"] == 1
assert result["limit"] == 20
assert result["total"] == 1
assert result["has_more"] is False
assert [item.id for item in result["data"]] == [log.id]
class TestWorkflowExecutionStatus:
@@ -409,8 +406,9 @@ class TestWorkflowExecutionStatus:
class TestAppGenerateServiceWorkflow:
"""Test AppGenerateService workflow integration."""
@pytest.mark.parametrize("sqlite_session", [()], indirect=True)
@patch.object(AppGenerateService, "generate")
def test_generate_accepts_workflow_args(self, mock_generate: MagicMock):
def test_generate_accepts_workflow_args(self, mock_generate: MagicMock, sqlite_session: Session):
"""Test generate accepts workflow-specific args."""
mock_generate.return_value = {"result": "success"}
@@ -419,15 +417,17 @@ class TestAppGenerateServiceWorkflow:
user=_make_end_user(),
args={"inputs": {"key": "value"}, "workflow_id": "workflow_123"},
invoke_from=InvokeFrom.SERVICE_API,
session=MagicMock(),
session=sqlite_session,
streaming=False,
)
assert result == {"result": "success"}
mock_generate.assert_called_once()
assert mock_generate.call_args.kwargs["session"] is sqlite_session
@pytest.mark.parametrize("sqlite_session", [()], indirect=True)
@patch.object(AppGenerateService, "generate")
def test_generate_raises_workflow_not_found_error(self, mock_generate: MagicMock):
def test_generate_raises_workflow_not_found_error(self, mock_generate: MagicMock, sqlite_session: Session):
"""Test generate raises WorkflowNotFoundError."""
mock_generate.side_effect = WorkflowNotFoundError("Workflow not found")
@@ -437,12 +437,13 @@ class TestAppGenerateServiceWorkflow:
user=_make_end_user(),
args={"workflow_id": "invalid_id"},
invoke_from=InvokeFrom.SERVICE_API,
session=MagicMock(),
session=sqlite_session,
streaming=False,
)
@pytest.mark.parametrize("sqlite_session", [()], indirect=True)
@patch.object(AppGenerateService, "generate")
def test_generate_raises_is_draft_workflow_error(self, mock_generate: MagicMock):
def test_generate_raises_is_draft_workflow_error(self, mock_generate: MagicMock, sqlite_session: Session):
"""Test generate raises IsDraftWorkflowError."""
mock_generate.side_effect = IsDraftWorkflowError("Workflow is draft")
@@ -452,12 +453,13 @@ class TestAppGenerateServiceWorkflow:
user=_make_end_user(),
args={"workflow_id": "draft_workflow"},
invoke_from=InvokeFrom.SERVICE_API,
session=MagicMock(),
session=sqlite_session,
streaming=False,
)
@pytest.mark.parametrize("sqlite_session", [()], indirect=True)
@patch.object(AppGenerateService, "generate")
def test_generate_supports_streaming_mode(self, mock_generate: MagicMock):
def test_generate_supports_streaming_mode(self, mock_generate: MagicMock, sqlite_session: Session):
"""Test generate supports streaming response mode."""
mock_stream = Mock()
mock_generate.return_value = mock_stream
@@ -467,7 +469,7 @@ class TestAppGenerateServiceWorkflow:
user=_make_end_user(),
args={"inputs": {}, "response_mode": "streaming"},
invoke_from=InvokeFrom.SERVICE_API,
session=MagicMock(),
session=sqlite_session,
streaming=True,
)
@@ -499,19 +501,23 @@ class TestWorkflowRunRepository:
assert hasattr(DifyAPIRepositoryFactory, "create_api_workflow_run_repository")
@patch("repositories.factory.DifyAPIRepositoryFactory.create_api_workflow_run_repository")
def test_workflow_run_repository_get_by_id(self, mock_factory):
"""Test workflow run repository get_workflow_run_by_id method."""
@pytest.mark.parametrize("sqlite_session", [(WorkflowRun,)], indirect=True)
def test_workflow_run_repository_get_by_id(self, sqlite_engine: Engine, sqlite_session: Session):
"""Test repository lookup against committed tenant-scoped state."""
run = _make_workflow_run(run_id=str(uuid.uuid4()))
mock_factory.return_value = _WorkflowRunRepositoryStub(run=run)
sqlite_session.add(run)
sqlite_session.commit()
from repositories.factory import DifyAPIRepositoryFactory
repo = DifyAPIRepositoryFactory.create_api_workflow_run_repository(sessionmaker())
repo = DifyAPIRepositoryFactory.create_api_workflow_run_repository(
sessionmaker(bind=sqlite_engine, expire_on_commit=False)
)
result = repo.get_workflow_run_by_id(tenant_id="tenant_123", app_id="app_456", run_id="run_789")
result = repo.get_workflow_run_by_id(tenant_id="tenant-1", app_id="app-1", run_id=run.id)
assert result == run
assert result is not None
assert result.id == run.id
assert repo.get_workflow_run_by_id(tenant_id="other-tenant", app_id="app-1", run_id=run.id) is None
class TestWorkflowRunDetailApi:
@@ -524,16 +530,17 @@ class TestWorkflowRunDetailApi:
with pytest.raises(NotWorkflowAppError):
handler(api, app_model=app_model, workflow_run_id="run")
def test_success(self, monkeypatch: pytest.MonkeyPatch) -> None:
run = _make_workflow_run(run_id="run")
repo = _WorkflowRunRepositoryStub(run=run)
workflow_module = sys.modules["controllers.service_api.app.workflow"]
monkeypatch.setattr(workflow_module, "db", _DbStub())
monkeypatch.setattr(
DifyAPIRepositoryFactory,
"create_api_workflow_run_repository",
lambda *_args, **_kwargs: repo,
)
@pytest.mark.parametrize("sqlite_session", [(WorkflowRun,)], indirect=True)
def test_success(
self,
monkeypatch: pytest.MonkeyPatch,
sqlite_engine: Engine,
sqlite_session: Session,
) -> None:
run = _make_workflow_run(run_id="run", tenant_id="t1", app_id="a1")
sqlite_session.add(run)
sqlite_session.commit()
_bind_sqlite_database(monkeypatch, sqlite_engine, sqlite_session)
api = WorkflowRunDetailApi()
handler = unwrap(api.get)
@@ -546,7 +553,8 @@ class TestWorkflowRunDetailApi:
class TestWorkflowRunApi:
def test_not_workflow_app(self, app: Flask) -> None:
@pytest.mark.parametrize("sqlite_session", [()], indirect=True)
def test_not_workflow_app(self, app: Flask, sqlite_session: Session) -> None:
api = WorkflowRunApi()
handler = unwrap(api.post)
app_model = _make_app_model(mode=AppMode.CHAT)
@@ -554,9 +562,10 @@ class TestWorkflowRunApi:
with app.test_request_context("/workflows/run", method="POST", json={"inputs": {}}):
with pytest.raises(NotWorkflowAppError):
handler(api, session=Mock(), app_model=app_model, end_user=end_user)
handler(api, session=sqlite_session, app_model=app_model, end_user=end_user)
def test_rate_limit(self, app: Flask, monkeypatch: pytest.MonkeyPatch) -> None:
@pytest.mark.parametrize("sqlite_session", [()], indirect=True)
def test_rate_limit(self, app: Flask, monkeypatch: pytest.MonkeyPatch, sqlite_session: Session) -> None:
monkeypatch.setattr(
AppGenerateService,
"generate",
@@ -570,7 +579,7 @@ class TestWorkflowRunApi:
with app.test_request_context("/workflows/run", method="POST", json={"inputs": {}}):
with pytest.raises(InvokeRateLimitHttpError):
handler(api, session=Mock(), app_model=app_model, end_user=end_user)
handler(api, session=sqlite_session, app_model=app_model, end_user=end_user)
def test_sandbox_billing_does_not_gate_default_workflow_run(
self, app: Flask, monkeypatch: pytest.MonkeyPatch
@@ -680,7 +689,8 @@ class TestWorkflowRunByIdApi:
else:
billing_get_info.assert_not_called()
def test_not_found(self, app: Flask, monkeypatch: pytest.MonkeyPatch) -> None:
@pytest.mark.parametrize("sqlite_session", [()], indirect=True)
def test_not_found(self, app: Flask, monkeypatch: pytest.MonkeyPatch, sqlite_session: Session) -> None:
workflow_module = sys.modules["controllers.service_api.app.workflow"]
monkeypatch.setattr(workflow_module.dify_config, "BILLING_ENABLED", False)
monkeypatch.setattr(
@@ -696,9 +706,10 @@ class TestWorkflowRunByIdApi:
with app.test_request_context("/workflows/1/run", method="POST", json={"inputs": {}}):
with pytest.raises(NotFound):
handler(api, session=Mock(), app_model=app_model, end_user=end_user, workflow_id="w1")
handler(api, session=sqlite_session, app_model=app_model, end_user=end_user, workflow_id="w1")
def test_draft_workflow(self, app: Flask, monkeypatch: pytest.MonkeyPatch) -> None:
@pytest.mark.parametrize("sqlite_session", [()], indirect=True)
def test_draft_workflow(self, app: Flask, monkeypatch: pytest.MonkeyPatch, sqlite_session: Session) -> None:
workflow_module = sys.modules["controllers.service_api.app.workflow"]
monkeypatch.setattr(workflow_module.dify_config, "BILLING_ENABLED", False)
monkeypatch.setattr(
@@ -714,7 +725,7 @@ class TestWorkflowRunByIdApi:
with app.test_request_context("/workflows/1/run", method="POST", json={"inputs": {}}):
with pytest.raises(BadRequest):
handler(api, session=Mock(), app_model=app_model, end_user=end_user, workflow_id="w1")
handler(api, session=sqlite_session, app_model=app_model, end_user=end_user, workflow_id="w1")
class TestWorkflowTaskStopApi:
@@ -748,28 +759,16 @@ class TestWorkflowTaskStopApi:
class TestWorkflowAppLogApi:
def test_success(self, app: Flask, monkeypatch: pytest.MonkeyPatch) -> None:
workflow_module = sys.modules["controllers.service_api.app.workflow"]
workflow_model_module = sys.modules["models.workflow"]
monkeypatch.setattr(workflow_module, "db", _DbStub())
monkeypatch.setattr(workflow_model_module, "db", _DbStub())
monkeypatch.setattr(workflow_module, "sessionmaker", _SessionMakerStub)
monkeypatch.setattr(
WorkflowAppService,
"get_paginate_workflow_app_logs",
lambda *_args, **_kwargs: _make_workflow_log_page(),
)
monkeypatch.setattr(
DifyAPIRepositoryFactory,
"create_api_workflow_run_repository",
lambda *_args, **_kwargs: _WorkflowRunRepositoryStub(
run=_make_workflow_run(
run_id="log-run-1",
created_at=datetime(2026, 1, 1, 1, tzinfo=UTC),
finished_at=datetime(2026, 1, 1, 1, 0, 2, tzinfo=UTC),
)
),
)
@pytest.mark.parametrize("sqlite_session", [(WorkflowRun, WorkflowAppLog, Account)], indirect=True)
def test_success(
self,
app: Flask,
monkeypatch: pytest.MonkeyPatch,
sqlite_engine: Engine,
sqlite_session: Session,
) -> None:
_persist_workflow_log(sqlite_session, tenant_id="tenant-1", app_id="a1")
_bind_sqlite_database(monkeypatch, sqlite_engine, sqlite_session)
api = WorkflowAppLogApi()
handler = unwrap(api.get)
@@ -803,18 +802,24 @@ class TestWorkflowRunDetailApiGet:
and we call the unwrapped method directly in tests.
"""
@patch("controllers.service_api.app.workflow.DifyAPIRepositoryFactory")
@patch("controllers.service_api.app.workflow.db")
@pytest.mark.parametrize("sqlite_session", [(WorkflowRun,)], indirect=True)
def test_get_workflow_run_success(
self,
mock_db,
mock_repo_factory,
app: Flask,
workflow_app: App,
monkeypatch: pytest.MonkeyPatch,
sqlite_engine: Engine,
sqlite_session: Session,
):
"""Test successful workflow run detail retrieval."""
run = _make_workflow_run(run_id="run-1")
mock_repo_factory.create_api_workflow_run_repository.return_value = _WorkflowRunRepositoryStub(run=run)
run = _make_workflow_run(
run_id="run-1",
tenant_id=workflow_app.tenant_id,
app_id=workflow_app.id,
)
sqlite_session.add(run)
sqlite_session.commit()
_bind_sqlite_database(monkeypatch, sqlite_engine, sqlite_session)
from controllers.service_api.app.workflow import WorkflowRunDetailApi
@@ -834,13 +839,12 @@ class TestWorkflowRunDetailApiGet:
"error": None,
"total_steps": 1,
"total_tokens": 10,
"created_at": 1767225600,
"finished_at": 1767225600,
"created_at": int(datetime(2026, 1, 1).timestamp()),
"finished_at": int(datetime(2026, 1, 1).timestamp()),
"elapsed_time": 0.1,
}
@patch("controllers.service_api.app.workflow.db")
def test_get_workflow_run_wrong_app_mode(self, mock_db, app: Flask):
def test_get_workflow_run_wrong_app_mode(self, app: Flask):
"""Test NotWorkflowAppError when app mode is not workflow or advanced_chat."""
from controllers.service_api.app.workflow import WorkflowRunDetailApi
@@ -902,46 +906,23 @@ class TestWorkflowAppLogApiGet:
``get`` is wrapped by ``@validate_app_token``.
"""
@patch("controllers.service_api.app.workflow.WorkflowAppService")
@patch("controllers.service_api.app.workflow.db")
@pytest.mark.parametrize("sqlite_session", [(WorkflowRun, WorkflowAppLog, Account)], indirect=True)
def test_get_workflow_logs_success(
self,
mock_db,
mock_wf_svc_cls,
app: Flask,
workflow_app: App,
monkeypatch: pytest.MonkeyPatch,
sqlite_engine: Engine,
sqlite_session: Session,
):
"""Test successful workflow log retrieval."""
mock_svc_instance = Mock()
mock_svc_instance.get_paginate_workflow_app_logs.return_value = _make_workflow_log_page()
mock_wf_svc_cls.return_value = mock_svc_instance
mock_repo = _WorkflowRunRepositoryStub(
run=_make_workflow_run(
run_id="log-run-1",
created_at=datetime(2026, 1, 1, 1, tzinfo=UTC),
finished_at=datetime(2026, 1, 1, 1, 0, 2, tzinfo=UTC),
)
)
# Mock sessionmaker(...).begin() context manager
mock_db.engine = object()
mock_db.session.get.return_value = None
_persist_workflow_log(sqlite_session, tenant_id=workflow_app.tenant_id, app_id=workflow_app.id)
_bind_sqlite_database(monkeypatch, sqlite_engine, sqlite_session)
from controllers.service_api.app.workflow import WorkflowAppLogApi
with app.test_request_context(
"/workflows/logs?page=1&limit=20",
method="GET",
):
with (
patch("controllers.service_api.app.workflow.sessionmaker", _SessionMakerStub),
patch("models.workflow.db", _DbStub()),
patch(
"repositories.factory.DifyAPIRepositoryFactory.create_api_workflow_run_repository",
return_value=mock_repo,
),
):
api = WorkflowAppLogApi()
result = unwrap(api.get)(api, app_model=workflow_app)
with app.test_request_context("/workflows/logs?page=1&limit=20", method="GET"):
api = WorkflowAppLogApi()
result = unwrap(api.get)(api, app_model=workflow_app)
assert result == _expected_workflow_log_pagination_payload()
@@ -7,18 +7,57 @@ Service API controller tests.
"""
import uuid
from collections.abc import Iterator
from dataclasses import dataclass
from unittest.mock import Mock
import pytest
from flask import Flask
from sqlalchemy import Engine
from sqlalchemy.orm import Session
from core.rag.index_processor.constant.index_type import IndexStructureType
from models.account import TenantStatus
from models.account import Account, Tenant, TenantAccountJoin, TenantAccountRole, TenantStatus
from models.base import TypeBase
from models.model import App, AppMode, EndUser
from tests.unit_tests.conftest import (
setup_mock_dataset_owner_execute_result,
setup_mock_tenant_owner_execute_result,
)
@dataclass(frozen=True)
class ServiceApiIdentity:
"""Persisted owner identity for service-API authentication tests."""
session: Session
tenant: Tenant
account: Account
membership: TenantAccountJoin
@pytest.fixture
def service_api_identity(sqlite_engine: Engine) -> Iterator[ServiceApiIdentity]:
"""Yield an isolated SQLite session with a real active tenant owner."""
TypeBase.metadata.create_all(
sqlite_engine,
tables=[Account.__table__, Tenant.__table__, TenantAccountJoin.__table__],
)
with Session(sqlite_engine, expire_on_commit=False) as session:
tenant = Tenant(name="Service API Workspace")
tenant.id = str(uuid.uuid4())
account = Account(name="Service API Owner", email=f"owner-{tenant.id}@example.com")
account.id = str(uuid.uuid4())
membership = TenantAccountJoin(
tenant_id=tenant.id,
account_id=account.id,
role=TenantAccountRole.OWNER,
)
account._current_tenant = tenant
session.add_all([tenant, account, membership])
session.commit()
yield ServiceApiIdentity(
session=session,
tenant=tenant,
account=account,
membership=membership,
)
@pytest.fixture
@@ -110,40 +149,6 @@ def mock_dataset_api_token(mock_tenant_id):
return token
class AuthenticationMocker:
"""
Helper class to set up common authentication mocking patterns.
Usage:
auth_mocker = AuthenticationMocker()
with auth_mocker.mock_app_auth(mock_api_token, mock_app_model, mock_tenant):
# Test code here
"""
@staticmethod
def setup_db_queries(mock_db, mock_app, mock_tenant, mock_account=None):
"""Configure mock_db to return app and tenant via session.get()."""
mock_db.session.get.side_effect = [mock_app, mock_tenant]
if mock_account:
setup_mock_tenant_owner_execute_result(mock_db, mock_tenant, mock_account)
@staticmethod
def setup_dataset_auth(mock_db, mock_tenant, mock_account):
"""Configure mock_db for dataset token authentication."""
mock_ta = Mock()
mock_ta.account_id = mock_account.id
setup_mock_dataset_owner_execute_result(mock_db, mock_tenant, mock_ta)
mock_db.session.get.return_value = mock_account
@pytest.fixture
def auth_mocker():
"""Provide an AuthenticationMocker instance."""
return AuthenticationMocker()
@pytest.fixture
def mock_dataset():
"""Create a mock Dataset model."""
@@ -0,0 +1,55 @@
"""State-based checks for shared service-API authentication fixtures."""
from uuid import uuid4
from sqlalchemy import select
from models.account import Account, Tenant, TenantAccountJoin, TenantAccountRole
from tests.unit_tests.conftest import (
persist_service_api_dataset_owner,
persist_service_api_tenant_owner,
)
from tests.unit_tests.controllers.service_api.conftest import ServiceApiIdentity
def test_service_api_identity_persists_tenant_scoped_owner(service_api_identity: ServiceApiIdentity) -> None:
identity = service_api_identity
owner_row = identity.session.execute(
select(Tenant, Account)
.join(TenantAccountJoin, Tenant.id == TenantAccountJoin.tenant_id)
.join(Account, TenantAccountJoin.account_id == Account.id)
.where(
Tenant.id == identity.tenant.id,
TenantAccountJoin.role == TenantAccountRole.OWNER,
)
).one()
assert owner_row == (identity.tenant, identity.account)
assert identity.account.current_tenant is identity.tenant
def test_shared_helpers_persist_real_app_and_dataset_owner_rows(service_api_identity: ServiceApiIdentity) -> None:
session = service_api_identity.session
app_tenant = Tenant(name="App Workspace")
app_tenant.id = str(uuid4())
app_owner = Account(name="App Owner", email=f"app-owner-{app_tenant.id}@example.com")
app_owner.id = str(uuid4())
app_membership = persist_service_api_tenant_owner(session, app_tenant, app_owner)
dataset_tenant = Tenant(name="Dataset Workspace")
dataset_tenant.id = str(uuid4())
dataset_membership = TenantAccountJoin(
tenant_id=dataset_tenant.id,
account_id=service_api_identity.account.id,
role=TenantAccountRole.OWNER,
)
persist_service_api_dataset_owner(session, dataset_tenant, dataset_membership)
assert session.get(TenantAccountJoin, app_membership.id) is app_membership
assert session.execute(
select(Tenant, TenantAccountJoin)
.join(TenantAccountJoin, Tenant.id == TenantAccountJoin.tenant_id)
.where(Tenant.id == dataset_tenant.id)
).one() == (dataset_tenant, dataset_membership)
@@ -574,6 +574,27 @@ def test_console_account_avatar_query_param_renders_as_query(monkeypatch: pytest
assert params["avatar"]["required"] is True
def test_console_agent_debug_conversation_refresh_body_is_optional(monkeypatch: pytest.MonkeyPatch):
from configs import dify_config
from controllers.console import bp as console_bp
monkeypatch.setattr(dify_config, "SWAGGER_UI_ENABLED", True)
app = Flask(__name__)
app.config["TESTING"] = True
app.register_blueprint(console_bp)
payload = app.test_client().get("/console/api/openapi.json").get_json()
operation = payload["paths"]["/agent/{agent_id}/debug-conversation/refresh"]["post"]
request_body = operation["requestBody"]
assert request_body["required"] is False
assert request_body["content"]["application/json"]["schema"] == {
"$ref": "#/components/schemas/AgentDebugConversationRefreshPayload"
}
assert "AgentDebugConversationRefreshPayload" in payload["components"]["schemas"]
def test_console_member_invite_documents_bad_request_response(monkeypatch: pytest.MonkeyPatch):
from configs import dify_config
from controllers.console import bp as console_bp
@@ -0,0 +1,67 @@
from unittest.mock import MagicMock, patch
from configs import dify_config
from controllers.web import site as site_module
from extensions.storage.storage_type import StorageType
from models.model import IconType, Site
def test_build_site_icon_url_uses_s3_presigned_url() -> None:
site = MagicMock(spec=Site)
site.icon_type = IconType.IMAGE
site.icon = "11111111-1111-4111-8111-111111111111"
with (
patch.object(dify_config, "EDITION", "CLOUD"),
patch.object(dify_config, "STORAGE_TYPE", StorageType.S3),
patch.object(site_module, "db") as mock_db,
patch.object(site_module, "FileService") as mock_file_service,
patch.object(site_module, "build_icon_url") as mock_build_icon_url,
):
mock_file_service.return_value.get_file_presigned_url.return_value = (
"https://s3.example.com/icon.png?signature=test"
)
result = site_module._build_site_icon_url(site=site, tenant_id="tenant-id")
assert result == "https://s3.example.com/icon.png?signature=test"
mock_file_service.assert_called_once_with(mock_db.engine)
mock_file_service.return_value.get_file_presigned_url.assert_called_once_with(
file_id="11111111-1111-4111-8111-111111111111",
tenant_id="tenant-id",
)
mock_build_icon_url.assert_not_called()
def test_build_site_icon_url_keeps_preview_url_for_self_hosted_s3() -> None:
site = MagicMock(spec=Site)
site.icon_type = IconType.IMAGE
site.icon = "11111111-1111-4111-8111-111111111111"
with (
patch.object(dify_config, "EDITION", "SELF_HOSTED"),
patch.object(dify_config, "STORAGE_TYPE", StorageType.S3),
patch.object(site_module, "FileService") as mock_file_service,
patch.object(site_module, "build_icon_url", return_value="https://api.example.com/files/icon/file-preview"),
):
result = site_module._build_site_icon_url(site=site, tenant_id="tenant-id")
assert result == "https://api.example.com/files/icon/file-preview"
mock_file_service.assert_not_called()
def test_build_site_icon_url_keeps_preview_url_for_non_s3_storage() -> None:
site = MagicMock(spec=Site)
site.icon_type = IconType.IMAGE
site.icon = "11111111-1111-4111-8111-111111111111"
with (
patch.object(dify_config, "EDITION", "CLOUD"),
patch.object(dify_config, "STORAGE_TYPE", StorageType.LOCAL),
patch.object(site_module, "FileService") as mock_file_service,
patch.object(site_module, "build_icon_url", return_value="https://api.example.com/files/icon/file-preview"),
):
result = site_module._build_site_icon_url(site=site, tenant_id="tenant-id")
assert result == "https://api.example.com/files/icon/file-preview"
mock_file_service.assert_not_called()
@@ -1,459 +1,120 @@
"""Test conversation variable handling in AdvancedChatAppRunner."""
"""SQLite-backed conversation-variable synchronization tests for AdvancedChatAppRunner."""
from unittest.mock import MagicMock, patch
from uuid import uuid4
from unittest.mock import MagicMock
import pytest
from sqlalchemy import select
from sqlalchemy.orm import Session
from core.app.apps.advanced_chat import app_runner as app_runner_module
from core.app.apps.advanced_chat.app_runner import AdvancedChatAppRunner
from core.app.entities.app_invoke_entities import AdvancedChatAppGenerateEntity, InvokeFrom
from factories import variable_factory
from graphon.variables import SegmentType
from models import ConversationVariable, Workflow
from models import ConversationVariable
MINIMAL_GRAPH = {
"nodes": [
APP_ID = "11111111-1111-1111-1111-111111111111"
CONVERSATION_ID = "22222222-2222-2222-2222-222222222222"
OTHER_CONVERSATION_ID = "22222222-2222-2222-2222-222222222223"
VAR_1_ID = "33333333-3333-3333-3333-333333333333"
VAR_2_ID = "33333333-3333-3333-3333-333333333334"
def _variable(variable_id: str, name: str, value: str):
return variable_factory.build_conversation_variable_from_mapping(
{
"id": "start",
"data": {
"type": "start",
"title": "Start",
},
"id": variable_id,
"name": name,
"value_type": SegmentType.STRING,
"value": value,
}
],
"edges": [],
}
)
def _patch_create_session(mock_session: MagicMock):
session_context = MagicMock()
session_context.__enter__.return_value = mock_session
session_context.__exit__.return_value = False
mock_session.begin.return_value.__enter__.return_value = mock_session
mock_session.begin.return_value.__exit__.return_value = False
return patch("core.app.apps.advanced_chat.app_runner.create_session", return_value=session_context)
def _runner(workflow_variables: list[object]) -> AdvancedChatAppRunner:
workflow = MagicMock()
workflow.conversation_variables = workflow_variables
conversation = MagicMock(app_id=APP_ID, id=CONVERSATION_ID)
return AdvancedChatAppRunner(
application_generate_entity=MagicMock(),
queue_manager=MagicMock(),
conversation=conversation,
message=MagicMock(),
dialogue_count=1,
variable_loader=MagicMock(),
workflow=workflow,
system_user_id="44444444-4444-4444-4444-444444444444",
app=MagicMock(),
workflow_execution_repository=MagicMock(),
workflow_node_execution_repository=MagicMock(),
)
class TestAdvancedChatAppRunnerConversationVariables:
"""Test that AdvancedChatAppRunner correctly handles conversation variables."""
def _bind_runner_sessions(monkeypatch: pytest.MonkeyPatch, sqlite_session: Session) -> None:
engine = sqlite_session.get_bind()
monkeypatch.setattr(
app_runner_module,
"create_session",
lambda: Session(engine, expire_on_commit=False),
)
def test_missing_conversation_variables_are_added(self):
"""Test that new conversation variables added to workflow are created for existing conversations."""
# Setup
app_id = str(uuid4())
conversation_id = str(uuid4())
workflow_id = str(uuid4())
# Create workflow with two conversation variables
workflow_vars = [
variable_factory.build_conversation_variable_from_mapping(
{
"id": "var1",
"name": "existing_var",
"value_type": SegmentType.STRING,
"value": "default1",
}
),
variable_factory.build_conversation_variable_from_mapping(
{
"id": "var2",
"name": "new_var",
"value_type": SegmentType.STRING,
"value": "default2",
}
),
]
# Mock workflow with conversation variables
mock_workflow = MagicMock(spec=Workflow)
mock_workflow.conversation_variables = workflow_vars
mock_workflow.tenant_id = str(uuid4())
mock_workflow.app_id = app_id
mock_workflow.id = workflow_id
mock_workflow.type = "chat"
mock_workflow.graph_dict = MINIMAL_GRAPH
mock_workflow.environment_variables = []
# Create existing conversation variable (only var1 exists in DB)
existing_db_var = MagicMock(spec=ConversationVariable)
existing_db_var.id = "var1"
existing_db_var.app_id = app_id
existing_db_var.conversation_id = conversation_id
existing_db_var.to_variable = MagicMock(return_value=workflow_vars[0])
# Mock conversation and message
mock_conversation = MagicMock()
mock_conversation.app_id = app_id
mock_conversation.id = conversation_id
mock_message = MagicMock()
mock_message.id = str(uuid4())
# Mock app config
mock_app_config = MagicMock()
mock_app_config.app_id = app_id
mock_app_config.workflow_id = workflow_id
mock_app_config.tenant_id = str(uuid4())
# Mock app generate entity
mock_app_generate_entity = MagicMock(spec=AdvancedChatAppGenerateEntity)
mock_app_generate_entity.app_config = mock_app_config
mock_app_generate_entity.inputs = {}
mock_app_generate_entity.query = "test query"
mock_app_generate_entity.files = []
mock_app_generate_entity.user_id = str(uuid4())
mock_app_generate_entity.invoke_from = InvokeFrom.SERVICE_API
mock_app_generate_entity.workflow_run_id = str(uuid4())
mock_app_generate_entity.task_id = str(uuid4())
mock_app_generate_entity.call_depth = 0
mock_app_generate_entity.single_iteration_run = None
mock_app_generate_entity.single_loop_run = None
mock_app_generate_entity.extras = {}
mock_app_generate_entity.trace_manager = None
# Create runner
runner = AdvancedChatAppRunner(
application_generate_entity=mock_app_generate_entity,
queue_manager=MagicMock(),
conversation=mock_conversation,
message=mock_message,
dialogue_count=1,
variable_loader=MagicMock(),
workflow=mock_workflow,
system_user_id=str(uuid4()),
app=MagicMock(),
workflow_execution_repository=MagicMock(),
workflow_node_execution_repository=MagicMock(),
def _persist_variable(session: Session, *, variable: object, conversation_id: str = CONVERSATION_ID) -> None:
session.add(
ConversationVariable.from_variable(
app_id=APP_ID,
conversation_id=conversation_id,
variable=variable,
)
)
session.commit()
# Mock database session
mock_session = MagicMock(spec=Session)
# First query returns only existing variable
mock_scalars_result = MagicMock()
mock_scalars_result.all.return_value = [existing_db_var]
mock_session.scalars.return_value = mock_scalars_result
@pytest.mark.parametrize("sqlite_session", [(ConversationVariable,)], indirect=True)
def test_missing_conversation_variables_are_added(monkeypatch: pytest.MonkeyPatch, sqlite_session: Session) -> None:
existing_variable = _variable(VAR_1_ID, "existing_var", "default1")
new_variable = _variable(VAR_2_ID, "new_var", "default2")
_persist_variable(sqlite_session, variable=existing_variable)
_persist_variable(sqlite_session, variable=new_variable, conversation_id=OTHER_CONVERSATION_ID)
_bind_runner_sessions(monkeypatch, sqlite_session)
# Track what gets added to session
added_items = []
variables = _runner([existing_variable, new_variable])._initialize_conversation_variables()
def track_add_all(items):
added_items.extend(items)
assert [variable.id for variable in variables] == [VAR_1_ID, VAR_2_ID]
persisted = sqlite_session.scalars(
select(ConversationVariable)
.where(ConversationVariable.conversation_id == CONVERSATION_ID)
.order_by(ConversationVariable.id)
).all()
assert [variable.id for variable in persisted] == [VAR_1_ID, VAR_2_ID]
mock_session.add_all.side_effect = track_add_all
# Patch the necessary components
with (
_patch_create_session(mock_session),
patch("core.app.apps.advanced_chat.app_runner.select") as mock_select,
patch.object(runner, "_init_graph") as mock_init_graph,
patch.object(
runner,
"handle_input_moderation",
return_value=(False, mock_app_generate_entity.inputs, mock_app_generate_entity.query),
),
patch.object(runner, "handle_annotation_reply", return_value=False),
patch("core.app.apps.advanced_chat.app_runner.WorkflowEntry") as mock_workflow_entry_class,
patch("core.app.apps.advanced_chat.app_runner.GraphRuntimeState") as mock_graph_runtime_state_class,
patch("core.app.apps.advanced_chat.app_runner.redis_client") as mock_redis_client,
patch("core.app.apps.advanced_chat.app_runner.RedisChannel") as mock_redis_channel_class,
):
# Mock GraphRuntimeState to accept the variable pool
mock_graph_runtime_state_class.return_value = MagicMock()
@pytest.mark.parametrize("sqlite_session", [(ConversationVariable,)], indirect=True)
def test_no_variables_creates_all(monkeypatch: pytest.MonkeyPatch, sqlite_session: Session) -> None:
workflow_variables = [
_variable(VAR_1_ID, "var1", "default1"),
_variable(VAR_2_ID, "var2", "default2"),
]
_bind_runner_sessions(monkeypatch, sqlite_session)
# Mock graph initialization
mock_init_graph.return_value = MagicMock()
variables = _runner(workflow_variables)._initialize_conversation_variables()
# Mock workflow entry
mock_workflow_entry = MagicMock()
mock_workflow_entry.run.return_value = iter([]) # Empty generator
mock_workflow_entry_class.return_value = mock_workflow_entry
assert [variable.id for variable in variables] == [VAR_1_ID, VAR_2_ID]
persisted = sqlite_session.scalars(select(ConversationVariable).order_by(ConversationVariable.id)).all()
assert [variable.id for variable in persisted] == [VAR_1_ID, VAR_2_ID]
# Run the method
runner.run()
# Verify that the missing variable was added
assert len(added_items) == 1, "Should have added exactly one missing variable"
@pytest.mark.parametrize("sqlite_session", [(ConversationVariable,)], indirect=True)
def test_all_variables_exist_no_changes(monkeypatch: pytest.MonkeyPatch, sqlite_session: Session) -> None:
workflow_variables = [
_variable(VAR_1_ID, "var1", "default1"),
_variable(VAR_2_ID, "var2", "default2"),
]
for variable in workflow_variables:
_persist_variable(sqlite_session, variable=variable)
_bind_runner_sessions(monkeypatch, sqlite_session)
# Check that the added item is the missing variable (var2)
added_var = added_items[0]
assert hasattr(added_var, "id"), "Added item should be a ConversationVariable"
# Note: Since we're mocking ConversationVariable.from_variable,
# we can't directly check the id, but we can verify add_all was called
assert mock_session.add_all.called, "Session add_all should have been called"
variables = _runner(workflow_variables)._initialize_conversation_variables()
def test_no_variables_creates_all(self):
"""Test that all conversation variables are created when none exist in DB."""
# Setup
app_id = str(uuid4())
conversation_id = str(uuid4())
workflow_id = str(uuid4())
# Create workflow with conversation variables
workflow_vars = [
variable_factory.build_conversation_variable_from_mapping(
{
"id": "var1",
"name": "var1",
"value_type": SegmentType.STRING,
"value": "default1",
}
),
variable_factory.build_conversation_variable_from_mapping(
{
"id": "var2",
"name": "var2",
"value_type": SegmentType.STRING,
"value": "default2",
}
),
]
# Mock workflow
mock_workflow = MagicMock(spec=Workflow)
mock_workflow.conversation_variables = workflow_vars
mock_workflow.tenant_id = str(uuid4())
mock_workflow.app_id = app_id
mock_workflow.id = workflow_id
mock_workflow.type = "chat"
mock_workflow.graph_dict = MINIMAL_GRAPH
mock_workflow.environment_variables = []
# Mock conversation and message
mock_conversation = MagicMock()
mock_conversation.app_id = app_id
mock_conversation.id = conversation_id
mock_message = MagicMock()
mock_message.id = str(uuid4())
# Mock app config
mock_app_config = MagicMock()
mock_app_config.app_id = app_id
mock_app_config.workflow_id = workflow_id
mock_app_config.tenant_id = str(uuid4())
# Mock app generate entity
mock_app_generate_entity = MagicMock(spec=AdvancedChatAppGenerateEntity)
mock_app_generate_entity.app_config = mock_app_config
mock_app_generate_entity.inputs = {}
mock_app_generate_entity.query = "test query"
mock_app_generate_entity.files = []
mock_app_generate_entity.user_id = str(uuid4())
mock_app_generate_entity.invoke_from = InvokeFrom.SERVICE_API
mock_app_generate_entity.workflow_run_id = str(uuid4())
mock_app_generate_entity.task_id = str(uuid4())
mock_app_generate_entity.call_depth = 0
mock_app_generate_entity.single_iteration_run = None
mock_app_generate_entity.single_loop_run = None
mock_app_generate_entity.extras = {}
mock_app_generate_entity.trace_manager = None
# Create runner
runner = AdvancedChatAppRunner(
application_generate_entity=mock_app_generate_entity,
queue_manager=MagicMock(),
conversation=mock_conversation,
message=mock_message,
dialogue_count=1,
variable_loader=MagicMock(),
workflow=mock_workflow,
system_user_id=str(uuid4()),
app=MagicMock(),
workflow_execution_repository=MagicMock(),
workflow_node_execution_repository=MagicMock(),
)
# Mock database session
mock_session = MagicMock(spec=Session)
# Query returns empty list (no existing variables)
mock_scalars_result = MagicMock()
mock_scalars_result.all.return_value = []
mock_session.scalars.return_value = mock_scalars_result
# Track what gets added to session
added_items = []
def track_add_all(items):
added_items.extend(items)
mock_session.add_all.side_effect = track_add_all
# Patch the necessary components
with (
_patch_create_session(mock_session),
patch("core.app.apps.advanced_chat.app_runner.select") as mock_select,
patch.object(runner, "_init_graph") as mock_init_graph,
patch.object(
runner,
"handle_input_moderation",
return_value=(False, mock_app_generate_entity.inputs, mock_app_generate_entity.query),
),
patch.object(runner, "handle_annotation_reply", return_value=False),
patch("core.app.apps.advanced_chat.app_runner.WorkflowEntry") as mock_workflow_entry_class,
patch("core.app.apps.advanced_chat.app_runner.GraphRuntimeState") as mock_graph_runtime_state_class,
patch("core.app.apps.advanced_chat.app_runner.ConversationVariable") as mock_conv_var_class,
patch("core.app.apps.advanced_chat.app_runner.redis_client") as mock_redis_client,
patch("core.app.apps.advanced_chat.app_runner.RedisChannel") as mock_redis_channel_class,
):
# Mock ConversationVariable.from_variable to return mock objects
mock_conv_vars = []
for var in workflow_vars:
mock_cv = MagicMock()
mock_cv.id = var.id
mock_cv.to_variable.return_value = var
mock_conv_vars.append(mock_cv)
mock_conv_var_class.from_variable.side_effect = mock_conv_vars
# Mock GraphRuntimeState to accept the variable pool
mock_graph_runtime_state_class.return_value = MagicMock()
# Mock graph initialization
mock_init_graph.return_value = MagicMock()
# Mock workflow entry
mock_workflow_entry = MagicMock()
mock_workflow_entry.run.return_value = iter([]) # Empty generator
mock_workflow_entry_class.return_value = mock_workflow_entry
# Run the method
runner.run()
# Verify that all variables were created
assert len(added_items) == 2, "Should have added both variables"
assert mock_session.add_all.called, "Session add_all should have been called"
def test_all_variables_exist_no_changes(self):
"""Test that no changes are made when all variables already exist in DB."""
# Setup
app_id = str(uuid4())
conversation_id = str(uuid4())
workflow_id = str(uuid4())
# Create workflow with conversation variables
workflow_vars = [
variable_factory.build_conversation_variable_from_mapping(
{
"id": "var1",
"name": "var1",
"value_type": SegmentType.STRING,
"value": "default1",
}
),
variable_factory.build_conversation_variable_from_mapping(
{
"id": "var2",
"name": "var2",
"value_type": SegmentType.STRING,
"value": "default2",
}
),
]
# Mock workflow
mock_workflow = MagicMock(spec=Workflow)
mock_workflow.conversation_variables = workflow_vars
mock_workflow.tenant_id = str(uuid4())
mock_workflow.app_id = app_id
mock_workflow.id = workflow_id
mock_workflow.type = "chat"
mock_workflow.graph_dict = MINIMAL_GRAPH
mock_workflow.environment_variables = []
# Create existing conversation variables (both exist in DB)
existing_db_vars = []
for var in workflow_vars:
db_var = MagicMock(spec=ConversationVariable)
db_var.id = var.id
db_var.app_id = app_id
db_var.conversation_id = conversation_id
db_var.to_variable = MagicMock(return_value=var)
existing_db_vars.append(db_var)
# Mock conversation and message
mock_conversation = MagicMock()
mock_conversation.app_id = app_id
mock_conversation.id = conversation_id
mock_message = MagicMock()
mock_message.id = str(uuid4())
# Mock app config
mock_app_config = MagicMock()
mock_app_config.app_id = app_id
mock_app_config.workflow_id = workflow_id
mock_app_config.tenant_id = str(uuid4())
# Mock app generate entity
mock_app_generate_entity = MagicMock(spec=AdvancedChatAppGenerateEntity)
mock_app_generate_entity.app_config = mock_app_config
mock_app_generate_entity.inputs = {}
mock_app_generate_entity.query = "test query"
mock_app_generate_entity.files = []
mock_app_generate_entity.user_id = str(uuid4())
mock_app_generate_entity.invoke_from = InvokeFrom.SERVICE_API
mock_app_generate_entity.workflow_run_id = str(uuid4())
mock_app_generate_entity.task_id = str(uuid4())
mock_app_generate_entity.call_depth = 0
mock_app_generate_entity.single_iteration_run = None
mock_app_generate_entity.single_loop_run = None
mock_app_generate_entity.extras = {}
mock_app_generate_entity.trace_manager = None
# Create runner
runner = AdvancedChatAppRunner(
application_generate_entity=mock_app_generate_entity,
queue_manager=MagicMock(),
conversation=mock_conversation,
message=mock_message,
dialogue_count=1,
variable_loader=MagicMock(),
workflow=mock_workflow,
system_user_id=str(uuid4()),
app=MagicMock(),
workflow_execution_repository=MagicMock(),
workflow_node_execution_repository=MagicMock(),
)
# Mock database session
mock_session = MagicMock(spec=Session)
# Query returns all existing variables
mock_scalars_result = MagicMock()
mock_scalars_result.all.return_value = existing_db_vars
mock_session.scalars.return_value = mock_scalars_result
# Patch the necessary components
with (
_patch_create_session(mock_session),
patch("core.app.apps.advanced_chat.app_runner.select") as mock_select,
patch.object(runner, "_init_graph") as mock_init_graph,
patch.object(
runner,
"handle_input_moderation",
return_value=(False, mock_app_generate_entity.inputs, mock_app_generate_entity.query),
),
patch.object(runner, "handle_annotation_reply", return_value=False),
patch("core.app.apps.advanced_chat.app_runner.WorkflowEntry") as mock_workflow_entry_class,
patch("core.app.apps.advanced_chat.app_runner.GraphRuntimeState") as mock_graph_runtime_state_class,
patch("core.app.apps.advanced_chat.app_runner.redis_client") as mock_redis_client,
patch("core.app.apps.advanced_chat.app_runner.RedisChannel") as mock_redis_channel_class,
):
# Mock GraphRuntimeState to accept the variable pool
mock_graph_runtime_state_class.return_value = MagicMock()
# Mock graph initialization
mock_init_graph.return_value = MagicMock()
# Mock workflow entry
mock_workflow_entry = MagicMock()
mock_workflow_entry.run.return_value = iter([]) # Empty generator
mock_workflow_entry_class.return_value = mock_workflow_entry
# Run the method
runner.run()
# Verify that no variables were added
assert not mock_session.add_all.called, "Session add_all should not have been called"
assert [variable.id for variable in variables] == [VAR_1_ID, VAR_2_ID]
persisted = sqlite_session.scalars(select(ConversationVariable)).all()
assert len(persisted) == 2
@@ -14,7 +14,8 @@ import pytest
from core.app.apps.agent_app.app_generator import AgentAppGenerator, AgentAppGeneratorError, AgentAppNotPublishedError
from core.app.entities.app_invoke_entities import InvokeFrom
from models.agent import AgentConfigDraftType, AgentSource
from models.agent import AgentConfigDraft, AgentConfigDraftType, AgentScope, AgentSource
from models.agent_config_entities import AgentSoulConfig
_SOUL_DICT = {
"model": {
@@ -95,7 +96,7 @@ class TestResolveDebugDraft:
created_by="creator-1",
updated_by="updater-1",
)
session = _FakeScalarSession([None, SimpleNamespace(id="agent-1"), _snapshot()])
session = _FakeScalarSession([None, _snapshot()])
draft = AgentAppGenerator._resolve_debug_draft(
tenant_id="t1",
@@ -110,6 +111,77 @@ class TestResolveDebugDraft:
assert session.added == [draft]
assert session.flush_count == 1
def test_stale_workflow_only_shared_draft_is_rebased_to_active_snapshot(self):
agent = SimpleNamespace(
id="agent-1",
scope=AgentScope.WORKFLOW_ONLY,
active_config_snapshot_id="snap-2",
created_by="creator-1",
updated_by="updater-1",
)
draft = AgentConfigDraft(
id="draft-1",
tenant_id="t1",
agent_id="agent-1",
draft_type=AgentConfigDraftType.DRAFT,
account_id=None,
draft_owner_key="",
base_snapshot_id="snap-1",
config_snapshot=AgentSoulConfig.model_validate({"prompt": {"system_prompt": "old"}}),
)
active_snapshot = SimpleNamespace(
id="snap-2",
config_snapshot_dict={"prompt": {"system_prompt": "new"}},
)
session = _FakeScalarSession([draft, active_snapshot])
resolved = AgentAppGenerator._resolve_debug_draft(
tenant_id="t1",
agent=agent,
draft_type=None,
account_id=None,
session=session,
)
assert resolved is draft
assert resolved.id == "draft-1"
assert resolved.base_snapshot_id == "snap-2"
assert resolved.config_snapshot_dict["prompt"]["system_prompt"] == "new"
assert session.flush_count == 1
def test_build_draft_is_not_rebased_to_active_snapshot(self):
agent = SimpleNamespace(
id="agent-1",
scope=AgentScope.WORKFLOW_ONLY,
active_config_snapshot_id="snap-2",
created_by="creator-1",
updated_by="updater-1",
)
draft = AgentConfigDraft(
id="build-draft-1",
tenant_id="t1",
agent_id="agent-1",
draft_type=AgentConfigDraftType.DEBUG_BUILD,
account_id="account-1",
draft_owner_key="account-1",
base_snapshot_id="snap-1",
config_snapshot=AgentSoulConfig.model_validate({"prompt": {"system_prompt": "build edit"}}),
)
session = _FakeScalarSession([draft])
resolved = AgentAppGenerator._resolve_debug_draft(
tenant_id="t1",
agent=agent,
draft_type=AgentConfigDraftType.DEBUG_BUILD.value,
account_id="account-1",
session=session,
)
assert resolved is draft
assert resolved.base_snapshot_id == "snap-1"
assert resolved.config_snapshot_dict["prompt"]["system_prompt"] == "build edit"
assert session.flush_count == 0
class TestResolveAgent:
def test_success_chains_to_resolve_by_id(self):
@@ -185,9 +257,12 @@ class TestResolveAgent:
def test_unpublished_imported_agent_remains_available_to_debugger(self):
bound_agent = SimpleNamespace(
id="agent-1",
scope=AgentScope.ROSTER,
source=AgentSource.IMPORTED,
active_config_snapshot_id="snap-1",
active_config_is_published=False,
created_by="creator-1",
updated_by="updater-1",
)
draft = SimpleNamespace(id="draft-1", draft_type="draft", config_snapshot_dict=_SOUL_DICT)
session = _FakeScalarSession([bound_agent, draft])
@@ -2,10 +2,10 @@ from __future__ import annotations
import logging
from contextlib import nullcontext
from types import SimpleNamespace
from unittest.mock import MagicMock
import pytest
from sqlalchemy import func, select
from sqlalchemy.orm import Session
from core.app.app_config.entities import (
AdvancedChatMessageEntity,
@@ -15,7 +15,13 @@ from core.app.app_config.entities import (
)
from core.app.apps.base_app_runner import AppRunner
from core.app.apps.exc import GenerateTaskStoppedError
from core.app.entities.app_invoke_entities import InvokeFrom
from core.app.apps.message_based_app_queue_manager import MessageBasedAppQueueManager
from core.app.entities.app_invoke_entities import (
AppGenerateEntity,
EasyUIBasedAppGenerateEntity,
InvokeFrom,
ModelConfigWithCredentialsEntity,
)
from core.app.entities.queue_entities import (
QueueAgentMessageEvent,
QueueLLMChunkEvent,
@@ -29,9 +35,9 @@ from graphon.model_runtime.entities.message_entities import (
PromptMessageRole,
TextPromptMessageContent,
)
from graphon.model_runtime.entities.model_entities import ModelPropertyKey
from graphon.model_runtime.entities.model_entities import AIModelEntity, ModelPropertyKey
from graphon.model_runtime.errors.invoke import InvokeBadRequestError
from models.model import AppMode
from models.model import App, AppMode, Message, MessageFile
class _DummyParameterRule:
@@ -40,13 +46,29 @@ class _DummyParameterRule:
self.use_template = use_template
class _QueueRecorder:
def __init__(self) -> None:
self.events: list[object] = []
class _TokenCountingModel:
token_count: int
def publish(self, event, pub_from):
_ = pub_from
self.events.append(event)
def __init__(self, token_count: int) -> None:
self.token_count = token_count
def get_llm_num_tokens(self, messages: list[AssistantPromptMessage]) -> int:
return self.token_count
def _queue_manager() -> MessageBasedAppQueueManager:
return MessageBasedAppQueueManager(
task_id="task-id",
user_id="user-id",
invoke_from=InvokeFrom.SERVICE_API,
conversation_id="conversation-id",
app_mode=AppMode.CHAT.value,
message_id="message-id",
)
def _published_events(queue_manager: MessageBasedAppQueueManager) -> list[object]:
return [message.event for message in queue_manager.listen()]
class _ClosableStream:
@@ -70,11 +92,11 @@ class TestAppRunner:
def test_recalc_llm_max_tokens_updates_parameters(self, monkeypatch: pytest.MonkeyPatch):
runner = AppRunner()
model_schema = SimpleNamespace(
model_schema = AIModelEntity.model_construct(
model_properties={ModelPropertyKey.CONTEXT_SIZE: 100},
parameter_rules=[_DummyParameterRule("max_tokens")],
)
model_config = SimpleNamespace(
model_config = ModelConfigWithCredentialsEntity.model_construct(
provider_model_bundle=object(),
model="mock",
model_schema=model_schema,
@@ -83,7 +105,7 @@ class TestAppRunner:
monkeypatch.setattr(
"core.app.apps.base_app_runner.ModelInstance",
lambda provider_model_bundle, model: SimpleNamespace(get_llm_num_tokens=lambda messages: 80),
lambda provider_model_bundle, model: _TokenCountingModel(80),
)
runner.recalc_llm_max_tokens(model_config, prompt_messages=[AssistantPromptMessage(content="hi")])
@@ -93,11 +115,11 @@ class TestAppRunner:
def test_recalc_llm_max_tokens_returns_minus_one_when_no_context(self, monkeypatch: pytest.MonkeyPatch):
runner = AppRunner()
model_schema = SimpleNamespace(
model_schema = AIModelEntity.model_construct(
model_properties={},
parameter_rules=[_DummyParameterRule("max_tokens")],
)
model_config = SimpleNamespace(
model_config = ModelConfigWithCredentialsEntity.model_construct(
provider_model_bundle=object(),
model="mock",
model_schema=model_schema,
@@ -106,17 +128,16 @@ class TestAppRunner:
monkeypatch.setattr(
"core.app.apps.base_app_runner.ModelInstance",
lambda provider_model_bundle, model: SimpleNamespace(get_llm_num_tokens=lambda messages: 10),
lambda provider_model_bundle, model: _TokenCountingModel(10),
)
assert runner.recalc_llm_max_tokens(model_config, prompt_messages=[]) == -1
def test_direct_output_streaming_publishes_chunks_and_end(self, monkeypatch: pytest.MonkeyPatch):
def test_direct_output_streaming_publishes_chunks_and_end(self):
runner = AppRunner()
queue = _QueueRecorder()
app_generate_entity = SimpleNamespace(model_conf=SimpleNamespace(model="mock"), stream=True)
monkeypatch.setattr("core.app.apps.base_app_runner.time.sleep", lambda _: None)
queue = _queue_manager()
model_config = ModelConfigWithCredentialsEntity.model_construct(model="mock")
app_generate_entity = EasyUIBasedAppGenerateEntity.model_construct(model_conf=model_config, stream=True)
runner.direct_output(
queue_manager=queue,
@@ -126,12 +147,13 @@ class TestAppRunner:
stream=True,
)
assert any(isinstance(event, QueueLLMChunkEvent) for event in queue.events)
assert isinstance(queue.events[-1], QueueMessageEndEvent)
events = _published_events(queue)
assert any(isinstance(event, QueueLLMChunkEvent) for event in events)
assert isinstance(events[-1], QueueMessageEndEvent)
def test_handle_invoke_result_direct_publishes_end_event(self):
runner = AppRunner()
queue = _QueueRecorder()
queue = _queue_manager()
llm_result = LLMResult(
model="mock",
prompt_messages=[],
@@ -145,11 +167,11 @@ class TestAppRunner:
stream=False,
)
assert isinstance(queue.events[-1], QueueMessageEndEvent)
assert isinstance(_published_events(queue)[-1], QueueMessageEndEvent)
def test_handle_invoke_result_invalid_type_raises(self):
runner = AppRunner()
queue = _QueueRecorder()
queue = _queue_manager()
with pytest.raises(NotImplementedError):
runner._handle_invoke_result(
@@ -160,7 +182,7 @@ class TestAppRunner:
def test_organize_prompt_messages_simple_template(self, monkeypatch: pytest.MonkeyPatch):
runner = AppRunner()
model_config = SimpleNamespace(mode="chat", stop=["STOP"])
model_config = ModelConfigWithCredentialsEntity.model_construct(mode="chat", stop=["STOP"])
prompt_template_entity = PromptTemplateEntity(
prompt_type=PromptTemplateEntity.PromptType.SIMPLE,
simple_prompt_template="hello",
@@ -172,7 +194,7 @@ class TestAppRunner:
)
prompt_messages, stop = runner.organize_prompt_messages(
app_record=SimpleNamespace(mode=AppMode.CHAT.value),
app_record=App(mode=AppMode.CHAT.value),
model_config=model_config,
prompt_template_entity=prompt_template_entity,
inputs={},
@@ -185,7 +207,7 @@ class TestAppRunner:
def test_organize_prompt_messages_advanced_completion_template(self, monkeypatch: pytest.MonkeyPatch):
runner = AppRunner()
model_config = SimpleNamespace(mode="completion", stop=["<END>"])
model_config = ModelConfigWithCredentialsEntity.model_construct(mode="completion", stop=["<END>"])
captured: dict[str, object] = {}
prompt_template_entity = PromptTemplateEntity(
prompt_type=PromptTemplateEntity.PromptType.ADVANCED,
@@ -202,7 +224,7 @@ class TestAppRunner:
monkeypatch.setattr("core.app.apps.base_app_runner.AdvancedPromptTransform.get_prompt", _fake_advanced_prompt)
prompt_messages, stop = runner.organize_prompt_messages(
app_record=SimpleNamespace(mode=AppMode.CHAT.value),
app_record=App(mode=AppMode.CHAT.value),
model_config=model_config,
prompt_template_entity=prompt_template_entity,
inputs={},
@@ -218,7 +240,7 @@ class TestAppRunner:
def test_organize_prompt_messages_advanced_chat_template(self, monkeypatch: pytest.MonkeyPatch):
runner = AppRunner()
model_config = SimpleNamespace(mode="chat", stop=["<END>"])
model_config = ModelConfigWithCredentialsEntity.model_construct(mode="chat", stop=["<END>"])
captured: dict[str, object] = {}
prompt_template_entity = PromptTemplateEntity(
prompt_type=PromptTemplateEntity.PromptType.ADVANCED,
@@ -237,7 +259,7 @@ class TestAppRunner:
monkeypatch.setattr("core.app.apps.base_app_runner.AdvancedPromptTransform.get_prompt", _fake_advanced_prompt)
prompt_messages, stop = runner.organize_prompt_messages(
app_record=SimpleNamespace(mode=AppMode.CHAT.value),
app_record=App(mode=AppMode.CHAT.value),
model_config=model_config,
prompt_template_entity=prompt_template_entity,
inputs={},
@@ -254,8 +276,8 @@ class TestAppRunner:
with pytest.raises(InvokeBadRequestError, match="Advanced completion prompt template is required"):
runner.organize_prompt_messages(
app_record=SimpleNamespace(mode=AppMode.CHAT.value),
model_config=SimpleNamespace(mode="completion", stop=[]),
app_record=App(mode=AppMode.CHAT.value),
model_config=ModelConfigWithCredentialsEntity.model_construct(mode="completion", stop=[]),
prompt_template_entity=PromptTemplateEntity(prompt_type=PromptTemplateEntity.PromptType.ADVANCED),
inputs={},
files=[],
@@ -263,18 +285,16 @@ class TestAppRunner:
with pytest.raises(InvokeBadRequestError, match="Advanced chat prompt template is required"):
runner.organize_prompt_messages(
app_record=SimpleNamespace(mode=AppMode.CHAT.value),
model_config=SimpleNamespace(mode="chat", stop=[]),
app_record=App(mode=AppMode.CHAT.value),
model_config=ModelConfigWithCredentialsEntity.model_construct(mode="chat", stop=[]),
prompt_template_entity=PromptTemplateEntity(prompt_type=PromptTemplateEntity.PromptType.ADVANCED),
inputs={},
files=[],
)
def test_handle_invoke_result_stream_routes_chunks_and_builds_message(
self, monkeypatch: pytest.MonkeyPatch, caplog: pytest.LogCaptureFixture
):
def test_handle_invoke_result_stream_routes_chunks_and_builds_message(self, caplog: pytest.LogCaptureFixture):
runner = AppRunner()
queue = _QueueRecorder()
queue = _queue_manager()
image_content = ImagePromptMessageContent(
url="https://example.com/image.png", format="png", mime_type="image/png"
@@ -286,11 +306,9 @@ class TestAppRunner:
prompt_messages=[AssistantPromptMessage(content="prompt")],
delta=LLMResultChunkDelta(
index=0,
message=AssistantPromptMessage.model_construct(
message=AssistantPromptMessage(
content=[
"a",
TextPromptMessageContent(data="b"),
SimpleNamespace(data="c"),
TextPromptMessageContent(data="abc"),
image_content,
]
),
@@ -305,21 +323,25 @@ class TestAppRunner:
agent=False,
)
assert isinstance(queue.events[0], QueueLLMChunkEvent)
assert isinstance(queue.events[-1], QueueMessageEndEvent)
assert queue.events[-1].llm_result.message.content == "abc"
events = _published_events(queue)
assert isinstance(events[0], QueueLLMChunkEvent)
assert isinstance(events[-1], QueueMessageEndEvent)
assert events[-1].llm_result.message.content == "abc"
assert "Received multimodal output but missing required parameters" in caplog.messages
def test_handle_invoke_result_stream_agent_mode_handles_multimodal_errors(
self, monkeypatch: pytest.MonkeyPatch, caplog: pytest.LogCaptureFixture
):
runner = AppRunner()
queue = _QueueRecorder()
queue = _queue_manager()
def raise_multimodal_error(**kwargs):
raise RuntimeError("failed to save image")
monkeypatch.setattr(
runner,
"_handle_multimodal_image_content",
MagicMock(side_effect=RuntimeError("failed to save image")),
raise_multimodal_error,
)
usage = LLMUsage.empty_usage()
@@ -353,22 +375,37 @@ class TestAppRunner:
tenant_id="tenant-id",
)
assert isinstance(queue.events[0], QueueAgentMessageEvent)
assert isinstance(queue.events[-1], QueueMessageEndEvent)
assert queue.events[-1].llm_result.usage == usage
events = _published_events(queue)
assert isinstance(events[0], QueueAgentMessageEvent)
assert isinstance(events[-1], QueueMessageEndEvent)
assert events[-1].llm_result.usage == usage
assert "Failed to handle multimodal image output" in caplog.messages
def test_handle_invoke_result_stream_commits_message_file_before_publish(self, monkeypatch: pytest.MonkeyPatch):
@pytest.mark.parametrize("sqlite_session", [()], indirect=True)
def test_handle_invoke_result_stream_commits_message_file_before_publish(
self,
monkeypatch: pytest.MonkeyPatch,
sqlite_session: Session,
):
runner = AppRunner()
runner._handle_multimodal_image_content = MagicMock(return_value="message-file-1")
session = MagicMock()
monkeypatch.setattr(
runner,
"_handle_multimodal_image_content",
lambda **kwargs: "message-file-1",
)
events: list[str] = []
session.commit.side_effect = lambda: events.append("commit")
original_commit = sqlite_session.commit
def commit():
events.append("commit")
original_commit()
monkeypatch.setattr(sqlite_session, "commit", commit)
monkeypatch.setattr(
"core.app.apps.base_app_runner.session_factory.create_session",
lambda: nullcontext(session),
lambda: nullcontext(sqlite_session),
)
queue = _QueueRecorder()
queue = _queue_manager()
original_publish = queue.publish
def publish(event, pub_from):
@@ -407,7 +444,7 @@ class TestAppRunner:
assert events == ["commit", "publish"]
def test_handle_invoke_result_stream_closes_generator_when_stopped(self):
def test_handle_invoke_result_stream_closes_generator_when_stopped(self, monkeypatch: pytest.MonkeyPatch):
runner = AppRunner()
chunk = LLMResultChunk(
model="stream-model",
@@ -416,9 +453,8 @@ class TestAppRunner:
)
stream = _ClosableStream([chunk])
queue_manager = SimpleNamespace(
publish=MagicMock(side_effect=GenerateTaskStoppedError("stopped")),
)
queue_manager = _queue_manager()
monkeypatch.setattr(queue_manager, "_is_stopped", lambda: True)
with pytest.raises(GenerateTaskStoppedError):
runner._handle_invoke_result_stream(
@@ -429,7 +465,11 @@ class TestAppRunner:
assert stream.closed is True
def test_handle_multimodal_image_content_fallback_return_branch(self, monkeypatch: pytest.MonkeyPatch):
@pytest.mark.parametrize("sqlite_session", [(MessageFile,)], indirect=True)
def test_handle_multimodal_image_content_fallback_return_branch(
self,
sqlite_session: Session,
):
runner = AppRunner()
class _ToggleBool:
@@ -442,19 +482,17 @@ class TestAppRunner:
self._index += 1
return value
content = SimpleNamespace(
# The fallback is reachable only when the fields change truthiness between the guard and branch checks.
content = ImagePromptMessageContent.model_construct(
url=_ToggleBool([False, False]),
base64_data=_ToggleBool([True, False]),
mime_type="image/png",
)
db_session = SimpleNamespace(add=MagicMock(), flush=MagicMock(), refresh=MagicMock())
monkeypatch.setattr("core.app.apps.base_app_runner.ToolFileManager", lambda: MagicMock())
queue_manager = SimpleNamespace(invoke_from=InvokeFrom.SERVICE_API, publish=MagicMock())
queue_manager = _queue_manager()
runner._handle_multimodal_image_content(
session=db_session,
session=sqlite_session,
content=content,
message_id="message-id",
user_id="user-id",
@@ -462,20 +500,20 @@ class TestAppRunner:
queue_manager=queue_manager,
)
db_session.add.assert_not_called()
queue_manager.publish.assert_not_called()
message_file_count = sqlite_session.scalar(select(func.count()).select_from(MessageFile))
assert message_file_count == 0
def test_check_hosting_moderation_direct_output_called(self, monkeypatch: pytest.MonkeyPatch):
runner = AppRunner()
queue = _QueueRecorder()
app_generate_entity = SimpleNamespace(stream=False)
queue = _queue_manager()
app_generate_entity = EasyUIBasedAppGenerateEntity.model_construct(stream=False)
direct_output_calls: list[dict[str, object]] = []
monkeypatch.setattr(
"core.app.apps.base_app_runner.HostingModerationFeature.check",
lambda self, application_generate_entity, prompt_messages: True,
)
direct_output = MagicMock()
monkeypatch.setattr(runner, "direct_output", direct_output)
monkeypatch.setattr(runner, "direct_output", lambda **kwargs: direct_output_calls.append(kwargs))
result = runner.check_hosting_moderation(
application_generate_entity=app_generate_entity,
@@ -484,7 +522,7 @@ class TestAppRunner:
)
assert result is True
assert direct_output.called
assert len(direct_output_calls) == 1
def test_fill_in_inputs_from_external_data_tools(self, monkeypatch: pytest.MonkeyPatch):
runner = AppRunner()
@@ -509,7 +547,7 @@ class TestAppRunner:
"core.app.apps.base_app_runner.InputModeration.check",
lambda self, app_id, tenant_id, app_config, inputs, query, message_id, trace_manager: (True, {}, ""),
)
app_generate_entity = SimpleNamespace(app_config=SimpleNamespace(), trace_manager=None)
app_generate_entity = AppGenerateEntity.model_construct(app_config=None, trace_manager=None)
result = runner.moderation_for_inputs(
app_id="app",
@@ -522,7 +560,12 @@ class TestAppRunner:
assert result == (True, {}, "")
def test_query_app_annotations_to_reply(self, monkeypatch: pytest.MonkeyPatch):
@pytest.mark.parametrize("sqlite_session", [()], indirect=True)
def test_query_app_annotations_to_reply(
self,
monkeypatch: pytest.MonkeyPatch,
sqlite_session: Session,
):
runner = AppRunner()
monkeypatch.setattr(
"core.app.apps.base_app_runner.AnnotationReplyFeature.query",
@@ -530,12 +573,12 @@ class TestAppRunner:
)
response = runner.query_app_annotations_to_reply(
app_record=SimpleNamespace(),
message=SimpleNamespace(),
app_record=App(),
message=Message(),
query="hello",
user_id="user",
invoke_from=InvokeFrom.WEB_APP,
session=MagicMock(),
session=sqlite_session,
)
assert response == "reply"
@@ -125,6 +125,9 @@ class TestPluginParameterEntities:
parameter = PluginParameter(name="p", label=self._label(), options="invalid") # type: ignore[arg-type]
assert parameter.options == []
def test_plugin_parameter_excludes_tool_specific_multiple_declaration(self):
assert "multiple" not in PluginParameter.model_fields
@pytest.mark.parametrize(
("parameter_type", "expected"),
[

Some files were not shown because too many files have changed in this diff Show More