Compare commits

...
Author SHA1 Message Date
fatelei 2bb01984ed fix: fix conflict 2026-07-28 19:03:11 +08:00
fatelei 77e41a37ee feat: agent can op file and fix version check 2026-07-28 18:36:06 +08:00
zxhlyh 79229b7ede fix(web): preserve IME composition in skill builder 2026-07-28 17:46:12 +08:00
zxhlyh 36ae648a63 fix(web): protect skill bindings in version view 2026-07-28 16:21:43 +08:00
zxhlyh c409d531da fix(web): paginate workspace skills list 2026-07-28 15:49:44 +08:00
autofix-ci[bot]andGitHub 511ca69e4e [autofix.ci] apply automated fixes 2026-07-28 07:17:11 +00:00
fatelei 4c7e60d7e7 fix: fix db version 2026-07-28 15:12:58 +08:00
fatelei a9e163403d fix: fix model not change 2026-07-28 14:52:16 +08:00
zxhlyhandfatelei bc136b89f9 fix(web): prevent duplicate skill autosaves 2026-07-28 14:52:16 +08:00
fatelei b1bd4f9a8b fix: fix model not update 2026-07-28 14:52:16 +08:00
fatelei 06d3927f05 fix: fix failed test 2026-07-28 14:52:16 +08:00
fatelei 5c3516cae8 fix: fix skill not in agent 2026-07-28 14:52:16 +08:00
fatelei 1c18d8ddbd fix: fix frontend issue and let draft check relax 2026-07-28 14:52:16 +08:00
fatelei 253dfe9351 fix: fix ci failed 2026-07-28 14:52:15 +08:00
fatelei 2d3999e984 fix: fix ci failed 2026-07-28 14:52:15 +08:00
fatelei a431cc726d fix: fix scroll and live mode can not enter 2026-07-28 14:52:15 +08:00
fatelei 08493d2429 fix: fix some bugs 2026-07-28 14:52:15 +08:00
fatelei 8dd0969006 chore: adjust skill entrance 2026-07-28 14:52:15 +08:00
fatelei 2d80d3c35c fix: fix delete skill ref 2026-07-28 14:52:14 +08:00
fatelei 4da764904c feat: support agent skill 2026-07-28 14:46:34 +08:00
fatelei 5c4f4fd1ef Merge branch 'skill' into deploy/dev 2026-07-28 14:44:57 +08:00
fatelei 54843971ac fix: fix model not change 2026-07-28 14:43:05 +08:00
zxhlyh 5ec5d3aeb9 fix(web): prevent duplicate skill autosaves 2026-07-28 14:30:17 +08:00
github-actions[bot]GitHubclaude[bot] <41898282+claude[bot]@users.noreply.github.com>yyh
137d4f3f60 chore(i18n): sync translations with en-US (#39671)
Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: yyh <[email protected]>
2026-07-28 06:11:36 +00:00
fatelei d661b53e49 fix: fix model not update 2026-07-28 14:10:49 +08:00
zxhlyh fd1e777f85 Merge branch 'codex/fix-skill-autosave-conflict' into deploy/dev 2026-07-28 14:04:25 +08:00
zxhlyh e5f77ce185 fix(web): prevent duplicate skill autosaves 2026-07-28 14:02:18 +08:00
Asuka MinatoGitHubautofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
e723b348cf test: use SQLite sessions in core app apps (#39103)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-07-28 05:54:17 +00:00
59fb603ec6 fix: reject trailing newlines in alphanumeric() validator (#39666) (#39667)
Co-authored-by: Harsh Kashyap <[email protected]>
2026-07-28 05:49:19 +00:00
Asuka MinatoGitHubautofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
63f072ebfb test: use SQLite sessions in core rag (#39105)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-07-28 05:47:22 +00:00
Asuka MinatoGitHubautofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
0913d04d33 test: use SQLite sessions in services retention (#39074)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-07-28 05:47:01 +00:00
Asuka MinatoandGitHub 80ff108fc0 test: use SQLite in Aliyun trace utilities (#38983) 2026-07-28 05:40:27 +00:00
Asuka MinatoandGitHub ea58129ebe test: move workflow app service coverage to unit tests (#38946) 2026-07-28 05:34:59 +00:00
-LAN-andGitHub 698869460c chore(docker): remove redundant feature preview env (#39035) 2026-07-28 05:34:35 +00:00
Asuka MinatoandGitHub f3f2f63110 test: use SQLite in LangSmith traces (#38986) 2026-07-28 05:26:52 +00:00
Asuka MinatoGitHubautofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
b100cdc382 test: use SQLite sessions in core app (#39099)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-07-28 05:22:47 +00:00
Asuka MinatoGitHubautofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
9e90b32991 test: use SQLite sessions in core datasource (#39104)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-07-28 05:22:18 +00:00
非法操作andGitHub b2d54cb2e9 chore: improve archived logs hint in logs list page (#39659) 2026-07-28 05:15:35 +00:00
yyhandGitHub 1e5e47b889 fix: validate plugin installation scope (#39669) 2026-07-28 04:13:20 +00:00
Asuka MinatoGitHubautofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>Byron.wang
dd28b0d165 test: move OAuth server service coverage to unit tests (#38931)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Byron.wang <[email protected]>
2026-07-28 04:09:32 +00:00
JoelandGitHub 49e74e2f58 fix: prevent the model selector footer from covering options (#39668) 2026-07-28 04:02:28 +00:00
e6e5d761c2 fix: prevent Safari from clipping the settings close button (#39664)
Co-authored-by: yyh <[email protected]>
2026-07-28 04:01:12 +00:00
Asuka MinatoandGitHub e1ce808567 test: move data source controller coverage to unit tests (#38924) 2026-07-28 03:51:22 +00:00
Asuka MinatoandGitHub 3c7ad816d9 test: use SQLite sessions in commands (#39080) 2026-07-28 03:29:02 +00:00
Asuka MinatoandGitHub f44dd343da test: use sqlite3 session in test_plugin_service (#38727) 2026-07-28 03:27:58 +00:00
Asuka MinatoandGitHub 2d9b2d50f3 test: use sqlite3 session in test_wraps (#38770) 2026-07-28 03:27:22 +00:00
Asuka MinatoandGitHub 003e0f9614 test: move message cleanup coverage to unit tests (#38932) 2026-07-28 03:26:43 +00:00
Asuka MinatoGitHubautofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>Byron.wang
da0979b373 test: use SQLite sessions in services core (#39090)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Byron.wang <[email protected]>
2026-07-28 03:20:27 +00:00
Asuka MinatoGitHubautofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>Byron.wang
81fb57639e test: use SQLite sessions in controllers service api (#39098)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Byron.wang <[email protected]>
2026-07-28 03:19:17 +00:00
Asuka MinatoandGitHub 1e61078e93 test: use SQLite sessions in services core (#39088) 2026-07-28 03:18:32 +00:00
Escape0707andGitHub b597bb1b17 test: separate human input unit and database paths (#39655) 2026-07-28 03:18:01 +00:00
Asuka MinatoGitHubautofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>Byron.wang
65ead05dfc test: use SQLite sessions in services plugin (#39084)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Byron.wang <[email protected]>
2026-07-28 03:16:19 +00:00
6f8ed69ee1 fix: fix mcp output_schema is optional (#39453)
Co-authored-by: yunlu.wen <[email protected]>
2026-07-28 02:31:18 +00:00
Yunlu WenGitHubautofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
59b879d2df chore: bump version to 1.16.1 (#39653)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-07-28 01:09:25 +00:00
Xiyuan ChenandGitHub a57b0b9b58 feat(plugin): allow disabling the tenant plugin model providers cache (#39632) 2026-07-27 23:53:59 +00:00
Xiyuan ChenandGitHub d8506efed6 fix(cli): decouple release script tests from the live compat window (#39658) 2026-07-27 23:50:05 +00:00
JingyiandGitHub b25b28cc76 fix(workflow): align block icon vector sizes (#39657) 2026-07-27 23:43:03 +00:00
yyhGitHubautofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
f68cabe226 fix(web): align tour trigger DOM order (#39654)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-07-27 13:00:16 +00:00
Asuka MinatoGitHubautofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>Byron.wang
f73f83c5d6 test: use SQLite sessions in core app (#39075)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Byron.wang <[email protected]>
2026-07-27 10:45:16 +00:00
Asuka MinatoGitHubautofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>Byron.wang
29801d6a65 test: use SQLite sessions in core tools (#39077)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Byron.wang <[email protected]>
2026-07-27 10:45:11 +00:00
Asuka MinatoGitHubautofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>Byron.wang
75016e8bfe test: use SQLite sessions in core ops (#39072)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Byron.wang <[email protected]>
2026-07-27 10:41:17 +00:00
yyhandGitHub b6dea7b2ba fix(workflow): preserve latest collaboration session (#39646) 2026-07-27 10:37:45 +00:00
yyhandGitHub b2b1cd7e97 chore: update workspace dependencies (#39641) 2026-07-27 10:27:31 +00:00
Asuka MinatoGitHubautofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>Byron.wang
ca63e01d45 test: use SQLite sessions in controllers service api (#39069)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Byron.wang <[email protected]>
2026-07-27 10:26:23 +00:00
yyhandGitHub 271b6e1f5c fix(ui): unify combobox trigger focus rings (#39643) 2026-07-27 10:21:32 +00:00
Asuka MinatoGitHubautofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>Byron.wang
609ddf9e01 test: use SQLite sessions in core memory (#39066)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Byron.wang <[email protected]>
2026-07-27 10:15:08 +00:00
Asuka MinatoGitHubautofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>Byron.wang
9fd1fea58c test: use SQLite sessions in services core (#39057)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Byron.wang <[email protected]>
2026-07-27 09:36:44 +00:00
JoelandGitHub 8de3b4d033 fix: prevent the agent build draft UI from flashing during apply (#39642) 2026-07-27 09:32:52 +00:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
d9c038daf2 chore(deps): bump the storage group across 1 directory with 3 updates (#39604)
Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-07-27 09:24:58 +00:00
Asuka MinatoGitHubautofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>Byron.wang
c9c057fd04 test: use SQLite sessions in rag retrieval (#39049)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Byron.wang <[email protected]>
2026-07-27 09:12:30 +00:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
85189be53d chore: bump pymdown-extensions from 10.21.2 to 11.0 in /dify-agent (#39618)
Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-07-27 09:10:22 +00:00
Asuka MinatoGitHubautofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
01e736aaf7 test: use sqlite3 session in test_extract_thread_messages (#38739)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-07-27 09:00:47 +00:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
a28969f564 chore(deps): bump golang.org/x/net from 0.53.0 to 0.55.0 in /dify-agent-runtime (#39025)
Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-07-27 08:58:40 +00:00
Asuka MinatoandGitHub 42f9610ba5 test: use sqlite3 session in test_api (#38733) 2026-07-27 08:57:51 +00:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
1a7ffbe5ee chore: bump gitpython from 3.1.52 to 3.1.54 in /api (#39619)
Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-07-27 08:57:44 +00:00
JoelandGitHub 481354ca70 refactor: make agent composer save and publish state consistent (#39637) 2026-07-27 08:43:45 +00:00
3c80857ea3 feat: add bearer auth to agent backend (#39622)
Co-authored-by: Copilot Autofix powered by AI <[email protected]>
2026-07-27 08:37:30 +00:00
Xiyuan ChenandGitHub 94702efbc2 fix: gate service API, MCP and trigger surfaces on enterprise license (#39635) 2026-07-27 08:23:15 +00:00
Asuka MinatoandGitHub 755f7b0e8b test: use sqlite3 session in test_workflow_comment_service (#38700) 2026-07-27 08:22:45 +00:00
Asuka MinatoandGitHub a1c9564b30 test: use sqlite3 session in test_llm_generator_missing (#38742) 2026-07-27 08:22:16 +00:00
Asuka MinatoandGitHub 97acd9c70b test: use sqlite3 session in test_utils (#38729) 2026-07-27 08:21:39 +00:00
Asuka MinatoandGitHub 71601bf76c test: use sqlite3 session in test_draft_var_loader_simple (#38702) 2026-07-27 08:20:18 +00:00
Asuka MinatoandGitHub 460efbf285 test: use sqlite3 session in test_workflow_run_service (#38701) 2026-07-27 08:19:32 +00:00
a752f43b8e test: use pristine file-backed SQLite fixtures (#39624)
Co-authored-by: Asuka Minato <[email protected]>
2026-07-27 08:07:05 +00:00
非法操作GitHubautofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
b3774bfe1c perf: add lightweight recent apps endpoint (#39625)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-07-27 08:00:39 +00:00
yyhandGitHub 4313d23889 perf(web): lazy-load Home creation modals (#39634) 2026-07-27 07:48:35 +00:00
林玮 (Jade Lin)andGitHub 26a43db6a4 fix(api): wait for workflow worker cleanup before task completion (#39614) 2026-07-27 07:34:25 +00:00
yyhandGitHub e5d40336b3 fix(workflow): lazy-load Loro collaboration runtime (#39631) 2026-07-27 07:15:13 +00:00
yyhandGitHub 58e2bcbba1 fix(web): stop reporting workspace status (#39626) 2026-07-27 06:22:37 +00:00
Yunlu WenGitHubautofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
e1b55f54e6 feat(agent): add a squid proxy for agent sandbox (#39544)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-07-27 06:21:20 +00:00
Yunlu WenGitHubautofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
989039db6e fix: use jinja sandbox (#39609)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-07-27 06:03:01 +00:00
JoelGitHub林玮autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
52428df1bd feat: add amptitude to new agent (#39608)
Co-authored-by: 林玮 (Jade Lin) <[email protected]>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-07-27 05:54:45 +00:00
yyhGitHubautofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
28d174603f fix: align workspace card plan ownership (#39616)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-07-27 05:01:02 +00:00
Xiyuan ChenandGitHub b6099d09ff feat(inner_api): add endpoint to invalidate plugin model providers cache (#39468) 2026-07-27 04:33:36 +00:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2962a7ea93 chore(deps): bump the github-actions-dependencies group with 6 updates (#39603)
Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-07-27 04:32:09 +00:00
zyssyz123andGitHub cc01189966 fix(agent): expose publish state from composer (#39615) 2026-07-27 04:05:38 +00:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
d563d6e7df chore: bump pypdf from 6.10.2 to 6.14.2 in /dify-agent (#39502)
Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-07-27 03:56:57 +00:00
CYJ1226andGitHub e4e99d198b fix: preserve spaces during recursive text splitting (#39498) 2026-07-27 03:21:26 +00:00
yyhandGitHub 34e6e5a049 refactor(web): simplify Dify logo theming (#39580) 2026-07-27 03:01:24 +00:00
林玮 (Jade Lin)GitHubautofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
7e6ba05464 feat(api): expose app mode in webapp site response (#39607)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-07-27 02:57:18 +00:00
Madan kumarandGitHub aeda37db68 fix(rag): stop the document cleaner from stripping valid characters ï, ¿, ¾ (#39215) 2026-07-27 02:53:33 +00:00
FFXNandGitHub 441f9f9ec0 fix: The expiration time for web app login to JWT is incorrectly set. (#39537) 2026-07-27 02:43:32 +00:00
f16c249b1e fix(api): honor array-element limit and byte budget for list[File] in VariableTruncator (#39218) (#39220)
Co-authored-by: Taranum Wasu <[email protected]>
Co-authored-by: Cursor <[email protected]>
2026-07-27 02:12:57 +00:00
FFXNGitHubautofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
858b3b74dc feat: add CE telemetry report (#39452)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-07-27 02:04:28 +00:00
d5c0e927c6 fix(web): stop chunk length/overlap inputs collapsing to an unusable width in narrow containers (#39600)
Co-authored-by: Claude Opus 4.8 (1M context) <[email protected]>
2026-07-27 02:03:17 +00:00
Asuka MinatoandGitHub b81737cfb3 test: use sqlite3 session in test_message_service (#38698) 2026-07-27 02:01:47 +00:00
fatelei 99929d1c16 fix: fix conflict 2026-07-27 08:12:58 +08:00
fatelei c699aa11db fix: fix failed test 2026-07-26 23:42:50 +08:00
fatelei 06cd0b56ac fix: fix skill not in agent 2026-07-26 21:00:38 +08:00
fatelei 241a9e1fec fix: fix frontend issue and let draft check relax 2026-07-26 21:00:38 +08:00
fatelei 3d79689fb5 fix: fix ci failed 2026-07-26 20:59:56 +08:00
fatelei 9f050f7957 fix: fix ci failed 2026-07-26 20:59:56 +08:00
fatelei cb691d47d3 fix: fix scroll and live mode can not enter 2026-07-26 20:57:29 +08:00
fatelei 04f5ee58d0 fix: fix some bugs 2026-07-26 20:57:29 +08:00
fatelei f5cff724f4 chore: adjust skill entrance 2026-07-26 20:57:29 +08:00
fatelei 1f4ccafdea fix: fix delete skill ref 2026-07-26 20:57:29 +08:00
fatelei 163049f6ca feat: support agent skill 2026-07-26 20:54:32 +08:00
fatelei 65e7507ca3 Merge branch 'skill' into deploy/dev 2026-07-24 13:38:15 +08:00
fatelei ba157f9604 fix: fix skill not in agent 2026-07-24 13:37:19 +08:00
fatelei 0dc913630e fix: fix conflict 2026-07-24 13:18:31 +08:00
fatelei 875cd30b1f fix: fix frontend issue and let draft check relax 2026-07-24 13:11:18 +08:00
fatelei aa4a32ae84 fix: fix ci failed 2026-07-24 11:13:44 +08:00
fatelei 8573e14777 fix: fix ci failed 2026-07-24 11:13:44 +08:00
fatelei 1855be234c fix: fix scroll and live mode can not enter 2026-07-24 11:13:44 +08:00
fatelei 9bb960ff12 fix: fix some bugs 2026-07-24 11:13:44 +08:00
fatelei 1c14c7d467 chore: adjust skill entrance 2026-07-24 11:13:44 +08:00
fatelei 61faec16ca fix: fix delete skill ref 2026-07-24 11:13:44 +08:00
fatelei 57c836e692 feat: support agent skill 2026-07-24 11:13:43 +08:00
samzong 626cc282b1 fix(web): echo OAuth state on authorize redirect (#39459)
Signed-off-by: samzong <[email protected]>
2026-07-23 06:36:58 -04:00
fatelei 52624d54e3 fix: fix db migration 2026-07-23 16:28:27 +08:00
fatelei 5ce038ef92 fix: fix db migration 2026-07-23 16:25:33 +08:00
fatelei 30f4d4c0c6 fix: fix db migration 2026-07-23 16:04:09 +08:00
fatelei 510679a7d1 fix: fix db migration 2026-07-23 15:38:05 +08:00
fatelei 9237f2a14a fix: fix conflict 2026-07-23 15:23:09 +08:00
fatelei bd178c7b29 Merge branch 'skill' into deploy/dev 2026-07-23 14:38:04 +08:00
fatelei d80947aa72 fix: fix scroll and live mode can not enter 2026-07-23 14:36:44 +08:00
fatelei 1618c37d26 Merge branch 'skill' into deploy/dev 2026-07-23 14:17:52 +08:00
fatelei 701ab64462 fix: fix some bugs 2026-07-23 14:14:28 +08:00
fatelei b3298800e9 chore: adjust skill entrance 2026-07-22 16:30:38 +08:00
fatelei 0f1c6b3f78 chore: remove agent v2 feature 2026-07-22 16:26:00 +08:00
林玮 (Jade Lin) 9b4b246aad refactor(api): simplify batch cleanup refresh state
(cherry picked from commit 1ded32d949)
2026-07-22 14:49:11 +08:00
林玮 (Jade Lin) 1bd654a289 refactor(api): simplify vector cache refresh metric
(cherry picked from commit fa585cf134)
2026-07-22 14:49:11 +08:00
林玮 (Jade Lin) fc70329bdb refactor(api): derive cleanup tenant from dataset
(cherry picked from commit b124ec86fc)
2026-07-22 14:49:11 +08:00
林玮 (Jade Lin) cd8a82fbd4 fix(api): invalidate vector space cache after cleanup
(cherry picked from commit a07e5c6eb4)
2026-07-22 14:49:11 +08:00
fatelei 550cb7eff5 fix: fix db migration 2026-07-22 14:42:11 +08:00
fatelei 2e748c16e9 Merge branch 'skill' into deploy/dev 2026-07-22 14:17:08 +08:00
fatelei 577012b66d fix: fix delete skill ref 2026-07-22 14:11:55 +08:00
fatelei 251c324180 Merge branch 'skill' into deploy/dev 2026-07-22 12:59:22 +08:00
fatelei 865a618fd5 Revert "feat: support agent skill"
This reverts commit 991116990a.
2026-07-22 12:48:29 +08:00
fatelei 991116990a feat: support agent skill 2026-07-22 11:45:50 +08:00
fatelei eb5d1da0e8 feat: support agent skill 2026-07-22 10:25:25 +08:00
fatelei dce3b7a7fc feat: use celery task 2026-07-10 15:20:06 +08:00
fatelei be386aba3b chore: add switch 2026-07-10 15:19:56 +08:00
JzoNg 02e51e7d7c Merge branch 'main' into deploy/dev 2026-07-10 10:28:07 +08:00
Xiyuan ChenandGitHub 96b6d4f2c0 Merge branch 'fix/socketio-user-connect-app-context' into deploy/dev 2026-07-08 22:02:43 -07:00
GareArc 4c84c5957d fix: run user_connect authorization inside Flask app context
Socket.IO event handlers run in plain gevent greenlets without a Flask
application context, so the db.session() call added in #38227 raises
RuntimeError: Working outside of application context, and every
user_connect is rejected as unauthorized. Wrap the authorization call in
sio.app.app_context(), matching the existing connect handler.
2026-07-08 21:57:07 -07:00
CodingOnStar 34613ecdc5 fix(app): update app creation logic to use database session and ensure RBAC access initialization 2026-07-09 12:00:16 +08:00
CodingOnStar 6a14245401 Merge remote-tracking branch 'origin/main' into deploy/dev 2026-07-09 11:49:16 +08:00
CodingOnStar a758ca2aef fix(amplitude): update API key for amplitude initialization 2026-07-09 11:37:28 +08:00
CodingOnStar 9e60d4e213 Merge branch 'fix/report-slug' into deploy/dev 2026-07-09 11:29:04 +08:00
CodingOnStar ef29c8442c refactor(amplitude): comment out amplitude initialization logic for future implementation 2026-07-09 11:26:59 +08:00
CodingOnStar a1b45415ac fix(web): preserve attribution from auth redirect 2026-07-09 11:00:37 +08:00
fatelei 7fc46d75bd chore: add switch 2026-07-08 18:08:02 +08:00
fatelei 953a4ef0ca feat: create app sync scope and workspace user 2026-07-08 16:12:40 +08:00
413 changed files with 38459 additions and 11486 deletions
+6 -6
View File
@@ -29,13 +29,13 @@ jobs:
steps: steps:
- name: Checkout code - name: Checkout code
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0 uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with: with:
fetch-depth: 0 fetch-depth: 0
persist-credentials: false persist-credentials: false
- name: Setup UV and Python - name: Setup UV and Python
uses: astral-sh/setup-uv@11f9893b081a58869d3b5fccaea48c9e9e46f990 # v8.3.2 uses: astral-sh/setup-uv@c771a70e6277c0a99b617c7a806ffedaca235ff9 # v9.0.0
with: with:
enable-cache: true enable-cache: true
python-version: ${{ matrix.python-version }} python-version: ${{ matrix.python-version }}
@@ -88,13 +88,13 @@ jobs:
steps: steps:
- name: Checkout code - name: Checkout code
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0 uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with: with:
fetch-depth: 0 fetch-depth: 0
persist-credentials: false persist-credentials: false
- name: Setup UV and Python - name: Setup UV and Python
uses: astral-sh/setup-uv@11f9893b081a58869d3b5fccaea48c9e9e46f990 # v8.3.2 uses: astral-sh/setup-uv@c771a70e6277c0a99b617c7a806ffedaca235ff9 # v9.0.0
with: with:
enable-cache: true enable-cache: true
python-version: ${{ matrix.python-version }} python-version: ${{ matrix.python-version }}
@@ -139,13 +139,13 @@ jobs:
steps: steps:
- name: Checkout code - name: Checkout code
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0 uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with: with:
fetch-depth: 0 fetch-depth: 0
persist-credentials: false persist-credentials: false
- name: Setup UV and Python - name: Setup UV and Python
uses: astral-sh/setup-uv@11f9893b081a58869d3b5fccaea48c9e9e46f990 # v8.3.2 uses: astral-sh/setup-uv@c771a70e6277c0a99b617c7a806ffedaca235ff9 # v9.0.0
with: with:
enable-cache: true enable-cache: true
python-version: "3.12" python-version: "3.12"
+3 -3
View File
@@ -20,7 +20,7 @@ jobs:
run: echo "autofix.ci updates pull request branches, not merge group refs." run: echo "autofix.ci updates pull request branches, not merge group refs."
- if: github.event_name != 'merge_group' - if: github.event_name != 'merge_group'
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0 uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- name: Check Docker Compose inputs - name: Check Docker Compose inputs
if: github.event_name != 'merge_group' if: github.event_name != 'merge_group'
@@ -84,12 +84,12 @@ jobs:
dify-agent/pyproject.toml dify-agent/pyproject.toml
dify-agent/uv.lock dify-agent/uv.lock
- if: github.event_name != 'merge_group' - if: github.event_name != 'merge_group'
uses: actions/setup-python@ece7cb06caefa5fff74198d8649806c4678c61a1 # v6.3.0 uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
with: with:
python-version: "3.11" python-version: "3.11"
- if: github.event_name != 'merge_group' - if: github.event_name != 'merge_group'
uses: astral-sh/setup-uv@11f9893b081a58869d3b5fccaea48c9e9e46f990 # v8.3.2 uses: astral-sh/setup-uv@c771a70e6277c0a99b617c7a806ffedaca235ff9 # v9.0.0
- name: Generate Docker Compose - name: Generate Docker Compose
if: github.event_name != 'merge_group' && steps.docker-compose-changes.outputs.any_changed == 'true' if: github.event_name != 'merge_group' && steps.docker-compose-changes.outputs.any_changed == 'true'
+2 -2
View File
@@ -97,7 +97,7 @@ jobs:
echo "PLATFORM_PAIR=${platform//\//-}" >> $GITHUB_ENV echo "PLATFORM_PAIR=${platform//\//-}" >> $GITHUB_ENV
- name: Login to Docker Hub - name: Login to Docker Hub
uses: docker/login-action@af1e73f918a031802d376d3c8bbc3fe56130a9b0 # v4.4.0 uses: docker/login-action@abd2ef45e78c5afb21d64d4ca52ee8550d9572c7 # v4.5.1
with: with:
username: ${{ env.DOCKERHUB_USER }} username: ${{ env.DOCKERHUB_USER }}
password: ${{ env.DOCKERHUB_TOKEN }} password: ${{ env.DOCKERHUB_TOKEN }}
@@ -199,7 +199,7 @@ jobs:
merge-multiple: true merge-multiple: true
- name: Login to Docker Hub - name: Login to Docker Hub
uses: docker/login-action@af1e73f918a031802d376d3c8bbc3fe56130a9b0 # v4.4.0 uses: docker/login-action@abd2ef45e78c5afb21d64d4ca52ee8550d9572c7 # v4.5.1
with: with:
username: ${{ env.DOCKERHUB_USER }} username: ${{ env.DOCKERHUB_USER }}
password: ${{ env.DOCKERHUB_TOKEN }} password: ${{ env.DOCKERHUB_TOKEN }}
+6 -6
View File
@@ -79,7 +79,7 @@ jobs:
ws2_app_id: ${{ steps.out.outputs.DIFY_E2E_WS2_APP_ID }} ws2_app_id: ${{ steps.out.outputs.DIFY_E2E_WS2_APP_ID }}
steps: steps:
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v4 - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v4
with: with:
ref: ${{ inputs.cli_ref || github.ref }} ref: ${{ inputs.cli_ref || github.ref }}
persist-credentials: false persist-credentials: false
@@ -123,7 +123,7 @@ jobs:
shell: bash shell: bash
steps: steps:
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v4 - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v4
with: with:
ref: ${{ inputs.cli_ref || github.ref }} ref: ${{ inputs.cli_ref || github.ref }}
persist-credentials: false persist-credentials: false
@@ -170,7 +170,7 @@ jobs:
shell: bash shell: bash
steps: steps:
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v4 - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v4
with: with:
ref: ${{ inputs.cli_ref || github.ref }} ref: ${{ inputs.cli_ref || github.ref }}
persist-credentials: false persist-credentials: false
@@ -233,7 +233,7 @@ jobs:
shell: bash shell: bash
steps: steps:
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v4 - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v4
with: with:
ref: ${{ inputs.cli_ref || github.ref }} ref: ${{ inputs.cli_ref || github.ref }}
persist-credentials: false persist-credentials: false
@@ -295,7 +295,7 @@ jobs:
shell: bash shell: bash
steps: steps:
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v4 - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v4
with: with:
ref: ${{ inputs.cli_ref || github.ref }} ref: ${{ inputs.cli_ref || github.ref }}
persist-credentials: false persist-credentials: false
@@ -351,7 +351,7 @@ jobs:
shell: bash shell: bash
steps: steps:
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v4 - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v4
with: with:
ref: ${{ inputs.cli_ref || github.ref }} ref: ${{ inputs.cli_ref || github.ref }}
persist-credentials: false persist-credentials: false
+1 -1
View File
@@ -23,7 +23,7 @@ jobs:
working-directory: ./cli working-directory: ./cli
steps: steps:
- name: Checkout - name: Checkout
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0 uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with: with:
persist-credentials: false persist-credentials: false
fetch-depth: 0 fetch-depth: 0
+2 -2
View File
@@ -35,7 +35,7 @@ jobs:
dify_tag: ${{ steps.resolve.outputs.dify_tag }} dify_tag: ${{ steps.resolve.outputs.dify_tag }}
steps: steps:
- name: Checkout - name: Checkout
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0 uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with: with:
persist-credentials: false persist-credentials: false
@@ -98,7 +98,7 @@ jobs:
DIFY_TAG: ${{ needs.validate.outputs.dify_tag }} DIFY_TAG: ${{ needs.validate.outputs.dify_tag }}
steps: steps:
- name: Checkout - name: Checkout
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0 uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with: with:
persist-credentials: false persist-credentials: false
fetch-depth: 1 fetch-depth: 1
+1 -1
View File
@@ -24,7 +24,7 @@ jobs:
shell: bash shell: bash
steps: steps:
- name: Checkout cli ref - name: Checkout cli ref
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0 uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with: with:
ref: ${{ inputs.cli_ref || github.ref }} ref: ${{ inputs.cli_ref || github.ref }}
persist-credentials: false persist-credentials: false
+1 -1
View File
@@ -30,7 +30,7 @@ jobs:
steps: steps:
- name: Checkout code - name: Checkout code
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0 uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with: with:
persist-credentials: false persist-credentials: false
+4 -4
View File
@@ -13,13 +13,13 @@ jobs:
steps: steps:
- name: Checkout code - name: Checkout code
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0 uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with: with:
fetch-depth: 0 fetch-depth: 0
persist-credentials: false persist-credentials: false
- name: Setup UV and Python - name: Setup UV and Python
uses: astral-sh/setup-uv@11f9893b081a58869d3b5fccaea48c9e9e46f990 # v8.3.2 uses: astral-sh/setup-uv@c771a70e6277c0a99b617c7a806ffedaca235ff9 # v9.0.0
with: with:
enable-cache: true enable-cache: true
python-version: "3.12" python-version: "3.12"
@@ -63,13 +63,13 @@ jobs:
steps: steps:
- name: Checkout code - name: Checkout code
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0 uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with: with:
fetch-depth: 0 fetch-depth: 0
persist-credentials: false persist-credentials: false
- name: Setup UV and Python - name: Setup UV and Python
uses: astral-sh/setup-uv@11f9893b081a58869d3b5fccaea48c9e9e46f990 # v8.3.2 uses: astral-sh/setup-uv@c771a70e6277c0a99b617c7a806ffedaca235ff9 # v9.0.0
with: with:
enable-cache: true enable-cache: true
python-version: "3.12" python-version: "3.12"
+1 -1
View File
@@ -24,7 +24,7 @@ jobs:
name: Require cherry-pick provenance name: Require cherry-pick provenance
runs-on: depot-ubuntu-24.04 runs-on: depot-ubuntu-24.04
steps: steps:
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0 - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with: with:
fetch-depth: 0 fetch-depth: 0
+1 -1
View File
@@ -9,6 +9,6 @@ jobs:
pull-requests: write pull-requests: write
runs-on: depot-ubuntu-24.04 runs-on: depot-ubuntu-24.04
steps: steps:
- uses: actions/labeler@b8dd2d9be0f68b860e7dae5dae7d772984eacd6d # v6.2.0 - uses: actions/labeler@bf12e9b00b37c5c0ca2b87b79b2daf7891dbda13 # v7.0.0
with: with:
sync-labels: true sync-labels: true
+1 -1
View File
@@ -47,7 +47,7 @@ jobs:
migration-changed: ${{ steps.changes.outputs.migration }} migration-changed: ${{ steps.changes.outputs.migration }}
sandbox-runtime-changed: ${{ steps.changes.outputs.sandbox-runtime }} sandbox-runtime-changed: ${{ steps.changes.outputs.sandbox-runtime }}
steps: steps:
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0 - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- uses: dorny/paths-filter@7b450fff21473bca461d4b92ce414b9d0420d706 # v4.0.2 - uses: dorny/paths-filter@7b450fff21473bca461d4b92ce414b9d0420d706 # v4.0.2
id: changes id: changes
with: with:
+1 -1
View File
@@ -18,7 +18,7 @@ jobs:
outputs: outputs:
external-e2e-changed: ${{ steps.changes.outputs.external_e2e }} external-e2e-changed: ${{ steps.changes.outputs.external_e2e }}
steps: steps:
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0 - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- uses: dorny/paths-filter@7b450fff21473bca461d4b92ce414b9d0420d706 # v4.0.2 - uses: dorny/paths-filter@7b450fff21473bca461d4b92ce414b9d0420d706 # v4.0.2
id: changes id: changes
with: with:
+2 -2
View File
@@ -17,12 +17,12 @@ jobs:
pull-requests: write pull-requests: write
steps: steps:
- name: Checkout PR branch - name: Checkout PR branch
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0 uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with: with:
fetch-depth: 0 fetch-depth: 0
- name: Setup Python & UV - name: Setup Python & UV
uses: astral-sh/setup-uv@11f9893b081a58869d3b5fccaea48c9e9e46f990 # v8.3.2 uses: astral-sh/setup-uv@c771a70e6277c0a99b617c7a806ffedaca235ff9 # v9.0.0
with: with:
enable-cache: true enable-cache: true
@@ -21,10 +21,10 @@ jobs:
if: ${{ github.event.workflow_run.conclusion == 'success' && github.event.workflow_run.pull_requests[0].head.repo.full_name != github.repository }} if: ${{ github.event.workflow_run.conclusion == 'success' && github.event.workflow_run.pull_requests[0].head.repo.full_name != github.repository }}
steps: steps:
- name: Checkout default branch (trusted code) - name: Checkout default branch (trusted code)
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0 uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- name: Setup Python & UV - name: Setup Python & UV
uses: astral-sh/setup-uv@11f9893b081a58869d3b5fccaea48c9e9e46f990 # v8.3.2 uses: astral-sh/setup-uv@c771a70e6277c0a99b617c7a806ffedaca235ff9 # v9.0.0
with: with:
enable-cache: true enable-cache: true
+2 -2
View File
@@ -17,12 +17,12 @@ jobs:
pull-requests: write pull-requests: write
steps: steps:
- name: Checkout PR branch - name: Checkout PR branch
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0 uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with: with:
fetch-depth: 0 fetch-depth: 0
- name: Setup Python & UV - name: Setup Python & UV
uses: astral-sh/setup-uv@11f9893b081a58869d3b5fccaea48c9e9e46f990 # v8.3.2 uses: astral-sh/setup-uv@c771a70e6277c0a99b617c7a806ffedaca235ff9 # v9.0.0
with: with:
enable-cache: true enable-cache: true
+3 -3
View File
@@ -21,7 +21,7 @@ jobs:
steps: steps:
- name: Checkout code - name: Checkout code
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0 uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with: with:
fetch-depth: 0 fetch-depth: 0
persist-credentials: false persist-credentials: false
@@ -45,7 +45,7 @@ jobs:
steps: steps:
- name: Checkout code - name: Checkout code
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0 uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with: with:
fetch-depth: 0 fetch-depth: 0
persist-credentials: false persist-credentials: false
@@ -72,7 +72,7 @@ jobs:
steps: steps:
- name: Checkout code - name: Checkout code
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0 uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with: with:
fetch-depth: 0 fetch-depth: 0
persist-credentials: false persist-credentials: false
+5 -5
View File
@@ -23,7 +23,7 @@ jobs:
steps: steps:
- name: Checkout code - name: Checkout code
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0 uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with: with:
persist-credentials: false persist-credentials: false
fetch-depth: 0 fetch-depth: 0
@@ -45,7 +45,7 @@ jobs:
- name: Setup UV and Python - name: Setup UV and Python
if: steps.changed-files.outputs.any_changed == 'true' if: steps.changed-files.outputs.any_changed == 'true'
uses: astral-sh/setup-uv@11f9893b081a58869d3b5fccaea48c9e9e46f990 # v8.3.2 uses: astral-sh/setup-uv@c771a70e6277c0a99b617c7a806ffedaca235ff9 # v9.0.0
with: with:
enable-cache: false enable-cache: false
python-version: "3.12" python-version: "3.12"
@@ -93,7 +93,7 @@ jobs:
steps: steps:
- name: Checkout code - name: Checkout code
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0 uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with: with:
persist-credentials: false persist-credentials: false
@@ -144,7 +144,7 @@ jobs:
steps: steps:
- name: Checkout code - name: Checkout code
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0 uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with: with:
persist-credentials: false persist-credentials: false
@@ -186,7 +186,7 @@ jobs:
steps: steps:
- name: Checkout code - name: Checkout code
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0 uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with: with:
fetch-depth: 0 fetch-depth: 0
persist-credentials: false persist-credentials: false
+1 -1
View File
@@ -24,7 +24,7 @@ jobs:
working-directory: sdks/nodejs-client working-directory: sdks/nodejs-client
steps: steps:
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0 - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with: with:
persist-credentials: false persist-credentials: false
+2 -2
View File
@@ -40,7 +40,7 @@ jobs:
steps: steps:
- name: Checkout repository - name: Checkout repository
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0 uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with: with:
fetch-depth: 0 fetch-depth: 0
token: ${{ secrets.GITHUB_TOKEN }} token: ${{ secrets.GITHUB_TOKEN }}
@@ -158,7 +158,7 @@ jobs:
- name: Run Claude Code for Translation Sync - name: Run Claude Code for Translation Sync
if: steps.context.outputs.CHANGED_FILES != '' if: steps.context.outputs.CHANGED_FILES != ''
uses: anthropics/claude-code-action@af0559ee4f514d1ef21826982bed13f7edc3c35e # v1.0.178 uses: anthropics/claude-code-action@be7b93b1907a4abad570368f3c74b6fe3807510b # v1.0.183
with: with:
anthropic_api_key: ${{ secrets.ANTHROPIC_API_KEY }} anthropic_api_key: ${{ secrets.ANTHROPIC_API_KEY }}
github_token: ${{ secrets.GITHUB_TOKEN }} github_token: ${{ secrets.GITHUB_TOKEN }}
+1 -1
View File
@@ -21,7 +21,7 @@ jobs:
steps: steps:
- name: Checkout repository - name: Checkout repository
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0 uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with: with:
fetch-depth: 0 fetch-depth: 0
+2 -2
View File
@@ -24,7 +24,7 @@ jobs:
steps: steps:
- name: Checkout code - name: Checkout code
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0 uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with: with:
persist-credentials: false persist-credentials: false
@@ -36,7 +36,7 @@ jobs:
remove_tool_cache: true remove_tool_cache: true
- name: Setup UV and Python - name: Setup UV and Python
uses: astral-sh/setup-uv@11f9893b081a58869d3b5fccaea48c9e9e46f990 # v8.3.2 uses: astral-sh/setup-uv@c771a70e6277c0a99b617c7a806ffedaca235ff9 # v9.0.0
with: with:
enable-cache: true enable-cache: true
python-version: ${{ matrix.python-version }} python-version: ${{ matrix.python-version }}
+2 -2
View File
@@ -21,7 +21,7 @@ jobs:
steps: steps:
- name: Checkout code - name: Checkout code
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0 uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with: with:
persist-credentials: false persist-credentials: false
@@ -33,7 +33,7 @@ jobs:
remove_tool_cache: true remove_tool_cache: true
- name: Setup UV and Python - name: Setup UV and Python
uses: astral-sh/setup-uv@11f9893b081a58869d3b5fccaea48c9e9e46f990 # v8.3.2 uses: astral-sh/setup-uv@c771a70e6277c0a99b617c7a806ffedaca235ff9 # v9.0.0
with: with:
enable-cache: true enable-cache: true
python-version: ${{ matrix.python-version }} python-version: ${{ matrix.python-version }}
+2 -2
View File
@@ -26,7 +26,7 @@ jobs:
steps: steps:
- name: Checkout code - name: Checkout code
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0 uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with: with:
persist-credentials: false persist-credentials: false
@@ -34,7 +34,7 @@ jobs:
uses: ./.github/actions/setup-web uses: ./.github/actions/setup-web
- name: Setup UV and Python - name: Setup UV and Python
uses: astral-sh/setup-uv@11f9893b081a58869d3b5fccaea48c9e9e46f990 # v8.3.2 uses: astral-sh/setup-uv@c771a70e6277c0a99b617c7a806ffedaca235ff9 # v9.0.0
with: with:
enable-cache: true enable-cache: true
python-version: "3.12" python-version: "3.12"
+4 -4
View File
@@ -29,7 +29,7 @@ jobs:
steps: steps:
- name: Checkout code - name: Checkout code
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0 uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with: with:
persist-credentials: false persist-credentials: false
@@ -62,7 +62,7 @@ jobs:
steps: steps:
- name: Checkout code - name: Checkout code
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0 uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with: with:
persist-credentials: false persist-credentials: false
@@ -100,7 +100,7 @@ jobs:
steps: steps:
- name: Checkout code - name: Checkout code
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0 uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with: with:
persist-credentials: false persist-credentials: false
@@ -132,7 +132,7 @@ jobs:
steps: steps:
- name: Checkout code - name: Checkout code
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0 uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with: with:
persist-credentials: false persist-credentials: false
+3 -1
View File
@@ -666,6 +666,7 @@ PLUGIN_REMOTE_INSTALL_PORT=5003
PLUGIN_REMOTE_INSTALL_HOST=localhost PLUGIN_REMOTE_INSTALL_HOST=localhost
PLUGIN_MAX_PACKAGE_SIZE=15728640 PLUGIN_MAX_PACKAGE_SIZE=15728640
PLUGIN_MODEL_SCHEMA_CACHE_TTL=3600 PLUGIN_MODEL_SCHEMA_CACHE_TTL=3600
PLUGIN_MODEL_PROVIDERS_CACHE_ENABLED=true
PLUGIN_MODEL_PROVIDERS_CACHE_TTL=86400 PLUGIN_MODEL_PROVIDERS_CACHE_TTL=86400
# Comma-separated marketplace plugin IDs whose latest versions are installed for newly registered users. # Comma-separated marketplace plugin IDs whose latest versions are installed for newly registered users.
# Example: langgenius/openai,langgenius/gemini # Example: langgenius/openai,langgenius/gemini
@@ -677,6 +678,8 @@ INNER_API_KEY_FOR_PLUGIN=QaHbTe77CtuXmsfyhR7+vRjI/+XbV1AaFy691iy+kGDv2Jvy0/eAh8Y
# Dify Agent backend # Dify Agent backend
AGENT_BACKEND_BASE_URL=http://localhost:5050 AGENT_BACKEND_BASE_URL=http://localhost:5050
# Bearer token sent to the Agent backend /runs API. Must match DIFY_AGENT_API_TOKEN on the server side.
AGENT_BACKEND_API_TOKEN=dify-agent-run-token-for-dev-only
AGENT_BACKEND_STREAM_READ_TIMEOUT_SECONDS=30 AGENT_BACKEND_STREAM_READ_TIMEOUT_SECONDS=30
AGENT_BACKEND_STREAM_MAX_RECONNECTS=3 AGENT_BACKEND_STREAM_MAX_RECONNECTS=3
AGENT_BACKEND_RUN_TIMEOUT_SECONDS=1200 AGENT_BACKEND_RUN_TIMEOUT_SECONDS=1200
@@ -729,7 +732,6 @@ OTEL_MAX_EXPORT_BATCH_SIZE=512
OTEL_METRIC_EXPORT_INTERVAL=60000 OTEL_METRIC_EXPORT_INTERVAL=60000
OTEL_BATCH_EXPORT_TIMEOUT=10000 OTEL_BATCH_EXPORT_TIMEOUT=10000
OTEL_METRIC_EXPORT_TIMEOUT=30000 OTEL_METRIC_EXPORT_TIMEOUT=30000
# Prevent Clickjacking # Prevent Clickjacking
ALLOW_EMBED=false ALLOW_EMBED=false
+59 -30
View File
@@ -1,10 +1,13 @@
import logging import logging
import time import time
from collections.abc import Callable
from typing import NamedTuple
import socketio import socketio
from flask import request from flask import request
from opentelemetry.trace import get_current_span from opentelemetry.trace import get_current_span
from opentelemetry.trace.span import INVALID_SPAN_ID, INVALID_TRACE_ID from opentelemetry.trace.span import INVALID_SPAN_ID, INVALID_TRACE_ID
from werkzeug.exceptions import Forbidden, HTTPException, ServiceUnavailable
from configs import dify_config from configs import dify_config
from contexts.wrapper import RecyclableContextVar from contexts.wrapper import RecyclableContextVar
@@ -42,6 +45,53 @@ _CONSOLE_EXEMPT_PREFIXES = (
"/console/api/activate/check", "/console/api/activate/check",
) )
_WEBAPP_EXEMPT_PREFIXES = ("/api/system-features",)
_INVALID_LICENSE_STATUSES = (LicenseStatus.INACTIVE, LicenseStatus.EXPIRED, LicenseStatus.LOST)
def _session_surface_error(license_status: LicenseStatus | None) -> HTTPException:
if license_status is None:
return UnauthorizedAndForceLogout("Unable to verify enterprise license. Please contact your administrator.")
return UnauthorizedAndForceLogout(f"Enterprise license is {license_status}. Please contact your administrator.")
def _bearer_surface_error(license_status: LicenseStatus | None) -> HTTPException:
"""Token-authed: forcing a logout is meaningless and license state must not leak."""
return Forbidden(description="license_required")
def _retryable_surface_error(license_status: LicenseStatus | None) -> HTTPException:
"""Webhook senders retry on 5xx but treat 4xx as permanent, disabling the subscription."""
return ServiceUnavailable(description="license_required")
class _LicenseGatedSurface(NamedTuple):
prefix: str
exempt_prefixes: tuple[str, ...]
build_error: Callable[[LicenseStatus | None], HTTPException]
# /files (plugin-daemon data plane), /inner/api (enterprise control plane) and /health
# stay ungated: blocking them breaks workflow execution or license recovery itself.
_LICENSE_GATED_SURFACES = (
_LicenseGatedSurface("/console/api/", _CONSOLE_EXEMPT_PREFIXES, _session_surface_error),
_LicenseGatedSurface("/api/", _WEBAPP_EXEMPT_PREFIXES, _session_surface_error),
_LicenseGatedSurface("/v1", (), _bearer_surface_error),
_LicenseGatedSurface("/mcp", (), _bearer_surface_error),
_LicenseGatedSurface("/triggers", (), _retryable_surface_error),
)
def _match_license_gated_surface(path: str) -> _LicenseGatedSurface | None:
for surface in _LICENSE_GATED_SURFACES:
if not path.startswith(surface.prefix):
continue
if any(path.startswith(exempt) for exempt in surface.exempt_prefixes):
return None
return surface
return None
# ---------------------------- # ----------------------------
# Application Factory Function # Application Factory Function
@@ -62,38 +112,17 @@ def create_flask_app_with_configs() -> DifyApp:
init_request_context() init_request_context()
RecyclableContextVar.increment_thread_recycles() RecyclableContextVar.increment_thread_recycles()
# Enterprise license validation for API endpoints (both console and webapp)
# When license expires, block all API access except bootstrap endpoints needed
# for the frontend to load the license expiration page without infinite reloads.
if dify_config.ENTERPRISE_ENABLED: if dify_config.ENTERPRISE_ENABLED:
is_console_api = request.path.startswith("/console/api/") surface = _match_license_gated_surface(request.path)
is_webapp_api = request.path.startswith("/api/") if surface is not None:
try:
license_status = EnterpriseService.get_cached_license_status()
except Exception:
logger.exception("Failed to check enterprise license status")
license_status = None
if is_console_api or is_webapp_api: if license_status is None or license_status in _INVALID_LICENSE_STATUSES:
if is_console_api: raise surface.build_error(license_status)
is_exempt = any(request.path.startswith(p) for p in _CONSOLE_EXEMPT_PREFIXES)
else: # webapp API
is_exempt = request.path.startswith("/api/system-features")
if not is_exempt:
try:
# Check license status (cached — see EnterpriseService for TTL details)
license_status = EnterpriseService.get_cached_license_status()
if license_status in (LicenseStatus.INACTIVE, LicenseStatus.EXPIRED, LicenseStatus.LOST):
raise UnauthorizedAndForceLogout(
f"Enterprise license is {license_status}. Please contact your administrator."
)
if license_status is None:
raise UnauthorizedAndForceLogout(
"Unable to verify enterprise license. Please contact your administrator."
)
except UnauthorizedAndForceLogout:
raise
except Exception:
logger.exception("Failed to check enterprise license status")
raise UnauthorizedAndForceLogout(
"Unable to verify enterprise license. Please contact your administrator."
)
# add after request hook for injecting trace headers from OpenTelemetry span context # add after request hook for injecting trace headers from OpenTelemetry span context
# Only adds headers when OTEL is enabled and has valid context # Only adds headers when OTEL is enabled and has valid context
+5 -1
View File
@@ -11,6 +11,7 @@ from clients.agent_backend.fake_client import FakeAgentBackendRunClient, FakeAge
def create_agent_backend_run_client( def create_agent_backend_run_client(
*, *,
base_url: str | None = None, base_url: str | None = None,
api_token: str | None = None,
use_fake: bool = False, use_fake: bool = False,
fake_scenario: str | FakeAgentBackendScenario = FakeAgentBackendScenario.SUCCESS, fake_scenario: str | FakeAgentBackendScenario = FakeAgentBackendScenario.SUCCESS,
stream_read_timeout_seconds: float = 30, stream_read_timeout_seconds: float = 30,
@@ -22,8 +23,11 @@ def create_agent_backend_run_client(
return FakeAgentBackendRunClient(scenario=FakeAgentBackendScenario(fake_scenario)) return FakeAgentBackendRunClient(scenario=FakeAgentBackendScenario(fake_scenario))
if base_url is None: if base_url is None:
raise ValueError("base_url is required when creating a real Agent backend client") raise ValueError("base_url is required when creating a real Agent backend client")
headers: dict[str, str] = {}
if api_token:
headers["Authorization"] = f"Bearer {api_token}"
return DifyAgentBackendRunClient( return DifyAgentBackendRunClient(
Client(base_url=base_url, stream_timeout=stream_read_timeout_seconds), Client(base_url=base_url, stream_timeout=stream_read_timeout_seconds, headers=headers),
stream_max_reconnects=stream_max_reconnects, stream_max_reconnects=stream_max_reconnects,
stream_timeout_seconds=stream_run_timeout_seconds, stream_timeout_seconds=stream_run_timeout_seconds,
) )
@@ -12,6 +12,11 @@ class AgentBackendConfig(BaseSettings):
default=None, default=None,
) )
AGENT_BACKEND_API_TOKEN: str | None = Field(
description="Bearer token for authenticating with the Agent backend /runs API.",
default=None,
)
AGENT_BACKEND_USE_FAKE: bool = Field( AGENT_BACKEND_USE_FAKE: bool = Field(
description="Use the deterministic in-process fake Agent backend client.", description="Use the deterministic in-process fake Agent backend client.",
default=False, default=False,
+42
View File
@@ -266,6 +266,12 @@ class PluginConfig(BaseSettings):
default=60 * 60, default=60 * 60,
) )
PLUGIN_MODEL_PROVIDERS_CACHE_ENABLED: bool = Field(
description="Whether tenant plugin model providers are cached in Redis. Disable when plugins are installed "
"by a system other than this one, which cannot invalidate the cache when a tenant's plugins change.",
default=True,
)
PLUGIN_MODEL_PROVIDERS_CACHE_TTL: PositiveInt = Field( PLUGIN_MODEL_PROVIDERS_CACHE_TTL: PositiveInt = Field(
description="TTL in seconds for caching tenant plugin model providers in Redis", description="TTL in seconds for caching tenant plugin model providers in Redis",
default=60 * 60 * 24, default=60 * 60 * 24,
@@ -816,6 +822,41 @@ class UpdateConfig(BaseSettings):
) )
class CommunityTelemetryConfig(BaseSettings):
"""
Configuration for anonymous self-hosted community telemetry.
"""
DISABLE_TELEMETRY: bool = Field(
description="Disable anonymous community telemetry",
default=False,
)
DO_NOT_TRACK: bool = Field(
description="Respect the standard do-not-track opt-out signal for telemetry",
default=False,
)
TELEMETRY_ENDPOINT: str = Field(
description="Endpoint for anonymous community telemetry events",
default="https://otel.dify.ai/v1/events",
)
TELEMETRY_FALLBACK_ENDPOINT: str = Field(
description="Fallback endpoint for anonymous community telemetry events",
default="https://otel.dify.cn/v1/events",
)
TELEMETRY_TIMEOUT_SECONDS: PositiveInt = Field(
description="HTTP timeout in seconds for anonymous community telemetry requests",
default=3,
)
TELEMETRY_HEARTBEAT_INTERVAL_MINUTES: PositiveInt = Field(
description="Celery beat interval in minutes for checking whether heartbeat telemetry is due",
default=30,
)
CI: bool = Field(
description="Whether the process is running in CI; telemetry is skipped when true",
default=False,
)
class WorkflowVariableTruncationConfig(BaseSettings): class WorkflowVariableTruncationConfig(BaseSettings):
WORKFLOW_VARIABLE_TRUNCATION_MAX_SIZE: PositiveInt = Field( WORKFLOW_VARIABLE_TRUNCATION_MAX_SIZE: PositiveInt = Field(
# 1000 KiB # 1000 KiB
@@ -1599,6 +1640,7 @@ class FeatureConfig(
TenantIsolatedTaskQueueConfig, TenantIsolatedTaskQueueConfig,
ToolConfig, ToolConfig,
UpdateConfig, UpdateConfig,
CommunityTelemetryConfig,
WorkflowConfig, WorkflowConfig,
WorkflowNodeExecutionConfig, WorkflowNodeExecutionConfig,
WorkspaceConfig, WorkspaceConfig,
+2
View File
@@ -144,6 +144,7 @@ from .workspace import (
models, models,
plugin, plugin,
rbac, rbac,
skills,
snippets, snippets,
tool_providers, tool_providers,
trigger_providers, trigger_providers,
@@ -225,6 +226,7 @@ __all__ = [
"saved_message", "saved_message",
"setup", "setup",
"site", "site",
"skills",
"snippet_workflow", "snippet_workflow",
"snippet_workflow_draft_variable", "snippet_workflow_draft_variable",
"snippets", "snippets",
-5
View File
@@ -257,7 +257,6 @@ class AgentAppDetailWithSite(GenericAppDetailWithSite):
debug_conversation_has_messages: bool = False debug_conversation_has_messages: bool = False
debug_conversation_message_count: int = 0 debug_conversation_message_count: int = 0
role: str | None = None role: str | None = None
active_config_is_published: bool = False
class AgentDebugConversationRefreshResponse(BaseModel): class AgentDebugConversationRefreshResponse(BaseModel):
@@ -410,10 +409,6 @@ def _serialize_agent_app_detail(
payload["debug_conversation_has_messages"] = message_count > 0 payload["debug_conversation_has_messages"] = message_count > 0
payload["debug_conversation_message_count"] = message_count payload["debug_conversation_message_count"] = message_count
payload["role"] = agent.role or "" payload["role"] = agent.role or ""
payload["active_config_is_published"] = roster_service.active_config_is_published(
tenant_id=app_model.tenant_id,
agent=agent,
)
return payload return payload
+77
View File
@@ -58,6 +58,7 @@ from services.app_service import (
AppResponseView, AppResponseView,
AppService, AppService,
CreateAppParams, CreateAppParams,
RecentAppMode,
StarredAppListParams, StarredAppListParams,
) )
from services.enterprise import rbac_service as enterprise_rbac_service from services.enterprise import rbac_service as enterprise_rbac_service
@@ -139,6 +140,10 @@ class AppListBaseQuery(BaseModel):
raise ValueError("Invalid UUID format in creator_ids.") from exc raise ValueError("Invalid UUID format in creator_ids.") from exc
class RecentAppListQuery(BaseModel):
limit: int = Field(default=8, ge=1, le=8, description="Number of recently modified apps to return (1-8)")
class AppListQuery(AppListBaseQuery): class AppListQuery(AppListBaseQuery):
pass pass
@@ -411,6 +416,33 @@ class AppPartial(AppResponseModel):
return to_timestamp(value) return to_timestamp(value)
class RecentAppResponse(ResponseModel):
id: str
name: str
icon_type: IconType | None = None
icon: str | None = None
icon_background: str | None = None
mode: RecentAppMode
author_name: str | None = None
updated_at: int
permission_keys: list[str] = Field(default_factory=list)
maintainer: str | None = None
@computed_field(return_type=str | None) # type: ignore[prop-decorator]
@property
def icon_url(self) -> str | None:
return build_icon_url(self.icon_type, self.icon)
@field_validator("updated_at", mode="before")
@classmethod
def _normalize_timestamp(cls, value: datetime | int) -> int:
return to_timestamp(value)
class RecentAppListResponse(ResponseModel):
data: list[RecentAppResponse]
class AppDetail(AppResponseModel): class AppDetail(AppResponseModel):
id: str id: str
name: str name: str
@@ -575,6 +607,8 @@ register_schema_models(
register_response_schema_models( register_response_schema_models(
console_ns, console_ns,
AppPartial, AppPartial,
RecentAppResponse,
RecentAppListResponse,
AppDetailWithSite, AppDetailWithSite,
AppPagination, AppPagination,
) )
@@ -699,6 +733,49 @@ class AppListApi(Resource):
return app_detail.model_dump(mode="json"), 201 return app_detail.model_dump(mode="json"), 201
@console_ns.route("/apps/recent")
class RecentAppListApi(Resource):
@console_ns.doc("list_recent_apps")
@console_ns.doc(description="Get recently modified apps for the home Continue Work section")
@console_ns.doc(params=query_params_from_model(RecentAppListQuery))
@console_ns.response(200, "Success", console_ns.models[RecentAppListResponse.__name__])
@setup_required
@login_required
@account_initialization_required
@enterprise_license_required
@with_session(write=False)
@with_current_user_id
@with_current_tenant_id
def get(self, current_tenant_id: str, current_user_id: str, session: Session):
"""Return the lightweight app cards needed by the Explore home page."""
args = query_params_from_request(RecentAppListQuery)
params = AppListParams(limit=args.limit)
permissions = enterprise_rbac_service.RBACService.MyPermissions.get(
current_tenant_id,
current_user_id,
session=session,
)
if dify_config.RBAC_ENABLED:
access_filter = resolve_app_access_filter(
current_tenant_id,
current_user_id,
session=session,
permissions=permissions,
)
access_filter.apply_to_params(params)
recent_apps = AppService().get_recent_apps(current_user_id, current_tenant_id, params, session)
permission_keys_map = permissions.app.permission_keys_by_resource_ids([app.id for app in recent_apps])
response_items = [
RecentAppResponse.model_validate(app, from_attributes=True).model_copy(
update={"permission_keys": permission_keys_map.get(app.id, [])}
)
for app in recent_apps
]
return dump_response(RecentAppListResponse, {"data": response_items}), 200
@console_ns.route("/apps/starred") @console_ns.route("/apps/starred")
class StarredAppListApi(Resource): class StarredAppListApi(Resource):
@console_ns.doc("list_starred_apps") @console_ns.doc("list_starred_apps")
+1 -1
View File
@@ -59,7 +59,7 @@ class TagBindingRemovePayload(BaseModel):
class TagListQueryParam(BaseModel): class TagListQueryParam(BaseModel):
type: Literal["knowledge", "app", "snippet", ""] = Field("", description="Tag type filter") type: TagType | Literal[""] = Field("", description="Tag type filter")
keyword: str | None = Field(None, description="Search keyword") keyword: str | None = Field(None, description="Search keyword")
+834
View File
@@ -0,0 +1,834 @@
"""Console API for workspace-level Skill Management."""
from __future__ import annotations
import io
from flask import request, send_file
from flask_restx import Resource
from pydantic import BaseModel, ConfigDict, Field, ValidationError
from controllers.common.fields import BinaryFileResponse
from controllers.common.schema import query_params_from_model, register_response_schema_models, register_schema_models
from controllers.console import console_ns
from controllers.console.wraps import (
account_initialization_required,
edit_permission_required,
setup_required,
with_current_tenant_id,
with_current_user,
)
from fields.base import ResponseModel
from libs import helper
from libs.helper import dump_response
from libs.login import login_required
from models.account import Account
from services.skill_management_service import (
SkillAssistMessagePayload,
SkillCreatePayload,
SkillDraftFileOperationPayload,
SkillDraftTreePayload,
SkillImportPayload,
SkillManagementService,
SkillManagementServiceError,
SkillMetadataPayload,
SkillPublishPayload,
SkillRestorePayload,
SkillVersionUpdatePayload,
)
_FILE_UPLOAD_PARAMS = {
"file": {
"description": "Skill draft file payload",
"in": "formData",
"type": "file",
"required": True,
},
}
class WorkspaceSkillsQuery(BaseModel):
keyword: str | None = Field(default=None, description="Search keyword matching skill name or description.")
page: int = Field(default=1, ge=1, le=99999, description="Page number.")
limit: int = Field(default=20, ge=1, le=100, description="Number of items per page.")
tag: list[str] = Field(
default_factory=list,
description="Skill tag filters. Repeat the parameter for multiple tags.",
)
class SkillDeletePayload(BaseModel):
model_config = ConfigDict(extra="forbid")
confirmation_name: str | None = Field(
default=None,
description="Required when deleting a referenced Skill. Must match the Skill name.",
)
class AgentSkillBindingsPayload(BaseModel):
model_config = ConfigDict(extra="forbid")
skill_ids: list[str] = Field(default_factory=list, description="Ordered Skill IDs bound to the Agent.")
class SkillFileQuery(BaseModel):
path: str = Field(description="Skill file path relative to the Skill root.")
version_id: str | None = Field(default=None, description="Optional published version ID. Omit for current draft.")
class SkillResponse(ResponseModel):
id: str
name: str
display_name: str
icon: str
description: str
tags: list[str] = Field(default_factory=list)
name_manually_edited: bool = False
visibility: str
latest_published_version_id: str | None = None
reference_count: int = 0
created_by: str | None = None
created_by_name: str | None = None
updated_by: str | None = None
updated_by_name: str | None = None
created_at: int
updated_at: int
class SkillFileResponse(ResponseModel):
id: str | None = None
path: str
kind: str
storage: str | None = None
mime_type: str | None = None
content: str | None = None
tool_file_id: str | None = None
size: int | None = None
hash: str | None = None
class SkillFilePreviewResponse(ResponseModel):
path: str
mime_type: str
content: str
size: int
hash: str
class SkillFileUploadResponse(ResponseModel):
id: str
name: str
mime_type: str
size: int
hash: str
class SkillDetailResponse(SkillResponse):
files: list[SkillFileResponse] = Field(default_factory=list)
class SkillListResponse(ResponseModel):
data: list[SkillResponse] = Field(default_factory=list)
has_more: bool = False
limit: int = 20
page: int = 1
total: int = 0
class SkillTagResponse(ResponseModel):
tag: str
count: int
class SkillTagListResponse(ResponseModel):
data: list[SkillTagResponse] = Field(default_factory=list)
class SkillVersionResponse(ResponseModel):
id: str
skill_id: str
version_number: int
version_name: str
publish_note: str
hash_code: str
archive_size: int
published_by: str | None = None
published_by_name: str | None = None
is_latest: bool = False
created_at: int
class SkillVersionListResponse(ResponseModel):
data: list[SkillVersionResponse] = Field(default_factory=list)
class SkillVersionDetailResponse(SkillVersionResponse):
files: list[SkillFileResponse] = Field(default_factory=list)
class SkillVersionDeleteResponse(ResponseModel):
id: str
deleted: bool
latest_published_version_id: str | None = None
class SkillReferenceResponse(ResponseModel):
type: str
agent_id: str
agent_icon: str | None = None
agent_icon_background: str | None = None
agent_icon_type: str | None = None
app_id: str | None = None
name: str
display_name: str
workflow_id: str | None = None
workflow_name: str | None = None
workflow_icon: str | None = None
workflow_icon_background: str | None = None
workflow_icon_type: str | None = None
workflow_version: str | None = None
node_id: str | None = None
node_name: str | None = None
class SkillReferenceListResponse(ResponseModel):
data: list[SkillReferenceResponse] = Field(default_factory=list)
class SkillDeleteResponse(ResponseModel):
id: str
deleted: bool
class AgentSkillBindingItemResponse(ResponseModel):
id: str
priority: int
name: str
display_name: str
icon: str
description: str
tags: list[str] = Field(default_factory=list)
status: str
file_count: int
latest_published_version_id: str | None = None
latest_published_at: int | None = None
updated_at: int
class AgentSkillBindingsResponse(ResponseModel):
agent_id: str
skill_ids: list[str] = Field(default_factory=list)
data: list[AgentSkillBindingItemResponse] = Field(default_factory=list)
register_schema_models(
console_ns,
WorkspaceSkillsQuery,
SkillCreatePayload,
SkillAssistMessagePayload,
SkillMetadataPayload,
SkillDraftFileOperationPayload,
SkillDraftTreePayload,
SkillPublishPayload,
SkillRestorePayload,
SkillVersionUpdatePayload,
SkillDeletePayload,
SkillFileQuery,
AgentSkillBindingsPayload,
)
register_response_schema_models(
console_ns,
SkillResponse,
SkillFileResponse,
SkillFilePreviewResponse,
SkillFileUploadResponse,
SkillDetailResponse,
SkillListResponse,
SkillTagResponse,
SkillTagListResponse,
SkillVersionResponse,
SkillVersionListResponse,
SkillVersionDetailResponse,
SkillVersionDeleteResponse,
SkillReferenceResponse,
SkillReferenceListResponse,
SkillDeleteResponse,
AgentSkillBindingItemResponse,
AgentSkillBindingsResponse,
BinaryFileResponse,
)
def _error_response(exc: SkillManagementServiceError) -> tuple[dict[str, object], int]:
body: dict[str, object] = {"code": exc.code, "message": exc.message}
if exc.details:
body["details"] = exc.details
return body, exc.status_code
@console_ns.route("/workspaces/current/skills")
class WorkspaceSkillsApi(Resource):
@console_ns.doc(params=query_params_from_model(WorkspaceSkillsQuery))
@console_ns.response(200, "Workspace skills", console_ns.models[SkillListResponse.__name__])
@setup_required
@login_required
@account_initialization_required
@with_current_tenant_id
def get(self, current_tenant_id: str):
query_input: dict[str, object] = {
"keyword": request.args.get("keyword"),
"tag": request.args.getlist("tag"),
}
if "limit" in request.args:
query_input["limit"] = request.args.get("limit")
if "page" in request.args:
query_input["page"] = request.args.get("page")
query = WorkspaceSkillsQuery.model_validate(query_input)
result = SkillManagementService().list_skills(
tenant_id=current_tenant_id,
keyword=query.keyword,
page=query.page,
limit=query.limit,
tags=[tag for tag in query.tag if tag],
)
return dump_response(SkillListResponse, result)
@console_ns.expect(console_ns.models[SkillCreatePayload.__name__])
@console_ns.response(201, "Skill created", console_ns.models[SkillDetailResponse.__name__])
@setup_required
@login_required
@account_initialization_required
@edit_permission_required
@with_current_user
@with_current_tenant_id
def post(self, current_tenant_id: str, current_user: Account):
try:
payload = SkillCreatePayload.model_validate(console_ns.payload or {})
result = SkillManagementService().create_skill(
tenant_id=current_tenant_id,
user_id=current_user.id,
payload=payload,
)
return dump_response(SkillDetailResponse, result), 201
except ValidationError as exc:
return {"code": "invalid_request", "message": str(exc)}, 400
except ValueError as exc:
return {"code": "invalid_request", "message": str(exc)}, 400
except SkillManagementServiceError as exc:
return _error_response(exc)
@console_ns.route("/workspaces/current/skills/files/upload")
class WorkspaceSkillFileUploadApi(Resource):
@console_ns.doc(consumes=["multipart/form-data"], params=_FILE_UPLOAD_PARAMS)
@console_ns.response(201, "Skill draft file uploaded", console_ns.models[SkillFileUploadResponse.__name__])
@setup_required
@login_required
@account_initialization_required
@edit_permission_required
@with_current_user
@with_current_tenant_id
def post(self, current_tenant_id: str, current_user: Account):
if "file" not in request.files:
return {"code": "no_file_uploaded", "message": "no file uploaded"}, 400
file = request.files["file"]
if not file.filename:
return {"code": "filename_missing", "message": "filename is required"}, 400
result = SkillManagementService().upload_file(
tenant_id=current_tenant_id,
user_id=current_user.id,
filename=file.filename,
content=file.stream.read(),
mime_type=file.mimetype,
)
return dump_response(SkillFileUploadResponse, result), 201
@console_ns.route("/workspaces/current/skills/tags")
class WorkspaceSkillTagsApi(Resource):
@console_ns.response(200, "Workspace Skill tags", console_ns.models[SkillTagListResponse.__name__])
@setup_required
@login_required
@account_initialization_required
@with_current_tenant_id
def get(self, current_tenant_id: str):
result = SkillManagementService().list_tags(tenant_id=current_tenant_id)
return dump_response(SkillTagListResponse, result)
@console_ns.route("/workspaces/current/skills/import")
class WorkspaceSkillImportApi(Resource):
@console_ns.doc(description="Import a Skill zip package from multipart form field `file`.")
@console_ns.response(201, "Skill imported", console_ns.models[SkillDetailResponse.__name__])
@setup_required
@login_required
@account_initialization_required
@edit_permission_required
@with_current_user
@with_current_tenant_id
def post(self, current_tenant_id: str, current_user: Account):
upload = request.files.get("file")
if upload is None:
return {"code": "invalid_request", "message": "file is required"}, 400
try:
payload = SkillImportPayload(content=upload.read(), filename=upload.filename or "skill.zip")
result = SkillManagementService().import_skill(
tenant_id=current_tenant_id,
user_id=current_user.id,
payload=payload,
)
return dump_response(SkillDetailResponse, result), 201
except (ValidationError, ValueError) as exc:
return {"code": "invalid_request", "message": str(exc)}, 400
except SkillManagementServiceError as exc:
return _error_response(exc)
@console_ns.route("/workspaces/current/skills/<string:skill_id>")
class WorkspaceSkillApi(Resource):
@console_ns.response(200, "Skill detail", console_ns.models[SkillDetailResponse.__name__])
@setup_required
@login_required
@account_initialization_required
@with_current_tenant_id
def get(self, current_tenant_id: str, skill_id: str):
try:
result = SkillManagementService().get_skill(tenant_id=current_tenant_id, skill_id=skill_id)
return dump_response(SkillDetailResponse, result)
except SkillManagementServiceError as exc:
return _error_response(exc)
@console_ns.expect(console_ns.models[SkillMetadataPayload.__name__])
@console_ns.response(200, "Skill updated", console_ns.models[SkillResponse.__name__])
@setup_required
@login_required
@account_initialization_required
@edit_permission_required
@with_current_user
@with_current_tenant_id
def patch(self, current_tenant_id: str, current_user: Account, skill_id: str):
try:
payload = SkillMetadataPayload.model_validate(console_ns.payload or {})
result = SkillManagementService().update_metadata(
tenant_id=current_tenant_id,
user_id=current_user.id,
skill_id=skill_id,
payload=payload,
)
return dump_response(SkillResponse, result)
except ValidationError as exc:
return {"code": "invalid_request", "message": str(exc)}, 400
except ValueError as exc:
return {"code": "invalid_request", "message": str(exc)}, 400
except SkillManagementServiceError as exc:
return _error_response(exc)
@console_ns.expect(console_ns.models[SkillDeletePayload.__name__])
@console_ns.response(200, "Skill deleted", console_ns.models[SkillDeleteResponse.__name__])
@setup_required
@login_required
@account_initialization_required
@edit_permission_required
@with_current_tenant_id
def delete(self, current_tenant_id: str, skill_id: str):
try:
payload = SkillDeletePayload.model_validate(console_ns.payload or {})
result = SkillManagementService().delete_skill(
tenant_id=current_tenant_id,
skill_id=skill_id,
confirmation_name=payload.confirmation_name,
)
return dump_response(SkillDeleteResponse, result)
except ValidationError as exc:
return {"code": "invalid_request", "message": str(exc)}, 400
except SkillManagementServiceError as exc:
return _error_response(exc)
@console_ns.route("/workspaces/current/skills/<string:skill_id>/duplicate")
class WorkspaceSkillDuplicateApi(Resource):
@console_ns.response(201, "Skill duplicated", console_ns.models[SkillDetailResponse.__name__])
@setup_required
@login_required
@account_initialization_required
@edit_permission_required
@with_current_user
@with_current_tenant_id
def post(self, current_tenant_id: str, current_user: Account, skill_id: str):
try:
result = SkillManagementService().duplicate_skill(
tenant_id=current_tenant_id,
user_id=current_user.id,
skill_id=skill_id,
)
return dump_response(SkillDetailResponse, result), 201
except SkillManagementServiceError as exc:
return _error_response(exc)
@console_ns.route("/workspaces/current/skills/<string:skill_id>/export")
class WorkspaceSkillExportApi(Resource):
@console_ns.response(200, "Published Skill zip archive")
@setup_required
@login_required
@account_initialization_required
@with_current_tenant_id
def get(self, current_tenant_id: str, skill_id: str):
try:
result = SkillManagementService().pull_published_archive(tenant_id=current_tenant_id, skill_id=skill_id)
return send_file(
io.BytesIO(result.payload),
mimetype=result.mime_type,
as_attachment=True,
download_name=result.filename,
)
except SkillManagementServiceError as exc:
return _error_response(exc)
@console_ns.route("/workspaces/current/skills/<string:skill_id>/assist/messages")
class WorkspaceSkillAssistMessageApi(Resource):
"""Stream read-only Skill Authoring suggestions from the default workspace model."""
@console_ns.expect(console_ns.models[SkillAssistMessagePayload.__name__])
@console_ns.response(200, "Skill Authoring assistant event stream")
@setup_required
@login_required
@account_initialization_required
@with_current_user
@with_current_tenant_id
def post(self, current_tenant_id: str, current_user: Account, skill_id: str):
try:
payload = SkillAssistMessagePayload.model_validate(console_ns.payload or {})
response = SkillManagementService().create_assistant_action_stream(
tenant_id=current_tenant_id,
skill_id=skill_id,
user_id=current_user.id,
message=payload.message,
attachments=payload.attachments,
model_payload=payload.model,
target_path=payload.target_path,
)
except ValidationError as exc:
return {"code": "invalid_request", "message": str(exc)}, 400
except SkillManagementServiceError as exc:
return _error_response(exc)
return helper.compact_generate_response(response)
@console_ns.route("/workspaces/current/skills/<string:skill_id>/files")
class WorkspaceSkillFilesApi(Resource):
@console_ns.expect(console_ns.models[SkillDraftFileOperationPayload.__name__])
@console_ns.response(200, "Draft file operation applied", console_ns.models[SkillDetailResponse.__name__])
@setup_required
@login_required
@account_initialization_required
@edit_permission_required
@with_current_user
@with_current_tenant_id
def patch(self, current_tenant_id: str, current_user: Account, skill_id: str):
try:
payload = SkillDraftFileOperationPayload.model_validate(console_ns.payload or {})
result = SkillManagementService().apply_draft_file_operation(
tenant_id=current_tenant_id,
user_id=current_user.id,
skill_id=skill_id,
payload=payload,
)
return dump_response(SkillDetailResponse, result)
except ValidationError as exc:
return {"code": "invalid_request", "message": str(exc)}, 400
except ValueError as exc:
return {"code": "invalid_request", "message": str(exc)}, 400
except SkillManagementServiceError as exc:
return _error_response(exc)
@console_ns.expect(console_ns.models[SkillDraftTreePayload.__name__])
@console_ns.response(200, "Draft files replaced", console_ns.models[SkillDetailResponse.__name__])
@setup_required
@login_required
@account_initialization_required
@edit_permission_required
@with_current_user
@with_current_tenant_id
def put(self, current_tenant_id: str, current_user: Account, skill_id: str):
try:
payload = SkillDraftTreePayload.model_validate(console_ns.payload or {})
result = SkillManagementService().replace_draft_tree(
tenant_id=current_tenant_id,
user_id=current_user.id,
skill_id=skill_id,
payload=payload,
)
return dump_response(SkillDetailResponse, result)
except ValidationError as exc:
return {"code": "invalid_request", "message": str(exc)}, 400
except ValueError as exc:
return {"code": "invalid_request", "message": str(exc)}, 400
except SkillManagementServiceError as exc:
return _error_response(exc)
@console_ns.route("/workspaces/current/skills/<string:skill_id>/files/preview")
class WorkspaceSkillFilePreviewApi(Resource):
@console_ns.doc(params=query_params_from_model(SkillFileQuery))
@console_ns.response(200, "Skill file text preview", console_ns.models[SkillFilePreviewResponse.__name__])
@setup_required
@login_required
@account_initialization_required
@with_current_tenant_id
def get(self, current_tenant_id: str, skill_id: str):
try:
query = SkillFileQuery.model_validate(
{
"path": request.args.get("path"),
"version_id": request.args.get("version_id"),
}
)
result = SkillManagementService().preview_file(
tenant_id=current_tenant_id,
skill_id=skill_id,
path=query.path,
version_id=query.version_id,
)
return dump_response(SkillFilePreviewResponse, result)
except ValidationError as exc:
return {"code": "invalid_request", "message": str(exc)}, 400
except ValueError as exc:
return {"code": "invalid_request", "message": str(exc)}, 400
except SkillManagementServiceError as exc:
return _error_response(exc)
@console_ns.route("/workspaces/current/skills/<string:skill_id>/files/content")
class WorkspaceSkillFileContentApi(Resource):
@console_ns.doc(params={**query_params_from_model(SkillFileQuery), "download": "Return as an attachment when 1."})
@console_ns.response(200, "Skill file content", console_ns.models[BinaryFileResponse.__name__])
@setup_required
@login_required
@account_initialization_required
@with_current_tenant_id
def get(self, current_tenant_id: str, skill_id: str):
try:
query = SkillFileQuery.model_validate(
{
"path": request.args.get("path"),
"version_id": request.args.get("version_id"),
}
)
result = SkillManagementService().pull_file(
tenant_id=current_tenant_id,
skill_id=skill_id,
path=query.path,
version_id=query.version_id,
)
return send_file(
io.BytesIO(result.payload),
mimetype=result.mime_type,
as_attachment=request.args.get("download") == "1",
download_name=result.filename,
)
except ValidationError as exc:
return {"code": "invalid_request", "message": str(exc)}, 400
except ValueError as exc:
return {"code": "invalid_request", "message": str(exc)}, 400
except SkillManagementServiceError as exc:
return _error_response(exc)
@console_ns.route("/workspaces/current/skills/<string:skill_id>/publish")
class WorkspaceSkillPublishApi(Resource):
@console_ns.expect(console_ns.models[SkillPublishPayload.__name__])
@console_ns.response(200, "Skill published", console_ns.models[SkillVersionResponse.__name__])
@setup_required
@login_required
@account_initialization_required
@edit_permission_required
@with_current_user
@with_current_tenant_id
def post(self, current_tenant_id: str, current_user: Account, skill_id: str):
try:
payload = SkillPublishPayload.model_validate(console_ns.payload or {})
result = SkillManagementService().publish_skill(
tenant_id=current_tenant_id,
user_id=current_user.id,
skill_id=skill_id,
payload=payload,
)
return dump_response(SkillVersionResponse, result)
except ValidationError as exc:
return {"code": "invalid_request", "message": str(exc)}, 400
except SkillManagementServiceError as exc:
return _error_response(exc)
@console_ns.route("/workspaces/current/skills/<string:skill_id>/restore")
class WorkspaceSkillRestoreApi(Resource):
@console_ns.expect(console_ns.models[SkillRestorePayload.__name__])
@console_ns.response(200, "Skill version restored", console_ns.models[SkillVersionResponse.__name__])
@setup_required
@login_required
@account_initialization_required
@edit_permission_required
@with_current_user
@with_current_tenant_id
def post(self, current_tenant_id: str, current_user: Account, skill_id: str):
try:
payload = SkillRestorePayload.model_validate(console_ns.payload or {})
result = SkillManagementService().restore_version(
tenant_id=current_tenant_id,
user_id=current_user.id,
skill_id=skill_id,
payload=payload,
)
return dump_response(SkillVersionResponse, result)
except ValidationError as exc:
return {"code": "invalid_request", "message": str(exc)}, 400
except SkillManagementServiceError as exc:
return _error_response(exc)
@console_ns.route("/workspaces/current/skills/<string:skill_id>/references")
class WorkspaceSkillReferencesApi(Resource):
@console_ns.response(200, "Skill references", console_ns.models[SkillReferenceListResponse.__name__])
@setup_required
@login_required
@account_initialization_required
@with_current_tenant_id
def get(self, current_tenant_id: str, skill_id: str):
try:
result = SkillManagementService().list_skill_references(tenant_id=current_tenant_id, skill_id=skill_id)
return dump_response(SkillReferenceListResponse, result)
except SkillManagementServiceError as exc:
return _error_response(exc)
@console_ns.route("/workspaces/current/skills/<string:skill_id>/versions")
class WorkspaceSkillVersionsApi(Resource):
@console_ns.response(200, "Skill versions", console_ns.models[SkillVersionListResponse.__name__])
@setup_required
@login_required
@account_initialization_required
@with_current_tenant_id
def get(self, current_tenant_id: str, skill_id: str):
try:
result = SkillManagementService().list_versions(tenant_id=current_tenant_id, skill_id=skill_id)
return dump_response(SkillVersionListResponse, result)
except SkillManagementServiceError as exc:
return _error_response(exc)
@console_ns.route("/workspaces/current/skills/<string:skill_id>/versions/<string:version_id>")
class WorkspaceSkillVersionApi(Resource):
@console_ns.response(200, "Skill version detail", console_ns.models[SkillVersionDetailResponse.__name__])
@setup_required
@login_required
@account_initialization_required
@with_current_tenant_id
def get(self, current_tenant_id: str, skill_id: str, version_id: str):
try:
result = SkillManagementService().get_version(
tenant_id=current_tenant_id,
skill_id=skill_id,
version_id=version_id,
)
return dump_response(SkillVersionDetailResponse, result)
except SkillManagementServiceError as exc:
return _error_response(exc)
@console_ns.expect(console_ns.models[SkillVersionUpdatePayload.__name__])
@console_ns.response(200, "Skill version updated", console_ns.models[SkillVersionResponse.__name__])
@setup_required
@login_required
@account_initialization_required
@edit_permission_required
@with_current_tenant_id
def patch(self, current_tenant_id: str, skill_id: str, version_id: str):
try:
payload = SkillVersionUpdatePayload.model_validate(console_ns.payload or {})
result = SkillManagementService().update_version(
tenant_id=current_tenant_id,
skill_id=skill_id,
version_id=version_id,
payload=payload,
)
return dump_response(SkillVersionResponse, result)
except ValidationError as exc:
return {"code": "invalid_request", "message": str(exc)}, 400
except SkillManagementServiceError as exc:
return _error_response(exc)
@console_ns.response(200, "Skill version deleted", console_ns.models[SkillVersionDeleteResponse.__name__])
@setup_required
@login_required
@account_initialization_required
@edit_permission_required
@with_current_user
@with_current_tenant_id
def delete(self, current_tenant_id: str, current_user: Account, skill_id: str, version_id: str):
try:
result = SkillManagementService().delete_version(
tenant_id=current_tenant_id,
user_id=current_user.id,
skill_id=skill_id,
version_id=version_id,
)
return dump_response(SkillVersionDeleteResponse, result)
except SkillManagementServiceError as exc:
return _error_response(exc)
@console_ns.route("/workspaces/current/agents/<string:agent_id>/skills")
class WorkspaceAgentSkillBindingsApi(Resource):
@console_ns.response(200, "Agent Skill bindings", console_ns.models[AgentSkillBindingsResponse.__name__])
@setup_required
@login_required
@account_initialization_required
@with_current_tenant_id
def get(self, current_tenant_id: str, agent_id: str):
result = SkillManagementService().list_agent_bindings(tenant_id=current_tenant_id, agent_id=agent_id)
return dump_response(AgentSkillBindingsResponse, result)
@console_ns.expect(console_ns.models[AgentSkillBindingsPayload.__name__])
@console_ns.response(200, "Agent Skill bindings replaced", console_ns.models[AgentSkillBindingsResponse.__name__])
@setup_required
@login_required
@account_initialization_required
@edit_permission_required
@with_current_user
@with_current_tenant_id
def put(self, current_tenant_id: str, current_user: Account, agent_id: str):
try:
payload = AgentSkillBindingsPayload.model_validate(console_ns.payload or {})
result = SkillManagementService().replace_agent_bindings(
tenant_id=current_tenant_id,
user_id=current_user.id,
agent_id=agent_id,
skill_ids=payload.skill_ids,
)
return dump_response(AgentSkillBindingsResponse, result)
except ValidationError as exc:
return {"code": "invalid_request", "message": str(exc)}, 400
except SkillManagementServiceError as exc:
return _error_response(exc)
__all__ = [
"WorkspaceAgentSkillBindingsApi",
"WorkspaceSkillApi",
"WorkspaceSkillDuplicateApi",
"WorkspaceSkillExportApi",
"WorkspaceSkillFilesApi",
"WorkspaceSkillImportApi",
"WorkspaceSkillPublishApi",
"WorkspaceSkillReferencesApi",
"WorkspaceSkillRestoreApi",
"WorkspaceSkillTagsApi",
"WorkspaceSkillVersionApi",
"WorkspaceSkillVersionsApi",
"WorkspaceSkillsApi",
]
+2
View File
@@ -23,6 +23,7 @@ from .knowledge import retrieval as _knowledge_retrieval
from .plugin import agent_config as _agent_config from .plugin import agent_config as _agent_config
from .plugin import agent_drive as _agent_drive from .plugin import agent_drive as _agent_drive
from .plugin import plugin as _plugin from .plugin import plugin as _plugin
from .plugin import skills as _skills
from .workspace import workspace as _workspace from .workspace import workspace as _workspace
api.add_namespace(inner_api_ns) api.add_namespace(inner_api_ns)
@@ -36,6 +37,7 @@ __all__ = [
"_mail", "_mail",
"_plugin", "_plugin",
"_runtime_credentials", "_runtime_credentials",
"_skills",
"_workspace", "_workspace",
"api", "api",
"bp", "bp",
@@ -0,0 +1,54 @@
"""Inner API for published workspace Skills.
These endpoints are called by trusted runtime services. They expose only
published Skill artifacts, never draft files or editable metadata.
"""
from __future__ import annotations
import io
from flask import request, send_file
from flask_restx import Resource
from pydantic import BaseModel, ValidationError
from controllers.console.wraps import setup_required
from controllers.inner_api import inner_api_ns
from controllers.inner_api.wraps import plugin_inner_api_only
from services.skill_management_service import SkillManagementService, SkillManagementServiceError
class _SkillTargetQuery(BaseModel):
tenant_id: str
def _target_query_from_request() -> _SkillTargetQuery:
return _SkillTargetQuery.model_validate({"tenant_id": request.args.get("tenant_id")})
def _error_response(exc: SkillManagementServiceError) -> tuple[dict[str, str], int]:
return {"code": exc.code, "message": exc.message}, exc.status_code
@inner_api_ns.route("/skills/<string:skill_id>/pull")
class PublishedSkillPullApi(Resource):
@setup_required
@plugin_inner_api_only
@inner_api_ns.doc("published_skill_pull")
def get(self, skill_id: str):
try:
query = _target_query_from_request()
result = SkillManagementService().pull_published_archive(tenant_id=query.tenant_id, skill_id=skill_id)
return send_file(
io.BytesIO(result.payload),
mimetype=result.mime_type,
as_attachment=True,
download_name=result.filename,
)
except ValidationError as exc:
return {"code": "invalid_request", "message": str(exc)}, 400
except SkillManagementServiceError as exc:
return _error_response(exc)
__all__ = ["PublishedSkillPullApi"]
+2 -1
View File
@@ -24,7 +24,7 @@ from extensions.ext_database import db
from fields.base import ResponseModel from fields.base import ResponseModel
from libs.helper import RateLimiter, dump_response, extract_remote_ip, to_timestamp from libs.helper import RateLimiter, dump_response, extract_remote_ip, to_timestamp
from models.account import TenantStatus from models.account import TenantStatus
from models.model import App, Site from models.model import App, AppMode, Site
from repositories.factory import DifyAPIRepositoryFactory from repositories.factory import DifyAPIRepositoryFactory
from services.feature_service import FeatureService from services.feature_service import FeatureService
from services.human_input_file_upload_service import HumanInputFileUploadService from services.human_input_file_upload_service import HumanInputFileUploadService
@@ -207,6 +207,7 @@ class HumanInputFormApi(Resource):
site=WebAppSiteResponse.from_app_site( site=WebAppSiteResponse.from_app_site(
tenant=tenant, tenant=tenant,
app_model=app_model, app_model=app_model,
mode=AppMode.value_of(app_model.mode),
site=site, site=site,
end_user_id=None, end_user_id=None,
features=features, features=features,
+5 -1
View File
@@ -14,7 +14,7 @@ from extensions.storage.storage_type import StorageType
from fields.base import ResponseModel from fields.base import ResponseModel
from libs.helper import build_icon_url from libs.helper import build_icon_url
from models.account import Tenant, TenantStatus from models.account import Tenant, TenantStatus
from models.model import App, EndUser, IconType, Site from models.model import App, AppMode, EndUser, IconType, Site
from services.feature_service import FeatureModel, FeatureService from services.feature_service import FeatureModel, FeatureService
from services.file_service import FileService from services.file_service import FileService
@@ -67,6 +67,7 @@ class WebAppCustomConfigResponse(ResponseModel):
class WebAppSiteResponse(ResponseModel): class WebAppSiteResponse(ResponseModel):
app_id: str app_id: str
mode: AppMode
end_user_id: str | None = None end_user_id: str | None = None
enable_site: bool enable_site: bool
site: WebSiteResponse site: WebSiteResponse
@@ -83,6 +84,7 @@ class WebAppSiteResponse(ResponseModel):
*, *,
tenant: Tenant, tenant: Tenant,
app_model: App, app_model: App,
mode: AppMode,
site: Site, site: Site,
end_user_id: str | None, end_user_id: str | None,
features: FeatureModel, features: FeatureModel,
@@ -109,6 +111,7 @@ class WebAppSiteResponse(ResponseModel):
return cls( return cls(
app_id=app_model.id, app_id=app_model.id,
mode=mode,
end_user_id=end_user_id, end_user_id=end_user_id,
enable_site=app_model.enable_site, enable_site=app_model.enable_site,
site=site_response, site=site_response,
@@ -167,6 +170,7 @@ class AppSiteApi(WebApiResource):
return WebAppSiteResponse.from_app_site( return WebAppSiteResponse.from_app_site(
tenant=tenant, tenant=tenant,
app_model=app_model, app_model=app_model,
mode=AppMode.value_of(app_model.mode_compatible_with_agent_with_session(session=db.session())),
site=site, site=site,
end_user_id=end_user.id, end_user_id=end_user.id,
features=features, features=features,
@@ -616,23 +616,34 @@ class AdvancedChatAppGenerator(MessageBasedAppGenerator):
message_snapshot = MessageSnapshot.from_message(message) message_snapshot = MessageSnapshot.from_message(message)
session.close() session.close()
# return response or stream generator try:
response = self._handle_advanced_chat_response( response = self._handle_advanced_chat_response(
application_generate_entity=application_generate_entity, application_generate_entity=application_generate_entity,
workflow=workflow_snapshot, workflow=workflow_snapshot,
queue_manager=queue_manager, queue_manager=queue_manager,
conversation=conversation_snapshot, conversation=conversation_snapshot,
message=message_snapshot, message=message_snapshot,
user=user, user=user,
stream=stream, stream=stream,
draft_var_saver_factory=self._get_draft_var_saver_factory( draft_var_saver_factory=self._get_draft_var_saver_factory(
invoke_from, invoke_from,
account=user, account=user,
tenant_id=application_generate_entity.app_config.tenant_id, tenant_id=application_generate_entity.app_config.tenant_id,
), ),
) )
converted_response = AdvancedChatAppGenerateResponseConverter.convert(
response=response,
invoke_from=invoke_from,
)
except BaseException:
self._join_worker_thread(worker_thread)
raise
return AdvancedChatAppGenerateResponseConverter.convert(response=response, invoke_from=invoke_from) if isinstance(converted_response, Generator):
return self._wrap_stream_with_worker_thread_join(converted_response, worker_thread)
self._join_worker_thread(worker_thread)
return converted_response
def _generate_worker( def _generate_worker(
self, self,
@@ -538,6 +538,7 @@ class AgentAppGenerator(MessageBasedAppGenerator):
request_builder=AgentAppRuntimeRequestBuilder(credentials_provider=credentials_provider), request_builder=AgentAppRuntimeRequestBuilder(credentials_provider=credentials_provider),
agent_backend_client=create_agent_backend_run_client( agent_backend_client=create_agent_backend_run_client(
base_url=dify_config.AGENT_BACKEND_BASE_URL, base_url=dify_config.AGENT_BACKEND_BASE_URL,
api_token=dify_config.AGENT_BACKEND_API_TOKEN,
use_fake=dify_config.AGENT_BACKEND_USE_FAKE, use_fake=dify_config.AGENT_BACKEND_USE_FAKE,
fake_scenario=dify_config.AGENT_BACKEND_FAKE_SCENARIO, fake_scenario=dify_config.AGENT_BACKEND_FAKE_SCENARIO,
stream_read_timeout_seconds=dify_config.AGENT_BACKEND_STREAM_READ_TIMEOUT_SECONDS, stream_read_timeout_seconds=dify_config.AGENT_BACKEND_STREAM_READ_TIMEOUT_SECONDS,
@@ -43,6 +43,7 @@ from core.workflow.nodes.agent_v2.runtime_request_builder import (
build_config_layer_config, build_config_layer_config,
build_knowledge_layer_config, build_knowledge_layer_config,
build_shell_layer_config, build_shell_layer_config,
load_runtime_agent_skill_configs,
) )
from models.agent_config_entities import AgentSoulConfig, AgentSoulToolsConfig from models.agent_config_entities import AgentSoulConfig, AgentSoulToolsConfig
from models.provider_ids import ModelProviderID from models.provider_ids import ModelProviderID
@@ -125,14 +126,22 @@ class AgentAppRuntimeRequestBuilder:
"cli_tool_count": len(agent_soul.tools.cli_tools), "cli_tool_count": len(agent_soul.tools.cli_tools),
} }
runtime_config_skills = load_runtime_agent_skill_configs(
tenant_id=context.dify_context.tenant_id,
agent_id=context.agent_id,
)
config_layer_config, config_warnings = build_config_layer_config( config_layer_config, config_warnings = build_config_layer_config(
agent_soul, agent_soul,
agent_id=context.agent_id, agent_id=context.agent_id,
config_version_id=context.agent_config_snapshot_id, config_version_id=context.agent_config_snapshot_id,
config_version_kind=context.agent_config_version_kind, config_version_kind=context.agent_config_version_kind,
runtime_config_skills=runtime_config_skills,
) )
append_runtime_warnings(metadata, config_warnings) append_runtime_warnings(metadata, config_warnings)
soul_prompt_resolver = build_config_aware_soul_mention_resolver(agent_soul) soul_prompt_resolver = build_config_aware_soul_mention_resolver(
agent_soul,
runtime_config_skills=runtime_config_skills,
)
knowledge_config = build_knowledge_layer_config(agent_soul) knowledge_config = build_knowledge_layer_config(agent_soul)
request = self._request_builder.build_for_agent_app( request = self._request_builder.build_for_agent_app(
+29
View File
@@ -1,3 +1,5 @@
import logging
import threading
from collections.abc import Generator, Mapping, Sequence from collections.abc import Generator, Mapping, Sequence
from contextlib import AbstractContextManager, nullcontext from contextlib import AbstractContextManager, nullcontext
from typing import TYPE_CHECKING, Any, Union, final from typing import TYPE_CHECKING, Any, Union, final
@@ -23,6 +25,10 @@ from services.workflow_draft_variable_service import DraftVariableSaver as Draft
if TYPE_CHECKING: if TYPE_CHECKING:
from graphon.variables.input_entities import VariableEntity from graphon.variables.input_entities import VariableEntity
logger = logging.getLogger(__name__)
_WORKER_THREAD_JOIN_TIMEOUT_SECONDS = 300
@final @final
class _DebuggerDraftVariableSaver: class _DebuggerDraftVariableSaver:
@@ -64,6 +70,29 @@ class _DebuggerDraftVariableSaver:
class BaseAppGenerator: class BaseAppGenerator:
_file_access_controller: DatabaseFileAccessController = DatabaseFileAccessController() _file_access_controller: DatabaseFileAccessController = DatabaseFileAccessController()
@staticmethod
def _join_worker_thread(worker_thread: threading.Thread) -> None:
# Bound the wait so a leaked app worker cannot occupy an execution slot indefinitely.
worker_thread.join(timeout=_WORKER_THREAD_JOIN_TIMEOUT_SECONDS)
if worker_thread.is_alive():
logger.warning(
"Possible app worker thread leak: thread_name=%s timeout_seconds=%s; "
"continuing without waiting further to avoid occupying an execution slot indefinitely",
worker_thread.name,
_WORKER_THREAD_JOIN_TIMEOUT_SECONDS,
)
@staticmethod
def _wrap_stream_with_worker_thread_join[ResponseT](
response_stream: Generator[ResponseT, None, None],
worker_thread: threading.Thread,
) -> Generator[ResponseT, None, None]:
"""Keep the producer owned by the response stream until both finish."""
try:
yield from response_stream
finally:
BaseAppGenerator._join_worker_thread(worker_thread)
@staticmethod @staticmethod
def _bind_file_access_scope( def _bind_file_access_scope(
*, *,
@@ -351,17 +351,28 @@ class PipelineGenerator(BaseAppGenerator):
user, user,
tenant_id=pipeline.tenant_id, tenant_id=pipeline.tenant_id,
) )
# return response or stream generator try:
response = self._handle_response( response = self._handle_response(
application_generate_entity=application_generate_entity, application_generate_entity=application_generate_entity,
workflow=workflow, workflow=workflow,
queue_manager=queue_manager, queue_manager=queue_manager,
user=user, user=user,
stream=streaming, stream=streaming,
draft_var_saver_factory=draft_var_saver_factory, draft_var_saver_factory=draft_var_saver_factory,
) )
converted_response = WorkflowAppGenerateResponseConverter.convert(
response=response,
invoke_from=invoke_from,
)
except BaseException:
self._join_worker_thread(worker_thread)
raise
return WorkflowAppGenerateResponseConverter.convert(response=response, invoke_from=invoke_from) if isinstance(converted_response, Generator):
return self._wrap_stream_with_worker_thread_join(converted_response, worker_thread)
self._join_worker_thread(worker_thread)
return converted_response
def single_iteration_generate( def single_iteration_generate(
self, self,
+21 -10
View File
@@ -405,17 +405,28 @@ class WorkflowAppGenerator(BaseAppGenerator):
tenant_id=app_model.tenant_id, tenant_id=app_model.tenant_id,
) )
# return response or stream generator try:
response = self._handle_response( response = self._handle_response(
application_generate_entity=application_generate_entity, application_generate_entity=application_generate_entity,
workflow=workflow, workflow=workflow,
queue_manager=queue_manager, queue_manager=queue_manager,
user=user, user=user,
draft_var_saver_factory=draft_var_saver_factory, draft_var_saver_factory=draft_var_saver_factory,
stream=streaming, stream=streaming,
) )
converted_response = WorkflowAppGenerateResponseConverter.convert(
response=response,
invoke_from=invoke_from,
)
except BaseException:
self._join_worker_thread(worker_thread)
raise
return WorkflowAppGenerateResponseConverter.convert(response=response, invoke_from=invoke_from) if isinstance(converted_response, Generator):
return self._wrap_stream_with_worker_thread_join(converted_response, worker_thread)
self._join_worker_thread(worker_thread)
return converted_response
def single_iteration_generate( def single_iteration_generate(
self, self,
@@ -39,15 +39,16 @@ class Jinja2TemplateTransformer(TemplateTransformer):
@override @override
def get_runner_script(cls) -> str: def get_runner_script(cls) -> str:
runner_script = dedent(f""" runner_script = dedent(f"""
import jinja2
import json import json
from base64 import b64decode from base64 import b64decode
from jinja2.sandbox import SandboxedEnvironment
# declare main function # declare main function
def main(**inputs): def main(**inputs):
# Decode base64-encoded template to handle special characters safely # Decode base64-encoded template to handle special characters safely
template_code = b64decode('{cls._template_b64_placeholder}').decode('utf-8') template_code = b64decode('{cls._template_b64_placeholder}').decode('utf-8')
template = jinja2.Template(template_code) env = SandboxedEnvironment()
template = env.from_string(template_code)
return template.render(**inputs) return template.render(**inputs)
# decode and prepare input dict # decode and prepare input dict
@@ -67,12 +68,13 @@ class Jinja2TemplateTransformer(TemplateTransformer):
@override @override
def get_preload_script(cls) -> str: def get_preload_script(cls) -> str:
preload_script = dedent(""" preload_script = dedent("""
import jinja2 from jinja2.sandbox import SandboxedEnvironment
from base64 import b64decode from base64 import b64decode
def _jinja2_preload_(): def _jinja2_preload_():
# prepare jinja2 environment, load template and render before to avoid sandbox issue # prepare jinja2 sandboxed environment, load template and render
template = jinja2.Template('{{s}}') env = SandboxedEnvironment()
template = env.from_string('{{s}}')
template.render(s='a') template.render(s='a')
if __name__ == '__main__': if __name__ == '__main__':
+1 -1
View File
@@ -519,7 +519,7 @@ class IndexingRunner:
def filter_string(text): def filter_string(text):
text = re.sub(r"<\|", "<", text) text = re.sub(r"<\|", "<", text)
text = re.sub(r"\|>", ">", text) text = re.sub(r"\|>", ">", text)
text = re.sub(r"[\x00-\x08\x0B\x0C\x0E-\x1F\x7F\xEF\xBF\xBE]", "", text) text = re.sub(r"[\x00-\x08\x0B\x0C\x0E-\x1F\x7F]", "", text)
# Unicode U+FFFE # Unicode U+FFFE
text = re.sub("\ufffe", "", text) text = re.sub("\ufffe", "", text)
return text return text
+38 -23
View File
@@ -66,7 +66,7 @@ from services.enterprise.plugin_manager_service import (
PreUninstallPluginRequest, PreUninstallPluginRequest,
) )
from services.errors.plugin import PluginInstallationForbiddenError from services.errors.plugin import PluginInstallationForbiddenError
from services.feature_service import FeatureService, PluginInstallationScope from services.feature_service import FeatureService, PluginInstallationPermissionModel, PluginInstallationScope
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
_provider_entities_adapter: TypeAdapter[list[ProviderEntity]] = TypeAdapter(list[ProviderEntity]) _provider_entities_adapter: TypeAdapter[list[ProviderEntity]] = TypeAdapter(list[ProviderEntity])
@@ -434,14 +434,18 @@ class PluginService:
exc_info=True, exc_info=True,
) )
@classmethod
def _fetch_plugin_model_providers_uncached(
cls, tenant_id: str, client: PluginModelClient | None
) -> tuple[ProviderEntity, ...]:
model_client = client or PluginModelClient()
return tuple(cls._to_provider_entity(provider) for provider in model_client.fetch_model_providers(tenant_id))
@classmethod @classmethod
def _fetch_and_cache_plugin_model_providers( def _fetch_and_cache_plugin_model_providers(
cls, tenant_id: str, client: PluginModelClient | None, *, refresh_generation: int | None cls, tenant_id: str, client: PluginModelClient | None, *, refresh_generation: int | None
) -> tuple[ProviderEntity, ...]: ) -> tuple[ProviderEntity, ...]:
model_client = client or PluginModelClient() providers = cls._fetch_plugin_model_providers_uncached(tenant_id, client)
providers = tuple(
cls._to_provider_entity(provider) for provider in model_client.fetch_model_providers(tenant_id)
)
generation = cls._load_plugin_model_providers_generation(tenant_id) generation = cls._load_plugin_model_providers_generation(tenant_id)
if generation is not None and generation == refresh_generation: if generation is not None and generation == refresh_generation:
cls._store_cached_plugin_model_providers(tenant_id, generation, providers) cls._store_cached_plugin_model_providers(tenant_id, generation, providers)
@@ -471,6 +475,9 @@ class PluginService:
are intentionally owned by this service so tenant isolation and cache are intentionally owned by this service so tenant isolation and cache
expiry are handled in one place. expiry are handled in one place.
""" """
if not dify_config.PLUGIN_MODEL_PROVIDERS_CACHE_ENABLED:
return cls._fetch_plugin_model_providers_uncached(tenant_id, client)
deadline = time.monotonic() + cls.PLUGIN_MODEL_PROVIDERS_LOCK_WAIT_TIMEOUT deadline = time.monotonic() + cls.PLUGIN_MODEL_PROVIDERS_LOCK_WAIT_TIMEOUT
while True: while True:
@@ -597,22 +604,30 @@ class PluginService:
return result return result
@staticmethod @staticmethod
def _check_marketplace_only_permission(): def _check_marketplace_only_permission() -> None:
""" """
Check if the marketplace only permission is enabled Check if the marketplace only permission is enabled
""" """
features = FeatureService.get_system_features() permission = PluginService._get_plugin_installation_permission()
if features.plugin_installation_permission.restrict_to_marketplace_only: if permission.restrict_to_marketplace_only:
raise PluginInstallationForbiddenError("Plugin installation is restricted to marketplace only") raise PluginInstallationForbiddenError("Plugin installation is restricted to marketplace only")
@staticmethod @staticmethod
def _check_plugin_installation_scope(plugin_verification: PluginVerification | None): def _get_plugin_installation_permission() -> PluginInstallationPermissionModel:
"""Resolve the validated policy and reject deny-all before any installation side effect."""
permission = FeatureService.get_plugin_installation_permission()
if permission.plugin_installation_scope == PluginInstallationScope.NONE:
raise PluginInstallationForbiddenError("Installing plugins is not allowed")
return permission
@staticmethod
def _check_plugin_installation_scope(plugin_verification: PluginVerification | None) -> None:
""" """
Check the plugin installation scope Check the plugin installation scope
""" """
features = FeatureService.get_system_features() permission = PluginService._get_plugin_installation_permission()
match features.plugin_installation_permission.plugin_installation_scope: match permission.plugin_installation_scope:
case PluginInstallationScope.OFFICIAL_ONLY: case PluginInstallationScope.OFFICIAL_ONLY:
if ( if (
plugin_verification is None plugin_verification is None
@@ -627,10 +642,10 @@ class PluginService:
raise PluginInstallationForbiddenError( raise PluginInstallationForbiddenError(
"Plugin installation is restricted to official and specific partners" "Plugin installation is restricted to official and specific partners"
) )
case PluginInstallationScope.NONE:
raise PluginInstallationForbiddenError("Installing plugins is not allowed")
case PluginInstallationScope.ALL: case PluginInstallationScope.ALL:
pass pass
case _:
raise PluginInstallationForbiddenError("Plugin installation policy is invalid")
@staticmethod @staticmethod
def get_debugging_key(tenant_id: str) -> str: def get_debugging_key(tenant_id: str) -> str:
@@ -900,7 +915,7 @@ class PluginService:
# check if plugin pkg is already downloaded # check if plugin pkg is already downloaded
manager = PluginInstaller() manager = PluginInstaller()
features = FeatureService.get_system_features() permission = PluginService._get_plugin_installation_permission()
try: try:
manager.fetch_plugin_manifest(tenant_id, new_plugin_unique_identifier) manager.fetch_plugin_manifest(tenant_id, new_plugin_unique_identifier)
@@ -912,7 +927,7 @@ class PluginService:
response = manager.upload_pkg( response = manager.upload_pkg(
tenant_id, tenant_id,
pkg, pkg,
verify_signature=features.plugin_installation_permission.restrict_to_marketplace_only, verify_signature=permission.restrict_to_marketplace_only,
) )
# check if the plugin is available to install # check if the plugin is available to install
@@ -967,11 +982,11 @@ class PluginService:
""" """
PluginService._check_marketplace_only_permission() PluginService._check_marketplace_only_permission()
manager = PluginInstaller() manager = PluginInstaller()
features = FeatureService.get_system_features() permission = PluginService._get_plugin_installation_permission()
response = manager.upload_pkg( response = manager.upload_pkg(
tenant_id, tenant_id,
pkg, pkg,
verify_signature=features.plugin_installation_permission.restrict_to_marketplace_only, verify_signature=permission.restrict_to_marketplace_only,
) )
PluginService._check_plugin_installation_scope(response.verification) PluginService._check_plugin_installation_scope(response.verification)
@@ -989,13 +1004,13 @@ class PluginService:
pkg = download_with_size_limit( pkg = download_with_size_limit(
f"https://github.com/{repo}/releases/download/{version}/{package}", dify_config.PLUGIN_MAX_PACKAGE_SIZE f"https://github.com/{repo}/releases/download/{version}/{package}", dify_config.PLUGIN_MAX_PACKAGE_SIZE
) )
features = FeatureService.get_system_features() permission = PluginService._get_plugin_installation_permission()
manager = PluginInstaller() manager = PluginInstaller()
response = manager.upload_pkg( response = manager.upload_pkg(
tenant_id, tenant_id,
pkg, pkg,
verify_signature=features.plugin_installation_permission.restrict_to_marketplace_only, verify_signature=permission.restrict_to_marketplace_only,
) )
PluginService._check_plugin_installation_scope(response.verification) PluginService._check_plugin_installation_scope(response.verification)
@@ -1069,7 +1084,7 @@ class PluginService:
if not dify_config.MARKETPLACE_ENABLED: if not dify_config.MARKETPLACE_ENABLED:
raise ValueError("marketplace is not enabled") raise ValueError("marketplace is not enabled")
features = FeatureService.get_system_features() permission = PluginService._get_plugin_installation_permission()
manager = PluginInstaller() manager = PluginInstaller()
try: try:
@@ -1079,7 +1094,7 @@ class PluginService:
response = manager.upload_pkg( response = manager.upload_pkg(
tenant_id, tenant_id,
pkg, pkg,
verify_signature=features.plugin_installation_permission.restrict_to_marketplace_only, verify_signature=permission.restrict_to_marketplace_only,
) )
# check if the plugin is available to install # check if the plugin is available to install
PluginService._check_plugin_installation_scope(response.verification) PluginService._check_plugin_installation_scope(response.verification)
@@ -1101,7 +1116,7 @@ class PluginService:
# collect actual plugin_unique_identifiers # collect actual plugin_unique_identifiers
actual_plugin_unique_identifiers = [] actual_plugin_unique_identifiers = []
metas = [] metas = []
features = FeatureService.get_system_features() permission = PluginService._get_plugin_installation_permission()
# check if already downloaded # check if already downloaded
for plugin_unique_identifier in plugin_unique_identifiers: for plugin_unique_identifier in plugin_unique_identifiers:
@@ -1119,7 +1134,7 @@ class PluginService:
response = manager.upload_pkg( response = manager.upload_pkg(
tenant_id, tenant_id,
pkg, pkg,
verify_signature=features.plugin_installation_permission.restrict_to_marketplace_only, verify_signature=permission.restrict_to_marketplace_only,
) )
# check if the plugin is available to install # check if the plugin is available to install
PluginService._check_plugin_installation_scope(response.verification) PluginService._check_plugin_installation_scope(response.verification)
+1 -1
View File
@@ -9,7 +9,7 @@ class CleanProcessor:
# remove invalid symbol # remove invalid symbol
text = re.sub(r"<\|", "<", text) text = re.sub(r"<\|", "<", text)
text = re.sub(r"\|>", ">", text) text = re.sub(r"\|>", ">", text)
text = re.sub(r"[\x00-\x08\x0B\x0C\x0E-\x1F\x7F\xEF\xBF\xBE]", "", text) text = re.sub(r"[\x00-\x08\x0B\x0C\x0E-\x1F\x7F]", "", text)
# Unicode U+FFFE # Unicode U+FFFE
text = re.sub("\ufffe", "", text) text = re.sub("\ufffe", "", text)
@@ -2023,6 +2023,8 @@ class DatasetRetrieval:
redis_client.zremrangebyscore(key, 0, current_time - 60000) redis_client.zremrangebyscore(key, 0, current_time - 60000)
request_count = redis_client.zcard(key) request_count = redis_client.zcard(key)
if request_count > knowledge_rate_limit.limit: if request_count > knowledge_rate_limit.limit:
# The rate-limit exception is raised after this block, so commit the audit row
# explicitly instead of relying on the Session context, which only closes it.
with session_factory.create_session() as session: with session_factory.create_session() as session:
rate_limit_log = RateLimitLog( rate_limit_log = RateLimitLog(
tenant_id=tenant_id, tenant_id=tenant_id,
@@ -2030,6 +2032,7 @@ class DatasetRetrieval:
operation="knowledge", operation="knowledge",
) )
session.add(rate_limit_log) session.add(rate_limit_log)
session.commit()
raise exc.RateLimitExceededError( raise exc.RateLimitExceededError(
"you have reached the knowledge base request rate limit of your subscription." "you have reached the knowledge base request rate limit of your subscription."
) )
+2 -2
View File
@@ -91,8 +91,8 @@ class FixedRecursiveCharacterTextSplitter(EnhanceRecursiveCharacterTextSplitter)
splits = re.split(r" +", text) splits = re.split(r" +", text)
else: else:
splits = text.split(separator) splits = text.split(separator)
if self._keep_separator: if self._keep_separator:
splits = [s + separator for s in splits[:-1]] + splits[-1:] splits = [s + separator for s in splits[:-1]] + splits[-1:]
else: else:
splits = list(text) splits = list(text)
if separator == "\n": if separator == "\n":
+2
View File
@@ -107,6 +107,8 @@ class MCPTool(Tool):
if self.entity.output_schema and result.structuredContent: if self.entity.output_schema and result.structuredContent:
for k, v in result.structuredContent.items(): for k, v in result.structuredContent.items():
yield self.create_variable_message(k, v) yield self.create_variable_message(k, v)
elif result.structuredContent:
yield self.create_json_message(result.structuredContent)
def _process_text_content(self, content: TextContent) -> Generator[ToolInvokeMessage, None, None]: def _process_text_content(self, content: TextContent) -> Generator[ToolInvokeMessage, None, None]:
"""Process text content and yield appropriate messages.""" """Process text content and yield appropriate messages."""
+1
View File
@@ -497,6 +497,7 @@ class DifyNodeFactory(NodeFactory):
), ),
"agent_backend_client": create_agent_backend_run_client( "agent_backend_client": create_agent_backend_run_client(
base_url=dify_config.AGENT_BACKEND_BASE_URL, base_url=dify_config.AGENT_BACKEND_BASE_URL,
api_token=dify_config.AGENT_BACKEND_API_TOKEN,
use_fake=dify_config.AGENT_BACKEND_USE_FAKE, use_fake=dify_config.AGENT_BACKEND_USE_FAKE,
fake_scenario=dify_config.AGENT_BACKEND_FAKE_SCENARIO, fake_scenario=dify_config.AGENT_BACKEND_FAKE_SCENARIO,
stream_read_timeout_seconds=dify_config.AGENT_BACKEND_STREAM_READ_TIMEOUT_SECONDS, stream_read_timeout_seconds=dify_config.AGENT_BACKEND_STREAM_READ_TIMEOUT_SECONDS,
@@ -38,6 +38,7 @@ from dify_agent.layers.shell import (
) )
from dify_agent.protocol import CreateRunRequest, DeferredToolResultsPayload from dify_agent.protocol import CreateRunRequest, DeferredToolResultsPayload
from pydantic import BaseModel, ValidationError from pydantic import BaseModel, ValidationError
from sqlalchemy.exc import OperationalError
from clients.agent_backend import ( from clients.agent_backend import (
AgentBackendModelConfig, AgentBackendModelConfig,
@@ -206,14 +207,22 @@ class WorkflowAgentRuntimeRequestBuilder:
"cli_tool_count": len(agent_soul.tools.cli_tools), "cli_tool_count": len(agent_soul.tools.cli_tools),
} }
runtime_config_skills = load_runtime_agent_skill_configs(
tenant_id=context.dify_context.tenant_id,
agent_id=context.agent.id,
)
config_layer_config, config_warnings = build_config_layer_config( config_layer_config, config_warnings = build_config_layer_config(
agent_soul, agent_soul,
agent_id=context.agent.id, agent_id=context.agent.id,
config_version_id=context.snapshot.id, config_version_id=context.snapshot.id,
config_version_kind="snapshot", config_version_kind="snapshot",
runtime_config_skills=runtime_config_skills,
) )
append_runtime_warnings(metadata, config_warnings) append_runtime_warnings(metadata, config_warnings)
soul_prompt_resolver = build_config_aware_soul_mention_resolver(agent_soul) soul_prompt_resolver = build_config_aware_soul_mention_resolver(
agent_soul,
runtime_config_skills=runtime_config_skills,
)
soul_prompt = expand_prompt_mentions(agent_soul.prompt.system_prompt, soul_prompt_resolver).strip() soul_prompt = expand_prompt_mentions(agent_soul.prompt.system_prompt, soul_prompt_resolver).strip()
knowledge_config = build_knowledge_layer_config(agent_soul) knowledge_config = build_knowledge_layer_config(agent_soul)
@@ -883,11 +892,16 @@ def append_runtime_warnings(metadata: dict[str, Any], warnings: list[dict[str, s
existing.extend(warnings) existing.extend(warnings)
def build_config_aware_soul_mention_resolver(agent_soul: AgentSoulConfig): def build_config_aware_soul_mention_resolver(
agent_soul: AgentSoulConfig,
*,
runtime_config_skills: Sequence[DifyConfigSkillConfig] = (),
):
"""Resolve config skill/file mentions and delegate the rest to Agent Soul.""" """Resolve config skill/file mentions and delegate the rest to Agent Soul."""
base_resolver = build_soul_mention_resolver(agent_soul) base_resolver = build_soul_mention_resolver(agent_soul)
skill_names = {item.name for item in agent_soul.config_skills if not item.is_missing} skill_names = {item.name for item in agent_soul.config_skills if not item.is_missing}
skill_names.update(item.name for item in runtime_config_skills)
file_names = {item.name for item in agent_soul.config_files if not item.is_missing} file_names = {item.name for item in agent_soul.config_files if not item.is_missing}
def _resolve(mention: object) -> str | None: def _resolve(mention: object) -> str | None:
@@ -905,12 +919,34 @@ def build_config_aware_soul_mention_resolver(agent_soul: AgentSoulConfig):
return _resolve return _resolve
def load_runtime_agent_skill_configs(*, tenant_id: str, agent_id: str) -> list[DifyConfigSkillConfig]:
"""Return workspace-bound Skills as prompt-safe runtime config skills."""
from services.skill_management_service import SkillManagementService
try:
runtime_skills = SkillManagementService().list_runtime_agent_skills(tenant_id=tenant_id, agent_id=agent_id)
except OperationalError as exc:
if "no such table: agent_skill_bindings" not in str(exc.orig):
raise
runtime_skills = []
return [
DifyConfigSkillConfig(
name=str(item["name"]),
description=str(item.get("description") or ""),
size=cast(int | None, item.get("size")),
mime_type=cast(str | None, item.get("mime_type")),
)
for item in runtime_skills
]
def build_config_layer_config( def build_config_layer_config(
agent_soul: AgentSoulConfig, agent_soul: AgentSoulConfig,
*, *,
agent_id: str | None = None, agent_id: str | None = None,
config_version_id: str | None = None, config_version_id: str | None = None,
config_version_kind: Literal["snapshot", "draft", "build_draft"] = "snapshot", config_version_kind: Literal["snapshot", "draft", "build_draft"] = "snapshot",
runtime_config_skills: Sequence[DifyConfigSkillConfig] = (),
) -> tuple[DifyConfigLayerConfig, list[dict[str, str]]]: ) -> tuple[DifyConfigLayerConfig, list[dict[str, str]]]:
"""Build the always-present Agent config layer from Agent Soul state. """Build the always-present Agent config layer from Agent Soul state.
@@ -927,8 +963,23 @@ def build_config_layer_config(
) )
) )
available_skills = [skill for skill in agent_soul.config_skills if not skill.is_missing] available_skills = [skill for skill in agent_soul.config_skills if not skill.is_missing]
skill_configs = [
DifyConfigSkillConfig(
name=skill.name,
description=skill.description,
size=skill.size,
mime_type=skill.mime_type,
)
for skill in available_skills
]
seen_skill_names = {skill.name for skill in skill_configs}
for skill in runtime_config_skills:
if skill.name in seen_skill_names:
continue
seen_skill_names.add(skill.name)
skill_configs.append(skill)
available_files = [file_ref for file_ref in agent_soul.config_files if not file_ref.is_missing] available_files = [file_ref for file_ref in agent_soul.config_files if not file_ref.is_missing]
skill_names = {skill.name for skill in available_skills} skill_names = {skill.name for skill in skill_configs}
file_names = {file_ref.name for file_ref in available_files} file_names = {file_ref.name for file_ref in available_files}
warnings: list[dict[str, str]] = [ warnings: list[dict[str, str]] = [
{ {
@@ -965,15 +1016,7 @@ def build_config_layer_config(
kind=config_version_kind, kind=config_version_kind,
writable=config_version_kind == "build_draft", writable=config_version_kind == "build_draft",
), ),
skills=[ skills=skill_configs,
DifyConfigSkillConfig(
name=skill.name,
description=skill.description,
size=skill.size,
mime_type=skill.mime_type,
)
for skill in available_skills
],
files=[ files=[
DifyConfigFileConfig( DifyConfigFileConfig(
name=file_ref.name, name=file_ref.name,
+27
View File
@@ -5,6 +5,7 @@ from typing import Any
import pytz # type: ignore[import-untyped] import pytz # type: ignore[import-untyped]
from celery import Celery, Task from celery import Celery, Task
from celery.schedules import crontab from celery.schedules import crontab
from celery.signals import beat_init
from typing_extensions import TypedDict from typing_extensions import TypedDict
from configs import dify_config from configs import dify_config
@@ -36,6 +37,19 @@ class CeleryBeatScheduleEntry(TypedDict):
schedule: crontab | timedelta schedule: crontab | timedelta
def _enqueue_initial_community_telemetry_heartbeat(sender: Any, **_: Any) -> None:
task_name = "community_telemetry.send_heartbeat"
if "community_telemetry_heartbeat" not in sender.app.conf.beat_schedule:
return
task = sender.app.tasks.get(task_name)
if task is not None:
task.apply_async()
beat_init.connect(_enqueue_initial_community_telemetry_heartbeat, weak=False)
def get_celery_ssl_options() -> CelerySSLOptionsDict | None: def get_celery_ssl_options() -> CelerySSLOptionsDict | None:
"""Get SSL configuration for Celery broker/backend connections.""" """Get SSL configuration for Celery broker/backend connections."""
# Only apply SSL if we're using Redis as broker/backend # Only apply SSL if we're using Redis as broker/backend
@@ -260,6 +274,19 @@ def init_app(app: DifyApp) -> Celery:
"schedule": timedelta(minutes=dify_config.API_TOKEN_LAST_USED_UPDATE_INTERVAL), "schedule": timedelta(minutes=dify_config.API_TOKEN_LAST_USED_UPDATE_INTERVAL),
} }
if (
dify_config.EDITION == "SELF_HOSTED"
and not dify_config.ENTERPRISE_ENABLED
and not dify_config.DISABLE_TELEMETRY
and not dify_config.DO_NOT_TRACK
and not dify_config.CI
):
imports.append("tasks.community_telemetry_task")
beat_schedule["community_telemetry_heartbeat"] = {
"task": "community_telemetry.send_heartbeat",
"schedule": timedelta(minutes=dify_config.TELEMETRY_HEARTBEAT_INTERVAL_MINUTES),
}
if dify_config.ENTERPRISE_ENABLED and dify_config.ENTERPRISE_TELEMETRY_ENABLED: if dify_config.ENTERPRISE_ENABLED and dify_config.ENTERPRISE_TELEMETRY_ENABLED:
imports.append("tasks.enterprise_telemetry_task") imports.append("tasks.enterprise_telemetry_task")
celery_app.conf.update(beat_schedule=beat_schedule, imports=imports) celery_app.conf.update(beat_schedule=beat_schedule, imports=imports)
+1
View File
@@ -383,6 +383,7 @@ class AgentAppComposerResponse(ResponseModel):
variant: Literal[ComposerVariant.AGENT_APP] variant: Literal[ComposerVariant.AGENT_APP]
agent: AgentComposerAgentResponse agent: AgentComposerAgentResponse
active_config_snapshot: AgentConfigSnapshotSummaryResponse | None = None active_config_snapshot: AgentConfigSnapshotSummaryResponse | None = None
active_config_is_published: bool
draft: AgentConfigDraftSummaryResponse | None = None draft: AgentConfigDraftSummaryResponse | None = None
agent_soul: AgentSoulConfig agent_soul: AgentSoulConfig
save_options: list[ComposerSaveStrategy] save_options: list[ComposerSaveStrategy]
+5 -1
View File
@@ -289,7 +289,11 @@ UUIDStr = Annotated[str, AfterValidator(_strict_uuid)]
def alphanumeric(value: str): def alphanumeric(value: str):
# check if the value is alphanumeric and underlined # check if the value is alphanumeric and underlined
if re.match(r"^[a-zA-Z0-9_]+$", value): # Use re.fullmatch instead of re.match to reject trailing newlines.
# In Python, '$' matches at end-of-string OR just before a trailing newline,
# so re.match accepts "tool_name\n". re.fullmatch requires the entire
# string to match. Regression for #39666 (sibling of #39234 / #39548).
if re.fullmatch(r"^[a-zA-Z0-9_]+$", value):
return value return value
raise ValueError(f"{value} is not a valid alphanumeric value") raise ValueError(f"{value} is not a valid alphanumeric value")
@@ -0,0 +1,114 @@
"""add workspace skill management
Revision ID: a4f8d2c9e1b0
Revises: 6f5a9c2d8e1b
Create Date: 2026-07-09 12:00:00.000000
"""
import sqlalchemy as sa
from alembic import op
from sqlalchemy.dialects import mysql
from models.types import StringUUID
# revision identifiers, used by Alembic.
revision = "a4f8d2c9e1b0"
down_revision = "6f5a9c2d8e1b"
branch_labels = None
depends_on = None
def _uuid_column(name: str, *, nullable: bool = False) -> sa.Column:
return sa.Column(name, StringUUID(), nullable=nullable)
def _long_text() -> sa.types.TypeEngine:
return sa.Text().with_variant(mysql.LONGTEXT(), "mysql")
def upgrade() -> None:
op.create_table(
"skills",
_uuid_column("id"),
_uuid_column("tenant_id"),
sa.Column("name", sa.String(length=64), nullable=False),
sa.Column("display_name", sa.String(length=128), nullable=False),
sa.Column("icon", sa.String(length=16), nullable=False, server_default="📄"),
sa.Column("description", sa.String(length=1024), nullable=False, server_default=""),
sa.Column("name_manually_edited", sa.Boolean(), nullable=False, server_default=sa.false()),
sa.Column("visibility", sa.String(length=32), nullable=False, server_default="workspace"),
_uuid_column("latest_published_version_id", nullable=True),
_uuid_column("created_by", nullable=True),
_uuid_column("updated_by", nullable=True),
sa.Column("created_at", sa.DateTime(), nullable=False, server_default=sa.func.current_timestamp()),
sa.Column("updated_at", sa.DateTime(), nullable=False, server_default=sa.func.current_timestamp()),
sa.PrimaryKeyConstraint("id", name="skill_pkey"),
sa.UniqueConstraint("tenant_id", "name", name="skill_tenant_name_unique"),
)
op.create_index("skills_tenant_updated_at_idx", "skills", ["tenant_id", "updated_at"])
op.create_table(
"skill_draft_files",
_uuid_column("id"),
_uuid_column("skill_id"),
sa.Column("path", sa.String(length=512), nullable=False),
sa.Column("kind", sa.String(length=32), nullable=False),
sa.Column("storage", sa.String(length=32), nullable=True),
sa.Column("mime_type", sa.String(length=255), nullable=True),
sa.Column("content_text", _long_text(), nullable=True),
_uuid_column("tool_file_id", nullable=True),
sa.Column("size", sa.BigInteger(), nullable=True),
sa.Column("hash", sa.String(length=255), nullable=True),
sa.Column("created_at", sa.DateTime(), nullable=False, server_default=sa.func.current_timestamp()),
sa.Column("updated_at", sa.DateTime(), nullable=False, server_default=sa.func.current_timestamp()),
sa.PrimaryKeyConstraint("id", name="skill_draft_file_pkey"),
sa.UniqueConstraint("skill_id", "path", name="skill_draft_file_skill_path_unique"),
)
op.create_index("skill_draft_files_skill_path_idx", "skill_draft_files", ["skill_id", "path"])
op.create_table(
"skill_versions",
_uuid_column("id"),
_uuid_column("skill_id"),
sa.Column("version_number", sa.Integer(), nullable=False),
sa.Column("version_name", sa.String(length=128), nullable=False, server_default=""),
sa.Column("publish_note", sa.String(length=1024), nullable=False, server_default=""),
sa.Column("manifest", _long_text(), nullable=False),
_uuid_column("archive_tool_file_id"),
sa.Column("hash_code", sa.String(length=255), nullable=False),
sa.Column("archive_size", sa.BigInteger(), nullable=False),
_uuid_column("published_by", nullable=True),
sa.Column("created_at", sa.DateTime(), nullable=False, server_default=sa.func.current_timestamp()),
sa.Column("updated_at", sa.DateTime(), nullable=False, server_default=sa.func.current_timestamp()),
sa.PrimaryKeyConstraint("id", name="skill_version_pkey"),
sa.UniqueConstraint("skill_id", "version_number", name="skill_version_skill_number_unique"),
)
op.create_index("skill_versions_skill_created_at_idx", "skill_versions", ["skill_id", "created_at"])
op.create_table(
"agent_skill_bindings",
_uuid_column("id"),
_uuid_column("tenant_id"),
_uuid_column("agent_id"),
_uuid_column("skill_id"),
sa.Column("priority", sa.Integer(), nullable=False),
_uuid_column("created_by", nullable=True),
sa.Column("created_at", sa.DateTime(), nullable=False, server_default=sa.func.current_timestamp()),
sa.Column("updated_at", sa.DateTime(), nullable=False, server_default=sa.func.current_timestamp()),
sa.PrimaryKeyConstraint("id", name="agent_skill_binding_pkey"),
sa.UniqueConstraint("tenant_id", "agent_id", "skill_id", name="agent_skill_binding_unique"),
sa.UniqueConstraint("tenant_id", "agent_id", "priority", name="agent_skill_binding_priority_unique"),
)
op.create_index("agent_skill_bindings_skill_idx", "agent_skill_bindings", ["tenant_id", "skill_id"])
def downgrade() -> None:
op.drop_index("agent_skill_bindings_skill_idx", table_name="agent_skill_bindings")
op.drop_table("agent_skill_bindings")
op.drop_index("skill_versions_skill_created_at_idx", table_name="skill_versions")
op.drop_table("skill_versions")
op.drop_index("skill_draft_files_skill_path_idx", table_name="skill_draft_files")
op.drop_table("skill_draft_files")
op.drop_index("skills_tenant_updated_at_idx", table_name="skills")
op.drop_table("skills")
@@ -18,31 +18,109 @@ branch_labels = None
depends_on = None depends_on = None
def _is_pg(conn) -> bool:
return conn.dialect.name == "postgresql"
def _uuid_column(name: str, *, nullable: bool = False, primary_key: bool = False) -> sa.Column:
kwargs = {"nullable": nullable, "primary_key": primary_key}
if primary_key and _is_pg(op.get_bind()):
kwargs["server_default"] = sa.text("uuidv7()")
return sa.Column(name, models.types.StringUUID(), **kwargs)
def _has_table(table_name: str) -> bool:
return sa.inspect(op.get_bind()).has_table(table_name)
def _has_column(table_name: str, column_name: str) -> bool:
return any(
column["name"] == column_name for column in sa.inspect(op.get_bind()).get_columns(table_name)
)
def _has_unique_constraint(table_name: str, constraint_name: str) -> bool:
return any(
constraint["name"] == constraint_name
for constraint in sa.inspect(op.get_bind()).get_unique_constraints(table_name)
)
def upgrade(): def upgrade():
if not _has_table("agent_debug_conversations"):
op.create_table(
"agent_debug_conversations",
_uuid_column("id", primary_key=True),
sa.Column("tenant_id", models.types.StringUUID(), nullable=False),
sa.Column("agent_id", models.types.StringUUID(), nullable=False),
sa.Column("app_id", models.types.StringUUID(), nullable=False),
sa.Column("account_id", models.types.StringUUID(), nullable=False),
sa.Column("conversation_id", models.types.StringUUID(), nullable=False),
sa.Column(
"draft_type",
sa.String(length=32),
nullable=False,
server_default=sa.text("'debug_build'"),
),
sa.Column("created_at", sa.DateTime(), server_default=sa.func.current_timestamp(), nullable=False),
sa.Column("updated_at", sa.DateTime(), server_default=sa.func.current_timestamp(), nullable=False),
sa.PrimaryKeyConstraint("id", name=op.f("agent_debug_conversation_pkey")),
sa.UniqueConstraint(
"tenant_id",
"agent_id",
"account_id",
"draft_type",
name=op.f("agent_debug_conversation_agent_account_draft_type_unique"),
),
)
op.create_index(
"agent_debug_conversation_conversation_idx",
"agent_debug_conversations",
["conversation_id"],
)
op.create_index(
"agent_debug_conversation_account_idx",
"agent_debug_conversations",
["tenant_id", "account_id"],
)
return
# Existing pointers have always represented Build chat because the Agent # Existing pointers have always represented Build chat because the Agent
# detail API exposes them as ``debug_conversation_id`` for that surface. # detail API exposes them as ``debug_conversation_id`` for that surface.
op.add_column( if not _has_column("agent_debug_conversations", "draft_type"):
op.add_column(
"agent_debug_conversations",
sa.Column(
"draft_type",
sa.String(length=32),
nullable=False,
server_default=sa.text("'debug_build'"),
),
)
if _has_unique_constraint(
"agent_debug_conversations", "agent_debug_conversations",
sa.Column(
"draft_type",
sa.String(length=32),
nullable=False,
server_default=sa.text("'debug_build'"),
),
)
op.drop_constraint(
"agent_debug_conversation_agent_account_unique", "agent_debug_conversation_agent_account_unique",
):
op.drop_constraint(
"agent_debug_conversation_agent_account_unique",
"agent_debug_conversations",
type_="unique",
)
if not _has_unique_constraint(
"agent_debug_conversations", "agent_debug_conversations",
type_="unique",
)
op.create_unique_constraint(
"agent_debug_conversation_agent_account_draft_type_unique", "agent_debug_conversation_agent_account_draft_type_unique",
"agent_debug_conversations", ):
["tenant_id", "agent_id", "account_id", "draft_type"], op.create_unique_constraint(
) "agent_debug_conversation_agent_account_draft_type_unique",
"agent_debug_conversations",
["tenant_id", "agent_id", "account_id", "draft_type"],
)
def downgrade(): def downgrade():
if not _has_table("agent_debug_conversations"):
return
debug_conversations = sa.table( debug_conversations = sa.table(
"agent_debug_conversations", "agent_debug_conversations",
sa.column("tenant_id", models.types.StringUUID()), sa.column("tenant_id", models.types.StringUUID()),
@@ -64,14 +142,23 @@ def downgrade():
), ),
) )
) )
op.drop_constraint( if _has_unique_constraint(
"agent_debug_conversations",
"agent_debug_conversation_agent_account_draft_type_unique", "agent_debug_conversation_agent_account_draft_type_unique",
):
op.drop_constraint(
"agent_debug_conversation_agent_account_draft_type_unique",
"agent_debug_conversations",
type_="unique",
)
if not _has_unique_constraint(
"agent_debug_conversations", "agent_debug_conversations",
type_="unique",
)
op.create_unique_constraint(
"agent_debug_conversation_agent_account_unique", "agent_debug_conversation_agent_account_unique",
"agent_debug_conversations", ):
["tenant_id", "agent_id", "account_id"], op.create_unique_constraint(
) "agent_debug_conversation_agent_account_unique",
op.drop_column("agent_debug_conversations", "draft_type") "agent_debug_conversations",
["tenant_id", "agent_id", "account_id"],
)
if _has_column("agent_debug_conversations", "draft_type"):
op.drop_column("agent_debug_conversations", "draft_type")
@@ -0,0 +1,30 @@
"""add telemetry fields to dify_setups
Revision ID: 6f5a9c2d8e1b
Revises: d2825e7b9c10
Create Date: 2026-07-23 12:00:00.000000
"""
import sqlalchemy as sa
from alembic import op
# revision identifiers, used by Alembic.
revision = "6f5a9c2d8e1b"
down_revision = "d2825e7b9c10"
branch_labels = None
depends_on = None
def upgrade():
with op.batch_alter_table("dify_setups", schema=None) as batch_op:
batch_op.add_column(sa.Column("instance_id", sa.String(length=255), nullable=True))
batch_op.add_column(sa.Column("install_reported_at", sa.DateTime(), nullable=True))
batch_op.add_column(sa.Column("last_heartbeat_at", sa.DateTime(), nullable=True))
def downgrade():
with op.batch_alter_table("dify_setups", schema=None) as batch_op:
batch_op.drop_column("last_heartbeat_at")
batch_op.drop_column("install_reported_at")
batch_op.drop_column("instance_id")
@@ -0,0 +1,21 @@
"""merge skill and agent debug conversation heads
Revision ID: e9f4a1b2c3d5
Revises: a4f8d2c9e1b0, d2825e7b9c10
Create Date: 2026-07-23 15:00:00.000000
"""
# revision identifiers, used by Alembic.
revision = "e9f4a1b2c3d5"
down_revision = ("a4f8d2c9e1b0", "d2825e7b9c10")
branch_labels = None
depends_on = None
def upgrade():
pass
def downgrade():
pass
+7
View File
@@ -113,6 +113,7 @@ from .provider import (
TenantDefaultModel, TenantDefaultModel,
TenantPreferredModelProvider, TenantPreferredModelProvider,
) )
from .skill import AgentSkillBinding, Skill, SkillDraftFile, SkillFileKind, SkillFileStorage, SkillVersion
from .snippet import CustomizedSnippet, SnippetType from .snippet import CustomizedSnippet, SnippetType
from .source import DataSourceApiKeyAuthBinding, DataSourceOauthBinding from .source import DataSourceApiKeyAuthBinding, DataSourceOauthBinding
from .task import CeleryTask, CeleryTaskSet from .task import CeleryTask, CeleryTaskSet
@@ -173,6 +174,7 @@ __all__ = [
"AgentRuntimeSessionOwnerType", "AgentRuntimeSessionOwnerType",
"AgentRuntimeSessionStatus", "AgentRuntimeSessionStatus",
"AgentScope", "AgentScope",
"AgentSkillBinding",
"AgentSource", "AgentSource",
"AgentStatus", "AgentStatus",
"ApiRequest", "ApiRequest",
@@ -246,6 +248,11 @@ __all__ = [
"RecommendedApp", "RecommendedApp",
"SavedMessage", "SavedMessage",
"Site", "Site",
"Skill",
"SkillDraftFile",
"SkillFileKind",
"SkillFileStorage",
"SkillVersion",
"SnippetType", "SnippetType",
"Tag", "Tag",
"TagBinding", "TagBinding",
+1
View File
@@ -249,6 +249,7 @@ class TagType(StrEnum):
KNOWLEDGE = "knowledge" KNOWLEDGE = "knowledge"
APP = "app" APP = "app"
SNIPPET = "snippet" SNIPPET = "snippet"
SKILL = "skill"
class DatasetMetadataType(StrEnum): class DatasetMetadataType(StrEnum):
+6 -3
View File
@@ -362,6 +362,9 @@ class DifySetup(TypeBase):
__table_args__ = (sa.PrimaryKeyConstraint("version", name="dify_setup_pkey"),) __table_args__ = (sa.PrimaryKeyConstraint("version", name="dify_setup_pkey"),)
version: Mapped[str] = mapped_column(String(255), nullable=False) version: Mapped[str] = mapped_column(String(255), nullable=False)
instance_id: Mapped[str | None] = mapped_column(String(255), nullable=True, default=None)
install_reported_at: Mapped[datetime | None] = mapped_column(sa.DateTime, nullable=True, default=None)
last_heartbeat_at: Mapped[datetime | None] = mapped_column(sa.DateTime, nullable=True, default=None)
setup_at: Mapped[datetime] = mapped_column( setup_at: Mapped[datetime] = mapped_column(
sa.DateTime, nullable=False, server_default=func.current_timestamp(), init=False sa.DateTime, nullable=False, server_default=func.current_timestamp(), init=False
) )
@@ -1114,14 +1117,14 @@ class ExporleBanner(TypeBase):
status: Mapped[BannerStatus] = mapped_column( status: Mapped[BannerStatus] = mapped_column(
EnumText(BannerStatus, length=255), EnumText(BannerStatus, length=255),
nullable=False, nullable=False,
server_default=sa.text("'enabled'::character varying"), server_default=sa.text("'enabled'"),
default=BannerStatus.ENABLED, default=BannerStatus.ENABLED,
) )
created_at: Mapped[datetime] = mapped_column( created_at: Mapped[datetime] = mapped_column(
sa.DateTime, nullable=False, server_default=func.current_timestamp(), init=False sa.DateTime, nullable=False, server_default=func.current_timestamp(), init=False
) )
language: Mapped[str] = mapped_column( language: Mapped[str] = mapped_column(
String(255), nullable=False, server_default=sa.text("'en-US'::character varying"), default="en-US" String(255), nullable=False, server_default=sa.text("'en-US'"), default="en-US"
) )
@@ -2664,7 +2667,7 @@ class Tag(TypeBase):
sa.Index("tag_name_idx", "name"), sa.Index("tag_name_idx", "name"),
) )
TAG_TYPE_LIST = ["knowledge", "app", "snippet"] TAG_TYPE_LIST = ["knowledge", "app", "snippet", "skill"]
id: Mapped[str] = mapped_column( id: Mapped[str] = mapped_column(
StringUUID, insert_default=lambda: str(uuid4()), default_factory=lambda: str(uuid4()), init=False StringUUID, insert_default=lambda: str(uuid4()), default_factory=lambda: str(uuid4()), init=False
+164
View File
@@ -0,0 +1,164 @@
"""Workspace-level Skill Management models.
These tables are the source of truth for reusable workspace Skills. Agent Soul
``config_skills`` and Agent Drive skill rows remain per-agent runtime/config
assets; they may consume a published Skill snapshot but do not own the Skill's
draft, metadata, version history, or Agent binding priority.
"""
from enum import StrEnum
import sqlalchemy as sa
from pydantic import BaseModel, ConfigDict
from sqlalchemy import Index, UniqueConstraint
from sqlalchemy.orm import Mapped, mapped_column
from models.base import Base, DefaultFieldsMixin
from models.types import EnumText, JSONModelColumn, LongText, StringUUID
class SkillFileKind(StrEnum):
"""Draft file entry kind."""
FILE = "file"
DIRECTORY = "directory"
class SkillFileStorage(StrEnum):
"""How a draft file's content is stored."""
TEXT = "text"
TOOL_FILE = "tool_file"
class SkillVersionManifestFile(BaseModel):
"""One file entry captured in a published Skill snapshot manifest."""
path: str
mime_type: str | None = None
size: int
hash: str
model_config = ConfigDict(extra="forbid")
class SkillVersionManifest(BaseModel):
"""Published Skill snapshot file index."""
files: list[SkillVersionManifestFile]
model_config = ConfigDict(extra="forbid")
class Skill(DefaultFieldsMixin, Base):
"""Workspace-level reusable Skill metadata and draft status."""
__tablename__ = "skills"
__table_args__ = (
sa.PrimaryKeyConstraint("id", name="skill_pkey"),
UniqueConstraint("tenant_id", "name", name="skill_tenant_name_unique"),
Index("skills_tenant_updated_at_idx", "tenant_id", "updated_at"),
)
tenant_id: Mapped[str] = mapped_column(StringUUID, nullable=False)
name: Mapped[str] = mapped_column(sa.String(64), nullable=False)
display_name: Mapped[str] = mapped_column(sa.String(128), nullable=False)
icon: Mapped[str] = mapped_column(sa.String(16), nullable=False, default="📄", server_default="📄")
description: Mapped[str] = mapped_column(sa.String(1024), nullable=False, default="", server_default="")
name_manually_edited: Mapped[bool] = mapped_column(
sa.Boolean,
nullable=False,
default=False,
server_default=sa.false(),
)
visibility: Mapped[str] = mapped_column(
sa.String(32),
nullable=False,
default="workspace",
server_default="workspace",
)
latest_published_version_id: Mapped[str | None] = mapped_column(StringUUID, nullable=True)
created_by: Mapped[str | None] = mapped_column(StringUUID, nullable=True)
updated_by: Mapped[str | None] = mapped_column(StringUUID, nullable=True)
class SkillDraftFile(DefaultFieldsMixin, Base):
"""One draft file or directory in a workspace Skill."""
__tablename__ = "skill_draft_files"
__table_args__ = (
sa.PrimaryKeyConstraint("id", name="skill_draft_file_pkey"),
UniqueConstraint("skill_id", "path", name="skill_draft_file_skill_path_unique"),
Index("skill_draft_files_skill_path_idx", "skill_id", "path"),
)
skill_id: Mapped[str] = mapped_column(StringUUID, nullable=False)
path: Mapped[str] = mapped_column(sa.String(512), nullable=False)
kind: Mapped[SkillFileKind] = mapped_column(EnumText(SkillFileKind, length=32), nullable=False)
storage: Mapped[SkillFileStorage | None] = mapped_column(EnumText(SkillFileStorage, length=32), nullable=True)
mime_type: Mapped[str | None] = mapped_column(sa.String(255), nullable=True)
content_text: Mapped[str | None] = mapped_column(LongText, nullable=True)
tool_file_id: Mapped[str | None] = mapped_column(StringUUID, nullable=True)
size: Mapped[int | None] = mapped_column(sa.BigInteger, nullable=True)
hash: Mapped[str | None] = mapped_column(sa.String(255), nullable=True)
class SkillVersion(DefaultFieldsMixin, Base):
"""Immutable published Skill snapshot.
``hash_code`` uniquely identifies a published version for downstream
execution audit. It includes Skill identity, version number, and archive
content digest instead of being only the archive content hash.
"""
__tablename__ = "skill_versions"
__table_args__ = (
sa.PrimaryKeyConstraint("id", name="skill_version_pkey"),
UniqueConstraint("skill_id", "version_number", name="skill_version_skill_number_unique"),
Index("skill_versions_skill_created_at_idx", "skill_id", "created_at"),
)
skill_id: Mapped[str] = mapped_column(StringUUID, nullable=False)
version_number: Mapped[int] = mapped_column(sa.Integer, nullable=False)
version_name: Mapped[str] = mapped_column(sa.String(128), nullable=False, default="", server_default="")
publish_note: Mapped[str] = mapped_column(sa.String(1024), nullable=False, default="", server_default="")
manifest: Mapped[SkillVersionManifest] = mapped_column(JSONModelColumn(SkillVersionManifest), nullable=False)
archive_tool_file_id: Mapped[str] = mapped_column(StringUUID, nullable=False)
hash_code: Mapped[str] = mapped_column(sa.String(255), nullable=False)
archive_size: Mapped[int] = mapped_column(sa.BigInteger, nullable=False)
published_by: Mapped[str | None] = mapped_column(StringUUID, nullable=True)
class AgentSkillBinding(DefaultFieldsMixin, Base):
"""Direct Agent-to-workspace-Skill binding.
``priority`` is retained as an internal ordering column for the current
schema constraints. Runtime Skill selection is Agent-driven and must not
treat it as a matching priority.
"""
__tablename__ = "agent_skill_bindings"
__table_args__ = (
sa.PrimaryKeyConstraint("id", name="agent_skill_binding_pkey"),
UniqueConstraint("tenant_id", "agent_id", "skill_id", name="agent_skill_binding_unique"),
UniqueConstraint("tenant_id", "agent_id", "priority", name="agent_skill_binding_priority_unique"),
Index("agent_skill_bindings_skill_idx", "tenant_id", "skill_id"),
)
tenant_id: Mapped[str] = mapped_column(StringUUID, nullable=False)
agent_id: Mapped[str] = mapped_column(StringUUID, nullable=False)
skill_id: Mapped[str] = mapped_column(StringUUID, nullable=False)
priority: Mapped[int] = mapped_column(sa.Integer, nullable=False)
created_by: Mapped[str | None] = mapped_column(StringUUID, nullable=True)
__all__ = [
"AgentSkillBinding",
"Skill",
"SkillDraftFile",
"SkillFileKind",
"SkillFileStorage",
"SkillVersion",
"SkillVersionManifest",
"SkillVersionManifestFile",
]
+768 -3
View File
@@ -1672,6 +1672,23 @@ Create a new application
| 200 | Import confirmed | **application/json**: [Import](#import)<br> | | 200 | Import confirmed | **application/json**: [Import](#import)<br> |
| 400 | Import failed | **application/json**: [Import](#import)<br> | | 400 | Import failed | **application/json**: [Import](#import)<br> |
### [GET] /apps/recent
**Return the lightweight app cards needed by the Explore home page**
Get recently modified apps for the home Continue Work section
#### Parameters
| Name | Located in | Description | Required | Schema |
| ---- | ---------- | ----------- | -------- | ------ |
| limit | query | Number of recently modified apps to return (1-8) | No | integer, <br>**Default:** 8 |
#### Responses
| Code | Description | Schema |
| ---- | ----------- | ------ |
| 200 | Success | **application/json**: [RecentAppListResponse](#recentapplistresponse)<br> |
### [GET] /apps/starred ### [GET] /apps/starred
Get applications starred by the current account Get applications starred by the current account
@@ -9534,7 +9551,7 @@ Remove one or more tag bindings from a target.
| Name | Located in | Description | Required | Schema | | Name | Located in | Description | Required | Schema |
| ---- | ---------- | ----------- | -------- | ------ | | ---- | ---------- | ----------- | -------- | ------ |
| keyword | query | Search keyword | No | string | | keyword | query | Search keyword | No | string |
| type | query | Tag type filter | No | string, <br>**Available values:** "", "app", "knowledge", "snippet" | | type | query | Tag type filter | No | string |
#### Responses #### Responses
@@ -10070,6 +10087,38 @@ Get list of available agent providers
| ---- | ----------- | ------ | | ---- | ----------- | ------ |
| 200 | Success | **application/json**: [AgentProviderListResponse](#agentproviderlistresponse)<br> | | 200 | Success | **application/json**: [AgentProviderListResponse](#agentproviderlistresponse)<br> |
### [GET] /workspaces/current/agents/{agent_id}/skills
#### Parameters
| Name | Located in | Description | Required | Schema |
| ---- | ---------- | ----------- | -------- | ------ |
| agent_id | path | | Yes | string |
#### Responses
| Code | Description | Schema |
| ---- | ----------- | ------ |
| 200 | Agent Skill bindings | **application/json**: [AgentSkillBindingsResponse](#agentskillbindingsresponse)<br> |
### [PUT] /workspaces/current/agents/{agent_id}/skills
#### Parameters
| Name | Located in | Description | Required | Schema |
| ---- | ---------- | ----------- | -------- | ------ |
| agent_id | path | | Yes | string |
#### Request Body
| Required | Schema |
| -------- | ------ |
| Yes | **application/json**: [AgentSkillBindingsPayload](#agentskillbindingspayload)<br> |
#### Responses
| Code | Description | Schema |
| ---- | ----------- | ------ |
| 200 | Agent Skill bindings replaced | **application/json**: [AgentSkillBindingsResponse](#agentskillbindingsresponse)<br> |
### [GET] /workspaces/current/customized-snippets ### [GET] /workspaces/current/customized-snippets
**List customized snippets with pagination and search** **List customized snippets with pagination and search**
@@ -11971,6 +12020,341 @@ Returns permission flags that control workspace features like member invitations
| ---- | ----------- | ------ | | ---- | ----------- | ------ |
| 200 | Success | **application/json**: [WorkspaceAccessMatrix](#workspaceaccessmatrix)<br> | | 200 | Success | **application/json**: [WorkspaceAccessMatrix](#workspaceaccessmatrix)<br> |
### [GET] /workspaces/current/skills
#### Parameters
| Name | Located in | Description | Required | Schema |
| ---- | ---------- | ----------- | -------- | ------ |
| keyword | query | Search keyword matching skill name or description. | No | string |
| limit | query | Number of items per page. | No | integer, <br>**Default:** 20 |
| page | query | Page number. | No | integer, <br>**Default:** 1 |
| tag | query | Skill tag filters. Repeat the parameter for multiple tags. | No | [ string ] |
#### Responses
| Code | Description | Schema |
| ---- | ----------- | ------ |
| 200 | Workspace skills | **application/json**: [SkillListResponse](#skilllistresponse)<br> |
### [POST] /workspaces/current/skills
#### Request Body
| Required | Schema |
| -------- | ------ |
| Yes | **application/json**: [SkillCreatePayload](#skillcreatepayload)<br> |
#### Responses
| Code | Description | Schema |
| ---- | ----------- | ------ |
| 201 | Skill created | **application/json**: [SkillDetailResponse](#skilldetailresponse)<br> |
### [POST] /workspaces/current/skills/files/upload
#### Request Body
| Required | Schema |
| -------- | ------ |
| Yes | **multipart/form-data**: { **"file"**: binary }<br> |
#### Responses
| Code | Description | Schema |
| ---- | ----------- | ------ |
| 201 | Skill draft file uploaded | **application/json**: [SkillFileUploadResponse](#skillfileuploadresponse)<br> |
### [POST] /workspaces/current/skills/import
Import a Skill zip package from multipart form field `file`.
#### Responses
| Code | Description | Schema |
| ---- | ----------- | ------ |
| 201 | Skill imported | **application/json**: [SkillDetailResponse](#skilldetailresponse)<br> |
### [GET] /workspaces/current/skills/tags
#### Responses
| Code | Description | Schema |
| ---- | ----------- | ------ |
| 200 | Workspace Skill tags | **application/json**: [SkillTagListResponse](#skilltaglistresponse)<br> |
### [DELETE] /workspaces/current/skills/{skill_id}
#### Parameters
| Name | Located in | Description | Required | Schema |
| ---- | ---------- | ----------- | -------- | ------ |
| skill_id | path | | Yes | string |
#### Request Body
| Required | Schema |
| -------- | ------ |
| Yes | **application/json**: [SkillDeletePayload](#skilldeletepayload)<br> |
#### Responses
| Code | Description | Schema |
| ---- | ----------- | ------ |
| 200 | Skill deleted | **application/json**: [SkillDeleteResponse](#skilldeleteresponse)<br> |
### [GET] /workspaces/current/skills/{skill_id}
#### Parameters
| Name | Located in | Description | Required | Schema |
| ---- | ---------- | ----------- | -------- | ------ |
| skill_id | path | | Yes | string |
#### Responses
| Code | Description | Schema |
| ---- | ----------- | ------ |
| 200 | Skill detail | **application/json**: [SkillDetailResponse](#skilldetailresponse)<br> |
### [PATCH] /workspaces/current/skills/{skill_id}
#### Parameters
| Name | Located in | Description | Required | Schema |
| ---- | ---------- | ----------- | -------- | ------ |
| skill_id | path | | Yes | string |
#### Request Body
| Required | Schema |
| -------- | ------ |
| Yes | **application/json**: [SkillMetadataPayload](#skillmetadatapayload)<br> |
#### Responses
| Code | Description | Schema |
| ---- | ----------- | ------ |
| 200 | Skill updated | **application/json**: [SkillResponse](#skillresponse)<br> |
### [POST] /workspaces/current/skills/{skill_id}/assist/messages
#### Parameters
| Name | Located in | Description | Required | Schema |
| ---- | ---------- | ----------- | -------- | ------ |
| skill_id | path | | Yes | string |
#### Request Body
| Required | Schema |
| -------- | ------ |
| Yes | **application/json**: [SkillAssistMessagePayload](#skillassistmessagepayload)<br> |
#### Responses
| Code | Description |
| ---- | ----------- |
| 200 | Skill Authoring assistant event stream |
### [POST] /workspaces/current/skills/{skill_id}/duplicate
#### Parameters
| Name | Located in | Description | Required | Schema |
| ---- | ---------- | ----------- | -------- | ------ |
| skill_id | path | | Yes | string |
#### Responses
| Code | Description | Schema |
| ---- | ----------- | ------ |
| 201 | Skill duplicated | **application/json**: [SkillDetailResponse](#skilldetailresponse)<br> |
### [GET] /workspaces/current/skills/{skill_id}/export
#### Parameters
| Name | Located in | Description | Required | Schema |
| ---- | ---------- | ----------- | -------- | ------ |
| skill_id | path | | Yes | string |
#### Responses
| Code | Description |
| ---- | ----------- |
| 200 | Published Skill zip archive |
### [PATCH] /workspaces/current/skills/{skill_id}/files
#### Parameters
| Name | Located in | Description | Required | Schema |
| ---- | ---------- | ----------- | -------- | ------ |
| skill_id | path | | Yes | string |
#### Request Body
| Required | Schema |
| -------- | ------ |
| Yes | **application/json**: [SkillDraftFileOperationPayload](#skilldraftfileoperationpayload)<br> |
#### Responses
| Code | Description | Schema |
| ---- | ----------- | ------ |
| 200 | Draft file operation applied | **application/json**: [SkillDetailResponse](#skilldetailresponse)<br> |
### [PUT] /workspaces/current/skills/{skill_id}/files
#### Parameters
| Name | Located in | Description | Required | Schema |
| ---- | ---------- | ----------- | -------- | ------ |
| skill_id | path | | Yes | string |
#### Request Body
| Required | Schema |
| -------- | ------ |
| Yes | **application/json**: [SkillDraftTreePayload](#skilldrafttreepayload)<br> |
#### Responses
| Code | Description | Schema |
| ---- | ----------- | ------ |
| 200 | Draft files replaced | **application/json**: [SkillDetailResponse](#skilldetailresponse)<br> |
### [GET] /workspaces/current/skills/{skill_id}/files/content
#### Parameters
| Name | Located in | Description | Required | Schema |
| ---- | ---------- | ----------- | -------- | ------ |
| download | query | Return as an attachment when 1. | No | string |
| path | query | Skill file path relative to the Skill root. | Yes | string |
| version_id | query | Optional published version ID. Omit for current draft. | No | string |
| skill_id | path | | Yes | string |
#### Responses
| Code | Description | Schema |
| ---- | ----------- | ------ |
| 200 | Skill file content | **application/json**: [BinaryFileResponse](#binaryfileresponse)<br> |
### [GET] /workspaces/current/skills/{skill_id}/files/preview
#### Parameters
| Name | Located in | Description | Required | Schema |
| ---- | ---------- | ----------- | -------- | ------ |
| path | query | Skill file path relative to the Skill root. | Yes | string |
| version_id | query | Optional published version ID. Omit for current draft. | No | string |
| skill_id | path | | Yes | string |
#### Responses
| Code | Description | Schema |
| ---- | ----------- | ------ |
| 200 | Skill file text preview | **application/json**: [SkillFilePreviewResponse](#skillfilepreviewresponse)<br> |
### [POST] /workspaces/current/skills/{skill_id}/publish
#### Parameters
| Name | Located in | Description | Required | Schema |
| ---- | ---------- | ----------- | -------- | ------ |
| skill_id | path | | Yes | string |
#### Request Body
| Required | Schema |
| -------- | ------ |
| Yes | **application/json**: [SkillPublishPayload](#skillpublishpayload)<br> |
#### Responses
| Code | Description | Schema |
| ---- | ----------- | ------ |
| 200 | Skill published | **application/json**: [SkillVersionResponse](#skillversionresponse)<br> |
### [GET] /workspaces/current/skills/{skill_id}/references
#### Parameters
| Name | Located in | Description | Required | Schema |
| ---- | ---------- | ----------- | -------- | ------ |
| skill_id | path | | Yes | string |
#### Responses
| Code | Description | Schema |
| ---- | ----------- | ------ |
| 200 | Skill references | **application/json**: [SkillReferenceListResponse](#skillreferencelistresponse)<br> |
### [POST] /workspaces/current/skills/{skill_id}/restore
#### Parameters
| Name | Located in | Description | Required | Schema |
| ---- | ---------- | ----------- | -------- | ------ |
| skill_id | path | | Yes | string |
#### Request Body
| Required | Schema |
| -------- | ------ |
| Yes | **application/json**: [SkillRestorePayload](#skillrestorepayload)<br> |
#### Responses
| Code | Description | Schema |
| ---- | ----------- | ------ |
| 200 | Skill version restored | **application/json**: [SkillVersionResponse](#skillversionresponse)<br> |
### [GET] /workspaces/current/skills/{skill_id}/versions
#### Parameters
| Name | Located in | Description | Required | Schema |
| ---- | ---------- | ----------- | -------- | ------ |
| skill_id | path | | Yes | string |
#### Responses
| Code | Description | Schema |
| ---- | ----------- | ------ |
| 200 | Skill versions | **application/json**: [SkillVersionListResponse](#skillversionlistresponse)<br> |
### [DELETE] /workspaces/current/skills/{skill_id}/versions/{version_id}
#### Parameters
| Name | Located in | Description | Required | Schema |
| ---- | ---------- | ----------- | -------- | ------ |
| skill_id | path | | Yes | string |
| version_id | path | | Yes | string |
#### Responses
| Code | Description | Schema |
| ---- | ----------- | ------ |
| 200 | Skill version deleted | **application/json**: [SkillVersionDeleteResponse](#skillversiondeleteresponse)<br> |
### [GET] /workspaces/current/skills/{skill_id}/versions/{version_id}
#### Parameters
| Name | Located in | Description | Required | Schema |
| ---- | ---------- | ----------- | -------- | ------ |
| skill_id | path | | Yes | string |
| version_id | path | | Yes | string |
#### Responses
| Code | Description | Schema |
| ---- | ----------- | ------ |
| 200 | Skill version detail | **application/json**: [SkillVersionDetailResponse](#skillversiondetailresponse)<br> |
### [PATCH] /workspaces/current/skills/{skill_id}/versions/{version_id}
#### Parameters
| Name | Located in | Description | Required | Schema |
| ---- | ---------- | ----------- | -------- | ------ |
| skill_id | path | | Yes | string |
| version_id | path | | Yes | string |
#### Request Body
| Required | Schema |
| -------- | ------ |
| Yes | **application/json**: [SkillVersionUpdatePayload](#skillversionupdatepayload)<br> |
#### Responses
| Code | Description | Schema |
| ---- | ----------- | ------ |
| 200 | Skill version updated | **application/json**: [SkillVersionResponse](#skillversionresponse)<br> |
### [GET] /workspaces/current/tool-labels ### [GET] /workspaces/current/tool-labels
#### Responses #### Responses
@@ -13243,6 +13627,7 @@ Model class for AI model.
| Name | Type | Description | Required | | Name | Type | Description | Required |
| ---- | ---- | ----------- | -------- | | ---- | ---- | ----------- | -------- |
| active_config_is_published | boolean | | Yes |
| active_config_snapshot | [AgentConfigSnapshotSummaryResponse](#agentconfigsnapshotsummaryresponse) | | No | | active_config_snapshot | [AgentConfigSnapshotSummaryResponse](#agentconfigsnapshotsummaryresponse) | | No |
| agent | [AgentComposerAgentResponse](#agentcomposeragentresponse) | | Yes | | agent | [AgentComposerAgentResponse](#agentcomposeragentresponse) | | Yes |
| agent_soul | [AgentSoulConfig](#agentsoulconfig) | | Yes | | agent_soul | [AgentSoulConfig](#agentsoulconfig) | | Yes |
@@ -13282,7 +13667,6 @@ Model class for AI model.
| Name | Type | Description | Required | | Name | Type | Description | Required |
| ---- | ---- | ----------- | -------- | | ---- | ---- | ----------- | -------- |
| access_mode | string | | No | | access_mode | string | | No |
| active_config_is_published | boolean | | No |
| api_base_url | string | | No | | api_base_url | string | | No |
| app_id | string | | No | | app_id | string | | No |
| backing_app_id | string | | No | | backing_app_id | string | | No |
@@ -14692,6 +15076,37 @@ Visibility and lifecycle scope of an Agent record.
| ---- | ---- | ----------- | -------- | | ---- | ---- | ----------- | -------- |
| result | string | | Yes | | result | string | | Yes |
#### AgentSkillBindingItemResponse
| Name | Type | Description | Required |
| ---- | ---- | ----------- | -------- |
| description | string | | Yes |
| display_name | string | | Yes |
| file_count | integer | | Yes |
| icon | string | | Yes |
| id | string | | Yes |
| latest_published_at | integer | | No |
| latest_published_version_id | string | | No |
| name | string | | Yes |
| priority | integer | | Yes |
| status | string | | Yes |
| tags | [ string ] | | No |
| updated_at | integer | | Yes |
#### AgentSkillBindingsPayload
| Name | Type | Description | Required |
| ---- | ---- | ----------- | -------- |
| skill_ids | [ string ] | Ordered Skill IDs bound to the Agent. | No |
#### AgentSkillBindingsResponse
| Name | Type | Description | Required |
| ---- | ---- | ----------- | -------- |
| agent_id | string | | Yes |
| data | [ [AgentSkillBindingItemResponse](#agentskillbindingitemresponse) ] | | No |
| skill_ids | [ string ] | | No |
#### AgentSkillRefConfig #### AgentSkillRefConfig
| Name | Type | Description | Required | | Name | Type | Description | Required |
@@ -21018,6 +21433,28 @@ Whitelist scopes accepted by RBAC app and dataset access config APIs.
| result | string | | Yes | | result | string | | Yes |
| updated_at | integer | | Yes | | updated_at | integer | | Yes |
#### RecentAppListResponse
| Name | Type | Description | Required |
| ---- | ---- | ----------- | -------- |
| data | [ [RecentAppResponse](#recentappresponse) ] | | Yes |
#### RecentAppResponse
| Name | Type | Description | Required |
| ---- | ---- | ----------- | -------- |
| author_name | string | | No |
| icon | string | | No |
| icon_background | string | | No |
| icon_type | [IconType](#icontype) | | No |
| icon_url | string | | Yes |
| id | string | | Yes |
| maintainer | string | | No |
| mode | string, <br>**Available values:** "advanced-chat", "agent-chat", "chat", "completion", "workflow" | *Enum:* `"advanced-chat"`, `"agent-chat"`, `"chat"`, `"completion"`, `"workflow"` | Yes |
| name | string | | Yes |
| permission_keys | [ string ] | | No |
| updated_at | integer | | Yes |
#### RecommendedAppDetailNullableResponse #### RecommendedAppDetailNullableResponse
| Name | Type | Description | Required | | Name | Type | Description | Required |
@@ -21626,6 +22063,186 @@ Simple provider entity response.
| title | string | | Yes | | title | string | | Yes |
| use_icon_as_answer_icon | boolean | | Yes | | use_icon_as_answer_icon | boolean | | Yes |
#### SkillAssistAttachmentPayload
| Name | Type | Description | Required |
| ---- | ---- | ----------- | -------- |
| mime_type | string | | No |
| name | string | | Yes |
| size | integer | | No |
| tool_file_id | string | | Yes |
#### SkillAssistMessagePayload
One user message and optional uploaded context for the read-only Skill Authoring assistant.
| Name | Type | Description | Required |
| ---- | ---- | ----------- | -------- |
| attachments | [ [SkillAssistAttachmentPayload](#skillassistattachmentpayload) ] | | No |
| message | string | | Yes |
| model | [SkillAssistModelPayload](#skillassistmodelpayload) | | No |
#### SkillAssistModelPayload
| Name | Type | Description | Required |
| ---- | ---- | ----------- | -------- |
| model | string | | Yes |
| model_settings | object | | No |
| plugin_id | string | | No |
| provider | string | | Yes |
#### SkillCreatePayload
| Name | Type | Description | Required |
| ---- | ---- | ----------- | -------- |
| description | string | | No |
| display_name | string | | No |
| icon | string, <br>**Default:** 📄 | | No |
| name | string | | No |
| tags | [ string ] | | No |
#### SkillDeletePayload
| Name | Type | Description | Required |
| ---- | ---- | ----------- | -------- |
| confirmation_name | string | Required when deleting a referenced Skill. Must match the Skill name. | No |
#### SkillDeleteResponse
| Name | Type | Description | Required |
| ---- | ---- | ----------- | -------- |
| deleted | boolean | | Yes |
| id | string | | Yes |
#### SkillDetailResponse
| Name | Type | Description | Required |
| ---- | ---- | ----------- | -------- |
| created_at | integer | | Yes |
| created_by | string | | No |
| created_by_name | string | | No |
| description | string | | Yes |
| display_name | string | | Yes |
| files | [ [SkillFileResponse](#skillfileresponse) ] | | No |
| icon | string | | Yes |
| id | string | | Yes |
| latest_published_version_id | string | | No |
| name | string | | Yes |
| name_manually_edited | boolean | | No |
| reference_count | integer | | No |
| tags | [ string ] | | No |
| updated_at | integer | | Yes |
| updated_by | string | | No |
| updated_by_name | string | | No |
| visibility | string | | Yes |
#### SkillDraftFileOperation
| Name | Type | Description | Required |
| ---- | ---- | ----------- | -------- |
| SkillDraftFileOperation | string | | |
#### SkillDraftFileOperationPayload
| Name | Type | Description | Required |
| ---- | ---- | ----------- | -------- |
| content | string | | No |
| expected_updated_at | integer | | No |
| hash | string | | No |
| mime_type | string | | No |
| operation | [SkillDraftFileOperation](#skilldraftfileoperation) | | Yes |
| path | string | | Yes |
| size | integer | | No |
| target_path | string | | No |
| tool_file_id | string | | No |
#### SkillDraftTreeItemPayload
| Name | Type | Description | Required |
| ---- | ---- | ----------- | -------- |
| content | string | | No |
| hash | string | | No |
| kind | [SkillFileKind](#skillfilekind) | | No |
| mime_type | string | | No |
| path | string | | Yes |
| size | integer | | No |
| storage | [SkillFileStorage](#skillfilestorage) | | No |
| tool_file_id | string | | No |
#### SkillDraftTreePayload
| Name | Type | Description | Required |
| ---- | ---- | ----------- | -------- |
| expected_updated_at | integer | | No |
| files | [ [SkillDraftTreeItemPayload](#skilldrafttreeitempayload) ] | | No |
#### SkillFileKind
Draft file entry kind.
| Name | Type | Description | Required |
| ---- | ---- | ----------- | -------- |
| SkillFileKind | string | Draft file entry kind. | |
#### SkillFilePreviewResponse
| Name | Type | Description | Required |
| ---- | ---- | ----------- | -------- |
| content | string | | Yes |
| hash | string | | Yes |
| mime_type | string | | Yes |
| path | string | | Yes |
| size | integer | | Yes |
#### SkillFileQuery
| Name | Type | Description | Required |
| ---- | ---- | ----------- | -------- |
| path | string | Skill file path relative to the Skill root. | Yes |
| version_id | string | Optional published version ID. Omit for current draft. | No |
#### SkillFileResponse
| Name | Type | Description | Required |
| ---- | ---- | ----------- | -------- |
| content | string | | No |
| hash | string | | No |
| id | string | | No |
| kind | string | | Yes |
| mime_type | string | | No |
| path | string | | Yes |
| size | integer | | No |
| storage | string | | No |
| tool_file_id | string | | No |
#### SkillFileStorage
How a draft file's content is stored.
| Name | Type | Description | Required |
| ---- | ---- | ----------- | -------- |
| SkillFileStorage | string | How a draft file's content is stored. | |
#### SkillFileUploadResponse
| Name | Type | Description | Required |
| ---- | ---- | ----------- | -------- |
| hash | string | | Yes |
| id | string | | Yes |
| mime_type | string | | Yes |
| name | string | | Yes |
| size | integer | | Yes |
#### SkillListResponse
| Name | Type | Description | Required |
| ---- | ---- | ----------- | -------- |
| data | [ [SkillResponse](#skillresponse) ] | | No |
| has_more | boolean | | No |
| limit | integer, <br>**Default:** 20 | | No |
| page | integer, <br>**Default:** 1 | | No |
| total | integer | | No |
#### SkillManifest #### SkillManifest
Validated metadata extracted from a Skill package. Validated metadata extracted from a Skill package.
@@ -21639,6 +22256,91 @@ Validated metadata extracted from a Skill package.
| name | string | | Yes | | name | string | | Yes |
| size | integer | | Yes | | size | integer | | Yes |
#### SkillMetadataPayload
| Name | Type | Description | Required |
| ---- | ---- | ----------- | -------- |
| display_name | string | | No |
| expected_updated_at | integer | | No |
| icon | string | | No |
| tags | [ string ] | | No |
#### SkillPublishPayload
| Name | Type | Description | Required |
| ---- | ---- | ----------- | -------- |
| publish_note | string | | No |
| version_name | string | | No |
#### SkillReferenceListResponse
| Name | Type | Description | Required |
| ---- | ---- | ----------- | -------- |
| data | [ [SkillReferenceResponse](#skillreferenceresponse) ] | | No |
#### SkillReferenceResponse
| Name | Type | Description | Required |
| ---- | ---- | ----------- | -------- |
| agent_icon | string | | No |
| agent_icon_background | string | | No |
| agent_icon_type | string | | No |
| agent_id | string | | Yes |
| app_id | string | | No |
| display_name | string | | Yes |
| name | string | | Yes |
| node_id | string | | No |
| node_name | string | | No |
| type | string | | Yes |
| workflow_icon | string | | No |
| workflow_icon_background | string | | No |
| workflow_icon_type | string | | No |
| workflow_id | string | | No |
| workflow_name | string | | No |
| workflow_version | string | | No |
#### SkillResponse
| Name | Type | Description | Required |
| ---- | ---- | ----------- | -------- |
| created_at | integer | | Yes |
| created_by | string | | No |
| created_by_name | string | | No |
| description | string | | Yes |
| display_name | string | | Yes |
| icon | string | | Yes |
| id | string | | Yes |
| latest_published_version_id | string | | No |
| name | string | | Yes |
| name_manually_edited | boolean | | No |
| reference_count | integer | | No |
| tags | [ string ] | | No |
| updated_at | integer | | Yes |
| updated_by | string | | No |
| updated_by_name | string | | No |
| visibility | string | | Yes |
#### SkillRestorePayload
| Name | Type | Description | Required |
| ---- | ---- | ----------- | -------- |
| publish_note | string | | No |
| version_id | string | | Yes |
| version_name | string | | No |
#### SkillTagListResponse
| Name | Type | Description | Required |
| ---- | ---- | ----------- | -------- |
| data | [ [SkillTagResponse](#skilltagresponse) ] | | No |
#### SkillTagResponse
| Name | Type | Description | Required |
| ---- | ---- | ----------- | -------- |
| count | integer | | Yes |
| tag | string | | Yes |
#### SkillToolInferenceResult #### SkillToolInferenceResult
| Name | Type | Description | Required | | Name | Type | Description | Required |
@@ -21647,6 +22349,60 @@ Validated metadata extracted from a Skill package.
| inferable | boolean | | Yes | | inferable | boolean | | Yes |
| reason | string | | No | | reason | string | | No |
#### SkillVersionDeleteResponse
| Name | Type | Description | Required |
| ---- | ---- | ----------- | -------- |
| deleted | boolean | | Yes |
| id | string | | Yes |
| latest_published_version_id | string | | No |
#### SkillVersionDetailResponse
| Name | Type | Description | Required |
| ---- | ---- | ----------- | -------- |
| archive_size | integer | | Yes |
| created_at | integer | | Yes |
| files | [ [SkillFileResponse](#skillfileresponse) ] | | No |
| hash_code | string | | Yes |
| id | string | | Yes |
| is_latest | boolean | | No |
| publish_note | string | | Yes |
| published_by | string | | No |
| published_by_name | string | | No |
| skill_id | string | | Yes |
| version_name | string | | Yes |
| version_number | integer | | Yes |
#### SkillVersionListResponse
| Name | Type | Description | Required |
| ---- | ---- | ----------- | -------- |
| data | [ [SkillVersionResponse](#skillversionresponse) ] | | No |
#### SkillVersionResponse
| Name | Type | Description | Required |
| ---- | ---- | ----------- | -------- |
| archive_size | integer | | Yes |
| created_at | integer | | Yes |
| hash_code | string | | Yes |
| id | string | | Yes |
| is_latest | boolean | | No |
| publish_note | string | | Yes |
| published_by | string | | No |
| published_by_name | string | | No |
| skill_id | string | | Yes |
| version_name | string | | Yes |
| version_number | integer | | Yes |
#### SkillVersionUpdatePayload
| Name | Type | Description | Required |
| ---- | ---- | ----------- | -------- |
| publish_note | string | | No |
| version_name | string | | No |
#### SnippetDependencyCheckResponse #### SnippetDependencyCheckResponse
| Name | Type | Description | Required | | Name | Type | Description | Required |
@@ -22114,7 +22870,7 @@ Non-sensitive bootstrap snapshot exposed before Console or Web authentication.
| Name | Type | Description | Required | | Name | Type | Description | Required |
| ---- | ---- | ----------- | -------- | | ---- | ---- | ----------- | -------- |
| keyword | string | Search keyword | No | | keyword | string | Search keyword | No |
| type | string, <br>**Available values:** "", "app", "knowledge", "snippet" | Tag type filter<br>*Enum:* `""`, `"app"`, `"knowledge"`, `"snippet"` | No | | type | [TagType](#tagtype)<br>string | Tag type filter | No |
#### TagListResponse #### TagListResponse
@@ -24300,6 +25056,15 @@ Workflow tool configuration
| ---- | ---- | ----------- | -------- | | ---- | ---- | ----------- | -------- |
| permission_keys | [ string ] | | No | | permission_keys | [ string ] | | No |
#### WorkspaceSkillsQuery
| Name | Type | Description | Required |
| ---- | ---- | ----------- | -------- |
| keyword | string | Search keyword matching skill name or description. | No |
| limit | integer, <br>**Default:** 20 | Number of items per page. | No |
| page | integer, <br>**Default:** 1 | Page number. | No |
| tag | [ string ] | Skill tag filters. Repeat the parameter for multiple tags. | No |
#### WorkspaceTenantResultResponse #### WorkspaceTenantResultResponse
| Name | Type | Description | Required | | Name | Type | Description | Required |
+7
View File
@@ -965,6 +965,12 @@ Returns Server-Sent Events stream.
| ---- | ---- | ----------- | -------- | | ---- | ---- | ----------- | -------- |
| tool_icons | object | Tool icon metadata keyed by tool name | No | | tool_icons | object | Tool icon metadata keyed by tool name | No |
#### AppMode
| Name | Type | Description | Required |
| ---- | ---- | ----------- | -------- |
| AppMode | string | | |
#### AppPermissionQuery #### AppPermissionQuery
| Name | Type | Description | Required | | Name | Type | Description | Required |
@@ -1646,6 +1652,7 @@ in form definition, or a variable while the workflow is running.
| custom_config | [WebAppCustomConfigResponse](#webappcustomconfigresponse) | | No | | custom_config | [WebAppCustomConfigResponse](#webappcustomconfigresponse) | | No |
| enable_site | boolean | | Yes | | enable_site | boolean | | Yes |
| end_user_id | string | | No | | end_user_id | string | | No |
| mode | [AppMode](#appmode) | | Yes |
| model_config | [WebModelConfigResponse](#webmodelconfigresponse) | | No | | model_config | [WebModelConfigResponse](#webmodelconfigresponse) | | No |
| plan | string | | Yes | | plan | string | | Yes |
| site | [WebSiteResponse](#websiteresponse) | | Yes | | site | [WebSiteResponse](#websiteresponse) | | Yes |
@@ -1,3 +1,5 @@
"""Unit tests for Aliyun trace utility transformations and database lookups."""
import json import json
from collections.abc import Mapping from collections.abc import Mapping
from typing import Any, cast from typing import Any, cast
@@ -25,11 +27,13 @@ from dify_trace_aliyun.utils import (
serialize_json_data, serialize_json_data,
) )
from opentelemetry.trace import Link, StatusCode from opentelemetry.trace import Link, StatusCode
from sqlalchemy.orm import Session
from core.rag.models.document import Document from core.rag.models.document import Document
from graphon.entities import WorkflowNodeExecution from graphon.entities import WorkflowNodeExecution
from graphon.enums import WorkflowNodeExecutionStatus from graphon.enums import WorkflowNodeExecutionStatus
from models import EndUser from models import EndUser
from models.enums import EndUserType
def test_get_user_id_from_message_data_no_end_user(monkeypatch: pytest.MonkeyPatch): def test_get_user_id_from_message_data_no_end_user(monkeypatch: pytest.MonkeyPatch):
@@ -40,35 +44,40 @@ def test_get_user_id_from_message_data_no_end_user(monkeypatch: pytest.MonkeyPat
assert get_user_id_from_message_data(message_data) == "account_id" assert get_user_id_from_message_data(message_data) == "account_id"
def test_get_user_id_from_message_data_with_end_user(monkeypatch: pytest.MonkeyPatch): @pytest.mark.parametrize("sqlite3_session", [(EndUser,)], indirect=True)
def test_get_user_id_from_message_data_with_end_user(monkeypatch: pytest.MonkeyPatch, sqlite3_session: Session) -> None:
message_data = MagicMock() message_data = MagicMock()
message_data.from_account_id = "account_id" message_data.from_account_id = "account_id"
message_data.from_end_user_id = "end_user_id" message_data.from_end_user_id = "end_user_id"
end_user_data = MagicMock(spec=EndUser) end_user_data = EndUser(
end_user_data.session_id = "session_id" id="end_user_id",
tenant_id="tenant_id",
mock_session = MagicMock() app_id="app_id",
mock_session.get.return_value = end_user_data type=EndUserType.BROWSER,
session_id="session_id",
)
sqlite3_session.add(end_user_data)
sqlite3_session.commit()
from dify_trace_aliyun.utils import db from dify_trace_aliyun.utils import db
monkeypatch.setattr(db, "session", mock_session) monkeypatch.setattr(db, "session", sqlite3_session)
assert get_user_id_from_message_data(message_data) == "session_id" assert get_user_id_from_message_data(message_data) == "session_id"
def test_get_user_id_from_message_data_end_user_not_found(monkeypatch: pytest.MonkeyPatch): @pytest.mark.parametrize("sqlite3_session", [(EndUser,)], indirect=True)
def test_get_user_id_from_message_data_end_user_not_found(
monkeypatch: pytest.MonkeyPatch, sqlite3_session: Session
) -> None:
message_data = MagicMock() message_data = MagicMock()
message_data.from_account_id = "account_id" message_data.from_account_id = "account_id"
message_data.from_end_user_id = "end_user_id" message_data.from_end_user_id = "end_user_id"
mock_session = MagicMock()
mock_session.get.return_value = None
from dify_trace_aliyun.utils import db from dify_trace_aliyun.utils import db
monkeypatch.setattr(db, "session", mock_session) monkeypatch.setattr(db, "session", sqlite3_session)
assert get_user_id_from_message_data(message_data) == "account_id" assert get_user_id_from_message_data(message_data) == "account_id"
@@ -1,5 +1,8 @@
"""Unit tests for LangSmith trace translation with SQLite-backed lookups."""
import collections import collections
from datetime import datetime, timedelta from datetime import datetime, timedelta
from types import SimpleNamespace
from typing import override from typing import override
from unittest.mock import MagicMock from unittest.mock import MagicMock
@@ -11,6 +14,7 @@ from dify_trace_langsmith.entities.langsmith_trace_entity import (
LangSmithRunUpdateModel, LangSmithRunUpdateModel,
) )
from dify_trace_langsmith.langsmith_trace import LangSmithDataTrace from dify_trace_langsmith.langsmith_trace import LangSmithDataTrace
from sqlalchemy.orm import Session
from core.ops.entities.trace_entity import ( from core.ops.entities.trace_entity import (
DatasetRetrievalTraceInfo, DatasetRetrievalTraceInfo,
@@ -24,6 +28,7 @@ from core.ops.entities.trace_entity import (
) )
from graphon.enums import BuiltinNodeTypes, WorkflowNodeExecutionMetadataKey from graphon.enums import BuiltinNodeTypes, WorkflowNodeExecutionMetadataKey
from models import EndUser from models import EndUser
from models.enums import EndUserType
def _dt() -> datetime: def _dt() -> datetime:
@@ -108,7 +113,8 @@ def test_trace_dispatch(trace_instance, monkeypatch: pytest.MonkeyPatch):
mocks["generate_name_trace"].assert_called_once_with(info) mocks["generate_name_trace"].assert_called_once_with(info)
def test_workflow_trace(trace_instance, monkeypatch: pytest.MonkeyPatch): @pytest.mark.parametrize("sqlite3_session", [()], indirect=True)
def test_workflow_trace(trace_instance, monkeypatch: pytest.MonkeyPatch, sqlite3_session: Session) -> None:
# Setup trace info # Setup trace info
workflow_data = MagicMock() workflow_data = MagicMock()
workflow_data.created_at = _dt() workflow_data.created_at = _dt()
@@ -137,10 +143,10 @@ def test_workflow_trace(trace_instance, monkeypatch: pytest.MonkeyPatch):
workflow_data=workflow_data, workflow_data=workflow_data,
) )
# Mock dependencies monkeypatch.setattr(
mock_session = MagicMock() "dify_trace_langsmith.langsmith_trace.db",
monkeypatch.setattr("dify_trace_langsmith.langsmith_trace.sessionmaker", lambda bind: lambda: mock_session) SimpleNamespace(engine=sqlite3_session.get_bind(), session=sqlite3_session),
monkeypatch.setattr("dify_trace_langsmith.langsmith_trace.db", MagicMock(engine="engine")) )
# Mock node executions # Mock node executions
node_llm = MagicMock() node_llm = MagicMock()
@@ -228,7 +234,10 @@ def test_workflow_trace(trace_instance, monkeypatch: pytest.MonkeyPatch):
assert call_args[4].run_type == LangSmithRunType.retriever assert call_args[4].run_type == LangSmithRunType.retriever
def test_workflow_trace_no_start_time(trace_instance, monkeypatch: pytest.MonkeyPatch): @pytest.mark.parametrize("sqlite3_session", [()], indirect=True)
def test_workflow_trace_no_start_time(
trace_instance, monkeypatch: pytest.MonkeyPatch, sqlite3_session: Session
) -> None:
workflow_data = MagicMock() workflow_data = MagicMock()
workflow_data.created_at = _dt() workflow_data.created_at = _dt()
workflow_data.finished_at = _dt() + timedelta(seconds=1) workflow_data.finished_at = _dt() + timedelta(seconds=1)
@@ -256,9 +265,10 @@ def test_workflow_trace_no_start_time(trace_instance, monkeypatch: pytest.Monkey
workflow_data=workflow_data, workflow_data=workflow_data,
) )
mock_session = MagicMock() monkeypatch.setattr(
monkeypatch.setattr("dify_trace_langsmith.langsmith_trace.sessionmaker", lambda bind: lambda: mock_session) "dify_trace_langsmith.langsmith_trace.db",
monkeypatch.setattr("dify_trace_langsmith.langsmith_trace.db", MagicMock(engine="engine")) SimpleNamespace(engine=sqlite3_session.get_bind(), session=sqlite3_session),
)
repo = MagicMock() repo = MagicMock()
repo.get_by_workflow_execution.return_value = [] repo.get_by_workflow_execution.return_value = []
mock_factory = MagicMock() mock_factory = MagicMock()
@@ -271,7 +281,10 @@ def test_workflow_trace_no_start_time(trace_instance, monkeypatch: pytest.Monkey
assert trace_instance.add_run.called assert trace_instance.add_run.called
def test_workflow_trace_missing_app_id(trace_instance, monkeypatch: pytest.MonkeyPatch): @pytest.mark.parametrize("sqlite3_session", [()], indirect=True)
def test_workflow_trace_missing_app_id(
trace_instance, monkeypatch: pytest.MonkeyPatch, sqlite3_session: Session
) -> None:
trace_info = MagicMock(spec=WorkflowTraceInfo) trace_info = MagicMock(spec=WorkflowTraceInfo)
trace_info.trace_id = "trace-1" trace_info.trace_id = "trace-1"
trace_info.message_id = None trace_info.message_id = None
@@ -287,15 +300,17 @@ def test_workflow_trace_missing_app_id(trace_instance, monkeypatch: pytest.Monke
trace_info.workflow_run_outputs = {} trace_info.workflow_run_outputs = {}
trace_info.error = "" trace_info.error = ""
mock_session = MagicMock() monkeypatch.setattr(
monkeypatch.setattr("dify_trace_langsmith.langsmith_trace.sessionmaker", lambda bind: lambda: mock_session) "dify_trace_langsmith.langsmith_trace.db",
monkeypatch.setattr("dify_trace_langsmith.langsmith_trace.db", MagicMock(engine="engine")) SimpleNamespace(engine=sqlite3_session.get_bind(), session=sqlite3_session),
)
with pytest.raises(ValueError, match="No app_id found in trace_info metadata"): with pytest.raises(ValueError, match="No app_id found in trace_info metadata"):
trace_instance.workflow_trace(trace_info) trace_instance.workflow_trace(trace_info)
def test_message_trace(trace_instance, monkeypatch: pytest.MonkeyPatch): @pytest.mark.parametrize("sqlite3_session", [(EndUser,)], indirect=True)
def test_message_trace(trace_instance, monkeypatch: pytest.MonkeyPatch, sqlite3_session: Session) -> None:
message_data = MagicMock() message_data = MagicMock()
message_data.id = "msg-1" message_data.id = "msg-1"
message_data.from_account_id = "acc-1" message_data.from_account_id = "acc-1"
@@ -321,10 +336,19 @@ def test_message_trace(trace_instance, monkeypatch: pytest.MonkeyPatch):
message_file_data=MagicMock(url="file-url"), message_file_data=MagicMock(url="file-url"),
) )
# Mock EndUser lookup end_user = EndUser(
mock_end_user = MagicMock(spec=EndUser) id="end-user-1",
mock_end_user.session_id = "session-id-123" tenant_id="tenant-1",
monkeypatch.setattr("dify_trace_langsmith.langsmith_trace.db.session.get", lambda model, pk: mock_end_user) app_id="app-1",
type=EndUserType.BROWSER,
session_id="session-id-123",
)
sqlite3_session.add(end_user)
sqlite3_session.commit()
monkeypatch.setattr(
"dify_trace_langsmith.langsmith_trace.db",
SimpleNamespace(engine=sqlite3_session.get_bind(), session=sqlite3_session),
)
trace_instance.add_run = MagicMock() trace_instance.add_run = MagicMock()
@@ -521,9 +545,13 @@ def test_update_run_error(trace_instance):
trace_instance.update_run(update_data) trace_instance.update_run(update_data)
@pytest.mark.parametrize("sqlite3_session", [()], indirect=True)
def test_workflow_trace_usage_extraction_error( def test_workflow_trace_usage_extraction_error(
trace_instance, monkeypatch: pytest.MonkeyPatch, caplog: pytest.LogCaptureFixture trace_instance,
): monkeypatch: pytest.MonkeyPatch,
caplog: pytest.LogCaptureFixture,
sqlite3_session: Session,
) -> None:
workflow_data = MagicMock() workflow_data = MagicMock()
workflow_data.created_at = _dt() workflow_data.created_at = _dt()
workflow_data.finished_at = _dt() + timedelta(seconds=1) workflow_data.finished_at = _dt() + timedelta(seconds=1)
@@ -576,8 +604,10 @@ def test_workflow_trace_usage_extraction_error(
mock_factory = MagicMock() mock_factory = MagicMock()
mock_factory.create_workflow_node_execution_repository.return_value = repo mock_factory.create_workflow_node_execution_repository.return_value = repo
monkeypatch.setattr("dify_trace_langsmith.langsmith_trace.DifyCoreRepositoryFactory", mock_factory) monkeypatch.setattr("dify_trace_langsmith.langsmith_trace.DifyCoreRepositoryFactory", mock_factory)
monkeypatch.setattr("dify_trace_langsmith.langsmith_trace.sessionmaker", lambda bind: lambda: MagicMock()) monkeypatch.setattr(
monkeypatch.setattr("dify_trace_langsmith.langsmith_trace.db", MagicMock(engine="engine")) "dify_trace_langsmith.langsmith_trace.db",
SimpleNamespace(engine=sqlite3_session.get_bind(), session=sqlite3_session),
)
monkeypatch.setattr(trace_instance, "get_service_account_with_tenant", lambda app_id: MagicMock()) monkeypatch.setattr(trace_instance, "get_service_account_with_tenant", lambda app_id: MagicMock())
trace_instance.add_run = MagicMock() trace_instance.add_run = MagicMock()
@@ -644,9 +674,11 @@ def _make_workflow_trace_info(
) )
def _patch_workflow_trace_deps(monkeypatch, trace_instance): def _patch_workflow_trace_deps(monkeypatch, trace_instance, sqlite3_session: Session) -> None:
monkeypatch.setattr("dify_trace_langsmith.langsmith_trace.sessionmaker", lambda bind: lambda: MagicMock()) monkeypatch.setattr(
monkeypatch.setattr("dify_trace_langsmith.langsmith_trace.db", MagicMock(engine="engine")) "dify_trace_langsmith.langsmith_trace.db",
SimpleNamespace(engine=sqlite3_session.get_bind(), session=sqlite3_session),
)
repo = MagicMock() repo = MagicMock()
repo.get_by_workflow_execution.return_value = [] repo.get_by_workflow_execution.return_value = []
factory = MagicMock() factory = MagicMock()
@@ -656,14 +688,17 @@ def _patch_workflow_trace_deps(monkeypatch, trace_instance):
trace_instance.add_run = MagicMock() trace_instance.add_run = MagicMock()
def test_workflow_trace_id_uses_message_id_not_external(trace_instance, monkeypatch: pytest.MonkeyPatch): @pytest.mark.parametrize("sqlite3_session", [()], indirect=True)
def test_workflow_trace_id_uses_message_id_not_external(
trace_instance, monkeypatch: pytest.MonkeyPatch, sqlite3_session: Session
) -> None:
"""Chatflow with external trace_id: LangSmith trace_id must be message_id, not external.""" """Chatflow with external trace_id: LangSmith trace_id must be message_id, not external."""
trace_info = _make_workflow_trace_info( trace_info = _make_workflow_trace_info(
message_id="msg-abc", message_id="msg-abc",
workflow_run_id="run-xyz", workflow_run_id="run-xyz",
trace_id="external-999", trace_id="external-999",
) )
_patch_workflow_trace_deps(monkeypatch, trace_instance) _patch_workflow_trace_deps(monkeypatch, trace_instance, sqlite3_session)
trace_instance.workflow_trace(trace_info) trace_instance.workflow_trace(trace_info)
@@ -677,14 +712,17 @@ def test_workflow_trace_id_uses_message_id_not_external(trace_instance, monkeypa
assert trace_info.metadata.get("external_trace_id") == "external-999" assert trace_info.metadata.get("external_trace_id") == "external-999"
def test_workflow_trace_id_pure_workflow_uses_run_id(trace_instance, monkeypatch: pytest.MonkeyPatch): @pytest.mark.parametrize("sqlite3_session", [()], indirect=True)
def test_workflow_trace_id_pure_workflow_uses_run_id(
trace_instance, monkeypatch: pytest.MonkeyPatch, sqlite3_session: Session
) -> None:
"""Pure workflow (no message_id) with external trace_id: trace_id must be workflow_run_id.""" """Pure workflow (no message_id) with external trace_id: trace_id must be workflow_run_id."""
trace_info = _make_workflow_trace_info( trace_info = _make_workflow_trace_info(
message_id=None, message_id=None,
workflow_run_id="run-xyz", workflow_run_id="run-xyz",
trace_id="external-999", trace_id="external-999",
) )
_patch_workflow_trace_deps(monkeypatch, trace_instance) _patch_workflow_trace_deps(monkeypatch, trace_instance, sqlite3_session)
trace_instance.workflow_trace(trace_info) trace_instance.workflow_trace(trace_info)
+4 -4
View File
@@ -1,12 +1,12 @@
[project] [project]
name = "dify-api" name = "dify-api"
version = "1.16.0" version = "1.16.1"
requires-python = "~=3.12.0" requires-python = "~=3.12.0"
dependencies = [ dependencies = [
# Legacy: mature and widely deployed # Legacy: mature and widely deployed
"bleach>=6.4.0,<7.0.0", "bleach>=6.4.0,<7.0.0",
"boto3>=1.43.46,<2.0.0", "boto3>=1.43.56,<2.0.0",
"celery>=5.6.3,<6.0.0", "celery>=5.6.3,<6.0.0",
"croniter>=6.2.2,<7.0.0", "croniter>=6.2.2,<7.0.0",
"dify-agent", "dify-agent",
@@ -193,10 +193,10 @@ dev = [
############################################################ ############################################################
storage = [ storage = [
"azure-storage-blob>=12.30.0,<13.0.0", "azure-storage-blob>=12.30.0,<13.0.0",
"bce-python-sdk==0.9.72", "bce-python-sdk==0.9.76",
"cos-python-sdk-v5>=1.9.44,<2.0.0", "cos-python-sdk-v5>=1.9.44,<2.0.0",
"esdk-obs-python>=3.26.6,<4.0.0", "esdk-obs-python>=3.26.6,<4.0.0",
"google-cloud-storage>=3.12.1,<4.0.0", "google-cloud-storage>=3.13.0,<4.0.0",
"opendal==0.46.0", "opendal==0.46.0",
"oss2>=2.19.1,<3.0.0", "oss2>=2.19.1,<3.0.0",
"supabase>=2.31.0,<3.0.0", "supabase>=2.31.0,<3.0.0",
+10 -2
View File
@@ -75,6 +75,7 @@ from services.errors.account import (
from services.errors.workspace import WorkSpaceNotAllowedCreateError, WorkspacesLimitExceededError from services.errors.workspace import WorkSpaceNotAllowedCreateError, WorkspacesLimitExceededError
from services.feature_service import FeatureService from services.feature_service import FeatureService
from services.plugin.plugin_auto_upgrade_service import PluginAutoUpgradeService from services.plugin.plugin_auto_upgrade_service import PluginAutoUpgradeService
from services.telemetry_service import CommunityTelemetryService
from tasks.delete_account_task import delete_account_task from tasks.delete_account_task import delete_account_task
from tasks.mail_account_deletion_task import send_account_deletion_verification_code from tasks.mail_account_deletion_task import send_account_deletion_verification_code
from tasks.mail_change_mail_task import ( from tasks.mail_change_mail_task import (
@@ -1598,7 +1599,9 @@ class TenantService:
return updated_accounts return updated_accounts
@staticmethod @staticmethod
def iter_member_account_id_batches(tenant_id: str, batch_size: int, *, session: Session) -> Iterator[list[str]]: def iter_member_account_id_batches(
tenant_id: str, batch_size: int, *, session: Session
) -> Iterator[list[str]]:
"""Yield workspace member account ids in bounded, ordered batches.""" """Yield workspace member account ids in bounded, ordered batches."""
offset = 0 offset = 0
while True: while True:
@@ -1953,7 +1956,7 @@ class RegisterService:
TenantService.create_owner_tenant_if_not_exist(account=account, is_setup=True, session=session) TenantService.create_owner_tenant_if_not_exist(account=account, is_setup=True, session=session)
dify_setup = DifySetup(version=dify_config.project.version) dify_setup = DifySetup(version=dify_config.project.version, instance_id=str(uuid.uuid4()))
session.add(dify_setup) session.add(dify_setup)
session.commit() session.commit()
except Exception as e: except Exception as e:
@@ -1966,6 +1969,11 @@ class RegisterService:
logger.exception("Setup account failed, email: %s, name: %s", email, name) logger.exception("Setup account failed, email: %s, name: %s", email, name)
raise ValueError(f"Setup failed: {e}") raise ValueError(f"Setup failed: {e}")
try:
CommunityTelemetryService.report_install(session=session)
except Exception:
logger.debug("Failed to report install telemetry", exc_info=True)
@classmethod @classmethod
def register( def register(
cls, cls,
+1
View File
@@ -405,6 +405,7 @@ class AgentComposerService:
"variant": ComposerVariant.AGENT_APP.value, "variant": ComposerVariant.AGENT_APP.value,
"agent": cls._serialize_agent(agent), "agent": cls._serialize_agent(agent),
"active_config_snapshot": cls._serialize_version(version), "active_config_snapshot": cls._serialize_version(version),
"active_config_is_published": bool(agent.active_config_snapshot_id and agent.active_config_is_published),
"draft": cls._serialize_draft(draft), "draft": cls._serialize_draft(draft),
"agent_soul": draft.config_snapshot_dict, "agent_soul": draft.config_snapshot_dict,
"save_options": [ComposerSaveStrategy.SAVE_TO_CURRENT_VERSION.value], "save_options": [ComposerSaveStrategy.SAVE_TO_CURRENT_VERSION.value],
+46 -20
View File
@@ -19,12 +19,11 @@ from __future__ import annotations
import hashlib import hashlib
import io import io
import posixpath import posixpath
import re
import zipfile import zipfile
import zlib import zlib
import yaml import yaml
from pydantic import BaseModel from pydantic import BaseModel, Field, ValidationError, field_validator
# Bounds — generous but finite so a hostile upload can't exhaust memory/disk. # Bounds — generous but finite so a hostile upload can't exhaust memory/disk.
_MAX_ARCHIVE_BYTES = 50 * 1024 * 1024 _MAX_ARCHIVE_BYTES = 50 * 1024 * 1024
@@ -33,7 +32,8 @@ _MAX_SKILL_MD_BYTES = 1 * 1024 * 1024
_MAX_ENTRIES = 5000 _MAX_ENTRIES = 5000
_ALLOWED_EXTENSIONS = (".zip", ".skill") _ALLOWED_EXTENSIONS = (".zip", ".skill")
_SKILL_MD_NAME = "SKILL.md" _SKILL_MD_NAME = "SKILL.md"
_HEADING_RE = re.compile(r"^\s*#\s+(.+?)\s*$", re.MULTILINE) _SKILL_NAME_PATTERN = r"^[a-z0-9]+(?:-[a-z0-9]+)*$"
_MAX_SKILL_DESCRIPTION_LENGTH = 1024
class SkillPackageError(Exception): class SkillPackageError(Exception):
@@ -53,13 +53,18 @@ class SkillPackageError(Exception):
class SkillManifest(BaseModel): class SkillManifest(BaseModel):
"""Validated metadata extracted from a Skill package.""" """Validated metadata extracted from a Skill package."""
name: str name: str = Field(min_length=1, max_length=64, pattern=_SKILL_NAME_PATTERN)
description: str description: str = Field(min_length=1, max_length=_MAX_SKILL_DESCRIPTION_LENGTH)
entry_path: str # path of SKILL.md inside the archive entry_path: str # path of SKILL.md inside the archive
files: list[str] # all (safe) file paths inside the archive files: list[str] # all (safe) file paths inside the archive
size: int # total uncompressed bytes size: int # total uncompressed bytes
hash: str # sha256 of the archive bytes hash: str # sha256 of the archive bytes
@field_validator("name", "description", mode="before")
@classmethod
def _strip_required_string(cls, value: object) -> object:
return value.strip() if isinstance(value, str) else value
class NormalizedSkillPackage(BaseModel): class NormalizedSkillPackage(BaseModel):
"""Canonical skill package bytes and metadata ready to store in agent drive.""" """Canonical skill package bytes and metadata ready to store in agent drive."""
@@ -108,14 +113,17 @@ class SkillPackageService:
normalized_size = sum(max(info.file_size, 0) for info in normalized_members.values()) normalized_size = sum(max(info.file_size, 0) for info in normalized_members.values())
name, description = self._parse_skill_md(skill_md) name, description = self._parse_skill_md(skill_md)
manifest = SkillManifest( try:
name=name, manifest = SkillManifest(
description=description, name=name,
entry_path=_SKILL_MD_NAME, description=description,
files=sorted(normalized_members), entry_path=_SKILL_MD_NAME,
size=normalized_size, files=sorted(normalized_members),
hash=hashlib.sha256(normalized_archive_bytes).hexdigest(), size=normalized_size,
) hash=hashlib.sha256(normalized_archive_bytes).hexdigest(),
)
except ValidationError as exc:
raise self._manifest_validation_error(exc) from exc
return NormalizedSkillPackage( return NormalizedSkillPackage(
manifest=manifest, manifest=manifest,
archive_bytes=normalized_archive_bytes, archive_bytes=normalized_archive_bytes,
@@ -123,6 +131,31 @@ class SkillPackageService:
strip_prefix=strip_prefix, strip_prefix=strip_prefix,
) )
@staticmethod
def _manifest_validation_error(exc: ValidationError) -> SkillPackageError:
first_error = exc.errors()[0]
loc = first_error["loc"]
field = loc[0] if loc else "manifest"
error_type = first_error["type"]
if field == "name":
code = "missing_skill_name" if error_type == "string_too_short" else "invalid_skill_name"
message = (
"SKILL.md frontmatter name is required"
if code == "missing_skill_name"
else "SKILL.md frontmatter name must be lowercase letters, numbers, and hyphens only, "
"must not start or end with a hyphen, and must be at most 64 characters"
)
return SkillPackageError(code, message, status_code=400)
if field == "description":
code = "missing_skill_description" if error_type == "string_too_short" else "invalid_skill_description"
message = (
"SKILL.md frontmatter description is required"
if code == "missing_skill_description"
else f"SKILL.md frontmatter description must be at most {_MAX_SKILL_DESCRIPTION_LENGTH} characters"
)
return SkillPackageError(code, message, status_code=400)
return SkillPackageError("invalid_skill_manifest", "SKILL.md frontmatter is invalid", status_code=400)
def _open_archive(self, *, content: bytes, filename: str) -> zipfile.ZipFile: def _open_archive(self, *, content: bytes, filename: str) -> zipfile.ZipFile:
self._check_extension(filename) self._check_extension(filename)
if not content: if not content:
@@ -280,13 +313,6 @@ class SkillPackageService:
frontmatter = cls._parse_frontmatter(content) frontmatter = cls._parse_frontmatter(content)
name = str(frontmatter.get("name") or "").strip() name = str(frontmatter.get("name") or "").strip()
description = str(frontmatter.get("description") or "").strip() description = str(frontmatter.get("description") or "").strip()
if not name:
heading = _HEADING_RE.search(content)
name = heading.group(1).strip() if heading else ""
if not name:
raise SkillPackageError(
"missing_skill_name", "SKILL.md must declare a name (frontmatter or top heading)", status_code=400
)
return name, description return name, description
@staticmethod @staticmethod
+81 -12
View File
@@ -46,6 +46,7 @@ from models.tools import ToolFile
from services.agent.config_skill_normalize_service import ConfigSkillNormalizeService from services.agent.config_skill_normalize_service import ConfigSkillNormalizeService
from services.agent.skill_package_service import SkillPackageError from services.agent.skill_package_service import SkillPackageError
from services.agent_drive_service import DriveFileRef from services.agent_drive_service import DriveFileRef
from services.skill_management_service import SkillManagementService, SkillManagementServiceError
class AgentConfigVersionKind(StrEnum): class AgentConfigVersionKind(StrEnum):
@@ -98,6 +99,7 @@ class ConfigPushPayload(BaseModel):
@dataclass(slots=True) @dataclass(slots=True)
class AgentConfigTarget: class AgentConfigTarget:
tenant_id: str
agent_id: str agent_id: str
version_id: str version_id: str
kind: AgentConfigVersionKind kind: AgentConfigVersionKind
@@ -146,6 +148,7 @@ class AgentConfigService:
user_id=user_id, user_id=user_id,
) )
return AgentConfigTarget( return AgentConfigTarget(
tenant_id=tenant_id,
agent_id=target.agent_id, agent_id=target.agent_id,
version_id=target.version_id, version_id=target.version_id,
kind=target.kind, kind=target.kind,
@@ -191,7 +194,7 @@ class AgentConfigService:
return { return {
"agent_id": target.agent_id, "agent_id": target.agent_id,
"config_version": self._config_version_payload(target), "config_version": self._config_version_payload(target),
"items": [self._serialize_skill_item(skill) for skill in target.agent_soul.config_skills], "items": self._skill_items_for_target(target),
} }
def list_files( def list_files(
@@ -233,10 +236,27 @@ class AgentConfigService:
config_version_kind=config_version_kind, config_version_kind=config_version_kind,
user_id=user_id, user_id=user_id,
) )
skill = self._require_skill(target.agent_soul, name=name) try:
file_id = self._available_skill_file_id(skill) skill = self._require_skill(target.agent_soul, name=name)
payload, mime_type = self._load_tool_file_bytes(tenant_id=tenant_id, file_id=file_id) file_id = self._available_skill_file_id(skill)
return ConfigDownload(filename=f"{skill.name}.zip", mime_type=mime_type or "application/zip", payload=payload) payload, mime_type = self._load_tool_file_bytes(tenant_id=tenant_id, file_id=file_id)
return ConfigDownload(
filename=f"{skill.name}.zip",
mime_type=mime_type or "application/zip",
payload=payload,
)
except AgentConfigServiceError as exc:
if exc.code != "config_skill_not_found":
raise
try:
result = SkillManagementService().pull_runtime_agent_skill(
tenant_id=tenant_id,
agent_id=agent_id,
name=name,
)
return ConfigDownload(filename=result.filename, mime_type=result.mime_type, payload=result.payload)
except SkillManagementServiceError as exc:
raise AgentConfigServiceError("config_skill_not_found", "config skill not found", status_code=404) from exc
def download_skill_url( def download_skill_url(
self, self,
@@ -279,9 +299,45 @@ class AgentConfigService:
config_version_kind=config_version_kind, config_version_kind=config_version_kind,
user_id=user_id, user_id=user_id,
) )
skill = self._require_skill(target.agent_soul, name=name) try:
file_id = self._available_skill_file_id(skill) skill = self._require_skill(target.agent_soul, name=name)
archive_bytes, _mime_type = self._load_tool_file_bytes(tenant_id=tenant_id, file_id=file_id) file_id = self._available_skill_file_id(skill)
archive_bytes, _mime_type = self._load_tool_file_bytes(tenant_id=tenant_id, file_id=file_id)
skill_item = self._serialize_skill_item(skill)
except AgentConfigServiceError as exc:
if exc.code != "config_skill_not_found":
raise
try:
workspace_archive = SkillManagementService().pull_runtime_agent_skill(
tenant_id=tenant_id,
agent_id=agent_id,
name=name,
)
except SkillManagementServiceError as skill_exc:
raise AgentConfigServiceError(
"config_skill_not_found",
"config skill not found",
status_code=404,
) from skill_exc
archive_bytes = workspace_archive.payload
skill_item = next(
(
item
for item in SkillManagementService().list_runtime_agent_skills(
tenant_id=tenant_id,
agent_id=agent_id,
)
if item["name"] == name
),
{
"id": name,
"name": name,
"description": "",
"size": None,
"hash": None,
"mime_type": "application/zip",
},
)
try: try:
archive_items, skill_md = self._inspect_skill_archive(archive_bytes) archive_items, skill_md = self._inspect_skill_archive(archive_bytes)
except (OSError, ValueError, zipfile.BadZipFile) as exc: except (OSError, ValueError, zipfile.BadZipFile) as exc:
@@ -291,7 +347,7 @@ class AgentConfigService:
status_code=500, status_code=500,
) from exc ) from exc
return { return {
**self._serialize_skill_item(skill), **skill_item,
"source": "config_skill_zip", "source": "config_skill_zip",
"files": archive_items, "files": archive_items,
"skill_md": skill_md, "skill_md": skill_md,
@@ -839,6 +895,7 @@ class AgentConfigService:
status_code=404, status_code=404,
) )
return AgentConfigTarget( return AgentConfigTarget(
tenant_id=tenant_id,
agent_id=agent_id, agent_id=agent_id,
version_id=version.id, version_id=version.id,
kind=config_version_kind, kind=config_version_kind,
@@ -1133,9 +1190,7 @@ class AgentConfigService:
return { return {
"agent_id": target.agent_id, "agent_id": target.agent_id,
"config_version": AgentConfigService._config_version_payload(target), "config_version": AgentConfigService._config_version_payload(target),
"skills": { "skills": {"items": AgentConfigService._skill_items_for_target(target)},
"items": [AgentConfigService._serialize_skill_item(skill) for skill in target.agent_soul.config_skills]
},
"files": { "files": {
"items": [ "items": [
AgentConfigService._serialize_file_item(file_ref) for file_ref in target.agent_soul.config_files AgentConfigService._serialize_file_item(file_ref) for file_ref in target.agent_soul.config_files
@@ -1145,6 +1200,20 @@ class AgentConfigService:
"note": target.agent_soul.config_note, "note": target.agent_soul.config_note,
} }
@staticmethod
def _skill_items_for_target(target: AgentConfigTarget) -> list[dict[str, object]]:
items = [AgentConfigService._serialize_skill_item(skill) for skill in target.agent_soul.config_skills]
seen_names = {str(item["name"]) for item in items}
for item in SkillManagementService().list_runtime_agent_skills(
tenant_id=target.tenant_id,
agent_id=target.agent_id,
):
if item["name"] in seen_names:
continue
seen_names.add(str(item["name"]))
items.append(item)
return items
@staticmethod @staticmethod
def _config_version_payload(target: AgentConfigTarget) -> dict[str, object]: def _config_version_payload(target: AgentConfigTarget) -> dict[str, object]:
return { return {
+84
View File
@@ -1,6 +1,7 @@
import json import json
import logging import logging
from collections.abc import Sequence from collections.abc import Sequence
from dataclasses import dataclass
from datetime import datetime from datetime import datetime
from typing import Any, Literal, NotRequired, TypedDict, cast, override from typing import Any, Literal, NotRequired, TypedDict, cast, override
@@ -41,6 +42,20 @@ from tasks.remove_app_and_related_data_task import remove_app_and_related_data_t
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
AppListSortBy = Literal["last_modified", "recently_created", "earliest_created"] AppListSortBy = Literal["last_modified", "recently_created", "earliest_created"]
RecentAppMode = Literal[
AppMode.COMPLETION,
AppMode.WORKFLOW,
AppMode.CHAT,
AppMode.ADVANCED_CHAT,
AppMode.AGENT_CHAT,
]
RECENT_APP_MODES: tuple[RecentAppMode, ...] = (
AppMode.COMPLETION,
AppMode.WORKFLOW,
AppMode.CHAT,
AppMode.ADVANCED_CHAT,
AppMode.AGENT_CHAT,
)
class AppListBaseParams(BaseModel): class AppListBaseParams(BaseModel):
@@ -65,6 +80,19 @@ class StarredAppListParams(AppListBaseParams):
pass pass
@dataclass(frozen=True)
class RecentAppListItem:
id: str
name: str
icon_type: IconType | None
icon: str | None
icon_background: str | None
mode: RecentAppMode
author_name: str | None
updated_at: datetime
maintainer: str | None
class CreateAppParams(BaseModel): class CreateAppParams(BaseModel):
name: str = Field(min_length=1) name: str = Field(min_length=1)
description: str | None = None description: str | None = None
@@ -323,6 +351,62 @@ class AppService:
return app_models return app_models
def get_recent_apps(
self,
user_id: str,
tenant_id: str,
params: AppListParams,
session: Session,
) -> list[RecentAppListItem]:
"""Return recently modified apps as one lightweight, non-paginated projection."""
filters = self._build_app_list_filters(user_id, tenant_id, params, session)
if not filters:
return []
stmt = (
sa.select(
App.id,
App.name,
App.icon_type,
App.icon,
App.icon_background,
App.mode,
Account.name.label("author_name"),
App.updated_at,
App.maintainer,
)
.outerjoin(Account, Account.id == App.created_by)
.where(*filters, App.mode.in_(RECENT_APP_MODES))
.order_by(App.updated_at.desc())
.limit(params.limit)
)
rows = session.execute(stmt).all()
return [
RecentAppListItem(
id=str(app_id),
name=name,
icon_type=icon_type,
icon=icon,
icon_background=icon_background,
mode=cast(RecentAppMode, mode),
author_name=author_name,
updated_at=updated_at,
maintainer=maintainer,
)
for (
app_id,
name,
icon_type,
icon,
icon_background,
mode,
author_name,
updated_at,
maintainer,
) in rows
]
def get_paginate_starred_apps( def get_paginate_starred_apps(
self, self,
user_id: str, user_id: str,
+48 -9
View File
@@ -1,6 +1,8 @@
import logging
from collections.abc import Mapping
from enum import StrEnum from enum import StrEnum
from pydantic import BaseModel, ConfigDict, Field from pydantic import BaseModel, ConfigDict, Field, ValidationError
from configs import dify_config from configs import dify_config
from constants.dsl_version import CURRENT_APP_DSL_VERSION from constants.dsl_version import CURRENT_APP_DSL_VERSION
@@ -10,6 +12,8 @@ from enums.hosted_provider import HostedTrialProvider
from services.billing_service import BillingInfo, BillingService from services.billing_service import BillingInfo, BillingService
from services.enterprise.enterprise_service import EnterpriseService from services.enterprise.enterprise_service import EnterpriseService
logger = logging.getLogger(__name__)
class FeatureResponseModel(BaseModel): class FeatureResponseModel(BaseModel):
model_config = ConfigDict(json_schema_serialization_defaults_required=True, protected_namespaces=()) model_config = ConfigDict(json_schema_serialization_defaults_required=True, protected_namespaces=())
@@ -131,6 +135,13 @@ class PluginInstallationPermissionModel(FeatureResponseModel):
restrict_to_marketplace_only: bool = False restrict_to_marketplace_only: bool = False
class _EnterprisePluginInstallationPermission(BaseModel):
model_config = ConfigDict(extra="ignore")
plugin_installation_scope: PluginInstallationScope = Field(alias="pluginInstallationScope")
restrict_to_marketplace_only: bool = Field(alias="restrictToMarketplaceOnly", strict=True)
class FeatureModel(FeatureResponseModel): class FeatureModel(FeatureResponseModel):
billing: BillingModel = BillingModel() billing: BillingModel = BillingModel()
education: EducationModel = EducationModel() education: EducationModel = EducationModel()
@@ -285,6 +296,14 @@ class FeatureService:
"""Return whether Enterprise plugin credential policies must be enforced.""" """Return whether Enterprise plugin credential policies must be enforced."""
return dify_config.ENTERPRISE_ENABLED return dify_config.ENTERPRISE_ENABLED
@classmethod
def get_plugin_installation_permission(cls) -> PluginInstallationPermissionModel:
"""Resolve the validated deployment-wide plugin installation policy."""
if not dify_config.ENTERPRISE_ENABLED:
return PluginInstallationPermissionModel()
return cls._resolve_plugin_installation_permission(EnterpriseService.get_info())
@classmethod @classmethod
def get_license(cls) -> LicenseModel: def get_license(cls) -> LicenseModel:
"""Return full license detail. Enterprise-only; requires an authenticated caller. """Return full license detail. Enterprise-only; requires an authenticated caller.
@@ -452,6 +471,33 @@ class FeatureService:
) )
return license_model return license_model
@classmethod
def _resolve_plugin_installation_permission(
cls, enterprise_info: Mapping[str, object]
) -> PluginInstallationPermissionModel:
if "PluginInstallationPermission" not in enterprise_info:
return PluginInstallationPermissionModel()
try:
permission = _EnterprisePluginInstallationPermission.model_validate(
enterprise_info["PluginInstallationPermission"]
)
except ValidationError as exc:
# Do not attach the exception because it may contain raw Enterprise configuration values.
logger.error( # noqa: TRY400
"Invalid Enterprise plugin installation permission; denying all plugin installations: %s",
exc.errors(include_input=False),
)
return PluginInstallationPermissionModel(
plugin_installation_scope=PluginInstallationScope.NONE,
restrict_to_marketplace_only=True,
)
return PluginInstallationPermissionModel(
plugin_installation_scope=permission.plugin_installation_scope,
restrict_to_marketplace_only=permission.restrict_to_marketplace_only,
)
@classmethod @classmethod
def _fulfill_params_from_enterprise(cls, features: SystemFeatureModel): def _fulfill_params_from_enterprise(cls, features: SystemFeatureModel):
enterprise_info = EnterpriseService.get_info() enterprise_info = EnterpriseService.get_info()
@@ -499,11 +545,4 @@ class FeatureService:
status=LicenseStatus(license_info.get("status", LicenseStatus.INACTIVE)) status=LicenseStatus(license_info.get("status", LicenseStatus.INACTIVE))
) )
if "PluginInstallationPermission" in enterprise_info: features.plugin_installation_permission = cls._resolve_plugin_installation_permission(enterprise_info)
plugin_installation_info = enterprise_info["PluginInstallationPermission"]
features.plugin_installation_permission.plugin_installation_scope = plugin_installation_info[
"pluginInstallationScope"
]
features.plugin_installation_permission.restrict_to_marketplace_only = plugin_installation_info[
"restrictToMarketplaceOnly"
]
File diff suppressed because it is too large Load Diff
+9
View File
@@ -12,6 +12,7 @@ from werkzeug.exceptions import NotFound
from models.dataset import Dataset from models.dataset import Dataset
from models.enums import TagType from models.enums import TagType
from models.model import App, Tag, TagBinding from models.model import App, Tag, TagBinding
from models.skill import Skill
from models.snippet import CustomizedSnippet from models.snippet import CustomizedSnippet
type _TagTypeLike = TagType | str type _TagTypeLike = TagType | str
@@ -282,5 +283,13 @@ class TagService:
) )
if not snippet: if not snippet:
raise NotFound("Snippet not found") raise NotFound("Snippet not found")
elif type == "skill":
skill = session.scalar(
select(Skill)
.where(Skill.tenant_id == current_user.current_tenant_id, Skill.id == target_id)
.limit(1)
)
if not skill:
raise NotFound("Skill not found")
else: else:
raise NotFound("Invalid binding type") raise NotFound("Invalid binding type")
+165
View File
@@ -0,0 +1,165 @@
import logging
import platform
import uuid
from datetime import datetime
from typing import Literal
import httpx
from sqlalchemy import select
from sqlalchemy.orm import Session
from configs import dify_config
from libs.datetime_utils import naive_utc_now
from models.model import DifySetup
logger = logging.getLogger(__name__)
TelemetryEvent = Literal["install", "heartbeat"]
SCHEMA_VERSION = 1
class CommunityTelemetryService:
@classmethod
def report_install(cls, *, session: Session) -> bool:
setup = cls._get_setup(session)
if setup is None:
return False
if setup.instance_id is None:
setup.instance_id = str(uuid.uuid4())
session.add(setup)
session.commit()
payload = cls._build_payload(setup, "install")
if not cls._send_event(payload):
return False
setup.install_reported_at = naive_utc_now()
session.add(setup)
session.commit()
return True
@classmethod
def report_heartbeat(cls, *, session: Session, now: datetime | None = None) -> bool:
setup = cls._get_setup(session)
if setup is None:
return False
if setup.instance_id is None:
setup.instance_id = str(uuid.uuid4())
session.add(setup)
session.commit()
now = now or naive_utc_now()
if not cls._is_heartbeat_due(setup, now):
return False
if setup.install_reported_at is None:
cls.report_install(session=session)
payload = cls._build_payload(setup, "heartbeat")
if not cls._send_event(payload):
return False
setup.last_heartbeat_at = now
session.add(setup)
session.commit()
return True
@classmethod
def _get_setup(cls, session: Session) -> DifySetup | None:
return session.scalar(select(DifySetup).order_by(DifySetup.setup_at.asc()).limit(1))
@classmethod
def _is_enabled(cls) -> bool:
return (
dify_config.EDITION == "SELF_HOSTED"
and not dify_config.ENTERPRISE_ENABLED
and not dify_config.DISABLE_TELEMETRY
and not dify_config.DO_NOT_TRACK
and not dify_config.CI
and bool(dify_config.TELEMETRY_ENDPOINT)
)
@classmethod
def _build_payload(cls, setup: DifySetup, event: TelemetryEvent) -> dict[str, str | int]:
payload: dict[str, str | int] = {
"event": event,
"instance_id": setup.instance_id or "",
"version": setup.version if event == "install" else dify_config.project.version,
"edition": dify_config.EDITION,
"deployment_type": "unknown",
"schema_version": SCHEMA_VERSION,
"os": cls._normalize_os(platform.system()),
"arch": cls._normalize_arch(platform.machine()),
"sent_at": cls._format_datetime(naive_utc_now()),
}
if event == "install":
payload["installed_at"] = cls._format_datetime(setup.setup_at)
return payload
@classmethod
def _send_event(cls, payload: dict[str, str | int]) -> bool:
if not cls._is_enabled():
return False
endpoints = [dify_config.TELEMETRY_ENDPOINT]
if dify_config.TELEMETRY_FALLBACK_ENDPOINT not in endpoints:
endpoints.append(dify_config.TELEMETRY_FALLBACK_ENDPOINT)
for endpoint in endpoints:
if not endpoint:
continue
try:
response = httpx.post(
endpoint,
json=payload,
timeout=dify_config.TELEMETRY_TIMEOUT_SECONDS,
)
response.raise_for_status()
return True
except httpx.RequestError:
logger.debug("Failed to send community telemetry event to %s", endpoint, exc_info=True)
except httpx.HTTPStatusError:
logger.debug("Community telemetry endpoint returned an error: %s", endpoint, exc_info=True)
return False
return False
@classmethod
def _is_heartbeat_due(cls, setup: DifySetup, now: datetime) -> bool:
if setup.instance_id is None:
return False
if setup.last_heartbeat_at is not None and setup.last_heartbeat_at.date() >= now.date():
return False
return True
@staticmethod
def _format_datetime(value: datetime) -> str:
return value.replace(microsecond=0).isoformat() + "Z"
@staticmethod
def _normalize_os(value: str) -> str:
os_name = value.lower()
if os_name in {"linux", "darwin", "windows"}:
return os_name
return "unknown"
@staticmethod
def _normalize_arch(value: str) -> str:
arch = value.lower()
if arch in {"x86_64", "amd64"}:
return "amd64"
if arch in {"aarch64", "arm64"}:
return "arm64"
if arch.startswith("arm"):
return "arm"
if arch in {"i386", "i686", "x86"}:
return "386"
return "unknown"
+9 -9
View File
@@ -278,14 +278,14 @@ class VariableTruncator(BaseTruncator):
target_length = self._array_element_limit target_length = self._array_element_limit
for i, item in enumerate(value): for i, item in enumerate(value):
# Dirty fix: # ``File`` is routed through ``_truncate_json_primitives`` (whose
# The output of `Start` node may contain list of `File` elements, # dedicated ``File`` branch returns the file as-is with its real
# causing `AssertionError` while invoking `_truncate_json_primitives`. # serialized size). That preserves the count cap
# # (``array_element_limit``) and the byte budget (``target_size``)
# This check ensures that `list[File]` are handled separately # for ``list[File]`` — the original "Dirty fix" branch above this
if isinstance(item, File): # loop bypassed both guarantees and reported ``used_size=2`` even
truncated_value.append(item) # when the returned array serialized to well over the budget.
continue # See https://github.com/langgenius/dify/issues/39218.
if i >= target_length: if i >= target_length:
return _PartResult(truncated_value, used_size, True) return _PartResult(truncated_value, used_size, True)
if i > 0: if i > 0:
@@ -295,7 +295,7 @@ class VariableTruncator(BaseTruncator):
break break
remaining_budget = target_size - used_size remaining_budget = target_size - used_size
if item is None or isinstance(item, (str, list, dict, bool, int, float, UpdatedVariable)): if item is None or isinstance(item, (str, list, dict, bool, int, float, File, UpdatedVariable)):
part_result = self._truncate_json_primitives(item, remaining_budget) part_result = self._truncate_json_primitives(item, remaining_budget)
else: else:
raise UnknownTypeError(f"got unknown type {type(item)} in array truncation") raise UnknownTypeError(f"got unknown type {type(item)} in array truncation")
+1 -1
View File
@@ -116,7 +116,7 @@ class WebAppAuthService:
@classmethod @classmethod
def _get_account_jwt_token(cls, account: Account) -> str: def _get_account_jwt_token(cls, account: Account) -> str:
exp_dt = datetime.now(UTC) + timedelta(minutes=dify_config.ACCESS_TOKEN_EXPIRE_MINUTES * 24) exp_dt = datetime.now(UTC) + timedelta(minutes=dify_config.ACCESS_TOKEN_EXPIRE_MINUTES)
exp = int(exp_dt.timestamp()) exp = int(exp_dt.timestamp())
payload = { payload = {
+1 -1
View File
@@ -27,7 +27,6 @@ class WorkspaceService:
tenant_info: dict[str, object] = { tenant_info: dict[str, object] = {
"id": tenant.id, "id": tenant.id,
"name": tenant.name, "name": tenant.name,
"plan": tenant.plan,
"status": tenant.status, "status": tenant.status,
"created_at": tenant.created_at, "created_at": tenant.created_at,
"trial_end_reason": None, "trial_end_reason": None,
@@ -44,6 +43,7 @@ class WorkspaceService:
tenant_info["role"] = tenant_account_join.role tenant_info["role"] = tenant_account_join.role
feature = FeatureService.get_features(tenant.id, exclude_vector_space=True) feature = FeatureService.get_features(tenant.id, exclude_vector_space=True)
tenant_info["plan"] = feature.billing.subscription.plan if feature.billing.enabled else None
can_replace_logo = feature.can_replace_logo can_replace_logo = feature.can_replace_logo
if can_replace_logo and TenantService.has_roles( if can_replace_logo and TenantService.has_roles(
@@ -22,6 +22,7 @@ def _create_agent_backend_client():
return None return None
return create_agent_backend_run_client( return create_agent_backend_run_client(
base_url=dify_config.AGENT_BACKEND_BASE_URL, base_url=dify_config.AGENT_BACKEND_BASE_URL,
api_token=dify_config.AGENT_BACKEND_API_TOKEN,
use_fake=dify_config.AGENT_BACKEND_USE_FAKE, use_fake=dify_config.AGENT_BACKEND_USE_FAKE,
fake_scenario=dify_config.AGENT_BACKEND_FAKE_SCENARIO, fake_scenario=dify_config.AGENT_BACKEND_FAKE_SCENARIO,
stream_read_timeout_seconds=dify_config.AGENT_BACKEND_STREAM_READ_TIMEOUT_SECONDS, stream_read_timeout_seconds=dify_config.AGENT_BACKEND_STREAM_READ_TIMEOUT_SECONDS,
@@ -457,7 +457,7 @@ def _publish_streaming_response(
@shared_task(queue=WORKFLOW_BASED_APP_EXECUTION_QUEUE) @shared_task(queue=WORKFLOW_BASED_APP_EXECUTION_QUEUE)
def workflow_based_app_execution_task( def workflow_based_app_execution_task(
payload: str, payload: str,
) -> Generator[Mapping[str, Any] | str, None, None] | Mapping[str, Any] | None: ) -> Mapping[str, Any] | None:
exec_params = AppExecutionParams.model_validate_json(payload) exec_params = AppExecutionParams.model_validate_json(payload)
logger.info("workflow_based_app_execution_task run with params: %s", exec_params) logger.info("workflow_based_app_execution_task run with params: %s", exec_params)
+19
View File
@@ -0,0 +1,19 @@
import logging
from celery import shared_task
from sqlalchemy.orm import sessionmaker
from extensions.ext_database import db
from services.telemetry_service import CommunityTelemetryService
logger = logging.getLogger(__name__)
@shared_task(name="community_telemetry.send_heartbeat", queue="schedule_executor")
def send_community_telemetry_heartbeat() -> None:
session_factory = sessionmaker(bind=db.engine, expire_on_commit=False)
with session_factory() as session:
try:
CommunityTelemetryService.report_heartbeat(session=session)
except Exception:
logger.debug("Failed to process community telemetry heartbeat", exc_info=True)
@@ -17,10 +17,11 @@ class _SimpleJinja2Renderer:
"""Minimal Jinja2-based renderer for integration tests (no code executor).""" """Minimal Jinja2-based renderer for integration tests (no code executor)."""
def render_template(self, template: str, variables: dict[str, object]) -> str: def render_template(self, template: str, variables: dict[str, object]) -> str:
from jinja2 import Template from jinja2.sandbox import SandboxedEnvironment
try: try:
return Template(template).render(**variables) env = SandboxedEnvironment()
return env.from_string(template).render(**variables)
except Exception as exc: except Exception as exc:
raise TemplateRenderError(str(exc)) from exc raise TemplateRenderError(str(exc)) from exc
@@ -1,494 +1,85 @@
"""Testcontainers integration tests for controllers.console.datasets.data_source endpoints.""" """Integration coverage for Notion page bindings backed by persisted documents."""
from __future__ import annotations from inspect import unwrap
from unittest.mock import MagicMock, patch
import inspect
from collections.abc import Iterator
from datetime import UTC, datetime
from unittest.mock import MagicMock, PropertyMock, patch
from uuid import uuid4 from uuid import uuid4
import pytest
from flask import Flask from flask import Flask
from sqlalchemy.orm import Session from sqlalchemy.orm import Session
from werkzeug.exceptions import NotFound
from controllers.console.datasets import data_source from controllers.console.datasets.data_source import DataSourceNotionListApi
from controllers.console.datasets.data_source import ( from models import Account
DataSourceApi,
DataSourceNotionDatasetSyncApi,
DataSourceNotionDocumentSyncApi,
DataSourceNotionIndexingEstimateApi,
DataSourceNotionListApi,
DataSourceNotionPreviewApi,
)
from core.rag.index_processor.constant.index_type import IndexStructureType
from models import Account, DataSourceOauthBinding
from models.dataset import Document from models.dataset import Document
from models.enums import DataSourceType, DocumentCreatedFrom, IndexingStatus from models.enums import DataSourceType, DocumentCreatedFrom, IndexingStatus
@pytest.fixture def test_notion_page_is_marked_bound_from_persisted_document(
def current_user() -> Account: flask_app_with_containers: Flask,
account = Account(name="Test User", email="[email protected]") db_session_with_containers: Session,
account.id = "u1" ) -> None:
return account tenant_id = str(uuid4())
dataset_id = str(uuid4())
account = Account(name="Test User", email="[email protected]")
account.id = str(uuid4())
document = Document(
tenant_id=tenant_id,
dataset_id=dataset_id,
position=1,
data_source_type=DataSourceType.NOTION_IMPORT,
data_source_info='{"notion_page_id": "page-1"}',
batch=f"batch-{uuid4()}",
name="Notion Page",
created_from=DocumentCreatedFrom.WEB,
created_by=str(uuid4()),
indexing_status=IndexingStatus.COMPLETED,
enabled=True,
)
db_session_with_containers.add(document)
db_session_with_containers.commit()
runtime = MagicMock(
get_online_document_pages=lambda **_kwargs: iter(
[
MagicMock(
result=[
MagicMock(
workspace_id="workspace-1",
workspace_name="Workspace",
workspace_icon=None,
pages=[
MagicMock(
page_id="page-1",
page_name="Page",
type="page",
parent_id="parent",
page_icon=None,
)
],
)
]
)
]
),
datasource_provider_type=lambda: None,
)
with (
@pytest.fixture flask_app_with_containers.test_request_context(f"/?credential_id=c1&dataset_id={dataset_id}"),
def mock_engine() -> Iterator[None]: patch(
with patch.object( "controllers.console.datasets.data_source.DatasourceProviderService.get_datasource_credentials",
type(data_source.db), return_value={"token": "token"},
"engine", ),
new_callable=PropertyMock, patch(
return_value=MagicMock(), "controllers.console.datasets.data_source.DatasetService.get_dataset",
return_value=MagicMock(data_source_type="notion_import"),
),
patch(
"core.datasource.datasource_manager.DatasourceManager.get_datasource_runtime",
return_value=runtime,
),
): ):
yield response, status = unwrap(DataSourceNotionListApi().get)(
DataSourceNotionListApi(), db_session_with_containers, tenant_id, account
class TestDataSourceApi:
@pytest.fixture
def app(self, flask_app_with_containers: Flask) -> Flask:
return flask_app_with_containers
def test_get_success(self, app: Flask) -> None:
api = DataSourceApi()
method = inspect.unwrap(api.get)
binding = DataSourceOauthBinding(
tenant_id="tenant-1",
access_token="token",
provider="notion",
source_info={
"workspace_name": "Workspace",
"workspace_id": "workspace-1",
"workspace_icon": None,
"total": 1,
"pages": [
{
"page_id": "page-1",
"page_name": "Page",
"page_icon": {"type": "emoji", "emoji": "P", "url": None},
"parent_id": "parent-1",
"type": "page",
}
],
},
)
binding.id = "b1"
binding.created_at = datetime(2026, 5, 25, 1, 2, 3, tzinfo=UTC)
binding.disabled = False
with (
app.test_request_context("/"),
patch(
"controllers.console.datasets.data_source.db.session.scalars",
return_value=MagicMock(all=lambda: [binding]),
),
):
response, status = method(api, "tenant-1")
assert status == 200
assert response["data"][0] == {
"id": "b1",
"provider": "notion",
"created_at": 1779670923,
"is_bound": True,
"disabled": False,
"source_info": {
"workspace_name": "Workspace",
"workspace_id": "workspace-1",
"workspace_icon": None,
"pages": [
{
"page_name": "Page",
"page_id": "page-1",
"page_icon": {"type": "emoji", "url": None, "emoji": "P"},
"parent_id": "parent-1",
"type": "page",
}
],
"total": 1,
},
"link": "http://localhost/console/api/oauth/data-source/notion",
}
def test_get_no_bindings(self, app: Flask) -> None:
api = DataSourceApi()
method = inspect.unwrap(api.get)
with (
app.test_request_context("/"),
patch(
"controllers.console.datasets.data_source.db.session.scalars",
return_value=MagicMock(all=lambda: []),
),
):
response, status = method(api, "tenant-1")
assert status == 200
assert response["data"] == []
def test_patch_enable_binding(self, app: Flask) -> None:
api = DataSourceApi()
method = inspect.unwrap(api.patch)
binding = MagicMock(id="b1", disabled=True)
session = MagicMock()
session.scalar.return_value = binding
with app.test_request_context("/"):
response, status = method(api, session, "tenant-1", "b1", "enable")
assert status == 200
assert binding.disabled is False
def test_patch_disable_binding(self, app: Flask) -> None:
api = DataSourceApi()
method = inspect.unwrap(api.patch)
binding = MagicMock(id="b1", disabled=False)
session = MagicMock()
session.scalar.return_value = binding
with app.test_request_context("/"):
response, status = method(api, session, "tenant-1", "b1", "disable")
assert status == 200
assert binding.disabled is True
def test_patch_binding_not_found(self, app: Flask) -> None:
api = DataSourceApi()
method = inspect.unwrap(api.patch)
session = MagicMock()
session.scalar.return_value = None
with app.test_request_context("/"):
with pytest.raises(NotFound):
method(api, session, "tenant-1", "b1", "enable")
def test_patch_enable_already_enabled(self, app: Flask) -> None:
api = DataSourceApi()
method = inspect.unwrap(api.patch)
binding = MagicMock(id="b1", disabled=False)
session = MagicMock()
session.scalar.return_value = binding
with app.test_request_context("/"):
with pytest.raises(ValueError):
method(api, session, "tenant-1", "b1", "enable")
def test_patch_disable_already_disabled(self, app: Flask) -> None:
api = DataSourceApi()
method = inspect.unwrap(api.patch)
binding = MagicMock(id="b1", disabled=True)
session = MagicMock()
session.scalar.return_value = binding
with app.test_request_context("/"):
with pytest.raises(ValueError):
method(api, session, "tenant-1", "b1", "disable")
class TestDataSourceNotionListApi:
@pytest.fixture
def app(self, flask_app_with_containers: Flask) -> Flask:
return flask_app_with_containers
def test_get_credential_not_found(self, app: Flask, current_user: Account) -> None:
api = DataSourceNotionListApi()
method = inspect.unwrap(api.get)
with (
app.test_request_context("/?credential_id=c1"),
patch(
"controllers.console.datasets.data_source.DatasourceProviderService.get_datasource_credentials",
return_value=None,
),
):
with pytest.raises(NotFound):
method(api, MagicMock(), "tenant-1", current_user)
def test_get_success_no_dataset_id(self, app: Flask, current_user: Account, mock_engine: None) -> None:
api = DataSourceNotionListApi()
method = inspect.unwrap(api.get)
page = MagicMock(
page_id="p1",
page_name="Page 1",
type="page",
parent_id="parent",
page_icon=None,
) )
online_document_message = MagicMock( assert status == 200
result=[ assert response["notion_info"][0]["pages"][0]["is_bound"] is True
MagicMock(
workspace_id="w1",
workspace_name="My Workspace",
workspace_icon="icon",
pages=[page],
)
]
)
with (
app.test_request_context("/?credential_id=c1"),
patch(
"controllers.console.datasets.data_source.DatasourceProviderService.get_datasource_credentials",
return_value={"token": "t"},
),
patch(
"core.datasource.datasource_manager.DatasourceManager.get_datasource_runtime",
return_value=MagicMock(
get_online_document_pages=lambda **kw: iter([online_document_message]),
datasource_provider_type=lambda: None,
),
),
):
response, status = method(api, MagicMock(), "tenant-1", current_user)
assert status == 200
def test_get_success_with_dataset_id(
self, app: Flask, current_user: Account, mock_engine: None, db_session_with_containers: Session
) -> None:
api = DataSourceNotionListApi()
method = inspect.unwrap(api.get)
tenant_id = str(uuid4())
dataset_id = str(uuid4())
page = MagicMock(
page_id="p1",
page_name="Page 1",
type="page",
parent_id="parent",
page_icon=None,
)
online_document_message = MagicMock(
result=[
MagicMock(
workspace_id="w1",
workspace_name="My Workspace",
workspace_icon="icon",
pages=[page],
)
]
)
dataset = MagicMock(data_source_type="notion_import")
document = Document(
tenant_id=tenant_id,
dataset_id=dataset_id,
position=1,
data_source_type=DataSourceType.NOTION_IMPORT,
data_source_info='{"notion_page_id": "p1"}',
batch=f"batch-{uuid4()}",
name="Notion Page",
created_from=DocumentCreatedFrom.WEB,
created_by=str(uuid4()),
indexing_status=IndexingStatus.COMPLETED,
enabled=True,
)
db_session_with_containers.add(document)
db_session_with_containers.commit()
with (
app.test_request_context(f"/?credential_id=c1&dataset_id={dataset_id}"),
patch(
"controllers.console.datasets.data_source.DatasourceProviderService.get_datasource_credentials",
return_value={"token": "t"},
),
patch(
"controllers.console.datasets.data_source.DatasetService.get_dataset",
return_value=dataset,
),
patch(
"core.datasource.datasource_manager.DatasourceManager.get_datasource_runtime",
return_value=MagicMock(
get_online_document_pages=lambda **kw: iter([online_document_message]),
datasource_provider_type=lambda: None,
),
),
):
response, status = method(api, db_session_with_containers, tenant_id, current_user)
assert status == 200
def test_get_invalid_dataset_type(self, app: Flask, current_user: Account, mock_engine: None) -> None:
api = DataSourceNotionListApi()
method = inspect.unwrap(api.get)
dataset = MagicMock(data_source_type="other_type")
with (
app.test_request_context("/?credential_id=c1&dataset_id=ds1"),
patch(
"controllers.console.datasets.data_source.DatasourceProviderService.get_datasource_credentials",
return_value={"token": "t"},
),
patch(
"controllers.console.datasets.data_source.DatasetService.get_dataset",
return_value=dataset,
),
):
with pytest.raises(ValueError):
method(api, MagicMock(), "tenant-1", current_user)
class TestDataSourceNotionPreviewApi:
@pytest.fixture
def app(self, flask_app_with_containers: Flask) -> Flask:
return flask_app_with_containers
def test_get_preview_success(self, app: Flask) -> None:
api = DataSourceNotionPreviewApi()
method = inspect.unwrap(api.get)
extractor = MagicMock(extract=lambda: [MagicMock(page_content="hello")])
with (
app.test_request_context("/?credential_id=c1"),
patch(
"controllers.console.datasets.data_source.DatasourceProviderService.get_datasource_credentials",
return_value={"integration_secret": "t"},
),
patch(
"controllers.console.datasets.data_source.NotionExtractor",
return_value=extractor,
),
):
response, status = method(api, "tenant-1", "p1", "page")
assert status == 200
class TestDataSourceNotionIndexingEstimateApi:
@pytest.fixture
def app(self, flask_app_with_containers: Flask) -> Flask:
return flask_app_with_containers
def test_post_indexing_estimate_success(self, app: Flask) -> None:
api = DataSourceNotionIndexingEstimateApi()
method = inspect.unwrap(api.post)
empty_rules: dict[str, object] = {}
payload: dict[str, object] = {
"notion_info_list": [
{
"workspace_id": "w1",
"credential_id": "c1",
"pages": [{"page_id": "p1", "type": "page"}],
}
],
"process_rule": {"rules": empty_rules},
"doc_form": IndexStructureType.PARAGRAPH_INDEX,
"doc_language": "English",
}
with (
app.test_request_context("/", method="POST", json=payload, headers={"Content-Type": "application/json"}),
patch(
"controllers.console.datasets.data_source.DocumentService.estimate_args_validate",
),
patch(
"controllers.console.datasets.data_source.IndexingRunner.indexing_estimate",
return_value=MagicMock(model_dump=lambda: {"total_pages": 1}),
),
):
response, status = method(api, MagicMock(), "tenant-1")
assert status == 200
class TestDataSourceNotionDatasetSyncApi:
@pytest.fixture
def app(self, flask_app_with_containers: Flask) -> Flask:
return flask_app_with_containers
def test_get_success(self, app: Flask) -> None:
api = DataSourceNotionDatasetSyncApi()
method = inspect.unwrap(api.get)
with (
app.test_request_context("/"),
patch(
"controllers.console.datasets.data_source.DatasetService.get_dataset",
return_value=MagicMock(),
),
patch(
"controllers.console.datasets.data_source.DocumentService.get_document_by_dataset_id",
return_value=[MagicMock(id="d1")],
),
patch(
"controllers.console.datasets.data_source.document_indexing_sync_task.delay",
return_value=None,
),
):
response, status = method(api, MagicMock(), "ds-1")
assert status == 200
def test_get_dataset_not_found(self, app: Flask) -> None:
api = DataSourceNotionDatasetSyncApi()
method = inspect.unwrap(api.get)
with (
app.test_request_context("/"),
patch(
"controllers.console.datasets.data_source.DatasetService.get_dataset",
return_value=None,
),
):
with pytest.raises(NotFound):
method(api, MagicMock(), "ds-1")
class TestDataSourceNotionDocumentSyncApi:
@pytest.fixture
def app(self, flask_app_with_containers: Flask) -> Flask:
return flask_app_with_containers
def test_get_success(self, app: Flask) -> None:
api = DataSourceNotionDocumentSyncApi()
method = inspect.unwrap(api.get)
with (
app.test_request_context("/"),
patch(
"controllers.console.datasets.data_source.DatasetService.get_dataset",
return_value=MagicMock(),
),
patch(
"controllers.console.datasets.data_source.DocumentService.get_document",
return_value=MagicMock(),
),
patch(
"controllers.console.datasets.data_source.document_indexing_sync_task.delay",
return_value=None,
),
):
response, status = method(api, MagicMock(), "ds-1", "doc-1")
assert status == 200
def test_get_document_not_found(self, app: Flask) -> None:
api = DataSourceNotionDocumentSyncApi()
method = inspect.unwrap(api.get)
with (
app.test_request_context("/"),
patch(
"controllers.console.datasets.data_source.DatasetService.get_dataset",
return_value=MagicMock(),
),
patch(
"controllers.console.datasets.data_source.DocumentService.get_document",
return_value=None,
),
):
with pytest.raises(NotFound):
method(api, MagicMock(), "ds-1", "doc-1")
@@ -97,6 +97,7 @@ class TestAppSiteApi:
assert result["end_user_id"] == end_user.id assert result["end_user_id"] == end_user.id
assert result["plan"] == "basic" assert result["plan"] == "basic"
assert result["enable_site"] is True assert result["enable_site"] is True
assert result["mode"] == AppMode.CHAT
@patch("controllers.web.site.FileService.get_file_presigned_url") @patch("controllers.web.site.FileService.get_file_presigned_url")
@patch("controllers.web.site.FeatureService.get_features") @patch("controllers.web.site.FeatureService.get_features")
@@ -178,6 +179,7 @@ class TestWebAppSiteResponse:
response = WebAppSiteResponse.from_app_site( response = WebAppSiteResponse.from_app_site(
tenant=tenant, tenant=tenant,
app_model=app_model, app_model=app_model,
mode=AppMode.CHAT,
site=_site_model(app_id=app_model.id), site=_site_model(app_id=app_model.id),
end_user_id="eu-1", end_user_id="eu-1",
features=FeatureModel(can_replace_logo=False, webapp_copyright_enabled=True), features=FeatureModel(can_replace_logo=False, webapp_copyright_enabled=True),
@@ -185,6 +187,7 @@ class TestWebAppSiteResponse:
) )
assert response.app_id == app_model.id assert response.app_id == app_model.id
assert response.mode == AppMode.CHAT
assert response.end_user_id == "eu-1" assert response.end_user_id == "eu-1"
assert response.enable_site is True assert response.enable_site is True
assert response.plan == "basic" assert response.plan == "basic"
@@ -209,6 +212,7 @@ class TestWebAppSiteResponse:
response = WebAppSiteResponse.from_app_site( response = WebAppSiteResponse.from_app_site(
tenant=tenant, tenant=tenant,
app_model=app_model, app_model=app_model,
mode=AppMode.CHAT,
site=site, site=site,
end_user_id=None, end_user_id=None,
features=FeatureModel(can_replace_logo=False, webapp_copyright_enabled=True), features=FeatureModel(can_replace_logo=False, webapp_copyright_enabled=True),
@@ -236,6 +240,7 @@ class TestWebAppSiteResponse:
response = WebAppSiteResponse.from_app_site( response = WebAppSiteResponse.from_app_site(
tenant=tenant, tenant=tenant,
app_model=app_model, app_model=app_model,
mode=AppMode.CHAT,
site=_site_model(app_id=app_model.id), site=_site_model(app_id=app_model.id),
end_user_id="eu-1", end_user_id="eu-1",
features=FeatureModel(can_replace_logo=True, webapp_copyright_enabled=True), features=FeatureModel(can_replace_logo=True, webapp_copyright_enabled=True),
@@ -4,7 +4,7 @@ import datetime
import json import json
import uuid import uuid
from decimal import Decimal from decimal import Decimal
from unittest.mock import MagicMock, patch from unittest.mock import patch
import pytest import pytest
from faker import Faker from faker import Faker
@@ -1172,65 +1172,8 @@ class TestMessagesCleanServiceIntegration:
# Verify all messages were deleted # Verify all messages were deleted
assert db_session_with_containers.query(Message).where(Message.id.in_(msg_ids)).count() == 0 assert db_session_with_containers.query(Message).where(Message.id.in_(msg_ids)).count() == 0
def test_from_time_range_validation(self):
"""Test that from_time_range raises ValueError for invalid inputs."""
policy = MagicMock(spec=BillingDisabledPolicy)
now = datetime.datetime.now()
with pytest.raises(ValueError, match="start_from .* must be less than end_before"):
MessagesCleanService.from_time_range(policy, now, now)
with pytest.raises(ValueError, match="batch_size .* must be greater than 0"):
MessagesCleanService.from_time_range(policy, now - datetime.timedelta(days=1), now, batch_size=0)
def test_from_time_range_success(self):
"""Test that from_time_range creates a service with correct parameters."""
policy = MagicMock(spec=BillingDisabledPolicy)
start = datetime.datetime(2024, 1, 1)
end = datetime.datetime(2024, 2, 1)
service = MessagesCleanService.from_time_range(policy, start, end)
assert service._start_from == start
assert service._end_before == end
def test_from_days_validation(self):
"""Test that from_days raises ValueError for invalid inputs."""
policy = MagicMock(spec=BillingDisabledPolicy)
with pytest.raises(ValueError, match="days .* must be greater than or equal to 0"):
MessagesCleanService.from_days(policy, days=-1)
with pytest.raises(ValueError, match="batch_size .* must be greater than 0"):
MessagesCleanService.from_days(policy, days=30, batch_size=0)
def test_from_days_success(self):
"""Test that from_days creates a service with correct parameters."""
policy = MagicMock(spec=BillingDisabledPolicy)
with patch("services.retention.conversation.messages_clean_service.naive_utc_now") as mock_now:
fixed_now = datetime.datetime(2024, 6, 1)
mock_now.return_value = fixed_now
service = MessagesCleanService.from_days(policy, days=10)
assert service._start_from is None
assert service._end_before == fixed_now - datetime.timedelta(days=10)
def test_batch_delete_message_relations_empty(self, db_session_with_containers: Session): def test_batch_delete_message_relations_empty(self, db_session_with_containers: Session):
"""Test that batch_delete_message_relations with empty list does nothing.""" """Test that batch_delete_message_relations with empty list does nothing."""
# Get execute call count before # Get execute call count before
MessagesCleanService._batch_delete_message_relations(db_session_with_containers, []) MessagesCleanService._batch_delete_message_relations(db_session_with_containers, [])
# No exception means success — empty list is a no-op # No exception means success — empty list is a no-op
def test_run_calls_clean_messages(self):
"""Test that run() delegates to _clean_messages_by_time_range."""
policy = MagicMock(spec=BillingDisabledPolicy)
service = MessagesCleanService(
policy=policy,
end_before=datetime.datetime.now(),
batch_size=10,
)
with patch.object(service, "_clean_messages_by_time_range") as mock_clean:
mock_clean.return_value = {"total_deleted": 5}
result = service.run()
assert result == {"total_deleted": 5}
mock_clean.assert_called_once()
@@ -3,7 +3,6 @@ from __future__ import annotations
import json import json
import uuid import uuid
from datetime import UTC, datetime, timedelta from datetime import UTC, datetime, timedelta
from types import SimpleNamespace
from unittest.mock import patch from unittest.mock import patch
import pytest import pytest
@@ -12,13 +11,13 @@ from sqlalchemy.orm import Session
from graphon.enums import WorkflowExecutionStatus from graphon.enums import WorkflowExecutionStatus
from models import EndUser, Workflow, WorkflowAppLog, WorkflowArchiveLog, WorkflowRun from models import EndUser, Workflow, WorkflowAppLog, WorkflowArchiveLog, WorkflowRun
from models.enums import AppTriggerType, CreatorUserRole, EndUserType, WorkflowRunTriggeredFrom from models.enums import CreatorUserRole, EndUserType, WorkflowRunTriggeredFrom
from models.workflow import WorkflowAppLogCreatedFrom from models.workflow import WorkflowAppLogCreatedFrom
from services.account_service import AccountService, TenantService from services.account_service import AccountService, TenantService
# Delay import of AppService to avoid circular dependency # Delay import of AppService to avoid circular dependency
# from services.app_service import AppService, CreateAppParams # from services.app_service import AppService, CreateAppParams
from services.workflow_app_service import LogView, WorkflowAppService from services.workflow_app_service import WorkflowAppService
from tests.test_containers_integration_tests.helpers import generate_valid_password from tests.test_containers_integration_tests.helpers import generate_valid_password
@@ -1627,73 +1626,3 @@ class TestWorkflowAppService:
end_user_item = next(d for d in result["data"] if d["created_by_end_user"] is not None) end_user_item = next(d for d in result["data"] if d["created_by_end_user"] is not None)
assert account_item["created_by_account"].id == account.id assert account_item["created_by_account"].id == account.id
assert end_user_item["created_by_end_user"].id == end_user.id assert end_user_item["created_by_end_user"].id == end_user.id
class TestLogView:
def test_details_and_proxy_attributes(self):
log = SimpleNamespace(id="log-1", status="succeeded")
view = LogView(log=log, details={"trigger_metadata": {"type": "plugin"}})
assert view.details == {"trigger_metadata": {"type": "plugin"}}
assert view.status == "succeeded"
class TestHandleTriggerMetadata:
def test_returns_empty_dict_when_metadata_missing(self):
service = WorkflowAppService()
assert service.handle_trigger_metadata("tenant-1", None) == {}
def test_enriches_plugin_icons(self):
service = WorkflowAppService()
meta = {
"type": AppTriggerType.TRIGGER_PLUGIN.value,
"icon_filename": "light.png",
"icon_dark_filename": "dark.png",
}
with patch(
"services.workflow_app_service.PluginService.get_plugin_icon_url",
side_effect=["https://cdn/light.png", "https://cdn/dark.png"],
) as mock_icon:
result = service.handle_trigger_metadata("tenant-1", json.dumps(meta))
assert result["icon"] == "https://cdn/light.png"
assert result["icon_dark"] == "https://cdn/dark.png"
assert mock_icon.call_count == 2
def test_non_plugin_metadata_without_icon_lookup(self):
service = WorkflowAppService()
meta = {"type": AppTriggerType.TRIGGER_WEBHOOK.value}
with patch("services.workflow_app_service.PluginService.get_plugin_icon_url") as mock_icon:
result = service.handle_trigger_metadata("tenant-1", json.dumps(meta))
assert result["type"] == AppTriggerType.TRIGGER_WEBHOOK.value
mock_icon.assert_not_called()
class TestSafeJsonLoads:
@pytest.mark.parametrize(
("value", "expected"),
[
(None, None),
("", None),
('{"k":"v"}', {"k": "v"}),
("not-json", None),
({"raw": True}, {"raw": True}),
],
)
def test_handles_various_inputs(self, value, expected):
assert WorkflowAppService._safe_json_loads(value) == expected
class TestSafeParseUuid:
def test_returns_none_for_short_or_invalid_values(self):
service = WorkflowAppService()
assert service._safe_parse_uuid("short") is None
assert service._safe_parse_uuid("x" * 40) is None
def test_returns_uuid_for_valid_string(self):
service = WorkflowAppService()
raw = str(uuid.uuid4())
result = service._safe_parse_uuid(raw)
assert result is not None
assert str(result) == raw
@@ -24,7 +24,10 @@ class TestWorkspaceService:
patch("services.workspace_service.dify_config") as mock_dify_config, patch("services.workspace_service.dify_config") as mock_dify_config,
): ):
# Setup default mock returns # Setup default mock returns
mock_feature_service.get_features.return_value.can_replace_logo = True feature = mock_feature_service.get_features.return_value
feature.can_replace_logo = True
feature.billing.enabled = True
feature.billing.subscription.plan = "professional"
mock_tenant_service.has_roles.return_value = True mock_tenant_service.has_roles.return_value = True
mock_dify_config.FILES_URL = "https://example.com/files" mock_dify_config.FILES_URL = "https://example.com/files"
@@ -112,7 +115,7 @@ class TestWorkspaceService:
assert result is not None assert result is not None
assert result["id"] == tenant.id assert result["id"] == tenant.id
assert result["name"] == tenant.name assert result["name"] == tenant.name
assert result["plan"] == tenant.plan assert result["plan"] == "professional"
assert result["status"] == tenant.status assert result["status"] == tenant.status
assert result["role"] == TenantAccountRole.OWNER assert result["role"] == TenantAccountRole.OWNER
assert result["created_at"] == tenant.created_at assert result["created_at"] == tenant.created_at
@@ -159,7 +162,7 @@ class TestWorkspaceService:
assert result is not None assert result is not None
assert result["id"] == tenant.id assert result["id"] == tenant.id
assert result["name"] == tenant.name assert result["name"] == tenant.name
assert result["plan"] == tenant.plan assert result["plan"] == "professional"
assert result["status"] == tenant.status assert result["status"] == tenant.status
assert result["role"] == TenantAccountRole.OWNER assert result["role"] == TenantAccountRole.OWNER
assert result["created_at"] == tenant.created_at assert result["created_at"] == tenant.created_at
@@ -214,7 +217,7 @@ class TestWorkspaceService:
assert result is not None assert result is not None
assert result["id"] == tenant.id assert result["id"] == tenant.id
assert result["name"] == tenant.name assert result["name"] == tenant.name
assert result["plan"] == tenant.plan assert result["plan"] == "professional"
assert result["status"] == tenant.status assert result["status"] == tenant.status
assert result["role"] == TenantAccountRole.NORMAL assert result["role"] == TenantAccountRole.NORMAL
assert result["created_at"] == tenant.created_at assert result["created_at"] == tenant.created_at
@@ -606,20 +609,23 @@ class TestWorkspaceService:
def test_get_tenant_info_should_not_include_cloud_fields_in_self_hosted( def test_get_tenant_info_should_not_include_cloud_fields_in_self_hosted(
self, db_session_with_containers: Session, mock_external_service_dependencies self, db_session_with_containers: Session, mock_external_service_dependencies
): ):
"""next_credit_reset_date and trial_credits should NOT appear in SELF_HOSTED mode.""" """Cloud-only billing data should not appear in SELF_HOSTED mode."""
fake = Faker() fake = Faker()
account, tenant = self._create_test_account_and_tenant( account, tenant = self._create_test_account_and_tenant(
db_session_with_containers, mock_external_service_dependencies db_session_with_containers, mock_external_service_dependencies
) )
mock_external_service_dependencies["dify_config"].DEPLOYMENT_EDITION = DeploymentEdition.COMMUNITY mock_external_service_dependencies["dify_config"].DEPLOYMENT_EDITION = DeploymentEdition.COMMUNITY
mock_external_service_dependencies["feature_service"].get_features.return_value.can_replace_logo = False feature = mock_external_service_dependencies["feature_service"].get_features.return_value
feature.can_replace_logo = False
feature.billing.enabled = False
mock_external_service_dependencies["tenant_service"].has_roles.return_value = False mock_external_service_dependencies["tenant_service"].has_roles.return_value = False
with patch("services.workspace_service.current_user", account): with patch("services.workspace_service.current_user", account):
result = WorkspaceService.get_tenant_info(tenant, db_session_with_containers) result = WorkspaceService.get_tenant_info(tenant, db_session_with_containers)
assert result is not None assert result is not None
assert result["plan"] is None
assert "next_credit_reset_date" not in result assert "next_credit_reset_date" not in result
assert "trial_credits" not in result assert "trial_credits" not in result
assert "trial_credits_used" not in result assert "trial_credits_used" not in result
@@ -6,6 +6,7 @@ import json
import os import os
import threading import threading
import time import time
from collections.abc import Iterator
from datetime import datetime, timedelta from datetime import datetime, timedelta
from pathlib import Path from pathlib import Path
from types import SimpleNamespace from types import SimpleNamespace
@@ -15,9 +16,12 @@ import pytest
import sqlalchemy as sa import sqlalchemy as sa
from click.testing import CliRunner from click.testing import CliRunner
from sqlalchemy.exc import OperationalError from sqlalchemy.exc import OperationalError
from sqlalchemy.orm import Session, SessionTransaction, sessionmaker
from graphon.model_runtime.entities.model_entities import ModelType from graphon.model_runtime.entities.model_entities import ModelType
from models import Dataset, DatasetPermission, DatasetPermissionEnum
from models.account import Tenant from models.account import Tenant
from models.base import TypeBase
from models.enums import CredentialSourceType from models.enums import CredentialSourceType
from models.provider import ProviderModel from models.provider import ProviderModel
from tests.helpers.legacy_model_type_migration import ( from tests.helpers.legacy_model_type_migration import (
@@ -59,6 +63,40 @@ def command_module():
) )
@pytest.fixture
def rbac_session(sqlite_engine: sa.Engine, monkeypatch: pytest.MonkeyPatch) -> Iterator[Session]:
"""Bind RBAC command reads to persisted SQLite dataset rows."""
TypeBase.metadata.create_all(
sqlite_engine,
tables=[Dataset.__table__, DatasetPermission.__table__],
)
factory = sessionmaker(bind=sqlite_engine, expire_on_commit=False)
monkeypatch.setattr("commands.rbac.session_factory.create_session", factory)
with factory() as session:
yield session
def _persist_dataset(
session: Session,
*,
dataset_id: str = "dataset-1",
tenant_id: str = "tenant-1",
permission: DatasetPermissionEnum = DatasetPermissionEnum.ONLY_ME,
created_by: str = "creator-account-1",
) -> Dataset:
dataset = Dataset(
id=dataset_id,
tenant_id=tenant_id,
name=f"Dataset {dataset_id}",
permission=permission,
created_by=created_by,
)
session.add(dataset)
session.commit()
return dataset
def _parse_json_lines(output: io.StringIO) -> list[dict[str, object]]: def _parse_json_lines(output: io.StringIO) -> list[dict[str, object]]:
return [json.loads(line) for line in output.getvalue().splitlines() if line.strip()] return [json.loads(line) for line in output.getvalue().splitlines() if line.strip()]
@@ -363,56 +401,35 @@ def test_dataset_permission_rbac_migration_maps_legacy_permissions_to_enum_scope
def test_dataset_permission_rbac_migration_uses_dataset_creator_as_operator( def test_dataset_permission_rbac_migration_uses_dataset_creator_as_operator(
command_module, command_module,
rbac_session: Session,
monkeypatch: pytest.MonkeyPatch, monkeypatch: pytest.MonkeyPatch,
) -> None: ) -> None:
rbac_module = importlib.import_module("commands.rbac") rbac_module = importlib.import_module("commands.rbac")
dataset_row = SimpleNamespace( _persist_dataset(rbac_session)
id="dataset-1",
tenant_id="tenant-1",
permission="only_me",
created_by="creator-account-1",
)
execute_results = [[dataset_row], [], []]
calls: list[dict[str, object]] = [] calls: list[dict[str, object]] = []
session_closed = False read_transaction_ended = False
class FakeExecuteResult:
def __init__(self, rows: list[object]) -> None:
self._rows = rows
def all(self) -> list[object]:
return self._rows
class FakeSession:
def __enter__(self):
return self
def __exit__(self, exc_type, exc, traceback) -> None:
nonlocal session_closed
session_closed = True
pass
def execute(self, stmt):
return FakeExecuteResult(execute_results.pop(0))
class FakeSessionFactory:
@staticmethod
def create_session() -> FakeSession:
return FakeSession()
def fake_replace_whitelist(**kwargs): def fake_replace_whitelist(**kwargs):
assert session_closed is True assert read_transaction_ended is True
calls.append(kwargs) calls.append(kwargs)
monkeypatch.setattr(rbac_module, "session_factory", FakeSessionFactory) def _record_transaction_end(session: Session, transaction: object) -> None:
monkeypatch.setattr(rbac_module.RBACService.DatasetAccess, "replace_whitelist", fake_replace_whitelist) nonlocal read_transaction_ended
del transaction
if session.get_bind() is rbac_session.get_bind():
read_transaction_ended = True
command_module.migrate_dataset_permissions_to_rbac.callback( sa.event.listen(Session, "after_transaction_end", _record_transaction_end)
tenant_id=None, monkeypatch.setattr(rbac_module.RBACService.DatasetAccess, "replace_whitelist", fake_replace_whitelist)
dataset_id=None, try:
batch_size=500, command_module.migrate_dataset_permissions_to_rbac.callback(
dry_run=False, tenant_id=None,
) dataset_id=None,
batch_size=500,
dry_run=False,
)
finally:
sa.event.remove(Session, "after_transaction_end", _record_transaction_end)
assert calls[0]["tenant_id"] == "tenant-1" assert calls[0]["tenant_id"] == "tenant-1"
assert calls[0]["account_id"] == "creator-account-1" assert calls[0]["account_id"] == "creator-account-1"
@@ -422,41 +439,19 @@ def test_dataset_permission_rbac_migration_uses_dataset_creator_as_operator(
def test_dataset_permission_rbac_migration_dry_run_outputs_structured_proposed_changes( def test_dataset_permission_rbac_migration_dry_run_outputs_structured_proposed_changes(
command_module, command_module,
rbac_session: Session,
monkeypatch: pytest.MonkeyPatch, monkeypatch: pytest.MonkeyPatch,
) -> None: ) -> None:
rbac_module = importlib.import_module("commands.rbac") rbac_module = importlib.import_module("commands.rbac")
dataset_row = SimpleNamespace( dataset = _persist_dataset(rbac_session, permission=DatasetPermissionEnum.PARTIAL_TEAM)
id="dataset-1", rbac_session.add(
tenant_id="tenant-1", DatasetPermission(
permission="partial_members", dataset_id=dataset.id,
created_by="creator-account-1", account_id="member-account-1",
tenant_id=dataset.tenant_id,
)
) )
permission_row = SimpleNamespace(dataset_id="dataset-1", account_id="member-account-1") rbac_session.commit()
execute_results = [[dataset_row], [permission_row], []]
class FakeExecuteResult:
def __init__(self, rows: list[object]) -> None:
self._rows = rows
def all(self) -> list[object]:
return self._rows
class FakeSession:
def __enter__(self):
return self
def __exit__(self, exc_type, exc, traceback) -> None:
pass
def execute(self, stmt):
return FakeExecuteResult(execute_results.pop(0))
class FakeSessionFactory:
@staticmethod
def create_session() -> FakeSession:
return FakeSession()
monkeypatch.setattr(rbac_module, "session_factory", FakeSessionFactory)
monkeypatch.setattr( monkeypatch.setattr(
rbac_module.RBACService.DatasetAccess, rbac_module.RBACService.DatasetAccess,
"replace_whitelist", "replace_whitelist",
@@ -1306,50 +1301,36 @@ def test_provider_models_processing_uses_same_plan_locking_and_transaction_entry
begin_calls: list[str] = [] begin_calls: list[str] = []
configure_calls: list[str] = [] configure_calls: list[str] = []
class _FakeBeginContext: def _record_begin(session: Session, transaction: SessionTransaction) -> None:
def __init__(self, phase: str) -> None: if session.get_bind() is sqlite_engine and transaction.parent is None:
self._phase = phase begin_calls.append(current_phase["name"])
def __enter__(self) -> None: def _fake_build_plan(self, session: Session, candidate, *, lock_rows: bool):
begin_calls.append(self._phase) assert session.get_bind() is sqlite_engine
def __exit__(self, exc_type, exc, tb) -> bool:
return False
class _FakeSession:
def __init__(self, phase: str) -> None:
self._phase = phase
def __enter__(self):
return self
def __exit__(self, exc_type, exc, tb) -> bool:
return False
def begin(self) -> _FakeBeginContext:
return _FakeBeginContext(self._phase)
def _fake_session_factory(engine: sa.Engine) -> _FakeSession:
return _FakeSession(current_phase["name"])
def _fake_build_plan(self, session, candidate, *, lock_rows: bool):
lock_rows_seen.append((current_phase["name"], lock_rows)) lock_rows_seen.append((current_phase["name"], lock_rows))
return SimpleNamespace(group_row_ids=[str(candidate.row.id)], winner=None, loser_rows=[]) return migration_module._ProviderModelGroupPlan(
group_row_ids=[str(candidate.row.id)],
winner=None,
loser_rows=[],
)
def _fake_emit_plan(self, plan, *, session, tx_id: str, business_key: dict[str, object]) -> None: def _fake_emit_plan(self, plan, *, session, tx_id: str, business_key: dict[str, object]) -> None:
return None return None
def _fake_configure(self, session) -> None: def _fake_configure(self, session: Session) -> None:
assert session.get_bind() is sqlite_engine
configure_calls.append(current_phase["name"]) configure_calls.append(current_phase["name"])
monkeypatch.setattr(migration_module, "_session_factory", _fake_session_factory)
monkeypatch.setattr(migration_module.Migration, "_build_provider_model_group_plan", _fake_build_plan) monkeypatch.setattr(migration_module.Migration, "_build_provider_model_group_plan", _fake_build_plan)
monkeypatch.setattr(migration_module.Migration, "_emit_provider_model_group_plan", _fake_emit_plan) monkeypatch.setattr(migration_module.Migration, "_emit_provider_model_group_plan", _fake_emit_plan)
monkeypatch.setattr(migration_module.Migration, "_configure_lock_timeout", _fake_configure) monkeypatch.setattr(migration_module.Migration, "_configure_lock_timeout", _fake_configure)
sa.event.listen(Session, "after_transaction_create", _record_begin)
dry_migration._process_provider_model_group(candidate, business_key) try:
current_phase["name"] = "apply" dry_migration._process_provider_model_group(candidate, business_key)
apply_migration._process_provider_model_group(candidate, business_key) current_phase["name"] = "apply"
apply_migration._process_provider_model_group(candidate, business_key)
finally:
sa.event.remove(Session, "after_transaction_create", _record_begin)
assert [phase for phase, _ in lock_rows_seen] == ["dry", "apply"] assert [phase for phase, _ in lock_rows_seen] == ["dry", "apply"]
assert lock_rows_seen[0][1] == lock_rows_seen[1][1] assert lock_rows_seen[0][1] == lock_rows_seen[1][1]
@@ -1392,6 +1373,22 @@ def test_process_load_balancing_model_config_row_logs_stacktrace_for_lock_timeou
sqlite_engine: sa.Engine, sqlite_engine: sa.Engine,
monkeypatch: pytest.MonkeyPatch, monkeypatch: pytest.MonkeyPatch,
) -> None: ) -> None:
create_minimal_legacy_model_type_schema(sqlite_engine)
created_at = datetime(2025, 1, 1, 12, 0, 0)
_insert_load_balancing_model_config(
sqlite_engine,
row_id="40000000-0000-0000-0000-000000000001",
tenant_id="tenant-1",
provider_name="openai",
model_name="gpt-4o-mini",
model_type="text-generation",
name="credential",
encrypted_config="{}",
credential_id="50000000-0000-0000-0000-000000000001",
enabled=True,
created_at=created_at,
updated_at=created_at,
)
output = io.StringIO() output = io.StringIO()
migration = migration_module.Migration( migration = migration_module.Migration(
tenant_id="tenant-1", tenant_id="tenant-1",
@@ -1401,37 +1398,18 @@ def test_process_load_balancing_model_config_row_logs_stacktrace_for_lock_timeou
model_types=(ModelType.LLM,), model_types=(ModelType.LLM,),
orm_models=(migration_module.LoadBalancingModelConfig,), orm_models=(migration_module.LoadBalancingModelConfig,),
) )
candidate = migration_module._RowWithRawModelType( candidate = migration._load_load_balancing_model_config_candidates(None)[0]
row=SimpleNamespace(id="lb-row-1"),
raw_model_type="text-generation",
canonical_model_type=ModelType.LLM,
)
lock_timeout_exc = OperationalError("SELECT 1", {}, SimpleNamespace(pgcode="55P03")) lock_timeout_exc = OperationalError("SELECT 1", {}, SimpleNamespace(pgcode="55P03"))
transaction_begins = 0
class _FakeBeginContext: def _record_begin(session: Session, transaction: SessionTransaction) -> None:
def __enter__(self) -> None: nonlocal transaction_begins
return None if session.get_bind() is sqlite_engine and transaction.parent is None:
transaction_begins += 1
def __exit__(self, exc_type, exc, tb) -> bool:
return False
class _FakeSession:
def __enter__(self):
return self
def __exit__(self, exc_type, exc, tb) -> bool:
return False
def begin(self) -> _FakeBeginContext:
return _FakeBeginContext()
def _fake_session_factory(engine: sa.Engine) -> _FakeSession:
return _FakeSession()
def _fake_reload(self, session, original_candidate, *, lock_rows: bool): def _fake_reload(self, session, original_candidate, *, lock_rows: bool):
raise lock_timeout_exc raise lock_timeout_exc
monkeypatch.setattr(migration_module, "_session_factory", _fake_session_factory)
monkeypatch.setattr(migration_module.Migration, "_configure_lock_timeout", lambda self, session: None) monkeypatch.setattr(migration_module.Migration, "_configure_lock_timeout", lambda self, session: None)
monkeypatch.setattr( monkeypatch.setattr(
migration_module.Migration, migration_module.Migration,
@@ -1439,17 +1417,22 @@ def test_process_load_balancing_model_config_row_logs_stacktrace_for_lock_timeou
_fake_reload, _fake_reload,
) )
migration._process_load_balancing_model_config_row(candidate) sa.event.listen(Session, "after_transaction_create", _record_begin)
try:
migration._process_load_balancing_model_config_row(candidate)
finally:
sa.event.remove(Session, "after_transaction_create", _record_begin)
lines = _parse_json_lines(output) lines = _parse_json_lines(output)
assert len(lines) == 1 assert len(lines) == 1
assert lines[0]["event"] == "lock_timeout_skipped" assert lines[0]["event"] == "lock_timeout_skipped"
attrs = cast(dict[str, object], lines[0]["attrs"]) attrs = cast(dict[str, object], lines[0]["attrs"])
assert attrs["table_name"] == "load_balancing_model_configs" assert attrs["table_name"] == "load_balancing_model_configs"
assert attrs["id"] == "lb-row-1" assert attrs["id"] == str(candidate.row.id)
assert attrs["error"] == str(lock_timeout_exc) assert attrs["error"] == str(lock_timeout_exc)
assert isinstance(attrs["stacktrace"], str) assert isinstance(attrs["stacktrace"], str)
assert "OperationalError" in attrs["stacktrace"] assert "OperationalError" in attrs["stacktrace"]
assert transaction_begins == 1
def test_process_load_balancing_model_config_row_logs_update_after_sql_execution( def test_process_load_balancing_model_config_row_logs_update_after_sql_execution(
@@ -1457,6 +1440,23 @@ def test_process_load_balancing_model_config_row_logs_update_after_sql_execution
sqlite_engine: sa.Engine, sqlite_engine: sa.Engine,
monkeypatch: pytest.MonkeyPatch, monkeypatch: pytest.MonkeyPatch,
) -> None: ) -> None:
create_minimal_legacy_model_type_schema(sqlite_engine)
created_at = datetime(2025, 1, 1, 12, 0, 0)
row_id = "40000000-0000-0000-0000-000000000002"
_insert_load_balancing_model_config(
sqlite_engine,
row_id=row_id,
tenant_id="tenant-1",
provider_name="openai",
model_name="gpt-4o-mini",
model_type="text-generation",
name="credential",
encrypted_config="{}",
credential_id="50000000-0000-0000-0000-000000000002",
enabled=True,
created_at=created_at,
updated_at=created_at,
)
migration = migration_module.Migration( migration = migration_module.Migration(
tenant_id="tenant-1", tenant_id="tenant-1",
engine=sqlite_engine, engine=sqlite_engine,
@@ -1465,42 +1465,33 @@ def test_process_load_balancing_model_config_row_logs_update_after_sql_execution
model_types=(ModelType.LLM,), model_types=(ModelType.LLM,),
orm_models=(migration_module.LoadBalancingModelConfig,), orm_models=(migration_module.LoadBalancingModelConfig,),
) )
candidate = migration_module._RowWithRawModelType( candidate = migration._load_load_balancing_model_config_candidates(None)[0]
row=SimpleNamespace(id="lb-row-1"),
raw_model_type="text-generation",
canonical_model_type=ModelType.LLM,
)
action_log: list[str] = [] action_log: list[str] = []
class _FakeBeginContext: def _record_begin(session: Session, transaction: SessionTransaction) -> None:
def __enter__(self) -> None: if session.get_bind() is sqlite_engine and transaction.parent is None:
action_log.append("begin") action_log.append("begin")
def __exit__(self, exc_type, exc, tb) -> bool: def _record_sql(
return False connection: sa.Connection,
cursor: object,
class _FakeSession: statement: str,
def __enter__(self): parameters: object,
return self context: object,
executemany: bool,
def __exit__(self, exc_type, exc, tb) -> bool: ) -> None:
return False del connection, cursor, parameters, context, executemany
if statement.lstrip().upper().startswith("UPDATE"):
def begin(self) -> _FakeBeginContext:
return _FakeBeginContext()
def execute(self, stmt) -> None:
action_log.append("sql_execute") action_log.append("sql_execute")
def _fake_session_factory(engine: sa.Engine) -> _FakeSession:
return _FakeSession()
def _fake_configure(self, session) -> None: def _fake_configure(self, session) -> None:
action_log.append("configure_lock_timeout") action_log.append("configure_lock_timeout")
def _fake_reload(self, session, original_candidate, *, lock_rows: bool): original_reload = migration_module.Migration._reload_load_balancing_model_config_candidate
def _record_reload(self, session: Session, original_candidate, *, lock_rows: bool):
action_log.append(f"reload_candidate:{lock_rows}") action_log.append(f"reload_candidate:{lock_rows}")
return candidate return original_reload(self, session, original_candidate, lock_rows=lock_rows)
def _fake_log_row_updated(self, *args, **kwargs) -> None: def _fake_log_row_updated(self, *args, **kwargs) -> None:
action_log.append("log_row_updated") action_log.append("log_row_updated")
@@ -1508,12 +1499,11 @@ def test_process_load_balancing_model_config_row_logs_update_after_sql_execution
def _fake_cache_cleanup(self, *, row_id: str, tx_id: str) -> None: def _fake_cache_cleanup(self, *, row_id: str, tx_id: str) -> None:
action_log.append("cache_cleanup") action_log.append("cache_cleanup")
monkeypatch.setattr(migration_module, "_session_factory", _fake_session_factory)
monkeypatch.setattr(migration_module.Migration, "_configure_lock_timeout", _fake_configure) monkeypatch.setattr(migration_module.Migration, "_configure_lock_timeout", _fake_configure)
monkeypatch.setattr( monkeypatch.setattr(
migration_module.Migration, migration_module.Migration,
"_reload_load_balancing_model_config_candidate", "_reload_load_balancing_model_config_candidate",
_fake_reload, _record_reload,
) )
monkeypatch.setattr(migration_module.Migration, "_log_row_updated", _fake_log_row_updated) monkeypatch.setattr(migration_module.Migration, "_log_row_updated", _fake_log_row_updated)
monkeypatch.setattr( monkeypatch.setattr(
@@ -1522,7 +1512,13 @@ def test_process_load_balancing_model_config_row_logs_update_after_sql_execution
_fake_cache_cleanup, _fake_cache_cleanup,
) )
migration._process_load_balancing_model_config_row(candidate) sa.event.listen(Session, "after_transaction_create", _record_begin)
sa.event.listen(sqlite_engine, "before_cursor_execute", _record_sql)
try:
migration._process_load_balancing_model_config_row(candidate)
finally:
sa.event.remove(sqlite_engine, "before_cursor_execute", _record_sql)
sa.event.remove(Session, "after_transaction_create", _record_begin)
assert action_log == [ assert action_log == [
"begin", "begin",
@@ -1532,6 +1528,10 @@ def test_process_load_balancing_model_config_row_logs_update_after_sql_execution
"log_row_updated", "log_row_updated",
"cache_cleanup", "cache_cleanup",
] ]
with Session(sqlite_engine) as session:
persisted = session.get(migration_module.LoadBalancingModelConfig, row_id)
assert persisted is not None
assert persisted.model_type == ModelType.LLM
def test_load_balancing_model_config_cache_delete_failure_logs_stacktrace( def test_load_balancing_model_config_cache_delete_failure_logs_stacktrace(
@@ -4,6 +4,7 @@ from dotenv import dotenv_values
BASE_API_AND_DOCKER_CONFIG_SET_DIFF: frozenset[str] = frozenset( BASE_API_AND_DOCKER_CONFIG_SET_DIFF: frozenset[str] = frozenset(
( (
"AGENT_BACKEND_API_TOKEN",
"APP_MAX_EXECUTION_TIME", "APP_MAX_EXECUTION_TIME",
"BATCH_UPLOAD_LIMIT", "BATCH_UPLOAD_LIMIT",
"CELERY_BEAT_SCHEDULER_TIME", "CELERY_BEAT_SCHEDULER_TIME",
@@ -43,6 +44,7 @@ BASE_API_AND_DOCKER_CONFIG_SET_DIFF: frozenset[str] = frozenset(
BASE_API_AND_DOCKER_COMPOSE_CONFIG_SET_DIFF: frozenset[str] = frozenset( BASE_API_AND_DOCKER_COMPOSE_CONFIG_SET_DIFF: frozenset[str] = frozenset(
( (
"AGENT_BACKEND_API_TOKEN",
"BATCH_UPLOAD_LIMIT", "BATCH_UPLOAD_LIMIT",
"CELERY_BEAT_SCHEDULER_TIME", "CELERY_BEAT_SCHEDULER_TIME",
"HTTP_REQUEST_MAX_CONNECT_TIMEOUT", "HTTP_REQUEST_MAX_CONNECT_TIMEOUT",
+56 -26
View File
@@ -1,11 +1,13 @@
import os import os
import shutil
from collections.abc import Iterator from collections.abc import Iterator
from pathlib import Path
from unittest.mock import MagicMock, patch from unittest.mock import MagicMock, patch
import pytest import pytest
from flask import Flask from flask import Flask
from sqlalchemy import create_engine from sqlalchemy import create_engine
from sqlalchemy.engine import Engine from sqlalchemy.engine import URL, Engine
from sqlalchemy.orm import Session, sessionmaker from sqlalchemy.orm import Session, sessionmaker
# Getting the absolute path of the current file's directory # Getting the absolute path of the current file's directory
@@ -35,7 +37,7 @@ os.environ.setdefault("OPENDAL_SCHEME", "fs")
os.environ.setdefault("OPENDAL_FS_ROOT", "/tmp/dify-storage") os.environ.setdefault("OPENDAL_FS_ROOT", "/tmp/dify-storage")
os.environ.setdefault("STORAGE_TYPE", "opendal") os.environ.setdefault("STORAGE_TYPE", "opendal")
from core.db.session_factory import configure_session_factory, session_factory import core.db.session_factory as session_factory_module
from extensions import ext_redis from extensions import ext_redis
from models.account import Account, Tenant, TenantAccountJoin, TenantAccountRole from models.account import Account, Tenant, TenantAccountJoin, TenantAccountRole
from models.base import TypeBase from models.base import TypeBase
@@ -111,42 +113,70 @@ def reset_secret_key():
dify_config.SECRET_KEY = original dify_config.SECRET_KEY = original
@pytest.fixture(scope="session")
def _unit_test_engine():
engine = create_engine("sqlite:///:memory:")
yield engine
engine.dispose()
@pytest.fixture @pytest.fixture
def sqlite_engine() -> Iterator[Engine]: def _sqlite_engine(_sqlite_database_template: Path, tmp_path: Path) -> Iterator[Engine]:
"""Create an isolated in-memory SQLite engine for tests that need a disposable database.""" """Create an engine over a pristine per-test copy of the SQLite schema."""
database_path = tmp_path / "unit-tests.sqlite3"
shutil.copyfile(_sqlite_database_template, database_path)
engine = create_engine(URL.create("sqlite", database=str(database_path)))
engine = create_engine("sqlite:///:memory:")
try: try:
yield engine yield engine
finally: finally:
engine.dispose() engine.dispose()
database_path.unlink(missing_ok=True)
@pytest.fixture @pytest.fixture(scope="session")
def sqlite_session(request: pytest.FixtureRequest, sqlite_engine: Engine) -> Iterator[Session]: def _sqlite_database_template(tmp_path_factory: pytest.TempPathFactory) -> Path:
"""Yield a SQLite session after creating the model tables passed through ``request.param``.""" """Create one empty full-schema SQLite database per pytest worker."""
models: tuple[type[TypeBase], ...] = request.param database_path = tmp_path_factory.mktemp("sqlite-template") / "unit-tests.sqlite3"
tables = [model.metadata.tables[model.__tablename__] for model in models] engine = create_engine(URL.create("sqlite", database=str(database_path)))
TypeBase.metadata.create_all(sqlite_engine, tables=tables) try:
session_factory = sessionmaker(bind=sqlite_engine, expire_on_commit=False) TypeBase.metadata.create_all(engine)
with session_factory() as session: finally:
yield session engine.dispose()
return database_path
@pytest.fixture(autouse=True) @pytest.fixture(autouse=True)
def _configure_session_factory(_unit_test_engine): def _sqlite_session_factory(
try: _sqlite_engine: Engine,
session_factory.get_session_maker() monkeypatch: pytest.MonkeyPatch,
except RuntimeError: ) -> sessionmaker[Session]:
configure_session_factory(_unit_test_engine, expire_on_commit=False) """Bind all unit-test Sessions to the pristine full-schema SQLite database."""
factory = sessionmaker(bind=_sqlite_engine, expire_on_commit=False)
monkeypatch.setattr(session_factory_module, "_session_maker", factory)
return factory
@pytest.fixture
def sqlite_engine(_sqlite_engine: Engine) -> Engine:
"""Expose the pristine full-schema SQLite engine to tests."""
return _sqlite_engine
@pytest.fixture
def sqlite_session_factory(_sqlite_session_factory: sessionmaker[Session]) -> sessionmaker[Session]:
"""Expose the shared SQLite session factory to tests."""
return _sqlite_session_factory
@pytest.fixture
def sqlite_session(_sqlite_session_factory: sessionmaker[Session]) -> Iterator[Session]:
"""Yield a session over the pristine full-schema SQLite database.
Legacy indirect model parameters remain accepted by pytest but are ignored.
Remove those decorators as their test files receive individual review.
"""
with _sqlite_session_factory() as session:
yield session
def persist_service_api_tenant_owner(session: Session, tenant: Tenant, owner: Account) -> TenantAccountJoin: def persist_service_api_tenant_owner(session: Session, tenant: Tenant, owner: Account) -> TenantAccountJoin:
@@ -115,6 +115,7 @@ def _agent_app_composer_response() -> dict:
"active_config_snapshot_id": "version-1", "active_config_snapshot_id": "version-1",
}, },
"active_config_snapshot": _version_response(), "active_config_snapshot": _version_response(),
"active_config_is_published": True,
"agent_soul": {}, "agent_soul": {},
"save_options": ["save_to_current_version"], "save_options": ["save_to_current_version"],
} }
@@ -376,7 +377,7 @@ def test_agent_app_list_and_create_use_agent_route(
assert created["app_id"] == "app-created" assert created["app_id"] == "app-created"
assert created["debug_conversation_id"] == "debug-conversation-created" assert created["debug_conversation_id"] == "debug-conversation-created"
assert created["role"] == "Created role" assert created["role"] == "Created role"
assert created["active_config_is_published"] is False assert "active_config_is_published" not in created
assert "bound_agent_id" not in created assert "bound_agent_id" not in created
create_call = cast(dict[str, object], captured["create"]) create_call = cast(dict[str, object], captured["create"])
create_params = cast(Any, create_call["params"]) create_params = cast(Any, create_call["params"])
@@ -487,7 +488,7 @@ def test_agent_app_detail_update_delete_resolve_app_from_agent_id(
assert detail["debug_conversation_has_messages"] is True assert detail["debug_conversation_has_messages"] is True
assert detail["debug_conversation_message_count"] == 2 assert detail["debug_conversation_message_count"] == 2
assert detail["role"] == "Resolved role" assert detail["role"] == "Resolved role"
assert detail["active_config_is_published"] is False assert "active_config_is_published" not in detail
assert "bound_agent_id" not in detail assert "bound_agent_id" not in detail
assert captured["get_app"] == {"app": app_model, "session": session} assert captured["get_app"] == {"app": app_model, "session": session}
with app.test_request_context( with app.test_request_context(
@@ -502,7 +503,7 @@ def test_agent_app_detail_update_delete_resolve_app_from_agent_id(
assert updated["debug_conversation_has_messages"] is True assert updated["debug_conversation_has_messages"] is True
assert updated["debug_conversation_message_count"] == 2 assert updated["debug_conversation_message_count"] == 2
assert updated["role"] == "Resolved role" assert updated["role"] == "Resolved role"
assert updated["active_config_is_published"] is False assert "active_config_is_published" not in updated
assert "bound_agent_id" not in updated assert "bound_agent_id" not in updated
update_call = cast(dict[str, object], captured["update"]) update_call = cast(dict[str, object], captured["update"])
assert update_call["app"] is app_model assert update_call["app"] is app_model
@@ -845,9 +846,6 @@ def test_agent_app_update_allows_empty_role(app: Flask, monkeypatch: pytest.Monk
monkeypatch.setattr( monkeypatch.setattr(
roster_controller.AgentRosterService, "count_agent_app_debug_conversation_messages", lambda _self, **kwargs: 0 roster_controller.AgentRosterService, "count_agent_app_debug_conversation_messages", lambda _self, **kwargs: 0
) )
monkeypatch.setattr(
roster_controller.AgentRosterService, "active_config_is_published", lambda _self, **kwargs: False
)
monkeypatch.setattr( monkeypatch.setattr(
roster_controller.FeatureService, roster_controller.FeatureService,
"get_system_features", "get_system_features",
@@ -1299,13 +1297,14 @@ def test_agent_composer_routes_resolve_app_from_agent_id(
composer_controller.AgentComposerService, "collect_validation_findings", collect_validation_findings composer_controller.AgentComposerService, "collect_validation_findings", collect_validation_findings
) )
monkeypatch.setattr(composer_controller.AgentComposerService, "get_agent_app_candidates", get_agent_app_candidates) monkeypatch.setattr(composer_controller.AgentComposerService, "get_agent_app_candidates", get_agent_app_candidates)
assert unwrap(AgentComposerApi.get)(AgentComposerApi(), MagicMock(), "tenant-1", agent_id)["variant"] == "agent_app" composer = unwrap(AgentComposerApi.get)(AgentComposerApi(), MagicMock(), "tenant-1", agent_id)
assert composer["variant"] == "agent_app"
assert composer["active_config_is_published"] is True
assert cast(dict[str, object], captured["load"])["agent_id"] == agent_id assert cast(dict[str, object], captured["load"])["agent_id"] == agent_id
with app.test_request_context(json=payload): with app.test_request_context(json=payload):
assert ( saved_composer = unwrap(AgentComposerApi.put)(AgentComposerApi(), MagicMock(), "tenant-1", account_id, agent_id)
unwrap(AgentComposerApi.put)(AgentComposerApi(), MagicMock(), "tenant-1", account_id, agent_id)["variant"] assert saved_composer["variant"] == "agent_app"
== "agent_app" assert saved_composer["active_config_is_published"] is True
)
assert cast(dict[str, object], captured["save"])["agent_id"] == agent_id assert cast(dict[str, object], captured["save"])["agent_id"] == agent_id
assert unwrap(AgentComposerValidateApi.post)(AgentComposerValidateApi(), MagicMock(), "tenant-1", agent_id) == { assert unwrap(AgentComposerValidateApi.post)(AgentComposerValidateApi(), MagicMock(), "tenant-1", agent_id) == {
"result": "success", "result": "success",
@@ -708,6 +708,121 @@ def test_app_list_api_attaches_permission_keys(app, app_module):
assert resp["data"][0]["permission_keys"] == ["app.acl.view_layout", "app.acl.edit"] assert resp["data"][0]["permission_keys"] == ["app.acl.view_layout", "app.acl.edit"]
def test_recent_app_list_api_returns_only_home_card_fields(app, app_module):
method = app_module.RecentAppListApi.get
while hasattr(method, "__wrapped__"):
method = method.__wrapped__
recent_app = SimpleNamespace(
id="app-1",
name="Recent App",
icon_type="emoji",
icon="🚀",
icon_background="#FFFFFF",
mode="chat",
author_name="Recent Author",
updated_at=_ts(15),
maintainer="acct-1",
)
get_recent_apps = MagicMock(return_value=[recent_app])
with app.test_request_context("/apps/recent?limit=8"):
with pytest.MonkeyPatch.context() as monkeypatch:
monkeypatch.setattr(dify_config, "RBAC_ENABLED", False)
monkeypatch.setattr(app_module.AppService, "get_recent_apps", get_recent_apps)
monkeypatch.setattr(
app_module.enterprise_rbac_service.RBACService.MyPermissions,
"get",
lambda tenant_id, account_id, session: app_module.enterprise_rbac_service.MyPermissionsResponse(
app=app_module.enterprise_rbac_service.ResourcePermissionSnapshot(
overrides=[
app_module.enterprise_rbac_service.ResourcePermissionKeys(
resource_id="app-1",
permission_keys=["app.acl.monitor"],
)
]
)
),
)
resp, status = method(app_module.RecentAppListApi(), "tenant-1", "acct-1", MagicMock())
assert status == 200
assert resp == {
"data": [
{
"id": "app-1",
"name": "Recent App",
"icon_type": "emoji",
"icon": "🚀",
"icon_background": "#FFFFFF",
"mode": "chat",
"author_name": "Recent Author",
"updated_at": int(_ts(15).timestamp()),
"permission_keys": ["app.acl.monitor"],
"maintainer": "acct-1",
"icon_url": None,
}
]
}
params = get_recent_apps.call_args.args[2]
assert params.limit == 8
assert "total" not in resp
assert "description" not in resp["data"][0]
assert "tags" not in resp["data"][0]
assert "workflow" not in resp["data"][0]
@pytest.mark.parametrize("mode", ["channel", "rag-pipeline", "agent"])
def test_recent_app_response_rejects_non_home_app_modes(app_module, mode: str) -> None:
with pytest.raises(ValidationError):
app_module.RecentAppResponse.model_validate(
{
"id": "app-1",
"name": "Recent App",
"mode": mode,
"updated_at": _ts(),
}
)
def test_recent_app_list_api_applies_rbac_visibility_filter(app, app_module):
method = app_module.RecentAppListApi.get
while hasattr(method, "__wrapped__"):
method = method.__wrapped__
get_recent_apps = MagicMock(return_value=[])
with app.test_request_context("/apps/recent"):
with pytest.MonkeyPatch.context() as monkeypatch:
monkeypatch.setattr(dify_config, "RBAC_ENABLED", True)
monkeypatch.setattr(app_module.AppService, "get_recent_apps", get_recent_apps)
monkeypatch.setattr(
app_module.enterprise_rbac_service.RBACService.MyPermissions,
"get",
lambda tenant_id, account_id, session: app_module.enterprise_rbac_service.MyPermissionsResponse(
workspace=app_module.enterprise_rbac_service.WorkspacePermissionSnapshot(
permission_keys=["app.create_and_management"]
)
),
)
monkeypatch.setattr(
app_module.enterprise_rbac_service.RBACService.AppAccess,
"whitelist_resources",
lambda tenant_id, account_id: SimpleNamespace(
unrestricted=False,
resource_ids=["app-shared"],
),
)
resp, status = method(app_module.RecentAppListApi(), "tenant-1", "acct-1", MagicMock())
assert status == 200
assert resp == {"data": []}
params = get_recent_apps.call_args.args[2]
assert params.accessible_app_ids == ["app-shared"]
assert params.include_own_apps is True
def test_app_list_api_limits_to_apps_created_by_current_user_without_view_permission(app, app_module): def test_app_list_api_limits_to_apps_created_by_current_user_without_view_permission(app, app_module):
method = app_module.AppListApi.get method = app_module.AppListApi.get
while hasattr(method, "__wrapped__"): while hasattr(method, "__wrapped__"):
@@ -1,18 +1,22 @@
from __future__ import annotations from __future__ import annotations
import inspect import inspect
from collections.abc import Callable from collections.abc import Callable, Iterator
from datetime import UTC, datetime from datetime import UTC, datetime
from typing import cast from typing import Literal, cast
from unittest.mock import MagicMock, PropertyMock, patch from unittest.mock import MagicMock, PropertyMock, patch
from uuid import uuid4 from uuid import UUID
import pytest import pytest
from flask import Flask from flask import Flask
from sqlalchemy import select
from sqlalchemy.orm import Session
from werkzeug.exceptions import NotFound
from controllers.console.datasets import data_source as module from controllers.console.datasets import data_source as module
from controllers.console.datasets.data_source import DataSourceApi, DataSourceNotionListApi from controllers.console.datasets.data_source import DataSourceApi, DataSourceNotionListApi
from models import Account, DataSourceOauthBinding from models import Account, DataSourceOauthBinding
from models.engine import db
ControllerMethod = Callable[..., tuple[dict[str, object], int]] ControllerMethod = Callable[..., tuple[dict[str, object], int]]
@@ -22,10 +26,15 @@ def unwrap(func: object) -> ControllerMethod:
@pytest.fixture @pytest.fixture
def flask_app() -> Flask: def flask_app() -> Iterator[Flask]:
app = Flask(__name__) app = Flask(__name__)
app.config["TESTING"] = True app.config["TESTING"] = True
return app app.config["SQLALCHEMY_DATABASE_URI"] = "sqlite:///:memory:"
db.init_app(app)
with app.app_context():
DataSourceOauthBinding.__table__.create(db.engine)
yield app
@pytest.fixture @pytest.fixture
@@ -35,9 +44,13 @@ def current_user() -> Account:
return account return account
def test_get_data_source_integrates_serializes_orm_binding(flask_app: Flask) -> None: TENANT_ID = "11111111-1111-1111-1111-111111111111"
BINDING_ID = "22222222-2222-2222-2222-222222222222"
def _add_binding(session: Session, *, disabled: bool) -> DataSourceOauthBinding:
binding = DataSourceOauthBinding( binding = DataSourceOauthBinding(
tenant_id="tenant-1", tenant_id=TENANT_ID,
access_token="token", access_token="token",
provider="notion", provider="notion",
source_info={ source_info={
@@ -55,24 +68,31 @@ def test_get_data_source_integrates_serializes_orm_binding(flask_app: Flask) ->
} }
], ],
}, },
disabled=disabled,
) )
binding.id = "binding-1" binding.id = BINDING_ID
binding.created_at = datetime(2026, 5, 25, 1, 2, 3, tzinfo=UTC) binding.created_at = datetime(2026, 5, 25, 1, 2, 3, tzinfo=UTC)
binding.disabled = False session.add(binding)
session.commit()
return binding
with (
flask_app.test_request_context("/"), def test_get_data_source_integrates_serializes_orm_binding(
patch.object(module.db.session, "scalars", return_value=MagicMock(all=lambda: [binding])), flask_app: Flask,
): ) -> None:
response, status = unwrap(DataSourceApi().get)(DataSourceApi(), "tenant-1") binding = _add_binding(db.session, disabled=False)
expected_created_at = int(binding.created_at.timestamp())
with flask_app.test_request_context("/"):
response, status = unwrap(DataSourceApi().get)(DataSourceApi(), TENANT_ID)
assert status == 200 assert status == 200
assert response == { assert response == {
"data": [ "data": [
{ {
"id": "binding-1", "id": BINDING_ID,
"provider": "notion", "provider": "notion",
"created_at": 1779670923, "created_at": expected_created_at,
"is_bound": True, "is_bound": True,
"disabled": False, "disabled": False,
"source_info": { "source_info": {
@@ -96,34 +116,75 @@ def test_get_data_source_integrates_serializes_orm_binding(flask_app: Flask) ->
} }
def test_get_data_source_integrates_preserves_empty_list_when_no_binding(flask_app: Flask) -> None: def test_get_data_source_integrates_preserves_empty_list_when_no_binding(
with ( flask_app: Flask,
flask_app.test_request_context("/"), ) -> None:
patch.object(module.db.session, "scalars", return_value=MagicMock(all=lambda: [])), with flask_app.test_request_context("/"):
): response, status = unwrap(DataSourceApi().get)(DataSourceApi(), TENANT_ID)
response, status = unwrap(DataSourceApi().get)(DataSourceApi(), "tenant-1")
assert status == 200 assert status == 200
assert response == {"data": []} assert response == {"data": []}
def test_patch_data_source_binding_uses_injected_session(flask_app: Flask) -> None: @pytest.mark.parametrize(
binding = MagicMock(disabled=True) ("disabled", "action", "expected_disabled"),
session = MagicMock() [(True, "enable", False), (False, "disable", True)],
session.scalar.return_value = binding )
@pytest.mark.parametrize("sqlite_session", [(DataSourceOauthBinding,)], indirect=True)
def test_patch_data_source_binding_updates_state(
flask_app: Flask,
sqlite_session: Session,
disabled: bool,
action: Literal["enable", "disable"],
expected_disabled: bool,
) -> None:
_add_binding(sqlite_session, disabled=disabled)
sqlite_session.expunge_all()
with flask_app.test_request_context("/"): with flask_app.test_request_context("/"):
response, status = unwrap(DataSourceApi().patch)(DataSourceApi(), session, "tenant-1", uuid4(), "enable") response, status = unwrap(DataSourceApi().patch)(
DataSourceApi(), sqlite_session, TENANT_ID, UUID(BINDING_ID), action
)
sqlite_session.flush()
sqlite_session.expire_all()
binding = sqlite_session.scalar(select(DataSourceOauthBinding).where(DataSourceOauthBinding.id == BINDING_ID))
assert status == 200 assert status == 200
assert response == {"result": "success"} assert response == {"result": "success"}
assert binding.disabled is False assert binding is not None
session.scalar.assert_called_once() assert binding.disabled is expected_disabled
session.add.assert_not_called()
session.commit.assert_not_called()
def test_notion_pre_import_pages_serializes_frontend_list_shape(flask_app: Flask, current_user: Account) -> None: @pytest.mark.parametrize("sqlite_session", [(DataSourceOauthBinding,)], indirect=True)
def test_patch_data_source_binding_rejects_unknown_binding(
flask_app: Flask,
sqlite_session: Session,
) -> None:
with flask_app.test_request_context("/"), pytest.raises(NotFound, match="Data source binding not found"):
unwrap(DataSourceApi().patch)(DataSourceApi(), sqlite_session, TENANT_ID, UUID(BINDING_ID), "enable")
@pytest.mark.parametrize(("disabled", "action"), [(False, "enable"), (True, "disable")])
@pytest.mark.parametrize("sqlite_session", [(DataSourceOauthBinding,)], indirect=True)
def test_patch_data_source_binding_rejects_current_state(
flask_app: Flask,
sqlite_session: Session,
disabled: bool,
action: Literal["enable", "disable"],
) -> None:
_add_binding(sqlite_session, disabled=disabled)
sqlite_session.expunge_all()
with flask_app.test_request_context("/"), pytest.raises(ValueError):
unwrap(DataSourceApi().patch)(DataSourceApi(), sqlite_session, TENANT_ID, UUID(BINDING_ID), action)
@pytest.mark.parametrize("sqlite_session", [()], indirect=True)
def test_notion_pre_import_pages_serializes_frontend_list_shape(
flask_app: Flask,
current_user: Account,
sqlite_session: Session,
) -> None:
page = MagicMock( page = MagicMock(
page_id="page-1", page_id="page-1",
page_name="Page", page_name="Page",
@@ -145,8 +206,6 @@ def test_notion_pre_import_pages_serializes_frontend_list_shape(flask_app: Flask
get_online_document_pages=MagicMock(return_value=iter([online_document_message])), get_online_document_pages=MagicMock(return_value=iter([online_document_message])),
datasource_provider_type=MagicMock(return_value="online_document"), datasource_provider_type=MagicMock(return_value="online_document"),
) )
session = MagicMock()
with ( with (
flask_app.test_request_context("/?credential_id=credential-1"), flask_app.test_request_context("/?credential_id=credential-1"),
patch.object( patch.object(
@@ -158,7 +217,7 @@ def test_notion_pre_import_pages_serializes_frontend_list_shape(flask_app: Flask
patch("core.datasource.datasource_manager.DatasourceManager.get_datasource_runtime", return_value=runtime), patch("core.datasource.datasource_manager.DatasourceManager.get_datasource_runtime", return_value=runtime),
): ):
response, status = unwrap(DataSourceNotionListApi().get)( response, status = unwrap(DataSourceNotionListApi().get)(
DataSourceNotionListApi(), session, "tenant-1", current_user DataSourceNotionListApi(), sqlite_session, "tenant-1", current_user
) )
assert status == 200 assert status == 200
@@ -183,3 +242,38 @@ def test_notion_pre_import_pages_serializes_frontend_list_shape(flask_app: Flask
} }
runtime.get_online_document_pages.assert_called_once() runtime.get_online_document_pages.assert_called_once()
assert runtime.get_online_document_pages.call_args.kwargs["datasource_parameters"] == {} assert runtime.get_online_document_pages.call_args.kwargs["datasource_parameters"] == {}
@pytest.mark.parametrize("sqlite_session", [()], indirect=True)
def test_notion_pre_import_pages_rejects_missing_credential(
flask_app: Flask,
current_user: Account,
sqlite_session: Session,
) -> None:
with (
flask_app.test_request_context("/?credential_id=credential-1"),
patch.object(module.DatasourceProviderService, "get_datasource_credentials", return_value=None),
pytest.raises(NotFound, match="Credential not found"),
):
unwrap(DataSourceNotionListApi().get)(DataSourceNotionListApi(), sqlite_session, TENANT_ID, current_user)
@pytest.mark.parametrize("sqlite_session", [()], indirect=True)
def test_notion_pre_import_pages_rejects_non_notion_dataset(
flask_app: Flask,
current_user: Account,
sqlite_session: Session,
) -> None:
dataset = MagicMock(data_source_type="other_type")
with (
flask_app.test_request_context("/?credential_id=credential-1&dataset_id=dataset-1"),
patch.object(
module.DatasourceProviderService,
"get_datasource_credentials",
return_value={"token": "token"},
),
patch.object(module.DatasetService, "get_dataset", return_value=dataset),
pytest.raises(ValueError, match="Dataset is not notion type"),
):
unwrap(DataSourceNotionListApi().get)(DataSourceNotionListApi(), sqlite_session, TENANT_ID, current_user)
@@ -0,0 +1,171 @@
"""Unit tests for controllers.console.datasets.data_source Notion endpoints."""
from __future__ import annotations
import inspect
from unittest.mock import MagicMock, patch
import pytest
from flask import Flask
from sqlalchemy.orm import Session
from werkzeug.exceptions import NotFound
from controllers.console.datasets.data_source import (
DataSourceNotionDatasetSyncApi,
DataSourceNotionDocumentSyncApi,
DataSourceNotionIndexingEstimateApi,
DataSourceNotionPreviewApi,
)
from core.rag.index_processor.constant.index_type import IndexStructureType
from models import Account
@pytest.fixture
def current_user() -> Account:
account = Account(name="Test User", email="[email protected]")
account.id = "u1"
return account
class TestDataSourceNotionPreviewApi:
def test_get_preview_success(self, app: Flask) -> None:
api = DataSourceNotionPreviewApi()
method = inspect.unwrap(api.get)
extractor = MagicMock(extract=lambda: [MagicMock(page_content="hello")])
with (
app.test_request_context("/?credential_id=c1"),
patch(
"controllers.console.datasets.data_source.DatasourceProviderService.get_datasource_credentials",
return_value={"integration_secret": "t"},
),
patch(
"controllers.console.datasets.data_source.NotionExtractor",
return_value=extractor,
),
):
response, status = method(api, "tenant-1", "p1", "page")
assert status == 200
class TestDataSourceNotionIndexingEstimateApi:
@pytest.mark.parametrize("sqlite_session", [()], indirect=True)
def test_post_indexing_estimate_success(self, app: Flask, sqlite_session: Session) -> None:
api = DataSourceNotionIndexingEstimateApi()
method = inspect.unwrap(api.post)
empty_rules: dict[str, object] = {}
payload: dict[str, object] = {
"notion_info_list": [
{
"workspace_id": "w1",
"credential_id": "c1",
"pages": [{"page_id": "p1", "type": "page"}],
}
],
"process_rule": {"rules": empty_rules},
"doc_form": IndexStructureType.PARAGRAPH_INDEX,
"doc_language": "English",
}
with (
app.test_request_context("/", method="POST", json=payload, headers={"Content-Type": "application/json"}),
patch(
"controllers.console.datasets.data_source.DocumentService.estimate_args_validate",
),
patch(
"controllers.console.datasets.data_source.IndexingRunner.indexing_estimate",
return_value=MagicMock(model_dump=lambda: {"total_pages": 1}),
),
):
response, status = method(api, sqlite_session, "tenant-1")
assert status == 200
class TestDataSourceNotionDatasetSyncApi:
@pytest.mark.parametrize("sqlite_session", [()], indirect=True)
def test_get_success(self, app: Flask, sqlite_session: Session) -> None:
api = DataSourceNotionDatasetSyncApi()
method = inspect.unwrap(api.get)
with (
app.test_request_context("/"),
patch(
"controllers.console.datasets.data_source.DatasetService.get_dataset",
return_value=MagicMock(),
),
patch(
"controllers.console.datasets.data_source.DocumentService.get_document_by_dataset_id",
return_value=[MagicMock(id="d1")],
),
patch(
"controllers.console.datasets.data_source.document_indexing_sync_task.delay",
return_value=None,
),
):
response, status = method(api, sqlite_session, "ds-1")
assert status == 200
@pytest.mark.parametrize("sqlite_session", [()], indirect=True)
def test_get_dataset_not_found(self, app: Flask, sqlite_session: Session) -> None:
api = DataSourceNotionDatasetSyncApi()
method = inspect.unwrap(api.get)
with (
app.test_request_context("/"),
patch(
"controllers.console.datasets.data_source.DatasetService.get_dataset",
return_value=None,
),
):
with pytest.raises(NotFound):
method(api, sqlite_session, "ds-1")
class TestDataSourceNotionDocumentSyncApi:
@pytest.mark.parametrize("sqlite_session", [()], indirect=True)
def test_get_success(self, app: Flask, sqlite_session: Session) -> None:
api = DataSourceNotionDocumentSyncApi()
method = inspect.unwrap(api.get)
with (
app.test_request_context("/"),
patch(
"controllers.console.datasets.data_source.DatasetService.get_dataset",
return_value=MagicMock(),
),
patch(
"controllers.console.datasets.data_source.DocumentService.get_document",
return_value=MagicMock(),
),
patch(
"controllers.console.datasets.data_source.document_indexing_sync_task.delay",
return_value=None,
),
):
response, status = method(api, sqlite_session, "ds-1", "doc-1")
assert status == 200
@pytest.mark.parametrize("sqlite_session", [()], indirect=True)
def test_get_document_not_found(self, app: Flask, sqlite_session: Session) -> None:
api = DataSourceNotionDocumentSyncApi()
method = inspect.unwrap(api.get)
with (
app.test_request_context("/"),
patch(
"controllers.console.datasets.data_source.DatasetService.get_dataset",
return_value=MagicMock(),
),
patch(
"controllers.console.datasets.data_source.DocumentService.get_document",
return_value=None,
),
):
with pytest.raises(NotFound):
method(api, sqlite_session, "ds-1", "doc-1")
@@ -0,0 +1,394 @@
from __future__ import annotations
from inspect import unwrap
from unittest.mock import MagicMock, PropertyMock, patch
import pytest
from flask import Flask
from controllers.console import console_ns
from controllers.console.workspace.skills import (
WorkspaceAgentSkillBindingsApi,
WorkspaceSkillAssistMessageApi,
WorkspaceSkillFilesApi,
WorkspaceSkillsApi,
WorkspaceSkillTagsApi,
WorkspaceSkillVersionApi,
)
from models.account import Account
from services.skill_management_service import SkillAssistAttachmentPayload, SkillManagementServiceError
@pytest.fixture
def app() -> Flask:
flask_app = Flask("test_workspace_skills")
flask_app.config["TESTING"] = True
return flask_app
@pytest.fixture
def current_user() -> Account:
user = Account(name="Test User", email="[email protected]")
user.id = "user-1"
return user
def _skill_detail() -> dict:
return {
"id": "skill-1",
"name": "finance-sop",
"display_name": "Finance SOP",
"icon": "📄",
"description": "",
"tags": [],
"name_manually_edited": False,
"visibility": "workspace",
"latest_published_version_id": None,
"reference_count": 0,
"created_by": "user-1",
"created_by_name": "Test User",
"updated_by": "user-1",
"updated_by_name": "Test User",
"created_at": 1,
"updated_at": 1,
"files": [
{
"id": "file-1",
"path": "SKILL.md",
"kind": "file",
"storage": "text",
"mime_type": "text/markdown",
"content": "---\nname: finance-sop\n---\n# Body",
"tool_file_id": None,
"size": 32,
"hash": "hash",
}
],
}
def test_create_skill_validates_payload_and_returns_detail(app: Flask, current_user: Account) -> None:
api = WorkspaceSkillsApi()
method = unwrap(api.post)
service = MagicMock()
service.create_skill.return_value = _skill_detail()
with (
app.test_request_context("/", method="POST"),
patch.object(type(console_ns), "payload", new_callable=PropertyMock, return_value={}),
patch("controllers.console.workspace.skills.SkillManagementService", return_value=service),
):
payload, status = method(api, "tenant-1", current_user)
assert status == 201
assert payload["id"] == "skill-1"
assert payload["files"][0]["path"] == "SKILL.md"
service.create_skill.assert_called_once()
assert service.create_skill.call_args.kwargs["tenant_id"] == "tenant-1"
assert service.create_skill.call_args.kwargs["user_id"] == "user-1"
def test_list_skills_uses_default_pagination_when_query_omits_page_and_limit(app: Flask) -> None:
api = WorkspaceSkillsApi()
method = unwrap(api.get)
service = MagicMock()
service.list_skills.return_value = {
"data": [],
"has_more": False,
"limit": 20,
"page": 1,
"total": 0,
}
with (
app.test_request_context("/?keyword=finance&tag=ops&tag=", method="GET"),
patch("controllers.console.workspace.skills.SkillManagementService", return_value=service),
):
payload = method(api, "tenant-1")
assert payload == {
"data": [],
"has_more": False,
"limit": 20,
"page": 1,
"total": 0,
}
service.list_skills.assert_called_once_with(
tenant_id="tenant-1",
keyword="finance",
page=1,
limit=20,
tags=["ops"],
)
def test_get_agent_skill_bindings_returns_card_data(app: Flask) -> None:
api = WorkspaceAgentSkillBindingsApi()
method = unwrap(api.get)
service = MagicMock()
service.list_agent_bindings.return_value = {
"agent_id": "agent-1",
"skill_ids": ["skill-1"],
"data": [
{
"id": "skill-1",
"priority": 0,
"name": "finance-sop",
"display_name": "Finance SOP",
"icon": "📄",
"description": "Handle finance.",
"tags": ["Finance"],
"status": "published",
"file_count": 2,
"latest_published_version_id": "version-1",
"latest_published_at": 123,
"updated_at": 124,
}
],
}
with (
app.test_request_context("/", method="GET"),
patch("controllers.console.workspace.skills.SkillManagementService", return_value=service),
):
payload = method(api, "tenant-1", "agent-1")
assert payload["skill_ids"] == ["skill-1"]
assert payload["data"][0]["display_name"] == "Finance SOP"
assert payload["data"][0]["file_count"] == 2
service.list_agent_bindings.assert_called_once_with(tenant_id="tenant-1", agent_id="agent-1")
def test_patch_skill_file_operation_validates_payload_and_returns_detail(app: Flask, current_user: Account) -> None:
api = WorkspaceSkillFilesApi()
method = unwrap(api.patch)
service = MagicMock()
service.apply_draft_file_operation.return_value = _skill_detail()
request_payload = {
"operation": "upsert_text",
"path": "references/policy.md",
"content": "Policy",
}
with (
app.test_request_context("/", method="PATCH"),
patch.object(type(console_ns), "payload", new_callable=PropertyMock, return_value=request_payload),
patch("controllers.console.workspace.skills.SkillManagementService", return_value=service),
):
payload = method(api, "tenant-1", current_user, "skill-1")
assert payload["id"] == "skill-1"
service.apply_draft_file_operation.assert_called_once()
call = service.apply_draft_file_operation.call_args.kwargs
assert call["tenant_id"] == "tenant-1"
assert call["user_id"] == "user-1"
assert call["skill_id"] == "skill-1"
assert call["payload"].operation == "upsert_text"
def test_patch_skill_file_operation_returns_error_details(app: Flask, current_user: Account) -> None:
api = WorkspaceSkillFilesApi()
method = unwrap(api.patch)
service = MagicMock()
service.apply_draft_file_operation.side_effect = SkillManagementServiceError(
"missing_skill_name",
"SKILL.md frontmatter name is required",
details={"path": "SKILL.md", "field": "name", "line": 2},
)
with (
app.test_request_context("/", method="PATCH"),
patch.object(
type(console_ns),
"payload",
new_callable=PropertyMock,
return_value={"operation": "delete", "path": "SKILL.md"},
),
patch("controllers.console.workspace.skills.SkillManagementService", return_value=service),
):
payload, status = method(api, "tenant-1", current_user, "skill-1")
assert status == 400
assert payload == {
"code": "missing_skill_name",
"message": "SKILL.md frontmatter name is required",
"details": {"path": "SKILL.md", "field": "name", "line": 2},
}
def test_list_skill_tags_returns_filter_options(app: Flask) -> None:
api = WorkspaceSkillTagsApi()
method = unwrap(api.get)
service = MagicMock()
service.list_tags.return_value = {"data": [{"tag": "finance", "count": 2}]}
with (
app.test_request_context("/", method="GET"),
patch("controllers.console.workspace.skills.SkillManagementService", return_value=service),
):
payload = method(api, "tenant-1")
assert payload == {"data": [{"tag": "finance", "count": 2}]}
service.list_tags.assert_called_once_with(tenant_id="tenant-1")
def test_get_skill_version_returns_version_detail(app: Flask) -> None:
api = WorkspaceSkillVersionApi()
method = unwrap(api.get)
service = MagicMock()
service.get_version.return_value = {
"id": "version-1",
"skill_id": "skill-1",
"version_number": 1,
"version_name": "Initial finance policy",
"publish_note": "Initial finance policy",
"hash_code": "hash-code",
"archive_size": 123,
"published_by": "user-1",
"published_by_name": "Li Wei",
"is_latest": True,
"created_at": 1,
"files": [
{
"id": None,
"path": "SKILL.md",
"kind": "file",
"storage": "text",
"mime_type": "text/markdown",
"content": "# Version",
"tool_file_id": None,
"size": 9,
"hash": "file-hash",
}
],
}
with (
app.test_request_context("/", method="GET"),
patch("controllers.console.workspace.skills.SkillManagementService", return_value=service),
):
payload = method(api, "tenant-1", "skill-1", "version-1")
assert payload["files"][0]["content"] == "# Version"
service.get_version.assert_called_once_with(
tenant_id="tenant-1",
skill_id="skill-1",
version_id="version-1",
)
def test_patch_skill_version_renames_version(app: Flask) -> None:
api = WorkspaceSkillVersionApi()
method = unwrap(api.patch)
service = MagicMock()
service.update_version.return_value = {
"id": "version-1",
"skill_id": "skill-1",
"version_number": 1,
"version_name": "Approval threshold",
"publish_note": "",
"hash_code": "hash-code",
"archive_size": 123,
"published_by": "user-1",
"published_by_name": "Li Wei",
"is_latest": True,
"created_at": 1,
}
with (
app.test_request_context("/", method="PATCH"),
patch.object(
type(console_ns),
"payload",
new_callable=PropertyMock,
return_value={"version_name": "Approval threshold"},
),
patch("controllers.console.workspace.skills.SkillManagementService", return_value=service),
):
payload = method(api, "tenant-1", "skill-1", "version-1")
assert payload["version_name"] == "Approval threshold"
service.update_version.assert_called_once()
assert service.update_version.call_args.kwargs["payload"].version_name == "Approval threshold"
def test_delete_skill_version_returns_new_latest(app: Flask, current_user: Account) -> None:
api = WorkspaceSkillVersionApi()
method = unwrap(api.delete)
service = MagicMock()
service.delete_version.return_value = {
"id": "version-2",
"deleted": True,
"latest_published_version_id": "version-1",
}
with (
app.test_request_context("/", method="DELETE"),
patch("controllers.console.workspace.skills.SkillManagementService", return_value=service),
):
payload = method(api, "tenant-1", current_user, "skill-1", "version-2")
assert payload == {"id": "version-2", "deleted": True, "latest_published_version_id": "version-1"}
service.delete_version.assert_called_once_with(
tenant_id="tenant-1",
user_id="user-1",
skill_id="skill-1",
version_id="version-2",
)
def test_skill_assistant_runs_agent_app_stream(app: Flask, current_user: Account) -> None:
api = WorkspaceSkillAssistMessageApi()
method = unwrap(api.post)
service = MagicMock()
assistant_app = MagicMock()
assistant_app.id = "assistant-app-1"
service.get_or_create_assistant_app.return_value = (assistant_app, "<skill_draft>draft</skill_draft>")
app_model = MagicMock()
app_response = MagicMock()
compact_response = MagicMock()
with (
app.test_request_context("/", method="POST"),
patch.object(
type(console_ns),
"payload",
new_callable=PropertyMock,
return_value={
"attachments": [
{
"tool_file_id": "tool-file-1",
"name": "requirements.md",
"mime_type": "text/markdown",
"size": 128,
}
],
"message": "Create an approval checklist.",
},
),
patch("controllers.console.workspace.skills.SkillManagementService", return_value=service),
patch(
"controllers.console.workspace.skills.db.session",
return_value=MagicMock(get=MagicMock(return_value=app_model)),
),
patch("controllers.console.workspace.skills.AppGenerateService.generate", return_value=app_response),
patch("controllers.console.workspace.skills.helper.compact_generate_response", return_value=compact_response),
):
response = method(api, "tenant-1", current_user, "skill-1")
assert response is compact_response
service.get_or_create_assistant_app.assert_called_once_with(
tenant_id="tenant-1",
skill_id="skill-1",
user_id="user-1",
attachments=[
SkillAssistAttachmentPayload(
tool_file_id="tool-file-1",
name="requirements.md",
mime_type="text/markdown",
size=128,
)
],
message="Create an approval checklist.",
model_payload=None,
)
@@ -1,17 +1,31 @@
""" """Unit tests for the Service API file-preview endpoint.
Unit tests for Service API File Preview endpoint
Ownership checks run against persisted message, file, app, and upload rows so the
tests exercise the same SQLAlchemy statements and tenant boundary as production.
Storage remains mocked because it is the external I/O boundary of the endpoint.
""" """
import logging import logging
import uuid from collections.abc import Iterator
from dataclasses import dataclass
from datetime import datetime
from decimal import Decimal
from typing import Protocol, cast from typing import Protocol, cast
from unittest.mock import Mock, patch from unittest.mock import Mock, patch
from uuid import uuid4
import pytest import pytest
from sqlalchemy import event
from sqlalchemy.engine import Engine
from sqlalchemy.orm import Session
from controllers.service_api.app.error import FileAccessDeniedError, FileNotFoundError from controllers.service_api.app.error import FileAccessDeniedError, FileNotFoundError
from controllers.service_api.app.file_preview import FilePreviewApi from controllers.service_api.app.file_preview import FilePreviewApi
from models.model import App, EndUser, Message, MessageFile, UploadFile from extensions.storage.storage_type import StorageType
from graphon.file import FileTransferMethod, FileType
from models.base import TypeBase
from models.enums import ConversationFromSource, CreatorUserRole
from models.model import App, AppMode, Message, MessageFile, UploadFile
class _FilePreviewLogRecord(Protocol): class _FilePreviewLogRecord(Protocol):
@@ -20,367 +34,252 @@ class _FilePreviewLogRecord(Protocol):
error: str error: str
@dataclass(frozen=True)
class _Database:
"""Expose the real test session through the interface used by the controller."""
session: Session
@dataclass(frozen=True)
class _PreviewRecords:
app: App
message: Message
message_file: MessageFile
upload_file: UploadFile
@pytest.fixture
def database(sqlite_engine: Engine) -> Iterator[_Database]:
"""Create only the tables required by file ownership validation."""
models = (App, Message, MessageFile, UploadFile)
tables = [TypeBase.metadata.tables[model.__tablename__] for model in models]
TypeBase.metadata.create_all(sqlite_engine, tables=tables)
with Session(sqlite_engine, expire_on_commit=False) as session:
yield _Database(session)
@pytest.fixture
def file_preview_api() -> FilePreviewApi:
"""Create the resource instance under test."""
return FilePreviewApi()
def _upload_file(*, tenant_id: str, file_id: str | None = None) -> UploadFile:
upload_file = UploadFile(
tenant_id=tenant_id,
storage_type=StorageType.LOCAL,
key="storage/key/test_file.jpg",
name="test_file.jpg",
size=1024,
extension="jpg",
mime_type="image/jpeg",
created_by_role=CreatorUserRole.ACCOUNT,
created_by=str(uuid4()),
created_at=datetime(2026, 1, 1),
used=True,
)
if file_id is not None:
upload_file.id = file_id
return upload_file
def _persist_preview_records(
session: Session,
*,
app_id: str | None = None,
app_tenant_id: str | None = None,
upload_tenant_id: str | None = None,
) -> _PreviewRecords:
app_id = app_id or str(uuid4())
app_tenant_id = app_tenant_id or str(uuid4())
upload_file = _upload_file(tenant_id=upload_tenant_id or app_tenant_id)
app = App(
id=app_id,
tenant_id=app_tenant_id,
name="Preview app",
description="",
mode=AppMode.CHAT,
icon_type=None,
icon="",
icon_background=None,
enable_site=True,
enable_api=True,
)
message = Message(
id=str(uuid4()),
app_id=app_id,
conversation_id=str(uuid4()),
_inputs={},
query="preview",
message={},
message_unit_price=Decimal(0),
answer="answer",
answer_unit_price=Decimal(0),
currency="USD",
from_source=ConversationFromSource.API,
)
message_file = MessageFile(
message_id=message.id,
type=FileType.IMAGE,
transfer_method=FileTransferMethod.LOCAL_FILE,
created_by_role=CreatorUserRole.ACCOUNT,
created_by=str(uuid4()),
upload_file_id=upload_file.id,
)
session.add_all([app, message, message_file, upload_file])
session.commit()
return _PreviewRecords(app=app, message=message, message_file=message_file, upload_file=upload_file)
class TestFilePreviewApi: class TestFilePreviewApi:
"""Test suite for FilePreviewApi""" """Exercise ownership validation and response construction."""
@pytest.fixture def test_validate_file_ownership_success(self, file_preview_api: FilePreviewApi, database: _Database):
def file_preview_api(self): records = _persist_preview_records(database.session)
"""Create FilePreviewApi instance for testing"""
return FilePreviewApi()
@pytest.fixture with patch("controllers.service_api.app.file_preview.db", database):
def mock_app(self): message_file, upload_file = file_preview_api._validate_file_ownership(
"""Mock App model""" records.upload_file.id, records.app.id
app = Mock(spec=App) )
app.id = str(uuid.uuid4())
app.tenant_id = str(uuid.uuid4())
return app
@pytest.fixture assert message_file.id == records.message_file.id
def mock_end_user(self): assert upload_file.id == records.upload_file.id
"""Mock EndUser model""" assert upload_file.tenant_id == records.app.tenant_id
end_user = Mock(spec=EndUser)
end_user.id = str(uuid.uuid4())
return end_user
@pytest.fixture def test_validate_file_ownership_file_not_found(self, file_preview_api: FilePreviewApi, database: _Database):
def mock_upload_file(self): with patch("controllers.service_api.app.file_preview.db", database):
"""Mock UploadFile model""" with pytest.raises(FileNotFoundError, match="File not found in message context"):
upload_file = Mock(spec=UploadFile) file_preview_api._validate_file_ownership(str(uuid4()), str(uuid4()))
upload_file.id = str(uuid.uuid4())
upload_file.name = "test_file.jpg"
upload_file.extension = "jpg"
upload_file.mime_type = "image/jpeg"
upload_file.size = 1024
upload_file.key = "storage/key/test_file.jpg"
upload_file.tenant_id = str(uuid.uuid4())
return upload_file
@pytest.fixture def test_validate_file_ownership_access_denied(self, file_preview_api: FilePreviewApi, database: _Database):
def mock_message_file(self): records = _persist_preview_records(database.session)
"""Mock MessageFile model"""
message_file = Mock(spec=MessageFile)
message_file.id = str(uuid.uuid4())
message_file.upload_file_id = str(uuid.uuid4())
message_file.message_id = str(uuid.uuid4())
return message_file
@pytest.fixture with patch("controllers.service_api.app.file_preview.db", database):
def mock_message(self): with pytest.raises(FileAccessDeniedError, match="not owned by requesting app"):
"""Mock Message model""" file_preview_api._validate_file_ownership(records.upload_file.id, str(uuid4()))
message = Mock(spec=Message)
message.id = str(uuid.uuid4())
message.app_id = str(uuid.uuid4())
return message
def test_validate_file_ownership_success( def test_validate_file_ownership_upload_file_not_found(self, file_preview_api: FilePreviewApi, database: _Database):
self, file_preview_api: FilePreviewApi, mock_app, mock_upload_file, mock_message_file, mock_message records = _persist_preview_records(database.session)
): database.session.delete(records.upload_file)
"""Test successful file ownership validation""" database.session.commit()
file_id = str(uuid.uuid4())
app_id = mock_app.id
# Set up the mocks with patch("controllers.service_api.app.file_preview.db", database):
mock_upload_file.tenant_id = mock_app.tenant_id with pytest.raises(FileNotFoundError, match="Upload file record not found"):
mock_message.app_id = app_id file_preview_api._validate_file_ownership(records.upload_file.id, records.app.id)
mock_message_file.upload_file_id = file_id
mock_message_file.message_id = mock_message.id
with patch("controllers.service_api.app.file_preview.db") as mock_db: def test_validate_file_ownership_tenant_mismatch(self, file_preview_api: FilePreviewApi, database: _Database):
# Mock scalar() for MessageFile and Message queries records = _persist_preview_records(database.session, upload_tenant_id=str(uuid4()))
mock_db.session.scalar.side_effect = [
mock_message_file, # MessageFile query
mock_message, # Message query
]
# Mock get() for UploadFile and App PK lookups
mock_db.session.get.side_effect = [
mock_upload_file, # UploadFile query
mock_app, # App query for tenant validation
]
# Execute the method with patch("controllers.service_api.app.file_preview.db", database):
result_message_file, result_upload_file = file_preview_api._validate_file_ownership(file_id, app_id) with pytest.raises(FileAccessDeniedError, match="tenant mismatch"):
file_preview_api._validate_file_ownership(records.upload_file.id, records.app.id)
# Assertions
assert result_message_file == mock_message_file
assert result_upload_file == mock_upload_file
def test_validate_file_ownership_file_not_found(self, file_preview_api: FilePreviewApi):
"""Test file ownership validation when MessageFile not found"""
file_id = str(uuid.uuid4())
app_id = str(uuid.uuid4())
with patch("controllers.service_api.app.file_preview.db") as mock_db:
# Mock MessageFile not found via scalar()
mock_db.session.scalar.return_value = None
# Execute and assert exception
with pytest.raises(FileNotFoundError) as exc_info:
file_preview_api._validate_file_ownership(file_id, app_id)
assert "File not found in message context" in str(exc_info.value)
def test_validate_file_ownership_access_denied(self, file_preview_api: FilePreviewApi, mock_message_file):
"""Test file ownership validation when Message not owned by app"""
file_id = str(uuid.uuid4())
app_id = str(uuid.uuid4())
with patch("controllers.service_api.app.file_preview.db") as mock_db:
# Mock MessageFile found but Message not owned by app via scalar()
mock_db.session.scalar.side_effect = [
mock_message_file, # MessageFile query - found
None, # Message query - not found (access denied)
]
# Execute and assert exception
with pytest.raises(FileAccessDeniedError) as exc_info:
file_preview_api._validate_file_ownership(file_id, app_id)
assert "not owned by requesting app" in str(exc_info.value)
def test_validate_file_ownership_upload_file_not_found(
self, file_preview_api: FilePreviewApi, mock_message_file, mock_message
):
"""Test file ownership validation when UploadFile not found"""
file_id = str(uuid.uuid4())
app_id = str(uuid.uuid4())
with patch("controllers.service_api.app.file_preview.db") as mock_db:
# Mock scalar() for MessageFile and Message
mock_db.session.scalar.side_effect = [
mock_message_file, # MessageFile query - found
mock_message, # Message query - found
]
# Mock get() for UploadFile - not found
mock_db.session.get.return_value = None
# Execute and assert exception
with pytest.raises(FileNotFoundError) as exc_info:
file_preview_api._validate_file_ownership(file_id, app_id)
assert "Upload file record not found" in str(exc_info.value)
def test_validate_file_ownership_tenant_mismatch(
self, file_preview_api: FilePreviewApi, mock_app, mock_upload_file, mock_message_file, mock_message
):
"""Test file ownership validation with tenant mismatch"""
file_id = str(uuid.uuid4())
app_id = mock_app.id
# Set up tenant mismatch
mock_upload_file.tenant_id = "different_tenant_id"
mock_app.tenant_id = "app_tenant_id"
mock_message.app_id = app_id
mock_message_file.upload_file_id = file_id
mock_message_file.message_id = mock_message.id
with patch("controllers.service_api.app.file_preview.db") as mock_db:
# Mock scalar() for MessageFile and Message queries
mock_db.session.scalar.side_effect = [
mock_message_file, # MessageFile query
mock_message, # Message query
]
# Mock get() for UploadFile and App PK lookups
mock_db.session.get.side_effect = [
mock_upload_file, # UploadFile query
mock_app, # App query for tenant validation
]
# Execute and assert exception
with pytest.raises(FileAccessDeniedError) as exc_info:
file_preview_api._validate_file_ownership(file_id, app_id)
assert "tenant mismatch" in str(exc_info.value)
def test_validate_file_ownership_invalid_input(self, file_preview_api: FilePreviewApi): def test_validate_file_ownership_invalid_input(self, file_preview_api: FilePreviewApi):
"""Test file ownership validation with invalid input""" with pytest.raises(FileAccessDeniedError, match="Invalid file or app identifier"):
# Test with empty file_id
with pytest.raises(FileAccessDeniedError) as exc_info:
file_preview_api._validate_file_ownership("", "app_id") file_preview_api._validate_file_ownership("", "app_id")
assert "Invalid file or app identifier" in str(exc_info.value)
# Test with empty app_id with pytest.raises(FileAccessDeniedError, match="Invalid file or app identifier"):
with pytest.raises(FileAccessDeniedError) as exc_info:
file_preview_api._validate_file_ownership("file_id", "") file_preview_api._validate_file_ownership("file_id", "")
assert "Invalid file or app identifier" in str(exc_info.value)
def test_build_file_response_basic(self, file_preview_api: FilePreviewApi, mock_upload_file): @pytest.mark.parametrize(
"""Test basic file response building""" ("as_attachment", "mime_type", "name", "extension", "size"),
mock_generator = Mock() [
(False, "image/jpeg", "test_file.jpg", "jpg", 1024),
(True, "image/jpeg", "test_file.jpg", "jpg", 1024),
(False, "text/html", "unsafe.html", "html", 1024),
(False, "video/mp4", "test_file.mp4", "mp4", 1024),
(False, "image/jpeg", "test_file.jpg", "jpg", 0),
],
)
def test_build_file_response(
self,
file_preview_api: FilePreviewApi,
as_attachment: bool,
mime_type: str,
name: str,
extension: str,
size: int,
):
upload_file = _upload_file(tenant_id=str(uuid4()))
upload_file.mime_type = mime_type
upload_file.name = name
upload_file.extension = extension
upload_file.size = size
response = file_preview_api._build_file_response(mock_generator, mock_upload_file, False) response = file_preview_api._build_file_response(Mock(), upload_file, as_attachment)
# Check response properties
assert response.mimetype == mock_upload_file.mime_type
assert response.direct_passthrough is True assert response.direct_passthrough is True
assert response.headers["Content-Length"] == str(mock_upload_file.size)
assert "Cache-Control" in response.headers assert "Cache-Control" in response.headers
assert ("Content-Length" in response.headers) is bool(size)
def test_build_file_response_as_attachment(self, file_preview_api: FilePreviewApi, mock_upload_file): if as_attachment or mime_type == "text/html":
"""Test file response building with attachment flag""" assert "attachment" in response.headers["Content-Disposition"]
mock_generator = Mock() assert response.headers["Content-Type"] == "application/octet-stream"
else:
response = file_preview_api._build_file_response(mock_generator, mock_upload_file, True) assert response.mimetype == mime_type
if mime_type == "text/html":
# Check attachment-specific headers assert response.headers["X-Content-Type-Options"] == "nosniff"
assert "attachment" in response.headers["Content-Disposition"] if mime_type.startswith("video/"):
assert mock_upload_file.name in response.headers["Content-Disposition"] assert response.headers["Accept-Ranges"] == "bytes"
assert response.headers["Content-Type"] == "application/octet-stream"
def test_build_file_response_html_forces_attachment(self, file_preview_api: FilePreviewApi, mock_upload_file):
"""Test HTML files are forced to download"""
mock_generator = Mock()
mock_upload_file.mime_type = "text/html"
mock_upload_file.name = "unsafe.html"
mock_upload_file.extension = "html"
response = file_preview_api._build_file_response(mock_generator, mock_upload_file, False)
assert "attachment" in response.headers["Content-Disposition"]
assert response.headers["Content-Type"] == "application/octet-stream"
assert response.headers["X-Content-Type-Options"] == "nosniff"
def test_build_file_response_audio_video(self, file_preview_api: FilePreviewApi, mock_upload_file):
"""Test file response building for audio/video files"""
mock_generator = Mock()
mock_upload_file.mime_type = "video/mp4"
response = file_preview_api._build_file_response(mock_generator, mock_upload_file, False)
# Check Range support for media files
assert response.headers["Accept-Ranges"] == "bytes"
def test_build_file_response_no_size(self, file_preview_api: FilePreviewApi, mock_upload_file):
"""Test file response building when size is unknown"""
mock_generator = Mock()
mock_upload_file.size = 0 # Unknown size
response = file_preview_api._build_file_response(mock_generator, mock_upload_file, False)
# Content-Length should not be set when size is unknown
assert "Content-Length" not in response.headers
@patch("controllers.service_api.app.file_preview.storage") @patch("controllers.service_api.app.file_preview.storage")
def test_get_method_integration( def test_components_use_validated_file(
self, self, mock_storage: Mock, file_preview_api: FilePreviewApi, database: _Database
mock_storage,
file_preview_api: FilePreviewApi,
mock_app,
mock_end_user,
mock_upload_file,
mock_message_file,
mock_message,
): ):
"""Test the full GET method integration (without decorator)""" records = _persist_preview_records(database.session)
file_id = str(uuid.uuid4()) generator = Mock()
app_id = mock_app.id
# Set up mocks with patch("controllers.service_api.app.file_preview.db", database):
mock_upload_file.tenant_id = mock_app.tenant_id message_file, upload_file = file_preview_api._validate_file_ownership(
mock_message.app_id = app_id records.upload_file.id, records.app.id
mock_message_file.upload_file_id = file_id )
mock_message_file.message_id = mock_message.id response = file_preview_api._build_file_response(generator, upload_file, False)
mock_generator = Mock() assert message_file.id == records.message_file.id
mock_storage.load.return_value = mock_generator assert response.mimetype == "image/jpeg"
mock_storage.load.assert_not_called()
with patch("controllers.service_api.app.file_preview.db") as mock_db:
# Mock scalar() for MessageFile and Message queries
mock_db.session.scalar.side_effect = [
mock_message_file, # MessageFile query
mock_message, # Message query
]
# Mock get() for UploadFile and App PK lookups
mock_db.session.get.side_effect = [
mock_upload_file, # UploadFile query
mock_app, # App query for tenant validation
]
# Test the core logic directly without Flask decorators
# Validate file ownership
result_message_file, result_upload_file = file_preview_api._validate_file_ownership(file_id, app_id)
assert result_message_file == mock_message_file
assert result_upload_file == mock_upload_file
# Test file response building
response = file_preview_api._build_file_response(mock_generator, mock_upload_file, False)
assert response is not None
# Verify storage was called correctly
mock_storage.load.assert_not_called() # Since we're testing components separately
@patch("controllers.service_api.app.file_preview.storage") @patch("controllers.service_api.app.file_preview.storage")
def test_storage_error_handling( def test_storage_error_remains_external(
self, self, mock_storage: Mock, file_preview_api: FilePreviewApi, database: _Database
mock_storage,
file_preview_api: FilePreviewApi,
mock_app,
mock_upload_file,
mock_message_file,
mock_message,
): ):
"""Test storage error handling in the core logic""" records = _persist_preview_records(database.session)
file_id = str(uuid.uuid4()) mock_storage.load.side_effect = OSError("Storage error")
app_id = mock_app.id
# Set up mocks with patch("controllers.service_api.app.file_preview.db", database):
mock_upload_file.tenant_id = mock_app.tenant_id _, upload_file = file_preview_api._validate_file_ownership(records.upload_file.id, records.app.id)
mock_message.app_id = app_id
mock_message_file.upload_file_id = file_id
mock_message_file.message_id = mock_message.id
# Mock storage error with pytest.raises(OSError, match="Storage error"):
mock_storage.load.side_effect = Exception("Storage error") mock_storage.load(upload_file.key, stream=True)
with patch("controllers.service_api.app.file_preview.db") as mock_db:
# Mock scalar() for MessageFile and Message queries
mock_db.session.scalar.side_effect = [
mock_message_file, # MessageFile query
mock_message, # Message query
]
# Mock get() for UploadFile and App PK lookups
mock_db.session.get.side_effect = [
mock_upload_file, # UploadFile query
mock_app, # App query for tenant validation
]
# First validate file ownership works
result_message_file, result_upload_file = file_preview_api._validate_file_ownership(file_id, app_id)
assert result_message_file == mock_message_file
assert result_upload_file == mock_upload_file
# Test storage error handling
with pytest.raises(Exception) as exc_info:
mock_storage.load(mock_upload_file.key, stream=True)
assert "Storage error" in str(exc_info.value)
def test_validate_file_ownership_unexpected_error_logging( def test_validate_file_ownership_unexpected_error_logging(
self, file_preview_api: FilePreviewApi, caplog: pytest.LogCaptureFixture self,
file_preview_api: FilePreviewApi,
database: _Database,
sqlite_engine: Engine,
caplog: pytest.LogCaptureFixture,
): ):
"""Test that unexpected errors are logged properly""" file_id = str(uuid4())
file_id = str(uuid.uuid4()) app_id = str(uuid4())
app_id = str(uuid.uuid4())
with patch("controllers.service_api.app.file_preview.db") as mock_db: def fail_statement(*_args: object) -> None:
# Mock database scalar to raise unexpected exception raise RuntimeError("Unexpected database error")
mock_db.session.scalar.side_effect = Exception("Unexpected database error")
# Execute and assert exception event.listen(sqlite_engine, "before_cursor_execute", fail_statement)
with caplog.at_level(logging.ERROR, logger="controllers.service_api.app.file_preview"): try:
with pytest.raises(FileAccessDeniedError) as exc_info: with patch("controllers.service_api.app.file_preview.db", database):
file_preview_api._validate_file_ownership(file_id, app_id) with caplog.at_level(logging.ERROR, logger="controllers.service_api.app.file_preview"):
with pytest.raises(FileAccessDeniedError, match="File access validation failed"):
file_preview_api._validate_file_ownership(file_id, app_id)
finally:
event.remove(sqlite_engine, "before_cursor_execute", fail_statement)
# Verify error message assert len(caplog.records) == 1
assert "File access validation failed" in str(exc_info.value) log_record = caplog.records[0]
assert log_record.getMessage() == "Unexpected error during file ownership validation"
# Verify logging was called with the structured context fields. The ``extra`` keys record = cast(_FilePreviewLogRecord, log_record)
# are attached to the LogRecord as attributes, so they are not in ``caplog.text``. assert record.file_id == file_id
assert len(caplog.records) == 1 assert record.app_id == app_id
log_record = caplog.records[0] assert record.error == "Unexpected database error"
assert log_record.getMessage() == "Unexpected error during file ownership validation"
record = cast(_FilePreviewLogRecord, log_record)
assert record.file_id == file_id
assert record.app_id == app_id
assert record.error == "Unexpected database error"

Some files were not shown because too many files have changed in this diff Show More