Commit Graph
1019 Commits
Author SHA1 Message Date
Yansong Zhang e9ee897973 fix: resolve remaining CI failures for style checks and unit tests
- Add model_features property and build_execution_context method to
  AgentAppRunner to fix mypy attr-defined errors
- Export WorkflowComment, WorkflowCommentReply, WorkflowCommentMention
  from models/__init__.py to fix import errors
- Add NestedNodeGraphRequest, NestedNodeGraphResponse,
  NestedNodeParameterSchema to services/workflow/entities.py
- Update test_agent_chat_app_runner: tests for invalid LLM mode and
  invalid strategy now reflect unified AgentAppRunner behavior
  (no longer raises ValueError for these cases)

Made-with: Cursor
2026-04-13 16:07:38 +08:00
Yansong Zhang b804c7ed47 fix: restore SandboxExpiredRecordsCleanConfig, remove debug logs
- Restore SandboxExpiredRecordsCleanConfig (billing/ops config that
  was mistakenly removed with sandbox execution code)
- Remove [DEBUG-AGENT] logging from app_generate_service.py

Made-with: Cursor
2026-04-13 14:39:48 +08:00
Yansong Zhang 94b3087b98 fix: resolve remaining CI failures
- app_model_config_service.py: add AppMode.AGENT to exhaustive match
- app_service.py: fix possibly unbound default_model_dict variable

Made-with: Cursor
2026-04-13 13:56:08 +08:00
zyssyz123andGitHub 3e0578a1c6 Merge branch 'main' into feat/new-agent-node 2026-04-13 13:43:47 +08:00
dev-miro26andGitHub f4d5e2f43d refactor(api): improve type safety in MCPToolManageService.execute_auth_actions (#34824) 2026-04-13 05:29:10 +00:00
Yansong Zhang c03b25a940 merge: resolve conflicts with origin/main
Conflicts resolved:
- workflow_app_runner.py: adopt main's DifyGraphInitContext pattern
- token_buffer_memory.py: adopt main's match/case, add AppMode.AGENT
- app_dsl_service.py: adopt main's match/case, add AppMode.AGENT

Made-with: Cursor
2026-04-13 12:52:56 +08:00
HeYinKazuneGitHubautofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
a111d56ea3 refactor: use sessionmaker in workflow_tools_manage_service.py (#34896)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-04-13 03:47:29 +00:00
wdeveloper16andGitHub 8436470fcb refactor: replace bare dict with TypedDicts in annotation_service (#34998) 2026-04-13 03:46:33 +00:00
Yansong Zhang 90cce7693f revert: remove all sandbox and skill related code
Remove ~12,900 lines of sandbox/skill code that was ported from
feat/support-agent-sandbox. This reverts to direct tool execution
(the original behavior before sandbox integration).

Removed:
- core/sandbox/ (SandboxBuilder, bash tools, providers, initializers)
- core/skill/ (SkillManager, assembler, entities)
- core/virtual_environment/ (5 provider implementations)
- core/zip_sandbox/ (archive operations)
- core/app_assets/ (asset management)
- core/app_bundle/ (bundle management)
- controllers/cli_api/ (DifyCli callback endpoints)
- services/sandbox/ (provider service)
- services/skill_service, app_asset_service, app_bundle_service
- models/sandbox.py, app_asset.py
- bin/dify-cli-* (3 platform binaries)
- web sandbox-provider-page and service
- SandboxLayer, _resolve_sandbox_context, _invoke_tool_in_sandbox
- CliApiConfig, DIFY_SANDBOX_CONTEXT_KEY
- sandbox-related migrations

Preserved: All Agent V2 core functionality (agent-v2 node, strategy
engine, transparent upgrade, LLM remapping, memory, context, tools
via direct execution).

Made-with: Cursor
2026-04-13 10:42:36 +08:00
-LAN-andGitHub 8c4ea5c898 fix: external dataset tenant checks for bound knowledge APIs (#34734) 2026-04-13 01:47:57 +00:00
GDS K SandGitHub 0862fd74b0 refactor: migrate _model_to_insertion_dict return type to TypedDict (#34988) 2026-04-12 09:50:55 +00:00
wdeveloper16GitHubautofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
0841b4c663 refactor(api): migrate tools, account, workflow and plugin services to SQLAlchemy 2.0 (#34966)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-04-12 01:45:27 +00:00
wdeveloper16GitHubautofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
440602f52a refactor(services): migrate summary_index_service to SQLAlchemy 2.0 select() API (#34971)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-04-12 01:37:16 +00:00
wdeveloper16GitHubautofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
510120410b refactor(services): migrate trigger_provider_service to SQLAlchemy 2.0 select() API (#34972)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-04-12 01:36:13 +00:00
wdeveloper16GitHubautofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
4ef67fef3a refactor(services): migrate builtin_tools_manage_service to SQLAlchemy 2.0 select() API (#34973)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-04-12 01:25:51 +00:00
wdeveloper16andGitHub 7bd5e80323 refactor(services): migrate datasource_provider_service to SQLAlchemy 2.0 select() API (#34974) 2026-04-12 01:23:24 +00:00
wdeveloper16GitHubautofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
7515eee0a8 refactor(services): migrate dataset_service and clear_free_plan_tenant_expired_logs to SQLAlchemy 2.0 select() API (#34970)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-04-12 01:21:52 +00:00
Sven WeidenmannandGitHub 859920a81f refactor: migrate verify_subscription_credentials return type to TypedDict (#34967) 2026-04-11 16:41:40 +00:00
wangxiaoleiandGitHub 65d66768c1 fix: fix tool output duplicate (#34962) 2026-04-11 15:07:31 +00:00
plindGitHubautofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2dc015b360 fix(api): default parent_mode to paragraph for hierarchical chunking via API (#34635)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-04-10 10:55:40 +00:00
Yansong Zhang 77c182f738 feat(api): propagate all app features in transparent upgrade
VirtualWorkflowSynthesizer._build_features() now extracts ALL legacy
app features from AppModelConfig into the synthesized workflow.features:

- opening_statement + suggested_questions
- sensitive_word_avoidance (keywords/API moderation)
- more_like_this
- speech_to_text / text_to_speech
- retriever_resource

Previously workflow.features was hardcoded to "{}", losing all these
features during transparent upgrade. Now AdvancedChatAppRunner's
moderation, opening text, and other feature layers work correctly
for transparently upgraded old apps.

Made-with: Cursor
2026-04-10 18:47:18 +08:00
corevibe555andGitHub 98eedf14dc refactor(services): replace Union with | syntax in service layer (#34905) 2026-04-10 07:44:47 +00:00
呆萌闷油瓶andGitHub cd3ee5bd5d fix: sqlalchemy.orm.exc.DetachedInstanceError (#34910) 2026-04-10 07:44:22 +00:00
wangxiaoleiGitHubautofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
bcd738d2e6 fix: fix orm_exc.DetachedInstanceError (#34904)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-04-10 07:13:59 +00:00
corevibe555GitHubautofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
488fcd4f83 refactor(services): replace Union with | syntax in service layer (batch 2) (#34906)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-04-10 07:05:31 +00:00
-LAN-andGitHub b90fe73c96 fix(api): prevent cross-tenant external API use-check disclosure (#34744) 2026-04-10 03:23:32 +00:00
dataCenter430andGitHub 1117b6e72d refactor: convert appmode misc if/elif to match/case (#30001) (#34869) 2026-04-10 00:35:12 +00:00
YBoyandGitHub 2352269ba9 refactor(api): type recommend app database retrieval dicts with TypedDicts (#34873) 2026-04-10 00:32:24 +00:00
aliworksx08andGitHub e143dbce50 refactor: migrate session.query to select API in webhook service (#34849) 2026-04-09 14:16:33 +00:00
aliworksx08andGitHub 8ad131bb3b refactor: migrate session.query to select API in file service (#34852) 2026-04-09 14:15:59 +00:00
wangxiaoleiandGitHub 41eeb1f2e7 fix: fix sqlalchemy.orm.exc.DetachedInstanceError (#34845) 2026-04-09 10:55:48 +00:00
Yansong Zhang e50c36526e fix(api): fix transparent upgrade SSE channel mismatch and chat mode routing
- workflow_execute_task: add AppMode.CHAT/AGENT_CHAT/COMPLETION to the
  AdvancedChatAppGenerator routing branch so transparently upgraded old
  apps can execute through the workflow engine.
- app_generate_service: use app_model.mode (not hardcoded AppMode.AGENT)
  for SSE event subscription channel, ensuring the subscriber and
  Celery publisher use the same Redis channel key.

Made-with: Cursor
2026-04-09 17:27:41 +08:00
-LAN-andGitHub d1e33ba9ea refactor(api): reduce Dify GraphInitParams usage (#34825) 2026-04-09 07:59:15 +00:00
Yansong Zhang e2e16772a1 fix(api): fix DSL import, memory loading, and remaining test coverage
1. DSL Import fix: change self._session.commit() to self._session.flush()
   in app_dsl_service.py _create_or_update_app() to avoid "closed transaction"
   error. DSL import now works: export agent app -> import -> new app created.

2. Memory loading attempt: added _load_memory_messages() to AgentV2Node
   that loads TokenBufferMemory from conversation history. However, chatflow
   engine manages conversations differently from easy-UI (conversation may
   not be in DB at query time, or uses ConversationVariablePersistenceLayer
   instead of Message table). Memory needs further investigation.

Test results:
- Multi-turn memory: Turn 1 OK, Turn 2 LLM doesn't see history (needs deeper fix)
- Service API with API Key: PASSED (answer="Sixteen" for 8+8)
- DSL Import: PASSED (status=completed, new app created)
- Token aggregation: PASSED (node=49, workflow=49)

Known: memory in multi-turn chatflow needs to use graphon's built-in
memory mechanism (MemoryConfig on node + ConversationVariablePersistenceLayer)
rather than direct DB query.

Made-with: Cursor
2026-04-09 14:47:55 +08:00
aliworksx08GitHubautofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
4c05316a7b refactor(api): deduplicate DSL shared entities into dsl_entities.py (#34762)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-04-09 06:04:18 +00:00
carlos4sandGitHub 66e588c8ca refactor(api): use sessionmaker in builtin tools manage service (#34812) 2026-04-09 05:58:38 +00:00
aliworksx08GitHubautofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
ee789db443 refactor: migrate session.query to select API in plugin services (#34817)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-04-09 05:49:59 +00:00
dataCenter430andGitHub 5f53748d07 refactor: convert ToolProviderType if/elif to match/case (#30001) (#34794) 2026-04-09 05:48:40 +00:00
carlos4sandGitHub a76a8876d1 refactor(api): use sessionmaker in datasource provider service (#34811) 2026-04-09 05:43:13 +00:00
carlos4sandGitHub be1f4b34f8 refactor(api): use sessionmaker in workflow & RAG pipeline services (#34805) 2026-04-09 05:42:39 +00:00
Yansong Zhang 482a004efe fix(api): fix duplicate answer and completion app upgrade issues
1. Remove StreamChunkEvent from AgentV2Node._run_without_tools():
   The agent-v2 node was yielding StreamChunkEvent during LLM streaming,
   AND the downstream answer node was outputting the same text via
   {{#agent.text#}} variable reference, causing "FourFour" duplication.
   Now text only flows through outputs.text -> answer node (single path).

2. Map inputs to query for completion app transparent upgrade:
   Completion apps send {inputs: {query: "..."}} not {query: "..."}.
   VirtualWorkflowSynthesizer route now extracts query from inputs
   when the top-level query is missing.

Verified:
- Old chat app: "What is 2+2?" -> "Four" (was "FourFour")
- Old completion app: {inputs: {query: "What is 3+3?"}} -> "3 + 3 = 6" (was failing)
- Old agent-chat app: still works

Made-with: Cursor
2026-04-09 12:02:43 +08:00
Yansong Zhang 7052257c8d fix(api): use lazy workflow persistence for transparent upgrade of old apps
VirtualWorkflowSynthesizer.ensure_workflow() creates a real draft
workflow on first call for a legacy app, persisting it to the database.
On subsequent calls, returns the existing draft.

This is needed because AdvancedChatAppGenerator's worker thread looks
up workflows from the database by ID. Instead of hacking the generator
to skip DB lookups, we treat this as a lazy one-time upgrade: the old
app gets a real workflow that can also be edited in the workflow editor.

Verified: old chat app created on main branch ("What is 2+2?" -> "Four")
and old agent-chat app ("Say hello" -> "Hello!") both successfully
execute through the Agent V2 engine with AGENT_V2_TRANSPARENT_UPGRADE=true.

Made-with: Cursor
2026-04-09 11:28:16 +08:00
Yansong Zhang edfcab6455 fix(api): add AGENT mode to app list filtering
Add AppMode.AGENT branch in get_paginate_apps() so that
filtering apps by mode=agent works correctly.
Discovered during comprehensive E2E testing.

14/14 E2E tests pass covering:
- A: New Agent app full lifecycle (create, draft, configs, publish, run)
- B: Old app creation compat (chat, completion, agent-chat, advanced-chat, workflow)
- C: App listing and filtering (all modes, agent filter)
- D: Workflow editor compat (block configs)
- E: DSL export

Made-with: Cursor
2026-04-09 10:54:05 +08:00
Yansong Zhang 66212e3575 feat(api): implement zero-migration transparent upgrade (Phase 8)
Add two feature-flag-controlled upgrade paths that allow existing apps
and LLM nodes to transparently run through the Agent V2 engine without
any database migration:

1. AGENT_V2_TRANSPARENT_UPGRADE (default: off):
   When enabled, old apps (chat/completion/agent-chat) bypass legacy
   Easy-UI runners. VirtualWorkflowSynthesizer converts AppModelConfig
   to an in-memory Workflow (start -> agent-v2 -> answer) at runtime,
   then executes via AdvancedChatAppGenerator. Falls back to legacy
   path on any synthesis error.

   VirtualWorkflowSynthesizer maps:
   - model JSON -> ModelConfig
   - pre_prompt/chat_prompt_config -> prompt_template
   - agent_mode.tools -> ToolMetadata[]
   - agent_mode.strategy -> agent_strategy
   - dataset_configs -> context
   - file_upload -> vision

2. AGENT_V2_REPLACES_LLM (default: off):
   When enabled, DifyNodeFactory.create_node() transparently remaps
   nodes with type="llm" to type="agent-v2" before class resolution.
   Since AgentV2NodeData is a strict superset of LLMNodeData, the
   mapping is lossless. With tools=[], Agent V2 behaves identically
   to LLM Node.

Both flags default to False for safety. Turn off = instant rollback.
46 existing tests pass. Flask starts successfully.

Made-with: Cursor
2026-04-09 10:30:52 +08:00
Yansong Zhang 44491e427c feat(api): enable all sandbox/skill controller routes and resolve dependencies (P0)
Resolve the full dependency chain to enable all previously disabled controllers:

Enabled routes:
- sandbox_files: sandbox file browser API
- sandbox_providers: sandbox provider management API
- app_asset: app asset management API
- skills: skill extraction API
- CLI API blueprint: DifyCli callback endpoints (/cli/api/*)

Dependencies extracted (64 files, ~8000 lines):
- models/sandbox.py, models/app_asset.py: DB models
- core/zip_sandbox/: zip-based sandbox execution
- core/session/: CLI API session management
- core/memory/: base memory + node token buffer
- core/helper/creators.py: helper utilities
- core/llm_generator/: context models, output models, utils
- core/workflow/nodes/command/: command node type
- core/workflow/nodes/file_upload/: file upload node type
- core/app/entities/: app_asset_entities, app_bundle_entities, llm_generation_entities
- services/: asset_content, skill, workflow_collaboration, workflow_comment
- controllers/console/app/error.py: AppAsset error classes
- core/tools/utils/system_encryption.py

Import fixes:
- dify_graph.enums -> graphon.enums in skill_service.py
- get_signed_file_url_for_plugin -> get_signed_file_url in cli_api.py

All 5 controllers verified: import OK, Flask starts successfully.
46 existing tests still pass.

Made-with: Cursor
2026-04-09 09:36:16 +08:00
dataCenter430andGitHub 9c4f897b9a refactor: convert segmentType if/elif to match/case in webhook_service.py (#30001) (#34770) 2026-04-09 00:36:28 +00:00
Jake ArmstrongandGitHub 8f46c9113c refactor(api): deduplicate ImportMode and ImportStatus enums from rag_pipeline_dsl_service (#34759) 2026-04-08 23:23:04 +00:00
carlos4sandGitHub 5aa4e23f54 refactor(api): use sessionmaker in end user, retention & cleanup services (#34765) 2026-04-08 23:21:28 +00:00
carlos4sandGitHub 1d971d3240 refactor(api): use sessionmaker in plugin & trigger services (#34764) 2026-04-08 23:18:26 +00:00
carlos4sandGitHub 4c70bfa8b8 refactor(api): use sessionmaker in trigger provider service & dataset… (#34774) 2026-04-08 23:13:38 +00:00