Commit Graph
29 Commits
Author SHA1 Message Date
349c3cf7b8 feat(api): Add image multimodal support for LLMNode (#17372)
Enhance `LLMNode` with multimodal capability, introducing support for
image outputs.

This implementation extracts base64-encoded images from LLM responses,
saves them to the storage service, and records the file metadata in the
`ToolFile` table. In conversations, these images are rendered as
markdown-based inline images.
Additionally, the images are included in the LLMNode's output as
file variables, enabling subsequent nodes in the workflow to utilize them.

To integrate file outputs into workflows, adjustments to the frontend code
are necessary.

For multimodal output functionality, updates to related model configurations
are required. Currently, this capability has been applied exclusively to
Google's Gemini models.

Close #15814.

Signed-off-by: -LAN- <[email protected]>
Co-authored-by: -LAN- <[email protected]>
2025-04-30 17:28:02 +08:00
-LAN-andGitHub 8e6f6d64a4 feat: re-add prompt messages to result and chunks in llm (#17883)
Signed-off-by: -LAN- <[email protected]>
2025-04-11 17:04:49 +08:00
+110 403e2d58b9 Introduce Plugins (#13836)
Signed-off-by: yihong0618 <[email protected]>
Signed-off-by: -LAN- <[email protected]>
Signed-off-by: xhe <[email protected]>
Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: takatost <[email protected]>
Co-authored-by: kurokobo <[email protected]>
Co-authored-by: Novice Lee <[email protected]>
Co-authored-by: zxhlyh <[email protected]>
Co-authored-by: AkaraChen <[email protected]>
Co-authored-by: Yi <[email protected]>
Co-authored-by: Joel <[email protected]>
Co-authored-by: JzoNg <[email protected]>
Co-authored-by: twwu <[email protected]>
Co-authored-by: Hiroshi Fujita <[email protected]>
Co-authored-by: AkaraChen <[email protected]>
Co-authored-by: NFish <[email protected]>
Co-authored-by: Wu Tianwei <[email protected]>
Co-authored-by: 非法操作 <[email protected]>
Co-authored-by: Novice <[email protected]>
Co-authored-by: Hiroki Nagai <[email protected]>
Co-authored-by: Gen Sato <[email protected]>
Co-authored-by: eux <[email protected]>
Co-authored-by: huangzhuo1949 <[email protected]>
Co-authored-by: huangzhuo <[email protected]>
Co-authored-by: lotsik <[email protected]>
Co-authored-by: crazywoola <[email protected]>
Co-authored-by: nite-knite <[email protected]>
Co-authored-by: Jyong <[email protected]>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: gakkiyomi <[email protected]>
Co-authored-by: CN-P5 <[email protected]>
Co-authored-by: CN-P5 <[email protected]>
Co-authored-by: Chuehnone <[email protected]>
Co-authored-by: yihong <[email protected]>
Co-authored-by: Kevin9703 <[email protected]>
Co-authored-by: -LAN- <[email protected]>
Co-authored-by: Boris Feld <[email protected]>
Co-authored-by: mbo <[email protected]>
Co-authored-by: mabo <[email protected]>
Co-authored-by: Warren Chen <[email protected]>
Co-authored-by: JzoNgKVO <[email protected]>
Co-authored-by: jiandanfeng <[email protected]>
Co-authored-by: zhu-an <[email protected]>
Co-authored-by: zhaoqingyu.1075 <[email protected]>
Co-authored-by: 海狸大師 <[email protected]>
Co-authored-by: Xu Song <[email protected]>
Co-authored-by: rayshaw001 <[email protected]>
Co-authored-by: Ding Jiatong <[email protected]>
Co-authored-by: Bowen Liang <[email protected]>
Co-authored-by: JasonVV <[email protected]>
Co-authored-by: le0zh <[email protected]>
Co-authored-by: zhuxinliang <[email protected]>
Co-authored-by: k-zaku <[email protected]>
Co-authored-by: luckylhb90 <[email protected]>
Co-authored-by: hobo.l <[email protected]>
Co-authored-by: jiangbo721 <[email protected]>
Co-authored-by: 刘江波 <[email protected]>
Co-authored-by: Shun Miyazawa <[email protected]>
Co-authored-by: EricPan <[email protected]>
Co-authored-by: crazywoola <[email protected]>
Co-authored-by: sino <[email protected]>
Co-authored-by: Jhvcc <[email protected]>
Co-authored-by: lowell <[email protected]>
Co-authored-by: Boris Polonsky <[email protected]>
Co-authored-by: Ademílson Tonato <[email protected]>
Co-authored-by: Ademílson Tonato <[email protected]>
Co-authored-by: IWAI, Masaharu <[email protected]>
Co-authored-by: Yueh-Po Peng (Yabi) <[email protected]>
Co-authored-by: Jason <[email protected]>
Co-authored-by: Xin Zhang <[email protected]>
Co-authored-by: yjc980121 <[email protected]>
Co-authored-by: heyszt <[email protected]>
Co-authored-by: Abdullah AlOsaimi <[email protected]>
Co-authored-by: Abdullah AlOsaimi <[email protected]>
Co-authored-by: Yingchun Lai <[email protected]>
Co-authored-by: Hash Brown <[email protected]>
Co-authored-by: zuodongxu <[email protected]>
Co-authored-by: Masashi Tomooka <[email protected]>
Co-authored-by: aplio <[email protected]>
Co-authored-by: Obada Khalili <[email protected]>
Co-authored-by: Nam Vu <[email protected]>
Co-authored-by: Kei YAMAZAKI <[email protected]>
Co-authored-by: TechnoHouse <[email protected]>
Co-authored-by: Riddhimaan-Senapati <[email protected]>
Co-authored-by: MaFee921 <[email protected]>
Co-authored-by: te-chan <[email protected]>
Co-authored-by: HQidea <[email protected]>
Co-authored-by: Joshbly <[email protected]>
Co-authored-by: xhe <[email protected]>
Co-authored-by: weiwenyan-dev <[email protected]>
Co-authored-by: ex_wenyan.wei <[email protected]>
Co-authored-by: engchina <[email protected]>
Co-authored-by: engchina <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: 呆萌闷油瓶 <[email protected]>
Co-authored-by: Kemal <[email protected]>
Co-authored-by: Lazy_Frog <[email protected]>
Co-authored-by: Yi Xiao <[email protected]>
Co-authored-by: Steven sun <[email protected]>
Co-authored-by: steven <[email protected]>
Co-authored-by: Kalo Chin <[email protected]>
Co-authored-by: Katy Tao <[email protected]>
Co-authored-by: depy <[email protected]>
Co-authored-by: 胡春东 <[email protected]>
Co-authored-by: Junjie.M <[email protected]>
Co-authored-by: MuYu <[email protected]>
Co-authored-by: Naoki Takashima <[email protected]>
Co-authored-by: Summer-Gu <[email protected]>
Co-authored-by: Fei He <[email protected]>
Co-authored-by: ybalbert001 <[email protected]>
Co-authored-by: Yuanbo Li <[email protected]>
Co-authored-by: douxc <[email protected]>
Co-authored-by: liuzhenghua <[email protected]>
Co-authored-by: Wu Jiayang <[email protected]>
Co-authored-by: Your Name <[email protected]>
Co-authored-by: kimjion <[email protected]>
Co-authored-by: AugNSo <[email protected]>
Co-authored-by: llinvokerl <[email protected]>
Co-authored-by: liusurong.lsr <[email protected]>
Co-authored-by: Vasu Negi <[email protected]>
Co-authored-by: Hundredwz <[email protected]>
Co-authored-by: Xiyuan Chen <[email protected]>
2025-02-17 17:05:13 +08:00
yihongandGitHub 56e15d09a9 feat: mypy for all type check (#10921) 2024-12-24 18:38:51 +08:00
c5f7d650b5 feat: Allow using file variables directly in the LLM node and support more file types. (#10679)
Co-authored-by: Joel <[email protected]>
2024-11-22 16:30:22 +08:00
Bowen LiangandGitHub 0ebe198ff1 chore: use DEBUG in dify_config instead of parsing raw system environment variable in place (#10437) 2024-11-08 09:34:11 +08:00
-LAN-andGitHub e61752bd3a feat/enhance the multi-modal support (#8818) 2024-10-21 10:43:49 +08:00
zhuhaoandGitHub b90ad587c2 refactor: move the embedding to the rag module and abstract the rerank runner for extension (#9423) 2024-10-17 19:12:42 +08:00
JyongandGitHub 4669eb24be add embedding input type parameter (#8724) 2024-09-24 21:53:50 +08:00
-LAN-andGitHub ed37439ef7 refactor(api/core): Improve type hints and apply ruff formatter in agent runner and model manager. (#8166) 2024-09-10 15:00:25 +08:00
Shota TotsukaandGitHub 430e100142 refactor: Add @staticmethod decorator in api/core (#7652) 2024-08-26 19:45:03 +08:00
4e7b6aec3a feat: support pinning, including, and excluding for model providers and tools (#7419)
Co-authored-by: GareArc <[email protected]>
2024-08-21 11:16:43 +08:00
Nam VuandGitHub dbc1ae45de chore: update docstrings (#7343) 2024-08-16 14:19:01 +08:00
-LAN-andGitHub cd7fa8027a fix(api/core/model_manager.py): Avoid mutation during iteration. (#6536) 2024-07-22 22:58:22 +08:00
5e6fc58db3 Feat/environment variables in workflow (#6515)
Co-authored-by: JzoNg <[email protected]>
2024-07-22 15:29:39 +08:00
chenxu9741andGitHub 6ef401a9f0 feat:add tts-streaming config and future (#5492) 2024-07-09 11:33:58 +08:00
-LAN-andGitHub 23fa3dedc4 fix(core): Fix incorrect type hints. (#5427) 2024-06-20 15:16:21 +08:00
takatostandGitHub 5986841e27 fix: issue where an error occurs when invoking TTS without selecting a voice (#5046) 2024-06-09 20:28:24 +08:00
takatostandGitHub d1dbbc1e33 feat: backend model load balancing support (#4927) 2024-06-05 00:13:04 +08:00
7753ba2d37 FEAT: NEW WORKFLOW ENGINE (#3160)
Co-authored-by: Joel <[email protected]>
Co-authored-by: Yeuoly <[email protected]>
Co-authored-by: JzoNg <[email protected]>
Co-authored-by: StyleZhang <[email protected]>
Co-authored-by: jyong <[email protected]>
Co-authored-by: nite-knite <[email protected]>
Co-authored-by: jyong <[email protected]>
2024-04-08 18:51:46 +08:00
Charlie.WeiGitHubluowei <glpat-EjySCyNjWiLqAED-YmwM>crazywoolacrazywoola
300d9892a5 tts add voice choose (#2391)
Co-authored-by: luowei <glpat-EjySCyNjWiLqAED-YmwM>
Co-authored-by: crazywoola <[email protected]>
Co-authored-by: crazywoola <[email protected]>
2024-02-15 22:41:18 +08:00
Bowen LiangandGitHub 063191889d chore: apply ruff's pyupgrade linter rules to modernize Python code with targeted version (#2419) 2024-02-09 15:21:33 +08:00
Bowen LiangandGitHub 14a19a3da9 chore: apply ruff's pyflakes linter rules (#2420) 2024-02-08 14:11:10 +08:00
takatostandGitHub 9f637ead38 bump version to 0.5.3 (#2306) 2024-02-01 18:11:57 +08:00
Charlie.WeiGitHubluowei <glpat-EjySCyNjWiLqAED-YmwM>crazywoolacrazywoolaYeuoly
6355e61eb8 tts models support (#2033)
Co-authored-by: luowei <glpat-EjySCyNjWiLqAED-YmwM>
Co-authored-by: crazywoola <[email protected]>
Co-authored-by: crazywoola <[email protected]>
Co-authored-by: Yeuoly <[email protected]>
2024-01-24 01:05:37 +08:00
Bowen LiangandGitHub cc9e74123c improve: introduce isort for linting Python imports (#1983) 2024-01-12 12:34:01 +08:00
24bdedf802 fix get embedding model provider in empty dataset (#1986)
Co-authored-by: jyong <[email protected]>
2024-01-10 20:48:16 +08:00
Charlie.WeiGitHubluowei <glpat-EjySCyNjWiLqAED-YmwM>crazywoolacrazywoola
5b24d7129e Azure openai init (#1929)
Co-authored-by: luowei <glpat-EjySCyNjWiLqAED-YmwM>
Co-authored-by: crazywoola <[email protected]>
Co-authored-by: crazywoola <[email protected]>
2024-01-09 19:17:47 +08:00
d069c668f8 Model Runtime (#1858)
Co-authored-by: StyleZhang <[email protected]>
Co-authored-by: Garfield Dai <[email protected]>
Co-authored-by: chenhe <[email protected]>
Co-authored-by: jyong <[email protected]>
Co-authored-by: Joel <[email protected]>
Co-authored-by: Yeuoly <[email protected]>
2024-01-02 23:42:00 +08:00