Frederick2313072
626e71cb3b
feat: implement content-based deduplication for document segments
...
- Add database index on (dataset_id, index_node_hash) for efficient deduplication queries
- Add deduplication check in SegmentService.create_segment and multi_create_segment methods
- Add deduplication check in DatasetDocumentStore.add_documents method to prevent duplicate embedding processing
- Skip creating segments with identical content hashes across the entire dataset
This prevents duplicate content from being re-processed and re-embedded when uploading documents with repeated content, improving efficiency and reducing unnecessary compute costs.
2025-09-20 06:28:14 +08:00
Frederick2313072
07047487c3
fix
2025-09-20 05:41:25 +08:00
Frederick2313072
c7064d44af
feat: local dev
2025-09-20 05:30:39 +08:00
lyzno1 and GitHub
e93bfe3d41
fix: resolve chat sidebar UI bugs for hover panel and dropdown menu ( #25813 )
2025-09-19 18:28:49 +08:00
GuanMu and GitHub
ab910c736c
feat(goto-anything): add RAG pipeline node search ( #25948 )
2025-09-19 18:28:13 +08:00
Yeuoly and GitHub
4047a6bb12
fix: ensure original response are maintained by yielding text messages in ApiTool ( #23456 ) ( #25973 )
2025-09-19 18:27:33 +08:00
df2478dc26
chore: translate i18n files and update type definitions ( #25964 )
...
Co-authored-by: WTW0313 <[email protected]
2025-09-19 18:27:09 +08:00
-LAN- and GitHub
4cc3f6045b
Run import-linter within make lint ( #25933 )
2025-09-19 18:26:43 +08:00
Joel and GitHub
1550316b8d
fix: undefined match the wrong output schema ( #25971 )
2025-09-19 17:03:09 +08:00
Wu Tianwei and GitHub
87394d2512
fix: enhance model parameter handling with advanced mode support and localization updates ( #25963 )
2025-09-19 15:47:52 +08:00
Wu Tianwei and GitHub
bad59c95bc
fix: update details display to conditionally show creator information ( #25952 )
2025-09-19 15:45:45 +08:00
Xiyuan Chen and GitHub
9f138ef246
Refactor WorkflowService to handle missing default credentials gracef… ( #25960 )
2025-09-19 00:45:35 -07:00
zxhlyh and GitHub
6453fc4973
fix: refresh datasource list after install datasource ( #25949 )
2025-09-19 11:03:45 +08:00
f62f926537
style: update GotoAnything component styling ( #25929 )
...
Co-authored-by: crazywoola <[email protected]
2025-09-19 10:36:43 +08:00
Yongtao Huang and GitHub
b3dafd913b
Chore: correct inconsistent logging and typo ( #25945 )
2025-09-19 10:36:16 +08:00
-LAN- and GitHub
b2d8a7eaf1
Fix: enforce editor-only access to chat message logs ( #25936 )
2025-09-18 21:59:51 +08:00
GuanMu and GitHub
3e54414191
chore: update post_create_command.sh to use dynamic workspace root for aliases ( #25913 )
2025-09-18 21:09:43 +08:00
a173546c8d
Fix: replace stdout prints with debug logging ( #25931 )
...
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
2025-09-18 21:03:20 +08:00
-LAN- and GitHub
aa69d90489
fix(makefile): correct uv project path for lint target ( #25818 )
2025-09-18 20:36:26 +08:00
-LAN- and GitHub
4ba1292455
refactor: replace print statements with proper logging ( #25773 )
2025-09-18 20:35:47 +08:00
Maries and GitHub
bb01c31f30
fix(api): enhance data handling in RagPipelineDslService to filter credentials ( #25926 )
2025-09-18 18:36:49 +08:00
Wu Tianwei and GitHub
cd90b2ca9e
refactor: replace useInvalid with useInvalidCustomizedTemplateList ( #25924 )
2025-09-18 18:17:20 +08:00
heyszt and GitHub
9a65350cf7
fix: rollback aliyun_trace icon ( #25921 )
2025-09-18 18:01:08 +08:00
quicksand and GitHub
680eb7a9f6
fix(datasets): retrieval_model null issue when updating dataset info ( #25907 )
2025-09-18 17:58:06 +08:00
crazywoola and GitHub
878420463c
fix: Message => str ( #25876 )
2025-09-18 17:57:57 +08:00
zxhlyh and GitHub
4692e20daf
fix: workflow header style ( #25922 )
2025-09-18 17:53:40 +08:00
QuantumGhost and GitHub
13fe2ca8fe
fix(api): fix single stepping variable loading ( #25908 )
2025-09-18 17:30:02 +08:00
zxhlyh and GitHub
1264e7d4f6
fix: use invalid last run ( #25911 )
2025-09-18 16:52:27 +08:00
4f45978cd9
fix: remote code execution in email endpoints ( #25753 )
...
Co-authored-by: crazywoola <[email protected]
2025-09-18 16:45:34 +08:00
5a0bf8e028
feat: make SQLALCHEMY_POOL_TIMEOUT configurable ( #25468 )
...
Co-authored-by: crazywoola <[email protected]
2025-09-18 16:13:56 +08:00
ffa163a8a8
refactor: simplify portal interactions and manage state in Configure component ( #25906 )
...
Co-authored-by: Copilot <[email protected]
2025-09-18 15:57:33 +08:00
Novice and GitHub
8f86f5749d
chore: Update the value of sys.dialogue_count to start from 1. ( #25905 )
2025-09-18 15:52:52 +08:00
00d3bf15f3
perf(web): optimize ESLint performance with concurrency flag and remove oxlint ( #25899 )
...
Co-authored-by: Claude <[email protected]
2025-09-18 15:50:42 +08:00
17hz and GitHub
7196c09e9d
chore(workflows): remove redundant eslint command from style workflow ( #25900 )
2025-09-18 15:50:09 +08:00
zxhlyh and GitHub
fadd9e0bf4
fix: workflow logs list ( #25903 )
2025-09-18 15:45:37 +08:00
zxhlyh and GitHub
d8b4bbe067
fix: datasource pinned list ( #25896 )
2025-09-18 14:52:33 +08:00
GuanMu and GitHub
24611e375a
fix: update Python base image to use bullseye variant ( #25895 )
2025-09-18 14:38:56 +08:00
lyzno1 and GitHub
ccec582cea
chore: add missing template translations in ja-JP ( #25892 )
2025-09-18 14:37:26 +08:00
Bowen Liang and GitHub
b2e4107c17
chore: improve opendal storage and ensure closing file after reading files in load_stream method ( #25874 )
2025-09-18 14:09:19 +08:00
quicksand and GitHub
87aa070486
feat(api/commands): add migrate-oss to migrate from Local/OpenDAL to … ( #25828 )
2025-09-18 14:09:00 +08:00
21230a8eb2
fix: handle None description in MCP tool transformation ( #25872 )
...
Co-authored-by: crazywoola <[email protected]
2025-09-18 13:11:38 +08:00
+10
85cda47c70
feat: knowledge pipeline ( #25360 )
...
Signed-off-by: -LAN- <[email protected] [email protected] [email protected] [email protected] [email protected] [email protected] [email protected] [email protected] [email protected] [email protected] [email protected] [email protected] [email protected] [email protected] [email protected] [email protected] [email protected]
2025-09-18 12:49:10 +08:00
zyssyz123 and GitHub
7dadb33003
fix: remove billing cache when add or delete app or member ( #25885 )
2025-09-18 12:18:07 +08:00
-LAN- and GitHub
b5a7e64e19
Fix incorrect API endpoint routing from PR #25628 ( #25778 )
2025-09-16 19:20:26 +08:00
b283b10d3e
Fix/lindorm vdb optimize ( #25748 )
...
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2025-09-16 16:54:18 +08:00
-LAN- and GitHub
ecb22226d6
refactor: remove Claude-specific references from documentation files ( #25760 )
2025-09-16 14:22:14 +08:00
8635aacb46
Enhance LLM model configuration validation to include active status c… ( #25759 )
...
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2025-09-15 23:15:53 -07:00
bdd85b36a4
ruff check preview ( #25653 )
...
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2025-09-16 12:58:12 +08:00
znn and GitHub
a0c7713494
chat remove transparency from chat bubble in dark mode ( #24921 )
2025-09-16 12:57:53 +08:00
abf4955c26
Feature: add test containers document indexing task ( #25684 )
...
Signed-off-by: NeatGuyCoding <[email protected]
2025-09-16 09:47:28 +08:00
Frederick2313072