21164 Commits

This Branch

This Branch

All Branches

Author	SHA1	Message	Date
congqixia	f5b06a3c9f	enhance: Invalidate collection cache when release collection (#37577) ... Related to #37395 --------- Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>	2024-11-12 10:16:29 +08:00
Chun Han	2d29dcd30c	enhance:refine group_strict_size parameter(#37482) (#37483) ... related: #37482 Signed-off-by: MrPresent-Han <chun.han@gmail.com> Co-authored-by: MrPresent-Han <chun.han@gmail.com>	2024-11-12 09:56:28 +08:00
binbin	21b68029a0	test: Add nullable test cases for bulk writer (#37572) ... issue: #36129 Signed-off-by: binbin lv <binbin.lv@zilliz.com>	2024-11-12 09:46:28 +08:00
Bingyi Sun	c1eccce2fa	enhance: enable multiple chunked segment by default (#37570) ... Signed-off-by: sunby <sunbingyi1992@gmail.com>	2024-11-12 09:20:28 +08:00
foxspy	81141bd18d	fix: use yaml.v2 as yaml parser (#37423) ... issue: #34298 Viper uses yaml.v2 as the parser. This PR will adopt the parsing logic from Viper to handle YAML files, ensuring maximum consistency in parsing. Signed-off-by: xianliang.li <xianliang.li@zilliz.com>	2024-11-11 21:26:27 +08:00
Zhen Ye	1b6edd0b4b	enhance: refactor the consumer grpc proto for reusing grpc stream for multi-consumer (#37564) ... issue: #33285 - Modify the proto of consumer of streaming service. - Make VChannel as a required option for streaming --------- Signed-off-by: chyezh <chyezh@outlook.com>	2024-11-11 17:24:29 +08:00
XuanYang-cn	5e6c3df253	fix: l0RowCount metrics value always empty (#37306) ... See also: #36953 --------- Signed-off-by: yangxuan <xuan.yang@zilliz.com>	2024-11-11 17:00:27 +08:00
wei liu	61a5b15ada	fix: Lost loading collection's updateTs after qc restart (#37538) ... issue: #37537 Signed-off-by: Wei Liu <wei.liu@zilliz.com>	2024-11-11 14:34:28 +08:00
congqixia	83d2729f94	enhance: Generate milvusclient mockery with latest proto (#37567) ... Related to #37558 --------- Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>	2024-11-11 14:28:30 +08:00
Ted Xu	31d0c84f67	fix: panic calclulating data size on writing binlogs (#37575) ... See: #37568 Signed-off-by: Ted Xu <ted.xu@zilliz.com>	2024-11-11 14:08:27 +08:00
Zhen Ye	fca946dee1	enhance: move the search utilities from querynode into new module (#37531) ... issue: #33285 - The search utilities will be shared between query node and streaming node. Signed-off-by: chyezh <chyezh@outlook.com>	2024-11-11 11:10:27 +08:00
congqixia	5e90f348fc	enhance: Handle legacy proxy load fields request (#37565) ... Related to #35415 In rolling upgrade, legacy proxy may dispatch load request wit empty load field list. The upgraded querycoord may report error by mistake that load field list is changed. This PR: - Auto field empty load field list with all user field ids - Refine the error messag when load field list updates - Refine load job unit test with service cases Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>	2024-11-11 10:14:26 +08:00
sre-ci-robot	a88fff130f	Update all contributors ... Signed-off-by: sre-ci-robot <sre-ci-robot@zilliz.com>	2024-11-11 00:01:25 +00:00
ThreadDao	f42869c0ce	test: remove case skip label (#37539) ... [test] remove case skip label issue: #33952 Signed-off-by: ThreadDao <yufen.zong@zilliz.com>	2024-11-10 16:14:27 +08:00
aoiasd	12951f0abb	enhance: rename tokenizer to analyzer and check analyzer params (#37478) ... relate: https://github.com/milvus-io/milvus/issues/35853 --------- Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>	2024-11-10 16:12:26 +08:00
sthuang	ff00a12805	enhance: RBAC custom privilege group ut coverage (#37558) ... issue: https://github.com/milvus-io/milvus/issues/37031 Signed-off-by: shaoting-huang <shaoting.huang@zilliz.com>	2024-11-09 20:40:25 +08:00
zhuwenxing	ffdde391e3	test: fix unstable testcase caused by server refactor (#37545) ... issue: https://github.com/milvus-io/milvus/issues/37533 Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>	2024-11-09 09:58:26 +08:00
sthuang	70605cf5b3	enhance: Support custom privilege group for RBAC (#37087) ... issue: #37031 --------- Signed-off-by: shaoting-huang <shaoting.huang@zilliz.com>	2024-11-09 08:44:28 +08:00
yihao.dai	ff9bdf7029	fix: Fix load slowly (#37454) ... When there're a lot of loaded collections, they would occupy the target observer scheduler’s pool. This prevents loading collections from updating the current target in time, slowing down the load process. This PR adds a separate target dispatcher for loading collections. issue: https://github.com/milvus-io/milvus/issues/37166 --------- Signed-off-by: bigsheeper <yihao.dai@zilliz.com>	2024-11-09 07:48:26 +08:00
XuanYang-cn	ce2fa3de01	fix: Set correct memory size (#37551) ... See also: #37549 Signed-off-by: yangxuan <xuan.yang@zilliz.com>	2024-11-09 01:42:26 +08:00
yihao.dai	ebc3c82761	fix: Fix subscription leak (#37382) ... Close (unsubscribe) the msg stream after completing the `PreCreatedTopic` check to prevent backlog issue. issue: https://github.com/milvus-io/milvus/issues/36021 Signed-off-by: bigsheeper <yihao.dai@zilliz.com>	2024-11-08 16:16:26 +08:00
cai.zhang	ae227e3934	enhance: Add integration test for stats task (#37506) ... issue: #33744 Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>	2024-11-08 15:48:26 +08:00
congqixia	0f514708ff	enhance: Accelerate Build stage by skipping cpp ut compilation (#37530) ... Since master branch run cpp unittest in jenken, the cpp unit test building could be skipped since the output is not needed in github runner. Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>	2024-11-08 14:06:26 +08:00
congqixia	dcc1b506dc	enhance: Add context trace for querycoord queryable check (#37524) ... When check health logic failed to collection not-queryable, the related reason is hard to find in log. This PR add context for log with trace id and print unqueryable collection info log. Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>	2024-11-08 14:00:26 +08:00
wei liu	a03157838b	enhance: Enable node assign policy on resource group (#36968) ... issue: #36977 with node_label_filter on resource group, user can add label on querynode with env `MILVUS_COMPONENT_LABEL`, then resource group will prefer to accept node which match it's node_label_filter. then querynode's can't be group by labels, and put querynodes with same label to same resource groups. Signed-off-by: Wei Liu <wei.liu@zilliz.com>	2024-11-08 11:18:27 +08:00
SimFG	bcb6420540	enhance: enable the lock scheduler for the rootcoord (#37513) ... /kind improvement Signed-off-by: SimFG <bang.fu@zilliz.com>	2024-11-08 10:20:26 +08:00
Xiaofan	e073906a19	enhance: optimize describe collection and index (#37490) ... fix #37489 combine multiple describe collection and list index into one call Signed-off-by: xiaofanluan <xiaofan.luan@zilliz.com>	2024-11-08 10:18:34 +08:00
Zhen Ye	49657c4690	enhance: add create segment message, enable empty segment flush (#37407) ... issue: #37172 - add redo interceptor to implement append context refresh. (make new timetick) - add create segment handler for flusher. - make empty segment flushable and directly change it into dropped. - add create segment message into wal when creating new growing segment. - make the insert operation into following seq: createSegment -> insert -> insert -> flushSegment. - make manual flush into following seq: flushTs -> flushsegment -> flushsegment -> manualflush. --------- Signed-off-by: chyezh <chyezh@outlook.com>	2024-11-08 10:16:34 +08:00
yihao.dai	81879425e1	enhance: Optimize the performance of stats task (#37374) ... 1. Increase the writer's `batchSize` to avoid multiple serialization operations. 2. Perform asynchronous upload of binlog files to prevent blocking the data processing flow. 3. Reduce multiple calls to `writer.Flush()`. issue: https://github.com/milvus-io/milvus/issues/37373 --------- Signed-off-by: bigsheeper <yihao.dai@zilliz.com>	2024-11-08 10:08:27 +08:00
Ted Xu	bc9562feb1	enhance: avoid memory copy and serde in mix compaction (#37479) ... See: #37234 --------- Signed-off-by: Ted Xu <ted.xu@zilliz.com>	2024-11-07 16:30:57 -08:00
sre-ci-robot	1f7ce9e9c1	Update all contributors ... Signed-off-by: sre-ci-robot <sre-ci-robot@zilliz.com>	2024-11-08 00:02:00 +00:00
congqixia	11f4fe0177	enhance: [GoSDK] move client pkg go files to sub one (#37492) ... Related to #31293 Client source files under client pkg cannot be evaluate correctly by codecov. This PR moves them to `milvusclient` sub-package to fix this issue and follow go major version best practice. --------- Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>	2024-11-08 07:32:26 +08:00
Bingyi Sun	40ba5a3414	fix: fix chunked segment term filter expression and add ut (#37392) ... issue: https://github.com/milvus-io/milvus/issues/37143 --------- Signed-off-by: sunby <sunbingyi1992@gmail.com>	2024-11-07 11:04:19 -08:00
congqixia	5310d3469f	fix: Escape brace of dumped JSON for index err message (#37504) ... Related to #37503 Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>	2024-11-07 18:58:25 +08:00
Bingyi Sun	7ec449fee5	enhance: regenerate datacoord mockery (#37501) ... Signed-off-by: sunby <sunbingyi1992@gmail.com>	2024-11-07 17:38:24 +08:00
yihao.dai	36bb87d9b6	fix: Fix large growing segment (#37388) ... Consider the `sealProportion` factor during segment allocation. issue: https://github.com/milvus-io/milvus/issues/37387 Signed-off-by: bigsheeper <yihao.dai@zilliz.com>	2024-11-07 17:12:25 +08:00
yihao.dai	994f52fab8	fix: Revert "enhance: Support db for bulkinsert (#37012)" (#37420) ... This reverts commit 6e90f9e8d9. issue: https://github.com/milvus-io/milvus/issues/31273 Signed-off-by: bigsheeper <yihao.dai@zilliz.com>	2024-11-07 17:02:25 +08:00
cai.zhang	de627644f5	enhance: Update the template expression proto to improve transmission efficiency (#37484) ... issue: #36672 Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>	2024-11-07 16:28:25 +08:00
cai.zhang	4dc684126e	enhance: Handoff growing segment after sorted (#37385) ... issue: #33744 1. Segments generated from inserts will be loaded as growing until they are sorted by primary key. 2. This PR may increase memory pressure on the delegator, but we need to test the performance of stats. In local testing, the speed of stats is greater than the insert speed. Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>	2024-11-07 16:08:24 +08:00
Ted Xu	e47bf21305	fix: parse error given duplicated plan cache key (#37334) ... See: #37016 --------- Signed-off-by: Ted Xu <ted.xu@zilliz.com>	2024-11-07 15:14:25 +08:00
wei liu	00f6d0ec51	fix: watch channel stuck due to misuse of timer.Reset (#37433) ... issue: #37166 cause the misuse of timer.Reset, which cause dispatcher failed to send msg to virtual channel buffer, and dispatcher do splitting again and again, which hold the dispatcher manager's lock, block watching channel progress. This PR fix the misuse of timer.Reset Signed-off-by: Wei Liu <wei.liu@zilliz.com>	2024-11-07 14:34:24 +08:00
smellthemoon	86fd3200be	enhance: refactor createIndex in RESTful API (#37235) ... Make the parameter input method consistent with miluvs-client. Signed-off-by: lixinguo <xinguo.li@zilliz.com> Co-authored-by: lixinguo <xinguo.li@zilliz.com>	2024-11-07 14:18:30 +08:00
Buqian Zheng	40b770cb7b	fix: add rescorer activation function for BM25 (#37481) ... issue: #35853 Signed-off-by: Buqian Zheng <zhengbuqian@gmail.com>	2024-11-07 12:08:25 +08:00
jaime	f348bd9441	feat: add segment,pipeline, replica and resourcegroup api for WebUI (#37344) ... issue: #36621 Signed-off-by: jaime <yun.zhang@zilliz.com>	2024-11-07 11:52:25 +08:00
smellthemoon	9b6dd23f8e	fix: wrong path spelling when use rootpath in segcore (#37453) ... #36532 Signed-off-by: lixinguo <xinguo.li@zilliz.com> Co-authored-by: lixinguo <xinguo.li@zilliz.com>	2024-11-07 11:22:25 +08:00
aoiasd	b4c749dcd5	fix: merge sort segment loss data (#37400) ... relate: https://github.com/milvus-io/milvus/issues/37238 Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>	2024-11-07 11:18:26 +08:00
XuanYang-cn	51ed2a61c8	fix: Correct dropped segment num metrics (#37410) ... See also: #31891 Signed-off-by: yangxuan <xuan.yang@zilliz.com>	2024-11-07 11:16:24 +08:00
congqixia	f813fb4563	enhance: [GoSDK] Remove example in main pkg to make ut script work (#37472) ... Related to #31293 Example with main func/package causes `go list` command failed with vcs error. This PR removes example files with main package. The example snippet shall be added back in runnable test format. --------- Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>	2024-11-07 10:50:25 +08:00
cai.zhang	aed3b94b5d	enhance: Refine error message for contains array (#37383) ... issue: #36221 Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>	2024-11-07 10:44:25 +08:00
Zhen Ye	cae9e1c732	fix: drop collection failed if enable streaming service (#37444) ... issue: #36858 - Start channel manager on datacoord, but with empty assign policy in streaming service. - Make collection at dropping state can be recovered by flusher to make sure that milvus consume the dropCollection message. - Add backoff for flusher lifetime. - remove the proxy watcher from timetick at rootcoord in streaming service. Also see the better fixup: #37176 --------- Signed-off-by: chyezh <chyezh@outlook.com>	2024-11-07 10:26:26 +08:00