21269 Commits

This Branch

This Branch

All Branches

Author	SHA1	Message	Date
MimoN	3cb9511c57	fix: update webui filters (#37885) ... issue: #36621 - fix the empty segment source - remove http request timeout limit - support more filters in seg and channel table Signed-off-by: mimo.oyn <ning.ouyang@zilliz.com>	2024-11-21 20:04:33 +08:00
tinswzy	e247ff9ee7	enhance: refine rootcoord/metatable interfaces to ensure that each method includes a ctx parameter (#37846) ... issue: #35917 Before enhancing log trace information, it's necessary to pass the context to the method entry point. This PR first refine the rootcoord/metatable interfaces to ensure that each method includes a ctx parameter. Signed-off-by: tinswzy <zhenyuan.wei@zilliz.com>	2024-11-21 19:30:32 +08:00
wei liu	965bda6e60	enhance: Add channel name to shard leader log in meta cache (#37856) ... Signed-off-by: Wei Liu <wei.liu@zilliz.com>	2024-11-21 19:24:31 +08:00
wei liu	97a44b62fd	fix: Data race in datacoord channel manager (#37866) ... issue: #37865 Signed-off-by: Wei Liu <wei.liu@zilliz.com>	2024-11-21 19:00:32 +08:00
XuanYang-cn	d7dcc752f1	enhance: Increase task capacity and clean illegal task (#37896) ... 1. taskQueueCapacity 256 is too small for production when we want to re-write the entire collection 2. tasks should be cleaned when unable to recover, or the meta will remain in etcd forever later. Signed-off-by: yangxuan <xuan.yang@zilliz.com>	2024-11-21 18:48:32 +08:00
wei liu	0a440e0d38	fix: Prevent simultaneous balance of segments and channels (#37850) ... issue: #33550 balance segment and balance segment execute at same time, which will cause bounch of corner case. This PR disable simultaneous balance of segments and channels Signed-off-by: Wei Liu <wei.liu@zilliz.com>	2024-11-21 17:56:55 +08:00
wei liu	2159526e80	fix: Deadlock in compaction handler (#37868) ... issue: #37803 Signed-off-by: Wei Liu <wei.liu@zilliz.com>	2024-11-21 17:26:46 +08:00
XuanYang-cn	70e6a00ba1	fix: Replace outer lock with concurrent map (#37817) ... See also: #37493 --------- Signed-off-by: yangxuan <xuan.yang@zilliz.com>	2024-11-21 16:08:33 +08:00
wei liu	7817ea6b8b	enhance: Remove unnecessary stack trace in error (#37816) ... Signed-off-by: Wei Liu <wei.liu@zilliz.com>	2024-11-21 15:10:32 +08:00
cai.zhang	c07f056b17	fix: Use the ID to retrieve the real name when collectionName is empty (#37859) ... issue: #36989 Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>	2024-11-21 14:28:32 +08:00
wei liu	b983ef9fca	fix: Channel may be released after balance (#37862) ... issue: #37830 casue dist handler doesn't set channel's version, so if channel checker try to dedup channel, it may release the new delegator after balance finished. this PR fix the way to set proper version for channel. Signed-off-by: Wei Liu <wei.liu@zilliz.com>	2024-11-21 10:40:31 +08:00
Zhen Ye	f3a36f8a29	fix: use global pool but not dedicated pool for every index (#37852) ... issue: #37851 - make a global thread pool at tantivy temporally. - set 1 but not 4 threads for inverted text index. Signed-off-by: chyezh <chyezh@outlook.com>	2024-11-20 20:44:32 +08:00
zhikunyao	714421997c	enhance: [skip e2e] update mergify to support macos 13 job (#37844) ... Signed-off-by: Zhikun Yao <zhikun.yao@zilliz.com>	2024-11-20 16:52:31 +08:00
presburger	c73ca14a9b	enhence: update git workflows macos (#37819) ... Signed-off-by: yusheng.ma <yusheng.ma@zilliz.com>	2024-11-20 14:51:55 +08:00
Buqian Zheng	511edd29fd	enhance: disallow get raw vector data of a BM25 Function output field (#37800) ... issue: https://github.com/milvus-io/milvus/issues/35853 Signed-off-by: Buqian Zheng <zhengbuqian@gmail.com>	2024-11-20 14:22:30 +08:00
sre-ci-robot	7ba85504ba	[automated] Bump milvus version to v2.4.16 (#37795) ... Bump milvus version to v2.4.16 Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>	2024-11-20 11:46:37 +08:00
congqixia	b8d31ebed8	enhance: Remove unnecessary segment clone updating dist (#37797) ... Related to #37630 Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>	2024-11-20 11:26:31 +08:00
nish112022	484c6b5c44	feat: Added code for Internal-tls (#36865) ... issue : https://github.com/milvus-io/milvus/issues/36864 I have a few questions regarding my approach.I will consolidate them here for feedback and review.Thanks --------- Signed-off-by: Nischay Yadav <nischay.yadav@ibm.com> Signed-off-by: Nischay <Nischay.Yadav@ibm.com>	2024-11-20 06:00:32 +08:00
congqixia	83c902992c	enhance: Prevent generate "null" search params (#37812) ... Preventing generating null search params in restful search request Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>	2024-11-19 19:54:32 +08:00
yihao.dai	b3fc53015d	fix: Avoid memory copy in RESTful search and query (#37674) ... Custom `jsonRender` that encodes JSON data directly into the response stream, it uses less memory since it does not buffer the entire JSON structure before sending it, unlike `c.JSON` in `HTTPReturn`, which serializes the JSON fully in memory before writing it to the response. Benchmark testing shows that, using the custom render incurs no performance loss and reduces memory consumption by nearly 50%. issue: https://github.com/milvus-io/milvus/issues/37671 --------- Signed-off-by: bigsheeper <yihao.dai@zilliz.com>	2024-11-19 17:24:31 +08:00
SimFG	3bbbf3cd95	enhance: improve the lock key of the create/drop collection task (#37759) ... /kind improvement Signed-off-by: SimFG <bang.fu@zilliz.com>	2024-11-19 15:50:32 +08:00
cqy123456	8216345b07	enhance: reduce copy of bitset and id conversion of brurtforce search (#37675) ... issue: https://github.com/milvus-io/milvus/issues/37798 Signed-off-by: cqy123456 <qianya.cheng@zilliz.com>	2024-11-19 15:48:40 +08:00
yihao.dai	b6612e02b4	enhance: Reduce GetIndexInfos calls (#37695) ... Batch `GetIndexInfos` calls for segments to reduce RPC calls. issue: https://github.com/milvus-io/milvus/issues/37634 --------- Signed-off-by: bigsheeper <yihao.dai@zilliz.com>	2024-11-19 14:24:31 +08:00
nico	866d512f6f	test: update test cases and sdk version (#37774) ... Signed-off-by: nico <cheng.yuan@zilliz.com>	2024-11-19 14:14:32 +08:00
cai.zhang	dae4160466	enhance: Whether to enable mergeSort mode when performing mixCompaction (#37664) ... issue: #37579 Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>	2024-11-19 11:28:31 +08:00
Xiaofan	33bfb25c73	enhance: refine meta cache log and logic (#37318) ... related to #36989 add more logs in proxy meta cache and make it clearer Signed-off-by: xiaofanluan <xiaofan.luan@zilliz.com>	2024-11-19 10:50:33 +08:00
congqixia	175f49ea60	enhance: [GoSDK] Use generic column base and add unit tests (#37768) ... Related to #31293 Previous column is written using go before 1.18, which does not support generic type. After migrating client pkg to main repo, generic base could be used to reduce lots of duplicated code. --------- Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>	2024-11-19 10:40:31 +08:00
congqixia	c79fbd5eab	fix: Load l0 delta for growings when using RemoteLoad (#37771) ... Related to #37574 Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>	2024-11-19 10:38:31 +08:00
wei liu	113c72c02e	fix: [skip e2e] unstable ut TestResourceManager (#37761) ... issue: #37760 Signed-off-by: Wei Liu <wei.liu@zilliz.com>	2024-11-19 10:24:31 +08:00
Bingyi Sun	6b82320953	fix: Fix using wrong upperbound when searching by pk (#37769) ... issue: https://github.com/milvus-io/milvus/issues/37649 Signed-off-by: sunby <sunbingyi1992@gmail.com>	2024-11-19 10:16:31 +08:00
wei liu	fceff6ed7f	enhance: Optimize param cost in search (#37738) ... Signed-off-by: Wei Liu <wei.liu@zilliz.com>	2024-11-19 10:14:40 +08:00
congqixia	6d86b9022e	enhance: Provide secondary index critria when filter leaderview (#37777) ... Related to #37630 --------- Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>	2024-11-19 10:12:30 +08:00
smellthemoon	3d28d99411	fix: to use the correct offset in span (#37780) ... #37734 Signed-off-by: lixinguo <xinguo.li@zilliz.com> Co-authored-by: lixinguo <xinguo.li@zilliz.com>	2024-11-18 21:56:30 +08:00
congqixia	97b7ebbc08	fix: Move init kafka pool into once (#37786) ... Introduced by #37744 Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>	2024-11-18 21:24:48 +08:00
wei liu	351463b67e	fix: L0 segment has been loaded to worker during channel balance (#37748) ... issue: #37703 --------- Signed-off-by: Wei Liu <wei.liu@zilliz.com>	2024-11-18 20:58:30 +08:00
MimoN	5065d778d6	fix: [skip e2e]update webui base api url (#37779) ... issue: #36621 - fix tools links in webui Signed-off-by: mimo.oyn <ning.ouyang@zilliz.com>	2024-11-18 20:40:31 +08:00
sthuang	2d72ad33f2	enhance: RBAC built in privilege groups (#37720) ... issue: #37031 Signed-off-by: shaoting-huang <shaoting.huang@zilliz.com>	2024-11-18 20:38:39 +08:00
aoiasd	16e206167c	enhance: analyzer length filter max should be close interval instead open interval (#37770) ... Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>	2024-11-18 19:30:31 +08:00
MimoN	82d2ac6be0	fix: update webui base api url (#37775) ... issue: #36621 - fixed the base api url in webui Signed-off-by: mimo.oyn <ning.ouyang@zilliz.com>	2024-11-18 19:10:33 +08:00
wei liu	261212ee4a	fix: Get shard client failed by client is closed (#37729) ... issue: #37718 This PR refine the shard client ref counter, dec ref counter won't release client anymore, and only permit shard client manager to remove client. --------- Signed-off-by: Wei Liu <wei.liu@zilliz.com>	2024-11-18 18:22:30 +08:00
wei liu	226fe900e7	enhance: Use batch to speed up list collections from meta kv (#37742) ... issue: #36228 Signed-off-by: Wei Liu <wei.liu@zilliz.com>	2024-11-18 16:58:30 +08:00
yihao.dai	0fc0d1a888	fix: Limit the concurrency of channel tasks (#37740) ... Limit the maximum concurrency of channel tasks for each DataNode to prevent excessive subscriptions from causing DataNode OOM. issue: https://github.com/milvus-io/milvus/issues/37665 Signed-off-by: bigsheeper <yihao.dai@zilliz.com>	2024-11-18 16:26:30 +08:00
aoiasd	e9391acf80	fix: bm25 brute force search need index params k1 and b (#37721) ... relate: https://github.com/milvus-io/milvus/issues/35853 --------- Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>	2024-11-18 15:44:31 +08:00
jaime	12ed40e125	fix: some missing metrics for WebUI restful API (#37747) ... issue: #36621 Signed-off-by: jaime <yun.zhang@zilliz.com>	2024-11-18 14:00:32 +08:00
Chun Han	f2a2fd6808	enhance: shallow copy search request for large nq cases(#37732) (#37749) ... related: #37732 Signed-off-by: MrPresent-Han <chun.han@gmail.com> Co-authored-by: MrPresent-Han <chun.han@gmail.com>	2024-11-18 13:06:32 +08:00
jaime	257ecab84b	enhance: remove collection queryable check from health check (#37712) ... Signed-off-by: jaime <yun.zhang@zilliz.com>	2024-11-18 10:50:38 +08:00
congqixia	b0bd290a6e	enhance: Use internal json(sonic) to replace std json lib (#37708) ... Related to #35020 Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>	2024-11-18 10:46:31 +08:00
congqixia	3743fc2b00	enhance: Use pool to limit kafka cgo thread number (#37744) ... Related to #37665 Thread number went rocket high when there is lots of kafka consumers on datanode. Since the internal implementation is CGO, using which directly will make cgo thread leaked. This PR add a worker pool for kafka API utilzing CGO calls to limit thread number. Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>	2024-11-18 10:42:31 +08:00
Zhen Ye	3f1614e9d9	enhance: add trace_id into segcore logs (#37656) ... issue: #37655 Signed-off-by: chyezh <chyezh@outlook.com>	2024-11-18 10:20:30 +08:00
Buqian Zheng	00edec2ebd	fix: incorrect dim set when creating BM25 doc sparse array (#37717) ... issue: https://github.com/milvus-io/milvus/issues/35853 Signed-off-by: Buqian Zheng <zhengbuqian@gmail.com>	2024-11-16 16:36:30 +08:00