9797 Commits

Author	SHA1	Message	Date
sre-ci-robot	0894ed0016	[automated] Update Knowhere Commit (#38082) ... Update Knowhere Commit Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>	2024-11-29 01:08:36 +08:00
Bingyi Sun	e6af806a0d	enhance: optimize self defined rust error (#37975) ... Prepare for issue: https://github.com/milvus-io/milvus/issues/37930 Signed-off-by: sunby <sunbingyi1992@gmail.com>	2024-11-28 20:30:36 +08:00
SimFG	302650ae0e	fix: use the default partition for the limit quota when the request partition name is empty (#38005) ... - issue: #37685 Signed-off-by: SimFG <bang.fu@zilliz.com>	2024-11-27 11:00:36 +08:00
SimFG	49ee46ec1d	enhance: support to config the default db properties (#38035) ... - issue: #38034 Signed-off-by: SimFG <bang.fu@zilliz.com>	2024-11-27 10:04:34 +08:00
SimFG	971b4f17ae	fix: add the db information in the dml message (#37969) ... - issue: #37966 Signed-off-by: SimFG <bang.fu@zilliz.com>	2024-11-27 10:02:35 +08:00
tinswzy	1dbb6cd7cb	enhance: refine the datacoord meta related interfaces (#37957) ... issue: #35917 This PR refines the meta-related APIs in datacoord to allow the ctx to be passed down to the catalog operation interfaces Signed-off-by: tinswzy <zhenyuan.wei@zilliz.com>	2024-11-26 19:46:34 +08:00
SimFG	2208b7c2ef	fix: the too long default root password does not take effect (#37983) ... - issue: #36987 Signed-off-by: SimFG <bang.fu@zilliz.com>	2024-11-26 17:24:35 +08:00
Zhen Ye	2b4f211d84	enhance: add switch for local rpc enabled (#37985) ... issue: #33285 - Add switch for local rpc --------- Signed-off-by: chyezh <chyezh@outlook.com>	2024-11-26 17:00:54 +08:00
aoiasd	db3c002788	fix:wrong warning log when idf oracle find L0 segment lacks (#37873) ... relate: https://github.com/milvus-io/milvus/issues/35853 Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>	2024-11-26 16:26:41 +08:00
congqixia	cb6542339e	enhance: Mark cgo thread with tag name (#38000) ... Related to #37999 This PR add `SetThreadName` API for marking cgo thread and utilize it when initializing cgo worker. --------- Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>	2024-11-26 11:22:35 +08:00
tinswzy	e76802f910	enhance: refine querycoord meta/catalog related interfaces to ensure that each method includes a ctx parameter (#37916) ... issue: #35917 This PR refine the querycoord meta related interfaces to ensure that each method includes a ctx parameter. Signed-off-by: tinswzy <zhenyuan.wei@zilliz.com>	2024-11-25 11:14:34 +08:00
Zhen Ye	fbb68ca370	enhance: make all index operation async scheduled by tokio (#37946) ... issue: #37851 related pr: https://github.com/milvus-io/tantivy/pull/3 Signed-off-by: chyezh <chyezh@outlook.com>	2024-11-25 10:12:34 +08:00
sre-ci-robot	ed73dfca3f	[automated] Update Knowhere Commit (#37965) ... Update Knowhere Commit Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>	2024-11-25 01:28:32 +08:00
Yinzuo Jiang	5a06faca39	feat: fp32 vector to fp16/bf16 vector conversion for RESTful API (#37556) ... RESTful API. The influenced API are as follows: - Handler. insert - HandlerV1. insert/upsert - HandlerV2. insert/upsert/search We do not modify search API in Handler/HandlerV1 because they do not support fp16/bf16 vectors. module github.com/milvus-io/milvus/pkg: Add `Float32ArrayToBFloat16Bytes()`, `Float32ArrayToFloat16Bytes()` and `Float32ArrayToBytes()`. These method will be used in GoSDK in the future. issue: #37448 Signed-off-by: Yinzuo Jiang <yinzuo.jiang@zilliz.com> Signed-off-by: Yinzuo Jiang <jiangyinzuo@foxmail.com>	2024-11-24 17:46:33 +08:00
zhagnlu	62af24c1a1	fix: change search latency metric from us unit to ms unit (#37806) ... #37805 Signed-off-by: luzhang <luzhang@zilliz.com> Co-authored-by: luzhang <luzhang@zilliz.com>	2024-11-24 17:26:33 +08:00
congqixia	1e76d2b9f1	fix: Use correct policy merging growing&l0 and add unit tests (#37950) ... Related to #37574 --------- Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>	2024-11-24 17:10:33 +08:00
Zhen Ye	b9a10a2f68	enhance: remove the rpc layer of coordinator when enabling standalone or mixcoord (#37815) ... issue: #37764 - add a local client to call local server directly for querycoord/rootcoord/datacoord. - enable local client if milvus is running mixcoord or standalone mode. --------- Signed-off-by: chyezh <chyezh@outlook.com>	2024-11-23 21:36:33 +08:00
sthuang	bd0811434b	fix: fix grant v2 service (#37945) ... issue: https://github.com/milvus-io/milvus/issues/37031 Signed-off-by: shaoting-huang <shaoting.huang@zilliz.com>	2024-11-22 19:42:32 +08:00
smellthemoon	df08d91c0f	enhance: error msg (#37901) ... #37900 Signed-off-by: lixinguo <xinguo.li@zilliz.com> Co-authored-by: lixinguo <xinguo.li@zilliz.com>	2024-11-22 18:40:32 +08:00
jaime	7bbfe86bcd	enhance: add list index and segment index retrieval API for WebUI (#37861) ... issue: #36621 Signed-off-by: jaime <yun.zhang@zilliz.com>	2024-11-22 16:58:34 +08:00
SimFG	7c5a8012cf	enhance: remove the collectionBasicInfo class in the proxy metacache (#37874) ... /kind improvement - issue: #37928 Signed-off-by: SimFG <bang.fu@zilliz.com>	2024-11-22 16:10:33 +08:00
Bingyi Sun	700a448a54	fix: Escape prefix before search in inverted index (#37925) ... issue: https://github.com/milvus-io/milvus/issues/37912 Signed-off-by: sunby <sunbingyi1992@gmail.com>	2024-11-22 14:10:33 +08:00
Bingyi Sun	06d73cf2e2	enhance: Remove raw tokenizer register. (#37886) ... tantivy already register raw tokenizer by default Signed-off-by: sunby <sunbingyi1992@gmail.com>	2024-11-22 12:02:32 +08:00
congqixia	b34bfb98a0	enhance: Refine Replica manager colle2Replicas secondary index (#37906) ... Related to #37630 This PR add a new util coll2Replicas secondary index to reduce map access & iteration while get replicas by collection --------- Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>	2024-11-22 11:54:32 +08:00
congqixia	1ed686783f	enhance: Use PrimaryKeys to replace interface slice for segment delete (#37880) ... Related to #35303 Reduce temporary memory usage for PK interface for segment delete. Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>	2024-11-22 11:52:33 +08:00
SimFG	923a661dfe	enhance: filter the fields instead of create a new response obj (#37845) ... /kind improvement Here you only need to filter out the system fields, and you don’t need to recreate a response, because recreating the response will cause this part to be easily missed when adding fields later. Signed-off-by: SimFG <bang.fu@zilliz.com>	2024-11-22 11:42:32 +08:00
congqixia	cfa1f1f141	enhance: Add thread watcher to provide actual thread num (#37905) ... Related to #37904 --------- Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>	2024-11-22 10:20:33 +08:00
congqixia	92e6ee6285	enhance: Use load pool for CreateTextIndex (#37898) ... Related to #37895 Only resolves the starving issue which caused goroutine leakage Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>	2024-11-22 10:06:33 +08:00
SimFG	54aaeda63f	fix: add the request ctx for stream pipeline interface (#37835) ... - issue: #37834 Signed-off-by: SimFG <bang.fu@zilliz.com>	2024-11-22 03:10:34 +08:00
sthuang	19572f5b06	enhance: RBAC new grant/revoke privilege (#37785) ... issue: https://github.com/milvus-io/milvus/issues/37031 also fix issues: https://github.com/milvus-io/milvus/issues/37843, https://github.com/milvus-io/milvus/issues/37842, https://github.com/milvus-io/milvus/issues/37887 Signed-off-by: shaoting-huang <shaoting.huang@zilliz.com>	2024-11-21 22:20:34 +08:00
wei liu	5f3601a6a5	fix: unstable integration test caused by paramtable.GetNodeID (#37909) ... issue: #37908 cause paramtable is global single instance, which cause paramtable.GetNodeID may return wrong server id in integration test. This PR use node.GetNodeID to replace paramtable.GetNodeID Signed-off-by: Wei Liu <wei.liu@zilliz.com>	2024-11-21 22:16:33 +08:00
Zhen Ye	1dc1a97e65	fix: use different thread pool for scheduler and merger (#37911) ... issue: #37895 related pr: https://github.com/milvus-io/tantivy/pull/2 Signed-off-by: chyezh <chyezh@outlook.com>	2024-11-21 21:34:33 +08:00
MimoN	3cb9511c57	fix: update webui filters (#37885) ... issue: #36621 - fix the empty segment source - remove http request timeout limit - support more filters in seg and channel table Signed-off-by: mimo.oyn <ning.ouyang@zilliz.com>	2024-11-21 20:04:33 +08:00
tinswzy	e247ff9ee7	enhance: refine rootcoord/metatable interfaces to ensure that each method includes a ctx parameter (#37846) ... issue: #35917 Before enhancing log trace information, it's necessary to pass the context to the method entry point. This PR first refine the rootcoord/metatable interfaces to ensure that each method includes a ctx parameter. Signed-off-by: tinswzy <zhenyuan.wei@zilliz.com>	2024-11-21 19:30:32 +08:00
wei liu	965bda6e60	enhance: Add channel name to shard leader log in meta cache (#37856) ... Signed-off-by: Wei Liu <wei.liu@zilliz.com>	2024-11-21 19:24:31 +08:00
wei liu	97a44b62fd	fix: Data race in datacoord channel manager (#37866) ... issue: #37865 Signed-off-by: Wei Liu <wei.liu@zilliz.com>	2024-11-21 19:00:32 +08:00
XuanYang-cn	d7dcc752f1	enhance: Increase task capacity and clean illegal task (#37896) ... 1. taskQueueCapacity 256 is too small for production when we want to re-write the entire collection 2. tasks should be cleaned when unable to recover, or the meta will remain in etcd forever later. Signed-off-by: yangxuan <xuan.yang@zilliz.com>	2024-11-21 18:48:32 +08:00
wei liu	0a440e0d38	fix: Prevent simultaneous balance of segments and channels (#37850) ... issue: #33550 balance segment and balance segment execute at same time, which will cause bounch of corner case. This PR disable simultaneous balance of segments and channels Signed-off-by: Wei Liu <wei.liu@zilliz.com>	2024-11-21 17:56:55 +08:00
wei liu	2159526e80	fix: Deadlock in compaction handler (#37868) ... issue: #37803 Signed-off-by: Wei Liu <wei.liu@zilliz.com>	2024-11-21 17:26:46 +08:00
XuanYang-cn	70e6a00ba1	fix: Replace outer lock with concurrent map (#37817) ... See also: #37493 --------- Signed-off-by: yangxuan <xuan.yang@zilliz.com>	2024-11-21 16:08:33 +08:00
wei liu	7817ea6b8b	enhance: Remove unnecessary stack trace in error (#37816) ... Signed-off-by: Wei Liu <wei.liu@zilliz.com>	2024-11-21 15:10:32 +08:00
cai.zhang	c07f056b17	fix: Use the ID to retrieve the real name when collectionName is empty (#37859) ... issue: #36989 Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>	2024-11-21 14:28:32 +08:00
wei liu	b983ef9fca	fix: Channel may be released after balance (#37862) ... issue: #37830 casue dist handler doesn't set channel's version, so if channel checker try to dedup channel, it may release the new delegator after balance finished. this PR fix the way to set proper version for channel. Signed-off-by: Wei Liu <wei.liu@zilliz.com>	2024-11-21 10:40:31 +08:00
Zhen Ye	f3a36f8a29	fix: use global pool but not dedicated pool for every index (#37852) ... issue: #37851 - make a global thread pool at tantivy temporally. - set 1 but not 4 threads for inverted text index. Signed-off-by: chyezh <chyezh@outlook.com>	2024-11-20 20:44:32 +08:00
Buqian Zheng	511edd29fd	enhance: disallow get raw vector data of a BM25 Function output field (#37800) ... issue: https://github.com/milvus-io/milvus/issues/35853 Signed-off-by: Buqian Zheng <zhengbuqian@gmail.com>	2024-11-20 14:22:30 +08:00
congqixia	b8d31ebed8	enhance: Remove unnecessary segment clone updating dist (#37797) ... Related to #37630 Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>	2024-11-20 11:26:31 +08:00
nish112022	484c6b5c44	feat: Added code for Internal-tls (#36865) ... issue : https://github.com/milvus-io/milvus/issues/36864 I have a few questions regarding my approach.I will consolidate them here for feedback and review.Thanks --------- Signed-off-by: Nischay Yadav <nischay.yadav@ibm.com> Signed-off-by: Nischay <Nischay.Yadav@ibm.com>	2024-11-20 06:00:32 +08:00
congqixia	83c902992c	enhance: Prevent generate "null" search params (#37812) ... Preventing generating null search params in restful search request Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>	2024-11-19 19:54:32 +08:00
yihao.dai	b3fc53015d	fix: Avoid memory copy in RESTful search and query (#37674) ... Custom `jsonRender` that encodes JSON data directly into the response stream, it uses less memory since it does not buffer the entire JSON structure before sending it, unlike `c.JSON` in `HTTPReturn`, which serializes the JSON fully in memory before writing it to the response. Benchmark testing shows that, using the custom render incurs no performance loss and reduces memory consumption by nearly 50%. issue: https://github.com/milvus-io/milvus/issues/37671 --------- Signed-off-by: bigsheeper <yihao.dai@zilliz.com>	2024-11-19 17:24:31 +08:00
SimFG	3bbbf3cd95	enhance: improve the lock key of the create/drop collection task (#37759) ... /kind improvement Signed-off-by: SimFG <bang.fu@zilliz.com>	2024-11-19 15:50:32 +08:00