test/milvus
1
0
Fork 0
mirror of https://gitee.com/milvus-io/milvus.git synced 2024-12-04 12:59:23 +08:00
Code Issues Actions 14 Packages Projects Releases Wiki Activity
20,090 Commits 96 Branches 105 Tags 185 MiB
04edb07d82
Commit Graph

34 Commits

Author SHA1 Message Date
XuanYang-cn
5e39aa9272
enhance: Make channel meta able to writer 200k plus segments (#33279) 
See also: #33125

---------

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2024-05-28 12:33:42 +08:00
yiwangdr
e895cfed84
fix: reduce redundant map operations in datacoord (#33343) 
More refactories will be added.
issue: #33342

Signed-off-by: yiwangdr <yiwangdr@gmail.com>
 2024-05-24 12:47:40 +08:00
yiwangdr
b1eacb2ae8
feat: datacoord/node watch based on rpc (#32036) 
issue: https://github.com/milvus-io/milvus/issues/25309

Signed-off-by: yiwangdr <yiwangdr@gmail.com>
 2024-05-07 15:49:30 +08:00
yiwangdr
037de8e4d3
enhance: speed up minor functions calls in datacoord (#32389) 
Related to https://github.com/milvus-io/milvus/issues/32165

1. nodeid based channel store access should use map access instead of
iteration.

2. The join-ish functions calls are slow when # collections/segments
increases (e.g. 10k).
e.g.
getNumRowsOfCollectionUnsafe is O(num_segments); GetAllCollectionNumRows
is of O(num_collections*num_segments).

Signed-off-by: yiwangdr <yiwangdr@gmail.com>
 2024-04-20 07:55:21 +08:00
congqixia
83da08c388
enhance: Use map instead of slice to maintain channel info (#32273) 
See also #32165

`ChannelManager.Match` is a frequent operation for datacoord. When the
collection number is large, iteration over all channels will cost lots
of CPU time and time consuming.

This PR change the data structure storing datanode-channel info to map
avoiding this iteration when checking channel existence.

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-04-16 15:57:19 +08:00
XuanYang-cn
a153950b10
Change channel to Interface (#27839) 
This PR changes `*channel` into RWChannel interface

See also: #25309

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2023-11-13 11:16:18 +08:00
xige-16
33c2012675
Add more metrics (#25081) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2023-06-26 17:52:44 +08:00
congqixia
ba84f52119
Fix watcher loop quit and channel shouldDrop logic (#23402) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-04-14 09:54:28 +08:00
jaime
c9d0c157ec
Move some modules from internal to public package (#22572) 
Signed-off-by: jaime <yun.zhang@zilliz.com>
 2023-04-06 19:14:32 +08:00
zhenshan.cao
1287ca699a
Refine usage of TimeRecorder.Record (#23142) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2023-03-30 18:56:22 +08:00
MrPresent-Han
77c9e33e70
support dml channel balancer on datacoord (#22324) (#22377) (#22692) 
Signed-off-by: MrPresent-Han <jamesharden11122@gmail.com>
 2023-03-20 10:01:56 +08:00
Enwei Jiao
697dedac7e
Use cockroachdb/errors to replace other error pkg (#22390) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2023-02-26 11:31:49 +08:00
congqixia
53782ab83e
Fill collection schema after DataCoord restarts (#21161) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-12-12 19:05:21 +08:00
Enwei Jiao
89b810a4db
Refactor all params into ParamItem (#20987) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>

Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2022-12-07 18:01:19 +08:00
SimFG
9c6436d72d
Add time log for methods of starting the node (#20313) 
Signed-off-by: SimFG <bang.fu@zilliz.com>

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2022-11-08 20:13:03 +08:00
wayblink
43ae6791e6
Make VChannelInfo read compatible with 2.0.2 (#17995) 
Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2022-07-07 21:58:22 +08:00
bigsheeper
230404499d
Check duplication before adding channel info in DataCoord (#17742) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-06-24 16:30:15 +08:00
Xiaofan
6426e3d037
Split Etcd save operation into small txns (#17386) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-06-08 15:08:07 +08:00
Nemo
3eb074a9b0
Update some comments for data coord with some minor code refactoring. (#15686) 
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-02-22 13:15:51 +08:00
Cai Yudong
891c202b73
Use GlobalParamTable for all components (#13885) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-12-23 18:39:11 +08:00
xiangzhou.guo
a87e690d53
Correct spelling error in code comment (#13413) 
Signed-off-by: xiangzhou.guo <xiangzhou.guo@zilliz.com>
 2021-12-15 15:59:17 +08:00
xiangzhou.guo
78d583a3bb
Correct spelling error (#13412) 
Signed-off-by: xiangzhou.guo <xiangzhou.guo@zilliz.com>
 2021-12-15 15:57:09 +08:00
Xiangyu Wang
405b3cd932
Use the same lint rules with golint in revive (#13324) 
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
 2021-12-14 15:31:07 +08:00
edward.zeng
260d3e124d
[skip ci] Refine datacoord channel store comment (#12991) 
Signed-off-by: Edward Zeng <jie.zeng@zilliz.com>
 2021-12-08 17:57:05 +08:00
congqixia
06ced4d1d1
Add DropVirtualChannel for DataCoord (#12361) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-11-29 22:35:41 +08:00
congqixia
54738d2ac5
[skip ci] Fix lint for datacoord/channel_store.og (#10964) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-10-30 21:08:00 +08:00
congqixia
5c0b5c42ca
Fix merge caused compile error (#10693) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-10-26 21:46:21 +08:00
sunby
7fbb469081
Change channel's field to upper case to log details (#10567) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-10-26 20:15:41 +08:00
sunby
9b8e1c657e
Split watch operations to many transactions (#10655) 
issue: #10633
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-10-26 19:38:20 +08:00
XuanYang-cn
acb4dd2069
[skip ci]Update license for dc channel manager (#10591) 
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2021-10-25 20:19:25 +08:00
congqixia
b028dc3918
[skip ci] Fix golint issue for datacoord/channel_store.go (#9995) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-10-15 20:44:38 +08:00
sunby
003a7ed7cc
[skip ci]Add comment for MarshalLogObject (#9920) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-10-14 21:26:33 +08:00
sunby
3a6f6d399f
[skip ci]Add comment for ChannelOpTypeNames (#9919) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-10-14 21:24:41 +08:00
sunby
044b06421a
Use etcd to watch channel on datanode (#7115) 
issue: #6578
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-10-14 15:44:34 +08:00