test/milvus
1
0
Fork 0
mirror of https://gitee.com/milvus-io/milvus.git synced 2024-12-02 03:48:37 +08:00
Code Issues Actions 12 Packages Projects Releases Wiki Activity
21,303 Commits 94 Branches 105 Tags 184 MiB
4cd79323bf
Commit Graph

39 Commits

Author SHA1 Message Date
Zhen Ye
cae9e1c732
fix: drop collection failed if enable streaming service (#37444) 
issue: #36858

- Start channel manager on datacoord, but with empty assign policy in
streaming service.
- Make collection at dropping state can be recovered by flusher to make
sure that
 milvus consume the dropCollection message.
- Add backoff for flusher lifetime.
- remove the proxy watcher from timetick at rootcoord in streaming
service.

Also see the better fixup: #37176

---------

Signed-off-by: chyezh <chyezh@outlook.com>
 2024-11-07 10:26:26 +08:00
wei liu
6e9fbd1630
enhance: Solve channel unbalance on datanode (#34984) 
issue: #33583
the old policy permit datanode has at most 2 more channels than other
datanode. so if milvus has 2 datanode and 2 channels, both 2 channels
will be assign to 1 datanode, left another datanode empty.

This PR refine the balance policy to solve channel unbalance on datanode

---------

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2024-07-26 17:35:47 +08:00
XuanYang-cn
314f4d995b
enhance: Tidy dc channel manager (#34515) 
See also: #34518

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2024-07-09 18:26:12 +08:00
XuanYang-cn
40e5c0be70
fix: Enable to balance released standby channel (#32986) 
See also: #32879

---------

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2024-05-14 11:51:32 +08:00
yiwangdr
b1eacb2ae8
feat: datacoord/node watch based on rpc (#32036) 
issue: https://github.com/milvus-io/milvus/issues/25309

Signed-off-by: yiwangdr <yiwangdr@gmail.com>
 2024-05-07 15:49:30 +08:00
congqixia
83da08c388
enhance: Use map instead of slice to maintain channel info (#32273) 
See also #32165

`ChannelManager.Match` is a frequent operation for datacoord. When the
collection number is large, iteration over all channels will cost lots
of CPU time and time consuming.

This PR change the data structure storing datanode-channel info to map
avoiding this iteration when checking channel existence.

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2024-04-16 15:57:19 +08:00
XuanYang-cn
623939c9f5
enhance: Remove not in use policies (#29448) 
THe results don't meet our requirements, and the code hasn't been
maintained for a long time.

See also: #29447

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2023-12-28 10:38:46 +08:00
XuanYang-cn
a153950b10
Change channel to Interface (#27839) 
This PR changes `*channel` into RWChannel interface

See also: #25309

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2023-11-13 11:16:18 +08:00
wei liu
14c8a90517
Fix auto balance block channel reassign after datanode restart (#28275) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-11-09 19:00:25 +08:00
SimFG
26f06dd732
Format the code (#27275) 
Signed-off-by: SimFG <bang.fu@zilliz.com>
 2023-09-21 09:45:27 +08:00
jaime
c9d0c157ec
Move some modules from internal to public package (#22572) 
Signed-off-by: jaime <yun.zhang@zilliz.com>
 2023-04-06 19:14:32 +08:00
MrPresent-Han
77c9e33e70
support dml channel balancer on datacoord (#22324) (#22377) (#22692) 
Signed-off-by: MrPresent-Han <jamesharden11122@gmail.com>
 2023-03-20 10:01:56 +08:00
aoiasd
148a024e05
Add tickle for datacoord watch event (#21193) 
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2023-02-15 16:20:34 +08:00
cai.zhang
e25f987a5c
Fix bug that calculated an extra DataNode of register policy (#21927) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2023-02-06 10:33:54 +08:00
Enwei Jiao
89b810a4db
Refactor all params into ParamItem (#20987) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>

Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2022-12-07 18:01:19 +08:00
Ten Thousand Leaves
0700e56008
Increase MaxWatchDuration and make it configurable (#20884) 
/kind improvement

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-11-29 17:19:14 +08:00
SimFG
c604d2f933
Make param names clearer in the policy.go (#20136) 
Signed-off-by: SimFG <bang.fu@zilliz.com>

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2022-10-31 11:35:34 +08:00
wayblink
43ae6791e6
Make VChannelInfo read compatible with 2.0.2 (#17995) 
Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2022-07-07 21:58:22 +08:00
Ten Thousand Leaves
51a9e54b21
Fix a bug that causes infinite loop (#17550) 
This will cause AddNode() to keep its lock forever so DataCoord would just hang

issue: #17366
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-06-15 20:50:10 +08:00
XuanYang-cn
42f643e727
Make DataNode release rather than delete when reassign (#17293) 
1. Reassgin now will assign to the original Node if no other nodes
   avaliable
2. Make AddNode balance async: ToRealse + Reassign

See also: #16114, #17270

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2022-06-02 13:56:04 +08:00
congqixia
6513be5c44
Extend watch/release channel logic in DataNode (#15925) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-03-09 15:39:58 +08:00
Nemo
3eb074a9b0
Update some comments for data coord with some minor code refactoring. (#15686) 
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-02-22 13:15:51 +08:00
godchen
3cc45eca9a
[skip e2e]Sort imports (#14810) 
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
 2022-01-05 14:08:42 +08:00
godchen
a039178339
[skip e2e]Sort imports (#14328) 
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
 2021-12-27 18:52:57 +08:00
edward.zeng
1848dae589
[skip e2e] Refine datacoord policy comment (#13794) 
Signed-off-by: Edward Zeng <jie.zeng@zilliz.com>
 2021-12-20 19:46:59 +08:00
godchen
c983af85c5
[skip e2e]Fix policy typo (#13645) 
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
 2021-12-17 19:46:42 +08:00
shaoyue
6777d24c65
[skip ci] Add comment for exported items (#12126) 
Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
 2021-11-19 18:05:25 +08:00
shaoyue
6322eb31e7
[skip ci] Add comments for exported things in policy.go (#11876) 
Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
 2021-11-16 13:21:16 +08:00
congqixia
2b07876827
[skip ci] Fix lint for AvgAssignRegisterPolicy (#10965) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-10-30 21:03:52 +08:00
sunby
7fbb469081
Change channel's field to upper case to log details (#10567) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-10-26 20:15:41 +08:00
sunby
2de3131bb3
Change default balance policy (#10563) 
issue: #10198
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-10-26 19:40:44 +08:00
XuanYang-cn
23c7691d59
[skip ci]Update license for dc policy (#10590) 
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2021-10-25 19:48:23 +08:00
sunby
044b06421a
Use etcd to watch channel on datanode (#7115) 
issue: #6578
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-10-14 15:44:34 +08:00
congqixia
be5c492e5a
Add datacoord unit tests (#7197) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-08-20 17:50:12 +08:00
sunby
6e34f4c7f1
Add a seal policy which restrict the lifetime of a segment (#7172) 
issue: #7164
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-08-20 15:42:12 +08:00
sunby
ee38d14f5d
Fix channel lost after data node crash (#6545) 
If we start up 2 data nodes and one of them crashes. We expect that all
channels of crashed node will be allcoated to the alive node. But now we
discover that these channels are lost after data node crash. The reason
is we pass a NodeInfo with empty channel info. We fix it and improve log
print.

issue: #6501
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-07-15 16:38:31 +08:00
sunby
a8e5fd2024
Refactor cluster in dataservice (#6356) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-07-12 11:03:52 +08:00
congqixia
d6259d2dd6
Refactor policy (#6003) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-06-23 14:28:08 +08:00
sunby
c863a193de
Rename filename from dataservice to datacoordinator (#5946) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-06-22 10:42:07 +08:00