Commit Graph

120 Commits

Author SHA1 Message Date
wayblink
eb5b0b7fc8
Move SegmentInfo out of VchannelInfo, leave Id only to decrease kv size. Get complete SegmentInfo through RPC (#17441)
Resolves: #17233 #16047

Signed-off-by: wayblink <anyang.wang@zilliz.com>
2022-06-16 12:00:10 +08:00
cai.zhang
ea5041aec2
Acquiring the segment reference lock on task level (#17544)
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
2022-06-15 21:38:10 +08:00
congqixia
f9553970f9
Add BindContext function for querycoord task scheduler (#17531)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-06-14 17:12:09 +08:00
Enwei Jiao
a5b008acec
ignore getReplica's error when handle rebalanceTask (#17469)
Signed-off-by: Enwei Jiao <jiaoew2011@gmail.com>
2022-06-10 14:50:08 +08:00
yah01
cc69c5cdd3
Make Cluster interface's methods called outside public (#17315)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-06-02 13:16:05 +08:00
yah01
f5bd519e49
Add retry mechanism for NodeDown LoadBalance (#17306)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-06-01 20:00:03 +08:00
cai.zhang
bcf3b7426a
Add distributed lock for segment refgerence (#16782)
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
2022-05-31 16:36:03 +08:00
congqixia
c88514bc49
Remove not used QueryChannel in Proxy and Query Cluster (#16856)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-05-30 19:50:04 +08:00
yah01
5872c5afb6
Fix updating shard leaders may lost some modifications (#17218)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-05-26 17:14:02 +08:00
yah01
de0ba6d495
Fix GetQuerySegmentInfo() returns incorrect result after LoadBalance (#17190)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-05-25 15:17:59 +08:00
Jiquan Long
75ca64f8c7
Refine task type logs to string (#17196)
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2022-05-25 03:08:00 +08:00
Bingyi Sun
ffaead6ad9
Add load meta in hand off task. (#17179)
issue: #16842

Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
2022-05-24 18:24:00 +08:00
Bingyi Sun
86728490a2
Fix partition not found (#17132)
issue: #16842

Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
2022-05-20 19:37:57 +08:00
yah01
7746a5b742
Add NodeIds field for QuerySegmentInfo (#17121)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-05-20 18:03:58 +08:00
yah01
dcfe472586
Fix LoadBalance doesn't save the modification to replicas' shards (#17064)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-05-19 16:51:57 +08:00
bigsheeper
9eeec4a2d5
Add collection load cache and InvalidateCollMetaCache by collID (#16882)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-05-19 10:13:56 +08:00
yah01
33c855dcd2
Fix LoadBalance doesn't remove the source nodes from segment (#17051)
If the triggerCondition isn't NodeDown, the removing won't happen.

Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-05-18 11:55:56 +08:00
Bingyi Sun
59bc0a7000
Add some log. (#16978)
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
2022-05-16 17:15:55 +08:00
yah01
e38c6f6c44
Fix load the same segments multiple times for manual LoadBalance (#16921)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-05-16 15:41:56 +08:00
yah01
2d0f908dba
Fix updating segments' NodeIds correctly after LoadBalance (#16854)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-05-10 15:47:53 +08:00
yah01
c82e2453eb
Modify the replicas' shard info after load balance (#16785)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-05-05 21:15:50 +08:00
yah01
20936f78da
Fix LoadBalance failed to sync segments to shard leader when the leader is offline (#16770)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-05-05 16:25:50 +08:00
yah01
93416d5524
Fix LoadBalance loads the same segment multiple times (#16734)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-04-29 15:19:47 +08:00
yah01
206b554d49
Sync the distribution of segments to shard leaders (#16685)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-04-28 16:01:48 +08:00
yah01
60f7fef3b3
Remove offline nodes from replica info after LoadBalanceTask done (#16653)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-04-26 18:23:46 +08:00
Xiaofan
c1ff0cec8f
Change QueryCoord Log level (#16590)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-04-26 11:29:54 +08:00
congqixia
69252f812d
Implement memory replica in Proxy, QueryNode and QueryCoord (#16470)
Related to #16298 #16291 #16154
Co-authored-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: yangxuan <xuan.yang@zilliz.com>
Co-authored-by: yah01 <yang.cen@zilliz.com>
Co-authored-by: Letian Jiang <letian.jiang@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-04-20 16:15:41 +08:00
XuanYang-cn
ad41e2a082
Tidy getQueryChannel for readability (#16096)
- Remove useless codes in query_coord.go
- Tidy slice initialization in LoadCollection task
- Add ut for getQueryChannel

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2022-03-21 11:25:24 +08:00
Jiquan Long
ffdad82b9c
Set load info in reschedule tasks (#16065)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2022-03-16 11:43:21 +08:00
zhenshan.cao
f3eeecf146
Reduce the number of metrics (#16030)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-03-15 21:51:21 +08:00
Jiquan Long
c984c4ea73
Set load meta info in load balance task (#16060)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2022-03-15 20:37:21 +08:00
congqixia
6a66faa2fb
Explictly specify Load operation meta (#16027)
Fix #15996

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-03-14 09:50:01 +08:00
yah01
7124e91d53
Refactor assigning loading segments task in QueryCoord (#15895)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-03-07 19:23:58 +08:00
Cai Yudong
3e862ecdfb
Merge msgChannelConfig and knowhereConfig into commonConfig (#15843)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-03-04 11:17:56 +08:00
cai.zhang
88527fc0a9
Fix metrics label conflict (#15839)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2022-03-03 16:05:57 +08:00
bigsheeper
e1bfd14db1
Add Prometheus metrics for QueryCoord (#15606)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Co-authored-by: jingkl <34296482+jingkl@users.noreply.github.com>
2022-02-28 16:51:55 +08:00
xige-16
1e8e3eb912
Fix wrong flushSegmentInfos in watchdmChannel request (#15740)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-02-24 19:23:52 +08:00
xige-16
55cdc5aa35
Get indexInfo and segment size by queryCoord (#14207)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-02-08 21:57:46 +08:00
Cai Yudong
4bccc85534
Use MsgChannelConfig in GlobalParams for all components (#15163)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-02-02 00:35:43 +08:00
xige-16
6336e2326c
Set triggerCondition when queryCoord reload loadbalanceTask meta (#15380)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-01-25 17:26:13 +08:00
xige-16
dfc6670f0f
Unsubscribe channel after query node down (#15230)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-01-17 17:37:37 +08:00
xige-16
384d3169d3
Don't remove nodeInfo when querycoord panic at loadBalanceTask (#15189)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-01-13 17:59:34 +08:00
xige-16
4e956ee10c
Panic when recover querynode failed (#15161)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-01-12 17:43:37 +08:00
xige-16
a304684b55
Delete useless var in watchDeletaChannelTask (#14641)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-01-04 19:21:31 +08:00
xige-16
d4ac010165
Fix not load delete log when handoff segment (#14758)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-01-04 14:27:18 +08:00
xige-16
95f0e9a4a6
Fix proxy send query before queryNode watchQueryChannel and search timeout (#14640)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-12-30 22:51:21 +08:00
xige-16
b9730cb564
Fix not load delete log when handoff segment (#14613)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-12-30 19:45:20 +08:00
xige-16
b063bfda8f
Add index info in loadSegmentRequest (#14524)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-12-30 19:09:33 +08:00
xige-16
903e67b0ec
Fix recovery failed after query node down (#14449)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-12-29 12:15:21 +08:00
xige-16
e3771bee3d
Fix data loss after balance task rescheduled (#14096)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-12-24 10:59:31 +08:00