yah01
39db27865f
Refine log, remove huge logs ( #18036 )
...
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-07-04 16:34:20 +08:00
yah01
4def7047b4
Fix the scheduler enqueues concurrent load tasks ( #17950 )
...
Signed-off-by: yah01 <yang.cen@zilliz.com>
related to #17850
2022-07-01 19:54:19 +08:00
yah01
66a04450ad
Fix LoadBalance Sync all segments to shard leaders ( #18002 )
...
Signed-off-by: yah <yang.cen@zilliz.com>
2022-07-01 19:18:24 +08:00
yah01
27eca2881a
Clean segments as releasing collection ( #17932 )
...
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-07-01 19:16:25 +08:00
Jiquan Long
47c9e77155
Fix querycoord ut data race ( #17980 )
...
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2022-07-01 10:58:18 +08:00
cai.zhang
01fc411566
Handle etcd compacted error ( #17886 )
...
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
2022-06-30 18:54:19 +08:00
yah01
ef6859a4d6
Refactor handoff handler ( #17806 )
...
Signed-off-by: yah01 <yang.cen@zilliz.com>
Co-authored-by: xiaofan-luan <xiaofan.luan@zilliz.com>
Co-authored-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-06-29 23:26:19 +08:00
congqixia
106f19a7b8
Fix querycoord cannot update deltaChannel info ( #17901 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-06-29 15:38:17 +08:00
yah01
797218a8ad
Fix handoff isn't processed correctly ( #17738 )
...
Signed-off-by: yah01 <yang.cen@zilliz.com>
Co-authored-by: Congqi Xia <congqi.xia@zilliz.com>
Co-authored-by: Congqi Xia <congqi.xia@zilliz.com>
2022-06-27 22:08:17 +08:00
cai.zhang
ef0fc37ad7
Fixed the bug that IndexCoord lost some meta information ( #17818 )
...
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
2022-06-27 21:52:17 +08:00
wayblink
46e0e2658b
Move datacoord related methods from meta to globalMetaBroker ( #17812 )
...
Signed-off-by: wayblink <anyang.wang@zilliz.com>
2022-06-27 21:04:17 +08:00
congqixia
6840486efc
Fix load progress complete before syncReplica ( #17828 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-06-27 20:52:16 +08:00
Bingyi Sun
9d2e5aacbc
Fix missing dml nodeIDs when recovering from old version ( #17800 )
...
issue: #17416
Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: sunby <bingyi.sun@zilliz.com>
2022-06-26 18:08:16 +08:00
yah01
8388478ef3
SyncReplicaSegments syncs all segments ( #17774 )
...
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-06-24 23:24:15 +08:00
yah01
94a5122044
Fix LoadBalance removes source nodes from DmChannel ( #17744 )
...
Only NodeDown LoadBalance should remove source nodes
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-06-24 12:40:14 +08:00
wayblink
1a336bbbf1
Fix: WatchDmChannelsRequest can be too large to save in etcd ( #17722 )
...
Signed-off-by: wayblink <anyang.wang@zilliz.com>
2022-06-24 10:54:15 +08:00
yah01
b4f21259ef
Fix SegmentChangeInfo is not set properly ( #17701 )
...
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-06-22 21:08:13 +08:00
yah01
a077bad84a
Make updating replica atomic, balance plan idempotent ( #17686 )
...
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-06-22 13:40:13 +08:00
yah01
1e87b54542
Add the offlines nodes previously crashed to offline nodes channel ( #17663 )
...
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-06-21 16:08:13 +08:00
wayblink
f2bd910df5
[skip e2e]Fix log mistake: WatchDmChannels -> WatchDeltaChannels ( #17643 )
...
Signed-off-by: wayblink <anyang.wang@zilliz.com>
2022-06-20 15:10:12 +08:00
congqixia
50cecc65ed
Fix mock querynode server session not revoked ( #17229 )
...
Revoke mock querynode server session when it's stopped
This PR reduces the running time of TestLoadBalanceIndexedSegmentsAfterNodeDown from 60+ seconds to less than 1+ seconds
Also related to #17212 #17215
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-06-18 18:22:12 +08:00
wayblink
074ec3060a
Support return dropped segments info in GetSegmentInfo rpc ( #17617 )
...
Signed-off-by: wayblink <anyang.wang@zilliz.com>
2022-06-17 18:24:12 +08:00
congqixia
785a5a757f
Use segment version instead of ref cnt ( #17609 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-06-17 17:38:12 +08:00
congqixia
cc3ecc4bd5
Make querycoord channel allocator respect context ( #17552 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-06-17 16:02:12 +08:00
yah01
f5fa93aa0b
Return err if failed to assign segments/channels to nodes ( #17616 )
...
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-06-17 12:42:11 +08:00
yah01
0f87763682
Fix LoadBalance may not update the shard leader ( #17608 )
...
This happens probably with concurrent updating replicas,
some goroutines modify the nodes list of replicas,
and the others modify the shard leaders of replicas
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-06-17 08:58:10 +08:00
yah01
3f42f5f345
Set the task state to TaskFailed if error occurs ( #17598 )
...
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-06-16 19:18:11 +08:00
Xiaofan
1f6fbf91b2
Fix pulsar unsubsribe issue ( #17562 )
...
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-06-16 17:28:11 +08:00
yah01
7d5c8c5f38
Fix bug not remove offline node ( #17560 )
...
The LoadBalance task won't remove the offline node if the node never load/watch any segment/dmchannel
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-06-16 16:00:10 +08:00
wayblink
eb5b0b7fc8
Move SegmentInfo out of VchannelInfo, leave Id only to decrease kv size. Get complete SegmentInfo through RPC ( #17441 )
...
Resolves : #17233 #16047
Signed-off-by: wayblink <anyang.wang@zilliz.com>
2022-06-16 12:00:10 +08:00
cai.zhang
ea5041aec2
Acquiring the segment reference lock on task level ( #17544 )
...
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
2022-06-15 21:38:10 +08:00
congqixia
f9553970f9
Add BindContext function for querycoord task scheduler ( #17531 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-06-14 17:12:09 +08:00
Enwei Jiao
a5b008acec
ignore getReplica's error when handle rebalanceTask ( #17469 )
...
Signed-off-by: Enwei Jiao <jiaoew2011@gmail.com>
2022-06-10 14:50:08 +08:00
Xiaofan
66f26943f8
Add Err code when task rollback ( #17472 )
...
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-06-09 21:08:07 +08:00
congqixia
97a871cc82
Make querycoord segment allocator respect context ( #17452 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-06-09 20:14:06 +08:00
Letian Jiang
dfaed5acdd
Add QueryServiceAvailable field in ShowCollections ( #17456 )
...
Signed-off-by: Letian Jiang <letian.jiang@zilliz.com>
2022-06-09 18:20:07 +08:00
yah01
a2d2ad88bd
Make assigning segments faster ( #17377 )
...
This improve the Load performance,
and let the LoadBalance fails fast, which allows us to retry it timely
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-06-06 16:52:05 +08:00
xige-16
8c69790383
Fix lost delete msg caused by loadSegment after watchDeltaChannel ( #17308 )
...
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-06-02 16:56:04 +08:00
yah01
cc69c5cdd3
Make Cluster interface's methods called outside public ( #17315 )
...
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-06-02 13:16:05 +08:00
yah01
f5bd519e49
Add retry mechanism for NodeDown LoadBalance ( #17306 )
...
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-06-01 20:00:03 +08:00
cai.zhang
bcf3b7426a
Add distributed lock for segment refgerence ( #16782 )
...
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
2022-05-31 16:36:03 +08:00
congqixia
c88514bc49
Remove not used QueryChannel in Proxy and Query Cluster ( #16856 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-05-30 19:50:04 +08:00
yah01
b09359b12f
Remove useless collection ID in error message ( #17269 )
...
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-05-30 14:40:01 +08:00
xige-16
3a63d6c98e
Fix load timeout in chaos ( #17241 )
...
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-05-26 22:28:03 +08:00
Letian Jiang
f2a27e0e64
Retry GetShardLeaders until service available or timeout ( #17183 )
...
Signed-off-by: Letian Jiang <letian.jiang@zilliz.com>
2022-05-26 20:28:02 +08:00
yah01
5872c5afb6
Fix updating shard leaders may lost some modifications ( #17218 )
...
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-05-26 17:14:02 +08:00
congqixia
7409bfc56d
Make allocateNode run async in case of block offline event ( #17185 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-05-25 18:53:59 +08:00
yah01
de0ba6d495
Fix GetQuerySegmentInfo() returns incorrect result after LoadBalance ( #17190 )
...
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-05-25 15:17:59 +08:00
congqixia
37d7d7baf8
Fix node state data race in querycoord ( #17198 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-05-25 10:43:59 +08:00
Jiquan Long
75ca64f8c7
Refine task type logs to string ( #17196 )
...
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2022-05-25 03:08:00 +08:00