Commit Graph

15043 Commits

Author SHA1 Message Date
binbin
9ad67e99dd
Update pymilvus version (#17667)
Signed-off-by: Binbin Lv <binbin.lv@zilliz.com>
2022-06-21 15:24:13 +08:00
ThreadDao
eeaae0799f
[test] Update some compaction test cases (#17658)
Signed-off-by: ThreadDao <yufen.zong@zilliz.com>
2022-06-21 15:10:12 +08:00
Jiquan Long
0282f69eb4
Add msg id to log in search path. (#17654)
issue: #17513
Add msg id to log in search path,
which should help us to trace a detailed request more easier.

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2022-06-21 13:30:12 +08:00
congqixia
37464a281d
Fix datasorter swap vector logic (#17661)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-06-21 11:38:12 +08:00
SimFG
eb1f0bc805
Add the consistency check tool (#17136)
Signed-off-by: SimFG <bang.fu@zilliz.com>
2022-06-21 10:28:12 +08:00
shaoyue
32a3ed0791
Add e2e test for RESTful APIs, fix json encoding for on proto_oneof fields (#17362)
Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
2022-06-20 22:06:12 +08:00
cai.zhang
ed94ecf847
Segments can't be compacted when they have reference lock (#17649)
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
2022-06-20 21:56:12 +08:00
cai.zhang
3610cc77bf
Fix bug for align segments (#17652)
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
2022-06-20 20:16:12 +08:00
zhuwenxing
c185747552
[skip e2e]Fix etcd version in first deployment for upgrading test (#17650)
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2022-06-20 19:12:12 +08:00
Bingyi Sun
4333a30f97
Use pulsar client go v0.6.8 (#17642)
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
2022-06-20 17:54:12 +08:00
cai.zhang
f123d657ed
Fix bug for dropping index (#17635)
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
2022-06-20 15:22:13 +08:00
Xiaofan
e36fce1fd4
Add log for mqtt msgstream seek (#17640)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-06-20 15:18:13 +08:00
wayblink
f2bd910df5
[skip e2e]Fix log mistake: WatchDmChannels -> WatchDeltaChannels (#17643)
Signed-off-by: wayblink <anyang.wang@zilliz.com>
2022-06-20 15:10:12 +08:00
congqixia
12b3a29db8
Fix segments info in drop virtual channel request is not set (#17641)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-06-20 13:40:12 +08:00
zhuwenxing
0bb415f593
[skip e2e]Update jenkinsfile for chaos and deploy test (#17630)
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
2022-06-20 11:16:13 +08:00
Xiaofan
d70f483806
Support rocksmq dynamic lru cache size (#17626)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-06-20 10:56:12 +08:00
ThreadDao
b491673218
[test] Add some compaction test cases (#17628)
Signed-off-by: ThreadDao <yufen.zong@zilliz.com>
2022-06-20 09:58:12 +08:00
Xiaofan
516d91470f
Fix Delta log loss after compaction (#17634)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-06-19 20:12:11 +08:00
congqixia
50cecc65ed
Fix mock querynode server session not revoked (#17229)
Revoke mock querynode server session when it's stopped
This PR reduces the running time of TestLoadBalanceIndexedSegmentsAfterNodeDown from 60+ seconds to less than 1+ seconds
Also related to #17212 #17215

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-06-18 18:22:12 +08:00
Xiaofan
144f7a2d80
Refine meta, remove redundant proxy data (#17631)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-06-18 12:50:11 +08:00
Letian Jiang
b15c24a554
Acquire reference lock of compacted from segments during index building (#17619)
- Accquire reference lock during index building
- Keep dropped segment record

Co-authored-by: Letian Jiang <letian.jiang@zilliz.com>
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Co-authored-by: Congqi Xia <congqi.xia@zilliz.com>
2022-06-18 01:24:11 +08:00
groot
7c69f4b338
Fix bulkload failed bug caused by dropping collection (#17624)
Signed-off-by: groot <yihua.mo@zilliz.com>
2022-06-17 21:14:11 +08:00
groot
371a44f018
Fix a bug of bulkload (#17620)
Signed-off-by: groot <yihua.mo@zilliz.com>
2022-06-17 20:52:11 +08:00
Jenny Li
d69a41f63d
[skip e2e] Upgrade chart to add kafka metrics for nightly (#17622)
Signed-off-by: Jenny Li <jing.li@zilliz.com>
2022-06-17 20:22:12 +08:00
congqixia
aa4b42c74b
Remove wait before release collection/partition (#17623)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-06-17 19:00:11 +08:00
wayblink
074ec3060a
Support return dropped segments info in GetSegmentInfo rpc (#17617)
Signed-off-by: wayblink <anyang.wang@zilliz.com>
2022-06-17 18:24:12 +08:00
cai.zhang
141a3bbe00
Mark deleted before drop index (#17490)
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
2022-06-17 18:08:12 +08:00
bigsheeper
92d06b2e30
Purge memory by the memory state and try to purge after each search (#17565)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-06-17 17:46:10 +08:00
congqixia
785a5a757f
Use segment version instead of ref cnt (#17609)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-06-17 17:38:12 +08:00
congqixia
cc3ecc4bd5
Make querycoord channel allocator respect context (#17552)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-06-17 16:02:12 +08:00
Jiquan Long
0e29c37499
Forbid system fields in user schema (#17613)
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2022-06-17 14:10:11 +08:00
yah01
f5fa93aa0b
Return err if failed to assign segments/channels to nodes (#17616)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-06-17 12:42:11 +08:00
SimFG
2008f60e50
Simplify the implementations of the ToPhysicalChannel and the ConvertChannelName func (#17605)
Signed-off-by: SimFG <bang.fu@zilliz.com>
2022-06-17 10:22:11 +08:00
yah01
0f87763682
Fix LoadBalance may not update the shard leader (#17608)
This happens probably with concurrent updating replicas,
some goroutines modify the nodes list of replicas,
and the others modify the shard leaders of replicas

Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-06-17 08:58:10 +08:00
binbin
380c788ec9
[skip e2e] Update compaction related test cases for code change (#17596)
Signed-off-by: Binbin Lv <binbin.lv@zilliz.com>
2022-06-16 21:20:11 +08:00
yanliang567
bea6fcd5fd
verify data_queryable state after bulk load (#17601)
Signed-off-by: yanliang567 <yanliang.qiao@zilliz.com>
2022-06-16 21:14:10 +08:00
cai.zhang
53a49a9255
GetIndexState only get the state generated call CreateIndex (#17549)
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
2022-06-16 20:12:11 +08:00
Bingyi Sun
00089de2c5
Use v0.6.7 pulsar go client (#17594)
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
2022-06-16 19:34:10 +08:00
yah01
3f42f5f345
Set the task state to TaskFailed if error occurs (#17598)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-06-16 19:18:11 +08:00
Jiquan Long
8e2797fe30
Print error when failed to broadcast message (#17595)
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2022-06-16 18:32:10 +08:00
shaoyue
04f836f516
Fix s3 delete file (#17574)
Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
2022-06-16 18:14:10 +08:00
Letian Jiang
be4077421e
Filter duplicated sealed segments which are both online and offline (#17593)
Signed-off-by: Letian Jiang <letian.jiang@zilliz.com>
2022-06-16 17:42:11 +08:00
Xiaofan
1f6fbf91b2
Fix pulsar unsubsribe issue (#17562)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-06-16 17:28:11 +08:00
Jiquan Long
2f66531fdf
Fix wrong IP distances (#17590)
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2022-06-16 16:34:11 +08:00
Jenny Li
0c2970f916
Add Image size after build docker image (#17591)
Signed-off-by: Jenny Li <jing.li@zilliz.com>
2022-06-16 16:28:11 +08:00
yah01
7d5c8c5f38
Fix bug not remove offline node (#17560)
The LoadBalance task won't remove the offline node if the node never load/watch any segment/dmchannel

Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-06-16 16:00:10 +08:00
Jenny Li
a6f0a23145
[skip e2e] Move feature label to exemptLabels (#17586)
Signed-off-by: Jenny Li <jing.li@zilliz.com>
2022-06-16 14:18:10 +08:00
groot
b5e62023ea
Return error when bulkload pending list hit limit (#17570)
Signed-off-by: groot <yihua.mo@zilliz.com>
2022-06-16 13:02:10 +08:00
wayblink
eb5b0b7fc8
Move SegmentInfo out of VchannelInfo, leave Id only to decrease kv size. Get complete SegmentInfo through RPC (#17441)
Resolves: #17233 #16047

Signed-off-by: wayblink <anyang.wang@zilliz.com>
2022-06-16 12:00:10 +08:00
Jenny Li
5cfc4a0fc8
[skip e2e] Upgrade codecov-action && add retry upload cov (#17575)
Signed-off-by: Jenny Li <jing.li@zilliz.com>
2022-06-16 09:52:10 +08:00