test/milvus
1
0
Fork 0
mirror of https://gitee.com/milvus-io/milvus.git synced 2024-12-04 21:09:06 +08:00
Code Issues Actions 14 Packages Projects Releases Wiki Activity
15,349 Commits 96 Branches 105 Tags 186 MiB
94ffa8a275
Commit Graph

5629 Commits

Author SHA1 Message Date
wayblink
c198e6c2d9
Add some debug log to explore ocassionally channel name mismatch issue in create collection (#18414) 
Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2022-07-27 13:06:30 +08:00
congqixia
e9d777b336
Fix ParseSegmentIDBinlog panicks with bad input (#18413) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-07-26 19:32:30 +08:00
yah01
9621b6615d
Limit the concurrency level for single load request (#18400) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-07-26 15:18:30 +08:00
congqixia
128c66f301
Fix segment DmChannel in querycoord for compatibility to 2.0.2 (#18397) 
Reload segment dm channel if empty channel found

Patch originally authored by @czs007
Unit test added by @congqixia

Co-authored-by: zhenshan.cao <zhenshan.cao@zilliz.com>
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Co-authored-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2022-07-25 14:14:31 +08:00
congqixia
45746bb736
Add ShowConfiguration proto definition (#18364) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-07-23 17:14:28 +08:00
xing.zhao
d90313a1a8
Use chunkManager rather than minio.Client in datanode garbage collection (#18092) 
Signed-off-by: xingzhao <xing.zhao@zilliz.com>

Co-authored-by: xingzhao <xing.zhao@zilliz.com>
 2022-07-22 22:10:28 +08:00
wgcn
f192c6c298
fix the search metric of querynode (#18367) 
Signed-off-by: wanggang11335 <wanggang11335@autohome.com.cn>

fix the search metric of querynode

Co-authored-by: wanggang11335 <wanggang11335@autohome.com.cn>
 2022-07-22 17:38:29 +08:00
jaime
3ec147acfe
Upgrade confluent kafka go version to 1.9.1 (#18293) 
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
 2022-07-22 13:28:28 +08:00
jaime
9672eae62c
Unify metastore catalog interface (#17772) 
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
 2022-07-22 10:20:29 +08:00
congqixia
f37e07b0f8
Use Get method of request in GetMetrics to avoid panicking (#18356) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-07-21 20:52:27 +08:00
SimFG
74fd28df9c
Refine some datanode code (#18351) 
Signed-off-by: SimFG <bang.fu@zilliz.com>
 2022-07-21 15:52:27 +08:00
Xiaofan
a6e1ddb53d
Fix node is not assigned to replica (#18350) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-07-21 12:02:27 +08:00
bigsheeper
cef8b1e7cc
Enable jemalloc (#18349) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-07-20 22:22:31 +08:00
Ten Thousand Leaves
cd2f64d30c
Fix a nil pointer issue (#18343) 
/kind bug

issue: #18270
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-07-20 17:56:30 +08:00
congqixia
d90308e9a0
Handle topic not exist in rqm & mqtt seek (#18314) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-07-20 15:42:29 +08:00
Ten Thousand Leaves
0ebe407221
Make DataNode not panic if stale segment is not found during flush (#18322) 
/kind improvement

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-07-20 15:06:30 +08:00
wei liu
a79c5e2366
fix sync problem during reblance (#18332) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2022-07-20 15:02:30 +08:00
congqixia
be6ca6358d
Use serialized size before compression in log meta (#18336) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-07-19 21:30:30 +08:00
bigsheeper
c038cd7a48
Fix addSegment error, avoid the risk of mem leaks (#18309) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-07-19 20:58:28 +08:00
Jeng.Gwan
db71c74cc7
Fix leak of memory in ParseMallocInfo while assert false (#18311) 
Signed-off-by: xaxys <zheng.guan@zilliz.com>
 2022-07-19 14:26:33 +08:00
cai.zhang
315df1ff15
Reduce the timer time of assigning index task scheduler (#18319) 
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
 2022-07-19 14:24:29 +08:00
Jiquan Long
70246f82fd
Reduce chunk row size (#18320) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2022-07-19 13:50:28 +08:00
congqixia
f7bd054adb
Set EnableGarbageCollection to true by default (#18305) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-07-18 16:34:28 +08:00
Xiaofan
c97563590c
fix missing handling channels while a node down (#18250) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-07-18 13:06:28 +08:00
Jeng.Gwan
638f6c36e9
Support to get real row count of segment (#18115) 
Signed-off-by: xaxys <zheng.guan@zilliz.com>
 2022-07-18 09:58:28 +08:00
congqixia
f0846fb79b
Handles DropPartitionMsg in datanode flowgraph (#18292) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-07-15 17:12:27 +08:00
Xiaofan
6d82ef8c20
fix garbage collector err handling (#18277) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-07-15 13:54:26 +08:00
bigsheeper
d278213e4f
Check load request both in impl and task (#18276) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-07-15 10:24:28 +08:00
bigsheeper
73aef14820
Improve check of purge memory and change purge place (#18271) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-07-15 10:22:27 +08:00
yah01
bb2ad23e55
GetQuerySegmentInfo returns sealed loaded segments (#18255) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-07-14 16:44:27 +08:00
cai.zhang
93d9dfad1a
Ignore task when IndexNode load empty index meta (#18266) 
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
 2022-07-14 15:08:28 +08:00
cai.zhang
a47a414dce
Fix bug for IndexNode pod kill test (#18177) 
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
 2022-07-13 16:16:25 +08:00
xige-16
54d17bc5da
Fix query too slow when insert multi repeated pk data (#18231) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-07-13 10:22:26 +08:00
congqixia
e5fe46124b
Fix isCompacting flag lost after segment clone (#18248) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-07-13 10:08:28 +08:00
Zach
84d73bd618
add package name in log contents (#18251) 
Signed-off-by: Zach41 <zongmei.zhang@zilliz.com>
 2022-07-12 23:02:26 +08:00
Jeng.Gwan
a926a9eb66
Fix redundant string copy of PlaceholderGroup (#18236) 
Signed-off-by: xaxys <zheng.guan@zilliz.com>
 2022-07-12 15:48:25 +08:00
Cai Yudong
015a2f0866
Let growing segment call knowhere brute search API (#18227) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2022-07-12 11:58:25 +08:00
bigsheeper
1da970af18
Remove useless statsService in QueryNode (#17395) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-07-11 15:12:25 +08:00
congqixia
a9aff3a678
Allocate ID for each insertMsg (#18217) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-07-11 14:38:24 +08:00
yah01
08612ac56c
Fix QueryCoord unit test stucks (#18197) 
Signed-off-by: yah <yang.cen@zilliz.com>
 2022-07-11 12:44:25 +08:00
Xiaofan
653dce4948
Fix pulsar error check (#18207) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-07-11 10:56:25 +08:00
congqixia
e61f61a9bd
Dedup insert msgs by MsgID in mqtt MsgPack (#18180) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-07-08 20:42:22 +08:00
bigsheeper
22508f36d3
Get search field id from search plan and log if loaded index when search segments (#18183) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-07-08 20:18:22 +08:00
cai.zhang
90faf4a212
Fix bug for IndexCoord recycles index files (#18168) 
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
 2022-07-08 16:54:22 +08:00
bigsheeper
0fb5e9c1a9
Remove load collection check in Proxy (#18166) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-07-08 16:12:25 +08:00
Ten Thousand Leaves
ecc250c20e
Improve logs to better cover segment flush lifecycle (#18157) 
/kind improvements

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-07-08 14:50:21 +08:00
zhenshan.cao
72910d65e0
Fix inconsistency in DataCoord (#18170) (#18171) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2022-07-08 13:48:21 +08:00
XuanYang-cn
53734bef58
Add seek msgID in DataNode (#17881) 
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2022-07-08 10:18:28 +08:00
wayblink
43ae6791e6
Make VChannelInfo read compatible with 2.0.2 (#17995) 
Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2022-07-07 21:58:22 +08:00
codeman
5fda8261d0
Fix: Unable to list newly created credential users (#18117) (#18153) 
Signed-off-by: kejiang <ke.jiang@zilliz.com>

Co-authored-by: kejiang <ke.jiang@zilliz.com>
 2022-07-07 21:16:22 +08:00
cai.zhang
0c883b215b
Fix bug for IndexCoord ut (#18148) 
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
 2022-07-07 18:52:21 +08:00
cai.zhang
44e59d6318
Fix bug for clean etcd in QueryNCoord (#18145) 
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
 2022-07-07 18:32:21 +08:00
XuanYang-cn
d9e8231728
Filter sealed segments (#18065) 
- Remove FilterThreshold in DataNode
- Alter filter logic in flowgraph DDNode

See also: #17943

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2022-07-07 18:26:20 +08:00
cai.zhang
58a9a0b79a
Add license for new files (#18138) 
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
 2022-07-07 17:44:21 +08:00
cai.zhang
c1a47c8941
Use struct channel instead of bool chan (#18140) 
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
 2022-07-07 16:20:22 +08:00
congqixia
543197c2be
Fix handoff segment unit test uses wrong node id (#18137) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-07-07 15:22:21 +08:00
cai.zhang
0ad15a22c8
IndexCoord handle events correctly (#17878) 
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
 2022-07-07 14:44:21 +08:00
congqixia
8f1ba6a0bb
Concurrent load segment stats log when initialize dataSyncService (#18125) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-07-07 14:32:21 +08:00
Jiquan Long
6df47c7faf
Change pulsar compression type to zstd-faster (#18121) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2022-07-07 11:16:22 +08:00
Enwei Jiao
ca88e37c98
fix receive messages before waitToFinish (#18101) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2022-07-07 10:00:21 +08:00
Ten Thousand Leaves
d4a1e94f32
Fix flush failure caused by empty stale segments (#18097) 
/bug

issue: #18096
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-07-06 18:10:21 +08:00
SimFG
79523ff8d4
Fix the wrong log tag in the test case (#18103) 
Signed-off-by: SimFG <bang.fu@zilliz.com>
 2022-07-06 15:38:20 +08:00
Zach
1c9647ff31
Merging multiple shard query/search requests into one rpc request if QueryNode is the leader of multiple shards of Collection (#17588) 
Signed-off-by: Zach41 <zongmei.zhang@zilliz.com>
 2022-07-06 15:06:21 +08:00
Xiaofan
a9b1d71a8f
change the etcd compareAndSwap interface (#18068) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-07-06 13:54:21 +08:00
Jiquan Long
44024c1b1d
Replace pulsar compression type zstd with lz4 (#18085) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2022-07-05 22:06:21 +08:00
bigsheeper
696d747afa
Check if loaded in Proxy when LoadCollection (#18074) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-07-05 21:16:20 +08:00
zhagnlu
257da153ce
Fix core dump when nq has no topk result (#17923) (#18051) 
Signed-off-by: zhagnlu <lu.zhang@zilliz.com>

Co-authored-by: zhagnlu <lu.zhang@zilliz.com>
 2022-07-05 19:48:20 +08:00
Cai Yudong
5edeeb5fcd
Define new SubSearchResult constructor to avoid memory copy for growing (#18000) 
segment search

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2022-07-05 17:14:20 +08:00
bigsheeper
7d6624fcad
Improve flow graph node name and log msgID in QueryCoord (#18043) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-07-05 16:46:23 +08:00
zhenshan.cao
749558d55b
Fix bug: get correct number of cpu (#17973) (#18030) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2022-07-05 16:40:20 +08:00
congqixia
63230f0280
Fix data race in ShardCluster.SyncReplicas (#18054) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-07-05 13:16:20 +08:00
congqixia
d652fa567a
Return NotShardLeader error code when dml channel is not watched (#18055) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-07-05 13:08:19 +08:00
Xiaofan
b31f937301
Fix unsubscribe with subscription not exist (#18042) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-07-05 10:12:20 +08:00
Bingyi Sun
1fd3ded815
Add querycoord v2 dir and proto (#18032) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
 2022-07-04 16:56:20 +08:00
xige-16
0702993106
Fix query not empty after delete string (#17953) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-07-04 16:54:20 +08:00
yah01
39db27865f
Refine log, remove huge logs (#18036) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-07-04 16:34:20 +08:00
zhenshan.cao
1647bebaf4
Count growing segment number to estimate cpu usage (#17820) (#18027) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2022-07-04 15:10:20 +08:00
zhenshan.cao
9bd35dd84e
Refine logs to reduce print (#17759) (#18028) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2022-07-04 11:10:20 +08:00
Jiquan Long
6954a5ba3e
Fix search successfully with invalid metric type (#17977) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2022-07-01 22:28:23 +08:00
yah01
4def7047b4
Fix the scheduler enqueues concurrent load tasks (#17950) 
Signed-off-by: yah01 <yang.cen@zilliz.com>

related to #17850
 2022-07-01 19:54:19 +08:00
congqixia
566f5b5766
Make querynode handle segmentChangeInfo with multiple vchannel (#18009) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-07-01 19:20:20 +08:00
yah01
66a04450ad
Fix LoadBalance Sync all segments to shard leaders (#18002) 
Signed-off-by: yah <yang.cen@zilliz.com>
 2022-07-01 19:18:24 +08:00
yah01
27eca2881a
Clean segments as releasing collection (#17932) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-07-01 19:16:25 +08:00
zhenshan.cao
f33b090819
Fix bug: datarace in proxy unittest (#17990) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2022-07-01 17:56:21 +08:00
Jiquan Long
47c9e77155
Fix querycoord ut data race (#17980) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2022-07-01 10:58:18 +08:00
bigsheeper
641fb96958
Return malloc_trim result and log if malloc_trim successfully (#17959) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-07-01 10:44:18 +08:00
codeman
6ab850bedf
Fix update credential failure (#17963) (#17970) 
Signed-off-by: kejiang <ke.jiang@zilliz.com>

Co-authored-by: kejiang <ke.jiang@zilliz.com>
 2022-06-30 21:34:18 +08:00
congqixia
f4177f4434
Remove growing according to offline as well (#17968) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-06-30 20:14:19 +08:00
cai.zhang
01fc411566
Handle etcd compacted error (#17886) 
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
 2022-06-30 18:54:19 +08:00
bigsheeper
cdbd75d4dd
Log delta channel timetick in DataNode (#17947) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-06-30 17:44:18 +08:00
zhenshan.cao
0baeb609dc
Fix bugs in monitoring metrics (#17894) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2022-06-30 17:26:19 +08:00
Xiaofan
a803f9e4b0
Fix Wait Group Race in Msgstream (#17931) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-06-30 10:38:18 +08:00
yah01
ef6859a4d6
Refactor handoff handler (#17806) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
Co-authored-by: xiaofan-luan <xiaofan.luan@zilliz.com>

Co-authored-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-06-29 23:26:19 +08:00
codeman
95a3830dc6
fix password comparison too slow (#17874) (#17897) 
Signed-off-by: kejiang <ke.jiang@zilliz.com>

Co-authored-by: kejiang <ke.jiang@zilliz.com>
 2022-06-29 20:02:18 +08:00
XuanYang-cn
ff3c4ab2a8
Fix compaction panic (#17840) 
See also: #17823

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2022-06-29 16:54:17 +08:00
Bingyi Sun
24b3ad22d7
Fix msgstream close block. (#17880) 
issue: #17513
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
 2022-06-29 16:24:20 +08:00
congqixia
c179986e6c
Fix shard cluster segments data race (#17903) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-06-29 16:20:19 +08:00
congqixia
106f19a7b8
Fix querycoord cannot update deltaChannel info (#17901) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-06-29 15:38:17 +08:00
Cai Yudong
a001412e12
Replace faiss::MetricType with knowhere::MetricType (#17891) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2022-06-29 14:20:19 +08:00
jaime
762e3b78a6
support configurable sasl mechanisms for kafka (#17542) 
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
 2022-06-29 10:32:17 +08:00