test/milvus
1
0
Fork 0
mirror of https://gitee.com/milvus-io/milvus.git synced 2024-12-04 04:49:08 +08:00
Code Issues Actions 14 Packages Projects Releases Wiki Activity
15,409 Commits 96 Branches 105 Tags 185 MiB
d08581d9b5
Commit Graph

5668 Commits

Author SHA1 Message Date
congqixia
0db281c4ff
Fix DataType::bool cast to double (#18690) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-08-17 16:14:49 +08:00
congqixia
d4c54d96b0
Dedup output fields for task query (#18673) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-08-16 18:16:48 +08:00
aoiasd
be933a75c0
Add log for NoReplicaAvailable Error in QueryCoord.GetShardLeaders (#18662) 
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2022-08-16 10:56:48 +08:00
Cai Yudong
d878cc2ec0
Rename real_topK_per_nq_ to topks_per_nq_prefix_sum_ in SearchResult struct (#18634) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2022-08-16 00:32:48 +08:00
SimFG
447ce8ee48
Handle some apis about the default role (#18647) 
Signed-off-by: SimFG <bang.fu@zilliz.com>

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2022-08-15 16:40:48 +08:00
XuanYang-cn
4c86cc63ba
Fix DN flowgraph delete buffer logs (#18657) 
1. Remove logs about not existing segments.
2. Group logs by timestamp.
3. Log changed segments only.
4. Pair the segments reference lock and unlock log by taskID.

Resolves: #18655

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2022-08-15 16:28:48 +08:00
aoiasd
8a70f2badb
Add ShowConfigurations for all remaining session (#18594) 
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2022-08-12 13:20:39 +08:00
codeman
174130c7d6
fix wrong operation model for user (#18614) (#18615) 
Signed-off-by: kejiang <ke.jiang@zilliz.com>

Signed-off-by: kejiang <ke.jiang@zilliz.com>
Co-authored-by: kejiang <ke.jiang@zilliz.com>
 2022-08-11 20:22:38 +08:00
xige-16
e40061b864
Update binlog event format (#18347) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>

Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-08-11 14:06:38 +08:00
codeman
6c3dbf0a07
Support meta storing on database (#17236) (#18541) 
Signed-off-by: kejiang <ke.jiang@zilliz.com>

Signed-off-by: kejiang <ke.jiang@zilliz.com>
Co-authored-by: kejiang <ke.jiang@zilliz.com>
 2022-08-11 12:12:38 +08:00
xige-16
111c608513
Add script to auto clang format (#18559) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>

Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-08-11 11:04:37 +08:00
xige-16
dbb96a8b70
Fix pk2offset occupy too much memory (#18325) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>

Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-08-11 10:48:38 +08:00
jaime
0e95f8a690
Refine indexcoord logs (#18589) 
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>

Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
 2022-08-11 10:00:38 +08:00
xige-16
cd04258122
Abort execute timeout search/query (#17729) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>

Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-08-11 09:58:41 +08:00
xige-16
29384ddbc0
Fix querycoord handle offline node with wrong nodeID (#18590) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-08-10 21:10:37 +08:00
SimFG
1366f130a2
Add a specific KeyNotExistError err for the metastore load interface (#18582) 
Signed-off-by: SimFG <bang.fu@zilliz.com>
 2022-08-10 17:20:41 +08:00
jaime
efa5dfaa7b
Fix Parse binlog path failure (#18583) 
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
 2022-08-10 17:04:37 +08:00
Jiquan Long
e7b3bacbec
Refine catalog of partition & alias. (#18546) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2022-08-10 10:22:38 +08:00
Ten Thousand Leaves
097f144db1
Improve slow Etcd logs by specifying keys (#18547) 
/kind improvement

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-08-09 21:14:37 +08:00
Ten Thousand Leaves
12023e8569
Fix a potential nullptr issue with metric (#18540) 
/kind improvement

related to: #18530

but not the root cause

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-08-09 21:12:42 +08:00
congqixia
179b496824
Add cgo worker pool for querynode (#18461) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-08-09 16:34:37 +08:00
bigsheeper
002f509808
Remove deprecated cacheSize param (#18566) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-08-09 15:54:36 +08:00
SimFG
c267ad612d
Support Role-Based Access Control II (#18528) 
Signed-off-by: SimFG <bang.fu@zilliz.com>
 2022-08-05 16:28:35 +08:00
Xiaofan
b427450593
Remove Load Delta Channel Request (#18497) 
Fix #18393

remove watch delta channel task related logic

Co-authored-by: xiaofan-luan <xiaofan.luan@zilliz.com>
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Co-authored-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-08-04 17:50:37 +08:00
jaime
c345df41a7
Fix kafka ut (#18527) 
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
 2022-08-04 16:02:35 +08:00
SimFG
ff0200210a
Support Role-Based Access Control (#18425) 
Signed-off-by: SimFG <bang.fu@zilliz.com>
 2022-08-04 11:04:34 +08:00
jaime
4a32c1bb8e
Enable kafka ut with a mocked kafka server (#18366) 
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
 2022-08-03 19:06:35 +08:00
congqixia
28aadc988d
Use vchannel name in datanode subName (#18519) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-08-03 17:10:36 +08:00
Cai Yudong
7cd37fc6dd
Merge FloatSearch() and BinarySearch() into SearchOnGrowing() (#18498) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2022-08-03 11:56:36 +08:00
aoiasd
a7ce0eb6f6
Add ShowConfigurations to QueryNode (#18426) 
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2022-08-03 01:12:33 +08:00
jaime
6e7831470c
Revert settings of Pulsar address and refine config code (#18494) 
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
 2022-08-02 21:26:33 +08:00
XuanYang-cn
43c7c1ff03
Fix indexcheck division by zero bug (#18482) 
See also: #18479

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2022-08-02 16:04:34 +08:00
congqixia
ab461c6e5e
Release collection resources when all partition released (#18488) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-08-02 11:40:34 +08:00
congqixia
318cdbebe9
Add heap for dmlChannels to get next channel (#18451) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-08-01 16:36:33 +08:00
congqixia
f0fe8dae0b
Close Node/Segment detector when close ShardCluster (#18476) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-08-01 13:50:33 +08:00
Enwei Jiao
283f5731d2
config from etcd (#18421) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2022-08-01 10:04:33 +08:00
jaime
c9174d55ba
Refine merge operation during compacting phase (#18399) 
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
 2022-07-28 14:52:31 +08:00
zhenshan.cao
a287a2b3fd
Return empty result in advance if all data filtered out (#18329) (#18438) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2022-07-28 12:36:30 +08:00
wei liu
a3ec44cd37
support dynamic change log level through http (#18430) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2022-07-28 11:36:30 +08:00
wayblink
c198e6c2d9
Add some debug log to explore ocassionally channel name mismatch issue in create collection (#18414) 
Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2022-07-27 13:06:30 +08:00
congqixia
e9d777b336
Fix ParseSegmentIDBinlog panicks with bad input (#18413) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-07-26 19:32:30 +08:00
yah01
9621b6615d
Limit the concurrency level for single load request (#18400) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-07-26 15:18:30 +08:00
congqixia
128c66f301
Fix segment DmChannel in querycoord for compatibility to 2.0.2 (#18397) 
Reload segment dm channel if empty channel found

Patch originally authored by @czs007
Unit test added by @congqixia

Co-authored-by: zhenshan.cao <zhenshan.cao@zilliz.com>
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Co-authored-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2022-07-25 14:14:31 +08:00
congqixia
45746bb736
Add ShowConfiguration proto definition (#18364) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-07-23 17:14:28 +08:00
xing.zhao
d90313a1a8
Use chunkManager rather than minio.Client in datanode garbage collection (#18092) 
Signed-off-by: xingzhao <xing.zhao@zilliz.com>

Co-authored-by: xingzhao <xing.zhao@zilliz.com>
 2022-07-22 22:10:28 +08:00
wgcn
f192c6c298
fix the search metric of querynode (#18367) 
Signed-off-by: wanggang11335 <wanggang11335@autohome.com.cn>

fix the search metric of querynode

Co-authored-by: wanggang11335 <wanggang11335@autohome.com.cn>
 2022-07-22 17:38:29 +08:00
jaime
3ec147acfe
Upgrade confluent kafka go version to 1.9.1 (#18293) 
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
 2022-07-22 13:28:28 +08:00
jaime
9672eae62c
Unify metastore catalog interface (#17772) 
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
 2022-07-22 10:20:29 +08:00
congqixia
f37e07b0f8
Use Get method of request in GetMetrics to avoid panicking (#18356) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-07-21 20:52:27 +08:00
SimFG
74fd28df9c
Refine some datanode code (#18351) 
Signed-off-by: SimFG <bang.fu@zilliz.com>
 2022-07-21 15:52:27 +08:00
Xiaofan
a6e1ddb53d
Fix node is not assigned to replica (#18350) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-07-21 12:02:27 +08:00
bigsheeper
cef8b1e7cc
Enable jemalloc (#18349) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-07-20 22:22:31 +08:00
Ten Thousand Leaves
cd2f64d30c
Fix a nil pointer issue (#18343) 
/kind bug

issue: #18270
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-07-20 17:56:30 +08:00
congqixia
d90308e9a0
Handle topic not exist in rqm & mqtt seek (#18314) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-07-20 15:42:29 +08:00
Ten Thousand Leaves
0ebe407221
Make DataNode not panic if stale segment is not found during flush (#18322) 
/kind improvement

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-07-20 15:06:30 +08:00
wei liu
a79c5e2366
fix sync problem during reblance (#18332) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2022-07-20 15:02:30 +08:00
congqixia
be6ca6358d
Use serialized size before compression in log meta (#18336) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-07-19 21:30:30 +08:00
bigsheeper
c038cd7a48
Fix addSegment error, avoid the risk of mem leaks (#18309) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-07-19 20:58:28 +08:00
Jeng.Gwan
db71c74cc7
Fix leak of memory in ParseMallocInfo while assert false (#18311) 
Signed-off-by: xaxys <zheng.guan@zilliz.com>
 2022-07-19 14:26:33 +08:00
cai.zhang
315df1ff15
Reduce the timer time of assigning index task scheduler (#18319) 
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
 2022-07-19 14:24:29 +08:00
Jiquan Long
70246f82fd
Reduce chunk row size (#18320) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2022-07-19 13:50:28 +08:00
congqixia
f7bd054adb
Set EnableGarbageCollection to true by default (#18305) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-07-18 16:34:28 +08:00
Xiaofan
c97563590c
fix missing handling channels while a node down (#18250) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-07-18 13:06:28 +08:00
Jeng.Gwan
638f6c36e9
Support to get real row count of segment (#18115) 
Signed-off-by: xaxys <zheng.guan@zilliz.com>
 2022-07-18 09:58:28 +08:00
congqixia
f0846fb79b
Handles DropPartitionMsg in datanode flowgraph (#18292) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-07-15 17:12:27 +08:00
Xiaofan
6d82ef8c20
fix garbage collector err handling (#18277) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-07-15 13:54:26 +08:00
bigsheeper
d278213e4f
Check load request both in impl and task (#18276) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-07-15 10:24:28 +08:00
bigsheeper
73aef14820
Improve check of purge memory and change purge place (#18271) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-07-15 10:22:27 +08:00
yah01
bb2ad23e55
GetQuerySegmentInfo returns sealed loaded segments (#18255) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-07-14 16:44:27 +08:00
cai.zhang
93d9dfad1a
Ignore task when IndexNode load empty index meta (#18266) 
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
 2022-07-14 15:08:28 +08:00
cai.zhang
a47a414dce
Fix bug for IndexNode pod kill test (#18177) 
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
 2022-07-13 16:16:25 +08:00
xige-16
54d17bc5da
Fix query too slow when insert multi repeated pk data (#18231) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-07-13 10:22:26 +08:00
congqixia
e5fe46124b
Fix isCompacting flag lost after segment clone (#18248) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-07-13 10:08:28 +08:00
Zach
84d73bd618
add package name in log contents (#18251) 
Signed-off-by: Zach41 <zongmei.zhang@zilliz.com>
 2022-07-12 23:02:26 +08:00
Jeng.Gwan
a926a9eb66
Fix redundant string copy of PlaceholderGroup (#18236) 
Signed-off-by: xaxys <zheng.guan@zilliz.com>
 2022-07-12 15:48:25 +08:00
Cai Yudong
015a2f0866
Let growing segment call knowhere brute search API (#18227) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2022-07-12 11:58:25 +08:00
bigsheeper
1da970af18
Remove useless statsService in QueryNode (#17395) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-07-11 15:12:25 +08:00
congqixia
a9aff3a678
Allocate ID for each insertMsg (#18217) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-07-11 14:38:24 +08:00
yah01
08612ac56c
Fix QueryCoord unit test stucks (#18197) 
Signed-off-by: yah <yang.cen@zilliz.com>
 2022-07-11 12:44:25 +08:00
Xiaofan
653dce4948
Fix pulsar error check (#18207) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-07-11 10:56:25 +08:00
congqixia
e61f61a9bd
Dedup insert msgs by MsgID in mqtt MsgPack (#18180) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-07-08 20:42:22 +08:00
bigsheeper
22508f36d3
Get search field id from search plan and log if loaded index when search segments (#18183) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-07-08 20:18:22 +08:00
cai.zhang
90faf4a212
Fix bug for IndexCoord recycles index files (#18168) 
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
 2022-07-08 16:54:22 +08:00
bigsheeper
0fb5e9c1a9
Remove load collection check in Proxy (#18166) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-07-08 16:12:25 +08:00
Ten Thousand Leaves
ecc250c20e
Improve logs to better cover segment flush lifecycle (#18157) 
/kind improvements

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-07-08 14:50:21 +08:00
zhenshan.cao
72910d65e0
Fix inconsistency in DataCoord (#18170) (#18171) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2022-07-08 13:48:21 +08:00
XuanYang-cn
53734bef58
Add seek msgID in DataNode (#17881) 
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2022-07-08 10:18:28 +08:00
wayblink
43ae6791e6
Make VChannelInfo read compatible with 2.0.2 (#17995) 
Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2022-07-07 21:58:22 +08:00
codeman
5fda8261d0
Fix: Unable to list newly created credential users (#18117) (#18153) 
Signed-off-by: kejiang <ke.jiang@zilliz.com>

Co-authored-by: kejiang <ke.jiang@zilliz.com>
 2022-07-07 21:16:22 +08:00
cai.zhang
0c883b215b
Fix bug for IndexCoord ut (#18148) 
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
 2022-07-07 18:52:21 +08:00
cai.zhang
44e59d6318
Fix bug for clean etcd in QueryNCoord (#18145) 
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
 2022-07-07 18:32:21 +08:00
XuanYang-cn
d9e8231728
Filter sealed segments (#18065) 
- Remove FilterThreshold in DataNode
- Alter filter logic in flowgraph DDNode

See also: #17943

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2022-07-07 18:26:20 +08:00
cai.zhang
58a9a0b79a
Add license for new files (#18138) 
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
 2022-07-07 17:44:21 +08:00
cai.zhang
c1a47c8941
Use struct channel instead of bool chan (#18140) 
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
 2022-07-07 16:20:22 +08:00
congqixia
543197c2be
Fix handoff segment unit test uses wrong node id (#18137) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-07-07 15:22:21 +08:00
cai.zhang
0ad15a22c8
IndexCoord handle events correctly (#17878) 
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
 2022-07-07 14:44:21 +08:00
congqixia
8f1ba6a0bb
Concurrent load segment stats log when initialize dataSyncService (#18125) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-07-07 14:32:21 +08:00
Jiquan Long
6df47c7faf
Change pulsar compression type to zstd-faster (#18121) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2022-07-07 11:16:22 +08:00
Enwei Jiao
ca88e37c98
fix receive messages before waitToFinish (#18101) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2022-07-07 10:00:21 +08:00
Ten Thousand Leaves
d4a1e94f32
Fix flush failure caused by empty stale segments (#18097) 
/bug

issue: #18096
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-07-06 18:10:21 +08:00