test/milvus
1
0
Fork 0
mirror of https://gitee.com/milvus-io/milvus.git synced 2024-12-04 12:59:23 +08:00
Code Issues Actions 14 Packages Projects Releases Wiki Activity
16,971 Commits 96 Branches 105 Tags 185 MiB
9d35a5f943
Commit Graph

6741 Commits

Author SHA1 Message Date
congqixia
eb690ef033
Fix querynodev2 stop logic (#23487) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-04-18 17:56:30 +08:00
congqixia
d83654c33f
Add Close method for ChannelManager in datacoord (#23493) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-04-18 17:54:31 +08:00
wayblink
b23723535c
Add log in AddImportSegment.getChannelLatestMsgID (#23480) 
Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2023-04-18 17:50:31 +08:00
congqixia
b5a73e6d1a
Add OWNERS files for querycoordv2 sub pkgs (#23489) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-04-18 15:52:30 +08:00
yah01
5aa0ddf77b
Refine field storage of sealed segment (#23464) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-04-18 11:24:30 +08:00
zhenshan.cao
4a32b842e8
Improve the check logic of channel remove (#23473) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2023-04-18 02:58:30 +08:00
wei liu
3ad9ff7a9a
fix release segment failed (#23462) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-04-17 18:46:30 +08:00
Bennu
f63d52944f
Update knowhere version from main to commit_id (#23437) 
Signed-off-by: Bennu-Li <yunmei.li@zilliz.com>
 2023-04-17 16:32:33 +08:00
aoiasd
695cc769df
Fix watch event timeout when IO Pool busy (#23293) 
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2023-04-17 11:02:30 +08:00
Jiquan Long
adfa415b74
Optimize log trace (#23431) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2023-04-17 11:00:30 +08:00
aoiasd
f603456b72
remove segment from datanode meta when segment dropped (#23398) 
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2023-04-17 10:58:30 +08:00
cqy123456
7ff6a3a246
Set query node thread pool size in milvus.yaml (#23286) 
Signed-off-by: cqy123456 <qianya.cheng@zilliz.com>
 2023-04-17 10:44:29 +08:00
yah01
7c4cafc83c
Fix task merge doesn't work (#23405) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-04-14 18:18:29 +08:00
Enwei Jiao
288582b2d9
Optimize mac build time (#23390) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2023-04-14 17:56:29 +08:00
zhenshan.cao
a88178a5ea
Add slow log for read (#23272) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2023-04-14 15:30:29 +08:00
yah01
bed8d6892e
Protect the QueryCoord meta from stale data migrated from old version (#23412) 
Fix #23411

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-04-14 14:42:28 +08:00
congqixia
ba84f52119
Fix watcher loop quit and channel shouldDrop logic (#23402) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-04-14 09:54:28 +08:00
congqixia
a9a5f76f74
Fix AddDistribution unit test (#23418) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-04-13 20:46:29 +08:00
yah01
dc0eb433ae
Add IoPoolSize config (#23360) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-04-13 14:50:28 +08:00
cai.zhang
43a9e175a3
Exit component process when session key is deleted (#21658) (#22164) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2023-04-12 20:12:28 +08:00
zhenshan.cao
dc5abe086c
Revert "Restrain gc files by collection (#22504)" (#23380) 
This reverts commit fa86de530d.

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2023-04-12 19:38:28 +08:00
yah01
296380d6e6
Support async refresh (#23107) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-04-12 15:06:28 +08:00
Jiquan Long
a940df94c7
Forbid counting entities with pagination (#23375) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2023-04-12 14:22:34 +08:00
congqixia
6cd73d4e2a
Wait safe signal before release growing (#23358) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-04-11 17:56:31 +08:00
Jiquan Long
ad7f3d4c3e
Support Count(*) on querynodev2 (#23321) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2023-04-11 15:40:31 +08:00
congqixia
0527d2706c
Fix offline set always nil (#23340) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-04-11 14:42:30 +08:00
wei liu
dbbd703667
fix balance generate unexpected task (#23299) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-04-11 14:38:30 +08:00
congqixia
f1f8ce01de
Remove querynode v1 package (#23289) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-04-11 11:28:31 +08:00
yihao.dai
0b354cbab1
Fix timetick block caused by dml task pop failed (#23277) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-04-11 11:00:31 +08:00
congqixia
03a4f60b6e
Fix knowhere cpp header files name (#23330) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-04-10 20:00:30 +08:00
XuanYang-cn
b90fa5f459
Refine codes of datanode buffer (#23168) 
This PR refines deltabuffer of datanode:
- Add last sync time for compacted segment, see also: #23210
- Ensure all deltabuffermanager handles all delete related operations
- Change usedMemory to atomic.Int64
- Remove allocator in delete buffer

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2023-04-10 18:42:30 +08:00
Jiquan Long
d85f673a95
Fix time tick lag too much (#23265) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2023-04-09 16:40:29 +08:00
cai.zhang
9288020da3
Fix bug for batch delete files on gcp of minio (#23052) (#23083) (#23090) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2023-04-09 16:22:30 +08:00
cai.zhang
bd65ef8e95
Fix bug for type params when indexing (#23242) (#23250) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2023-04-09 16:18:29 +08:00
Xiaofan
680ad482b7
Check balance checker chore to 10s (#23304) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2023-04-09 16:14:32 +08:00
MrPresent-Han
ba02e70301
fix bug when syncing distribution without schema and remove unused logic(#23085) (#23216) 
Signed-off-by: MrPresent-Han <jamesharden11122@gmail.com>
 2023-04-07 19:32:29 +08:00
wei liu
9f127dae47
enable balance channel (#23227) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-04-07 19:06:28 +08:00
congqixia
132c010271
Fix proxy panicking when QueryCoord offline (#23254) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-04-07 17:30:28 +08:00
Cai Yudong
55202ce7d9
Support show knowhere prometheus metrics (#23102) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2023-04-07 15:34:29 +08:00
Cai Yudong
5f4673fd16
Optimize unittest to save runtime (#23248) 
Signed-off-by: Yudong Cai <yudong.cai@zilliz.com>
 2023-04-07 14:20:29 +08:00
congqixia
32be374c33
Fix MetaCache ut internal not changed (#23276) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-04-07 13:20:28 +08:00
jaime
c9d0c157ec
Move some modules from internal to public package (#22572) 
Signed-off-by: jaime <yun.zhang@zilliz.com>
 2023-04-06 19:14:32 +08:00
Jiquan Long
8530dfadb4
Fix metric RootCoordInsertChannelTimeTick (#23236) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2023-04-06 15:30:29 +08:00
Jiquan Long
2c1c3aecfc
Fix metric ProxySyncTimeTickLag (#23235) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2023-04-06 15:28:33 +08:00
congqixia
8db24e0a83
Support queryHook in querynodev2 (#23140) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-04-06 14:32:29 +08:00
aoiasd
1a485044a9
fix NumConsumers metric not work (#23167) 
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2023-04-05 08:18:28 +08:00
congqixia
424570b2b8
Remove annoy related unit tests (#23229) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-04-05 08:14:28 +08:00
congqixia
4a53018e5f
Remove legacy annoy_inner_error (#23220) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-04-04 18:08:24 +08:00
Cai Yudong
ef63e64ded
Remove ANNOY index type (#23189) 
Signed-off-by: Yudong Cai <yudong.cai@zilliz.com>
 2023-04-04 16:30:27 +08:00
congqixia
3af3fe52d8
Remove partition target check in querynodev2 (#23188) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-04-04 13:36:32 +08:00
Cai Yudong
9ad6d9f1a0
Rename RAFT_IVF_FLAT/RAFT_IVF_PQ to GPU_IVF_FLAT/GPU_IVF_PQ (#23194) 
Signed-off-by: Yudong Cai <yudong.cai@zilliz.com>
 2023-04-04 11:44:26 +08:00
yah01
7da870f512
Remove useCustomConfig and simpilify the session type (#23166) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-04-03 20:10:24 +08:00
yihao.dai
17d3190fcc
Fix DescribeCollection failed with undefined msg type (#23192) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-04-03 19:52:24 +08:00
Enwei Jiao
4d2e379b09
Add prometheus-cpp (#23199) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2023-04-03 19:22:23 +08:00
smellthemoon
ce91f61e5a
Pre-create topics (#21907) 
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2023-04-03 16:44:23 +08:00
congqixia
42168f67fc
Add OWNERS file for qnv2 (#23196) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-04-03 16:22:24 +08:00
congqixia
e3e3ac1525
Use peek segment info in querynodev2 (#23183) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-04-03 16:10:24 +08:00
yah01
2a760c108c
Add metrics for QueryNodeV2 (#23163) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-04-03 15:24:24 +08:00
yihao.dai
b6f69fe7f2
Check if loaded before delegator search/query (#23162) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-04-03 14:26:23 +08:00
yihao.dai
9cecdc4f4e
Fix querynode panic (#23181) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-04-03 14:18:24 +08:00
MrPresent-Han
afd874b736
enhance segment balance by considering global rowCount(##22914) (#23056) 
Signed-off-by: MrPresent-Han <jamesharden11122@gmail.com>
Co-authored-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2023-04-03 14:16:25 +08:00
yihao.dai
a0ca4d6108
Improve search/query logs in Proxy (#23144) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-04-03 10:54:23 +08:00
Andy Pan
b8c1859aed
Bump ants to v2.7.2 for performance improvement (#22936) 
Signed-off-by: Andy Pan <i@andypan.me>
Signed-off-by: Andy Pan <panjf2000@gmail.com>
 2023-04-02 08:54:23 +08:00
Ten Thousand Leaves
1596abf6e8
Add lock for segment manager during bulk insert (#23111) 
/kind bug

issue: #23106

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2023-03-31 20:16:24 +08:00
XuanYang-cn
be0827417a
Fix buffer calculate memorysize negative (#23152) 
Along with the following small patches:
- And fix logging dup segmentIDs for delete buffer status
- Add String method for PriorityQueue to debug

See also: #23105

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2023-03-31 18:30:23 +08:00
congqixia
127867b873
Add ratedgroup for some info/warning log (#23095) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-03-31 15:22:23 +08:00
aoiasd
bd5fab1e53
Remove merge policy of proxy RoundRobin policy (#23021) 
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2023-03-31 14:36:22 +08:00
smellthemoon
22aeb72eba
Fix LoadSegment req don't fill in collection id (#23136) 
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2023-03-31 11:28:23 +08:00
Cai Yudong
7612c75c47
Let RAFT_IVF_PQ param accept m=0 (#23134) 
Signed-off-by: Yudong Cai <yudong.cai@zilliz.com>
 2023-03-31 11:22:22 +08:00
yah01
75737c65ac
Refine error handle of QueryCoord (#23068) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-03-31 10:54:29 +08:00
zhenshan.cao
1287ca699a
Refine usage of TimeRecorder.Record (#23142) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2023-03-30 18:56:22 +08:00
yihao.dai
a182594bf5
Add FlushAll support (#22942) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-03-30 11:28:25 +08:00
wei liu
74da53c027
fix update load percentage (#23054) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-03-30 10:48:23 +08:00
Jiquan Long
0f2c3ac832
Optimize log (#23121) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2023-03-30 10:44:22 +08:00
Enwei Jiao
50dab2d394
Add tracing for retrieve (#23033) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2023-03-29 18:10:02 +08:00
aoiasd
970dcb3161
Fix panic because timestamp was nil (#23096) 
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2023-03-29 17:20:04 +08:00
Xiaofan
653d9ecf91
Change RocksMQ page size to 64M (#23078) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2023-03-29 17:08:04 +08:00
yah01
73f1f15c1c
Fix DataCoord's meta may be broken with DataNode of prior version (#23030) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-03-29 16:58:02 +08:00
Cai Yudong
3febb5e45a
Remove gpu mode param check (#23074) 
Signed-off-by: Yudong Cai <yudong.cai@zilliz.com>
 2023-03-29 14:24:01 +08:00
wei liu
e2096965c7
fix leader view (#23038) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-03-29 14:06:02 +08:00
smellthemoon
b1992e84cc
Delete dup check (#23076) 
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2023-03-29 10:06:01 +08:00
yah01
dc6d4b913a
Fix partitions may be not recovered with double load partitions (#23061) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-03-28 21:38:02 +08:00
smellthemoon
846de72fb3
fix to reduce memory (#23023) 
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2023-03-28 21:36:06 +08:00
yah01
826c1fb850
Support mmap on QueryNodeV2 (#23073) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-03-28 21:30:05 +08:00
Xiaofan
d0510210df
Add config for disable BF load (#22996) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2023-03-28 20:06:10 +08:00
XuanYang-cn
9691315c21
Fix flush deltabuf with segmentID=0 (#23059) 
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2023-03-28 19:14:01 +08:00
wei liu
ed60b434fa
fix unstable ut (#23070) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-03-28 19:06:01 +08:00
aoiasd
5d172d0f4f
Refine errors of datanode (#22852) 
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2023-03-28 19:04:00 +08:00
zhenshan.cao
e3c3c949c4
Fix datarace of flow_graph_manager_test.go in package datanode (#23043) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2023-03-28 10:42:00 +08:00
Bingyi Sun
d6bfe8fbb9
Change frequent logs to debug level (#23012) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: sunby <bingyi.sun@zilliz.com>
 2023-03-28 07:12:01 +08:00
zhenshan.cao
d55c860383
Align the maximum dim of the diskann index and collection (#23039) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2023-03-28 00:06:00 +08:00
yah01
ae0f467c02
Fix segment/channel may be re-loaded/subscribed (#22969) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-03-27 18:28:00 +08:00
Enwei Jiao
ef57ba3bac
Remove easylogging and replace by GLog (#22979) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2023-03-27 18:22:02 +08:00
yihao.dai
b21f03682a
Fix flowgraph manager test (#23009) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-03-27 18:00:05 +08:00
congqixia
4008ee6d60
Remove datanode ddNode forward delta logic (#23005) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-03-27 14:18:00 +08:00
yah01
081572d31c
Refactor QueryNode (#21625) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
Co-authored-by: Congqi Xia <congqi.xia@zilliz.com>
Co-authored-by: aoiasd <zhicheng.yue@zilliz.com>
 2023-03-27 00:42:00 +08:00
cai.zhang
977943463e
Check index params and type params (#22988) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2023-03-26 22:15:59 +08:00
Xiaofan
5a914aedf3
Make Datacoord TT to be configurable (#22956) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2023-03-24 22:24:04 +08:00
XuanYang-cn
da58ba6a49
Fix endless appending sids (#22977) 
See also: #22976

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2023-03-24 18:00:03 +08:00
Cai Yudong
8aebc6f3b7
Remove faiss GPU index support (#22966) 
Signed-off-by: Yudong Cai <yudong.cai@zilliz.com>
 2023-03-24 17:53:58 +08:00