test/milvus
1
0
Fork 0
mirror of https://gitee.com/milvus-io/milvus.git synced 2024-12-02 20:09:57 +08:00
Code Issues Actions 12 Packages Projects Releases Wiki Activity
17,686 Commits 94 Branches 105 Tags 184 MiB
3421956afa
Commit Graph

224 Commits

Author SHA1 Message Date
SimFG
f9e2d00f91
Prevent exclusive consumer exception in pulsar (#25376) 
Signed-off-by: SimFG <bang.fu@zilliz.com>
 2023-07-12 17:26:30 +08:00
yihao.dai
4c93495587
Add segment size metric in querynode (#25406) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-07-12 14:26:28 +08:00
Xiaofan
dbf0130803
Faster garbage collect on compacted data (#25088) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2023-07-12 10:14:29 +08:00
congqixia
bcd3abf644
Separate segments/shards reduce latency metrics (#25475) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-07-11 18:18:28 +08:00
groot
96c987ed62
Bulkinsert supports partition keys (#25284) 
Signed-off-by: yhmo <yihua.mo@zilliz.com>
 2023-07-11 15:18:28 +08:00
yah01
8b06941da3
Reduce the load memory usage predict factor to 2 (#25469) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-07-11 13:06:28 +08:00
smellthemoon
d63323d117
Add rate limit and deny write in upsert (#25351) 
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2023-07-11 11:20:34 +08:00
Cai Yudong
5aad0ceec3
HNSW support binary vector (#25417) 
Signed-off-by: Yudong Cai <yudong.cai@zilliz.com>
 2023-07-10 18:18:28 +08:00
Xiaofan
b437195e17
Set compaction default value to be true (#25424) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2023-07-08 11:04:25 +08:00
wei liu
342cfcad46
decrease default value of group max nq (#25380) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-07-07 15:40:25 +08:00
wei liu
c84496a1ba
fix sq count metrics (#25287) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-07-06 08:46:26 +08:00
wei liu
951c312ef9
refine grafana metrics for replica selection (#25319) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-07-05 17:00:25 +08:00
congqixia
efdd71c640
Make cgo pool size larger than worker pool size (#25318) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-07-05 16:56:26 +08:00
foxspy
31173727b2
growing segment index memory opt & get vector bugfix (#25272) 
Signed-off-by: xianliang <xianliang.li@zilliz.com>
 2023-07-05 00:04:25 +08:00
chyezh
d7d61f529c
[Feature|Pick] enable scheduler policy and add user-task-polling policy (#24839) 
Signed-off-by: chyezh <ye.zhen@zilliz.com>
 2023-07-03 18:24:25 +08:00
Gao
de6e4817a2
Support dynamic tuning config (#25152) 
Signed-off-by: chasingegg <chao.gao@zilliz.com>
 2023-07-03 15:18:24 +08:00
smellthemoon
b30517d303
Enlarge timeout to prevent health check failure (#25173) 
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2023-06-29 17:10:23 +08:00
yihao.dai
37fe3393d1
Remove const to enable move semantics and improve schema util functions (#25193) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-06-29 11:04:22 +08:00
congqixia
1357ef7043
Merge get worker process for same nodeID (#25185) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-06-28 18:34:24 +08:00
wei liu
3a222e976d
fix consumer close may block forever (#25120) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-06-27 14:04:45 +08:00
SimFG
a1e6428bc7
Fix the kafka panic when sending the message to a closed channel (#25116) (#25117) 
Signed-off-by: SimFG <bang.fu@zilliz.com>
 2023-06-27 10:42:44 +08:00
wei liu
17796743dd
add metrics for replica selection (#25037) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-06-27 09:52:44 +08:00
xige-16
33c2012675
Add more metrics (#25081) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2023-06-26 17:52:44 +08:00
wei liu
7b999b42bd
enable config policy on replica selection (#25067) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-06-25 19:46:44 +08:00
jaime
18df2ba6fd
[Cherry-Pick] Support Database (#24769) 
Support Database(#23742)
Fix db nonexists error for FlushAll (#24222)
Fix check collection limits fails (#24235)
backward compatibility with empty DB name (#24317)
Fix GetFlushAllState with DB (#24347)
Remove db from global meta cache after drop database (#24474)
Fix db name is empty for describe collection response (#24603)
Add RBAC for Database API (#24653)
Fix miss load the same name collection during recover stage (#24941)

RBAC supports Database validation (#23609)
Fix to list grant with db return empty (#23922)
Optimize PrivilegeAll permission check (#23972)
Add the default db value for the rbac request (#24307)

Signed-off-by: jaime <yun.zhang@zilliz.com>
Co-authored-by: SimFG <bang.fu@zilliz.com>
Co-authored-by: longjiquan <jiquan.long@zilliz.com>
 2023-06-25 17:20:43 +08:00
chyezh
ccf3f0066f
[Pick] Enable max result window limit (#24986) 
Signed-off-by: chyezh <ye.zhen@zilliz.com>
 2023-06-25 14:42:43 +08:00
xige-16
04082b3de2
Migrate the ability to upload and download binlog to cpp (#22984) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2023-06-25 14:38:44 +08:00
congqixia
6a1eff3487
Fix kafka consumer double close (#25051) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-06-25 14:16:44 +08:00
congqixia
00f8f236d7
Revert close kafka internal producer (#25072) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-06-25 12:38:43 +08:00
congqixia
a3630e17a0
Refine grpc Status handling: retry legacy only for Unimplemented (#25041) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-06-25 10:30:43 +08:00
wei liu
0d51f55c48
enable config each collection has different rate limit (#24139) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-06-21 15:02:42 +08:00
PowderLi
3f4356df10
fix the spelling of field (#25008) 
Signed-off-by: PowderLi <min.li@zilliz.com>
 2023-06-21 14:00:42 +08:00
yihao.dai
9f5756f16e
Add cluster validation interceptor to resolve the ip-reuse issue (#25001) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-06-21 11:36:46 +08:00
congqixia
4fda4f3951
Close kafka internal consumer&producer properly (#24998) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-06-20 21:14:41 +08:00
Xiaofan
af1d84e5e1
Support sync when cp is lagged (#24989) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2023-06-20 10:00:41 +08:00
cai.zhang
c9e456c6eb
Remove metric_type check and fix some minor bugs (#24921) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
Co-authored-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2023-06-19 09:54:41 +08:00
cai.zhang
c81bdda55a
Hide autoindex params details (#24851) (#24935) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
Co-authored-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2023-06-16 14:30:40 +08:00
cqy123456
a519213316
Update knowhere version, update diskann api and generate cache nodes in build process (#24898) 
Signed-off-by: cqy123456 <qianya.cheng@zilliz.com>
 2023-06-16 14:20:39 +08:00
yihao.dai
c73219a54d
Limit the number of concurrent sync tasks and allow only one sync task for the same segment (#24881) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-06-16 14:14:39 +08:00
MrPresent-Han
ce5cb3c0c5
[skip e2e] modify default used balancer for 2.3 (#24937) 
Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
 2023-06-16 14:00:45 +08:00
smellthemoon
db31e88a73
Add length check when insert and upsert (#24759) 
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2023-06-15 10:24:38 +08:00
wayblink
bfae6b49af
Remove datanode timetick mq, use rpc to report instead (#23156) 
Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2023-06-14 14:16:38 +08:00
wei liu
a3437e0ab5
refactor replica selection and retry policy on channel (#24367) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-06-13 10:20:37 +08:00
yihao.dai
e81eb56b92
Add metric of waiting for tSafe latency (#24765) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-06-09 11:26:35 +08:00
congqixia
41af0a98fa
Use go-api/v2 for milvus-proto (#24770) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-06-09 01:28:37 +08:00
yihao.dai
2086b4f75d
Add RBAC for FlushAll (#24751) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-06-08 20:28:35 +08:00
yah01
ebd0279d3f
Check error by Error() and NoError() for better report message (#24736) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-06-08 15:36:36 +08:00
Enwei Jiao
d3af451d92
Upgrade golangci-lint (#24707) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2023-06-07 19:34:36 +08:00
Enwei Jiao
4048caa8ca
Use only one go.mod (#24669) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2023-06-07 14:22:36 +08:00
chyezh
1593278f9d
add independent nq limit option (#24632) 
Signed-off-by: chyezh <ye.zhen@zilliz.com>
 2023-06-07 10:38:36 +08:00
chyezh
f97127ae55
add nats mq wrappers (#24445) 
bug fixup, configurable natsmq, add unittest, pass e2e.



move natsmq to pkg project

Signed-off-by: chyezh <ye.zhen@zilliz.com>
Co-authored-by: yiwangdr <yiwangdr@gmail.com>
 2023-06-07 10:00:37 +08:00
Gao
ea62922829
Disable search params rewrite while searching with filter (#24625) 
Signed-off-by: chasingegg <chao.gao@zilliz.com>
 2023-06-06 14:28:35 +08:00
xige-16
732fe54775
Support partition Key (#24047) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2023-06-06 10:24:34 +08:00
congqixia
31880ab427
Use conc.PoolOption instead of ants.Option (#24585) 
- Add conc.PoolOption to setup conc.Pool
- Change panic default behavior
- Make future has error when job panicks

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-06-01 15:52:31 +08:00
xige-16
af42d7c2af
Fix crash when insert duplicate field datas (#24559) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2023-05-31 20:32:31 +08:00
smellthemoon
bb62cf4e60
Support upsert api rbac (#24550) 
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2023-05-31 19:01:09 +08:00
wei liu
71f2012488
enable collection rate limit when only global limit set (#24452) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-05-31 16:47:29 +08:00
yihao.dai
19164ea753
Make binlog size metric to collect healthy binlog only (#24557) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-05-31 16:45:29 +08:00
zhenshan.cao
f7924724b0
Add consistency_level paramter in search/query request (#24499) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2023-05-30 21:01:29 +08:00
MrPresent-Han
b09e7aeaf7
support detailed task metrics(#23414) (#24507) 
Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
 2023-05-30 14:59:28 +08:00
Jiquan Long
29ae1229b6
Support AutoIndex (#24387) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2023-05-29 20:35:28 +08:00
wei liu
1deac47069
reduce grpc client dial timout (#24427) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-05-29 10:03:28 +08:00
wei liu
8e3ba74648
fix qc service unstable ut (#24340) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-05-24 18:49:25 +08:00
cai.zhang
e206265d01
Fill is_dynamic after requery (#24345) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2023-05-24 11:09:26 +08:00
cai.zhang
d16e18fd34
Add output fields for search/query results (#24302) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2023-05-23 10:19:26 +08:00
cai.zhang
ce67d024ed
Fill is_dynamic flag for output field data (#24283) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2023-05-22 14:01:25 +08:00
wei liu
8965ea2a08
refine err msg about no available node in replica (#24256) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-05-22 11:59:26 +08:00
congqixia
7e9ef36de4
Fix TestConfigFromRemote/close_manager ut is not stable (#24279) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-05-22 09:59:25 +08:00
yihao.dai
3f96c335bb
Add receivedNQ metric and change receivedBytes metric to collection level (#24199) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-05-21 22:23:24 +08:00
smellthemoon
0b72cf2c2e
log out error (#24260) 
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2023-05-19 18:25:24 +08:00
congqixia
3a66e1de65
Use suite for integration test (#24253) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-05-19 15:57:24 +08:00
Jiquan Long
bd343550a5
Support to manage connections (#24224) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2023-05-19 12:51:23 +08:00
MrPresent-Han
7744573d3d
support parms for import maxfilesize(#24191) (#24192) 
Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
 2023-05-18 18:59:27 +08:00
Jiquan Long
a98c79b6a6
Fix flat index can be created with invalid metric type (#24180) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2023-05-18 11:07:26 +08:00
cai.zhang
008285f849
Support dynamic schema for create collection (#24176) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2023-05-18 09:33:24 +08:00
yah01
99cc24b283
Fix compaction doesn't support JSON data (#24151) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-05-17 19:09:22 +08:00
yihao.dai
7384d83d2c
Support rate limit based on growing segments size (#24121) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-05-17 09:57:22 +08:00
congqixia
73a181d226
Fix get vector it timeout and improve some string const usage (#24141) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-05-16 17:41:22 +08:00
cai.zhang
ccd685013a
Json key must be enclosed in double quotes (#24113) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2023-05-16 16:19:22 +08:00
yah01
3ea4a39078
Fix insert node doesn't handle JSON data while merging (#24126) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-05-16 16:09:23 +08:00
congqixia
e68f5cf08e
Fix integration tests logic (#24063) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-05-15 14:45:21 +08:00
wei liu
4c956fab73
enable config collection level rate limit (#24012) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-05-12 18:13:26 +08:00
yah01
7d0c47dd65
Check the length of JSON (#24035) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-05-12 11:33:20 +08:00
yah01
400364483d
Improve predict the resource usage (#23968) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-05-11 15:33:24 +08:00
congqixia
5aa9db0d38
Add collection level auto compaction enabled config (#24013) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-05-10 17:45:20 +08:00
cai.zhang
9715a850fa
Support expr with json field (#23804) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2023-05-10 10:19:19 +08:00
congqixia
de4b4dafef
Refresh etcd source config with Searializable option (#23954) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-05-09 11:52:40 +08:00
yihao.dai
fc010ee86b
Improve the metric of stored binlog size to the level of collection and segment (#23859) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-05-07 20:52:38 +08:00
Enwei Jiao
086f3bd748
Add it for refresh config (#23773) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2023-05-06 17:34:39 +08:00
congqixia
2cacda0d95
Fix Int8~Int32 empty result set type (#23852) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-05-06 14:16:39 +08:00
Jiquan Long
7be7e6f360
Refactor check logic of index parameters (#23856) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2023-05-06 10:40:39 +08:00
MrPresent-Han
b517bc9e6a
refine balance mechanism including:(#23454) (#23763) (#23791) 
1. balance granuity to replica to avoid influence unrelated replicas
2. avoid balance back and forth

Signed-off-by: MrPresent-Han <jamesharden11122@gmail.com>
 2023-05-04 12:22:40 +08:00
yihao.dai
8c060215e9
Add collection number quota per DB (#23656) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-04-28 11:02:35 +08:00
smellthemoon
45fbe1d1a7
Change proxy max shard num (#23777) 
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2023-04-27 22:20:35 +08:00
Gao
e467e0d451
Seperate reading build and load params for diskann (#23758) 
Signed-off-by: chasingegg <chao.gao@zilliz.com>
 2023-04-27 20:30:35 +08:00
yah01
eab94489ba
Refine the merge algorithm (#23767) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-04-27 18:26:35 +08:00
smellthemoon
912cf4ef0f
Change some configurations, include change the defaultChannelNum to 16 (#23617) 
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2023-04-27 14:26:35 +08:00
Xiaofan
016311ad48
Fix Has Collection Failed on datacoord (#23710) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2023-04-27 09:50:34 +08:00
yihao.dai
ed8836cd15
Add disk quota at the collection level (#23704) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-04-26 21:52:36 +08:00
SimFG
5cd21893c8
Fix superusers' password verification problem (#23733) 
Signed-off-by: SimFG <bang.fu@zilliz.com>
 2023-04-26 21:16:34 +08:00