test/milvus
1
0
Fork 0
mirror of https://gitee.com/milvus-io/milvus.git synced 2024-12-01 03:18:29 +08:00
Code Issues Actions 11 Packages Projects Releases Wiki Activity
18,602 Commits 94 Branches 105 Tags 184 MiB
69a71e11d1
Commit Graph

281 Commits

Author SHA1 Message Date
congqixia
93a877f55e
Make qcv2 target&leader observer execute in parallel (#27844) 
- Add `taskDispatcher` to submit and run task async safely
- Change `LeaderObeserver` and `TargetObserver` schedule and manual check action to submitting task into dispatcher
- Fix logic problem in collection observer when manual check return false

See also #27494

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-10-24 10:14:11 +08:00
Xiaofan
2ea7579dbb
Reduce rpc size for GetRecoveryInfoV2 (#27483) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2023-10-23 21:44:09 +08:00
zhagnlu
6060dd7ea8
Add chunk manager request timeout (#27692) 
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2023-10-23 20:08:08 +08:00
congqixia
4a6790b63a
Make KeyLock generic (#27854) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-10-23 17:40:11 +08:00
SimFG
9b0ecbdca7
Support to replicate the mq message (#27240) 
Signed-off-by: SimFG <bang.fu@zilliz.com>
 2023-10-20 14:26:09 +08:00
zhenshan.cao
020ad9a6bc
Rectify wrong exception messages associated with Array datatype (#27769) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2023-10-19 17:24:07 +08:00
smellthemoon
4b0ec156b3
Set channel work pool size in datanode (#27728) 
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2023-10-19 08:28:08 +08:00
yah01
9467de79c3
Use pool to block exceeded tasks (#27767) 
Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-10-19 03:30:06 +08:00
yihao.dai
49b3a12804
Return newly defined merr instead of grpc unimplemented err (#27751) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-10-18 15:32:11 +08:00
congqixia
2f201c25e2
Remove deprecated io/ioutil usage (#27747) 
`io/ioutil` package is deprecated, use `io`,`os` package replacement
also added golangci-lint rule to block future reference

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
Co-authored-by: guoguangwu <guoguangwu@magic-shield.com>
 2023-10-17 20:32:09 +08:00
Futao Wei
599012a340
Implement SearchByPks path for Search (#25882) 
Signed-off-by: unfode <forrest.futao.wei@gmail.com>
 2023-10-16 15:34:08 +08:00
wayblink
e3f2122618
Expose metrics of stanby coordinators (#27698) 
Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2023-10-16 15:04:09 +08:00
wei liu
43f2452f4b
speed up shard leader cache update frequency (#27640) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-10-16 10:52:10 +08:00
yah01
be980fbc38
Refine state check (#27541) 
Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-10-11 21:01:35 +08:00
foxspy
2b46bd1f08
fix diskann searchCache param compatibility (#27557) 
Signed-off-by: xianliang <xianliang.li@zilliz.com>
 2023-10-11 16:57:34 +08:00
yah01
2df9908c6a
Map old error code to new error (#27616) 
Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-10-11 12:43:33 +08:00
yah01
3759857bc5
Refine Proxy errors (#27499) 
Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-10-09 10:09:33 +08:00
XuanYang-cn
56c94cdfa7
Add channel manager in DataNode (#27308) 
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2023-10-08 21:37:33 +08:00
yah01
41495ed266
Improve the error message for getting all indexes of collection (#27389) 
Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-10-08 21:23:32 +08:00
PowderLi
8d3069b1db
update openssl to 3.1.2 (#27399) 
deal with root path's normalization

Signed-off-by: PowderLi <min.li@zilliz.com>
 2023-10-08 19:17:31 +08:00
congqixia
5d558623fe
Add revive sub-lints and fix existing problems (#27495) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-10-07 20:53:38 +08:00
congqixia
80eb5434d3
Fix var-naming caused by old PRs (#27501) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-10-07 14:43:32 +08:00
yah01
8394b3a1ec
Block creating new error from status reason (#27426) 
Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-10-07 11:29:32 +08:00
yah01
63ac43a3b8
Refine errors for import (#27379) 
Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-09-30 10:31:28 +08:00
yihao.dai
106c17f304
Make read ahead policy in ChunkCache configurable (#27291) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-09-28 15:47:27 +08:00
congqixia
258e1ccd66
Refine querynode scheduler lifetime (#26915) 
This PR refines scheduler lifetime control:
- Move private tri-state into lifetime package
- Make scheduler block incoming "Add" task
- Make scheduler Stop wait until all previously accepted task done

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-09-28 10:21:26 +08:00
MrPresent-Han
505e8509b7
refine the default values for segcore threadpool size (#27406) 
/kind improvement

Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
 2023-09-27 20:43:28 +08:00
wei liu
4071132f6a
reload loading collection when qc recover (#27300) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-09-27 11:43:28 +08:00
SimFG
9fb4c27a90
Fix rootcoord failure to start when reaching the limit of role num (#27361) 
Signed-off-by: SimFG <bang.fu@zilliz.com>
 2023-09-27 11:03:26 +08:00
congqixia
a3dd2756cf
Add predicates for TxnKV operations (#27365) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-09-27 10:21:26 +08:00
yah01
6539a5ae2c
Refine DataCoord status (#27262) 
Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-09-26 17:15:27 +08:00
cai.zhang
30a6c6072e
Fix bug for loading array data panic (#27343) 
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2023-09-26 12:23:26 +08:00
wayblink
7dd0be1b2c
Enable bulkinsert binlog data with partitionkey (#27241) 
Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2023-09-26 10:25:25 +08:00
wei liu
27046e242b
add timeout for graceful stop (#27326) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-09-26 10:15:25 +08:00
foxspy
5db4a0489e
dynamic index version control (#27335) 
Co-authored-by: longjiquan <jiquan.long@zilliz.com>
 2023-09-25 21:39:27 +08:00
foxspy
370b6fde58
milvus support multi index engine (#27178) 
Co-authored-by: longjiquan <jiquan.long@zilliz.com>
 2023-09-22 09:59:26 +08:00
congqixia
67a471e09c
Fix gofumpt issue in scann_checker_test.go (#27305) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-09-22 00:39:26 +08:00
Gao
ab2d8dd0a7
Add ScaNN index param checker (#27268) 
Signed-off-by: chasingegg <chao.gao@zilliz.com>
 2023-09-21 14:51:25 +08:00
SimFG
26f06dd732
Format the code (#27275) 
Signed-off-by: SimFG <bang.fu@zilliz.com>
 2023-09-21 09:45:27 +08:00
yihao.dai
fe01d54eca
Set kafka read timeout to 10s and make it configurable (#27238) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-09-20 22:01:27 +08:00
smellthemoon
d1f6825b86
Specify componet ip (#27161) 
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2023-09-20 11:49:25 +08:00
yah01
9baff1b81c
Rename error to match the nameing rules (#27214) 
Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-09-20 10:59:24 +08:00
Xiaofan
6635398a6d
Fix Bin log concurrency by adding a pool (#27189) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2023-09-19 18:05:22 +08:00
cai.zhang
a362bb1457
Support array datatype (#26369) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2023-09-19 14:23:23 +08:00
XuanYang-cn
10116b85ac
Release before remove when releasing flowgraphs (#27191) 
GetAndRemove removes the fg from manager immediately,
while the flowgraph is still releasing. This PR will remove
the fg from flowgraphManager AFTER flowgraphs released.

- Add Remove for ConcurrentMap
- Move collections() into flowgraph manager

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2023-09-19 11:53:22 +08:00
yah01
b9ab18d692
Replace error code with merr for RateLimiter (#27176) 
Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-09-19 10:37:23 +08:00
congqixia
cc9974979f
Add staticcheck linter and fix existing problems (#27174) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-09-19 10:05:22 +08:00
PowderLi
4feb3fa7c6
support azure (#26398) 
Signed-off-by: PowderLi <min.li@zilliz.com>
 2023-09-19 10:01:23 +08:00
yihao.dai
4b2802033d
Fix datanode panic due to concurrent compaction and delete processing (#27167) 
Co-authored-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2023-09-18 19:35:29 +08:00
Enwei Jiao
8e17bc3013
Use merr to handle segcore errors (#27137) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2023-09-18 14:07:22 +08:00
MrPresent-Han
7939f0e7d5
enable ctx traceId for assignsegment on dc(#26972) (#27108) (#27030) 
Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
 2023-09-18 11:39:20 +08:00
xige-16
488b423e1b
Fix high cpu usage caused by proto.size (#27054) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2023-09-17 20:55:21 +08:00
yah01
0f2883ef9d
Optimize the generic zero value (#27102) 
allocate by declare,
this make it possible to allocate the value in stack,
faster and less presure for GC

Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-09-14 19:05:20 +08:00
congqixia
f0d0651989
Do not reset connection immediately if grpc code is Canceled or DeadlineExceeded (#27014) 
We found lots of connection reset & canceled due to recent retry change
Current implementation resets connection no matter what the error code is
To sync behavior to previous retry, skip reset connection only if cancel error happens too much.

Also adds a config item for minResetInterval for grpc reset connection

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-09-13 15:01:18 +08:00
chyezh
791e6ef6c6
[Improvement] add pulsar metrics and fix timeout (#26907) 
Signed-off-by: chyezh <ye.zhen@zilliz.com>
 2023-09-13 12:03:19 +08:00
yah01
00c65fa0d7
Refine QueryNode errors (#27013) 
Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-09-12 16:07:18 +08:00
yah01
45d9fb5929
Fix wrong error codes & names (#26904) 
Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-09-12 14:41:18 +08:00
congqixia
9e0977622d
Remove config content output before credential mask implemented (#26993) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-09-11 19:33:17 +08:00
congqixia
44f308fff5
Handle nil status in merr.OK (#26940) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-09-08 18:43:16 +08:00
Xu Tong
9166011c4a
Add float16 vector (#25852) 
Signed-off-by: Writer-X <1256866856@qq.com>
 2023-09-08 10:03:16 +08:00
XuanYang-cn
7f1ae35e72
Add timeout in dispatcher, AsConsumer and Seek (#26686) 
See also: #25309

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2023-09-08 09:51:17 +08:00
congqixia
af5c01082b
Refine delegator lifetime control (#26881) 
- Add SafeChan interface in lifetime package
- Embed SafeChan into  interface
- Replace private lifetime struct in delegator package with
- Refine delegator on-going task lifetime control and wait all accepted task done
- Fix potential goroutine leakage from  if delegator closed concurrently

/kind improvement

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-09-07 10:11:15 +08:00
yiwangdr
337edc321b
tikv integration (#26246) 
Signed-off-by: yiwangdr <yiwangdr@gmail.com>
 2023-09-07 07:25:14 +08:00
SimFG
28681276e2
Improve the retry of the rpc client (#26795) 
Signed-off-by: SimFG <bang.fu@zilliz.com>
 2023-09-06 17:43:14 +08:00
congqixia
7e2fa55440
Fix misuse of waitgroup in metrics mutex ut (#26865) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-09-06 12:39:14 +08:00
Enwei Jiao
fb0705df1b
Decouple basetable and componentparam (#26725) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2023-09-05 10:31:48 +08:00
wei liu
1097776477
stop heartbeat if reach heartbeat limit (#26728) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-09-04 17:51:48 +08:00
XuanYang-cn
b2e7cbdf4b
Remove TimeTravel in compactor (#26785) 
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2023-09-04 17:41:48 +08:00
zhagnlu
7056e9c0f7
Increase minio log level to avoid unnecessary log (#26776) 
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2023-09-04 17:25:48 +08:00
yihao.dai
4340cbfba2
Merge syncCPLagTooBehind policy into syncPeriodically policy (#26713) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-09-04 14:37:47 +08:00
yah01
3349db4aa7
Refine errors to remove changes breaking design (#26521) 
Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-09-04 09:57:09 +08:00
congqixia
e8f1b1736e
Remove log.Error(err.error())-style log (#26783) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-09-01 13:09:01 +08:00
smellthemoon
e56b0018e4
Remove merr mask logic to make it ok to compare resp.status with origin error (#26773) 
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2023-09-01 11:09:00 +08:00
Enwei Jiao
d206ebf220
Change default config files (#26780) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2023-09-01 09:19:00 +08:00
Cai Yudong
8dc16b599b
Add binary metric types SUBSTRUCTURE/SUPERSTRUCTURE back (#26766) 
Signed-off-by: Yudong Cai <yudong.cai@zilliz.com>
 2023-08-31 20:07:00 +08:00
XuanYang-cn
86e9f1818f
Change default compaction timeout to 15mins (#26757) 
See also: #26566

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2023-08-31 18:07:00 +08:00
Gao
7e36f819b4
Make autoindex config clean (#26732) 
Signed-off-by: chasingegg <chao.gao@zilliz.com>
 2023-08-31 10:25:01 +08:00
congqixia
d343888f3c
Fix pkg codecov & static-check script (#26720) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-08-31 09:59:01 +08:00
jaime
c603f1c244
Remove mysql metastore (#26633) 
Signed-off-by: jaime <yun.zhang@zilliz.com>
 2023-08-29 14:36:26 +08:00
congqixia
ee7aef9272
Make pulsar request timeout configurable (#26525) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-08-23 20:46:23 +08:00
wei liu
7af0f7d90c
avoid concurrent sub/unsub on same channel (#26454) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-08-23 10:10:22 +08:00
jaime
b82fc8d6e7
[skip e2e] Add connection log for etcd and grpc (#26536) 
Signed-off-by: jaime <yun.zhang@zilliz.com>
 2023-08-22 21:20:24 +08:00
SimFG
d13ca54414
Change the default gracefulStopTimeout and only warn when fail to refresh policy cache (#26450) 
Signed-off-by: SimFG <bang.fu@zilliz.com>
 2023-08-22 09:04:21 +08:00
cai.zhang
94846995bf
Refine error message for field not exist (#26330) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2023-08-21 11:52:20 +08:00
yihao.dai
c3128aaef3
Accelerate server id interceptor validation (#26468) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-08-20 21:24:19 +08:00
wayblink
c5a1b41f95
Update session ttl to 60s (#26346) 
Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2023-08-18 22:52:20 +08:00
wei liu
79ccf06cf6
refine proxy to querynode heartbeat interval (#26426) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-08-18 14:26:19 +08:00
cqy123456
7e3323a518
update dimension range of diskann. (#26405) 
Signed-off-by: cqy123456 <qianya.cheng@zilliz.com>
 2023-08-17 20:26:20 +08:00
jaime
4a388cf450
[skip e2e] Align default value with milvus.yaml file (#25619) 
Signed-off-by: jaime <yun.zhang@zilliz.com>
 2023-08-17 20:24:19 +08:00
yihao.dai
63b86b32a6
Add server id validation interceptor (#26395) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-08-17 20:20:20 +08:00
wayblink
ab8214fef9
Add ut to ensure hash result (#26417) 
Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2023-08-17 18:26:21 +08:00
XuanYang-cn
be30838cf9
Refine some comments (#26401) 
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2023-08-17 10:54:18 +08:00
wei liu
4aac7a6642
refine error reason of query/search failed (#26342) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-08-16 19:38:18 +08:00
wei liu
74133a3996
refine retry on grpc (#26360) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-08-16 19:36:18 +08:00
yah01
9723787141
Calculate memory usage without page cache (#26389) 
Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-08-16 17:04:17 +08:00
SimFG
19f210974f
Support to update the hook config in the runtime (#26350) 
Signed-off-by: SimFG <bang.fu@zilliz.com>
 2023-08-16 16:03:33 +08:00
SimFG
8b54914287
Add go.mod file for the milvus/pkg package (#26359) 
Signed-off-by: SimFG <bang.fu@zilliz.com>
 2023-08-16 14:49:32 +08:00
congqixia
4742049ecf
Fix unstable TestSingleflight unit test (#26363) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-08-16 10:07:33 +08:00
congqixia
f8bcf60e55
Adjust some confusing Warning log to INFO (#26356) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-08-15 17:49:33 +08:00
cqy123456
376642b933
reset diskann min dim limit (#26344) 
Signed-off-by: cqy123456 <qianya.cheng@zilliz.com>
 2023-08-15 16:45:34 +08:00